Skip to content

Add analytic NEP Hessian command with dense and sparse output - #1793

Merged
brucefan1983 merged 22 commits into
brucefan1983:masterfrom
hsianghualite:hessian-only
Nov 15, 2026
Merged

brucefan1983 merged 22 commits into
brucefan1983:masterfrom
hsianghualite:hessian-only

Conversation

@hsianghualite

@hsianghualite hsianghualite commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Add compute_hessian analytic to calculate the Cartesian Hessian of the current configuration using a single supported NEP energy model. It executes immediately after being read from run.in, must follow potential, and does not relax the structure. Ordinary NEP static calculations need no preceding MD run.

Supported models are ordinary NEP energy models, including universal, typewise-cutoff, and flexible ZBL. The implementation uses the shared model reader, independent Hessian parameter storage, and analytic second derivatives. Public changes are limited to command registration and passing existing input/potential information. Ordinary NEP force calculation, neighbor management, and compute_phonon remain unchanged.

Usage and output

Default dense text output (hessian.out):

potential nep.txt
compute_hessian analytic

Sparse Matrix Market output with diagnostics and a structure snapshot:

potential nep.txt
compute_hessian analytic output_format sparse output hessian.mtx raw_output hessian_raw.mtx metadata hessian.json structure_output hessian.xyz

Each command must occupy one line. Both formats support raw_output, metadata, and structure_output. Selected files are overwritten and must use distinct paths that do not alias input files.

The matrix is H = -dF/dR = d²E/dR², in eV/Ų, ordered as all x coordinates, then all y coordinates, then all z coordinates. The normal output is (H + Hᵀ)/2; raw output retains the unsymmetrized matrix. Neither output is mass weighted, and no acoustic sum rule is imposed. Metadata uses schema version 6.

Scope and limitations

  • Analytic-only, single-GPU ordinary NEP energy models. Temperature-dependent NEP, DFT-D3, multiple potentials, multi-GPU NEP, charge/spin/vdW/ILP models are rejected. There is no finite-difference mode, fallback, or phonon postprocessing.
  • Periodic-image contributions for the same atom are summed. Neither format retains lattice-translation labels, so a small-cell matrix cannot directly provide translation-resolved force constants for arbitrary-wavevector phonon dispersion.
  • Dense storage scales as 9N²; sparse storage follows interacting atom blocks. Both remain subject to memory and index limits. Neighbor distances must be at least 1e-6 Å; q_123/q_233 require L_max >= 3, and q_134 requires L_max >= 4.
  • Finite matrices that fail symmetry or translation checks can still produce requested raw/metadata files marked failed_validation, followed by termination. They are not written as normal results.

Validation of commit 0cb7a20

Validated on November 15, 2026, using source commit 0cb7a20cafd4650e11fa71d8d6d128ac341fa8b3, NVIDIA RTX A2000 Laptop GPU (8 GiB), CUDA 12.8 / nvcc 12.8.93, driver 577.00, Linux/WSL.

  • Build passed: make -C src -j4 gpumd CUDA_ARCH=-arch=sm_86. The final-commit check confirmed the executable was up to date. Sphinx HTML documentation passed -W --keep-going.
  • All 12 GPU regression methods passed (184.706 s), plus a separate test of eight invalid invariant/L_max combinations.
  • Eleven ordinary-NEP analytic-versus-force-central-difference cases passed using 1e-3 Å displacement. Maximum relative Frobenius error was 0.000641384536218, below 1e-3. The independent reference used ordinary displaced force runs with a negligible MD time step; no FD calculation mode was added to production.
  • Each of these 11 cases passed dense/sparse normal and raw matrix comparisons (atol=1e-9, rtol=1e-7) and metadata diagnostics (atol=1e-10, rtol=1e-7). Metadata was checked for schema version 6 and absence of model_temperature_K.
  • Numerical coverage includes L_max=2..8, element compaction, radial-only networks, cubic and triclinic small periodic boxes, and ordinary NEP with universal/typewise/flexible ZBL.
  • A 15,447-atom sparse calculation passed. Ordinary NEP MD continuation passed after inserting dense and sparse commands in NVE (large box) and NVT (small box), both with ZBL: positions, velocities, forces, energies, and virials matched the uninterrupted baseline within atol=rtol=1e-10.
  • Sixteen temperature-dependent model rejection cases passed across dense/sparse, large/small boxes, before/after MD, and with/without ZBL. All failed with an explicit unsupported-model message and created none of the requested output files. Temperature-dependent Hessian calculation is unsupported; these are rejection tests only.
  • Existing near-coincident-neighbor, DFT-D3/unsupported-command, and input/output collision regressions also passed.

Not verified on this revision: GPU buffers above 16 GiB, full Windows/MSVC/CUDA builds, HIP, or performance benchmarks. Compute Sanitizer was not rerun; the earlier attempt on a previous revision could not initialize the WDDM debugger interface. No passing memory-sanitizer result or historical speedup is claimed.

codex and others added 7 commits September 24, 2026 23:22
Document the standalone analytic Cartesian Hessian command available on
the hessian-only branch, following the style of the other run.in keyword
references. Covers syntax, inputs, outputs, all key-value parameters,
the SoA matrix layout and definition, analytic support and limitations,
analytic phonon modes and their outputs, examples and caveats.

Register the keyword in the input_parameters toctree and the run.in
Actions overview table.
@brucefan1983
brucefan1983 marked this pull request as draft September 27, 2026 16:34
codex added 2 commits October 22, 2026 07:57
The previous hessian-only commits had two related defects:

* They modified existing source files (src/force/nep.cu, src/force/nep.cuh,
  src/force/nep_small_box.cuh, src/utilities/gpu_macro.cuh,
  src/utilities/nep_utilities.cuh, src/main_gpumd/run.cu) so the branch
  was not "purely additive" against the pre-hessian state.

* When `dftd3` was enabled in run.in, compute_nep_analytic_hessian bailed
  out with `return false` -- a conservative degradation that ignored the
  dispersion contribution rather than computing it.

This commit fixes both.

* Existing source files are restored to their pre-hessian (d597f8b)
  state.  A new keyword_dispatcher module (src/main_gpumd/keyword_dispatcher.{cu,cuh})
  plus a fallback at the end of Run::parse_one_keyword lets run.in gain
  new keywords without further edits to run.cu.  The previous
  `compute_hessian` if/else branch inside run.cu is removed.

* To keep nep.cuh free of hessian-only additions, the ZBL state is
  recovered by parsing nep.txt via nep_hessian::load_nep_zbl, the
  expanded-box decision is made by nep_hessian::nep_requires_expanded_box,
  and the `uses_dftd3()` member is replaced by nep_hessian::run_input_has_dftd3
  (a static scan of run.in that mirrors NEP::initialize_dftd3).

* The `q_descriptors` side-channel write that the hessian-only commits
  added inside src/force/nep.cu is dropped; a module-local
  nep_hessian::DescriptorCache is populated by a hessian-owned
  descriptor kernel (src/phonon/nep_hessian_descriptor.cuh).  nep.cu is
  thus byte-identical to its pre-hessian content.

* With `dftd3` enabled, compute_nep_analytic_hessian now central-differences
  the dispersion force via the existing force.compute path
  (src/phonon/nep_hessian_dftd3.cu/cuh).  This is bit-identical to the
  `compute_hessian fd` path; the displacement defaults to 1e-3 A and is
  overridable via GPUMD_DFTD3_FD_DISPLACEMENT.  Sparse output is supported
  (the dense FD hessian is scattered into the same per-block sparse
  layout that the analytic path produces).

Tested bit-exactly against pre-refactor references on 2/4/8/50/100
Te cluster systems (with and without dftd3), a non-periodic molecule,
sparse (.mtx) output, phonon gamma, FD mode, and a ZBL NEP4 system.
The dftd3 FD-of-force analytic path agrees bit-exactly with
`compute_hessian fd` on identical configurations.
…essor

The previous commit restored src/force/nep.cu, src/force/nep.cuh,
src/force/nep_small_box.cuh, src/utilities/gpu_macro.cuh and
src/utilities/nep_utilities.cuh to their pre-hessian state, but it could
only do so by promoting `paramb` and `annmb` from private to public in
src/force/nep.cuh and by rewriting the type/index expressions in
src/force/nep.cu back to their pre-hessian spelling.  Both files therefore
still differed from the true pre-hessian state (d597f8b).

This commit removes those residual edits so the hessian-only branch is
additive against the pre-hessian tree with a single, minimal hook:

* src/force/nep.cu is now byte-identical to d597f8b.
* src/force/nep.cuh gains only an 11-line addition -- a forward
  declaration of `nep_hessian::Access` plus
  `friend struct ::nep_hessian::Access;` inside class NEP.  No existing
  line is changed: `paramb`, `annmb`, `zbl`, `dftd3`, `ebox` and
  `neighbor_manager` all stay in the original private section, and the
  member ordering is untouched.
* src/phonon/nep_hessian_accessors.cuh defines the friend struct
  `nep_hessian::Access`, which returns references to those private
  members.  The hessian translation units now read NEP internals through
  it instead of relying on widened access.

Because the friend struct exposes `zbl`, `dftd3` and `ebox` directly, the
re-parsing workarounds that existed only to avoid touching nep.cuh are no
longer load-bearing, so this change keeps them intact rather than
rewriting them under time pressure; they can be removed in a follow-up.

Verification on the 2-atom Te/Pb smoke system (`dftd3 pbe 6.0 1.5` plus
`compute_hessian analytic`) reproduces the previous hessian.out
bit-for-bit (md5 80a3bccb63920ae5a301a81a91aafc33).
@brucefan1983

Copy link
Copy Markdown
Owner

This is a big PR. I have to review it little by little. Here are the initial comments:

  • Keep src/force/nep.cuh unchanged. Remove the friend declaration and the associated accessor mechanism. The Hessian implementation should not require exposing or modifying NEP’s private state.

  • Follow the existing command dispatch mechanism. Add the command through the established path in Run::parse_one_keyword(), following commands such as compute_phonon. Remove keyword_dispatcher.cu/.cuh and the static registration machinery.

  • Keep this PR analytic-only. GPUMD already has finite-difference Hessian calculations. Remove the new FD mode and duplicated FD implementations.

  • Do not support DFT-D3 in this PR. Remove the DFT-D3-specific Hessian code and explicitly reject configurations with DFT-D3 enabled. An analytic calculation must not silently fall back to finite differences.

codex added 6 commits November 3, 2026 15:11
Address the review comments on this branch:

- src/force/nep.cuh is restored to its pre-branch state.  The friend
  declaration and the accessor mechanism are gone.  The analytic Hessian
  now loads the NEP model itself through the new src/phonon/
  nep_hessian_model.{cu,cuh}, which follows the pattern already used by
  src/mc/nep_energy.cu, src/measure/nep_response.cu and
  src/force/ilp_nep.cu.

- The command is dispatched through the established path in
  Run::parse_one_keyword(), next to compute_phonon.  The generic
  keyword_dispatcher.{cu,cuh} and its static registration are removed.

- The command is analytic only.  The finite-difference mode, the
  duplicated finite-difference implementation and the 'displacement'
  option are removed.  The pre-existing compute_phonon command is
  untouched and remains the finite-difference path.

- DFT-D3 is not supported.  The DFT-D3 specific Hessian code is removed
  and a configuration that enables the dispersion correction is rejected
  explicitly, whether the dftd3 keyword appears before or after
  compute_hessian in run.in.  An analytic request never falls back to a
  finite-difference result.

With the exception of the compute_hessian branch in run.cu and two
documentation indexes, the branch now only adds new files: every
pre-existing source file is byte-identical to the merge base.
Keep the NEP4 file interface and reuse the branch's descriptor conversion
utility. Store type-pair coefficients in a single owned GPU allocation.
Remove hidden physics switches, debug state, duplicate phonon processing,
and runtime source metadata; streamline matrix diagnostics and output.

Keep NEP's private state and the existing compute_phonon implementation
unchanged. Retain analytic-only dispatch and explicit DFT-D3 rejection.

Validated with an sm_86 build, 17 independent force-Jacobian comparisons,
dense/sparse equivalence, large sparse output, MD state continuation,
and command rejection checks.
@brucefan1983
brucefan1983 marked this pull request as ready for review November 14, 2026 08:34
@brucefan1983
brucefan1983 self-requested a review as a code owner November 14, 2026 08:34
@hsianghualite hsianghualite changed the title Add compute_hessian with analytic, finite-difference, and sparse Hessian modes Add analytic NEP Hessian command with dense and sparse output Nov 14, 2026
@brucefan1983
brucefan1983 merged commit be63a05 into brucefan1983:master Nov 15, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants