Skip to content

Fix: validate dav_subspace correction directions before normalization - #8142

Merged
mohanchen merged 4 commits into
deepmodeling:developfrom
Critsium-xy:codex/fix-8127-dav-direction
Nov 4, 2026
Merged

mohanchen merged 4 commits into
deepmodeling:developfrom
Critsium-xy:codex/fix-8127-dav-direction

Conversation

@Critsium-xy

Copy link
Copy Markdown
Collaborator

An already converged Ritz pair can have a zero or tiny Davidson correction even with an ample basis. dav_subspace currently normalizes that direction before checking it, then reports a misleading insufficient-basis diagnosis. This change checks residuals and direction norms before division while retaining ordinary Davidson expansion and the outer SCF stopping criteria.

Reminder

  • I have read AGENTS.md and docs/developers_guide/agent_governance.md.
  • I have linked the issue and described the remaining verification limits.
  • I have added focused regression tests and listed commands/results.
  • I have described INPUT/docs and core-module impact below.
  • No governance exception is requested.

Linked Issue

Related to #8127. The correction-handling defect is reproduced and repaired on CPU. The original FeRh GPU trajectory has not been rerun, so this draft does not claim end-to-end verification of that case.

What's changed?

  • Preserve the full Ritz coefficients/eigenvalues while compacting active corrections, including the all-converged return path.
  • Deflate only finite, numerically tiny unpreconditioned residuals that also satisfy the requested band tolerance. The cutoff is min(ethr_band, max(1e-12, epsilon<Real>*max(1,abs(eigenvalue)))); loose energy tolerances therefore do not skip ordinary expansions.
  • Measure the remaining preconditioned directions before normalization using the shared CPU/GPU operations and pool reductions. Normalize usable small directions; reject zero/nonfinite directions with measured norms, both thresholds, eigenvalue, original band and explicitly pool-local k-point context.
  • Preserve the existing context-free interface for LR-TDDFT and Python callers. Shared normalization kernels for other solvers are unchanged.
  • Cover exact/rotated Ritz pairs, partial and noncontiguous convergence, nonuniform positive overlap, complex/spinor-shaped storage, padding, restart/repeated solves, zero/NaN directions, and loose/strict tolerance discrimination. Register the corresponding GPU test target for GPU builds.

Unit Tests and/or Case Tests for my changes

Verified on 2026-10-14 from develop f264b4022fd477734de0f3ace723f6da9e306943 plus submitted commit fcf6e5c. An independent reviewer checked the source hashes, raw logs and numerical comparisons. Linux CPU: GCC 12.2.0, Open MPI 4.1.4, OpenBLAS 0.3.21, FFTW 3.3.10, LibXC 5.2.3 and ELPA 2022.11.001.

export OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1
cmake -S src -B build -G Ninja -DBUILD_TESTING=ON -DCMAKE_BUILD_TYPE= \
  -DENABLE_MPI=ON -DENABLE_LCAO=ON -DBLA_VENDOR=OpenBLAS -DMKLROOT= -DMKL_ROOT= \
  -DENABLE_ELPA=ON -DENABLE_LIBXC=ON -DENABLE_FLOAT_FFTW=ON \
  -DENABLE_JSON=ON -DGIT_SUBMODULE=OFF -DCMAKE_INSTALL_PREFIX="$root/install"
cmake --build build --target abacus_basic_para MODULE_HSOLVER_dav_subspace MODULE_HSOLVER_pw -j4
ctest --test-dir build -N -R '^MODULE_HSOLVER_(dav_subspace|dav_subspace_parallel|pw)$'
ctest --test-dir build -j1 -V --output-on-failure --no-tests=error \
  -R '^MODULE_HSOLVER_(dav_subspace|dav_subspace_parallel|pw)$'
build/abacus_basic_para --version

All commands passed: 3/3 CTest entries, comprising 14 single-rank regression tests, 8 cases under two MPI ranks, and 2 existing PW tests. The full executable build includes the unchanged LR-TDDFT caller. Version: v3.11.0-beta10; executable SHA256: 9a622f980a41690822af99e04eda1aef34ee4b0966c1e12d305bba1cec2551fb.

Before the repair, compatible exact/tiny/large-preconditioner fixtures exited 1 at the old norm guard; the mixed exact/inexact fixture aborted at the positive-norm assertion. These are CPU mechanism reproductions, not the original FeRh GPU run.

Runtime checks, using four MPI ranks and one OpenMP thread:

# From src/tests/01_PW:
bash ../integrate/Autotest.sh -a "$root/build/abacus_basic_para" -n 4 -o 1 -j 1 -r '^024_PW_DS$'
# From each of 038_PW_NC_GGA_GRAD2 and scf_deltaspin4:
timeout 120 mpirun -np 4 "$root/build/abacus_basic_para"
# Independent comparison of their completed outputs:
python3 verify_runtime_outputs.py .

The ordinary PW harness passed. Both additional calculations exited 0 and converged. Their standard property collector failed because the CPU host lacks bc; a local independent parser then compared all eight energy/per-atom-energy/force/stress values against unchanged references, reproducing the collector's six-decimal absolute-sum arithmetic. All comparisons passed with the original energy/force/stress thresholds 1e-7 / 1e-4 / 1e-3; no inputs or references were relaxed. Noncollinear total force is 18.267150, and constrained-spin total force is 21.281736, both matching the stored references. These are supplemental numerical checks, not a claimed pass of the two interrupted standard collectors.

Fresh git diff --cached --check and python tools/03_code_analysis/agent_governance_check.py --staged passed, with the documentation rationale below accounting for the sole warning.

Not run: CUDA/ROCm compilation or runtime, the original FeRh GPU case (archive attachment and three required PP/orbital files unavailable), Python package runtime, or a full physical USPP case. CPU real/complex single-precision tests and complex-double MPI coverage do not establish the existing real/float MPI or real-GPU paths. The precision-scaled cutoff is a safeguard, not a general error bound for ill-conditioned overlap matrices.

Governance Notes

  • INPUT/docs: No documentation update required. No INPUT names, defaults, metadata or outer SCF density/energy criteria change.
  • Core impact: Limited to Davidson correction selection/normalization and diagnostic context in HSolverPW; the old general-operator interface is retained. No new global dependencies, header includes, default arguments or direct MPI calls.
  • Quality debt: Whole-file scores are 49/89/82/91 for the solver source/header/PW caller/test. The solver source was already below the suggested 60 line (56 at baseline); existing length/complexity plus explicit safeguards remain nonblocking debt, without an unrelated refactor.
  • Exceptions requested: None.

mohanchen
mohanchen previously approved these changes Oct 15, 2026
@mohanchen
mohanchen dismissed their stale review October 15, 2026 01:27

the PR is still a draft

@Critsium-xy
Critsium-xy marked this pull request as ready for review October 15, 2026 03:36
@Critsium-xy

Copy link
Copy Markdown
Collaborator Author

@ABACUS-PR-BOT Please review this PR. I notice that the original issue seems to be provided by AI, so you'd better check whether it is necessary to solve the issue, and whether there is a risk of over-engineering.

@ABACUS-PR-BOT ABACUS-PR-BOT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Static review at 9609384: this narrow fix is justified. An exact initial Ritz pair reaches cal_grad before the eigenvalue-change check, so the old path can assert or divide by zero despite an ample basis. Checking residuals, preserving Ritz data and filtering converged directions are proportionate; moving the old guard alone still aborts valid eigenpairs. I found no blocking defect or need for a broader redesign.

The original FeRh GPU cause and GPU performance remain unverified. Fresh whitespace/governance checks passed; I ran no builds or runtime tests, and CI diagnostics were unavailable.

@mohanchen mohanchen added Diago Issues related to diagonalizaiton methods Refactor Refactor ABACUS codes labels Oct 15, 2026
Resolve the conflicts with the hsolver test-directory rename and the
empty-block leading-dimension change in dav_subspace:

- cal_grad: keep develop's matrix_ld leading dimension together with this
  branch's compacted active_vcc coefficients.
- Move the dav_subspace test into source_hsolver/unittests and keep only its
  target in that CMakeLists; the linear/TDDFT targets now live under
  module_linalg/unittests on develop.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@ABACUS-PR-BOT ABACUS-PR-BOT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The merged Davidson safeguard remains a proportionate fix; the source review found no new correctness regression. Conflict resolutions and affected callers were checked statically; no build or runtime tests were run.

Preserve the dav_subspace test targets alongside develop's CMake formatting and updates. Local compilation and tests were not run, as requested.
@mohanchen
mohanchen merged commit 2d1da70 into deepmodeling:develop Nov 4, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Diago Issues related to diagonalizaiton methods Refactor Refactor ABACUS codes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants