Skip to content

refactor(lcao): decouple basis indexing from block-cyclic layouts - #8295

Merged
mohanchen merged 68 commits into
deepmodeling:developfrom
kirk0830:pr/06-explicit-layout-v2
Nov 13, 2026
Merged

mohanchen merged 68 commits into
deepmodeling:developfrom
kirk0830:pr/06-explicit-layout-v2

Conversation

@kirk0830

@kirk0830 kirk0830 commented Nov 7, 2026 •

Copy link
Copy Markdown
Collaborator

Key design: the atom basis function layout (ityp, iat, l, izeta, m) := mu is not necessary to be coupled with the block-cyclic distribution mechanism: one is about the matrix layout, the other is about how matrix is distributed

Summary

  • Continue the longer Parallel_Orbitals dissolution plan by passing the block-cyclic basis and matrix layouts explicitly instead of relying on implicit inheritance or forwarding.
  • Introduce BasisFlattenedIndexing and AtomBasisView as a single basis-indexing boundary shared by HContainer, LCAO operators, and their consumers.
  • Migrate Hamiltonian, TDDFT, LR/BSE, density-matrix, HS I/O, solver, and MD test paths to use explicit para_2d() layouts.
  • Remove the obsolete per-element LCAO offset-vector path.
  • Rename BlockCyclicDistribution to Parallel_Blkcyclic; the class documentation defines Blkcyclic as the ABACUS shorthand for block-cyclic. This addresses issue Rename block_cyclic_distribution and normalize its BLACS constants #8299.

The PR-scoped diff is limited to this refactor. Upstream synchronization added no CSR-reader, workflow, documentation, or repository-local agent-memory changes to the PR.

Behavior

No INPUT, output-format, numerical, or public workflow behavior changes. Documentation updates are not required because this is an internal ownership and interface refactor; the affected contracts are documented inline.

Regression coverage

The first revision of this PR retained a duplicate inline nspin=4 Mulliken path that bypassed the already corrected collect_MW_nspin4 helper. Existing unit tests exercised only the helper, so they did not cover the production Output_Mulliken::collect_MW() path where the regression occurred. Integration and CUDA CI caught the resulting spin2/spin4 sign reversal.

The new OutputMullikenSpinTest.Nspin4ProductionPathPreservesMySign test calls Output_Mulliken::collect_MW() directly. It fails with the old implementation (My=-1 instead of +1) and passes after restoring the corrected helper path.

Verification

The final class-name rename and interface-cleanup state was verified at 7ba7fa9bc with abacus-dev/1.0, GNU 11.4.0, and OMP_NUM_THREADS=1:

cmake --build build -j 8
ctest --test-dir build --output-on-failure \
  -R "MODULE_BASE_parallel_blkcyc|MODULE_LCAO_tddft_boundary_fix_test|MODULE_IO_mulliken_test|MODULE_IO_output_mat_sparse_test|MODULE_LR_state_ovlp|MODULE_LR_state_track|MODULE_LR_spectrum_mo_output_test"

cmake --build build-exx -j 8
ctest --test-dir build-exx --output-on-failure \
  -R "MODULE_BASE_parallel_blkcyc|MODULE_LCAO_tddft_boundary_fix_test|MODULE_IO_mulliken_test|MODULE_IO_output_mat_sparse_test|MODULE_LR_state_ovlp|MODULE_LR_state_track|MODULE_LR_spectrum_mo_output_test"

git diff --check upstream/develop..HEAD
python3 tools/03_code_analysis/agent_governance_check.py \
  --base upstream/develop --head HEAD --format text
  • 7ba7fa9bc: default CPU+MPI+LCAO+testing build [100%]; 9/9 focused tests passed, including MODULE_BASE_parallel_blkcyc and its four-rank MPI test.
  • 7ba7fa9bc: LibRI/LibXC build [100%]; the same 9/9 focused tests passed.
  • 1fd814ea2: pre-rename production integration case scf_out_mul_spin4 passed with both 1 and 4 MPI ranks.
  • 7ba7fa9bc: governance check has no blockers and git diff --check is clean.

After merging upstream/develop at 52bc79789, the current head 8462a195f was rebuilt with the same module environment and LibRI/LibXC/EXX enabled:

cmake -S . -B build-merge-exx -G Ninja \
  -DENABLE_LCAO=ON -DENABLE_MPI=ON -DENABLE_LIBXC=ON \
  -DENABLE_LIBRI=ON -DBUILD_TESTING=ON \
  -DFFTW3_DIR=/opt/apps/fftw-3.3.10
cmake --build build-merge-exx -j 8
ctest --test-dir build-merge-exx --output-on-failure \
  -R "MODULE_(BASE_(parallel_blkcyc|atom_basis_view|basis_flattened_indexing)|LCAO_hcontainer|RI_(exx_info|hybrid_gauge_phase|hybrid_image_domain|exx_workspace|EXX)|LCAO_tddft_boundary_fix_test|IO_(mulliken_test|output_mat_sparse_test)|LR_(state_ovlp|state_track|spectrum_mo_output_test))"
git diff --check upstream/develop..HEAD
python3 tools/03_code_analysis/agent_governance_check.py \
  --base upstream/develop --head HEAD --format text
  • 8462a195f: EXX/LibRI/LibXC CPU+MPI+LCAO+testing build [100%] (4850/4850).
  • 8462a195f: 23/23 focused tests passed, including the four-rank block-cyclic MPI test, seven HContainer tests, the upstream EXX/RI regression tests, Mulliken, boundary correction, and LR layout tests.
  • 8462a195f: git diff --check is clean; governance has warnings only and the global dependency budget is net -5.
  • The PR-scoped diff excludes AGENTS.md, CONTEXT.md, ADRs, developer-governance documents, and .qoder.

Review follow-up at 77c9095f4:

  • Retain the EKinetic and Overlap pair-level cal_force_IJR / cal_stress_IJR interfaces. Their legacy Parallel_Orbitals* parameter is replaced by explicit BasisFlattenedIndexing and Parallel_Blkcyclic inputs.
  • Restore the RT-TDDFT term gather_eij and name distribution parameters/members para_layout.
  • Rebuild the complete EXX/LibRI/LibXC CPU+MPI+LCAO+testing configuration successfully; the final incremental dependency tree reached 16/16 targets.
  • 8/8 focused tests pass: EKinetic/Overlap serial, MPI, and three-center variants; RT evolve/band-energy; and the T_NL_cd check.
  • git diff --check is clean and governance has warnings only.

Upstream synchronization at d7789c6cf:

  • Merged upstream/develop through a2beda0ea, including the RT/MD boundary-test split from test(rt): move boundary tests out of MD unit tests #8300.
  • Preserved the upstream ownership split: MODULE_MD_base_mpi now contains only MD-state tests, while boundary correction and the MD-to-RT contract live under the RT test fixture.
  • Adapted the new shared fixture to this PR's explicit BasisFlattenedIndexing / Parallel_Blkcyclic interface and renamed parallel_blkcyc.cpp build input.
  • Complete EXX/LibRI/LibXC CPU+MPI+LCAO+testing build passed (1132/1132 incremental targets).
  • 16/16 boundary, MD/RT contract, td-folding, and MD-base tests passed across serial, 4-rank MPI, and OpenMP variants.
  • git diff --check is clean.

Governance Exception

  • Rule: Unit test directory layout
  • Reason: This long-running PR predates upstream Governance: block unit tests outside a flat unittests/ directory and sources there without a test_ prefix #8245. Its interface migration already required edits to existing tests in legacy test/ directories before the new rule landed.
  • Scope: Existing test files changed only to consume the explicit basis/layout interfaces; no new production behavior is introduced by these paths.
  • User or maintenance risk: Moving all touched legacy tests in this already-large refactor would mix an orthogonal repository-wide test-layout migration into the layout-decoupling PR and make review and rollback substantially harder.
  • Why the normal rule cannot be followed now: The tests are causally required by the interface migration, while the rename/relocation itself is unrelated to Parallel_Orbitals removal and would create a large, noisy second refactor.
  • Follow-up cleanup plan: Migrate the affected legacy test directories to flat unittests/ layouts in a dedicated follow-up PR, using Governance: block unit tests outside a flat unittests/ directory and sources there without a test_ prefix #8245 as the tracking governance rule.
  • Requested approver: Repository maintainer reviewing this PR

Auto-merge is intentionally disabled.

kirk0830 and others added 30 commits October 28, 2026 06:34
Purely additive: no call site changes, inheritance still in place.
para_2d() (const + non-const) returns *this while Parallel_Orbitals derives
from Parallel_2D, giving the two escape hatches a const getter cannot serve
(LR_Util::setup_2d_division mutation, test-side descriptor writes).
get_blacs_ctxt() completes the inherited getter family, which covers the
other seven public data members.
…bitals, migrate module_operator_lcao call sites
… (getter-equiv, derived-layer sync, value-semantics static_assert, MPI+serial dual coverage)
…arallel_2D to using alias, full-repo getter-ization

- new class BlockCyclicDistributionInformation owns all Parallel_2D responsibilities:
  data members (snake_case + trailing underscore), initialize/set/set_serial/set_coord,
  move semantics, getter family, global_to_local_*, owner_*, owns_index
- Parallel_2D demoted to 'using Parallel_2D = BlockCyclicDistributionInformation;' (removal in S5)
  with DEPRECATED-ALIAS/DELETION-COMMIT markers
- all legacy Parallel_2D direct field accesses replaced by getters (A0 burn-down = 0)
- dual build (MPI+serial) zero error; parallel_orbitals + composition tests green;
  D0 four-case byte-compare ok=36 diff=0
…Information

- Parallel_Orbitals member Parallel_2D para_2d_ -> BlockCyclicDistributionInformation para_2d_
- para_2d() accessor return types updated to BlockCyclicDistributionInformation& / const&
- composition test binds para_2d() to const BlockCyclicDistributionInformation&
- Parallel_2D alias untouched (deletion in S5); setup_2d_division signature unchanged
- dual build zero error; MPI ctest 7/7, serial ctest 3/3; D0 four cases byte-identical (ok=36 diff=0)
- all content from codex S2 run
… / gtest / legacy make / pyabacus) - zero changes needed
…ll references to BlockCyclicDistributionInformation

- delete parallel_2d.{h,cpp} (alias header + empty TU) and parallel_2d_test.{cpp,sh}
- rewrite ~127 files: Parallel_2D whole-word refs -> BlockCyclicDistributionInformation, drop parallel_2d.h includes
- burn-down == 0: git grep Parallel_2D / parallel_2d.h / build entries all zero
- build verification performed by computer-agent (serial+MPI zero-error, related tests green, D0 4-case byte-diff clean)
@kirk0830
kirk0830 requested a review from mohanchen November 7, 2026 08:58

@ABACUS-PR-BOT ABACUS-PR-BOT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new governance regression expects an unimplemented checker rule and needs to be aligned with the checker. The other inline notes are non-blocking cleanup.

Static review at 1fd814e: git diff --check and the governance checker completed successfully (warnings only). Builds, numerical/runtime tests and CI diagnostics were not run.

@kirk0830

kirk0830 commented Nov 7, 2026 •

Copy link
Copy Markdown
Collaborator Author

@ABACUS-PR-BOT All problems have been solved, could you review again?

@AsTonyshment AsTonyshment left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall!

Just a heads-up: our current guidance on forward declarations is a bit contradictory. The ABACUS C++ guide recommends avoiding them, while agent_governance_check.py and even the review bot recommend the opposite.

This is also a debated topic in software engineering, with major projects taking different approaches. For example, Google generally discourages forward declarations, while LLVM encourages minimizing header inclusions and using forward declarations where appropriate. It's essentially a trade-off between maintainability (e.g., explicit dependencies and avoiding fragile declarations) and compile-time efficiency (e.g., reduced header coupling and unnecessary recompilation).

We're planning to put together an updated version of the ABACUS development guidelines to clear up inconsistencies like this. No need to change anything here over this, though :) I'm just pointing it out, and we can leave the discussion of whether forward declarations are preferable for another time.

@kirk0830

kirk0830 commented Nov 8, 2026 •

Copy link
Copy Markdown
Collaborator Author

@AsTonyshment Thanks for your review.
As for the forward declaration and header only, I can share part of my opinions here: personally, I use forward declaration with much care, and I think it should not be allowed in a wide range presently, because it may hide the real design problem, especially hide the coupling between modules. Even for now AI can help me code many, I still spend much time on answering questions from myself about:

  1. What is this name indicates?
  2. What it (a class/variable/struct/...) IS?

As for the compile time management, I am not sure about the preference of other developers, so I cannot give a clear suggestion.

C++ provides many language features allowing developers coding in many different ways, but maintainers should be cautious on this feature before the code is "dry" or atomic enough.

Cheers,
Yike

@AsTonyshment AsTonyshment linked an issue Nov 8, 2026 that may be closed by this pull request

@ABACUS-PR-BOT ABACUS-PR-BOT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both verification filters omit the renamed layout tests; the inline note identifies the update.

Static incremental review at adbb919: full and incremental git diff --check passed. Builds, runtime tests, numerical probes and CI diagnostics were not run; the reported test counts remain author-reported.

Comment thread source/source_base/CMakeLists.txt Resolved
@kirk0830

kirk0830 commented Nov 8, 2026

Copy link
Copy Markdown
Collaborator Author

@AsTonyshment update: to enhance the readability of the forward declaration I have to make at the intermediate stage of the big refactor, I append the inline comments to point to the place where the classes are really declared. In the next PR, the number of forward declaration used will be decreased significantly because of the introduction of Matrix2d container.

Comment thread source/source_io/module_hs/lat_r_csr.h Resolved
@mohanchen mohanchen added the Refactor Refactor ABACUS codes label Nov 9, 2026
…ayout-v2

# Conflicts:
#	source/source_hamilt/module_hcontainer/hcontainer.cpp
…ayout-v2

# Conflicts:
#	source/source_lcao/module_rt/test/test_boundary_fix.cpp
#	source/source_md/unittests/CMakeLists.txt
#	source/source_md/unittests/test_md_base.cpp
@kirk0830

Copy link
Copy Markdown
Collaborator Author

Conflict resolution is pushed at d7789c6cf. Local verification passed: complete EXX/LibRI/LibXC CPU+MPI+LCAO+testing build (1132/1132 incremental targets) and 16/16 boundary/MD-RT/td-folding/MD-base serial and MPI tests. The PR body now includes a complete Governance Exception for the retroactive #8245 legacy-test-layout rule; a maintainer needs to apply governance-exception-approved if branch protection requires that blocker to be downgraded.

@ABACUS-PR-BOT ABACUS-PR-BOT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The incremental source changes introduce no new implementation defect identified by static review; the inline note preserves a removed lifecycle regression case.

Reviewed d7789c6cf against a2beda0ea, including contributor changes and merge resolution. Full and incremental git diff --check passed. Builds, product tests, numerical/MPI/GPU checks and CI diagnostics were not run; author-reported results were not independently verified.

@mohanchen
mohanchen merged commit d1db1cc into deepmodeling:develop Nov 13, 2026
17 of 19 checks passed
mohanchen pushed a commit to mohanchen/abacus-mc that referenced this pull request Nov 13, 2026
Resolve conflicts with the Parallel_Blkcyclic refactor (deepmodeling#8291, deepmodeling#8295) and
the new LCAO_Orbitals parameter of write_eband_terms.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Refactor Refactor ABACUS codes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Rename block_cyclic_distribution and normalize its BLACS constants

4 participants