Repository navigation
fix(ase): backport bundled calculator and parser corrections - #8319
QuantumMisaka wants to merge 11 commits into
Conversation
ABACUS-PR-BOT
left a comment
There was a problem hiding this comment.
Source inspection identified two compatibility issues and two non-blocking follow-ups below. Reviewed the complete diff and affected callers, including ASE 3.23/3.29 source APIs. Governance and Python syntax checks passed; the full whitespace check flagged preserved fixture formatting only. Builds, tests and the engine were not run; author-reported results remain unverified, and CI diagnostics were unavailable.
| # the parser; otherwise a malformed mapping leaks a bare IndexError. | ||
| reference = _read_stru_reference( | ||
| directory, stru_file, self.atomorder, log_path=log | ||
| ) |
There was a problem hiding this comment.
The new mandatory STRU reread rejects valid noncollinear input such as mag 1.0 -1.0 0.0: _parse_coord_line() recognizes a vector only when the y component starts with a digit, then rejects the remaining negative token. This prevents ordinary SCF result collection before either output reader runs. Please accept signed vector components and cover a negative-y moment through the public SCF handoff.
There was a problem hiding this comment.
Fixed in 770926e. _parse_coord_line() now parses all three Cartesian moment components with float(), including negative values. Added a public SCF handoff regression with mag 0.0 -1.0 0.0; the offline suite passes on ASE 3.23 and 3.29.
There was a problem hiding this comment.
Follow-up: signed Cartesian moments remain supported after the keyword-parser cleanup, including the negative-y public SCF handoff case. Its regression is maintained in the standalone suite and run externally against the bundled implementation; the imported test directory has been removed from this PR (329bdf8).
| if vector_moments: | ||
| results.setdefault('natoms', len(atoms)) | ||
| if 'energy_without_entropy' in atoms.info: | ||
| results['energy_without_entropy'] = atoms.info['energy_without_entropy'] |
There was a problem hiding this comment.
After a successful calculation with Fermi or entropy data, inherited calc.export_properties() now raises: ASE 3.23/3.29 reconstructs Properties(self.results), which accepts neither efermi nor energy_without_entropy; vector magmoms also violates its shape schema. This affects export and the experimental atoms.get_properties(["energy"]) path, while ordinary energy getters remain usable. Please preserve the extra metadata while providing a schema-compatible export and regression.
There was a problem hiding this comment.
Fixed in 770926e. Abacus.export_properties() now filters unknown metadata and skips vector magnetic moments that do not match ASE’s versioned scalar schema; efermi, energy_without_entropy and vector magmoms remain in calc.results. Added coverage for both export_properties() and Atoms.get_properties(["energy"]).
There was a problem hiding this comment.
Follow-up: export_properties still projects through ASE Properties, while read_results reuses the already-validated SinglePointDFTCalculator properties without projecting twice. Extra metadata and vector moments remain in calc.results. The public export/get_properties regressions pass; their source now stays in the standalone suite (329bdf8).
| Both ASE `energy` and `free_energy` retain ABACUS's force-consistent | ||
| `E_KohnSham`. When an entropy term is present, `energy_without_entropy` is | ||
| provided separately; a missing term remains unknown. Noncollinear vector | ||
| moments are retained without modifying ASE's global property definitions. |
There was a problem hiding this comment.
Non-blocking: the recommended examples/soc.py and tests/magnetic.py still replace ASE’s global magnetic-property definitions. The new handoff no longer needs that workaround; removing it would align these consumers with the documented behavior and avoid rejecting later collinear results in the same process.
There was a problem hiding this comment.
Addressed in 770926e: removed the global ASE magnetic-property mutation from examples/soc.py and tests/magnetic.py. The calculator now handles vector moments locally.
There was a problem hiding this comment.
The global magnetic-property mutations remain removed from both examples/soc.py and tests/magnetic.py. This consumer correction is retained in the reduced diff; it is not part of the test-directory removal (329bdf8).
|
|
||
| @pytest.mark.parametrize("scale", [1.234564321, 1.2345652]) | ||
| def test_lts_rounded_cell_nonunit_alat_and_nonorthogonal_vectors(tmp_path, scale): | ||
| """The precise Cartesian header, rather than its rounded Angstrom line, sets alat.""" |
There was a problem hiding this comment.
Non-blocking: this reader still takes alat from the rounded lattice constant (Angstrom) line, so this test establishes tolerance acceptance rather than precise-header selection. Likewise, lines 414–415 describe a second spin channel although their fixture declares nspin=1 and one E_Fermi. Please align both descriptions with the paths the assertions actually exercise.
There was a problem hiding this comment.
Corrected in 770926e: the fixture is an nspin=1 log with one Fermi value; the comment now describes that value directly.
There was a problem hiding this comment.
The imported test file has now been removed from the ABACUS PR. The standalone counterpart describes tolerance acceptance with rounded lattice output and the single Fermi value in its nspin=1 fixture; it does not claim precise-header selection or a second spin channel. The bundled README states the fixed-tolerance limitation (329bdf8).
|
The compatibility findings are fixed in 770926e and the four inline replies document the changes. The standalone counterpart is tracked separately as QuantumMisaka/abacuslite#16 and remains draft pending the ABACUS-first maintenance decision. |
kirk0830
left a comment
There was a problem hiding this comment.
I have only reviewed a very limited of this PR, there are 7000 net increase line, a detailed review seems not plausible. However, as for as what I have found, there are still several lines, designs need to be re-considered, for a long-term maintainability.
| values = list(atomorder) | ||
| except (TypeError, ValueError) as exc: | ||
| raise RuntimeError( | ||
| f"scf atomorder 无法解析(STRU: {stru_path};期望 {natoms} 个整数索引;" |
There was a problem hiding this comment.
please use English instead of Chinese, thanks
There was a problem hiding this comment.
Converted this diagnostic to English. The SCF errors now identify the relevant STRU/log and the rejected condition; the language cleanup is included in 329bdf8.
| or not valid_integer | ||
| or sorted(values) != list(range(natoms)) | ||
| ): | ||
| raise RuntimeError( |
There was a problem hiding this comment.
similarly, use English instead of Chinese
There was a problem hiding this comment.
Converted the atom-order diagnostic to English in 329bdf8. It reports the received indices and the expected integer permutation.
| f"fail-closed)" | ||
| ) from exc | ||
|
|
||
| valid_integer = all( |
There was a problem hiding this comment.
for this boolean is used only once, it is usually not recommended to explicitly create such a variable, instead, use the expression at the rhs of the = operator to keep the code clean
There was a problem hiding this comment.
Removed the one-use valid_integer variable and placed the check directly in the permutation condition. The adjacent comment explains the integer, range and uniqueness requirements (329bdf8).
| isinstance(value, (int, np.integer)) and not isinstance(value, (bool, np.bool_)) | ||
| for value in values | ||
| ) | ||
| if ( |
There was a problem hiding this comment.
in-line annotations are needed to enhance the code readability of this conditional statement
There was a problem hiding this comment.
Added an English explanation: the mapping needs one non-boolean integer per atom, and sorting against range(natoms) checks range and uniqueness. This validation runs before reader indexing can hide an invalid mapping (329bdf8).
| ) as exc: | ||
| suffix = f";log: {log_path}" if log_path is not None else "" | ||
| raise RuntimeError( | ||
| f"scf STRU 身份信息无效(STRU: {stru_path}{suffix};{exc};fail-closed)" |
There was a problem hiding this comment.
Converted the STRU/coordinate diagnostics here to English and kept the filename and failing condition in the error (329bdf8).
| cell_inverse = _invert_cell(cell, f"STRU {stru_path}") | ||
|
|
||
| if coord_type.startswith('d'): | ||
| positions_species = coordinates @ cell |
There was a problem hiding this comment.
what's the meaning of the variable "positions_species"? you use the "_" as "of"?
There was a problem hiding this comment.
Renamed this to species_positions: Cartesian positions in the species-grouped STRU order, before applying the mapping back to ASE order. The unused return fields and redundant array copies have also been removed (329bdf8).
| 'path': stru_path, | ||
| 'positions': np.asarray(positions_species, dtype=float)[mapping].copy(), | ||
| 'cell': np.asarray(cell, dtype=float).copy(), | ||
| 'cell_inverse': np.asarray(cell_inverse, dtype=float).copy(), |
There was a problem hiding this comment.
what is the necessity to pass the inverse of the cell in this function? cell is also passed, the inverse should be calculated elsewhere
There was a problem hiding this comment.
Removed cell_inverse from the reference dictionary because no caller consumed it. The reference cell is still checked for singularity; the matching routine computes the candidate inverse where the periodic residual needs it (329bdf8).
| scale = float(traj[0]['alat_in_angstrom']) | ||
| if not np.isfinite(scale) or scale <= 0: | ||
| raise ValueError(f"Invalid lattice constant in Angstrom: {scale!r}") | ||
| return direct, scale |
There was a problem hiding this comment.
this is not what a commonly metadata like
There was a problem hiding this comment.
Renamed the helper to _log_coordinate_system. Its two results are the logged fractional/Cartesian convention and the Cartesian scale in Angstrom. It reuses the already-validated raw trajectory during SCF collection (329bdf8).
| raise | ||
| except (AssertionError, KeyError, OSError, TypeError, ValueError, IndexError) as exc: | ||
| raise RuntimeError( | ||
| f"scf running log 原始帧校验失败:{log_path}({exc};fail-closed)" |
There was a problem hiding this comment.
The frame-matching diagnostics are now in English, including the last-frame coordinate and cell differences. The cleanup is included in 329bdf8.
| return _frame_coordinate_metadata(log_path, legacyio=legacyio)[0] | ||
|
|
||
|
|
||
| def _validate_raw_scf_trajectory(log_path, expected, *, legacyio=None): |
There was a problem hiding this comment.
this function has a non-trivial cognitive complexity due to too deep conditional statements...what does this function do?
There was a problem hiding this comment.
This check prevents atom reordering from silently dropping an extra atom in malformed output. I separated the per-frame checks from the trajectory loop and reduced duplicate matching bookkeeping and result conversion. The raw cardinality check remains before indexing; malformed-frame and atom-order regressions still pass (329bdf8).
| result['mag'] = ('Cartesian', | ||
| list(map(float, fields[idx+1:idx+4]))) | ||
| idx += 4 | ||
| elif idx + 3 < len(fields): |
There was a problem hiding this comment.
actually, this is a quite fragile parsing strategy if rely on the number of lines and this is where the ase-abacus project always failed in the past, which furtherly caused the endless bugfix... Do you have any better idea on it to enhance the robustness of these lines?
There was a problem hiding this comment.
Replaced the positional look-ahead with keyword-delimited option parsing. Counts now validate the declared grammar: three coordinates, mobility/velocity triplets, one or three magnetic components, and paired angle1/angle2 values. Signed/scientific values and valid trailing bare mobility triplets are covered, as are truncated or ambiguous records. This uses small standalone cases and existing fixtures (329bdf8).
|
|
||
| Parameters | ||
| ---------- | ||
| fileobj : str | Path | TextIOWrapper |
There was a problem hiding this comment.
I would like to suggest you have a look at how the quantum espresso implements this function, there should be some reasons the TextIOWrapper are there.
There was a problem hiding this comment.
I checked ase.io.espresso.read_espresso_out: its reader decorator accepts a filename or stream, and the implementation reads the stream. Both ABACUS readers now accept str/Path and caller-owned named text streams. The filename is retained to locate eig_occ.txt or MD_dump; the reader leaves caller-owned streams open. Named-stream and path cases pass (329bdf8).
| trajectory = read_traj_from_running_log(abacus_lines) \ | ||
| if fileobj.name != 'running_md.log' else \ | ||
| read_traj_from_md_dump(fileobj.parent / 'MD_dump') | ||
| is_md_dump = fileobj.name == 'running_md.log' |
There was a problem hiding this comment.
we do not always use the Java like "is_" as the boolean prefix in this project.
There was a problem hiding this comment.
Replaced is_md_dump with md_calculation in the latest reader. The legacy reader separately names geometry_from_dump, so the variable identifies the selected data source rather than suggesting that it defines periodicity (329bdf8).
| positions=frame['coords'][ind], | ||
| cell=frame['cell'], | ||
| cell=cell, | ||
| pbc=is_md_dump, |
There was a problem hiding this comment.
confusing in semantics: pbc or not depends on whether is MD dumped?
There was a problem hiding this comment.
Agreed: periodicity does not depend on the output file. Both readers now set pbc=True for the ABACUS periodic simulation cell, including SCF output. Existing SCF/MD fixtures verify this independently of whether MD_dump supplies the trajectory (329bdf8).
| ind = ind or list(range(len(frame['elem']))) | ||
| positions = frame['coords'][ind] | ||
| if is_md_log: | ||
| if uses_md_dump: |
There was a problem hiding this comment.
confusing semantics. it is not the md dump directly determines the unit. either use another variable name or add in-line annotations
There was a problem hiding this comment.
Separated md_calculation from geometry_from_dump and documented the format units at the conversion: MD_dump lattice vectors are in alat and its Cartesian positions are in Angstrom; running-log cells are already in Angstrom, while positions follow the logged coordinate convention. The conversion and non-unit-alat regressions pass (329bdf8).
| cell=frame['cell'], | ||
| positions=positions, | ||
| cell=cell, | ||
| pbc=is_md_log, |
There was a problem hiding this comment.
The pbc expression here no longer depends on the calculation/file branch: both readers return pbc=True for the periodic ABACUS cell. Geometry source selection and unit conversion are handled separately (329bdf8).
| requires-python = ">=3.10" | ||
| dependencies = [ | ||
| "ase>=3.22.0", | ||
| "ase>=3.23.0", |
There was a problem hiding this comment.
The existing calculator imports BaseProfile, CalculatorTemplate and GenericFileIOCalculator from ase.calculators.genericfileio. I reproduced ModuleNotFoundError with ASE 3.22.1; ASE 3.23.0 provides that module. The bundled implementation passes 130 external regressions on 3.23.0. This corrects the declared minimum rather than adding a new ASE API dependency (329bdf8).
| @@ -0,0 +1,33 @@ | |||
| INPUT_PARAMETERS | |||
There was a problem hiding this comment.
can't the testfiles in the integration tests folder satisfy the functionalities being tested? adding quite many additional files without maximizing the reuse is strongly discouraged.
There was a problem hiding this comment.
Removed the newly imported tests/unit and tests/fixtures trees and their added CI step. ABACUS keeps its existing inline and engine integration harness. The standalone suite can be run externally against the bundled source; its own duplicate tests/data were consolidated. The resulting PR changes 8 existing files, +760/-178, with no new recorded logs (329bdf8).
|
Following the maintainability review, the PR is now limited to 8 existing files (+760/-178) at 329bdf8. The imported offline regression directories, recorded logs and added CI step are removed; ABACUS keeps its existing harness. Standalone tests and documentation were also consolidated in QuantumMisaka/abacuslite#16, and release artifacts contain no tests, logs or development history. The follow-up removes redundant SCF matching state, array copies and property conversion, and leaves downstream abacus.json markers untouched. Keyword parsing, named streams, periodicity, units and the ASE minimum are addressed individually in the review threads. Verification: 130 external regressions against the bundled source on ASE 3.23 and 3.29, 55 existing inline tests, and one native SCF integration passed; the standalone source and rebuilt installed wheel each passed 210 tests. The PR description records skipped cases and the pre-existing energy-reference mismatch. Hosted checks for this new head are still running. Threads are left open for reviewer confirmation. |
The bundled ASE calculator can collect SCF results for a different structure and mishandle magnetic moments, output units and ASE property export. This PR corrects the existing calculator and readers.
Changes
1e-4 Åidentity tolerance (rtol=0). Result parsing does not readabacus.json.Scope and provenance
8 existing files, +746 / -181 lines. No new regression directories or recorded logs are included. Existing inline tests, engine integration tests and the ASE workflow remain the verification entry points in ABACUS. Offline regressions and fixtures belong to the standalone development repository; the same suite was run externally against this bundled implementation.
Adapted from
QuantumMisaka/abacuslite@39ab1d5and signed-occupation fix5d3da5d, with the matching review follow-up in QuantumMisaka/abacuslite#16. Related: #8165, #8175. The bundled implementation remains maintained while external-package adoption is pending.Verification
Python 3.12; source imports selected explicitly with
PYTHONPATH:PYTHONPATH=interfaces/ASE_interface python -m pytest interfaces/ASE_interface/abacuslite/{core.py,io/generalio.py,io/latestio.py,io/legacyio.py,utils/ksampling.py} -k 'not test_calculator_results' -qOMP_NUM_THREADS=1:PYTHONPATH=interfaces/ASE_interface python -m unittest discover -s interfaces/ASE_interface/tests -p scf.py -vpython3 tools/03_code_analysis/agent_governance_check.py --stagedandgit diff --cached --check: passed. Independent review caught a bare-mobility parsing regression; the corrected input forms now have focused standalone coverage and passed re-review.The deselected energy assertion was previously reproduced on the original base and candidate with the installed LTS engine (
-194.9530533315versus-194.953050937eV); its reference is unchanged. Current develop ABACUS was not rebuilt locally. Hosted CI remains responsible for that engine build.Governance
No engine INPUT parameters or
source/modules change. README explains result behavior and the ASE dependency floor. The fixed matching tolerance is not a universal printing-error bound for arbitrarily large or skewed cells. Tests and provenance remain available in the linked standalone repository; no new fixture duplication is requested.Scientific geometry review follow-up
Normalize Direct and Cartesian running-log positions to Cartesian Angstrom at both public ASE reader boundaries; raw-frame dictionaries remain file-native. Remove compensating conversion from SCF matching to avoid double scaling. Public periodic flags describe the simulation box even for vacuum systems; README documents explicit nonperiodic analysis and current-position stream reads.
Eight numeric regressions failed before the fix and pass afterward (both readers, both coordinate formats, nonunit alat, triclinic/vacuum cells and reordering). The finite-positive alat rejection remains covered. Bundled external suite: 139 passed on both ASE3.23 and3.29; inline55 passed/2 skipped/1 known baseline deselected; native LTS SCF1 passed. Standalone rebuilt-wheel219 passed/19 skipped and ATST installed-consumer1462 passed/37 skipped. Independent review checked runtime equivalence between bundled and standalone implementations. New hosted CI is pending.