Repository navigation
Add sbd.analysis.caop_variance: variance and PT2 of a CAOP state (upstream caop-stat-evaluator) - #90
Merged
Merged
Conversation
sbd.analysis.variance gives the energy variance and Epstein-Nesbet PT2 correction of a state saved with savename, by upstream SBD's stochastic evaluator (bound as external_observables, CPU and OpenMP-offload backends) or exactly with PySCF for small systems. state_energy, load_state and load_gdb_shards read TPB and GDB states on any number of ranks. examples/analysis adds variance.py, extrapolate.py and a README. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PySCF's cistring.strs2addr packs a string into an int64 and raises NotImplementedError for norb >= 64, though the FCI routines the exact method uses (contract_2e, make_hdiag) work there. Address each string by its position in PySCF's own make_strings list instead, which is in address order and returns occupied-orbital lists from 64 orbitals on. method="auto" can reach the exact method at that size, as a full space of a few electrons in 64+ orbitals fits under exact_limit. Tested with H2 embedded at orbitals 30 and 64 of 65 against H2 alone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r floor
variance() and variance.py take observable ("both", "variance" or "pt2"),
first_batch (the first batch's ID, so a later call adds independent batches
instead of repeating them) and minimum_abs_denominator (refuse a PT2 with a
near-zero denominator), all passed straight to upstream's evaluator, which
already accepted them. Each batch's ID is recorded in the result. The README
names upstream's gdb-stat-evaluator as the method and describes the options.
Tests: the same batch ID repeats a batch and later IDs give new draws; the
unrequested observable is NaN; an impossible denominator floor is refused;
and, under MPI, a GDB state saved by every rank is analysed on half of them,
as upstream's own test reads 2 saved shards on 4 ranks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On the Thrust backend, _create_sbd_config applied DeviceConfig after sbd_config, and DeviceConfig always wrote use_precalculated_dets and max_memory_gb_for_determinants from its own defaults. A value set in sbd_config was reset, so run_sqd_sbd.py and run_sqd_enlarge_subspace_sbd.py ignored --sbd_use_precalculated_dets and --sbd_max_memory_gb_for_determinants. DeviceConfig is now applied first and writes only the arguments it was given; those two arguments are deprecated in favour of sbd_config, which takes precedence over them. Also bind upstream's Thrust-only TPB field thrust_collapse_loops, present at the vendored pin but not exposed, with --thrust_collapse_loops in run_sbd_diag.py and --sbd_thrust_collapse_loops in the two SQD drivers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Upstream SBD has no OpenMP-offload GDB code, so gdb_diag on the gpu-omp module ran entirely on the host while a GPU had been asked for; only run_gdb_diag.py warned. gdb_diag now raises ValueError on that module, and run_gdb_diag.py and run_gdb_heatbath.py exit with an error for --device gpu-omp, or auto resolving to it, pointing at cpu, or gpu (Thrust, NVIDIA GPUs only). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… cpu_subspace - vendor/sbd-upstream moves from 93ebabe to r-ccs-cms/sbd 95bf45a (25 commits): the opposite-spin 2-RDM sign fix (#98), Davidson/Lanczos subspace refactor and TPB performance work (#95, #99), the CAOP stat evaluator (#94) and the CAOP spectrum app (#96, #100). - Upstream's Thrust builds now MPI_Abort the job when omp_get_max_threads() is 1, which ends Python with nothing to catch. tpb_diag, tpb_diag_from_files and gdb_diag on Thrust raise ValueError on every rank first instead. - TPB_SBD.cpu_subspace and GDB_SBD.cpu_subspace (Thrust only, as upstream declares them): keep the Davidson (and TPB's Lanczos) subspace vectors in host memory instead of GPU memory, as upstream's apps' --cpu_subspace. - Tests: test_rdm_open_shell.py, the opposite-spin 2-RDM block on upstream's H3 doublet against PySCF, which fails at the old pin on every backend and passes now; test_openmp_threads.py, the one-thread refusal on Thrust; test_cpu_subspace.py, the same energy with cpu_subspace on and off for TPB and GDB, and the field present on the Thrust backend only. Also expose it in the drivers as --cpu_subspace / --sbd_cpu_subspace; it is ignored on the backends that do not have it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
variance() took the PT2 reference energy as optional and, for the stochastic method, recomputed it from the saved state with a zero-iteration gdb_diag. That recompute is redundant -- the diagonalization that saved the state returned the energy -- and it ties variance() to a GDB solve, which the OpenMP-offload backend cannot run. Upstream's evaluator takes the energy as a required input (--reference-energy), as caop_variance does here. The energy is now a required argument of variance() and --energy a required flag of examples/analysis/variance.py. state_energy() stays, for checking that a state and an FCIDUMP belong together; variance() no longer calls it. The test helper that saves GDB states skips when gdb_diag refuses the backend, so the tests keep running wherever GDB does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…caop_diag.py CAOP diagonalizes a Hamiltonian written as creation/annihilation operators (hard-core bosons, spins, fermions: lattice models such as Hubbard) over a selected basis, as upstream's caop_selected_basis_diagonalization app does, from a --hamfile and basis files. The library is templated on the scalar type, so real and complex amplitudes are both bound in the existing modules and chosen per call (complex=); upstream's app picks one with -D_COMPLEX. CAOP runs on cpu and gpu (Thrust). Upstream has no OpenMP-offload path for it, so that module sets __sbd_has_caop__ = False and the Python wrapper refuses it. examples/caop/run_caop_diag.py takes upstream's option names and by default runs upstream's 8-site hard-core boson example. It exits with sys.exit(main()); switch to sbd.mpi_util.run(main) once #74 lands. Tests (each solve in a subprocess, since CAOP fixes the process-wide word count): upstream's boson chain over its 16-state basis, and upstream's 4-site Hubbard ring (t=-1, U=4) as a --hamfile in upstream's block layout, both against an exact diagonalization built in NumPy from the same terms; the ring also against GDB on upstream's FCIDUMP of the model; complex amplitudes give the real energy; the offload backend and an empty basis are refused. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…a CAOP state sbd.spectrum.green_function runs upstream's caop-spectrum on a ground state saved by caop_diag_from_files: block Lanczos in the excitation basis, for single-particle c/cdag on chosen orbitals or for general operators given as CAOP operator files, with upstream's deflation tolerances. green_function_from_coefficients reevaluates saved coefficients on a new grid or broadening without rerunning (--read-coefficients), and spectral_function gives the spectral matrix. Real and complex amplitudes, cpu and gpu (Thrust) backends. examples/caop/run_caop_spectrum.py runs it with upstream's option names and CSV output, and the CAOP README continues from the ground-state example to the Green's function. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…P states sbd.analysis.caop_variance runs upstream SBD's caop-stat-evaluator on a state saved by sbd.caop_diag_from_files: the stochastic external variance and Epstein-Nesbet PT2. As upstream, the energy is required for PT2 and not for variance alone. Upstream's loader maps the saved amplitudes onto the basis files and normalizes the result, with no record of the scalar type and no size check, so a real/complex mismatch, an unnormalized state or a basis that is not the state's gave a wrong answer with no error, and a bad Hamiltonian file or duplicate basis line hung or aborted the job. The binding checks the state and the Hamiltonian file on every rank first and refuses together, naming the cause. examples/analysis/variance.py takes a CAOP state with --hamfile, beside the TPB and GDB states it took, with its help grouped by solver, and examples/README.md shows the workflows end to end, from a solve through its check to the Green's function. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Upstream's Thrust GDB solve waits forever in its ring exchange when a b_comm shard is empty. Placement can empty a shard the caller filled: equal-bra-a, the default, deals out whole alpha strings, so fewer distinct alpha strings than b_comm_size leaves ranks with nothing. Check every rank after placement and raise on all of them together. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- caop_check_hamfile also reads each term's modes and the statistics, and caop_check_modes refuses a mode at or beyond system_size, which upstream's solve never checks. The spectrum and the variance reuse both. - omp_set_dynamic(0) before the solve, as upstream's caop-spectrum app: the stored Hamiltonian (methods 1/3) keeps per-thread slots built by a fixed team. - The Thrust t > b comment names the real cause: the mult returns early on an empty slide and skips the Allreduce the other ranks wait in. - README: the block runs from this folder, without cd. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s only; label reevaluated rows - On the Thrust backend with the on-the-fly Hamiltonian, t_comm_size above the b ranks is refused: the mult returns early on an empty slide and skips the Allreduce the other ranks wait in. - The Hamiltonian's and each operator file's modes are checked against sites, and each operator's statistics against H's, on every rank; upstream checks them on some ranks only, so a bad file hung the others. - omp_set_dynamic(0), as upstream's app: the stored Hamiltonian's per-thread slots. - caop_spectrum_from_coefficients returns orbitals and operator_files; the driver's --read-coefficients CSV labels its rows as the generating run did. - eta defaults to 0.1 and --points to 201, as upstream's caop-spectrum. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The workflow page's blocks each start in examples/, and the CAOP block writes its own input files; the analysis README runs its CAOP check after the CAOP README's first two steps. - No OpenMPI-only mpirun -x in the READMEs and driver docstrings: set OMP_NUM_THREADS in the shell, which works with every MPI. - The analysis README names step 1's h2o energy as such. - Top README: CAOP in the overview; the variance section points to the examples instead of repeating them; heatbath_batch_size's default is 1000000. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> # Conflicts: # README.md
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> # Conflicts: # docs/apidocs/sbd.rst # examples/gdb/run_gdb_diag.py # examples/tpb/README.md # examples/tpb/run_sbd_diag.py # examples/tpb/run_sqd_enlarge_subspace_sbd.py # examples/tpb/run_sqd_sbd.py # python/bindings.cpp
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> # Conflicts: # docs/apidocs/index.rst
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> # Conflicts: # README.md # examples/analysis/README.md # examples/analysis/variance.py # python/analysis.py
The state is solved and saved with b_comm_size = ranks and evaluated from those shards on the same ranks; the result must match the exact values and agree across ranks. The other numerical check runs in one process. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # python/bindings.cpp
…s defaults Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # docs/apidocs/sbd.rst # examples/caop/README.md # python/bindings.cpp
…-the-fly H on the GPU The README, the release note and green_function's docstring said the spectrum runs on the cpu and gpu backends "like CAOP", which overstated the GPU part. As upstream's, only applying the Hamiltonian on the fly (--hamiltonian-mode on-the-fly, store_hamiltonian=False) runs on the GPU; the stored Hamiltonian, the default, and the rest of the work run on the CPU. The README now says so up front, with both the driver option and the Python argument. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # examples/caop/README.md
The page grew driver by driver and no longer read as a whole: the upstream README links were reduced to app folders, variance was a bare bullet, the "Differences" and "Running it" sections were the solve's only without saying so, the GPU rules were split across two places, and Checks listed only the solve's checks, without saying they are this package's and not upstream's. Now: a table of the three drivers with their upstream app READMEs; how they fit (solve, check the state, then its Green's function, and the settings each later run must repeat); a section per driver, the variance one a short pointer to analysis/README.md#caop-states; MPI; GPU for all three; and Checks grouped by driver, stating that the bindings refuse these before upstream runs, with upstream unchanged. Headings keep the anchors other pages link to. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # examples/caop/README.md # python/bindings.cpp
… and limits Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 2-site Hubbard examples ran every CAOP driver on one process, so nothing showed the state saved over several files and read back with --shards / --wavefunction-shards. They now run on 4 ranks (--b_comm_size 4), in the workflow page, the CAOP README and the analysis README's CAOP section; the coefficient re-evaluation stays one process, as it reads no state. analysis/README.md's "Doing it in the same run" now shows caop_variance after caop_diag_from_files in one job, and its Device note says CAOP's evaluator runs from the loaded backend, Thrust included, unlike TPB and GDB's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… grid The solve, the variance and the Green's function run with MPI and OpenMP and may each use their own rank count; reevaluating saved coefficients is a single process, as upstream's app, since it reads no state. The MPI section now says so, and how each of the three sets h, b and t. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Binds upstream SBD's
caop-stat-evaluator: the stochastic external energy variance and Epstein–Nesbet PT2 of a CAOP ground state, assbd.analysis.variancegives for TPB and GDB states.sbd.analysis.caop_variance(state, hamfile, basisfiles, energy, ...)takes a state saved bysbd.caop_diag_from_files(..., savename=...), the basis files it was solved over, and upstream's evaluator options, and returns the sameVarianceResultasvariance(). As upstream,energymay beNoneonly forobservable="variance". It runs on the CPU with MPI + OpenMP, from thecpuorgpu(Thrust) module.examples/analysis/variance.py --hamfileruns it, with the CAOP options in their own group; the TPB/GDB options and their--energyrequirement are unchanged.Checks. Upstream reads the state onto the basis by determinant and normalizes it, with no type tag or size check, so a mismatch gave a wrong answer with exit status 0. The bindings refuse on every rank, before upstream does any work: a saved state that does not match the basis files given (its determinant count, which determinants they are, its word size, real or complex, and its norm); a duplicated determinant; a Hamiltonian file upstream cannot read; a mode beyond
sites; more saved files than basis ranks;h_comm_sizenot dividing the rank count.Docs.
examples/README.mdshows how the examples fit together: for molecules, solve then check; for lattice and spin models, solve, check, then the Green's function.sbd.spectrumamong the features and no longer says CAOP is unsupported; its variance section now points to the examples.examples/analysis/README.mdgets a CAOP states section.examples/caop/README.mdis reorganized around the three CAOP drivers: what each computes and its upstream app, how they fit together, a section per driver, what runs on the GPU, and the checks grouped by driver.heatbath_batch_sizedefault, which is 1000000 (upstream's and the binding's), not 200000000, and replaces the OpenMPI-onlympirun -xin two driver docstrings.Tests (
test_caop_variance.py): variance and PT2 of a Hubbard subspace against exact values from upstream's definitions, computed in NumPy; the variance-only observable without an energy; each refusal; thevariance.py --hamfiledriver end to end. Under MPI: bad inputs refused on every rank, and a state saved over every rank against the exact values, identical on every rank.Checked on an 8× H100 server: the full suite passes on the
cpu,gpuandgpu-ompbackends, as does the MPI suite on 2 and 4 ranks on each.🤖 Generated with Claude Code