Status (2026-10-07): A, B1, B2 and C merged (#205–#208), with D's docs; what remains is the 0.7.0 release (and publishing compose-api-client to PyPI), each on Jim's go, and the open questions below.
(2026-10-13: D is done. The workflow merged in #210, and compose-api-client 0.7.6 is on PyPI, published by a
workflow_dispatch run on 2026-10-10. No GitHub release has been made since 0.5.0, so the on-release trigger has never
fired. The spec now has 49 operations, each with a command; --json on every command came in #221.) Three PRs, each merged on green with a merge
commit. Publishing to PyPI and deploying are separate goes from Jim.
| Step | What | State |
|---|---|---|
| A | Generation pipeline: deterministic, in-repo, drift-checked in CI; regenerate the stale client | done #205 |
| B1 | compose-api-client as a workspace package at clients/python, brought in with its history (C6) |
done #206 |
| B2 | The hand-written application layer, compose_api_client.ext |
done #207 |
| C | The compose-api CLI on ext and the generated client; a spec-coverage test |
done #208 |
| D | Docs (docs/cli.md, CLAUDE.md) done in #208; PyPI trusted-publisher workflow; release with 0.7.0 |
done: workflow #210; trusted publisher added; 0.7.6 published 2026-10-10 |
compose-api has no command line. Today a user drives it with curl, the generated client in Python, or pbest's
run_simulation_and_wait. The goals:
- Exercise every operation. One command per operation; a test fails if the spec gains an operation the CLI does not claim.
- Build on a client generated from the OpenAPI spec, httpx-based, which pytest drives against the FastAPI app in-process.
- Two layers above the generated code, with different concerns:
- an application layer for programs that integrate the service (waiting, downloading, typed errors);
- the CLI, for people and scripts (output formats, exit codes, configuration).
The precedent is sms-api's (viva-api's) plan for its own core CLI, decision D8 of
plan-core.md: "an independent CLI
targeting only core, built on the OpenAPI client generated from the core spec … no hand-rolled URLs, no server
imports". That CLI is deferred to its Phase 2 (P8) and not built yet. This plan is the same shape, done first here.
There is no auth and there are no websockets. compose_api/api/main.py mounts curated, simulation, results
and compute (prefix /core).
| Method | Path | operationId | Notes |
|---|---|---|---|
| GET | /health |
check_health_health_get |
{docs, version} |
| GET | /version |
get_version_version_get |
|
| POST | /simulation/run |
run-simulation |
multipart uploaded_file; interval_time, batch_submission, simulator; 400 detail is a string or {message, violations} |
| POST | /curated/copasi |
run-copasi |
multipart sbml; start_time, duration, num_data_points |
| POST | /curated/tellurium |
run-tellurium |
multipart sbml; start_time, end_time, num_data_points |
| GET | /results/simulation/status |
get-simulation-status |
404 until the SLURM job exists |
| GET | /results/simulations/status/batch |
get-simulations-status-batch |
a GET with a JSON body (list[int]) |
| GET | /results/simulation/results/file |
get-simulation-results-file |
zip |
| GET | /results/simulator/build/status |
get-simulator-build-status |
404 as above |
| GET | /core/simulator/list · /core/processes/list · /core/steps/list |
get-simulator-list · get-processes-list · get-steps-list |
pbest calls four of these by operationId (run-simulation, get-simulations-status-batch, get-simulation-status,
get-simulation-results-file). Renaming any of them breaks pbest.
-
Generator:
openapi-python-client0.29.1 (dev,<0.30), run byscripts/generate-api-client.shfrommake clients. Its output,compose_api/api/client/, is committed and ships inside the server wheel.- It gives attrs
Client/AuthenticatedClientwrappinghttpx.Clientandhttpx.AsyncClient, withsync,sync_detailed,asyncioandasyncio_detailedper operation, and 23 typed models. - Tests drive it in-process:
tests/fixtures/api_fixtures.py::in_memory_api_clientinstallshttpx.AsyncClient(transport=ASGITransport(app=app)).
- It gives attrs
-
Stale: the client was generated 2026-09-08; the spec was regenerated 2026-10-06.
run_simulationhas nosimulatorparameter (#190).- No 400/404 is parsed: a 404 raises
UnexpectedStatusor returnsNone.
-
Not reproducible. Regenerating today differs from the committed tree in every file, mostly formatting:
- The committed tree is raw template output (unsorted imports, an unused
from urllib.parse import quote). - A regeneration on a machine with
ruffon PATH runs the generator's default post-hooks, which format with ruff's defaults, not this repo's settings.
So the output depends on the machine, and a CI drift check is impossible until the post-hooks are pinned.
- The committed tree is raw template output (unsorted imports, an unused
-
make clientshas three problems:- It refuses to run without
LIB_DIR, because it also writes into the separatecompose-api-clientrepository (PyPI 0.2.0, which pbest uses). - It calls a bare
openapi-python-client. - It runs the spec script with
python3rather thanuv run.
- It refuses to run without
-
No CI step checks the spec or the client.
strategy.md plans one uv workspace with three distributions:
viva-toolkit: pbest renamed, the local-or-remote run command line;compose-api-client: the generated client plus its one hand-written helper, outside the generated tree;compose-api: the service.
Phase 2b's gate is "make clients writes only in-repo". This plan does 2b's client half. It doesn't wait for pbest to
be absorbed.
atlantis is viva-api's typer CLI (app/cli.py, about 3,700 lines). It already has a compose group with the same
verbs: run, status with several IDs, results, simulators, processes, steps, build-status, copasi and
tellurium.
Worth copying:
- typer sub-apps per noun (
cli.add_typer(compose_cli, name="compose")); - one error handler (a decorator plus
main()) that renders connection, HTTP, JSON and other errors as a panel and pullsdetailout of FastAPI error bodies; - exit codes 1 (error), 2 (bad input), 130 (Ctrl-C);
- opt-in polling with a status line per tick and a final panel;
- a spinner while submitting;
CliRunnertests with the service factory patched, plushttpx.MockTransportfor the HTTP layer.
Not worth copying:
- Its HTTP layer is hand-written.
E2EDataServicebuilds URLs from string literals and returns the server's own pydantic models. Its route-drift test explains why: atlantis ships inside viva-api, and that generator returned bareAnyfor 20 of 20 operations.- Neither reason holds here. A standalone client cannot import server models, and this repo's generated operations are typed.
- The hand-written layer also needs an AST test just to notice that a URL stopped existing. Generated code goes stale too, but then a drift check finds it mechanically.
- No global options: every command repeats
--base-url, and there is no output-format option. - Packaging: the CLI ships in the server's distribution with typer, textual and marimo as core dependencies, so
pip install viva-apiinstalls the server stack. strategy.md forbids that here: "a laptop install must not pull the server".
| # | Decision | Why |
|---|---|---|
| C1 | Keep openapi-python-client, pinned, with a committed config. |
httpx-native, sync and async; typed attrs models; httpx_args and set_*_httpx_client accept ASGITransport/MockTransport, so pytest needs no server. Already in use. Rejected: openapi-generator (Java, urllib3 by default; openapitools.json is a leftover to delete); a hand-written httpx client (atlantis's road, above); commercial SDK generators (not justified for 12 operations). |
| C2 | The generated code is never edited; behaviour goes in ext. |
Regeneration must be free. pbest's utils/run_simulation_and_wait.py shows what happens otherwise: hand-written code inside a generated tree, with a standing warning not to overwrite it. |
| C3 | The CLI is hand-written typer over ext, with a spec-coverage test. |
A spec-driven runtime CLI (restish-style) gets coverage for free but has poor UX: raw query and body flags, no --wait, no file handling. The CLI's real concerns are presentation, waiting and files. The coverage test gives back what that approach would have guaranteed. |
| C4 | compose-api-client is a workspace package with an optional [cli] extra (typer, rich). The command is compose-api. |
The base install is httpx and attrs only, for applications; strategy.md's packaging constraint. The command drives the service; viva-toolkit builds and runs documents. They are two tools with two names. |
| C6 | clients/python comes in from compose-api-client at tag 0.2.0 by git subtree, with its history (30 commits, Ezequiel Valencia), and the root overrides pbest's compose-api-client==0.2.0 pin so it resolves to the workspace package. (Found while starting B, 2026-10-07.) |
pbest pins the client exactly, so a workspace package of that name at any other version cannot resolve, and two packages cannot both install compose_api_client. The override is safe for the service: it imports only pbest.utils.input_types and the containerization code, never pbest.execution.remote (the client's one user), and the regenerated client keeps every name that module uses (Client, api.simulation.run_simulation, models, types.File, utils.run_simulation_and_wait). Bringing it in with history is strategy.md's rule 1 ("merge with history, never by copying") applied to phase 2b. |
| C5 | The API is unchanged by this work. The GET-with-body batch endpoint stays. | Moving ids to the query string would change pbest's call signature. That's a separate decision, recorded in §Open questions. |
scripts/openapi-python-client.yaml, committed:post_hooks:ruff check --fixandruff format, run throughuv runagainst this repo's ruff configuration (line length, isort).project_name_override/package_name_overridefor the target package.- The output is then the same on every machine.
make clients:uv run python compose_api/api/openapi_spec.py, thenuv run openapi-python-client generate --config …, writing only in-repo.LIB_DIR, when set, also writes the external repository, for the 0.2.x line pbest pins; the 0.2.x repository is retired at strategy phase 5.
make check-clients, added tomake checkso CI runs it inquality:- generate the spec and the client into a temporary directory;
diffboth against the committed copies;- on any difference, fail with "run
make clients".
- Regenerate and commit the new client. It picks up
simulatorand the 400/404 handling. Adjust the few tests that relied on a 404 raising. - Delete
openapitools.jsonand the commented-out openapi-generator lines.
As built (B1, 2026-10-07):
git subtree addof compose-api-client at tag 0.2.0 intoclients/python, then the old repository's tooling is dropped (its own uv.lock, pre-commit, Makefile,publish.sh,main.py,.idea), and a test that submitted to production is dropped too. That test returns in B2 as acluster_onlytest ofext.run_and_wait.- Version: follows the service (0.6.0 now).
tag.shbumps both. - The wheel: built alone, it installs on Python 3.11 with httpx, attrs and dateutil only, with no FastAPI or SQLAlchemy.
- Typing: the script adds a
py.typed, so mypy reads the generated types in the service's tests. - The root project:
- depends on the package only in its dev group (the service never calls the client; its tests do);
- overrides pbest's pin (C6);
- keeps the generated files out of ruff and mypy, by path.
- Gotcha:
uv pip installrun from the repository root applies that override and fetches 0.2.0 from PyPI. Run ad-hoc installs from outside the repository.
clients/python/ is a uv workspace member: distribution compose-api-client, version locked to the service.
compose_api_client/(generated): the client moves here fromcompose_api/api/client/(C6: by subtree from the external repository, then regenerated in place; its hand-writtenutils/is kept).- Nothing outside this repository imports
compose_api.api.client(pbest importscompose_api_client), so the in-repo copy is deleted rather than shimmed; the server's tests importcompose_api_client.
- Nothing outside this repository imports
compose_api_client/ext/(hand-written, outside the generated tree). Its only dependencies are httpx and attrs.
| Piece | What |
|---|---|
ComposeSession(base_url=… | client=…, tokens=, timeout) / AsyncComposeSession |
Context managers that own per-attempt guarded clients (TokenSource / AsyncTokenSource, _DestinationGuard, trust_env=False, follow_redirects=False, 1-renewal read replay on 401). |
submit(file, simulator=, interval=, batch=), submit_curated(engine, sbml, …) |
The SimulationExperiment |
status(id) → JobState |
.status, .terminal, .ok, .slurm_job. A 404 means SUBMITTING, not an error, until a deadline. |
wait(id, poll=, timeout=, on_update=), build_status(id), wait_build(id) |
The final HpcRun |
results(id) -> bytes, download(id, dest, extract=False) |
The zip, saved or unpacked |
run_and_wait(...) |
Submit, wait, download. Replaces pbest's helper and the tests' check_experiment_run. |
| Errors | NotFound, BadRequest(detail, violations), ServerError, ApiTimeout, mapped from the *_detailed responses. |
tests/simulators/utils.py::check_experiment_run moves onto run_and_wait.
As built (B2, 2026-10-07):
- Results are split:
download(id, dest) -> Pathsaves the zip;extract(id, dest) -> list[Path]unpacks it and refuses paths that would escapedest.run_and_waitreturns the files written as a list, empty unless the job completed. Splitting them avoids handing every caller aPath | list[Path]union. statuses(ids)is the batch call.token=selectsAuthenticatedClient(a Bearer header) for #192.- Regeneration:
make clientscarriesext/(andutils/) across, andcheck-clientsignores both. - Typing:
extis under mypy strict. - Tests:
tests/client/test_ext.py:httpx.MockTransport, no service.tests/client/test_ext_in_process.py: the real app throughin_process, Postgres only. It pins 404 →submittingon status, 404 →NotFoundon results, and the partial batch answer.tests/client/test_ext_live.py: a read-only check of production under--slurm-backend cluster.check_experiment_runnow waits throughAsyncComposeSession.wait. That makes the service's own tests the first user ofext.
Later, not in this plan: an ensemble helper (seed blocks, a manifest, resume, retry, splitting on timeout) can
generalise viva-pde-particle's compose_client.EnsembleRun once this ships on PyPI.
compose_api_client/cli/, entry point compose-api = compose_api_client.cli:main, installed with
compose-api-client[cli].
Global options, on the root callback and not repeated per command (unlike atlantis):
| Option | Env | Default |
|---|---|---|
--url |
COMPOSE_API_URL |
https://compose.cam.uchc.edu |
--timeout |
COMPOSE_API_TIMEOUT |
300 s |
--output {table,json,yaml} |
COMPOSE_API_OUTPUT |
table on a TTY, json when piped |
--profile |
COMPOSE_API_CLI_PROFILE |
production (--token CLI flag removed in 0.8.0; COMPOSE_API_TOKEN is env-only for automation) |
--verbose / --quiet |
--verbose logs the HTTP exchanges |
Commands. Each operationId belongs to exactly one command:
| Command | Operation(s) |
|---|---|
health, version |
check_health_health_get, get_version_version_get |
simulators list · processes list · steps list |
get-simulator-list · get-processes-list · get-steps-list |
run FILE [--simulator NAME] [--interval T] [--batch] [--wait] [--download DIR] [--extract] |
run-simulation |
curated copasi SBML --start --duration --points [--wait …] |
run-copasi |
curated tellurium SBML --start --end --points [--wait …] |
run-tellurium |
status ID… [--watch] |
one ID: get-simulation-status; several: get-simulations-status-batch |
results ID [-o FILE | --extract DIR] |
get-simulation-results-file |
wait ID… [--poll S] [--timeout S] |
status, through ext.wait |
build-status ID [--wait] |
get-simulator-build-status |
openapi [--server] [--diff] |
The bundled spec; --server fetches the live /openapi.json; --diff reports client/server skew |
Behaviour:
-
Exit codes:
Code Meaning 0 Success, or the job completed 1 The job ended failed, timed out, out of memory or cancelled 2 Usage error 3 API error, with detailand theviolationsrendered4 Not found 130 Interrupted run --waitreturns the job's outcome, socompose-api run x.omex --wait && …can be scripted. -
Waiting: a rich status line per poll on a TTY and nothing when piped; a final panel or JSON record.
-
Errors: one handler, rendering as atlantis does;
--verboseadds the traceback. -
Registry: commands declare their operationIds with a decorator,
@claims("run-simulation"). The coverage test reads that registry.
As built (C, 2026-10-07), with three deviations from the design above:
--outputisauto|table|json, with no YAML. YAML would add a PyYAML dependency for little gain.- Exit code 5 means a
--wait-timeoutpassed. The plan had folded that into 1, but scripts want to tell "the job failed" from "I gave up waiting". - The CLI tests run over
httpx.MockTransport, not the in-process app. The CLI is synchronous, and the app is reachable in process only through an async transport, with fixtures bound to pytest's event loop. The CLI's own concerns (output, exit codes, files, error rendering) are tested over the mock. What the service actually answers is pinned one layer down, intests/client/test_ext_in_process.py.
Other details:
- Layout:
- The module is
compose_api_client/cli/commands.py(named so it doesn't shadow the exportedapp). resultstakes--out FILE|DIRor--extract DIR.- The spec ships inside the package as
openapi.json, generated with the client. It is whatopenapiprints and what--diffcompares. - The client package's ruff target is Python 3.11, extending the root config.
- The module is
- Tested on production 0.6.0 (2026-10-07):
health,version,simulators list,status(single and batch),results --extractandopenapi --diff(no skew) all behave as documented.run experiment.omex --simulator viva-pde-particle --wait --download out/ --extractran simulation 4195 (SLURM job 4137519) tocompletedin 31 s and unpacked its.pber.- The wheel with
[cli]installs and runs on Python 3.11 without FastAPI.
- Docs:
docs/cli.mdis a guide followed by a command reference.make cli-docsgenerates the reference from the typer app;make check-clientsfails when it is stale.
- Coverage test. It fails CI on drift in either direction:
- every operationId in the committed spec is claimed by exactly one command;
- the registry names no operation the spec lacks.
- CLI tests:
CliRunnerwith the session factory pointed atComposeSession.in_process(app), a fixture besidein_memory_api_client. They cover:- each command once;
- the output formats;
- every exit code;
run --wait --downloadagainst the existing SLURM-mocked fixtures;- the rendering of 400 violations and 404s.
exttests: polling, deadlines and error mapping throughhttpx.MockTransport; downloads intotmp_path.- Docs:
docs/cli.md: install, connect, a tutorial (run, wait, download), and a command reference generated bytyper compose_api_client.cli utils docs;- a
CLAUDE.mdregeneration section; - the strategy.md 2b row.
- Release: with the service, 0.7.0, alongside #192. Publishing
compose-api-clientto PyPI is a separate go.
As built (publishing, 2026-10-07):
- The workflow:
.github/workflows/publish-client.ymlpublishescompose-api-clientto PyPI through a trusted publisher (GitHub OIDC; no API token is stored anywhere).- It runs when a GitHub release is published (the same go that deploys the docs and archives to Zenodo), or by hand with a tag.
- It checks that the client's version is the tag, builds, and smoke-tests the wheel on Python 3.11 (it must import,
run
compose-api --help, pull in no FastAPI, and passscripts/client-smoke.py, which prepares private session state and closes the sign-in listener: the code--helpnever reaches whose behaviour depends on the Python version).cli-platforms.ymlruns the same script on 3.11 on Linux, macOS and Windows for every client change, andmake checktypes the hand-written client a second time as Python 3.11. - It then uploads from the
pypienvironment, which accepts deployments only from tags andmain. Add required reviewers there for a manual gate.
- The PyPI side, once, by an owner of the existing project (0.1.0–0.2.0 were uploaded with a personal token from
the old repository): at https://pypi.org/manage/project/compose-api-client/settings/publishing/, add a GitHub
publisher with owner
biosimulations, repositorycompose-api, workflowpublish-client.ymland environmentpypi. - The first publishable release is 0.7.0. Tag 0.6.0 predates
clients/python. - PyPI goes from 0.2.0 straight to 0.7.0, because the client now versions with the service. pbest keeps its
==0.2.0pin until it is bumped (the four-step release in CLAUDE.md). (2026-10-13: pbest#55 bumped it to==0.7.6on 2026-10-10, but pbest has not released since 0.6.3, so compose-api'spbest==0.6.3pin and itsoverride-dependenciesentry for the client stay.)
make check-clientspasses on a clean tree. Adding a route without runningmake clientsmakes it fail.uv run pytest -m "not slurm"is green, including the coverage test andtests/cli/. CItests-slurmis green.uv buildforclients/pythoninstalls into a clean venv with neither FastAPI nor SQLAlchemy. Thenpip install 'compose-api-client[cli]'givescompose-api --help.- Against production (0.6.0):
compose-api healthcompose-api simulators listcompose-api run experiment.omex --simulator viva-pde-particle --wait --download out/, which exits 0 and leaves the.pberinout/compose-api status <id> --output jsoncompose-api openapi --server --diff, which reports no skew
- Moving the client changes import paths for pbest's 0.2.x line. Mitigation: the external repository keeps
publishing 0.2.x from
LIB_DIRuntil strategy phase 5. The in-repo shim covers this repository's own imports. - Generator upgrades reformat everything. Mitigation: pin
<0.30and upgrade deliberately in its own PR. The drift check makes the diff visible. - The FastAPI 0.141 upload schema needs
_restore_binary_format(generator issue #1417). The drift check compares the post-processed spec, so a fix upstream only shows up as a clean diff.
- Auth: decided in 0.8.0 (
compose-api auth login|signup|status|logoutwith OS keyring storage and PKCE/device flows;--tokenflag removed,COMPOSE_API_TOKENenv-only for automation). This supersedes the alternative design in #310 (a stored token file,--tokenkept). - A TUI or GUI (atlantis has both; not needed here).
- Folding atlantis's
composegroup into this CLI. viva-api could later delegate tocompose-api-client, which is viva-api's decision. - Ensembles: a later
exthelper; viva-pde-particle has its own today.
- The batch-status endpoint: keep the GET-with-body, or add a query-string form (
?ids=1&ids=2) beside it? The addition would be non-breaking, the replacement would break pbest. Proposed: add the query form in 0.7.0 and keep the body form until pbest moves. (2026-10-13: still open; only the body form exists,routers/results.py.) The command name:Settled:compose-api, as shipped in #208.compose-api(proposed: names the service), orcompose, which is shorter but generic and collides withdocker composemuscle memory?- When the 0.2.x external repository stops being written: (2026-10-13: still open; it waits on a pbest release that carries the 0.7.6 pin.) at strategy phase 5 (proposed), or as soon as pbest pins a release from this repository (0.7.0 or later; the client now versions with the service)?
Prebuilt simulators can't be discovered through the API.Answered (2026-10-13):GET /registry/environments(compose-api environments list) names them, andsimulations list --simulator NAMEfilters by one. (Found building C.)simulators listreturns container definitions, with no names. The namesrun --simulatoraccepts (PREBUILT_SIMULATORS, for exampleviva-pde-particle) exist only in the deployment config. Proposed: aGET /core/simulator/prebuilt(name → image digest), and acompose-api simulators prebuiltcommand for it.