pystagegate

Automated quality assurance methods for demographic outputs. Work in progress.

Contact: Christoffer.Soderberg@ons.gov.uk

Instuctions

This package uses uv for virtual environments and dependency management.

• Installation
• Getting started with uv

Running the pipeline

Install the package into your Python environment of choice:

pip install git+https://github.com/ONSdigital/pystagegate.git

Download a copy of the configuration and save it to your working directory.

Running from the command line

Open your Python environment of choice and in your working directory use the console command run_prov_fin to run the provisional-final migration data quality assurance pipeline:

run_prov_fin pipeline_config.json

Similiary, there is a console command for the sex-ratio pipeline:

run_sex_ratio pipeline_config.json

And also the national profile pipeline:

run_national_profile pipeline_config.json

Running in Jupyter Notebooks

Import the prov_fin_main function from the pipelines module:

from pystagegate.pipelines import prov_fin_main

Call the function with the filepath to the configuration JSON file as an argument (or pass a Python dictionary):

prov_fin_main("pipeline_config.json")

See the demo for more detail.

Pure synthetic data for testing is available in tests/data

Contributing

• uv sync to read dependencies
• uv run pytest for testing
• uv run ruff check and uv run ruff format for linting