Auracle is a reference architecture for agentic customer support. A language model classifies each inbound message, finds the documents that answer it and drafts a reply. The system acts alone only when two independent gates pass. Every reply to a customer still waits for a person to approve it.
Who it is for. Teams who build or evaluate support automation and want a design they can read, run and test end to end. The code is TypeScript on Postgres, with no orchestration framework.
What it is not. It is not a hosted product, and nothing publishes it to npm. It is a working system to study, run on your own infrastructure and adapt.
Status. Version 0.1.0 is the first public release. Read its release notes. CHANGELOG.md records each release. Every number this repository publishes comes from a synthetic corpus. So the numbers test the pipeline, not the quality of the answers.
License. Apache License 2.0. See LICENSE and NOTICE.
| Term | Meaning |
|---|---|
| Ticket | A customer's request, as the helpdesk holds it. |
| Turn | One inbound message of a ticket's conversation. |
| Intent | What a turn asks for, such as a refund or a copy of an invoice. The classifier names it. |
| Classifier | The step that asks the model for a turn's intent. |
| Temperature | The model setting that lets repeated samples of one question differ. Above zero, they can. |
| Retriever | The step that finds the documents that may answer a turn. |
| Composer | The step that drafts a reply from the retrieved documents, or abstains. |
| Run | One pass of the pipeline over one turn, or over a ticket that has no conversation. |
| Case | An item a person examines: a turn to label, or the runs of one ticket a reviewer takes. Each turn of the gold set is a case. |
| Ledger | The Postgres database that records every ticket, turn, run and action. |
| Step journal | The ledger table where a run records each step before it acts. |
| Coverage | The share of a draft's claims that the retrieved documents support. Higher is better. |
| Agreement | The share of the nine classifier samples that name the same intent. Higher is better. |
| Policy engine | The code that decides, from the evidence and the rules, whether the system acts alone. It calls no model. |
| Gate | A test that a run must pass before it continues. The agreement gate reads agreement, and the groundedness gate reads coverage. |
| Agreement threshold | Agreement below it escalates the turn. |
| Block threshold, review threshold | The groundedness gate's two thresholds. Coverage below the block threshold escalates the turn, and coverage below the review threshold parks it. |
| Internal write | An action that changes a helpdesk ticket and sends nothing to the customer, such as closing or routing it. |
| Automatic path | The route where the system acts alone, with no person. Only an internal write can take it. |
| Abstain | The composer's answer when no retrieved document supports a reply. |
| Park | Keep an action waiting until a person approves it. A customer reply waits as a helpdesk draft. It waits in the ledger when the helpdesk cannot send that copy (DEC-44). An internal write waits in the ledger. |
| Escalate | Hand the turn to a person, with no action for them to approve. A parked turn has an action ready, and an escalated turn has none. |
| Outbox | The table an action waits in until the relay sends it. It commits with the state change that caused it. |
| Adapter | The code that connects the system to one helpdesk or mailbox. It declares what it supports. |
| Idempotency | An adapter's guarantee that a repeated send of one action has the effect of one send. |
| Read-back | An adapter's ability to read an action's result back from the helpdesk. It tells the system whether a send arrived. |
| Consumer | The daemon that takes each turn through the pipeline. |
| Relay | The daemon that sends each outbox action to the helpdesk adapter. |
| Daemon | A long-running process: the webhook, the IMAP poller, the consumer, the relay or the console. |
| Gold set | The labeled synthetic turns that fit and measure the gate thresholds. Each turn in it is a case. |
| Fit half, report half | The gold set's two halves. A threshold comes from the fit half, and a published rate comes from the report half. |
| Risk tier | The cost of an error for one intent: low, medium or high. A person sets it, and a high-tier intent always goes to a person. |
| Control band | A daily metric with the range it should stay inside. |
| Containment | The share of cases the policy engine sends down the automatic path. Neither direction is better in itself: read it beside action match. |
| Routing decision | The policy engine's answer for a turn: act alone, park or escalate. |
| Action match | The share of cases whose routing decision matches the recorded outcome. Higher is better. |
| Stub model | A deterministic stand-in for the model. It returns a fixed answer and costs nothing. |
| Artifact | The JSON file an offline job writes, with its inputs and its result. |
| Corpus | The synthetic tickets, documents and gold set under fixtures/. The generator, scripts/generate-fixtures.mjs, writes all of it. |
| Replay | An offline run of the classifier and the policy engine over the gold set's cases. Action match scores it against each case's recorded outcome. |
| Interval | A 95 percent confidence interval around a rate. A narrower interval is more certain. |
| Register | The decision register, which holds every design decision. |
| DEC | A numbered decision in the register. DEC-01 to DEC-51 are the design authority. |
The register's appendix defines every technical name it uses.
One turn moves through these steps. The architecture map draws the same path.
- Intake. A webhook or an IMAP mailbox delivers the message. Intake stores it as a ticket and a turn in the ledger.
- Classification. The classifier asks the model for the turn's intent nine times, at a temperature above zero (DEC-05).
- Retrieval. The retriever finds the documents that may answer the turn. It merges a keyword ranking and a vector ranking by rank alone (DEC-21).
- Composition. The composer drafts a reply that cites its sources, or it abstains (DEC-07).
- Groundedness. A check tests each claim of the draft against the retrieved documents, and that gives the draft's coverage (DEC-07).
- Policy. The policy engine reads agreement, coverage and the risk tier of the intent. It decides whether the system acts alone, parks the action for a person, or escalates (DEC-03, DEC-13).
- Action. The policy engine applies its rules before it reads the gates. A customer reply parks, whatever the gates read (DEC-12). So do an irreversible internal write (DEC-03, DEC-34) and a high-tier intent (DEC-13). So does an internal write through an adapter that lacks idempotency or read-back (DEC-06). Any other internal write then reaches the gates. It escalates below the agreement threshold or the block threshold. It parks below the review threshold. At or above all three, it takes the automatic path, through the outbox.
- Approval. A person approves each parked action, or sends back or closes its case. They do it in the helpdesk or in the operator console (DEC-11, DEC-40, DEC-44).
The two gates stay separate (DEC-05). Each one catches a failure the other cannot see, and one blended score can pass while either gate fails.
Adapters connect the system to a helpdesk. The repository ships three: a reference adapter that sends nothing, an IMAP and SMTP mailbox, and FreeScout.
This path takes a fresh clone to passing tests and a running console with a seeded history. It runs on your own machine and needs no model API key. The next section, Run the live pipeline, adds the key.
- Node 22.22 or later, on major version 22, 24, or 26 and later. vitest
does not run on Node 23 or 25.
.nvmrcnames the major version CI uses. - pnpm 11.20.0. On Node 22 or 24, run
corepack enable. Node 26 and later ship no corepack, so runnpm install -g pnpm@11.20.0there. - Docker with Compose, for Postgres and a test mail server.
- Python 3.10 or later, for the documentation and guard checks.
- git.
git clone https://github.com/synaptiai/auracle.git
cd auracle
./scripts/bootstrap.sh
docker compose --profile test up -d --wait
export DATABASE_URL=postgres://auracle:auracle@localhost:5433/auracle
export READER_DATABASE_URL=postgres://auracle_reader:auracle_reader@localhost:5433/auracle
export MAIL_HOST=127.0.0.1
pnpm run db:migrate
pnpm run verify:fast./scripts/bootstrap.shinstalls the dependencies from the lockfile, installs the commit hook and copies.env.exampleto.env. It also runscorepack enable. It overwrites no file that already exists.- The compose stack starts Postgres with pgvector, its vector-search
extension, on port 5433. It also starts GreenMail, a test mail server, on
ports 3025 and 3143. Every port binds
127.0.0.1.--waitreturns once both answer, so the migration never meets a database that does not answer. - The three variables point every command at that stack. Some commands read
no
.envfile, so export them in your shell. pnpm run db:migratecreates the schema andauracle_reader, the read-only database role the console's readers connect as.pnpm run verify:fastruns the type check, the lint and the unit suites. It also runs the documentation, style and guard checks and the fixtures diff. It skips the database and mail suites and the console's tests. CONTRIBUTING.md lists what each verify command covers.
pnpm run verify is the full check a pull request must pass. It adds the
database and mail suites and the console's own tests, so it needs both
services:
pnpm run verifypnpm run reviewers -- --add reviewer@example.test
pnpm run seed-history
CONSOLE_DEV_REVIEWER=reviewer@example.test pnpm run dev:consoleThen open http://127.0.0.1:8787. The console shows the queue of cases that need a person.
reviewers -- --addputs a reviewer on the roster. The console serves only a reviewer on the roster.seed-historywrites 45 days of synthetic traffic through the real ledger path. It needs no model.CONSOLE_DEV_REVIEWERnames that reviewer for a console run from source on loopback, the local address127.0.0.1. A deployed console reads the reviewer from a trusted proxy instead, as the runbook describes.
The daemons process real tickets, and the consumer calls a model. Before any
daemon takes real traffic, read SECURITY.md and the
compliance position below. The webhook accepts every
request until you set WEBHOOK_SECRET, and the console needs a proxy that
authenticates each reviewer.
Put an OpenRouter key in .env as
OPENROUTER_API_KEY first. Without it, the consumer refuses to start with
OPENROUTER_API_KEY is not set.
The offline jobs that call a model need the key too, and some need more:
| Command | Also needs |
|---|---|
pnpm run induce |
Nothing more |
pnpm run load-docs -- <dir> |
A directory of markdown documents to embed, and DATABASE_URL |
pnpm run calibrate |
CALIBRATION_MODEL_VER, the classifier model to calibrate |
pnpm run fit-groundedness |
GROUNDEDNESS_MODEL_VER, the model that checks each claim |
pnpm run report-groundedness |
GROUNDEDNESS_MODEL_VER, and a pair from fit-groundedness |
Each model version must equal the consumer's MODEL_VER. The consumer refuses
a threshold from a different model. A job that lacks what it needs
names it and refuses to start. The jobs replay, gold-report and triage
use a deterministic stub model and need no key.
The runbook covers each daemon, each variable and the container image.
| Document | What it gives you |
|---|---|
| The architecture map | Diagrams of the system, with a legend for every box |
| The quick map | Which decisions each source directory answers to |
| The decision register | DEC-01 to DEC-51, the only design authority |
| The runbook | Every daemon, command and variable an operator uses |
| The V2 roadmap | What is left, and what blocks each item |
| The contributing guide | From a clean checkout to a merged change |
| The review guide | The three passes every change makes |
| The playbook | The workshop pack that precedes the design. The register supersedes it. It is an HTML file, so open it in a browser |
This section states the compliance position, as DEC-19 requires. No machine check can decide who controls the data, so this document carries it.
Personal data stays in the runtime path (DEC-15). The system removes no personal data from a model call. Instead, you sign a Data Processing Agreement with the model provider. It binds the provider to process the data on your instructions alone. You are the data controller (section 6 of the register).
The model data guarantee is configurable (DEC-17). Retention is a
contract term, not a technical setting. You choose the data position with the
provider, and you document your choice. Set AURACLE_ZDR_ONLY=1 to ask the
provider for zero retention on every call. Absence means off, as DEC-17
requires.
The default chat model answers under that setting (DEC-26). The default
is z-ai/glm-5.3-flash, and MODEL_VER overrides it. Not every model offers
a zero-retention endpoint. A model that offers none fails the call, so the
setting never degrades to silent retention. You own the pin and the position
it implies.
The ledger stores customer message text (DEC-17, DEC-19).
turn.body_text holds each inbound message. turn.approved_reply_text holds
the reply a person approves, and turn.saved_reply_text holds a reply a
reviewer saves. outbox.reply holds the reply text the relay sends from a
record, when the ledger holds it. Each lives for the life of its row.
The step journal stores drafted replies and cited documents (DEC-17,
DEC-19). step_journal.result holds the claims of each draft, on every run
that composed one. Together the claims are the drafted reply, so the journal
holds it whether or not a person sees it.
The journal also holds the body of each document a claim cited. It holds the
title, owner and review date of every document the run retrieved. A run that
parks holds the draft a second time, under the park step. A closed case holds
the reason its reviewer records, beside that reviewer's name. An erasure path
that clears only the turn columns leaves all of this behind. The runbook
table "What the ledger stores about a customer" names each place that keeps
that text.
The console shows each reviewer's figures to every reviewer (DEC-45, DEC-46). The team screen has a row for each active reviewer. The row gives the cases each reviewer takes and their time on support, over seven days. It also gives the documents each reviewer authors and the cases each one has now.
The console has no roles, so every reviewer on the roster sees every row. These are figures about named members of your staff. You are the data controller, so you decide whether your staff see them.
You own retention and erasure. This repository ships no retention job and no purge job. Plan the retention period and the erasure path before you carry real traffic.
Labeled real history lives outside this repository (DEC-18, DEC-31). The
offline jobs accept a labeled half drawn from your own mailbox. Such a half
holds real customer text, so it stays under corpus/, which git ignores. That
directory holds your real history, not the synthetic corpus. Keep corpus/
ignored.
An artifact fitted on such a half records gold_source: "real" and belongs to
your deployment alone. You own its retention and its erasure.
Prompt logging stays off (section 6). It is a setting in your provider account, and this repository never turns it on. If you turn it on there, the provider gains irrevocable commercial rights to the inputs and the outputs. In return, the provider lowers its price by 1 percent.
No standard yet verifies Article 50(2) text marking (section 4.9). The register records that EU AI Act Article 50(1) applies now. It records that Article 50(2) applies from 2 December 2026, after a grace period of four months. That grace period covers only systems on the market before 2 August 2026. This system is not one of them.
So on the register's reading, Article 50(2) already applies to this system, from 2 August 2026. Confirm the date that applies to your deployment with legal advice.
The system adds provenance metadata to each generated text (DEC-33). The metadata is structured, so a machine can read it after export. A line of disclosure text does not survive export, so it cannot mark the text alone. Adapters carry the metadata in headers where the channel permits them.
C2PA, the content provenance standard, covers images, audio and video. Text has no equivalent standard. The architecture cannot decide whether this method satisfies Article 50(2). Get legal advice before you go live.
Synthetic fixtures only (DEC-18). The fixtures contain invented values only. Never commit a real email address, phone number, name or account identifier.
The published numbers live in gold-report/v9.json. They exercise the
pipeline, and they do not measure answer quality. The artifact says so
itself: scored_by reads stub.
The offline stub model returns one fixed label for each case text. So the classifier agrees with itself on every sample, and the agreement gate never fires. The report also pins coverage at 1.0, so the groundedness gate never fires either. What remains is the routing decision over synthetic turns.
The artifact scores the report half of the gold set, and first turns only (DEC-31, DEC-39). A later turn carries the outcome of the whole conversation, so it grades no single decision. The numbers below cover the first turns of the report half:
- Containment: 0.9424. Interval 0.9336 to 0.9501. 3074 cases.
- Action match: 0.8630. Interval 0.8504 to 0.8747. 3074 cases.
Containment is a property of the corpus. A high-tier intent always goes to a person (DEC-03, DEC-13). So the policy engine never sends its cases down the automatic path. In the corpus a few intents are common and most are rare. The high-tier intents are among the rare ones. Read the figure as a statement about the corpus, never as a measure of the system.
Action match compares the routing decision with the observable outcome, such as a refund or an escalation (DEC-32). An intent label records only a prediction, so it cannot grade the system.
These rates move whenever the corpus changes. The generator draws every value
again, so both halves change together. Run pnpm run gold-report to score the
current corpus.
The artifact records an agreement threshold of 0.60, and its source. A
gate_source of seed names the default in section 4.5 of the register. A
gate_source of calibration names the calibration artifact that fits it.
Read the two fields together, because a fitted gate and a seeded gate publish
different numbers. gate_calibration_scope names the turns a calibration
reads. scope names the turns the report scores.
coverage_fit_version reads 3, which is the fit the next section describes.
The report pins coverage at 1.0, so the groundedness gate cannot move a rate.
Read that field as provenance, not as a result.
The groundedness gate has two thresholds. They decide an internal write that no earlier rule holds back and that clears the agreement gate. Coverage below the block threshold escalates the turn, and the write never runs. Coverage below the review threshold parks the write for a person.
Each rule of step 7 that parks an action holds it before this gate. So neither threshold decides a customer reply or a high-tier intent. Neither decides an irreversible write, or a write through an adapter that lacks a guarantee.
pnpm run fit-groundedness fits the pair on the fit half, which publishes
nothing (DEC-31). The reference fit holds block 0.67 and review 0.83, from
z-ai/glm-5.3-flash. It records a spend of at least 0.1484 US dollars.
Read this pair as a pilot. The reference fit uses GROUNDEDNESS_LIMIT,
so it reads a prefix of the fit half: about a quarter of its drafts. The
corpus orders its intents, so that slice covers fewer than half of them. It
holds every high-tier draft and no low-tier draft.
The slice shows that supported and unsupported drafts separate, and that the job selects a pair rather than refusing. It is not a fit of the whole half.
These figures come from that slice, and they choose the pair:
- Block: negative predictive value 0.958. Interval 0.920 to 0.979. 192 drafts.
- Review: precision 0.998. Interval 0.990 to 1.000. 539 drafts.
Negative predictive value is the share of blocked drafts that truly lack support. Precision is the share of passed drafts that truly have it. Higher is better for both. The fit requires at least 0.9 and 0.985, and each lower bound clears its requirement.
No report-half run covers this pair yet. The figures above choose the
thresholds, so DEC-31 forbids reading them as a measurement of those
thresholds. Measure the pair 0.67 and 0.83 on the report half with
pnpm run report-groundedness before you trust it.
The reference fit does not ship as a live artifact. Git ignores the fitted
files under groundedness/. A committed fit moves your gate to a number
fitted on this synthetic corpus. groundedness/reference-v3.json is a
tracked copy you can read, and no loader finds it.
To measure that pair, copy groundedness/reference-v3.json to
groundedness/v3.json and set GROUNDEDNESS_MODEL_VER=z-ai/glm-5.3-flash.
The copy also moves the thresholds of every process in that checkout. The loader
reads the highest version, so move any fit of your own out of groundedness/
first. Otherwise fit your own.
DEC-09 names three ways to score a replay: action match, a sample a person
reviews, and judge triage. pnpm run triage runs the third. A judge selects
the cases that a person should review, and it records each selection with its
reason. By default the judge is the deterministic stub. The job never
produces a published number.
A judge shares the failures of the system it tests, so its output is a review queue only. On the synthetic gold set this job is a demonstration. The runbook says how to feed it live cases.
- CONTRIBUTING.md takes you from a clean checkout to a merged change.
- SECURITY.md says how to report a vulnerability privately, and what the default setup does not protect.
- CODE_OF_CONDUCT.md applies to everyone who takes part.
- Open an issue for a bug. Ask a question in Discussions.