| title | Observability | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| description | Track cost, latency, models, providers, and failures across requests and provider attempts. | |||||||||
| icon | activity | |||||||||
| keywords |
|
|||||||||
| canonical | https://manifest.build/llm-gateway/docs/observability |
Every request through the gateway is recorded: which model served it, what it cost, how long it took, and what failed. The dashboard is built on that record, and you can query the same numbers from your own code.
The gateway tracks two things, and the difference matters once fallback is involved:
| What it is | When you get more than one | |
|---|---|---|
| Request | One call from your agent to the gateway | Never. One call in, one Request. |
| Provider attempt | One call from the gateway to a provider | Every fallback step and every Autofix retry adds another |
A request that succeeds on the first try is one Request with one attempt. A request that fails twice before succeeding is still one Request, with three attempts. A request blocked by the gateway, like a hard limit or a malformed body, is one Request with one attempt that never reached a provider. That attempt is listed under the model you asked for, or No provider when the request named none, and carries the error that stopped it.
This is why the request count and the provider call count don't match, and why cost is attributed to attempts: a failed attempt can still burn tokens.
Spend, tokens, and request volume over time, across all harnesses or one. Includes the provider and model mix. The full log, newest first. Filter it, then open any row for the request body, every provider attempt, and the routing decision. Per-provider usage, spend, and error rates. Useful for spotting one provider degrading. Threshold rules per harness, plus email provider setup.On Cloud, the Overview covers up to 365 days of history.
Opening a request in the log shows the whole story: the tier that routed it, the model and provider that served it, input and output tokens, computed cost, latency, the auth type used (API key, subscription, or local), and every fallback hop with its own status. Failures carry an error code, so a gateway rejection like M100 reads differently from a provider's own 500.
That's all metadata. To read the messages themselves, turn on request logs for the harness. Open a request, pick an attempt, and its Messages tab shows the body sent to the provider and the body that came back.
Local models record `cost = 0` with real token counts and latency, since nothing was billed. Custom providers do the same, because the gateway can't infer pricing for a model it doesn't know.A limit rule watches one metric for one harness over a rolling period, and acts when the threshold is crossed.
| Field | Options |
|---|---|
| Metric | Tokens or cost |
| Threshold | The number to watch for |
| Period | Hour, day, week, or month |
| Action | Every rule emails you. Turn on blocking to also stop requests over the threshold |
Blocking is the gateway half of this and is covered in hard limits. The rest of this section is the alerting half.
Navigate to your harness's **Limits** page in the dashboard. Click **+ Create rule**. Pick the metric, threshold, and period. Every rule emails you; turn on **Block requests when exceeded** to also stop requests over the threshold. The rule takes effect immediately.Alerts are evaluated hourly, so an email can lag the threshold by up to an hour. Blocking is not: it's checked on every request. Each rule emails once per period, so a rule that stays over its threshold won't flood your inbox.
Alerts go to your account email. Check it's current in your profile settings.
On the Limits page, under Configure email provider, pick Resend, Mailgun, or SendGrid. Enter the API key. Mailgun also asks for a sending domain. Notification email defaults to your account email; change it to send alerts elsewhere. Test & Connect sends a real test email and saves nothing if it fails. The saved provider appears as a card on the same page, where you can edit or remove it.
Your workspace holds one provider, shared by every harness. Removing it stops alerts for all of them.
Alerts use the saved provider first, and fall back to the EMAIL_* environment variables when no provider is saved. Those variables also send verification and password-reset email, which the saved provider never handles.
Without an email provider from either path, alerts are skipped. Blocking rules still work, since they don't need to send anything.
A harness can query its own numbers with its mnfst_ key, which is useful for showing a budget in your own UI or stopping work before a limit blocks you.
curl https://app.manifest.build/api/v1/agent/usage \
-H "Authorization: Bearer mnfst_YOUR_KEY"| Endpoint | Returns |
|---|---|
GET /api/v1/agent/usage |
Token usage for the calling harness |
GET /api/v1/agent/costs |
Cost data for the calling harness |
Self-hosted installs swap the base URL for their own host.
Separately from your dashboard, each self-hosted install sends a small daily report so the project can see version adoption and which providers deserve attention. It's aggregates only, never prompts or content, and one environment variable turns it off. Full field list and opt-out: Data and telemetry.
- LLM Gateway — routing, fallback, and hard limits
- Request logs — the request and response bodies
- Autofix
- Error codes
- Glossary