[Nurozen]

Description

Rails and actions cannot tell which conversation a request belongs to, or which two checks are the two legs of one model call. A rail that reports to an external service sees a stream of unrelated checks, and nothing can pair a request with its response.

This adds two request-scoped context variables, conversation_id and roundtrip_id, set by the embedding application and read by rails and actions. No public API signature changes.

with conversation_id(thread_id):
    response = await rails.generate_async(messages=messages)

This adds no server-side state

Worth stating plainly, because it was the main concern on the 28 Oct call: Guardrails keeps scaling statelessly. NeMo Guardrails does not interpret either ID, does not pass them to a model, and does not store them. They are labels for the duration of a block, carried in context variables, and nothing persists them.

The identity already exists at the boundary. The server accepts guardrails.thread_id, and the LangChain middleware runs on a LangGraph thread. This stops that identity being thrown away before a rail can see it. The server derives the conversation ID from the thread_id the request already carries, so a deployment using threads gets this with no client change, and an ID the application set itself wins over the request's thread.

Storing prior turns server-side so a client need not resend them is a different feature, and we agreed on the call that it belongs at the deployment layer rather than here: a horizontally scaled Guardrails behind a load balancer would need a persistent store the compute nodes read from, or consistent hashing, and that is a NeMo Helix question. This PR deliberately does not go there. Parking it as discussed; we are looking at Helix and will bring a proposal.

thread_id is now rejected if it holds a control character or leading or trailing whitespace. An ID that was only ever a datastore key can now leave the process in an HTTP header, and an unsendable one would fail checks that should pass.

Three fixes make the IDs reach a rail

• AsyncWorkQueue ran each work item in the worker's context rather than the submitter's, so any context variable a caller set was dropped on the way in. The queue now copies the submitter's context. This stands on its own: it applies to every context variable, not only these two.
• IORails.generate_async and check_async open a round trip, since one call covers both legs of a model call. A caller that already opened one keeps its ID, which is what lets an agent framework pair two separate check_async calls.
• A stream is pulled by whoever consumes the iterator, in a context that is not the one that created it, so the IDs are captured at creation and restored on each pull.

Related Issue(s)

• Fixes feature: request-scoped conversation and round-trip IDs for rails and actions #2495
• Issue assignee: @Nurozen

Raised as its own PR at Monika's request on 19 Oct ("agree session ids is important, love the idea, please do a separate PR").

Verification

make pre-commit and make test pass, rebased on current develop: 12821 passed, 201 skipped.

Six failures remain, in tests/server/test_api.py, tests/server/test_schema_utils.py and tests/integrations/langchain/llm/test_langchain_integration.py. They reproduce on a clean develop checkout in the same environment and are unrelated: they reach the network and fail on a NotFoundError from the OpenAI client.

New tests cover both IDs across LLMRails, IORails, the server, the LangChain middleware and streaming, including concurrent requests each seeing their own ID.

AI Assistance

• No AI tools were used.
• AI tools were used; a human reviewed and can explain every change (tool: Claude Code).

Checklist

• I've read the CONTRIBUTING guidelines.
• This PR links to a triaged issue assigned to me.
• My PR title follows the project commit convention.
• I've updated the documentation if applicable.
• I've added tests if applicable.
• I've noted any verification beyond CI and any checks I couldn't run.
• I did not update generated changelog files manually.
• I addressed all CodeRabbit, Greptile, and other review comments, or replied with why no change is needed.
• @mentions of the person or team responsible for reviewing proposed changes.

[Nurozen]

Self-review pass on this branch turned up one bug, now fixed in 58650de.

_thread_id fell back to runtime.config, which LangGraph's Runtime does not declare in any 1.x release, confirmed via dataclasses.fields. The execution_info leg does not cover the gap: on 1.1.3 the field exists but carries no thread_id, so the conversation ID was unset on everything up to somewhere in 1.1.4 to 1.1.9, which includes every version langchain 1.0.x allows. Measured across 1.0.0, 1.0.10, 1.1.2, 1.1.3, 1.1.9, 1.2.0 and 1.2.7 with real checkpointed StateGraph runs: both legs returned None on the first four.

It now falls back to langgraph.config.get_config(), which returns the thread ID on every version tested, and catches the RuntimeError that call raises outside a runnable context. The old test pinned a SimpleNamespace(execution_info=None, config=...) runtime that LangGraph never produces; it now runs the hooks inside a real node of a checkpointed graph.

Two things I am leaving open rather than changing, happy to take direction:

• The round-trip ID is keyed on the AI-message count, so a trimming middleware such as SummarizationMiddleware can hold that count steady and make later calls rejoin a stale ID. Either key it on something trimming cannot rewind, or document the limitation.
• Tool legs are out of scope here. Once #2499 lands on the same middleware its tool checks run outside this round-trip scope, so pairing for tool legs needs deciding in one place or the other.

[Nurozen]

@tgasser-nv, per our conversation in HiddenLayer meeting