Repository navigation
Conversation
github-actions
Bot
added
status: needs triage
Nurozen
marked this pull request as ready for review
November 6, 2026 19:52
1 task done
Nurozen
force-pushed
the
feat/hiddenlayer-runtime-security-rail
branch
from
November 6, 2026 22:37
f497870 to
d1c5ed2
Compare
AsyncWorkQueue workers ran every item in the context they started in. A context variable set by the caller of generate_async or check_async never reached the rails, and one set by the first caller (whose submit starts the workers lazily) was frozen into every later request. Each item now runs in a copy of its submitter's context, so request-scoped context variables reach the work and changes made by one item do not leak into the next. Signed-off-by: Justin Brooks <justin.lance.brooks@gmail.com>
Rails that report to an external service have no way to learn which conversation a request belongs to: IORails actions receive only the request messages, and the context message role does not reach them. nemoguardrails.context now holds a conversation_id_var, a conversation_id() context manager that sets it for the calls made inside the block, and a get_conversation_id() getter for rails and actions. It works for generate, stream and check on both engines, is never added to the messages, and is restored when the block exits. Concurrent requests each see their own ID. Signed-off-by: Justin Brooks <justin.lance.brooks@gmail.com>
A chat completion that names a thread now runs its rails inside conversation_id(thread_id), so rails and actions that report to an external service can group the checks of one conversation with get_conversation_id(). This covers generate and stream; for streams each chunk is pulled with the ID set, so rails that run while the response body is sent, such as streaming output rails, see it too. An ID the application running the server already set is kept. Requests without a thread_id see no ID, and the Colang 2.x rejection and thread history replay are unchanged. Signed-off-by: Justin Brooks <justin.lance.brooks@gmail.com>
GuardrailsMiddleware now runs its input and output checks with the graph's thread ID (runtime.execution_info.thread_id, else runtime.config["configurable"]["thread_id"]) as the request-scoped conversation ID, so rails that group checks by conversation see one conversation per thread. A conversation ID the application already set is kept, and nothing is set outside a checkpointed run. Signed-off-by: Justin Brooks <justin.lance.brooks@gmail.com>
A rail that reports to an external service has to pair the check of what goes to the model with the check of what comes back, and the engine is the only thing that knows the two halves belong to the same call. `nemoguardrails.context` now holds a `roundtrip_id_var`, a `roundtrip_id()` context manager for callers that know where a model call starts, a `new_roundtrip_id()` that mints one only when nothing is set, and a `get_roundtrip_id()` getter for rails and actions. `generate_async`, `stream_async` and the two `IORails` queue entry points open a round trip, so a caller that runs a whole call through one of them has nothing to set. A stream resumes in the context of whoever pulls the chunk, which for an HTTP response body is not the caller that created it. Both IDs are therefore captured where the stream is created and re-applied around each pull rather than held across the yields of the generator, so a library caller that stores an iterator and drains it elsewhere still runs its rails with the conversation and round trip the request was made in. An agent framework is the case the engine cannot see: `GuardrailsMiddleware` runs the two legs of one model call as separate `check_async` calls in separate graph nodes, so it carries the ID in the agent state and publishes it for the duration of each check. The ID is recorded against the model call it was opened for: the first hook to reach a call opens the round trip and the rest join it, and an ID left behind by an earlier call is ignored. That keeps both legs on one ID when `InputRailsMiddleware` is composed with `OutputRailsMiddleware`, and stops a checkpointed thread reporting a turn's response leg under the previous turn's ID. The after-model hook prefers an ID the application set. The new tests run real agents built with `create_agent`, which is what exercises the state schema that carries the ID between graph nodes. Signed-off-by: Justin Brooks <justin.lance.brooks@gmail.com>
The conversation ID fell back to runtime.config["configurable"]["thread_id"] when the runtime carried no execution info, but a LangGraph Runtime has no config attribute, and the releases that langchain 1.0 pins have no execution info either, so on those releases the input and output checks of a checkpointed run reported no conversation ID at all. The fallback now reads the thread ID of the run in progress from langgraph.config.get_config(), which every release populates for the node being executed, and still yields nothing outside a run. The test that pinned the old fallback described a runtime LangGraph does not produce; both shapes are now exercised from inside a real graph invoked on a thread. Signed-off-by: Justin Brooks <justin.lance.brooks@gmail.com>
thread_id was length-checked only. It is the conversation ID of every rail that runs for the request, and a rail that reports to an external service puts it on the wire, where a control character or surrounding whitespace makes the message unsendable. A thread ID ending in a space therefore failed every check made for that conversation, or, with a rail configured to fail open, silently disabled it. The server now rejects a thread ID that is not printable ASCII or that is surrounded by whitespace, so the caller learns at the boundary. Signed-off-by: Justin Brooks <justin.lance.brooks@gmail.com>
LangGraph binds the run config to a node's context by creating the node's task with asyncio's context argument, which arrived in Python 3.11. Below that an async node runs without it, get_config() raises, and the fallback that reads the thread ID from the run config yields nothing, so checks given a runtime that carries no thread ID of its own report no conversation. A runtime that does carry one still names the conversation on every supported Python, and so does the run config on the sync path, which is now pinned too. The async expectation follows what each version can deliver. Signed-off-by: Justin Brooks <justin.lance.brooks@gmail.com>
…tors Behavior is unchanged on every production path. - stream_with_request_ids takes a str round-trip ID and always applies it: both engines pass a minted ID, so the None guard only served a test fake, which now passes ensure_roundtrip_id() like the engines do. - _thread_id_from_run_config drops the isinstance guards kept from the getattr(runtime, "config") draft; get_config() returns a RunnableConfig or raises RuntimeError, which is already caught. - _roundtrip_id_for_call returns the recorded ID directly: the before-model hook writes the ID and its call index together, and the ID is never empty. - LLMRails._generate_async takes keyword-only arguments without defaults, since its only caller passes all five. - The middleware API reference no longer calls OutputRailsMiddleware's abefore_model a no-op; it now opens the round trip for the output check. Signed-off-by: Justin Brooks <justin.lance.brooks@gmail.com>
…nerator stream_async(generator=...) returned the output-rail iterator directly, before the point where the non-generator path mints a round-trip ID and captures the conversation ID. Those rails run lazily, as the consumer pulls chunks, so they ran under whatever context drained the response body: no round trip at all, and either no conversation or, when the consumer had set one of its own, somebody else's. A rail that reports to an external service filed its checks under the wrong conversation and no one saw an error. The branch now captures the IDs of the creating request and applies them around each pull, the same shape the other streaming paths use, because an async generator resumes in its consumer's context and cannot hold a token across a yield. A round-trip ID the caller already opened is kept rather than replaced. Signed-off-by: Justin Brooks <justin.lance.brooks@gmail.com>
Nurozen
force-pushed
the
feat/hiddenlayer-runtime-security-rail
branch
from
November 6, 2026 22:44
d1c5ed2 to
9b3b508
Compare
The thread ID validator rejected every character outside printable ASCII, so a deployment naming threads in its own language started getting a 422 on requests that had worked. The field had accepted any string of 16 to 255 characters. A rail percent-encodes the ID before it goes on the wire, so a non-ASCII ID needs no protecting. A control character or surrounding whitespace is almost always a client defect, and is still rejected where the caller learns why. Signed-off-by: Justin Brooks <justin.lance.brooks@gmail.com>
The hook minted the round-trip ID whether or not it had an input check to pair it with, so a middleware configured for output only started writing two keys into the graph state on every model call and driving an event loop where it had previously returned at once. OutputRailsMiddleware is the case that shows it. The ID is still what pairs the two legs when both run. With no before-model leg to pair with, the output check opens the round trip itself rather than reporting under no ID at all. Also imports langgraph lazily. It is not a declared dependency of this package, and reading a LangGraph run config is the only thing that needs it. Restores the assertions in the existing middleware tests that this had changed. Signed-off-by: Justin Brooks <justin.lance.brooks@gmail.com>
The HiddenLayer AI Runtime Security rail evaluates a guarded model call in legs that share one round-trip ID: the input check and tool-result checks go to the request-evaluation route, the output check and tool-call checks to the response-evaluation route. Both carry an OpenAI Chat Completions body holding the current turn only, and both answer 200 with the payload to forward and the verdict in the HL-Runtime-Action and HL-Runtime-Protection-Mode headers. Add the three modules that own that wire contract: - _payloads builds each body, picks the route for a surface, derives the round-trip UUID, and assembles the headers. A conversation ID that cannot be sent as a header is dropped rather than truncated, because truncating can map two conversations onto one session. - _token fetches and caches the OAuth2 client-credentials access token, sharing one in-flight fetch per credentials. - _verdict maps the response headers onto a rail outcome, applies a redaction from the returned body, and ignores the body on a block. Signed-off-by: Justin Brooks <justin.lance.brooks@gmail.com>
The three actions resolve the rail's settings, post each leg to its route and return the outcome the verdict headers describe. Settings resolve from the config field, then the environment variable, then the default. The client ID and secret are read from the environment only. An api_url that is not https, outside a loopback host, is rejected at load. Failures fall into three classes. A config defect raises whatever fail_open says. A transport failure, which includes a 200 that carries no action to apply, is what fail_open covers. Everything else is a verdict and is applied: an evaluation HiddenLayer could not complete arrives as a 200 whose action its deployment's fail setting chose, and a redaction the rail cannot carry out blocks. Signed-off-by: Justin Brooks <justin.lance.brooks@gmail.com>
The manifest declares four surfaces. Input and output run on both engines and can rewrite the message; the tool-call and tool-result surfaces are IORails only, as every per-tool surface is. Register the rail in the catalog, the telemetry feature list, the engine support matrix and the config schema snapshot, and document the two routes, the session ID as the only source of conversation history, and the protection modes that tell a policy's own action apart from one the integration or Monitor Mode changed. Signed-off-by: Justin Brooks <justin.lance.brooks@gmail.com>
A tool result content list made only of non-text parts, such as an MCP resource or an image, joined to the empty string, so the HiddenLayer tool result check sent an empty tool message and evaluated nothing. Send such a list as its JSON encoding instead, as untyped parts already were. Signed-off-by: Justin Brooks <justin.lance.brooks@gmail.com>
The session-header guard accepted any ASCII ID without a NUL, so a CR, LF or edge whitespace reached the HL-Runtime-Session-Id header and httpx refused to serialize the request: every HiddenLayer check on that conversation failed in transport, allowing its traffic under fail_open and blocking it otherwise. Drop an ID outside the header field-value set through the same one-shot warning as an over-long one. Signed-off-by: Justin Brooks <justin.lance.brooks@gmail.com>
_roundtrip_id() falls back to the request ID, which lives in a process-global context variable that another test may have left set: tests/guardrails/test_telemetry.py makes reset_request_id raise on purpose, so traced_request leaves one behind. The case that asserts a fresh uuid4 per leg then saw a request ID and derived the same uuid5 twice. Pin the variable for the module instead of reading whatever the process is carrying. Signed-off-by: Justin Brooks <justin.lance.brooks@gmail.com>
The rail's comments, docstrings and docs called each check of a model call a "leg", a term HiddenLayer's API does not use. Say what the API says: a model call is one round-trip under `HL-Roundtrip-Id`, holding its request evaluations (the input check and tool-result checks) and its response evaluations (tool-call checks and the output check). Signed-off-by: Justin Brooks <justin.lance.brooks@gmail.com>
The rail built OpenAI Chat Completions requests and chat.completion responses around every check, so a change in how HiddenLayer models an interaction landed in the integration rather than in the service. Each body now carries the check as the library holds it, named by a `check` field: a user message, a bot message, a tool call, or a tool result with the call it answers. The call-ID synthesizer, the tool-name resolver, the arguments encoder and the content-part flattener are gone, along with the `unknown` model placeholder; a 200 comes back in the same shape, with `user_message` or `bot_message` replaced on REDACT. Validation that belongs to the service moves there. The session ID is percent-encoded and sent as the conversation names it, with no `nemo:` namespace and no byte or character checks, because HiddenLayer scopes it by tenant, integration and project. The round-trip ID is sent as the framework holds it rather than mapped onto a UUIDv5. The token path keeps a token until it expires and refreshes once on a 401. The single-flight fetch registry, its per-event-loop keys and the compare-and-invalidate it needed are removed: they bought one saved fetch on a cold start, and nothing in the library offers auth plumbing to replace them with. The declared tool is now sent with a tool-call check as well as a tool-result check, so a policy reads what the model was offered on both. Signed-off-by: Justin Brooks <justin.lance.brooks@gmail.com>
Under fail_open a rejected check is a turn nothing evaluated, so the rail now avoids sending one where it can. An empty user or bot message is allowed without a call. The contract gives user_message and bot_message a minimum length of one, so an empty check is a 422, and an empty turn holds nothing to evaluate in any case. The generate path had no such guard, where check() already skips its input rails with no user content. The two ID bounds degrade gracefully again, as they did before the round-trip and session IDs were sent as the framework holds them. A round-trip ID over 256 bytes is mapped onto a UUID derived from it, so a model call's checks still pair under one ID; a conversation ID over 2048 bytes is dropped with one warning, so the checks are evaluated without history rather than under a truncated ID two conversations could share. Losing an over-long ID costs the history, not the evaluation. The failure-handling docs said fail_open covered transport failures only, and two lines earlier described it allowing a check the server rejected. It covers any call that ends without a verdict, whatever ended it. Signed-off-by: Justin Brooks <justin.lance.brooks@gmail.com>
…erate path A user turn whose content is a list of content parts goes out as the framework holds it rather than flattened by the rail: HiddenLayer reads the parts and owns the translation. The IORails suite drives the real path against the recording stub, so the case asserts the body that left the rail, not a unit boundary. Signed-off-by: Justin Brooks <justin.lance.brooks@gmail.com>
… can The rail sends the round-trip ID as the framework holds it, and mapped only an over-long one onto a derived UUID. Two other values end the same way, in a check that never reaches HiddenLayer and a turn nothing evaluated. The nil UUID is a 422: every consumer downstream reads it as "no round trip", so the contract rejects it rather than record one. A caller passing it got a rejected check on every surface. A non-ASCII ID never leaves the client at all. httpx encodes a header value as ASCII, so it raises before the request goes out, which is a transport failure and so fails open. The pre-reshape UUID mapping covered this incidentally; sending the ID as held reopened it. All three are now one question, whether the ID can be sent as held, and one answer, the UUIDv5 already derived for the over-long case. The mapping is deterministic, so a model call's checks still pair, and each reason warns once on its own. Signed-off-by: Justin Brooks <justin.lance.brooks@gmail.com>
A 401 dropped the cached token unconditionally. Two callers holding the same rejected token would both invalidate: the first refreshes the cache, the second then discards that fresh token and fetches again, and every concurrent 401 costs another round trip. Drop the entry only when it still holds the token that was rejected. Also point the catalog link at the rendered Fern route. The destination used the source directory name, but docs/index.yml places the page under the Third-Party APIs section, whose slug is third-party, so the published page is at /configure-guardrails/guardrail-catalog/third-party/hiddenlayer. Signed-off-by: Justin Brooks <justin.lance.brooks@gmail.com>
…y-rail The rail predates the add-library-rail and review-library-rail skills. This closes the deviations an audit against them turned up. HTTP composition now goes through ensure_retrying_http_client and http_call rather than wrapping an injected client by hand, so an injected client is never taken ownership of and only what the factory builds is closed. The hand-rolled one-shot connection retry is gone: resilience belongs in the RetryPolicy. That costs a behaviour the policy cannot express. retry_transport_errors covers timeouts and connection errors together, and a rail runs inline, so retrying either would spend another full request timeout before the caller hears anything. Transport errors are declared off and a connection error now resolves to the fail setting on the first failure. Errors are rooted at HiddenLayerRailError in errs.py, and a failed check raises HiddenLayerCheckError rather than a bare RuntimeError. _payloads.py is now request.py, the name the skill's layout table uses. fail_open's description said it covered transport failures only, while the code fails open on any non-success status, including a 401 or 403 from a revoked key. An operator reading the config deserves to know that. Adds the recorded-suite entry the skill makes mandatory for a library rail, with its two config dirs and the credentials fixture. The cassettes are not recorded here: verdicts come from project policy, so recording needs a tenant with fixed policies rather than a key alone. HL- joins the filtered header prefixes. HL-Project-Id and HL-Runtime-Session-Id identify a tenant and a conversation, and neither belongs in a committed cassette. VCR matches on method, scheme, host, port, path, query and body, so filtering headers does not affect replay. Signed-off-by: Justin Brooks <justin.lance.brooks@gmail.com>
… header An `hl-` prefix also matched the response, so a recorded cassette lost `HL-Runtime-Action`: the verdict the cassette exists to replay. Naming the request headers keeps the tenant, project and conversation identifiers out of a committed cassette while the verdict stays in it. Found by recording against a live deployment, where every block case replayed as an unusable response. Signed-off-by: Justin Brooks <justin.lance.brooks@gmail.com>
The rail imported _header from nemoguardrails.http.retry, reaching across a package boundary for a private symbol. The lookup is four lines and the rail already carries its own copy for the token response. The fail-closed path also logged nothing before raising, so an operator running fail-closed had no record of what failed. Every other vendor rail logs at error level first. Signed-off-by: Justin Brooks <justin.lance.brooks@gmail.com>
Nurozen
force-pushed
the
feat/hiddenlayer-runtime-security-rail
branch
from
November 7, 2026 04:47
9b3b508 to
fbfe35a
Compare
Records the cassettes the suite was added without: an allowed input, a blocked input, a redacted output, a failed token exchange, and a two-round-trip conversation. They replay with the network blocked. The output case checks a redaction rather than a block. A policy rewriting a reply is what the rail's transform path exists for, and asserting the rewrite that comes back is worth more than a second refusal. The conversation case runs four checks under one conversation ID. The verdicts it asserts are per turn: the session itself is a filtered header, so what it buys is on the service's side, where the four checks are recorded as one conversation rather than four unrelated ones. The project is left to the recorder's environment rather than pinned in the config, since a project ID identifies a tenant. Re-recording needs one whose policy enforces; a detect-only project records an allow for every case. Signed-off-by: Justin Brooks <justin.lance.brooks@gmail.com>
Author
|
@tgasser-nv, per our conversation in HiddenLayer meeting |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds a
hiddenlayerrail to the guardrail catalog, covering all four check surfaces against HiddenLayer AI Runtime Security.Opened first, ahead of #2496 and #2498, per Monika on the 28 Oct call: get the rail through for proof of life.
The rail sends each check in the shape NeMo hands it over: a user message, a bot message, a tool call, or a tool result with the call it answers. HiddenLayer decides by the customer's own policy what to rewrite and returns the whole rewritten message, which the rail swaps in. No per-field redaction config lives in Guardrails, which is the split Tim and I landed on during the call.
It fails open by default. A guardrail that raises on a timeout turns a transient dependency failure into a failed request for the whole application.
Tool surfaces block today, and that is the thing #2498 fixes
On a
tool_callortool_resultthe rail blocks where it would otherwise redact, because the engine honours a transform only on the conversational surfaces. That is the workaround I described on the call:the agent stops instead of continuing with safe content, and the user gets nothing to act on. Once #2498 lands these two surfaces redact like the others, and the change is confined to the rail's verdict mapping.
Scanning context rather than one message is why the session identity in #2496 matters here: a prompt injection can take several turns to assemble, so HiddenLayer evaluates a window of history, and without a stable session ID the window has to be reconstructed heuristically.
About the size of this PR
The commit and file counts here cover both this PR and #2496, because a PR to this repo can only base on a branch in this repo and the two are stacked. The rail's own change is confined to
nemoguardrails/library/hiddenlayer/, its docs page, its tests and its recorded-suite entry.To see only that, compare the two branches directly:
hiddenlayerai/Guardrails@feat/request-scoped-conversation-id...feat/hiddenlayer-runtime-security-rail
The recorded suite
.agents/skills/add-library-raillanded after this rail was written. Auditing against it turned up eleven deviations, all fixed in this branch, and the recorded-suite entry it makes mandatory, which is now here: an allowed input, a blocked input, a redacted output, a failed token exchange, and a two-round-trip conversation. They replay with the network blocked, so CI needs no HiddenLayer credential.The output case asserts a redaction rather than a block. A policy rewriting a reply is what the rail's transform path exists for, and the rewrite that comes back is worth more than a second refusal.
The conversation case runs four checks under one conversation ID. The verdicts it asserts are per turn: the session is a filtered header, so what it buys is on HiddenLayer's side, where the four checks are recorded as one conversation rather than four unrelated ones.
The project is left to the recorder's environment rather than pinned in the config, since a project ID identifies a tenant. Re-recording needs
HIDDENLAYER_CLIENT_ID,HIDDENLAYER_CLIENT_SECRETand aHIDDENLAYER_PROJECT_IDwhose policy enforces; a detect-only project records an allow for every case.Related Issue(s)
Verification
make pre-commitandmake testpass, rebased on currentdevelop: 12821 passed, 201 skipped.Six failures remain, in
tests/server/test_api.py,tests/server/test_schema_utils.pyandtests/integrations/langchain/llm/test_langchain_integration.py. They reproduce on a cleandevelopcheckout in the same environment and are unrelated to this change: they reach the network and fail on aNotFoundErrorfrom the OpenAI client.Beyond CI, the rail was run end to end against a live HiddenLayer deployment. One payload across three surfaces returned
REDACTwith the text rewritten oninputandoutput, andBLOCKwith the body as sent ontool_result, which is the promotion described above. A two-turn conversation under one session ID had the second turn's output redacted on the strength of the first turn's content, exercising the IDs from #2496.Not run: a recorded suite, for the reason above.
On release timing
This does not need to block 0.25 on 11 Nov. Per the call, a partner rail does not have to ride the release cycle so long as we can build from a GitHub tag or commit rather than a published wheel, and we can. We need it demoable rather than in production.
AI Assistance
Checklist