Repository navigation
docs(examples): add Deep Agents example checking every tool call with NeMo Relay - #2591
Open
mkatariya90 wants to merge 1 commit into
Open
mkatariya90 wants to merge 1 commit into
mkatariya90 wants to merge 1 commit into
Conversation
Contributor
|
Staged Fern docs preview: https://nvidia-preview-pr-2591.docs.buildwithfern.com/nemo/guardrails |
… NeMo Relay Add examples/scripts/nemo_relay, a tutorial for checking each tool call of a Deep Agents agent with NeMo Guardrails' Relay plugin and the hooks server: a tool and argument allowlist, CEL rules, an optional LLM judge, deny feedback to the model, and failing closed. Map each control to the OWASP Top 10 for LLM and Agentic Applications. Test the policies in-process, and the agent through Relay with a scripted model. Closes #2590 Signed-off-by: Monika Katariya <mkatariya@nvidia.com>
mkatariya90
force-pushed
the
examples/nemo-relay-deepagents
branch
from
November 10, 2026 18:10
71e3753 to
8d3e061
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds
examples/scripts/nemo_relay, a tutorial for agent builders: a Deep Agents agent whose every tool call NeMo Relay checks with NeMo Guardrails' Relay plugin andnemoguardrails hooks-server.configs/policy, model-free): a tool and argument allowlist withallow_unlisted_tools: false(no shell, nodelete), and CEL rules that capread_logat 500 lines and allowhttp_requestonly as aGETtohttps://*.internal.example.configs/policy_judgeadds an LLM-judged tool safety check onhttp_requestfor secrets in allowed requests.relay_plugin.py: activates the plugin for a block of code, withstartup = "required", so a plugin that doesn't load stops the agent instead of leaving calls unchecked.deny_feedback.py: middleware that hands a deny back to the model asToolMessage(status="error"), so the agent corrects the call. A failed check propagates, and the run stops.agent.py: passes Deep Agents' built-ingeneral-purposesubagent toadd_nemo_relay_integrationexplicitly. Deep Agents doesn't give it the main agent's middleware, so without this its tool calls run unchecked.Supporting changes: a
deepagents_example_hooks_serverfixture inintegration_tests/relay_plugin/conftest.py,deepagentsadded to the hash-locked relay plugin test requirements, and cross-links from the plugin docs andintegration_tests/README.md.Areas for careful review:
integration_tests/relay_plugin/requirements.txtwas regenerated withdeepagents; it may conflict with fix(deps): bump pytest from 9.1.1 to 9.2.0 in /integration_tests/relay_plugin in the relay-plugin-tests group #2579 if that merges first.RuntimeErrormessage prefixes (guardrail rejected:andinternal error: NeMo Guardrails check failed for tool call). A typed exception from Relay would make this sturdier.Related Issue(s)
Verification
make test TEST=tests/server/test_hooks_nemo_relay_example.py: 23 passed.integration_tests/relay_plugin(whole suite, through the built plugin): 43 passed, including the 7 new Deep Agents tests.ruff check,ruff format --check, license headers, andty check: pass.agent.pyin its final form against a live model (CI drives it with a scripted model), and the docs build.AI Assistance
Checklist
@Pouyanpi @tgasser-nv