Skip to content

docs(examples): add Deep Agents example checking every tool call with NeMo Relay - #2591

Open
mkatariya90 wants to merge 1 commit into
developfrom
examples/nemo-relay-deepagents
Open

mkatariya90 wants to merge 1 commit into
developfrom
examples/nemo-relay-deepagents

Conversation

@mkatariya90

Copy link
Copy Markdown
Collaborator

Description

Adds examples/scripts/nemo_relay, a tutorial for agent builders: a Deep Agents agent whose every tool call NeMo Relay checks with NeMo Guardrails' Relay plugin and nemoguardrails hooks-server.

  • Policy (configs/policy, model-free): a tool and argument allowlist with allow_unlisted_tools: false (no shell, no delete), and CEL rules that cap read_log at 500 lines and allow http_request only as a GET to https://*.internal.example. configs/policy_judge adds an LLM-judged tool safety check on http_request for secrets in allowed requests.
  • relay_plugin.py: activates the plugin for a block of code, with startup = "required", so a plugin that doesn't load stops the agent instead of leaving calls unchecked.
  • deny_feedback.py: middleware that hands a deny back to the model as ToolMessage(status="error"), so the agent corrects the call. A failed check propagates, and the run stops.
  • agent.py: passes Deep Agents' built-in general-purpose subagent to add_nemo_relay_integration explicitly. Deep Agents doesn't give it the main agent's middleware, so without this its tool calls run unchecked.
  • README: step-by-step tutorial, with each control mapped to the OWASP Top 10 for LLM Applications 2025 and for Agentic Applications 2026.

Supporting changes: a deepagents_example_hooks_server fixture in integration_tests/relay_plugin/conftest.py, deepagents added to the hash-locked relay plugin test requirements, and cross-links from the plugin docs and integration_tests/README.md.

Areas for careful review:

Related Issue(s)

Verification

  • make test TEST=tests/server/test_hooks_nemo_relay_example.py: 23 passed.
  • integration_tests/relay_plugin (whole suite, through the built plugin): 43 passed, including the 7 new Deep Agents tests.
  • ruff check, ruff format --check, license headers, and ty check: pass.
  • The same policy and wiring were exercised in a larger Deep Agents demo against a live NIM model, with both configs, including stopping the hooks server mid-run.
  • Not run: agent.py in its final form against a live model (CI drives it with a scripted model), and the docs build.

AI Assistance

  • No AI tools were used.
  • AI tools were used; a human reviewed and can explain every change (tool: Cursor).

Checklist

  • I've read the CONTRIBUTING guidelines.
  • This PR links to a triaged issue assigned to me.
  • My PR title follows the project commit convention.
  • I've updated the documentation if applicable.
  • I've added tests if applicable.
  • I've noted any verification beyond CI and any checks I couldn't run.
  • I did not update generated changelog files manually.
  • I addressed all CodeRabbit, Greptile, and other review comments, or replied with why no change is needed.
  • @mentions of the person or team responsible for reviewing proposed changes.

@Pouyanpi @tgasser-nv

@mkatariya90 mkatariya90 self-assigned this Nov 10, 2026
@github-actions github-actions Bot added status: needs triage New issues that have not yet been reviewed or categorized. size: L needs: signing labels Nov 10, 2026
@github-actions

Copy link
Copy Markdown
Contributor

… NeMo Relay

Add examples/scripts/nemo_relay, a tutorial for checking each tool call of a Deep Agents agent with NeMo Guardrails' Relay plugin and the hooks server: a tool and argument allowlist, CEL rules, an optional LLM judge, deny feedback to the model, and failing closed. Map each control to the OWASP Top 10 for LLM and Agentic Applications.

Test the policies in-process, and the agent through Relay with a scripted model.

Closes #2590

Signed-off-by: Monika Katariya <mkatariya@nvidia.com>
@mkatariya90
mkatariya90 force-pushed the examples/nemo-relay-deepagents branch from 71e3753 to 8d3e061 Compare November 10, 2026 18:10

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size: L status: needs triage New issues that have not yet been reviewed or categorized.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feature: Deep Agents example for checking tool calls with NeMo Relay and the hooks server

1 participant