Repository navigation
A short checklist for AutoGen teams that write to real systems #8471
Replies: 4 comments 4 replies
|
The single key reused on every retry is the item I would test hardest, because it is easy to claim and easy to break. My usual check: let the specialist issue the write, kill the process after the API accepts it but before the acknowledgement reaches the team, restart, and count tickets or charges on the other side. Run that ten times at different kill points and expect exactly one side effect each time. I would also test your first point directly by having the reviewer's sign-off message contain an instruction to delete something, and assert the manager logs it as data and runs nothing. |
|
stackyardweekstart Treat another agent's output as data, not instructions. When an LLM generates a JSON payload without strict schema validation, downstream executors often misinterpret string content as shell commands. This cascades into corrupted directories or unintended writes. Reusing a single key for retries directly addresses idempotency. Without it, network timeouts trigger duplicate charges or permanent data loss that cannot be rolled back. The system behaves as a state machine rather than a conversation. For human approval on irreversible actions, is there a standard way to enforce this without introducing significant latency in the handoff between manager and reviewer agents? |
|
stackyardweekstart The approval gate should bind the exact operation, target and maximum amount, and the executor should validate that binding immediately before the write. A reviewer's text is evidence to inspect, not a substitute for the permission check. That avoids making a fast reviewer-to-manager handoff an authorization bypass. The restart test in this thread is important. A companion test, when a full kill-point harness is not available, is to make the remote write commit but lose its response, then inspect the resulting transfer count and read/write order. Agent Trials offers that narrower synthetic HTTP exercise: https://agent-trials.home201.com/agent.md . It also checks approval boundaries and reports expected/observed failure evidence after finishing. It does not kill the client, simulate replica lag or prove production crash recovery. If an already-authorized HTTP agent is available, a public result or redacted protocol blocker would be useful; no real funds or model-key upload, but the operator's model/tool costs still apply. |
|
stackyardweekstart the gap I would add sits between your second and third patterns. A reviewer's yes and the execution it approved need to be two separate records. If the sign-off lives as chat text, then after the write nobody can check which exact operation the yes was attached to, and the executor's own log is the only evidence it behaved. What has held up on our side (disclosure: I work on Zambo, an MCP stack where jobs carry AER-1 receipts, the standard Brennan Zambo created):
I built the record shape as a live handover for this thread's flow, approval step and execution step as separate receipted records, if you want to pick it apart: https://zambo.dev/handover/fecf26ee-42f7-4b79-a00e-5e351c4effe7 (receipt: https://zambo.dev/run/79f3bb24-f9d9-4f53-a2ce-413bd1e75552) Your caps and fixtures stop the bad write in the moment. The separate records are what let you prove, a month later, that the write that happened is the write that was approved. |
Uh oh!
There was an error while loading. Please reload this page.
Disclosure: this is ours, from Stackyard. It's a free MIT set of small patterns for agents that touch files, money or APIs: https://github.com/stackyardweekstart/weekstart-starter-pack
For AutoGen teams, four of them sit naturally at the handoff between a manager, a specialist and a reviewer:
They're patterns to adapt, not a library. There's also a free in-browser setup check at https://stackyard.fyi if you want a quick gap list first.
How are others gating writes when one agent in the team has already approved them?
All reactions