Benchmarks that check what an agent actually committed, agents whose every decision is inspectable,
and the data plumbing underneath. Founder of Thrill AI · MS in Data Science, George Mason '25.
🏘️ Settlement: a village sim where you can read why every agent did what it did
Six residents bid for jobs, gather, build and recruit toward one objective you set. Open any of them to see the decision, the route and the memory behind it. It doubles as a testbed: point npx settlement-testbed at your own agent's HTTP endpoint and get a report anyone can re-verify.
🛒 IndicOrderBench: catches voice agents that place the wrong order
A voice agent can read back "one paneer wrap, no onion" and still commit two wraps with onion. IndicOrderBench scores the order that actually reached the backend, in English and Hinglish, with a CI exit code. No judge model, and the demo needs no API keys.
uvx --from indicorderbench iob demo --out demo-out
🔐 aegis-shred: delete a user by destroying one key
Crypto-shredding for application data: each user gets their own encryption key, so erasing a person makes every copy of their data unreadable, backups included. Rust core, Python package, aegis CLI. Alpha, not yet independently audited.
libredb-studio ★1.0k: 8 merged PRs and listed in CONTRIBUTORS. I built the one-command Keycloak SSO demo stack, fixed MySQL schema diffs emitting unquoted column defaults, bound the demo stacks to loopback only (#960, #1004) and shipped four docs fixes (#938, #939, #940, #1657).
| aegiseval | Red-teaming a tool-using RAG agent: threat model, 260-prompt golden set, validated LLM judge, mitigations measured before and after |
| MerchantLens | Databricks lakehouse where access rules are enforced by the query engine, not by asking the agent nicely |
| mcp-rust-sdk | Rust SDK for building Model Context Protocol servers |
| pulse | Realtime task board that shows its own latency budget live |
| rocm-devops-starter | Probe, smoke-test training run, container and CI for PyTorch on AMD GPUs, with a table of exactly what ran on real hardware |
| Contract & Invoice Intelligence | Checks invoices against their contracts and tracks precision/recall/F1 on the flags |
Stack I reach for: Python · TypeScript · Rust · SQL · FastAPI · Next.js · LangGraph · MCP · Databricks · Postgres · Docker · GitHub Actions
Now: getting IndicOrderBench and aegis-shred to their first outside users, and publishing measured ordering accuracy for real LLM voice stacks. Open to AI engineering, data/analytics engineering and data analyst roles.
If something here is useful to you, a ⭐ or an issue with a real failure you've seen helps more than anything.