All pull requests

Pull requests

• feat: promptlens import builds golden sets from production traces

  #67·brainsparker opened Oct 25, 2026Owner·

  ·

• feat: multi-sample runs with pass@k, pass^k, flaky-case detection and a --fail-on-flaky CI gate

  #66·brainsparker opened Oct 9, 2026Owner·

  ·

• feat: multi-run judge sampling with flip-rate reporting and --max-flip-rate CI gate

  #65·brainsparker opened Oct 8, 2026Owner·

  ·

• feat: deterministic assertions that run before the judge, with a --fail-on-assertions CI gate

  #64·brainsparker opened Sep 30, 2026Owner·

  ·

• feat: deterministic assertions with judge gating and a --fail-on-assertions CI gate

  #63·brainsparker opened Sep 25, 2026Owner·

  ·

• feat: deterministic assertions with a --fail-on-assertion CI gate

  #62·brainsparker opened Sep 23, 2026Owner·

  ·

• feat: judge stability sampling with a disagreement gate for CI

  #61·brainsparker opened Sep 20, 2026Owner·

  ·

• feat: cost and latency budgets with --fail-on-budget CI gate

  #60·brainsparker opened Sep 19, 2026Owner·

  ·

• feat: deterministic assertions, zero-cost judge mode, and --fail-on-assertion CI gate

  #59·brainsparker opened Sep 10, 2026Owner·

  ·

• feat: deterministic trajectory assertions for agent evals with --fail-on-assertions CI gate

  #58·brainsparker opened Sep 1, 2026Owner·

  ·

• feat: multi-judge consensus scoring with agreement signal

  #57·brainsparker opened Aug 31, 2026Owner·

  ·

• feat: deterministic trajectory assertions for agent tool-call evaluation

  #56·brainsparker opened Aug 30, 2026Owner·

  ·

• feat: deterministic checks, zero-cost assertions that run alongside the LLM judge

  #55·brainsparker opened Aug 28, 2026Owner·

  ·

• feat: OpenTelemetry (OTLP) exporter following the GenAI semantic conventions

  #54·brainsparker opened Aug 27, 2026Owner·

  ·

• feat: multi-sample judging with stability metadata and noise-aware quality gate

  #53·brainsparker opened Aug 26, 2026Owner·

  ·

• feat: deterministic checks with judge-free runs and a --fail-on-checks CI gate

  #52·brainsparker opened Aug 25, 2026Owner·

  ·

• feat: judge spend guard with deterministic checks, result caching, and a per-run judge budget

  #51·brainsparker opened Aug 24, 2026Owner·

  ·

• feat: deterministic assertions (is_json, json_schema, contains, regex) that skip the LLM judge on failure

  #49·brainsparker opened Aug 23, 2026Owner·

  ·

• feat: add promptlens compare for run-over-run regression detection

  #48·brainsparker opened Aug 21, 2026Owner·

  ·

• feat: add promptlens compare for run-to-run regression detection

  #47·brainsparker opened Aug 18, 2026Owner·

  ·

• feat: add promptlens compare for run-to-run regression detection

  #45·brainsparker opened Aug 16, 2026Owner·

  ·

• feat: add promptlens compare command for baseline regression detection

  #44·brainsparker opened Aug 14, 2026Owner·

  ·