AI Version Regression Explainer: What Regressed, in Business Terms
The Version Regression Explainer replays real recorded decisions through both versions and reports which output keys moved, which rules churned, and whether 'same answer, different rules' counts as equivalence. Then the AI explains what changed in business terms.
Marcus Osei
QA Engineering Lead
'POST /rulesets/:id/regression' replays real recorded decisions through both versions and reports: changed outputs (which keys moved, in how many scenarios, first example), rule churn (rules that stopped or started firing), equivalence verdict, and coincidental agreement.
**Coincidental agreement** — same output, different rules — is counted separately from an identical run. A migration that claims parity on outputs but produces them through different rule paths is not a behavioural equivalent; the next out-of-distribution input might diverge. This distinction is what the replay measures and what a structural diff cannot.
**The AI step** is called after the measurement: 'POST /ai/review-diff' is given the structural diff and observed fire counts. The answer is joined onto the structural diff — a changed rule the review ignores is a visible gap; a rule the review names that's not in the diff is an orphan.
**CLI integration**: 'dm regression run --ruleset loan-eligibility --base v11 --target v12 --corpus last-30-days' 'dm regression assert --run <runId> --max-changed-scenarios 0'
Exits 0 if assertion holds, 1 if it fails. Run URL printed to stdout for PR annotation.
Target Topics & Keywords
Ready to evaluate DecisionManager?
Plans without a public rate card. Live demo runs in the browser with no signup. ODM export inventory stays on your machine. Free trial — no card, does not auto-convert.