t1-p3-06draft4–6h
Drift and regression alerting
requires: t1-p3-05
Spec
A scheduled job that re-runs the Phase 2 eval set nightly and alerts - Slack webhook or email - when the aggregate score drops more than a defined threshold from baseline. Commit the schedule definition and the threshold.
Why this matters
This closes the loop from Phase 2. An eval set that runs once is a report; an eval set that runs nightly is a regression test, and it is the only thing that catches a model provider changing behaviour underneath you.
Verify
No result yet. Run `learn verify t1-p3-06` locally, or push to trigger CI.
repo
unrun
contract
unrun