t1-p2-02draft4–6h

Automated eval runner

requires: t1-p2-01

Run all 50 questions through the pipeline, recording per question the retrieved chunks, generated answer, latency, and token cost. Write one run file per execution to evals/runs/<run-id>.json, referencing the eval set version used.

Recording retrieved chunks alongside the answer is what makes the failure taxonomy possible. Without it you can see that an answer was wrong but never whether retrieval or generation caused it - and those need opposite fixes.

No result yet. Run `learn verify t1-p2-02` locally, or push to trigger CI.

contractcontract: t1_p2_02_eval_runner
unrun
artifactartifact: evals/runs/
unrun