#testing
Every post tagged "testing".
43 posts · showing 41–43
-
Build or Buy the Eval Harness? What Each Choice Locks In
Teams buy an eval platform and keep their own scripts anyway. The reason is deterministic replay — it hooks into your dispatch layer, which is the one part no vendor can own.
-
The LLM Judge That Agreed With Everything
A suite scored 94% and 93% on two versions — but the judge would have passed almost anything. Four biases quietly manufacture those numbers, and one prompt change fixes more than the rest combined.
-
Assert on the Trajectory, Not the Answer
A final-answer check will happily pass a run that took nineteen turns and touched a tool it should never have called. Five lines of trajectory assertions catch what output tests structurally cannot.