#testing
Every post tagged "testing".
43 posts · showing 21–30
-
Why Agents Struggle Where There's No Feedback Signal
Coding agents work well because compilers exist, not because code is easy. Effectiveness tracks whether a domain has a fast, independent, specific verdict — and that's frequently something you can build.
-
Code Review When Most Code Is Agent-Written
Review practices assumed an author who had reasons, understood the system, and made idiosyncratic mistakes. Agent output breaks all three — and the bugs that survive are the ones that look correct.
-
Autonomous Refactors: What Makes One Safe Enough to Merge
Refactoring is what agents do best and what most easily produces an unreviewable disaster — for the same reason. One property separates the two, and it isn't how hard the change looks.
-
From Prompt Edit to Signal in Under a Minute
A six-minute iteration buys you ten experiments a day; a thirty-second one buys a hundred. Four fixes, about an afternoon of work, and the seam that everything else depends on.
-
How Many Eval Cases Do You Actually Need?
18/20 versus 19/20 is not an improvement — it's noise wearing a number. What your suite size can and cannot detect, and the paired design that makes a hundred cases behave like seven hundred.
-
The QA Agent That Wrote Tests That Always Passed
Coverage climbed, the build stayed green, and a real regression sailed through. An agent asked for passing tests writes passing tests — and the easiest passing test verifies nothing.
-
Content Boundaries: What Your Agent Should Refuse, and How to Test It
Providers handle the general case. Scope, regulatory limits, commitments, and how your agent represents you are yours — and the test suite needs both halves, or you'll ship something that refuses everything.
-
Eval Cases Where the Right Answer Is "I Don't Know"
Suites measure whether the agent got it right, never whether it should have answered at all. Five case types where the correct response is a question — plus the paired cases that stop you from just making it timid.
-
Retry Semantics: Which Tools Are Safe to Call Twice
Agents repeat tool calls for at least five ordinary reasons — none of them bugs. Whether that's harmless or a duplicate charge is a property of the tool, and usually one nobody wrote down.
-
Red-Teaming Your Own Agent: A Practical Injection Suite
Trying a few "ignore your instructions" prompts tests the easiest attack against the strongest defense. A real suite is a matrix of carriers, payloads, and targets — asserting on actions taken, not words said.