#evaluation
Every post tagged "evaluation".
98 posts · showing 31–40
-
What Happens to QA When Agents Write the Tests
Cheap test generation makes suite quality matter more, not less — because the default outcome is a large, green suite that verifies nothing. Someone has to own whether it catches things.
-
Evaluation Engineer: The Job Description Writing Itself
The furthest-formed of the emerging roles: well-defined, consistently needed, consistently unstaffed. It's everyone's dependency and nobody's job, which is exactly why.
-
Agent Operations: The Role Without a Title Yet
Someone in every agent-running organization does this job without being hired for it. It has no name, no allocation, and no incident that forces it — until whoever was doing it informally leaves.
-
The Jobs That Get Created Are Hard to Name in Advance
"New jobs will be created" sounds like faith because the examples are missing. There's a structural reason for that — and one observable signal that's better than any forecast.
-
Prompt Engineering Was the Beginning, Not the Skill
It got named as a discipline when the prompt was the entire interface. It's now one input among several — and if better prompts have stopped helping, five specific things are usually why.
-
Lock-In Risk in Agent Tooling, and How to Keep Optionality
The API call is trivially portable — the coupling forms in four other places, and the worst one isn't technical at all. One question tells you how exposed you are.
-
The Moat Isn't the Model — It's the Loop Around It
"The model isn't the moat" is now widely said and stops one step short. What compounds shares a property — it accumulates from watching real usage, which is why it's invisible from outside.
-
What a Harness Actually Is, and Why It's the Real Product
A model takes messages and returns tool calls. Everything else is the harness — and it explains almost all the variance between agent systems that share the same model.
-
If Intelligence Gets Cheap, What Becomes Scarce?
More useful than asking what gets cheap. Everything on the scarce list has one of two properties — and neither can be produced on demand, which is exactly why they stay scarce.
-
What Falling Inference Costs Change About Agent Design
Relaxed cost pressure mainly buys back the verification steps you declined. It doesn't rehabilitate filling the context or adding turns — those were quality problems wearing a cost disguise.