#evaluation
Every post tagged "evaluation".
98 posts · showing 21–30
-
Agent Vocabulary: What the Terms Actually Mean
People discussing the same word are frequently discussing different things. Four of the distinctions are load-bearing — and when an agent conversation isn't converging, it's usually one of them.
-
The First Agent to Deploy in a Company That Has None
The first one shapes how much latitude every later one gets. Which means the right choice usually isn't the most valuable use case — it's the one most likely to succeed where people can see it.
-
The Second-System Effect in Agent Deployments
The first agent works because it was scoped well. The second aims at exactly the cases the first one escalated — which is the residual that resisted automation for structural reasons.
-
When to Stop Improving an Agent
There's always another failing case, so without a stopping rule the work either never ends or ends arbitrarily. The signal isn't the error rate — it's effort per point of improvement.
-
How to Run an Agent Pilot That Produces a Decision
Most pilots end without a decision because they were designed to produce a demo. Write the threshold before you build — and sort the failures by cause, because "no-go" and "fix one thing" look identical in the average.
-
Buying vs Building Agent Capability as a Company
Framed as a technology decision, which is why it goes wrong. One question settles it: if a competitor bought exactly what you're buying, would they have what you have?
-
Compliance and Audit for Agent-Written Systems
The audit questions don't change — who's responsible, how do you know it's correct, show me this case. They become answerable differently, and an actor field saying "agent-service" answers the first one not at all.
-
Why "Just Add an Agent" Fails as a Product Strategy
Usage goes curious, then low, then the feature sits there — and the team upgrades the model, which doesn't help, because capability was never the problem.
-
The Use Cases Everyone Tries and Nobody Ships
Six applications get attempted everywhere and stall at the same four points. Each has a narrower version that ships — and it's usually less exciting than the pitch that funded it.
-
The Use Cases Agents Genuinely Solve Today
A criterion beats a list: complete inputs, a mechanical check, bounded blast radius. Everything clearing that bar is working in production — and the largest realized value is the least discussed.