All posts
Everything published here, newest first.
316 posts · showing 291–300
2026
10 posts-
Build or Buy the Eval Harness? What Each Choice Locks In
Teams buy an eval platform and keep their own scripts anyway. The reason is deterministic replay — it hooks into your dispatch layer, which is the one part no vendor can own.
-
The Research Agent That Cited the Wrong Page
Every source was real and had been read. Two citations still pointed at the wrong page — one at a source that disagreed. Attribution is a lookup, and generation isn't a lookup.
-
Where to Put the Approval: Human-in-the-Loop Is a Design, Not a Checkbox
A confirm dialog nobody reads has all the latency of oversight and none of the protection. Four placements, a routing rule based on reversibility, and the metric that tells you your gate has become a rubber stamp.
-
The LLM Judge That Agreed With Everything
A suite scored 94% and 93% on two versions — but the judge would have passed almost anything. Four biases quietly manufacture those numbers, and one prompt change fixes more than the rest combined.
-
What Belongs in Long-Term Memory (and What's Just Transcript)
Embedding every turn gives you a transcript with a search box, not memory. Three categories deserve to outlive a conversation — and the contradiction problem needs deciding before it arrives, not after.
-
One Search Tool or Five? Getting Tool Granularity Right
Five narrow search tools or one with filters? The choice doesn't remove errors, it relocates them — and one of the two places is much cheaper to defend.
-
The Planner That Plans Too Much: When Upfront Decomposition Hurts
An agent found the root cause in step one and executed the remaining seven steps anyway. Plans in context outweigh the evidence that should kill them — unless revision is something the loop can actually do.
-
Prompt, Tool, or Code? Where Each Piece of Logic Belongs
A rule the agent keeps breaking usually isn't a prompting failure — it's deterministic logic filed under "advice." Two questions place any behavior in the right one of three homes.
-
Streaming an Agent's Progress Without Lying About It
Token streaming goes quiet exactly when an agent gets slow — inside the tool call. The fix is an event stream from the loop, and the hard part is not inventing certainty the loop doesn't have.
-
Chain, Blackboard, or Broadcast? Three Ways Agents Share State
Chains lose the evidence, blackboards fight over it, broadcasts drown in it. Pick by which failure your task can absorb — and log provenance from day one or debugging becomes guesswork.