#observability
Every post tagged "observability".
21 posts · showing 1–10
-
One Query Is Rarely Enough: Iterative Retrieval for Agents
The search wasn't broken — it answered the question as typed, which isn't the question that was asked. Making retrieval a tool the agent calls repeatedly needs one signal most retrievers refuse to give.
-
What I'd Tell a Team Starting Their First Agent Project
Seven things, none of them about model capability, and most of them cost a quarter before they're visible. The first one eliminates more candidate projects than anything else — and should be applied at the idea stage.
-
Tracing Standards for Agents: What Fits and What's Missing
A run is a trace and a turn is a span — the mapping is cleaner than expected. Two things break: payloads far too big for span attributes, and sampling that discards exactly the runs you need.
-
Latency Budgets for Agents: Where the Seconds Actually Go
"Try a faster model" addresses maybe a third of the time in a typical run. Split the twelve seconds five ways and the real culprits — turn count and tool latency — are usually the ones nobody measured.
-
Audit Logs That Survive an Incident
Audit logs get designed for compliance and used during incidents. One question — who caused what, to whom, under what authority — decides every field, and most logs can't answer the authority part.
-
Bisecting a Bad Run: Finding the Change That Broke It
It worked last week. Six things could have changed, and they vary independently — so bisect across the dimensions first, then within one. And diff the raw arrays for what's missing, not what's different.
-
Treating Prompts Like Code: Versioning, Review, and Rollback
A one-line edit in a web console changes behavior across every request, with no history and no way back. Prompts in the repo, rendered hashes in the logs, eval results in the PR — and a quarterly prune.
-
Observability for Agents: What to Log Beyond the Trace
A trace explains one run and nothing about your system. The two layers above it — run records and aggregates — plus three agent-specific signals that standard APM will never suggest.
-
Canary Releases for Prompt Changes
A sentence added to fix one behavior routinely changes three others. Ship prompts like behavior: versioned by rendered hash, split by stable key, and compared on tool distribution — the row nobody collects.
-
Building an Eval Set From Production Traces
Hand-written cases test what you thought of. A user rephrasing the same request three times is a labelled failure sitting in your logs — here's how to turn a week of traces into a suite that catches the real ones.