#observability
Every post tagged "observability".
40 posts · showing 21–30
-
What I'd Tell a Team Starting Their First Agent Project
Seven things, none of them about model capability, and most of them cost a quarter before they're visible. The first one eliminates more candidate projects than anything else — and should be applied at the idea stage.
-
Tracing Standards for Agents: What Fits and What's Missing
A run is a trace and a turn is a span — the mapping is cleaner than expected. Two things break: payloads far too big for span attributes, and sampling that discards exactly the runs you need.
-
Latency Budgets for Agents: Where the Seconds Actually Go
"Try a faster model" addresses maybe a third of the time in a typical run. Split the twelve seconds five ways and the real culprits — turn count and tool latency — are usually the ones nobody measured.
-
Audit Logs That Survive an Incident
Audit logs get designed for compliance and used during incidents. One question — who caused what, to whom, under what authority — decides every field, and most logs can't answer the authority part.
-
Bisecting a Bad Run: Finding the Change That Broke It
It worked last week. Six things could have changed, and they vary independently — so bisect across the dimensions first, then within one. And diff the raw arrays for what's missing, not what's different.
-
Treating Prompts Like Code: Versioning, Review, and Rollback
A one-line edit in a web console changes behavior across every request, with no history and no way back. Prompts in the repo, rendered hashes in the logs, eval results in the PR — and a quarterly prune.
-
Observability for Agents: What to Log Beyond the Trace
A trace explains one run and nothing about your system. The two layers above it — run records and aggregates — plus three agent-specific signals that standard APM will never suggest.
-
Canary Releases for Prompt Changes
A sentence added to fix one behavior routinely changes three others. Ship prompts like behavior: versioned by rendered hash, split by stable key, and compared on tool distribution — the row nobody collects.
-
Building an Eval Set From Production Traces
Hand-written cases test what you thought of. A user rephrasing the same request three times is a labelled failure sitting in your logs — here's how to turn a week of traces into a suite that catches the real ones.
-
Cost Attribution: Finding Which Feature Is Eating Your Token Budget
Agent spend is dominated by a tail of expensive runs, so every averaged chart hides the cause. Five tags on each model call, one table of your twenty priciest runs, and the mystery usually resolves in an afternoon.