Tools of the Trade
Frameworks, SDKs, platforms, build-vs-buy.
26 posts · showing 21–26
-
Treating Prompts Like Code: Versioning, Review, and Rollback
A one-line edit in a web console changes behavior across every request, with no history and no way back. Prompts in the repo, rendered hashes in the logs, eval results in the PR — and a quarterly prune.
-
Observability for Agents: What to Log Beyond the Trace
A trace explains one run and nothing about your system. The two layers above it — run records and aggregates — plus three agent-specific signals that standard APM will never suggest.
-
Model Routing: Using the Cheap Model for the Cheap Steps
Classification, extraction, and formatting move to a cheaper tier safely. Tool selection and planning don't — and the failure looks like a tool-description bug, so you'll debug the wrong thing for a week.
-
Vector Database or Postgres? Choosing Storage for Agent Memory
Most agent memory reads are filtered similarity or plain structured lookup — relational workloads wearing an embedding. Write down your ten most common retrievals before you add a second database.
-
Build or Buy the Eval Harness? What Each Choice Locks In
Teams buy an eval platform and keep their own scripts anyway. The reason is deterministic replay — it hooks into your dispatch layer, which is the one part no vendor can own.
-
Framework or Plain Loop? Four Questions That Actually Decide It
The core agent loop is under a hundred lines — that's never what a framework saves you. Four situational questions decide it, and the one about 3am debugging should carry the most weight.