Multi-Tenancy for Agent Products: The Boundaries That Must Hold

Building an agent product for many customers adds isolation requirements that a single-tenant deployment never confronts. Some are the ordinary multi-tenancy problems. Several are specific to agents, and those are the ones that get missed because no existing checklist covers them.

The four boundaries

Data. Tenant A's agent must not read tenant B's data. Standard, and the standard solutions apply — scope every query at the query layer, never filter after retrieval.

Memory. Agent memory is a second data store, often built later, often without the same discipline. A memory retrieval that searches globally and filters by tenant afterwards is one bug from a cross-tenant leak, and memory stores tend to escape the review that primary data access gets.

Context. Nothing from one tenant's run may persist into another's. This sounds automatic and isn't: caches keyed carelessly, a shared summarizer that batches across tenants, a "recent examples" feature that pulls from all traffic. ⚠️ Few-shot examples harvested from production are a genuine leak vector — one tenant's data becomes another tenant's prompt.

Cost and capacity. One tenant's runaway agent must not consume the rate limit or budget that others depend on. Not a security boundary, but it's the one that produces incidents most often.

Where agents differ from ordinary SaaS

Tool credentials are per-tenant. Each tenant connects their own systems, so the tool layer must resolve credentials from the run's tenant, every call, with no fallback to a default. A fallback to a service account is how a misconfigured tenant silently reads someone else's data.

Configuration is per-tenant and affects behavior. Custom instructions, enabled tools, thresholds. This means "the agent" isn't one thing, so a bug can exist for one tenant and not another, and your eval suite tests a configuration nobody actually runs.

✅ Worth running evals against a sample of real tenant configurations, not only the default. This catches the class of problem where a tenant's custom instruction interacts badly with a change that tested fine.

Prompt caching is a tenant boundary. A shared prefix is fine; the moment tenant-specific content enters the cached region, cache entries must be tenant-scoped. Cache keys that omit tenant identity are both a correctness and a confidentiality problem.

Rate limits are shared upstream. Your model provider's limit is global to your account. Without per-tenant budgets, one tenant's batch job degrades everyone. Per-tenant quotas — enforced in your queue, not hoped for — are the fix.

Enforce at the boundary, not in the agent

The rule that makes this tractable: tenant identity is established once, at run start, and flows through the infrastructure — never through the model.

run = start_run(tenant_id="t_88", user_id="u_12", ...)
# every tool call, every memory read, every log line
# derives tenant from run context, not from arguments

If tenant were a tool parameter, an injected instruction could change it. Making it structurally impossible for the model to influence is worth more than any number of prompt-level rules, and it's the single design decision that determines whether the rest is defensible.

🔍 The tests that must exist

Write these before you need them:

• Tenant A's run requests tenant B's data by ID → fails at the data layer.
• Tenant A's memory query → returns nothing from tenant B, verified against a store containing both.
• An injection attempting to change tenant identity → structurally impossible.
• Tenant A exhausts its budget → tenant B's runs unaffected.
• A cached prefix from tenant A → not served to tenant B.

The last one is easy to get wrong and easy to test: run the same request under two tenants with different custom instructions, and check the outputs differ appropriately.

Noisy neighbours, concretely

Agent workloads are burstier than typical SaaS, so isolation needs to be more than nominal:

• Per-tenant concurrency caps, so one tenant can't occupy every worker.
• Per-tenant token budgets over a window, with a defined behavior at exhaustion — queue, degrade, or reject, decided in advance.
• Separate queues, or at least fair scheduling, so a large batch doesn't starve interactive requests from other tenants.
• Per-tenant cost visibility, because "which customer is unprofitable" arrives eventually and retrofitting the tagging loses the history.

The takeaway

Four boundaries — data, memory, context, and capacity — with tenant identity established at run start and carried by infrastructure rather than by anything the model can influence. Scope memory queries as carefully as primary data, key caches by tenant, never harvest few-shot examples across tenants, and give every tenant its own concurrency and token budget. Then write the negative tests, because the failures here are the ones that end customer relationships.