Constraint Decay: Why AI Agents Break Your Backend Rules at Scale

Your AI agent worked perfectly in the demo. It called the right APIs, respected the rate limits, stayed inside the permission boundary you defined. You shipped it.

Three weeks later it’s hammering a third-party endpoint, ignoring a schema constraint you thought you’d enforced, and someone in finance is asking why a report pulled data it shouldn’t have access to.

This isn’t a prompt engineering failure. It’s constraint decay — and a new piece of research from May 2026 has given it a name and a mechanism.

What the Research Actually Found

A paper published on arXiv in early May 2026 — Constraint Decay: The Fragility of LLM Agents in Backend Code Generation — hit the top of Hacker News with 217 points and 116 comments. The core finding is straightforward to state and uncomfortable to sit with:

LLM agents are functional under no constraints. Performance degrades systematically as structural constraints increase.

The researchers (Francesco Dente et al.) tested agents against the same API specification under varying degrees of structural constraint — things like authentication rules, rate limit policies, and schema restrictions. Without constraints, agents produced working code reliably. With constraints layered in, compliance dropped — not linearly, but in a pattern the researchers called “decay”: the more constraint you impose, the more the agent begins to violate the constraints it understood earlier.

The practical implication: your agent didn’t “forget” the rule. The rule was deprioritized when the agent was juggling enough other context. And in production, agents always juggle more context than they do in demos.

How Constraint Decay Shows Up in Production

This isn’t a theoretical risk. Fiddler AI’s production data puts the failure rate for AI agents in production at between 70% and 95%. Amazon’s engineering team, documenting lessons from internal agent deployments, identified poorly defined tool schemas as a leading cause — agents invoking irrelevant APIs, expanding context unnecessarily, silently producing incorrect results.

Constraint decay is one of the specific mechanisms behind that number. Here’s how it typically manifests:

Authentication drift. The agent knows the rule: use the read-only token for this endpoint. Under low complexity, it does. Add a few more tool calls, a branching decision, an error-recovery path — and the agent starts reaching for the higher-permission token because it’s in context and the constraint was four steps back. You don’t find out until an audit log review or a security incident.

Rate limit violations. You’ve told the agent to stay under 10 requests per minute to an external API. It respects this when the task is simple. Under a longer-horizon task — researching a customer account, generating a complex report — the rate limit instruction is still technically in the system prompt, but the agent has compressed and partially discarded earlier instructions in favor of staying on task. By request 14, nobody counted.

Schema constraint erosion. You’ve defined a strict output schema for a data pipeline agent. The first 20 runs are clean. On run 21, the task is more complex, the context window is fuller, and the agent produces a field that doesn’t match the schema. The pipeline accepts it anyway because you didn’t enforce validation at the receiving end. The bad data is now downstream.

In each case, the constraint wasn’t absent from the prompt. It decayed in the agent’s operational behavior under the pressure of complexity.

Why This Is Especially Dangerous in the Cloud

Most SaaS agent platforms give you a single enforcement point: the system prompt. You write instructions, the model follows them, and the platform logs outputs if you’ve paid for that tier.

That’s a thin safety net for a mechanism that, by definition, degrades under complexity. There’s no enforcement layer between the agent’s output and the backend system it’s touching. The only check is whether the LLM remembered the rule at the moment it mattered — and the research says that reliability goes down as task complexity goes up.

For teams in regulated industries — healthcare, finance, legal — this isn’t a reliability nuisance. It’s a compliance exposure. An agent that pulls data it shouldn’t have accessed, or writes to a table under a permission it wasn’t supposed to use, doesn’t become less of a problem because the violation was probabilistic rather than intentional. The audit finding is the same.

The Fix Is Architectural, Not Prompt-Based

The instinct when you read this research is to write better system prompts. Add more emphasis. Use ALL CAPS. Put the constraint in three places.

This doesn’t solve the problem because the problem isn’t that the agent didn’t understand the constraint. It’s that understanding a constraint in context and enforcing a constraint at execution are two different things, and prompts only address the first one.

The fixes that work are structural:

1. Enforce constraints at the tool layer, not the prompt layer.

If an agent should never call endpoint X with token Y, that rule should live in the tool definition — not as an instruction the model reads, but as a hard restriction in the function schema the model is calling. Structured tool schemas that encode constraints prevent the agent from expressing intent it’s not authorized to act on, regardless of what it decided to prioritize in its chain of thought.

Amazon’s lesson is instructive here: well-defined tool schemas aren’t just documentation for the model. They’re the actual constraint enforcement mechanism. If the schema says “this parameter accepts values 1-100,” an agent can’t send 500 — the tool refuses the call.

2. Add a gateway enforcement layer.

A self-hosted AI gateway gives you a chokepoint between your agents and your backend systems. Every outbound call from an agent goes through the gateway, which can enforce rate limits, check permissions, and log the full request — independent of what the agent decided to do. The agent can have constraint decay all it wants; the gateway enforces the rules at the boundary.

This is categorically different from telling the agent to respect rate limits. The gateway will refuse the 11th request in a minute regardless of whether the agent thought it was within budget.

3. Validate outputs structurally, not semantically.

If an agent produces data for a downstream pipeline, validate the schema at ingestion — not by asking the agent to self-check. Schema validation catches constraint drift before bad data propagates. This is standard software engineering practice that gets skipped because agents feel more like conversation partners than data pipelines. They’re still data pipelines. Treat them that way.

4. Build audit trails that log at the agent layer, not just the output layer.

Knowing what an agent produced is less useful than knowing what it called, in what order, with what permissions. An audit log that captures tool calls with their inputs, the timestamp, and the permission state at the time of the call gives you the forensic data to detect constraint drift before it becomes a compliance incident. It also makes the patterns visible — if you see authentication drift happening on tasks above a certain complexity threshold, you have actionable data to tighten the tool schema before the next run.

Putting It Together: What a Constraint-Safe Agent Stack Looks Like

The research frames this as a structural problem in how LLM agents are built today. The practical architecture that addresses it has three layers:

• Model layer: well-structured tool definitions that encode constraints in schema, not just description. The model can’t call what the schema doesn’t allow.
• Gateway layer: an enforcement chokepoint that applies rate limits, permission checks, and logging independent of model behavior. Deployed on your own infrastructure so you control the policy.
• Data layer: output schema validation before anything hits a database or downstream system. Don’t rely on the model having remembered the constraint.

None of these layers requires a more capable model. The constraint decay paper found that higher-capability models also show the decay pattern — it’s a structural property of how LLM agents handle competing context, not a capability floor problem. Better prompting doesn’t fix it. Architectural enforcement does.

The Honest Assessment

If you’re running AI agents in production today without enforcement at the tool, gateway, and output layers, you’re accepting a risk that scales with task complexity. Simple agents on simple tasks will probably be fine. The constraint drift becomes meaningful when agents are handling multi-step workflows, calling multiple backends, or running in production environments where the task distribution is less predictable than it was in your test cases.

The 70–95% production failure rate isn’t an indictment of AI agents as a category. It’s a description of where most current deployments are architecturally. The teams that are shipping reliable agent deployments aren’t using better models or better prompts — they’re enforcing constraints in the right places.

---

If you’re evaluating how to run AI agents against your own systems with enforcement at the gateway layer and audit trails out of the box, Auxot is built for exactly this. Or start with the tutorials to see how tool schemas and gateway-enforced rate limits work in practice.