AI Observability: How to Know What Your Agents Are Actually Doing

AI agents deployed in production without structured observability create compliance gaps you cannot close retroactively — and the longer they run, the larger the exposure. Langfuse, the open-source LLM observability platform, has become a de facto standard specifically because most teams shipped agents first and instrumented them later. If you cannot reconstruct exactly what your agent did, why, and what it cost for any given run, you are not ready for a compliance audit.

What this article covers:

• What agent observability actually means: traces, logs, and cost tracking
• What HIPAA, GDPR, and SOC 2 require for AI agent audit trails
• What a complete, auditable agent run record looks like
• A five-step process to add observability to agents you are already running

Why can’t most teams answer basic questions about what their AI agents did?

Most teams deploying AI agents focus on capability: can the agent do the task? The monitoring question comes later — usually after something breaks, or after a compliance officer asks to see a record of what your AI has been doing with customer data.

The gap is real. One 2026 study on enterprise agent deployments found that a significant portion of teams piloting agents cannot reliably reproduce a failed agent run after the fact. Once the context window closes, the reasoning chain is gone. All you have is the output.

That’s fine for a toy project. It’s not acceptable for any agent that:

• Touches customer or patient data
• Makes decisions that affect people (approvals, pricing, triage)
• Operates under HIPAA, GDPR, SOC 2, or any other regulatory regime
• Costs real money per run that you need to track and allocate

The agents that matter most in your business are the ones that need the most scrutiny — and those are exactly the ones where “I checked the output and it looked fine” isn’t enough.

What does AI agent observability actually mean?

Observability in traditional software means you can understand the internal state of a system from its external outputs — logs, metrics, traces. For AI agents, that definition gets more complicated.

An agent run isn’t a single function call. It’s a sequence: a prompt is constructed, a model is called, a response is parsed, tools might be invoked, more model calls happen, and eventually an output is produced. Any of those steps can fail, behave unexpectedly, or produce results you need to audit.

Full agent observability covers three things:

1. Traces — The complete reasoning chain for a single agent run. Which model was called, with what prompt, at what time. What tool calls were made. What the intermediate results were. How long each step took. Traces let you reproduce and debug any specific run.

2. Logs — A persistent, structured record of agent activity over time. Not raw text — structured data you can query: which agents ran, for which users, with what inputs and outputs. Logs are what auditors and compliance teams actually look at.

3. Cost tracking — Token consumption per run, per model, per agent, per user or team. If you’re running agents at scale without this, you’re flying blind on costs. Usage-based pricing surprises are now a documented driver of teams abandoning cloud AI entirely.

What do HIPAA, GDPR, and SOC 2 require for AI agent logging?

For teams in healthcare, finance, legal, or any regulated industry, observability isn’t optional — it’s a requirement with teeth.

Under HIPAA, any agent that touches protected health information (PHI) must produce an audit trail that shows what data was accessed, when, by whom (or by what system), and for what purpose. The Office for Civil Rights expects immutable logs. “The AI handled it” is not a satisfying answer in a breach investigation.

GDPR’s accountability principle has the same implication: if you’re processing personal data through an AI agent, you need to be able to demonstrate that the processing was lawful, minimal, and auditable. A model call to a third-party provider with user data attached is a data processing event — it needs to be logged.

SOC 2 Type II audits increasingly include AI workloads. Auditors want to see that access to sensitive systems (including AI models that might see sensitive data) is controlled, logged, and reviewable.

The practical question isn’t whether you need audit logs — if you’re in a regulated industry, you do. The question is whether you’re building that capability in from the start or retrofitting it after a compliance conversation forces the issue.

Retrofitting is significantly harder.

What should you be able to retrieve for any AI agent run?

Here’s what you should be able to answer for any agent run:

• Identity: Which agent? Which user or team triggered it?
• Input: What was the full prompt sent to the model? What context files or retrieved data were included?
• Model: Which provider and model handled it? What were the parameters (temperature, max tokens)?
• Tool calls: Did the agent invoke external tools? What were the inputs and outputs of each call?
• Output: What did the agent return?
• Timing: When did the run start and end? How long did each step take?
• Cost: How many tokens were consumed? What did it cost at current provider rates?
• Outcome: Did it succeed, fail, or time out? If it failed, where?

That’s a specific, auditable record. Compare it to the logging situation at most teams deploying agents today: “we have the output in Slack” or “we log the final response to a database.”

The difference matters enormously if you ever need to answer a compliance question, debug a production issue, or investigate why an agent gave someone the wrong information.

What observability tools are teams actually using for AI agents?

Langfuse — an open-source LLM observability platform that came out of YC W23 — has become a de facto standard for teams that take this seriously. It integrates with OpenTelemetry, LangChain, the OpenAI SDK, and others. It gives you traces, user sessions, prompt management, and cost dashboards in one place.

The fact that Langfuse is gaining this much traction is itself a signal: teams are realizing they shipped agents without instrumentation and are now going back to add it.

Other tools in this space include Arize, Portkey, and Braintrust. They’re all attacking the same problem: agents are increasingly production systems, and production systems need observability.

The pattern that’s emerging in 2026 is “proof chains, not just logs” — borrowed from the distributed systems world. A log tells you what happened. A proof chain tells you what happened, why, and that the record itself hasn’t been tampered with. For compliance-heavy environments, that distinction is starting to matter.

How does your deployment model affect your AI observability options?

If you’re running agents on a SaaS platform, observability is whatever that platform gives you — which may be comprehensive, limited, or positioned as a paid add-on. You’re dependent on their data model, their retention policies, and their ability to produce the format your auditors want.

If you’re self-hosting your agent infrastructure, you control the logging layer entirely. You can define the schema, the retention period, the access controls, and the export format. Your audit trail lives on your servers, not in a vendor’s database that you can’t directly query.

For teams in regulated industries, this is often the deciding factor. It’s not just that you want observability — it’s that you need to own it, in a form you can hand to an auditor without going through a vendor support ticket.

What are the first steps to add observability to your existing AI agents?

If you’re operating agents without adequate observability, here’s where to start:

Step 1: Define what a “complete run record” means for your context. Write it down. What fields does your compliance team need? What would an auditor ask to see? This is your schema.

Step 2: Instrument at the agent level, not just the application level. Logging the final output isn’t enough. You need to capture the full prompt, any tool calls, intermediate model responses, and timing for each step.

Step 3: Make logs structured and queryable. A text file of agent outputs is not useful for compliance. JSON logs that you can query by user, by date range, by agent, by outcome — that’s what matters.

Step 4: Add cost attribution. Even if it’s not a compliance requirement, token costs per agent and per user will become important as you scale. Do this now while it’s easy.

Step 5: Test your audit story before you need it. Pick a specific agent run from last week. Can you reconstruct exactly what happened? If not, your observability is a gap you need to close before a real audit forces the issue.

How does Auxot handle AI agent observability?

Auxot logs every agent interaction by default — the full prompt, the model, the tool calls, the output, timing, and cost. Logs are stored on your own infrastructure, not ours. You define who has access.

For teams in regulated industries, this means your audit trail exists, is structured, and is yours to query and export. No support ticket required to pull records for a compliance review.

Every agent run in Auxot is associated with the specific user who triggered it, the agent configuration that handled it, and the model that processed it. If an auditor asks what your AI did with a patient’s record on a specific date, you can answer that question from your own data.

If you want to see how this works in practice, start with the tutorials →

Or, if you’re ready to run agents with built-in observability on your own hardware: install Auxot →

---

AI observability is a fast-moving space. The frameworks are getting better. The compliance requirements are getting clearer. The teams that instrument their agents correctly from the start will spend a lot less time retrofitting controls after the fact.