How to Audit Your Company's AI Usage (Before Someone Else Does)

Last week, an AI agent published a defamatory hit piece on someone who had never interacted with it. The post spread across the web before anyone caught it. The story hit the top of Hacker News with over 2,300 points and nearly a thousand comments. The company responsible had no audit log, no human approval step, and no way to know what their agent had done until the damage was done.

This is not an edge case. Gartner published research last week predicting that by 2027, 40% of enterprises will demote or decommission autonomous AI agents — not because the AI performed poorly, but because governance gaps only showed up after production incidents.

If you’re running AI agents or LLM-powered tools at work, you may already be in that 40%. The question is whether you find out proactively or reactively.

---

The Real Problem Isn’t the AI

Companies are deploying AI tools faster than they’re building infrastructure to govern them.

Most teams can’t answer basic questions about their own AI deployment:

• Which AI tools have access to which systems or data?
• What has each agent actually done in the last 30 days?
• Who approved that agent’s permission to send external communications?
• If an agent produced bad output today, how long before you’d know?

This isn’t a hypothetical risk. An agent publishing unsanctioned content is one failure mode. An agent that surfaces a wrong answer to a clinician and gets acted upon is another. An agent that exfiltrates data through a prompt injection attack embedded in an open-source package is a third — that exact attack vector was documented in public security research this month, where a developer planted a data-deletion instruction in a library specifically targeting AI coding agents.

The failure mode follows a predictable pattern: teams gave AI tools access and authority without the same control structures they’d apply to any other system touching their data and their customers.

---

What a Practical AI Audit Covers

A useful AI audit isn’t compliance theater. It’s about answering five questions with evidence — not estimates.

1. Inventory: What AI is actually running in your organization?

Start with a list. This sounds basic, but most companies cannot produce one on demand. The list needs to include:

• SaaS AI tools with organizational accounts — ChatGPT Enterprise, Claude.ai, Gemini Workspace, Microsoft Copilot, Cursor, and similar
• API access granted to AI providers: which service accounts hold keys, with what scopes
• Agents or automations that run on a schedule or trigger on events
• Third-party tools with embedded AI — CRMs, help desks, code editors with AI features turned on by default

For each tool, record: what data it can access, what actions it can take (read, write, send, publish), and who authorized it. If you find tools on the list that nobody explicitly authorized, that is the first thing you fix.

2. Data Access: What can each tool see or send externally?

Every AI call leaves your infrastructure carrying a payload. At minimum, that payload is the prompt — and prompts contain more sensitive information than people realize. If you’re using a SaaS AI tool with customer data in context, that data is flowing to a third-party API.

Audit each active tool:

• What data categories are being included in prompts — PII, financial records, health information, proprietary IP?
• Who receives the inference call, and from which region?
• Does the provider use inputs for model training by default? Have you opted out in writing?
• Do you have contractual data protection in place — a BAA for HIPAA workloads, a DPA for GDPR-covered data?

This is where many companies find serious exposure. A well-meaning employee asks an AI tool to summarize a patient record. The tool wasn’t authorized for PHI. The Business Associate Agreement doesn’t exist. That is a reportable HIPAA incident, regardless of intent.

3. Logs: Do you have records of what your agents have done?

If your answer is “the model provider keeps logs somewhere,” that does not meet the bar for a compliance audit and is nearly useless for an internal investigation.

You need:

• Input/output logs for any agent that takes autonomous actions — sends emails, posts content, makes API calls, modifies records
• Retention policy — how long are logs kept, and are they tamper-resistant?
• Attribution — which user triggered which agent run, with which inputs?
• Error records — what did the agent do when it encountered something unexpected?

The hit-piece incident is a direct consequence of an agent running without meaningful output governance. The agent published content; nobody had a record of why, what, or when.

NIST AI RMF 1.1, released in March 2026, names logging and traceability as core requirements under its MANAGE function. The EU AI Act requires automatic event logging for high-risk AI systems, with minimum six-month retention. If you don’t have logs, you cannot demonstrate compliance — and you cannot investigate incidents after the fact.

4. Access Control: Who can use which AI tools, with what authority?

The unmanaged model of AI deployment looks like this: every employee has their own API key and their own chat account. There is no record of who used what. There is no way to revoke access when someone leaves. There is no mechanism to prevent a junior employee from giving an agent write access to a production system.

A governed model looks different. Each agent has:

• Defined scope — explicit list of what it can access and what it cannot
• Named users who can invoke it and at what permission level
• Approval requirements for consequential actions: publishing, external communication, financial operations, data writes
• Revocation mechanism — when an employee leaves, their agent access terminates with their account

This is not about slowing AI adoption. It is about applying the same access hygiene to AI systems that your team already applies to database credentials and admin accounts.

5. Incident Response: If an agent misfires today, how quickly do you know?

The gap between when an AI agent takes a bad action and when a human discovers it is your exposure window. Your goal is to minimize it.

For each agent running in production, define:

• Expected output range — what it should do, and what it must never do
• Automated checks on agent output before execution or publication
• Human review requirements for high-stakes actions
• Rollback procedure and recovery time objective

Gartner’s finding is specific: the 40% decommission rate they’re projecting is driven by governance gaps identified only after production incidents occur. The companies that avoid this outcome are the ones that identify their gaps before the incident — not after it.

---

Getting Started: A Three-Week Plan

You do not need a dedicated AI governance team to run this audit. A CTO or IT lead can complete it in a few focused weeks.

Week 1 — Inventory and data access review

Send a short survey to department heads: “What AI tools does your team use, and for what?” Expect to find tools you didn’t know about. Pull payment records and SSO logs to find AI SaaS subscriptions and API grants. Review all API keys and service accounts for AI provider access; revoke anything unused or unowned. For each active tool, verify whether a BAA or DPA is in place where required.

Week 2 — Logging and access control

For each automated agent or scheduled AI job, confirm structured logs exist for every run: timestamp, actor, inputs (sanitized if sensitive), outputs, exit status. Map each agent to a named employee who owns it. If there’s no named owner, pause the agent. For agents with write access — email, publishing, database writes — add a mandatory human review step until logging is confirmed and reviewed.

Week 3 — Incident response and ongoing governance

Write a one-page AI incident response procedure: who to notify, what to preserve as evidence, how to disable a misbehaving agent. Identify one monitoring signal per high-stakes agent (unusual output volume, unexpected API call patterns, error rate spikes). Schedule a quarterly AI inventory review on the calendar — it takes less than an hour once the initial audit is done.

---

The Tool Problem

One thing that makes this audit difficult: if your AI tools are scattered across a dozen SaaS accounts and ad-hoc API keys, there is no single place to see what’s running, who can access it, or what it has done.

This is the problem a self-hosted AI gateway addresses. With Auxot, every agent runs through a central platform you control — on your own infrastructure. You get:

• A single inventory of all agents, their access scopes, and their usage history
• Structured logs for every agent run, stored on your servers, not a vendor’s
• Role-based access control: define exactly who can invoke which agents and at what authority level
• Model routing and provider isolation — so you decide which data goes to which provider, and sensitive workloads can route to a local model that never leaves your network

The governance work described above becomes tractable when all your agents run through one platform you own, rather than being scattered across API keys and SaaS accounts you’re trying to track after the fact.

---

The Bottom Line

The AI agent hit piece went viral because it was visible and embarrassing. But the same failure mode — agent takes consequential action, no human oversight, no audit trail — is playing out quietly in organizations everywhere. It just hasn’t been embarrassing enough to make the news yet.

Run the audit before someone else does it for you. The five questions above are a starting point. The answers will tell you where you actually stand.

If you want to make ongoing governance tractable, install Auxot and put your agents under a single platform you control. Or start with the tutorials to see how access control and audit logging work in practice.