AI Agent Observability: The Monitoring Layer Most Teams Skipped

Most teams instrument their AI agents the way they instrument a web service. Latency, error rate, throughput, maybe token count. Then an agent quietly does something wrong for three weeks and nobody notices until a customer escalates.

The reason is structural. Conventional application monitoring answers "did it work?". Agent failures are almost never failures of that kind. The request succeeded, the latency was fine, no exception was raised, and the agent called the wrong tool with entirely plausible arguments and produced a confident, wrong result.

That class of failure is invisible to APM by construction. Agent observability exists to make it visible.

Why non-determinism breaks conventional monitoring

Three properties of agentic systems defeat the assumptions APM is built on.

Same input, different execution. A deterministic service given identical input takes an identical path. An agent may reason differently, call different tools, or take a different number of steps. Aggregate metrics over runs that are not comparable produce averages that describe nothing real.

Failure is semantic, not structural. The interesting failures do not throw. They return successfully with the wrong content. No status code encodes "technically fine, substantively incorrect".

The interesting behaviour is in the middle. A conventional trace cares about the boundaries: request in, response out. For an agent, the valuable information is the intermediate reasoning and the sequence of tool calls. If you only capture the endpoints, you have discarded the part that explains the outcome.

The four layers of an agent observability stack

Traces are the foundation. A trace should capture the complete run: triggering input, model and version, each reasoning step, every tool call with arguments and results, retrieved context with its source, token usage per step, and the final output. Tool calls with arguments are the single most valuable field, because that is the record of what the agent did rather than what it said. Without it you can read an agent's explanation of its actions, which is not the same thing and occasionally not true.

Evaluations turn quality into a measured signal. A held-out set of representative cases with known good outcomes, scored on every release and sampled continuously in production. Scoring mixes deterministic checks — did it call the expected tool, is the output valid against schema, did it stay within scope — with quality judgements, usually from a model-based grader. Running this only pre-release is the common mistake. Agent quality drifts when the underlying model updates, when a prompt changes, and when real input distribution shifts away from your test set.

Cost attribution matters more for agents than for conventional services because the marginal cost per request is both significant and highly variable. A single agent run can span dozens of model calls. Without attribution by feature, tenant and workflow, you get an aggregate bill nobody can act on. With it, you can see that one feature consumes 60% of inference spend, or that a retry loop is quietly burning tokens. This connects directly to inference cost budgeting, which becomes tractable only once attribution exists.

Audit logs are the compliance-facing layer. Distinct from debugging traces in retention, immutability and access control: which agent, acting under which identity, took which action against which system, and did a human review it. Debugging traces can expire in weeks. Audit records may need to survive years and resist modification.

The signals worth alerting on

Not everything instrumented deserves a page. The signals that reliably indicate something is wrong with an agent are less obvious than the ones that indicate something is wrong with a service.

Unusual tool sequences are a strong indicator, because an agent calling tools in an order it has never used before is either encountering a novel situation or being manipulated. Step-count anomalies matter too: an agent that normally completes in four steps taking nineteen is stuck in a loop or confused. Sharp movement in evaluation scores between releases catches quality regressions that no test suite flagged. Retrieval quality degradation — context being pulled that does not match the query — is often the earliest symptom of an embedding or index problem, which we have written about in the context of embedding staleness.

Latency and error rate still belong on the dashboard. They are simply not where agent problems usually appear.

Where observability meets regulation

This is the part that changes the business case, and it is new in 2026.

The EU AI Act became applicable on 2 August 2026, and high-risk systems now carry enforceable expectations around record-keeping, traceability and demonstrable human oversight. Separately, agent security research found roughly half of enterprise agents running with no oversight or logging at all.

The instrumentation that satisfies an engineer debugging a bad agent run is largely the same instrumentation that satisfies an assessor asking how a decision was reached. Model version, inputs, actions taken, human review — the same fields serve both.

That means observability is no longer only an engineering quality investment. Built once, with retention and immutability considered, it discharges a meaningful share of regulatory evidence obligations. Built without that consideration, you will build it twice.

A pragmatic implementation order

Start with traces on your highest-consequence agent, capturing tool calls with arguments. This single step usually reveals more about actual agent behaviour than teams expect, and it is the prerequisite for everything else.

Add cost attribution next, because it is comparatively easy once traces exist and it almost always surfaces immediate savings that fund the rest of the work.

Build the evaluation set third. This is the highest-effort component because it requires curating representative cases with known good outcomes, and it is the one most often skipped. It is also what converts agent quality from an opinion into a number.

Formalise audit logging last, once you know which systems are high-risk and what retention they require. Doing this first, before classification, tends to produce over-retention of everything and under-protection of the records that matter.

Ortem Technologies builds production agent systems with tracing, evaluation and cost attribution instrumented from the first sprint rather than added after the first incident. If you are running agents you cannot currently inspect, see our cloud and DevOps services, our AI agent development work, or talk to our engineers.