Blog · AI engineering

AI coding agents need a logging contract, not a better prompt.

An AI coding agent can make a change compile, pass the tests and still leave the system harder to operate. Production needs more than generated code. It needs evidence that follows a contract.

Die-cut sticker: an engineer inspects structured event cards moving from a coding robot through a validation gate to an observability console

Production judges the change last#

An AI coding agent can make a change compile, pass the tests and still leave the system harder to operate. The code ships. The next incident arrives. The only new log says that something went wrong.

I do not consider that change complete.

As I argued in Faster coding is not faster delivery, accelerating implementation does not automatically accelerate the complete software delivery lifecycle. Logging is a good example. Code generation gets faster, but someone still has to make the result understandable in production.

That requirement cannot remain an informal preference shared between experienced developers. It needs a contract.

Humans are quietly repairing agent-generated logs#

A 2026 empirical preprint examined 4,550 agentic pull requests across 81 open-source repositories. In the 77 repositories with comparable data, agents changed logging less often than humans in 58.4 percent.

Among the 1,308 agentic pull requests where at least one studied instruction channel was observable, only 61, or 4.7 percent, contained explicit logging instructions. The authors report that agents failed to comply with constructive logging requests in 67 percent of the analysed cases. Humans performed 72.5 percent of the post-generation logging-statement modifications.

These numbers need context. This is a preprint, not a universal benchmark. It covers a specific set of open-source repositories, agents and workflows. It does not prove that every coding agent handles logging badly.

The paper also found similar log-message lengths for agents and humans, while the apparent density difference largely narrowed after controlling for pull request size. It does not support the idea that agents always under-log. It does identify a familiar failure mode: an agent can satisfy the visible task while missing the operational requirement.

Use an event code and a clear message#

I want every important log record to contain four elements: a stable event code or event name, a short technical message, structured fields that describe the event, and trace context that connects it to the wider operation.

The event code and the message have different purposes. The code gives dashboards, alerts and queries a stable identifier. The message tells a human what happened without requiring a catalogue lookup. Using only text makes automation depend on wording. Using only a code makes incident analysis unnecessarily slow. Use both.

Structured fields must add context without exposing credentials, tokens, email addresses or unnecessary personal data. The OpenTelemetry Logs Data Model defines event name, severity, trace ID, body and attributes. The OWASP Logging Cheat Sheet sets the security boundary. I prefer existing models to a new convention per project.

Give each log level one meaning#

A logging contract also needs stable severity semantics.

01

DEBUG

Diagnostic detail for development or targeted investigation.

02

INFO

A significant lifecycle event or state transition completed normally.

03

WARN

An unexpected or degraded condition occurred, but the operation can continue.

04

ERROR

The current operation failed and requires recovery, retry or human attention.

A caught exception is not automatically an error. If the application handles it and completes the requested operation normally, ERROR can create a false incident signal. The reverse is also true. A failed business operation must not disappear into DEBUG because the application remained alive.

AGENTS.md is guidance, not enforcement#

An AGENTS.md file is still useful. OpenAI describes it as a place for repository-specific commands, conventions, review expectations and constraints. It gives every coding session the same starting point.

I would keep the logging section short: required structure, level semantics, sensitive data, approved examples and the verification command. Not a twenty-page manual.

More importantly, repository guidance cannot prove that generated code follows the rule. The 2026 study gives us a practical warning: natural-language instructions alone are unreliable for this non-functional requirement.

Move repeatable rules into CI#

Anything that can be verified deterministically should leave the prompt and enter the toolchain.

A shared logging API can require an event code and structured fields. A controlled catalogue or enum can prevent duplicate and invalid codes. Static analysis can reject System.out, printStackTrace, prohibited field names and unsupported logging patterns. Tests can verify that critical workflows emit the expected event code and outcome without asserting the complete human message.

CI should run these checks on human and agent-generated changes. The standard must not depend on who produced the pull request.

Automated controls cannot decide whether a business event deserves a log or whether the selected context will help during an incident. That remains engineering judgment. The controls remove mechanical defects so reviewers can focus on those decisions.

When the same logging correction appears twice in review, I treat it as a process defect. The next step is to improve the contract, example or automated check.

Build evidence, not volume#

Good logging is not about producing more records. It is about leaving enough stable evidence to reconstruct what happened without guessing.

That principle also supports the production workflow I described in Claude in the enterprise. An agent can investigate an incident only if the system gives it reliable facts to correlate. Unstructured noise does not become useful context because a model can read it.

The model can draft the logging call. The contract defines its shape. CI verifies the parts that a machine can verify. A human still owns why the event exists and whether it will help when the system fails.

That arrangement will survive the next model, the next incident and the next engineer.