Why Your Agent Logs Should Pre-Date the LLM Call

Most agent logs start at the LLM response and miss the most important data: what the agent decided to send. The pre-call log line, with rationale, and how to use it to debug regressions.

The pre-call gap most logs have

Most agent logs start at the LLM response: "the model said X." The data the model used to say X is lost.

When the model says something wrong, you cannot debug without that data. The operator stares at a wrong answer with no context.

Pre-call logging fills the gap. Log what was sent before what comes back. The five extra log lines per run are worth their weight.

What to log pre-call

The full prompt: system, messages, tool definitions. The size if too large to log raw; hash the body and log key fields.

The model name and version. "GPT-4o" is not enough; include the date version.

The tools available at call time. The agent's available tools change per run; the snapshot matters.

The agent's working memory snapshot. The structured state the agent had assembled before the call.

What to log post-call

The full response: text, tool calls, structured output if applicable.

Token counts (input/output/cache).

Latency from request-sent to response-received.

Any errors or partial responses.

Why this matters when something breaks

Bug type 1: the model got it wrong. Pre-call log shows the prompt was correct; the bug is the model. Switch model or prompt-engineer.

Bug type 2: the prompt was missing context. Pre-call log shows the missing field; the bug is in the agent's working-memory assembly. Fix upstream.

Bug type 3: the prompt was correct but malformed. Pre-call log shows the malformation; the bug is in the prompt template. Fix the template.

Managing log size

Full prompts are large. Log raw for the first 1% of runs (debug sample); hash the rest with a body-stored-elsewhere reference.

The body store is cheap object storage. Pull on demand when debugging. Day-to-day operations do not pay the size cost.

Retention: hot logs 7 days, body store 30 days. Past 30 days, only the hash and key fields remain.