Design evals that catch agent steps, not outputs

You are about to ship an agent that can search, edit, call tools, or move money. If its value depends on the path it takes, your evals must score that path, not only the final answer.

Most agent evals still ask only whether the answer arrived. For a chatbot, that can be enough. For an agent that acts across systems, the path is the product. A correct answer produced by a forbidden step is a compliance failure.

A passing output can hide a forbidden step

In the django-11555 task, a coding agent delivered a successful fix after using a prohibited grep operation, yet the output metric passed the run while the trajectory metric failed it. The user did not receive a wrong patch. The user received a patch made by an agent that broke the rule it was supposed to follow.

When the rule limits the agent to an allowed search path, the violation is a routing problem. The trajectory is where the product's safety claim lives. A routing violation is also a trust violation. If your product promise is that the agent stays inside a safe retrieval path, output-only checks will miss the exact behavior that makes users distrust it. The final answer can look polished. The logs can show the agent used a forbidden path.

Designers often treat the trajectory as an implementation detail. They miss the interface between the model and the system. Every tool call, file read, search query, and state change is a decision the user cannot see but can feel. If the agent reads a file it should not read, or calls an endpoint it should not call, the user may not know why the answer feels off. The eval should catch it before the user does.

Stable output scores can hide unstable behavior

The evaluation used 30 Django tasks from the same era of the codebase. In that comparison, plain-prompt enforcement satisfied the vector-search-only rule 77% of the time; tool-flag enforcement satisfied it 60% of the time. If you track only the final answer, you will not see which enforcement mechanism is doing the work.

Across the retrieval-augmented generation variants, test_passed stayed in the 70% to 77% band, while vector-search-only adherence ranged from 60% to 100%. Output quality can look stable while behavior swings wildly. Designing for the average ships the exceptions.

Keep behavior scoring separate from output quality. Do not average a compliance score into a quality score. A dashboard that blends the two scores will show a healthy average and a broken rule. One axis asks whether the answer is useful. The other asks whether the agent stayed inside the rules that make the product safe, auditable, and supportable. When the axes diverge, use that divergence to decide what to fix before release.

Two scorers can share a blind spot

Deterministic checks can be brittle. They can misread a log, overcount a leak, or flag a harmless step. A check can be wrong, but a wrong check is visible and fixable. The first deterministic leak check overcounted leaks by a factor of four, and the LLM judge made the same mistake. Agreement between two scorers can be shared blindness.

An agent metric measures, in numbers or judgments, how well an agent met its target; a scorer is a code-based or LLM-judge routine that calculates it. If the metric asks whether the agent stayed within the allowed tools, the scorer has to inspect the trajectory, not just the final answer. If the metric asks whether the answer met the required behavior, a model judge may help, but a deterministic scorer still owns the forbidden steps.

Audit the scorer before you trust the score. Run a small set of known-good and known-bad trajectories. Check that the deterministic scorer flags the bad ones and ignores the harmless ones. Check that the model judge does not inherit the same blind spot. Fix the scorer before you act on the score.

Score the rule, then fail the run

The redesign is a trajectory eval rule. List the steps that must happen, the steps that must not happen, and attach a deterministic check to each one. If any check fails, the run fails, even when the output passes.

• Write the rule as a step: the agent must use the vector-search-only path and must not use the prohibited grep command.
• Attach a deterministic scorer to each step, checking the actual tool calls, file accesses, or API events.
• Fail the run on any violation, then report the output score and the behavior score separately.

When the output passes but the trajectory fails, block the run, repair the step, and rerun the eval.