Fork-style evals for long-running AI agents

Final success hides the bad turn

Your long-running agent can finish a task and still make the user distrust it. The final artifact looks fine. The path was ugly: it skipped a confirmation, rewrote a user-owned file, or chose a brittle shortcut that would fail under one more constraint. If your eval only checks the last state, you are certifying the destination while ignoring the route.

Teams measure pass rate, latency, and cost, then ship. Users feel the difference earlier. They notice when the agent asks for permission at the wrong moment, or when it commits to a plan that cannot be undone. The fix is to evaluate the decision, not just the outcome.

A useful receipt is Taste-Bench, a dataset built from release v3. The paper 'The Tasteful Agent: Measuring and Improving Taste in Long-Horizon Tasks' has Wenbo Pan and eight other authors. The arXiv paper 2609.25804 had version 1 submitted on 22 Sep 2026 and version 2 on 23 Sep 2026. The engineering subset has 390 items. The research subset has 112 items. The method is the part to steal: fork-style questions ask the model to choose between two plausible next moves in a real trajectory, then test whether the choice survives a reversed option order. Taste-Bench scores a fork question correct only if the model chooses the right option in both the original and reversed orderings, so chance performance is 25%.

A fork card turns a trace into a test

The card is small enough to review in a pull request and specific enough to catch a regression. It has one trajectory fork, the context shown, two options, the correct option, the evidence that decides it, and the reversed-order scoring rule.

• Mine one trajectory. Pick a completed run where the final output passed but the user would have felt friction. Retain the saved trace with timestamps, tool calls, user-visible messages, and the state before the risky move.
• Cut the fork at the decision point. Stop the trace at the moment the agent could have chosen differently. Keep only the context the agent actually had: user request, prior tool results, constraints, and visible state. Trim the prompt to a size your eval harness can run; Taste-Bench evaluation requests are limited to 65,536 input tokens.
• Write the two options as user-visible choices. Make both options plausible. One should be the better move; the other should be the tempting mistake. Do not make the correct option the longer, more cautious one every time. Users do not reward verbosity; they reward recoverable progress. The options should be short enough for a product designer to read without the trace.
• Record the evidence that decides it. Name the fact that makes one option right: a permission boundary, a rollback cost, a user preference, a tool limitation, or a safety constraint. The rationale should be one sentence a teammate can check against the trace.
• Add the reversed-order scoring rule. Attach the rule to the card so the test fails when the model merely prefers the first option, the second option, or the longer option.
• Run it as a regression eval before deploy. Add the card to the suite that runs on every model, prompt, or tool change. A red build should appear when a mid-run choice flips, even if the final task still succeeds.

Source the cards from user friction

The best cards come from moments where users would have paused, not from synthetic edge cases invented in a meeting. If a trajectory shows the agent choosing a path that saves time but destroys trust, that is the fork you need.

Keep the suite small and sharp. A handful of high-signal cards beats a large pile of vague success checks. When a card fails, do not patch the prompt blindly. Read the trace, find the missing constraint, and update the context the agent sees. Then rerun the fork card and ship only when the mid-run choice stays stable.