How to log AI benchmark scores so model comparisons survive an index change

You are choosing a model from a score table, and the number will change when the benchmark index changes. GPT-6 Astra launched on September 3 under Intelligence Index v4.1.1 at 61, tying GPT-5.6 Sol and trailing Claude Fable 5.1's 66 by five points. After v4.2, Claude Fable 5.1 scored 57 and GPT-6 Astra scored 55. Intelligence Index v4.3, published September 7, gave OpenAI's GPT-6 Astra and Anthropic's Claude Fable 5.1 the same 53-point score.

Artificial Analysis released Intelligence Index v4.2 on September 4, retiring GPQA-Diamond after saturation. The release added AA-Briefcase and GDP.pdf, the latter covering long-context reasoning across 4,592 PDF pages. Private test sets made up 20 percent of the Intelligence Index in v4.1, 40 percent in v4.2, and 45 percent in v4.3. Intelligence Index v4.3 upgraded Terminal-Bench from v2.1 to v4.0 and replaced τ³-Banking with AutomationBench-AA, a 657-task business workflow benchmark.

The score moved because the ruler moved. A model can look better, worse, or tied because the index changed, and a real regression can hide inside that change. A number-only table cannot tell which story is true.

Fix the ruler before you compare the models

Benchmark versioning is the missing field in most model selection evals. The score is the reading; the index version is the ruler. Treat the ruler as part of the measurement. Rulers also change when benchmarks saturate. Claude 2 solved just 1.96% of SWE-bench's real GitHub issues in late 2023, while leading research systems and agents reached 71.7% by 2024; the benchmark community then added tougher versions, including a human-verified subset and a Pro tier.

Log the score with its provenance

The fix is small: store the score as a tuple, not a scalar. A tuple forces provenance.

• Record the index version with every score. The row names the index version before the number.
• Freeze the model version and the date. The row identifies the build and the test date.
• Note test-mix changes. The field lists what was added, removed, or reweighted.
• Compare only same-index scores. Rows from other index versions stay hidden.
• Label cross-version differences as index changes.

Keep the fields together in the same row. When the score moves to a new slide, the provenance moves with it, not left behind. Teams often paste a leaderboard number into a slide without the index version. That turns a model comparison into a ruler comparison. Put the version in the same cell as the score, or in the row header, so it cannot be dropped in review. When a stakeholder asks why the number moved, the answer should name the cause before anyone argues about quality: the model changed, the index changed, or the test mix changed.

When the index changes, say so

Do not explain a score drop as model decay until you have ruled out the index. New versions that add harder tests, remove saturated tests, or shift private-test weight move the number for a structural reason. Say that in the same sentence where you cite the score. Good notes name the direction and the cause, not a vague model improvement.

TruthInsightBench uses a fixed LLM-based judge to score six dimensions via 29 artifact-grounded items, and its automated deterministic aggregation without per-instance human grading allows the evaluation to be rerun as agents evolve. In the TruthInsightBench baseline, Claude Code v2.1.220, Codex CLI v0.149.0, OpenScience v2.0.1, and DeepSeek Harness v0.1.0rc7 were run on one frozen base model across the same 40 tasks and protocol, and their total scores ranged from 58.4 to 60.3 out of 100 without reliable pairwise differences. Fixed protocols make scores auditable. Without rerun capability, treat the score as a snapshot. Mark a comparison provisional in the launch doc and date it clearly when the change cannot be named.