P&L Reconciliation Break Attribution
AAL-D-005 measures whether frontier models can adjudicate the daily P&L reconciliation every trading desk runs: front office reports a number, product control independently computes another, a risk explain accounts for part of the gap, and the model must decide whether the residual is a real break or a fully-explained move — then attribute the break to a root cause. The dataset is 250 cases (162 breaks across 15 BRK-* categories, 88 explained-move traps) evaluated across six models — Claude Sonnet 5, Claude Sonnet 4.6, GPT-4o, Gemini 2.5 Pro, DeepSeek V3.2, and Kimi K3 — three runs each, deterministic scoring, no model permitted to perform arithmetic. Results pending a results file (2026-10-12): no D-005 results file has been found, so the model results on this page have been removed until it is located.
Benchmark design: breaks vs. explained-move traps
Each case presents three views of a day's P&L — front-office estimate, product-control books, and a risk explain — and prints the unexplained residual and the materiality threshold. 162 cases are real breaks spanning 15 root causes (stale marks, missing or new trades, amendments, sign errors, FX, fees, corporate actions, day-count, sensitivities that do not reconcile, and more). 88 are explained-move traps: large P&L swings that look like breaks but are fully accounted for (a trade booked today, a dividend already applied, a residual below the materiality threshold, a known T+1 timing adjustment). The model reads, attributes, and recommends; it is forbidden from computing, so all arithmetic is generator-side and ground-truth by construction.
Detection is solved; quantification is the wall
Results pending a results file (2026-10-12): no D-005 results file has been found, so the model results on this page have been removed until it is located.
The finding: 'reasoning model' does not predict performance
Results pending a results file (2026-10-12): no D-005 results file has been found, so the model results on this page have been removed until it is located.
Methodology
Cases are generated deterministically; ground truth is established by independent Python repricing with a full audit trail (every reported residual reconciles under a separately-written recompute check before anything ships). Six models, three runs each at vendor-recommended sampling (reasoning models at their specified temperature/top-p, not forced to temperature 0), deterministic scoring on detection, false-break rate, explained-move accuracy, category attribution, field identification, value extraction, escalation, resolution action, and break owner. Scorer locked at d005-score-v1.0.1. A pre-publication gate independently re-runs the dataset recompute, the scorer contract tests, and per-model error/null checks.
The deployment bottleneck
Results pending a results file (2026-10-12): no D-005 results file has been found, so the model results on this page have been removed until it is located.
AI assistance in construction
Cases and ground truth for this dataset were built by the author with AI assistance. Which model assisted was not recorded at the time, so we cannot say whether it belongs to a model family evaluated here. Where the project's commit records name an assistant, it is a Claude model, and Claude models are among those we evaluate. Scoring is deterministic code that no model influences. The exposure is in how the answers were built, not in how they are graded.
How to cite
BibTeX: @dataset{aal_d_005_2026, title={AAL-D-005: P&L Reconciliation Break Attribution Dataset}, author={{AI Alpha Labs}}, year={2026}, month={aug}, version={1.0}, publisher={AI Alpha Labs}, url={https://aialphalabs.ai/research/AAL-D-005}, note={250 cases, 162 breaks across 15 categories and 88 explained-move traps}}. APA style: AI Alpha Labs. (2026). AAL-D-005: P&L Reconciliation Break Attribution Dataset (Version 1.0). Retrieved from https://aialphalabs.ai/research/AAL-D-005.
Reproducibility status
The evaluation scripts and raw model outputs for D-005 are not in the public repository and are not currently available to us, so the prompt construction for this dataset cannot be independently verified. The dataset and its generators are published, and a scan of the case input fields found no reference to the benchmark name. The figures on this page are as originally recorded.