Margin Call Dispute Detection — Dataset
AAL-D-002 is the second benchmark dataset in the AI Alpha Labs evaluation suite. Where AAL-D-001 tested trade confirmation exception identification, AAL-D-002 tests a downstream, mathematically heavier workflow: margin call dispute processing. Each case presents a counterparty margin call notice alongside an internal calculation. Models must determine whether a dispute exists, classify it within a 17-category taxonomy, compute the correct call amount, recommend an action, determine whether the undisputed amount should be paid, and flag whether escalation is required.
The dataset covers bilateral OTC CSA portfolios, exchange-cleared positions, and securities lending agreements.
Coverage
Six instrument types: interest rate swaps, FX forwards and NDFs, cross-currency swaps, listed futures, equity total return swaps, credit default swaps, and securities lending. Seventeen dispute categories including DIS-PRICE, DIS-THRESH, DIS-HAIRCUT, DIS-ELGBLTY, DIS-FX, DIS-NETTING, DIS-SIMM, DIS-STALE, DIS-DUPE, DIS-SETTLED, DIS-DIR, and DIS-CSA. Difficulty ranges from easy (clean matching calls, simple single-category disputes) to complex (multi-leg portfolios, direction inversions, collateral eligibility violations, SIMM version mismatches).
Workflow complexity
AAL-D-002 is materially harder than AAL-D-001. Correct handling requires applying MTA and rounding conventions, computing threshold-adjusted exposure, applying haircuts to collateral, reconciling netting sets, determining undisputed-payment obligations, and recognizing duplicate or stale calls. The arithmetic burden is substantially higher — this is deliberate, to test whether the arithmetic weakness confirmed in AAL-D-001 persists when math is the primary deliverable.
Scoring
Each case specifies per-field scoring criteria: dispute detection (binary), category identification, field identification, correct call amount (numeric match within tolerance), disputed amount, undisputed payment flag, and escalation requirement. Tolerances are tiered by call size: ±$1k for calls under $500k, ±$5k for calls $500k–$5M, ±$10k for calls above $5M.
Sample case preview
Case AAL-D-002-017 · Bilateral OTC IRS Portfolio · Independent Amount Dispute. Counterparty margin call notice: $5,200,000.00 USD. Internal calculation: $4,850,000.00 USD. Dispute amount: $350,000.00. The counterparty applied a 10% independent-amount factor to the entire notional of $350,000,000; the CSA limits the IA to the in-the-money subset of $180,000,000. Ground truth flags dispute category DIS-IA-METHODOLOGY (Independent Amount calculation), computes correct call amount as $4,850,000.00, recommends action PAY-UNDISPUTED, sets undisputed amount as $4,500,000.00, and flags escalation-required as false. Scoring criteria: dispute category must be exact among 17 categories; correct call amount must match within $1,000.00; undisputed amount must match within $500.00; action and escalation boolean must be exact.
v1.0.1 correction: case 250 named the dataset
On 2 Oct 2026 we found that case AAL-D-002-250 in v1.0 had two sentences in its context that named the dataset: one called its portfolio the largest in the dataset, the other called it the final case in the v1.0 dataset. Those sentences could tell a model it was being tested. Version 1.0.1 removes both. It is the only case that differs, and its ground truth is unchanged. The published D-002 results were produced on v1.0. The stored GPT-4o results include case 250, and GPT-4o answered it correctly on all three runs. Excluding the case moves GPT-4o's amount accuracy from 76.08% to 75.88% and changes no other GPT-4o metric by more than 0.2 percentage points. We did not recompute figures for the other models. A scan of the dataset files found no other reference to the benchmark name in the case text shown to models, apart from the case identifiers in AAL-D-008, which are disclosed on that page. The scan script, check_prompt_leakage.py, is in the public repository.
AI assistance in construction
Cases and ground truth for this dataset were built by the author with AI assistance. Which model assisted was not recorded at the time, so we cannot say whether it belongs to a model family evaluated here. Where the project's commit records name an assistant, it is a Claude model, and Claude models are among those we evaluate. Scoring is deterministic code that no model influences. The exposure is in how the answers were built, not in how they are graded.
How to cite
BibTeX: @dataset{aal_d_002_2026, title={AAL-D-002: Margin Call Dispute Identification Dataset}, author={{AI Alpha Labs}}, year={2026}, month={jul}, version={1.0}, publisher={AI Alpha Labs}, url={https://aialphalabs.ai/research/AAL-D-002}, note={250 cases across 6 asset classes with 17-category dispute taxonomy}}. APA style: AI Alpha Labs. (2026). AAL-D-002: Margin Call Dispute Identification Dataset (Version 1.0). Retrieved from https://aialphalabs.ai/research/AAL-D-002. MLA style: AI Alpha Labs. AAL-D-002: Margin Call Dispute Identification Dataset, version 1.0, July 2026, aialphalabs.ai/research/AAL-D-002.
Related work — ISDA's own extraction benchmark
ISDA published its own AI benchmark on CSA clause extraction in May 2025, testing eight models on pulling thresholds, minimum transfer amounts, and rounding terms from 60 CSAs. ISDA reported an accuracy figure above 90% (its own self-reported result) but measured extraction only — it never asked a model to calculate a margin call amount from the extracted terms. That is the exact next step AAL-D-002 tests: given equivalent contract terms, can a model compute the correct call amount, not just locate the terms that determine it. ISDA's own scope, by its own design, stops short of the arithmetic question this dataset is built to answer.