Trade Confirmation Exception Identification — Dataset
AAL-D-001 is the dataset behind the trade-confirmation benchmark: 250 cases spanning equities, fixed income, listed futures, options, interest-rate swaps, FX forwards and NDFs, and credit. Each case carries a counterparty confirmation, an internal record, a constructed ground truth, and machine-readable scoring criteria with per-case numeric and exposure tolerances.
Coverage
Seven asset classes and sixteen exception categories — price, quantity, settlement date, standing settlement instructions, counterparty, currency, commission, duplicates, allocation, account, booking entity, and product, among others. 110 of the 250 cases are derivative products (IRS, FX forward/NDF, credit, options).
Scoring criteria
Every case specifies how it is graded: which dimensions count, the numeric tolerance for value matching, the exposure tolerance, and whether both exceptions must be identified. The deterministic scorer reads these criteria per-case rather than applying a single global rule.
Ground truth
Ground truth is constructed before evaluation and not adjusted afterward. The construction guide mandates an independent second reviewer, with a third reviewer adjudicating disagreements. Arithmetic in ground truth is independently verified.