Arithmetic ceiling confirmed across two frontier models on AAL-D-002
AAL-F-006 confirmed the arithmetic ceiling for GPT-4o across two datasets. AAL-RS-008 now extends the finding to Claude Sonnet 4.6 on AAL-D-002: 80.4% amount accuracy against 99.3% detection — a 19-point gap on the same workflow. The ceiling is now confirmed across two models, two labs, and two datasets. Neither prompt design nor model family closes the gap. The architectural implication is structural: arithmetic must be routed to deterministic code; the LLM layer handles detection, classification, and extraction.
What the two-model comparison shows
Claude Sonnet edges GPT-4o on every arithmetic metric, but both sit in the same 76–80% band while both exceed 99% on detection. The ceiling is not a GPT-4o artifact, not a prompt artifact, and not a dataset artifact. It is a consistent structural property of autoregressive text generation applied to multi-step financial arithmetic chains.
Zero false positives: the other joint finding
Across 1,128 combined clean-call runs from both models, neither produced a false positive. This is not a trivial result on a 17-category taxonomy with complex threshold and netting logic. Both frontier models can be trusted not to generate spurious dispute flags on clean margin calls — which is often the practical gate for production deployment.
Design implication
Hybrid deterministic/LLM architecture — routing classification to the model, arithmetic to code — is now empirically mandated across both evaluated models. A pure-LLM pipeline leaves ~20 points of amount accuracy on the table and adds unnecessary token cost. Full cost treatment: AAL-RS-009 (forthcoming).