Gemini 2.5 Flash on AAL-D-001
The fourth complete model evaluation on AAL-D-001. Gemini 2.5 Flash was run three times across all 250 cases using the same multi-run methodology as the Pro evaluation. Flash matches Pro on detection, classification, and dual-exception recall while demonstrating higher escalation accuracy in v1.0 — a difference subsequently traced to prompt specification rather than model capability. In v1.1 both models converge to near-identical performance across all metrics.
Protocol
Three independent runs per case across all 250 cases of AAL-D-001 (750 observations, 0 errors). Deterministic scorer as in all prior evaluations. The detection interval is a Wilson 95% interval computed by case (the 3 runs of each case grouped). Source: aal-eval-platform eval_out_v2/eval_results_gemini_gemini-2.5-flash.json.
Correction (Oct 12, 2026)
Previously shown: 99.2% detection (98.3–99.6), which could not be traced to a results file. Replaced with the full v2 run (eval_out_v2/eval_results_gemini_gemini-2.5-flash.json): 746/750 = 99.5% detection, 95% interval 97.6–99.9% by case. The other metrics on this page were also replaced with the v2 figures, and figures that could not be traced to a results file were removed.
Flash vs Pro on the v2 runs
On the same 250 cases (eval_out_v2 scorecards), Flash and Pro both detect 746/750 (99.5%). Escalation accuracy is 98.1% for Flash and 99.0% for Pro; exposure accuracy is 79.5% for Flash and 76.0% for Pro (rescored 2026-10-12 on the corrected key; for the later runs, the prompt in our run scripts contained the same unit error, and these runs' answers are consistent with it, so 7 cases removed, 57 cases); false-positive rate is 0.0% (0/228) for Flash and 0.9% (2/228) for Pro.
Practical implication
On these runs Flash's accuracy is close to Pro's on every metric listed above. Exposure accuracy is 79.5% for Flash and 76.0% for Pro (rescored 2026-10-12 on the corrected key; for the later runs, the prompt in our run scripts contained the same unit error, and these runs' answers are consistent with it, so 7 cases removed, 57 cases).