A 30B open-weight model ties frontier detection on AAL-D-006
Run against the AAL-D-006 collateral eligibility and substitution set (250 cases, 3 runs, 0 errors), Muse Glimmer — Meta's 30B open-weight model, deployable locally on a single consumer GPU — outperforms DeepSeek V4 Pro on detection (95.7% vs 93.1%) and outperforms both Sonnet 5 (68.1%) and DeepSeek V4 Pro (66.9%) on substitution validity (78.6%). Its value accuracy of 16.3% sits within the 15.4% to 57.8% band the frontier models occupy on this dataset. Held against Spark 1.2 — the closed, frontier-tier version of the same model family — the pattern holds inside a single vendor's own lineup: Spark 1.2 scores higher across the board (100% detection, 92.6% substitution, 31.1% value) but hits an equivalent wall on value. Not published in the flagship comparison table: Glimmer's 30B parameter count makes a head-to-head against much larger frontier models a size-mismatched comparison.
This entry isolates the finding it actually supports.
Scale did not predict value accuracy here
A 30-billion-parameter model that can run inside a bank's own walls detects within the frontier range on D-006 and beats two frontier models on substitution. Its value accuracy (16.3%) is low, but so is the field's on this dataset: Grok 4.5 scores 15.4%. On D-006, a larger model did not mean a better number. Later datasets show value accuracy depends on the task: frontier models reach 66.5% to 88.1% on AAL-D-007, and two reach 100% on AAL-D-008 when conventions are stated.
The on-prem case
For regulated firms that can't send collateral schedules to a vendor API, Glimmer-class local models are a realistic path for the triage layer: detect, categorize, propose a substitute, all on hardware already inside the building, with an open-weight model anyone can independently reproduce results against. Valuation should still route to deterministic code: on this dataset no model, open or closed, computed the number reliably enough to book.
Methodology
Same AAL-D-006 dataset, prompts, and deterministic scorer as the flagship comparison — 250 cases, 3 runs, 0 errors, temp 0.7 / top-p 0.9. Not blended into the flagship table: comparing a 30B open-weight model against much larger frontier models on a single ranked table would overstate the significance of any gap in either direction.