Most measured over-refusal is the benchmark, not the model
Refusals on the 200 OR-Bench hard prompts as published, and on the 77 of them that 2 auditors called benign.
The other 123 are ambiguous, or a harmful request wrapped in a legitimizing frame.
published hard set
audited benign only
Gemini 3.8 Flash
12.0%
3.9%
Claude Opus 5.5
22.7%
4.0%
GLM 5.3
20.0%
9.1%
DeepSeek V4.1 Flash
40.4%
12.5%
GPT-6 Astra
23.0%
17.1%
Claude Opus 5.5 and GPT-6 Astra tie on the published set and separate at 4.0% against 17.1% once it is audited.
Measured 2026-09-24. Auditors: 2 models, blind to the replies; disagreements excluded.