Posts tagged benchmarks

2 posts about benchmarks.

• Over-Refusal Measured: Most of It Is the Benchmark, Not the Model

  September 28, 2026 · over-refusal · safety · benchmarks · evals

  We ran XSTest and OR-Bench over ten models, then audited the prompts. Only 77 of 200 hard ones were clearly benign, and Opus 5.5 drops from 23% refused to 4%.

• Jev vs Flash LLMs: 7x Cheaper on Workflows, Same Cost on One Label

  September 22, 2026 · jev · classification · llm-selection · benchmarks

  TypeSafe's Jev vs 5 flash LLMs, measured: 5-37x cheaper and 11-27x faster on 12-question workflows, same cost as Qwen or GLM on one label, never most accurate.