📊 BENCHMARKS & EVALS
- Nonobench v1.2 tests 43 LLMs on nonogram puzzles
Nonobench v1.2 found GPT-6 Astra was perfect on 15×15 nonograms while open models failed 20×20 Hard mode.- The benchmark is for developers comparing private and open-weight LLM reasoning on nonogram puzzles.
- GPT-6 Astra scored 30/30 on 15×15 puzzles, while DeepSeek V4 Pro was the top open-weight model at 83%.
- On ten random 20×20 Hard puzzles, Opus 5.5 solved 8/10 and every tested open-weight model scored 0/10.
- The benchmark publishes every prompt and output but remains OpenRouter-only, with no local execution available.
Sources 📎 Coverage: reddit.com · 👁 via r/LocalLLaMA