Airbench leaderboard compares AI models and agent harnesses

🚀 RELEASES & LAUNCHES

• DecisionTune 1.0 brings 395M offline model for bounded decisions
  DecisionTune released a 395M Apache-2.0 encoder for local option selection and yes/no decisions.
  • For developers building local agents, routing, triage, and tool-selection workflows.
  • The ModernBERT-large model uses a 4 KB scoring head to select options or return P(yes) without generating text.
  • It scores 29.57 on Decision Index 0.2.1 and runs in a reported 25.9 ms median per request.
  • Weights are 1.58 GB and support PyTorch, MLX on Apple silicon, and ONNX for offline execution.
  • The model gives local applications probability outputs without a network trip or generative-model parsing.
  Sources 📄 Source: github.com · 📎 Coverage: huggingface.co · 👁 via r/LocalLLaMA

📊 BENCHMARKS & EVALS

• Airbench leaderboard compares AI models and agent harnesses on real tasks
  Airbench published a leaderboard comparing models and agent harnesses across 49 tasks.
  • For local AI builders choosing models, runtimes, and agent harnesses.
  • The benchmark covers 49 challenges spanning math, vision, coding, and computer use, including simulated stores and email.
  • Claude Code Opus 5.5 scored 100% in 14m26s; a local GLM-5.3 Flash setup matched 100% in 31m13s.
  • Qwen3.8 Flash Next reached 96% in 7m41s with opencode and 94% in 11m31s with omp.
  • The leaderboard compares task success and speed across model-and-harness combinations and lets users test their own setups.
  Sources 📄 Source: airbench.ai · 📎 Coverage: reddit.com · 👁 via r/LocalLLaMA

🔬 RESEARCH

• 📄 Source for vLLM 0.31.0 adds GPU weight caching and serving upgrades — github.com