HFlow evaluation finds open-weight VLMs near Gemini on egocentric data

📊 BENCHMARKS & EVALS

• HFlow evaluation finds open-weight VLMs near Gemini on egocentric data
  HFlow found Gemma 4 26B-A4B matched Gemini 2.5 Flash on egocentric evaluation at 19x lower cost.

  • For builders processing large-scale first-person video, including privacy-sensitive workloads.
  • Using Build AI’s Egocentric-10K dataset and prompts, Gemma 4 26B-A4B agreed with Gemini 2.5 Flash’s results 90.87% of the time versus the 91.65% baseline.
  • GLM 5.3 Flash scored 91.00%, Qwen 3.8 27B 90.79%, and Inkling Small 85.21%.
  • Gemma 4 26B-A4B was reported as 19x cheaper than Gemini 2.5 Flash; Gemma and Qwen can be self-hosted without data egress.
  Sources 📄 Source: github.com · 📎 Coverage: reddit.com · 👁 via r/LocalLLaMA

• Reddit user proposes an Agentic Coding Index for model intelligence density
  A Reddit user proposed an Agentic Coding Index to measure coding capability per model parameter.

  • The analysis targets developers comparing LLMs for autonomous coding tasks.
  • It aggregates SWE-bench Pro, DeepSWE v1.1, Terminal-Bench, Code Arena Elo, and LiveCodeBench v6.
  • Its density formula uses a 50-point neutral baseline, a 2.5354 scale, and an 8-billion-parameter lower bound.
  • The index adjusts agentic coding performance for model size instead of ranking models by benchmark scores alone.
  Sources 📎 Coverage: reddit.com · 👁 via r/LocalLLaMA