Benchmark compares Qwen3.8-27B inference providers at 1–64 concurrency

🚀 RELEASES & LAUNCHES

• UPDATE: Benchmark compares Qwen3.8-27B inference providers at 1–64 concurrency
  A benchmark compared Qwen3.8-27B inference throughput and latency across five serving setups.
  • The comparison targets teams serving Qwen3.8-27B through hosted providers or self-managed vLLM.
  • Together, Fireworks, Doubleword, vanilla vLLM, and g factor were tested on 2× H100 80GB GPUs at concurrency levels from 1 to 64.
  • Together reached 189.61 tokens per second at concurrency 1 and 2,701.47 at concurrency 64; g factor with MTP4 reached 133.00 and 2,611.88.
  • Fireworks throughput stayed nearly flat from 1,146 tokens per second at concurrency 32 to 1,145 at 64, while p95 TTFT rose from 2.69 to 7.71 seconds.
  • The measurements compare tensor parallelism, data parallelism, speculative decoding, and queueing trade-offs for deployment decisions.
  Sources 📄 Source: g-ftech.com · 📎 Coverage: dev.to · 👁 via r/LocalLLaMA

📊 BENCHMARKS & EVALS

• Thinkingbox benchmarks agent reliability in stateful business workflows
  Thinkingbox introduced a benchmark for evaluating agents in stateful business workflows.
  • For developers evaluating agents that handle multi-turn business workflows.
  • Thinkingbox provides a sandboxed benchmark testing state persistence and policy adherence.
  • The benchmark measures reliability beyond single successful runs and typical code or web-agent tests.
  Sources 📎 Coverage: huggingface.co · 👁 via Hugging Face Blog