📊 BENCHMARKS & EVALS
- llama.cpp fork boosts Qwen3.8-27B inference on dual Tesla P100s
A llama.cpp fork speeds Qwen3.8-27B inference on two Tesla P100 GPUs.- Targets local-LLM users running Qwen3.8-27B on two Tesla P100-PCIE-16GB cards.
- The fork supports Q6_K weights with a q4_0 KV cache and reports 1.75× faster decode and 1.85× faster prefill than upstream.
- A Reddit test reported about 60 tokens per second using two P100s capped at 175W each.
- The optimization enables higher-throughput inference on older, low-cost Pascal GPUs without reported accuracy loss.
- The project is a personal, unreviewed fork with some changes specific to Qwen3.8-27B’s model shape.