📊 BENCHMARKS & EVALS
- Halogen 0.12.0 runs Qwen3.8-Flash-Next at 1M context on Strix Halo
Halogen 0.12.0 enables Qwen3.8-Flash-Next inference at 1M context on Strix Halo.- For Qwen3.8-Flash-Next users running AMD Strix Halo systems, including the Ryzen AI Max+ 395 with 128 GB RAM.
- Halogen 0.12.0 serves the model through an OpenAI-compatible endpoint with a 1,048,576-token context configuration.
- At about 1,004,581 tokens, reported performance reached 38.3 tok/s decode and 937 tok/s prefill, with a 17.9-minute cold prefill.
- The runtime reports 23.0 seconds of prefill and 6.1 seconds of decode for a 32K prompt, or 29.1 seconds total, using 5.53 bits per weight across 179.55B parameters.