🚀 RELEASES & LAUNCHES
Xolotl inference engine runs 552B DeepSeek-V4.1-Flash on a single RTX 4090 at ~27 tok/s
A developer built Xolotl, a Strata-based inference engine that runs the full 552B DeepSeek-V4.1-Flash on one RTX 4090 at ~27 tok/s decode.- Aimed at local-LLaMA hobbyists and developers running large MoE models on consumer hardware.
- Xolotl spreads weights across 24GB VRAM, 90GB pinned DDR4 RAM, and NVMe, loading only the experts each token needs.
- Reported results: 27 tok/s decode, 180-320 tok/s prefill, 512K context, 21/21 HumanEval, 10/10 tool calls, 3/3 agentic bug-fix loops.
- Also runs GLM-5.3-Flash, Qwen3.8-Flash-Next, and Qwen3.8-27B; an uncensored 1017 quant (152 GiB) of V4.1 is on Hugging Face.
- Matters because it shows frontier-scale MoE models can run on a single gaming PC, not just multi-GPU servers.
Red Curtain runs Fish Audio's Drama 3 TTS on a 16 GB AMD Radeon, no 48 GB NVIDIA card needed
A developer built Red Curtain, a tool that runs Fish Audio's Drama 3 speech model on a 16 GB AMD Radeon RX 9070 despite its official 48 GB NVIDIA requirement.- Aimed at local AI hobbyists and AMD GPU owners who want to run Fish Audio's Drama 3 text-to-speech model without high-end NVIDIA hardware.
- Drama 3 is a 35B-parameter MoE model with ~3B active parameters per token; Red Curtain keeps the ~32 GB of FP8 experts in system RAM and caches only used experts on the GPU, running the rest in BF16 on the card.
- On the author's RX 9070 it uses ~11-12 GB VRAM and ~35 GB RAM, loads in ~17 s, and generates 3.3 s of speech in ~13 s (11.6 s in ~37 s) — slower than realtime but usable for short takes.
- Ships with a small web UI supporting voice cloning from a reference clip, multi-speaker scripts, and Fish's editor tags like [laughing], [sighing], and .
- Shows Drama 3 can run on consumer AMD cards via expert offloading, though a known issue can cause takes to go silent after the first line.