🔬 RESEARCH
- T-Tech researchers present LIMe, cutting LLM training compute ~15% without quality loss
Researchers from T-Tech presented LIMe (Layer-Integrated Memory) at EMNLP 2026, an attention extension that trains ~1B-parameter models with 15% less compute at equal loss.- Aimed at LLM builders and researchers training Transformer models, especially deep architectures.
- LIMe adds learnable per-head, per-layer routing weights so attention can reuse key-value representations from earlier layers, not just the previous one, with no change to hidden state size.
- In ~1B-parameter experiments, LIMe matched baseline LLaMA training loss with 15% less compute and beat competing approaches on language modeling and multi-step reasoning benchmarks, with largest gains in synthetic arithmetic and deeper models.
- The authors attribute the gains to mitigating representation collapse: LIMe yields higher value-vector entropy and better token separability.
- Paper at arXiv (2502.09245) and official code on GitHub; commenters note the paper is ~2 years old and untested at larger scales.
- It matters because it offers a lightweight architecture change that could cut training costs, though independent testing showed per-step overhead that may offset gains in practice.
📊 BENCHMARKS & EVALS
- Community prunes NVIDIA's 30B Nemotron speech model from 67.5GB to 10GB with no retraining
A Reddit user pruned NVIDIA's new 30B speech LLM from 67.5GB to 10GB with zero retraining, keeping transcription quality close to the original.- The pruned model targets local/self-hosted users running NVIDIA's Nemotron 3.5 Transcribe 30B MoE speech model (about 4B active parameters) on consumer hardware.
- Pruning shrank the model from 67.5GB to 10GB with zero retraining; it still transcribes at 5.16% WER versus the teacher model's 4.12%.
- The pruned checkpoint is available on Hugging Face as nemotron-3.5-transcribe-mini-64e.
- The base model is an open-weight checkpoint handling transcription in 40 languages, translation, diarization, and audio Q&A, and topped the Open ASR English leaderboard at 5.33% WER.
- The result shows large speech models can be compressed ~85% with modest accuracy loss, making high-quality transcription feasible on much smaller hardware.
💰 BUSINESS & FUNDING
- The Verge: AI data centers strain Arizona's water as bipartisan backlash grows
The Verge reports that Arizona's booming data center and chip industries are colliding with the state's shrinking water supply, sparking bipartisan local opposition.- Arizona residents, officials, and activists are the focus of The Verge's feature on backlash against new hyperscale data center developments from Phoenix to Tucson.
- TSMC's Phoenix fab complex uses nearly 5 million gallons of water daily from Phoenix's municipal supply; its projected Arizona investment has grown from $12 billion to $265 billion, with 3,500+ jobs created.
- Arizona hosts 172 data centers; Phoenix is the second-largest US data center hub with ~707 MW of IT capacity, and data centers use 7.4% of the state's power.
- Microsoft's Goodyear data center is estimated to use ~56 million gallons of potable water annually — equivalent to 670 households or 1.9% of the town's population.
- Arizona will soon lose access to nearly 30% of its Colorado River water, and new, larger data center proposals are facing mass bipartisan resistance unlike earlier developments.