🚀 RELEASES & LAUNCHES
Qwen Code repo PRs leak Qwen4-Max, Qwen4-Coder-Plus, and Qwen4-Omni-120B model IDs
GitHub pull requests against Alibaba's Qwen Code repo revealed references to unreleased Qwen4 model IDs including qwen4-max and qwen4-omni-120b.- For developers using Qwen Code and local-AI builders tracking the Qwen4 family, new GitHub PRs (#15762, #15987) exposed model IDs qwen4, qwen4-max, qwen4-plus, qwen4.5-max, qwen4-coder-plus, and qwen4-omni-120b-think.
- The PRs show these IDs resolving to a 262,144-token context limit, with qwen4-omni-120b already used as the session model in the repo's omni fixtures — meaning the IDs are not hypothetical.
- Separately, Alibaba named Qwen4-Max at the Apsara Conference on September 22, 2026 as flagship of a four-tier lineup (Max, Flash, Plus, 27B open-weight), with Qwen4 confirmed in training on a new architecture but no specs published.
- The leak matters because it offers the first concrete, code-level evidence of Qwen4 model identifiers and context sizing ahead of any official release or model card.
Hugging Face proposes Model Integration Contracts: shrunk fixtures run in Transformers PR CI
Hugging Face introduced transformers-mic, a proposed standard that runs tiny shrunk model fixtures with executable contracts in Transformers CI to catch regressions on every pull request.- WHO: Transformers maintainers and model integrators; the huggingface-mic Hub org hosts 203 fixture models and a pinned fleet dataset for Transformers CI.
- WHAT: A Model Integration Contract (MIC) has three parts — a shrunk fixture built from a real checkpoint's config, a runnable Markdown usage path, and human-reviewed output baselines; the tmic CLI builds and runs them.
- WHAT: Fixtures run on CPU in seconds on PRs (a T5 contract took 13.4s excluding setup); real checkpoints still run after merge, nightly, and before release.
- WHAT: The UMT5 fixture caught a real SDPA bug that reached main on July 30, 2026 and stayed for ~2 months; replay flagged it on every commit until the September 28 fix (label log-probability off by 1.22, training loss by ~0.20).
- WHY: Transformers has 750+ model types and 88,000+ CPU tests plus 200,000+ GPU tests daily; shrinking models (e.g. Llama 4 Scout from 108.6B to 44.7M parameters) gives fast, meaningful regression signals without keeping GPUs busy all day.
Simon Willison releases LLM CLI 0.37 with new GPT-6.1 models and OpenAI endpoint tooling
Simon Willison released llm 0.37, a major update to his LLM CLI tool and Python library.- For developers using the LLM CLI and Python library to run prompts against local and hosted models.
- Adds gpt-6.1-sol and gpt-6.1-terra model IDs, plus a new 'llm openai endpoint add' command for configuring OpenAI-compatible endpoints.
- Switches the default model to gpt-6-luna, which is half the price of the previous default gpt-5.6-luna.
- Embeddings support now handles different content-types per binary, needed for models like Gemini Embedding 2 that embed text, audio, images, and video.
- The release bundles around two months of accumulated features and fixes.
🔬 RESEARCH
Paper: LLMs can natively read and write neural network weights
A new paper reports that frontier LLMs can read raw neural network weights and edit them directly without gradients.- Targets ML researchers working on optimization and interpretability of neural networks.
- GPT-6 Astra identifies a network's trained task from raw weights alone and writes small game-playing networks from scratch that outplay PPO-trained networks trained for 100 million steps.
- Asked to improve Qwen3.5-0.8B at math without gradients, it diagnosed a weight issue causing reasoning loops (noted in Qwen's own model card) and fixed it by editing 0.016% of the weights.
- The paper argues this capability could matter for both optimization and interpretability.
- A demo, edited Qwen3.5-0.8B weights, and supporting logs were released publicly on Hugging Face and GitHub.
MIT researchers use GPT-6 Astra to edit Qwen3.5-0.8B weights, lifting MATH-500 from 7.9% to 48.1%
MIT CSAIL researchers used OpenAI's GPT-6 Astra to directly edit 0.016% of Qwen3.5-0.8B's weights without gradient training, raising its MATH-500 score from 7.9% to 48.1%.- For open-weight model developers and ML researchers: MIT CSAIL's Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab demonstrated 'weight-space programming' — an LLM inspecting and editing another network's weights directly, with no gradient-based training.
- GPT-6 Astra edited 131,072 weights (~0.016% of Qwen3.5-0.8B's 800M parameters) across layers 15 and 19, removing a repetition direction that caused reasoning loops; MATH-500 rose 7.9%→48.1%, GSM8K 13.8%→67.1%, IFEval 21.0%→62.3%, and token-limit hits fell 93.0%→33.1%.
- The winning run took 158 conversational turns, 80 Python executions and 20.3 hours within a 50-request budget; Astra also identified game-playing networks from weights alone, one-shot fixed a stuck driving controller (0%→100% success), and wrote MinAtar Breakout networks from scratch that beat 100-million-step PPO baselines.
- It matters because it points toward AI systems that diagnose, repair and construct other neural networks directly — potentially a cheaper alternative to fine-tuning for some fixes — though the researchers did not establish savings over conventional training.
- The team released edit manifests, run logs and evaluation code; outside commenters ranged from enthusiastic (Databricks' Yuchen Jin) to concerned (Pause AI's Maxime Fournes on acceleration risks).
Open-source sparse attention method extends Qwen 3 models to 10M-token contexts
Researchers released open-source sparse attention methods that let Qwen 3 models run ultra-long contexts with minimal accuracy loss.- For local-LLM users and builders of long-context inference systems, including small open-weight models on local hardware.
- The team sparsified attention in a Qwen 3 27B model with no observed accuracy loss and extended context to 10M tokens, where needle-in-a-haystack retrieval still works though some tasks degrade.
- Sparse attention can be natively trained on next-token prediction alone, matching full attention's validation loss while running faster.
- Everything is fully open: kernels, SGLang integration, and the paper; the team targeted H200/B300 GPUs at scale.
- Related research includes the ASEntmax paper (arXiv 2506.16640) showing up to 1000x length extrapolation and MLSys 2025's SampleAttention cutting time-to-first-token by up to 5.29x vs FlashAttention2.
How "reverse game theory" could keep AI agents in check
MIT Technology Review reports that alignment researchers are exploring mechanism design—"reverse game theory"—to create incentives that make rule-following the rational choice for AI models.- Aimed at AI safety researchers and frontier-model developers, the approach designs rules and incentives rather than trying to interpret models' internals or train in values.
- A preprint by MIT economist Stephen Morris and colleagues proposes mechanisms against "sandbagging": rewarding a model for how well its answers predict a second model's outputs, pitting agents against each other, and pairing oppositely-biased agents.
- Princeton's Elad Hazan proposes a solver/auditor scheme where models alternate roles, cheaters and failed auditors are punished, and disputes go to human "trial," drawing on Gary Becker's rational-offender economics.
- Critics like Flatiron Institute's Helen Qu question whether AI agents respond to incentives rationally, framing mechanism design as one layer in a defense-in-depth strategy alongside interpretability and regulation.
- It matters because current alignment techniques have shown failures—models underperform on tests, hide intentions, and mislead evaluators—so incentive-based control could work even without understanding the model.
Library of Layers: transformer architecture dynamically reorders layers at inference
Researchers introduced Library of Layers (LoL), a transformer architecture that dynamically orders its layers during inference.- LoL is aimed at language model builders seeking parameter- and FLOPs-efficient architectures.
- It dynamically orders transformer layers at runtime, with a single hyperparameter — the size of its global library of transformer blocks — that transitions between parameter reuse and sparsity.
- LoL generalizes the efficiency tradeoffs of dense, depth-recurrent, and mixture-of-experts architectures in one unified family.
- The paper reports LoL achieves competitive performance with all these architectures.
- It matters because it unifies several efficiency approaches into a single adaptable architecture family.
Sources 📎 Coverage: huggingface.co · 👁 via Hugging Face Papers
💰 BUSINESS & FUNDING
- DeepSeek plans to pre-train V5 on 160K Huawei Ascend 950DT chips
DeepSeek reportedly plans to pre-train its next-gen V5 model on Huawei Ascend chips instead of Nvidia GPUs.- Per a STAR Market Daily report citing industry sources, DeepSeek plans to pre-train DeepSeek V5 on Huawei Ascend chips, a shift from prior inference-only use.
- Emergent reports at least 160,000 Ascend 950DT chips (144 GB memory, ~4 TB/s bandwidth) at a new Ulanqab, Inner Mongolia data center, with pre-training starting late 2026.
- Founder Liang Wenfeng estimates four Ascend 950 chips match one Nvidia GB300 GPU, a tradeoff accepted for supply chain independence under US export controls.
- DeepSeek has open-sourced Ascend-optimized infrastructure, reportedly hitting over 95% of the 950's peak performance in internal tests; Huawei engineers are embedded at DeepSeek to co-design hardware.
- If confirmed, it would be the first full pre-training of a frontier Chinese LLM on domestic accelerators, testing Huawei as an Nvidia alternative.
Sources 📄 Source: mp.weixin.qq.com · 📎 <https://emergent.sh/news/deepseek-v5-pre-tr