NAVER AI Lab's DLoop cuts speculative-decoding verification overhead

🔬 RESEARCH

• NAVER AI Lab’s DLoop cuts speculative-decoding verification overhead
  NAVER AI Lab researchers proposed DLoop, a speculative-decoding method that delivers 5%–41% higher wall-clock speedups.

  • For LLM inference engineers using EAGLE-3, DFlash, Domino, DSpark, or multi-token prediction.
  • DLoop runs multiple confident draft stages before verifying accumulated tokens with the target model.
  • Loop-aware training supports parallel drafting with unverified hidden states while preserving lossless decoding.
  • The paper reports 5%–41% wall-clock speedup by reducing target-model forward passes.
  Sources 📄 Source: arxiv.org · 📎 Coverage: cctest.ai · 👁 via r/LocalLLaMA

• EngramEdit enables targeted LLM fact updates through conditional memory
  Researchers introduced EngramEdit for updating LLM facts through conditional memory.

  • It targets LLM builders using Engram-style conditional memory for factual knowledge updates.
  • EngramEdit freezes the Transformer backbone, computes target memory representations across multiple expressions, and jointly updates shared n-gram embeddings.
  • The method penalizes changes to frequently reused embeddings to limit interference with unrelated knowledge.
  • Experiments report near-perfect edit success, transfer to unseen expressions and multi-hop reasoning, and nearly 3× the strongest baseline’s CoT accuracy.
  • Unrelated knowledge and general capabilities were largely preserved as edits accumulated.
  Sources 📄 Source: arxiv.org · 📎 Coverage: cctest.ai · 👁 via r/LocalLLaMA

• Apple researchers propose Stepped MoE for configurable 1B–4B edge models
  Apple researchers published Stepped MoE for configurable 1B–4B edge inference.

  • For on-device and edge LLM deployments constrained by DRAM, compute, or disk.
  • The framework combines elastically nested subnetworks with input-adaptive segment-level MoE routing.
  • One shared model supports 1B, 2B, 3B, or 4B active-parameter configurations at inference time.
  • The configurations were 2–5% more accurate than dense counterparts on knowledge-intensive benchmarks, with similar latency.
  • Shared parameters reduce device disk usage while allowing serving capacity to match available hardware.
  Sources 📄 Source: arxiv.org · 📎 Coverage: paperswithcode.co · 👁 via r/LocalLLaMA

💰 BUSINESS & FUNDING

• Sophos cuts threat investigations 96% with OpenAI Daybreak
  Sophos used OpenAI Daybreak agents to cut threat-response time to 89 seconds.
  • Sophos uses Daybreak in its Fusion AI-native cyber defense system for more than 625,000 organizations.
  • Daybreak agents reduced average case response time from 38 minutes to 89 seconds.
  • AI now resolves 52% of Sophos MDR cases end-to-end while human oversight remains in place.
  • The system processes data from more than 500 integrations and helps analysts handle 1,000–2,000 daily cases.
  Sources 📎 Coverage: openai.com · 👁 via OpenAI News