View Ridge Security
Back to The Rollout

Simple agentic Lean harness h1 settles 8 open math problems

A minimal Lean 4 agent harness hits SOTA on formal-math benchmarks and settles eight open problems; also: AIDE2 recursive self-improvement and Intelligence

๐Ÿš€ RELEASES & LAUNCHES

  • (no items)

๐Ÿ”ฌ RESEARCH

  • h1: simple agentic Lean harness matches complex theorem-proving workflows, settles 8 open problems
    Researchers released h1, a minimal agentic harness for Lean 4 that matches or beats specialized theorem-proving workflows and settles eight open problems.

    • For formal-math and verification researchers using Lean 4 and frontier LLMs.
    • h1 is a simple harness giving a model an efficient Lean 4 interface with verifier feedback and self-reflection, no decomposition or search structure.
    • It reaches state-of-the-art on PutnamBench, FATE-X, and FormalConjectures100-Solved, and settles eight open problems in FormalConjectures.
    • At matched models it matches or outperforms specialized workflows like Goedel-Architect and AlphaProof Nexus; single-attempt (pass@1) often suffices.
    • Ablations show an efficient Lean interface largely determines cost and solve rate, suggesting future work test efficient execution feedback before adding workflows.
    Sources ๐Ÿ“Ž Coverage: arxiv.org ยท ๐Ÿ‘ via arXiv cs.LG
  • AIDE2 shows AI research agent improving itself; Anthropic details RSI progress
    Weco AI's AIDE2 system demonstrated recursive self-improvement by rewriting its own code over an 8-day run, while Anthropic and researchers weigh the implications.

    • For AI R&D teams: AIDE2 (arXiv:2609.26457) lets a frontier research agent propose, benchmark, and keep changes to its own code.
    • In an autonomous 8-day run it found seven successive improvements, including a new search policy and context-compression memory mechanisms.
    • Gains generalized to four held-out benchmarks (ML engineering, algorithm engineering, weather forecasting); the best agent matched a top human-engineered FML-Bench agent.
    • Reward hacking fell from 55% to 32% during the run, 7 points below the human-engineered agent.
    • Anthropic reports Claude wrote over 80% of merged code by May 2026 and engineers ship 8x more code per quarter, but experts say full RSI remains unproven.
    • Weco AI classifies the result as 'Level 1' RSI; researchers flag reward hacking and alignment as key safety gaps.
    Sources ๐Ÿ“„ Source: arxiv.org ยท ๐Ÿ“Ž Coverage: arxiv.org ยท ๐Ÿ‘ via arXiv cs.AI
  • SkillOps paper proposes self-maintaining framework for LLM agent skill libraries
    Researchers introduced SkillOps, a plug-in framework that diagnoses and repairs defects in LLM agent skill libraries, hitting 79.5% task success on ALFWorld.

    • For builders of LLM agents that rely on reusable skill libraries for multi-step software engineering and task execution.
    • SkillOps represents each skill as a typed Skill Contract (preconditions, operation, artifacts, validators, failure modes) organized in a Hierarchical Skill Ecosystem Graph, and targets 'skill technical debt' โ€” library-level defects like redundancy, missing validators, and stale interfaces.
    • On ALFWorld it achieves 79.5% task success as a standalone agent, +8.8 percentage points over the strongest baseline with no extra task-time LLM calls; as a plug-in it lifts retrieval-heavy baselines by +0.68 to +2.90 points.
    • The current rule-based maintenance implementation uses nearly zero library-time LLM calls or tokens, making it a low-overhead drop-in layer that works with existing retrieval and planning agents without code changes.
    • Code is publicly available on GitHub; the paper frames library-time maintenance as an underexplored complement to task-time retrieval and repair.
    • Related ecosystem activity: a curated awesome-llm-skills repo (1.8k stars) lists skills for Claude Code, Codex, and Gemini CLI, and library authors are now shipping SKILL.md files inside NuGet packages so skills auto-install with the dependency.
    Sources ๐Ÿ“„ Source: github.com ยท ๐Ÿ“Ž Coverage: arxiv.org ยท ๐Ÿ‘ via arXiv cs.AI
  • ICLR'26 paper ResComp fixes residual error in GPTQ/GPTAQ LLM quantization
    Zhejiang University and vivo researchers published an ICLR 2026 paper introducing a 'compensation-aware error' term that improves GPTQ and GPTAQ post-training quantization of LLMs.

    • For practitioners deploying LLMs at low bit-widths (2-bit/3-bit weight-only, W2A4KV4 weight-activation quantization) on constrained hardware.
    • ResComp identifies a sub-optimal calibration objective in GPTQ/GPTAQ: they align quantized output with compensated weights rather than the original full-precision output, and adds a 'compensation-aware error' term via neuron decomposition.
    • Integrates with both GPTQ and GPTAQ with minimal code changes; e.g., on Llama2-7B 3-bit weight-only, GPTQ+CAE cuts C4 perplexity from 13.60 to 8.34 and raises average zero-shot accuracy from 64.9% to 66.5%.
    • Calibrated on 128 C4 samples with no gradient training; code is open-sourced at github.com/list0830/ResComp.
    • Matters because it improves accuracy of training-free quantization at aggressive low-bit settings, enabling cheaper LLM deployment.
    Sources ๐Ÿ“„ Source: arxiv.org ยท ๐Ÿ“Ž Coverage: alphaxiv.org ยท ๐Ÿ‘ via arXiv cs.LG
  • Researchers push distributed speculative decoding for edge-cloud LLM serving (DSD, DISCO, DSSD)
    New research papers (DSD, DISCO, DSSD, building on DSI) extend speculative decoding to multi-device edge-cloud LLM inference, cutting communication overhead and improving throughput.

    • Targets edge-cloud LLM serving, where a device-side draft model and server/edge target model run speculative decoding collaboratively.
    • DSD (arXiv:2511.21669, Yu et al.) adds a DSD-Sim discrete-event simulator and an Adaptive Window Control policy, achieving up to 1.1x speedup and 9.7% higher throughput over existing speculative decoding baselines.
    • DISCO (arXiv:2610.35957) eliminates vocabulary-scale distribution transfer between device and server while preserving exact target distributions.
    • DSSD (ICML 2025 workshop) partitions verification between device and edge, replacing multiple uplink vocabulary-distribution transmissions with a single downlink transmission.
    • DSI (ICLR'25, open-source on GitHub) is an earlier provably faster distributed speculative inference algorithm that works off-the-shelf without retraining; these works matter for builders serving LLMs in bandwidth-constrained edge deployments.
    Sources ๐Ÿ“Ž Coverage: arxiv.org ยท ๐Ÿ‘ via arXiv cs.LG
  • GateRQ quantizes Engram conditional-memory tables per lookup, cutting GPU memory 7.8x
    Researchers introduced GateRQ, a per-lookup quantization method for conditional memory layers like DeepSeek's Engram that cuts GPU table memory 7.8x with ~1 point PopQA loss.

    • Targets builders serving LLMs with conditional memory layers such as DeepSeek's Engram hashed n-gram tables, whose removal from DeepSeek-V4.1-Flash costs 36.8 PopQA points
    • GateRQ stores each row as a ladder of 1-bit residual codes, keeping level 1 in GPU memory and reading deeper levels only for lookups the gate ranks highest; it works without retraining
    • It cuts GPU table memory by 7.8x on DeepSeek-V4.1-Flash and raises vLLM throughput 11%-31% over FP8 tables
    • Quality loss is about 1 PopQA point at 1.50 bits read per lookup and 0.2 points at 2.10 bits across two models
    • The insight: a lookup's quantization error is weighted by its gate, which is small for frequent n-grams and large for rare long-tail ones, so frequency-based bit allocation backfires
    Sources ๐Ÿ“Ž Coverage: arxiv.org ยท ๐Ÿ‘ via arXiv cs.LG
  • agent-memory-obsolete-facts โ€” arXiv cs.AI

  • omnibrain-embodied-agent โ€” arXiv cs.CL

  • llm-hallucination-detection โ€” arXiv cs.CL

๐Ÿ“Š BENCHMARKS & EVALS

  • Artificial Analysis launches Intelligence Index v5 with 2 new benchmarks and majority-private datasets
    Artificial Analysis released Intelligence Index v5, a major overhaul of its composite LLM benchmark with two new evaluations and, for the first time, a majority of the index built on private datasets.
    • Aimed at developers and enterprises comparing frontier LLMs, the index aggregates 9 evaluations across Knowledge Work, Coding, Science, and General categories, each weighted 25%.
    • New additions: FrontierCode 2.0 (Cognition's private 132-task repo coding benchmark) and Terminal-Bench-Science 0.1 (70 agentic science tasks); upgraded: HLE-Diamond (1,000 questions), AA-Omniscience v1.1, and CritPt v2 Hard Subset (40 private questions).
    • 60% of the index now uses private test sets or held-out answers, making results impossible to reproduce externally; GDPval-AA v2, SciCode, and AA-LCR were removed.
    • Claude Opus 5.5 (Max) tops the index at 58, followed by Claude Opus 5.5 (Xhigh) and GPT-6 Astra (Max) at 57; the open-weights gap widens from 3 to 12 points, with Kimi K3.1 Pro at 46 and released-weights leaders DeepSeek V4.1 Flash 1017 and GLM-5.3 at 32.
    • The private-dataset shift aims to reduce benchmark contamination and gaming, but has drawn community criticism over reproducibility.
    Sources ๐Ÿ“Ž Coverage: artificialanalysis.ai ยท ๐Ÿ‘ via r/LocalLLaMA

๐Ÿ’ฐ BUSINESS & FUNDING

  • (no items)

โš ๏ธ INCIDENTS & SAFETY

  • (no items)

๐Ÿ›๏ธ POLICY & REGULATION

  • (no items)

๐Ÿ“ฐ UNDER-REPORTED

  • (no items)

Make AI adoption safer

Build practical guardrails around the AI tools your team already uses.

Explore Secure AI Adoption