Papers with Code paper Sep 30

Scaling Laws for Looped Mixture of Experts

Looped transformers and Mixture-of-Experts (MoE) offer complementary routes to efficient scaling: recurrence increases computational depth at fixed parameters, while MoE sparsity e...

Papers with Code paper Sep 30

Initialization Improves LLM-Driven Discovery

Large Language Models (LLMs) have been used for novel discovery of algorithms, theorems, drugs, and other tasks through the use of harnesses that prompt an LLM to iteratively optim...

Papers with Code paper Sep 30

Semifactual Credit-Augmented Policy Optimization

Reinforcement learning with verifiable rewards (RLVR) has improved the reasoning capabilities of large language models (LLMs), yet their predictions remain sensitive to task-irrele...

Papers with Code paper Sep 30

SparseEngine: Sparse-First Inference Engine

Long-context LLM agents accumulate interaction histories that strain KV-cache memory and attention computation. Although sparse attention reduces these costs, heterogeneous cache r...