Virtual Head Attention
The design of attention mechanisms in long-context large language models faces an ``impossible triangle'' among KV cache size, training and prefill FLOPs, and model quality. GQA ac...
Papers
FlashMask-3: Efficient and Expressive Mask-Aware Distributed Attention
Long-context training of large language and multimodal models couples structured attention masks (document, prefix-LM, share-question, sliding-window, block-sparse) with multi-node...
Copy-Paste Language Models
Transformers are intuitively considered to be efficient at copying from their inputs. We actually show that the ability of LLMs to copy in context can be improved through explicit ...
Efficient Scene-Text Refinement via Pixel-Space Grouped Image Generation
Generated images often have coherent layouts and visual content, yet local errors in small text and complex characters still limit their readability and practical use. Scene-text r...
Diff Diffusion: Predicting Edit Paths with Diffusion Models for Efficient Code Editing
Diffusion large language models (dLLMs) are appealing for code generation because they can update many tokens in parallel, yet many are optimized for sequence synthesis rather than...
Failure Recovery via Belief Updates and World Modeling for Long-Horizon Agents
Long-horizon agents often struggle to recover from failures whose causes lie several steps earlier, beyond what local reflection or backtracking can resolve. We introduce BELIEFWEA...
Hybrid Reasoning Agents
How can we build an agent that thinks beyond human language, yet still acts through it in the external environment? Modern LLM agents still reason and interact through text, while ...
Are We Testing Too Much? A Survey of Benchmark Compression for Efficient Evaluation of Large Language Models
Benchmarks are essential for evaluating and improving large language models (LLMs). Yet many widely used benchmarks are redundant, making evaluation unnecessarily expensive. Benchm...
λ_0: Scaling Open-World Egocentric Human Data for Humanoid Loco-Manipulation
Humanoid loco-manipulation requires coordinated whole-body interaction with the physical world, yet collecting robot demonstrations at scale remains costly. Human demonstrations of...
MA-Bench: Measuring Operational Planning in Language Models
How well can a language model turn a large, incomplete and partly contradictory body of documents into a plan when no single correct answer exists? We introduce Mission Analysis (M...
How Far Can Coding Agents Go in Visual Generation?
Modern image generation largely relies on specialized generative models that leave the process of image construction implicit in learned parameters. We explore an alternative: gene...
PreciseCoder: Faithful and Generalizable Debugging Beyond Unit-Test Rewards
Reinforcement learning (RL) from unit tests is the common recipe for post-training code models, but a unit test only asks whether the final program passes. We analyze what this uni...
GreenWAM: A World Foundation Model for Robotics
A robot needs a world model that predicts how its actions will change the scene and a policy that chooses those actions. Models are usually built and evaluated for only one of thes...
FlashVSR+: Efficient Video Super-Resolution via Hybrid Attention and Reward-Yield Distribution Matching
Recent video super-resolution (VSR) models achieve efficient one-step generation through diffusion distillation. However, high-resolution inference remains costly, and distilled mo...
Many Right Answers, One Training Example: Removing Responder Noise from GRPO for LLM Recommenders
A recommendation request on Reddit draws several replies, each naming different titles. When an LLM recommender is trained on such threads with reinforcement learning (RL), which r...
Simplifying and Improving On-Policy Distillation for Few-Step Diffusion
Few-step distillation reduces the sampling cost of diffusion and flow models, but existing approaches often require auxiliary score models or specialized architectures. We show tha...
Model-Based Online Decision Making via Generative Trajectory Planning
Online decision-making faces a tension between efficient inference and long-horizon reasoning. Value-based methods rely on short-horizon bootstrapping that limits the propagation o...
Scaling Online Optimal Transport Minibatches for Flow Matching
A central challenge in scaling flow models is maintaining stability in training and integration. In flow matching, this stability depends critically on how source samples are paire...
SWE-MISBEHAVIOR: Benchmarking and Mitigating Boundary Violations in Software Coding Agents
We present SWE-MISBEHAVIOR, the first taxonomy and evaluation framework for diagnosing boundary-violating misbehaviors in coding agents. It contains 7.4K human-in-the-loop synthesi...
WorldRecon: Generative Multi-View Mesh Reconstruction for Large-Scale 3D World
Reconstructing complete, high-quality meshes of large-scale scenes from unposed multi-view images requires faithful geometric grounding while plausibly completing surfaces that are...
Agent-Reflex: Teaching Language Agents to Act Reflectively
Large language models have evolved from single-turn chatbots into interactive agents capable of long-horizon reasoning. In real deployments, MCP tool services can return noisy feed...
Enabling Token-wise Importance Sampling in Off-Policy RLVR with Proximal KL Regularization
Off-policy data in reinforcement learning with verifiable rewards (RLVR) can arise from asynchronous generation, rollout-training mismatch, and response reuse. Token-wise importanc...
Reading Right, Acting Wrong: Diagnosing Decision Failures of System One Models in Agent Loops
System One models choose an agent's next action by scoring a supplied set of options in a single pass. This makes them fast and cheap, but when they choose wrongly there is nothing...
Jeva: System One Models from First Principles
System one models enable direct, low-latency decision-making. Yet recent practice of predicting an answer from serialized text does not, by itself, respect the structure of a decis...