AI & ML
TAILROLL: Tail-Aware Rollout Acceleration for RL-Based LLM Post-Training
arXiv cs.DC0 views
arXiv:2611.22206v1 Announce Type: new Abstract: RL post-training of large language models (LLMs) is bottlenecked by rollout: heavy-tailed responses leave engines idle as each step waits for the longest response. Asynchronous training masks, but does not eliminate, idle time. We distinguish imbalance waste (uneven placement and decode contention) from straggler waste (waiting for the longest response). Tail length, not average load, often sets the step bound. We propose TAILROLL, an online scheduler integrated into an open-source RL framework. It combines stateless in-flight migration and sufficiency-gated termination. Migration moves decoding requests between engines at chunk boundaries and resumes them with one re-prefill under a freeness-based, cost-aware tail-balancing policy. Termination stops a response when its GRPO group has enough completed samples for advantage estimation, masking its partial output from the training loss. Across ten configurations, TAILROLL reduces rollout bubble ratio by 21% to 74% and cross-engine KV-cache imbalance by 44% to 79%. In end-to-end RL training, it cuts rollout time by 24% to 34%, full step time by 21% to 22% on long-CoT RL, and p95 request latency by 28% to 37%, while raising generation throughput by up to 33% with comparable or higher final reward.
Source:https://arxiv.org/abs/2611.22206PreviousCollecting Infrastructure-Level Measures for Energy Analysis in Modern Computing InfrastructuresNextXM-Sparse: High-Throughput Sparse-Attention LLM Serving over Scale-up Memory Pools
Related posts

Preparing for a post-quantum future
0TechRadar

75+ best Black Friday deals 2026 in the UK — GTA 6, Kindles, Switch 2, Shark vacuums, LG OLED TVs, and more
0TechRadar

Why no 40fps mode for GTA 6? — Rockstar says the ‘complexity of simulating and rendering our world’ requires it to ‘carefully balance how we spend every millisecond of both the central processor and t
0TechRadar

'Enterprises need to connect AI to the right data': Report finds over half of firms have abandoned AI projects over unclear goals
0TechRadar

$3.26 million in SpaceX AI data center revenue will be pumped back into the City of Memphis
0Tom's Hardware

The Downfall of Jensen Huang — a timeline of allegations and scandals that brought down the king of AI infrastructure
0Tom's Hardware