AI & ML
TAILROLL: Tail-Aware Rollout Acceleration for RL-Based LLM Post-Training
arXiv cs.DC0 views
arXiv:2611.22206v1 Announce Type: new Abstract: RL post-training of large language models (LLMs) is bottlenecked by rollout: heavy-tailed responses leave engines idle as each step waits for the longest response. Asynchronous training masks, but does not eliminate, idle time. We distinguish imbalance waste (uneven placement and decode contention) from straggler waste (waiting for the longest response). Tail length, not average load, often sets the step bound. We propose TAILROLL, an online scheduler integrated into an open-source RL framework. It combines stateless in-flight migration and sufficiency-gated termination. Migration moves decoding requests between engines at chunk boundaries and resumes them with one re-prefill under a freeness-based, cost-aware tail-balancing policy. Termination stops a response when its GRPO group has enough completed samples for advantage estimation, masking its partial output from the training loss. Across ten configurations, TAILROLL reduces rollout bubble ratio by 21% to 74% and cross-engine KV-cache imbalance by 44% to 79%. In end-to-end RL training, it cuts rollout time by 24% to 34%, full step time by 21% to 22% on long-CoT RL, and p95 request latency by 28% to 37%, while raising generation throughput by up to 33% with comparable or higher final reward.
Source:https://arxiv.org/abs/2611.22206PreviousCollecting Infrastructure-Level Measures for Energy Analysis in Modern Computing InfrastructuresNextXM-Sparse: High-Throughput Sparse-Attention LLM Serving over Scale-up Memory Pools
Related posts
Fluent English is not evidence of ASL understanding: lessons from building a video translator
0dev.to

Audit Trails and Explainability for Agentic AI in Code Events
0dev.to
By 3 p.m. the daily high was already final on 78% of days. In Madrid, 32% (49 airports, one year)
0dev.to

This smell startup says its AI nose is as good as a human’s
0MIT Technology Review

Debugging Claude Fable 5.2 Code Art: Truncated SVG, Broken Canvas, Working Render
0dev.to
Hydra · Coeziune OM–IA — the complete DOI registry (2026-11-18): 76 published works
0dev.to OpenSource