AI & ML
eFAIR: Fair LLM Serving under Hourly Electricity Commitments
arXiv cs.DC0 views
arXiv:2611.24119v1 Announce Type: new Abstract: Datacenters can procure electricity in day-ahead markets by committing to energy purchases for individual hours. For LLM services operating within these hourly budgets, maximizing aggregate throughput alone may still leave individual requests waiting or receiving much slower service than desired. Fair serving therefore requires improving service for poorly served requests while reserving electricity for later arrivals whose timing and demand are unknown. To compare service across requests, we define normalized throughput as each request's actual output rate divided by the product of its standalone throughput and its service entitlement, a positive weight representing its desired service level. Using this measure, we formulate fair serving as leximin optimization under an hourly electricity constraint, prioritizing requests with the lowest normalized throughput. To address this problem, we present eFAIR, an online scheduler that coordinates request dispatching, electricity budget allocation, and GPU frequency selection through a common normalized-throughput target. Specifically, request dispatching prioritizes requests by desired service level and progress; electricity budget allocation reserves electricity for current and expected future work; and frequency selection uses electricity savings to raise GPU frequencies for requests with higher desired service levels. Evaluations on H20 and H200 configurations, each with 32 replicas, show that eFAIR improves the tenth percentile of normalized throughput by 28.0%-31.0% over the best budget-feasible baseline, while keeping absolute deviations from the hourly electricity commitment below 0.26%. eFAIR also improves throughput balance among concurrent requests and aligns electricity allocation more closely with service demand over time. Together, these results demonstrate how eFAIR improves request-level fairness within an hourly commitment.
Source:https://arxiv.org/abs/2611.24119PreviousLEDGE: An Efficient Simulator for Distributed LLM Serving on Heterogeneous Edge DevicesNextMultilevel Graph Partitioning on GPU Supercomputers
Related posts
Mesa 26.3.1 Ships Fixes For Intel Jay Compiler & RADV Driver
0Phoronix

Report finds workers are staying around at their jobs — but many say that's because they feel stuck
0TechRadar

95+ best Black Friday deals 2026 in the UK — GTA 6, Shokz OpenRun, Shark vacuums, Oura Ring, Kindle, Switch 2, and more
0TechRadar
Gig Work Is Invading Nursing Homes. The Risks Can Be Deadly
0Wired
4 Request-Validation Gotchas When Migrating OpenAI Clients to GPT-6.2 Reasoning Models
0dev.to
Razorpay webhook not verified: anyone can fake a payment
0dev.to