AI & ML
LEDGE: An Efficient Simulator for Distributed LLM Serving on Heterogeneous Edge Devices
arXiv cs.DC0 views
arXiv:2611.24085v1 Announce Type: new Abstract: Evaluating distributed large language model (LLM) serving on heterogeneous edge devices requires exploring diverse model partitions, device placements, and network conditions, making repeated physical experiments costly. An efficient simulator for edge LLM serving must support diverse, fine-grained placement configurations, capture runtime behavior, and accurately predict execution times while keeping simulation overhead low. We present LEDGE, a lightweight simulator for fast, fine-grained evaluation of distributed LLM serving on heterogeneous edge devices without executing tensor operations. LEDGE constructs event graphs supporting non-uniform tensor and pipeline parallelism, estimates execution costs using reusable performance models calibrated from end-to-end measurements, and simulates shared-link contention. We evaluate LEDGE on a heterogeneous testbed of 10 Jetson and Raspberry Pi devices, covering 17 model--precision combinations across 4 LLM families. On Qwen deployments, LEDGE achieves request-level mean absolute percentage errors below 3.6% and a 135.3 times aggregate speedup over physical inference. Performance model calibrated on one LLM family yield 7.44% zero-shot end-to-end latency MAPE across three other families. Further simulations show how shared-link contention affects placement decisions and network topology constrains scaling. Across four edge environments, LEDGE completes 26,773 simulation runs in 83.8 minutes, enabling efficient exploration of deployment strategies under diverse resource conditions and serving objectives. The source code is publicly available at https://github.com/gdfwj/LEDGE.
Source:https://arxiv.org/abs/2611.24085PreviousLocality-Aware Scheduling for Stream Processing Applications in Serverless EnvironmentsNexteFAIR: Fair LLM Serving under Hourly Electricity Commitments
Related posts
Chaos, then control: the maturity model most AI rollouts skip over
0Stack Overflow Blog
ChatGPT at a venture firm: an engineer's checklist for deal data
0dev.to
I checked 183 vibe-coded apps. 92% are almost blank to AI crawlers.
0dev.to
What a Rule Engine, a Model and a Graph Can Each Know About a Mule Chain
0dev.to

I made the model end every post with a question. The replies stayed the same
0dev.to
How powerful is modern "low-end" hardware?
0dev.to