AI & ML
A Fast, Programmable GPU-to-CPU Invocation Layer for GPU-Initiated Communication
arXiv cs.DC0 views
arXiv:2611.23848v1 Announce Type: new Abstract: Modern GPU workloads increasingly rely on device-initiated communication to let GPU kernels communicate directly without synchronizing with the host at kernel boundaries. Existing libraries commonly favor placing transport control on GPU threads, an approach known as GPUDirect Async (GDA), and treat reverse offload to host proxy threads as a slower fallback. We argue that reverse offload can outperform GDA and should be a first-class communication path. We further propose exposing reverse offload as a programming interface that lets users define workload-specific communication operations. We present KIWI, a programmable reverse-offload framework that achieves sub-microsecond one-way device-to-host latency and hundreds of millions of operations per second. We address practical challenges of scaling reverse offload, including contention on shared queues, polling overhead, and load imbalance across proxies. KIWI-P2P, a SHMEM-like library built on KIWI, outperforms state-of-the-art GDA implementations in both latency and bandwidth. More importantly, KIWI-EP redesigns low-latency mixture-of-experts dispatch and combine kernels around workload-defined operations, lowering expert-parallel communication latency by up to 2.3x over the best existing backend on AMD GPUs and time per output token in vLLM by up to 29%. The source code of KIWI is available at https://github.com/AMDResearch/kiwi.
Source:https://arxiv.org/abs/2611.23848PreviousStephen: Verifying Sequencer-Aided Coordination Systems with Cross-View ProphecyNextNeural Memory Orchestration
Related posts

With the biggest IPO of all time, Anthropic’s ‘good guy’ reputation is on the line
0The Verge
Read the Hints First, Then Guess: A Better Way to Play a Music Daily
0dev.to

Did the attack really work?
0dev.to
Synergizing Evidence Retrieval and Utilization for Self-Improving Multimodal Scientific Document Reasoning
0dev.to
How to build a favicon set from a logo you already have
0dev.to
Understanding AI Computing — Part 4
0dev.to