AI & ML
Dissecting an Unexplored SASS Control-Code Field: Barrier Exemption and Programmable Warp Priority
arXiv cs.DC0 views
arXiv:2611.24437v1 Announce Type: cross Abstract: Every instruction in NVIDIA's native GPU ISA (SASS) carries a control code that guides hardware scheduling, and prior work has decoded its known fields to optimize GPU kernels. However, one control-code field remains unexplored: it appears over 500000 times in NVIDIA's closed-source libraries such as cuBLAS and cuDNN, yet nvcc never emits it when compiling user programs. This paper presents the first systematic study of this control-code field across GPU architectures from Volta to Blackwell. Using instruction-level microbenchmarks, we show that the field controls two independent hardware mechanisms: barrier exemption (EXEMPT), which allows instructions blocked by a pending barrier to issue early during the barrier wait, and programmable warp priority (START, TILE, and END), which moves warps across three priority levels in the hardware scheduler. We identify the conditions under which each mechanism improves GPU kernel performance and implement a post-compilation binary optimizer that places these marks directly in compiled binaries without modifying source code or the compiler. Evaluated across six GPUs spanning four architectures, our optimizations speed up representative kernels by up to 31.9%, and the optimizer accelerates production kernels in llama.cpp and vLLM by up to 5.4% and end-to-end LLM inference by up to 3.6%.
Source:https://arxiv.org/abs/2611.24437PreviousOpenGNS: An Open Dataset of the Gradient Noise Scale across Vision, Language, and Diffusion ModelsNextEfficient simulation of variational quantum circuits: representation, accuracy and CPU/GPU allocation
Related posts
What Does a Harness Buy? Tokens, Mostly
0arXiv cs.SE
Prior Knowledge or Search? A Study of LLM Agents in Hardware-Aware Code Optimization
0arXiv cs.SE
Live Noise Fingerprints for Quantum Program Analysis
0arXiv cs.SE
Biomedical Knowledge Composition: A Software Engineering Perspective
0arXiv cs.SE
MicroAssert: Microarchitecture-Aware Structured Assertion Generation for CPU Verification
0arXiv cs.SE
Reducing Inference Cost of Coding Agents with Timeout Cascades
0arXiv cs.SE