AI & ML
Computed Permutation Trellis Coding: Near-Optimal Quantization from Two Multiplications and a 64-Byte Table
arXiv math.IT0 views
arXiv:2611.22019v1 Announce Type: new Abstract: We propose Computed Permutation Trellis Coding (CPTC), a low-bit quantization format for tensors written once and read many times. Formats that round each value on its own, such as INT4 and FP4, waste a large part of their bits against the information-theoretic ideal. Trellis codes recover this loss by encoding values jointly, but prior decoders either need a large lookup table for their best quality or compute values with a hash that restricts what they can decode. CPTC's decoder multiplies its state by two coefficients and adds the top bits of the products to index a 64-byte table. Because that sum is not uniformly distributed, the rarely used ends of a small table can hold values far out in the tails. One decoder serves every bit width from 1 to 6, reproduces the standard integer and floating-point formats exactly, and comes within 0.07 bits of the ideal at 4 bits. It matches a random-permutation trellis code whose table is thousands of times larger, and exceeds every QTIP codebook. In LLM weight quantization without fine-tuning, CPTC has the lowest KL divergence to the unquantized model at every tested bit width on Llama 3.1 8B and 70B Instruct against QTIP's codebooks (by up to 27%). The decoder has been built into the AWS Trainium 4 AI accelerator, where it runs at the throughput of the native FP4 and FP8 formats for a negligible fraction of the systolic array's area. Fused encoding kernels quantize an 8B model's linear layers in under two minutes on one GPU and, in cycle-level simulation, on one Trainium 4 chip.
Source:https://arxiv.org/abs/2611.22019PreviousAn upper bound on non-overlapping codesNextSubsystem Stabilizer $(r,\delta)$-Locally Recoverable Codes: Characterization, Bounds and Constructions
Related posts
Classified communications network at the National Security Agency goes down, defense official says
0Phys.org Tech
Le Groupe Figaro and OpenAI partner to bring Le Figaro journalism to ChatGPT
0OpenAI Blog
Web 上已经有 GPT-6.2,本地 Codex 却没有?先对入口,再更新客户端
0dev.to
영지식 증명 프라이버시와 규제 준수: 우리는 정말 두 마리 토끼를 다 잡을 수 있을까?
0dev.to
My context grew 6x in three months. That decides the cache question
0dev.to AI
How to connect Joist to any MCP client: Claude, ChatGPT, Cursor
0dev.to AI