Overview
A GGUF conversion of Tencent’s Qwen3.8-27B-DSpine-b16 draft model, intended for speculative decoding with a separate Qwen3.8-27B target model in a llama.cpp fork. It is not a standalone chat model. The maintainer is aj9o9.
Best use cases
Use it to accelerate Qwen3.8-27B generation, especially code, math, and tool-call workloads. The supplied RTX 3090 tests also show gains on mixed chat tasks, but prose and image-heavy requests may benefit less; tune draft length for your GPU and workload.
Limitations
Requires both a compatible Qwen3.8-27B target GGUF and the dspine branch of the llama.cpp fork; stock llama.cpp does not load these files. Longer drafts consume more VRAM and reduce available context: on the tested 24 GB RTX 3090 setup, Q4_K_M with 15 drafted tokens fit up to context 65,536, versus 98,304 with 7 drafted tokens. Speculative decoding can produce different output from non-speculative runs, including at temperature 0. Performance figures are hardware-, backend-, prompt-, and setting-dependent; community results use differing test conditions.
How it compares
On the tested RTX 3090, Q4_K_M DSpine at 15 drafted tokens reached 90.1 tok/s on a mixed short-context task set, compared with 51.7 tok/s for DFlash2 and 28.2 tok/s without speculation. Code and math were the strongest measured tasks; prose was slower than code and math. The same test setup reported maximum context of 65,536 for DSpine at 15 drafts, 98,304 for DFlash2 at 7 drafts, and 176,128 without speculation. These are results for the stated setup, not general rankings. Other GGUF alternatives listed in the supplied sources include Qwen3.8-27B-S-mirai-GGUF, Qwen-3.8-27B-ROCmFP4-FAST-GGUF, Qwen3.8-27B-i1-IQ4_KS_KT-GGUF, and Qwen3.8-27B-GGUF.
Technical specifications
This is a five-layer speculative drafter that reads hidden states from five target-model layers and drafts a 16-token block in parallel. Its described mechanisms include causal block attention, a gated code bridge after each layer, and serial last-write refinement over candidate predecessor tokens. Available files are Q4_K_M (1.26 GB; code-bridge tensors remain BF16), Q8_0 (2.20 GB), and BF16 (4.06 GB, described as intended for re-quantizing). The files contain only the drafter. The supplied RTX 3090 benchmark used a Q4_K_M Qwen3.8-27B target, q8_0 KV caches, one slot, and the CUDA dspine branch. It measured 90.1 tok/s on a mixed task set at temperature 0; task-specific results were 128.7 tok/s for code, 122.8 for math, 62.0 for summarize/translate, and 41.6 for prose/story. These are reported measurements, not guarantees. The supplied abstract describes DSpine research results on Qwen3-4B and Qwen3-8B in SGLang; it does not establish those results as measurements of this GGUF conversion or its Qwen3.8-27B use.
Model inputs and outputs
Inputs
Use a compatible Qwen3.8-27B target GGUF together with one of the drafter GGUF files. The documented llama.cpp server setup accepts a target path (-m) and drafter path (-md), with controls including --spec-draft-n-max, context size (-c), slot count (-np), and optional probabilistic draft sampling. The README warns that auto-fit may not reserve enough memory for the drafter; set context size explicitly. On 24 GB cards, two slots with 15-token drafts do not fit according to the supplied guidance.
Outputs
The drafter proposes token blocks for speculative verification by the target model; it does not independently provide final text generation. The decoder can select a draft chain greedily or by sampling. Output may differ from a run without a drafter because verification batches tokens and can change near-tied token choices; the source says code and tool calls usually match, while prose and image descriptions can diverge.
Getting started
git clone -b dspine https://github.com/Ajay9o9/llama.cpp.git cd llama.cpp cmake -B build -DGGML_CUDA=ON cmake --build build --config Release -j ./build/bin/llama-server \ -m Qwen3.8-27B-Q4_K_M.gguf \ -md Qwen3.8-27B-DSpine-b16-Q4_K_M.gguf \ --spec-type draft-dflash \ --spec-draft-n-max 15 \ -ngl 999 --spec-draft-ngl 999 \ -fa on --jinja -c 32768 -np 1 \ --cache-type-k q8_0 --cache-type-v q8_0 \ --spec-draft-type-k q8_0 --spec-draft-type-v q8_0 The example is the documented CUDA setup. Adjust --spec-draft-n-max and -c for your GPU and memory; the source recommends testing draft sizes such as 2, 3, 4, 7, and 15.
Frequently asked questions
Q: Can I use this as a standalone model?
A: No. These files contain only the draft model; pair one with a Qwen3.8-27B target GGUF.
Q: Will stock llama.cpp load the files?
A: No. The supplied README says they require the dspine branch of the Ajay9o9 llama.cpp fork; stock llama.cpp refuses to load them.
Q: Which quantization should I start with?
A: The README recommends Q4_K_M, whose code-bridge tensors remain BF16. Q8_0 is larger and uses more VRAM, while BF16 is described as a full-precision file for re-quantizing.
Q: What draft length should I set?
A: The documented default of 3 uses only part of the 16-token block. The source suggests trying 2, 3, 4, 7, and 15; 15 is a starting point for CUDA, ROCm/HIP, and SYCL, while AMD Vulkan may favor 2 or 3.
Q: How much context fits on a 24 GB GPU?
A: In the reported RTX 3090 setup with a Q4_K_M target and q8_0 KV cache, the maximum context was 65,536 with 15 drafted tokens and 98,304 with 7. Actual capacity depends on the setup.
Q: Does speculative decoding preserve identical output?
A: Not necessarily, even at temperature 0. Batched verification can change near-tied token choices; the source reports that code and tool calls usually match, while prose and image descriptions can diverge.
Q: What does the supplied research abstract establish?
A: It reports DSpine research results on Qwen3-4B and Qwen3-8B, including SGLang serving tests. It does not establish those figures as benchmarks for this Qwen3.8-27B GGUF conversion.
This is a simplified guide to an AI model called Qwen3.8-27B-DSpine-b16-GGUF maintained by aj9o9. If you like these kinds of analysis, join AIModels.fyi or follow us on Twitter.
