Qwen 3.8 27B: Run It Locally on One GPU with LM Studio

Qwen 3.8 27B is an Apache-2.0 dense model with a vision encoder, released 14 August 2026. Dense matters here: it isn't mixture-of-experts, so all 27 billion parameters participate in every token. The card documents 64 layers, a hybrid Gated DeltaNet and Gated Attention architecture, and multi-token prediction training.

It runs on one GPU. Whether it runs well depends on choices that have nothing to do with the model file, because the file is only part of the memory bill — context, KV cache, runtime overhead, and the vision projector all compete for the same VRAM.

Exact file sizes, and what they imply

Weights measured from the LM Studio community and Unsloth repositories. The hardware column is a derived starting point, not an official requirement from anyone.

Unsloth's community guide estimates 16–19 GB combined RAM and VRAM for 4-bit, 23–26 GB for 6-bit, 31 GB for 8-bit, 56 GB for BF16. LM Studio lists a 16 GB minimum system memory figure for the model but publishes no official VRAM-by-quant table.

The practical read: a 16 GB card runs this model, but the clean path is UD-Q3_K_XL or smaller with a modest context. Q4_K_M's 16.81 GB file already exceeds the card before cache and runtime, so it needs partial CPU offload — which works, and costs you speed through data movement.

Q4 is the sensible evaluation default on 24 GB and on Apple Silicon with enough unified memory. GGUF for Windows, Linux and macOS through llama.cpp runtimes; MLX only on Apple Silicon. Two- and three-bit quants genuinely open up smaller hardware, but test instruction following, code edits, tool calls and domain terminology before accepting the trade.

Setup, in an order that avoids surprises

1. Update LM Studio and its runtime — lms runtime update. Older llama.cpp or MLX builds may reject the architecture outright.
2. Estimate before loading: lms load --estimate-only <model> --context-length 32768 --gpu 1.0. The estimator accounts for context, GPU offload, Flash Attention, and vision models.
3. Set context explicitly. Start at 16,384 or 32,768. Don't rely on the app's first-load default, which changes between builds, and don't treat the model's 262,144-token native limit as a local default — that's a model capability, not a promise about your workstation.
4. Lower the reasoning effort. The card lists xhigh as default. Use medium or low for routine coding and disable thinking where the task doesn't benefit.
5. Check the vision projector if you need images. The community GGUF repo ships a 0.93 GB BF16 mmproj, and LM Studio treats a matching mmproj in the same folder as the projector. If the catalogue download omitted it, put it beside the GGUF before loading.
6. Start the server: lms server start --port 1234, then curl http://localhost:1234/v1/models and use the returned ID rather than guessing it. Step 4 is the one people skip and then blame their hardware for. Simon Willison's August tests make the case: an 8,192-token context got consumed by xhigh thinking, and one SVG task took 21 minutes at xhigh against 137 seconds with reasoning disabled. One practitioner's observation rather than a benchmark, but it's a strong argument for turning the dial down before buying a bigger GPU.

Four checkpoints before you connect anything with file access:

Wiring Qwen Code to it

Qwen Code is the open-source terminal coding agent; Qwen Coder is a model family. Different products, often conflated. The CLI needs Node.js 22+.

npm install -g @qwen-code/qwen-code@latest 

Then a provider entry in ~/.qwen/settings.json pointing baseUrl at http://localhost:1234/v1, with the id set to the exact value /v1/models returned and contextWindowSize matched to the context you actually loaded. That last detail matters: the fallback for an unknown alias can exceed a realistic local allocation.

Qwen Code sends tool definitions on every request, so the local server and model template have to support tool calling. Run /model to select the LM Studio entry, /status to confirm the version, then point it at a disposable test repository and watch a harmless tool call before widening permissions. A local coding agent carries different operational risk from a chat window — check patch accuracy, command selection, and recovery from tool errors first.

What the benchmarks do and don't show

Qwen reports 73.0 on Terminal Bench 2.1, 61.7 on SWE-bench Pro, 90.3 on LiveCodeBench v6, and 89.2 on GPQA Diamond. The card notes most agentic evaluations used Claude Code at 256K context and that several suites were developed in-house.

Those are the vendor's numbers. I found no independent reproduction of those suites for this model. Willison's work is a useful practitioner check on latency and reasoning behaviour — he reported roughly 15–30 tokens/second through LM Studio on both a 128 GB M5 Max and a DGX Spark, which he attributes to the memory-bandwidth demands of a dense architecture — but it isn't a reproduction of the benchmark table.

Decide from a fixed evaluation set reflecting your codebase, tools, latency target, and acceptable failure rate.

When not to run it locally

Local shifts spend from per-token fees to capital cost, power, updates, security patching, capacity planning, monitoring, and time debugging runtime changes. Go hosted when usage is intermittent and the GPU would idle, when you need elastic concurrency or low latency across regions, when nobody owns quant validation and runtime upgrades, or when the local quant fails your quality threshold.

Go local for sustained utilization, data-boundary requirements, offline operation, or predictable capacity. A hybrid keeps sensitive workflows local and bursts to hosted for larger contexts.

One thing local inference doesn't do: create compliance. Prompts and outputs staying on your infrastructure tightens the data boundary. A regulated deployment still needs access control, audit logs, retention rules, model risk review, security testing, incident response, and evidence the system behaves within policy.

Full guide — the complete quant table with sources, the LM Studio steps, and the Qwen Code config: teamvoy.com/blog/qwen-3-8-27b

Written by Zhanna Yuskevych, CPO at Teamvoy. More engineering writing at teamvoy.com/blog.