AI & ML
The Local LLM Weight Classes, September 2026: What Actually Fits on Your Machine
Kirill Lukyanov Dev.to (EN Zone)
2 views
A $20,000 Mac Studio buys you the same number of tokens as $20,000 of cloud API credit. The difference is that the cloud hands them over on demand, and the Mac needs fourteen years of uninterrupted generation to produce them.
That number is the reason I stopped sorting open-weight models by benchmark score and started sorting them by what physically fits. Here is the September 2026 snapshot — with a date stamped on it, because half of these figures will be stale by December.
The map changed, and it moved east
By May 2026, Chinese models accounted for roughly 61% of all tokens routed through OpenRouter. That share grew on a market that quadrupled in the same period — the router went from about 5 trillion tokens a week in April 2025 to more than 20 trillion by April 2026.
On Hugging Face the picture is starker. Qwen crossed one billion cumulative downloads and passed Llama. There are over 200,000 Qwen-tagged models on the hub, and roughly 40% of every new LLM derivative created there is built on a Qwen base.
Vendor
Share of routed tokens
Xiaomi (MiMo)
~21%
DeepSeek
~17.6%
Anthropic
~15.4%
Google
~13% (down from 37%)
Meta
<1%
Meta invented open weights as a competitive strategy in 2023. In September 2026 its routing share is under one percent. More on that below.
The ranking, for context
Current aggregate standing of open-weight models (BenchLM, September 2026):
#
Model
Vendor
Index
Context
1
Qwen3.8 Max
Alibaba
72.4
1M
2
GLM-5.3
Z.ai
68.5
1M
3
Qwen3.8-27B
Alibaba
68.4
262K
4
GLM-5.2
Z.ai
68.2
1M
5
GLM-5.3-Flash
Z.ai
66.1
1M
8
Kimi K2.7 Code
Moonshot
65.4
256K
13
MiniMax M3
MiniMax
63.9
1M
Look at row three. Qwen3.8-27B is a dense 27-billion-parameter model under Apache 2.0, and it sits four index points behind the leader. The leader is a 2.4-trillion-parameter network you cannot run anywhere except a server rack. Third place fits inside a single gaming GPU.
That collapse — from "the good models are unreachable" to "four index points" — is the actual story of the year.
Weight class 1: laptop, 8–16 GB
Model
Weights (Q4)
Fits in
Good for
Phi-4-mini
~3.5 GB
8 GB, CPU-only
logic, math, weak machines
Gemma 4 E4B
~6 GB
16 GB (~12.5 GB with full 128K context)
best generalist at 16 GB
Qwen3 8B
~6 GB
16 GB comfortably
chat, document work
Qwen3 14B
~9 GB
12–16 GB VRAM
code, refactoring
The trap everyone falls into: model weight is not total memory use. Gemma 4 E4B at full 128K context roughly doubles its footprint, from ~6 GB to ~12.5 GB. If your math says "just barely fits," it doesn't — the KV cache grows with the conversation and you will hit the wall mid-task.
Weight class 2: workstation, 24–64 GB
This is where the interesting thing happens.
Qwen3.8-27B in Q4_K_M needs 16.5–19.5 GB of weights plus about 1.5 GB of runtime overhead. That fits entirely inside an RTX 3090 or 4090 — cards that are plentiful on the used market. Native vision-language, 262K context, Apache 2.0 with no conditions attached. The base repo pulled 415,000 downloads within days of the August 14 release.
On Apple Silicon, a 64 GB Mac mini M4 runs it at roughly 7 tokens/sec. Not fast, but usable for non-interactive work.
Also in this class: Qwen3-Coder-Next 80B-A3B (sparse MoE, 3B active — the most realistic self-hosted coding model in the current lineup) and gpt-oss-120b at ~65 GB in MXFP4.
If you buy one machine for local inference, buy for this class. 32 GB is the floor, 48 GB is the working sweet spot, 64 GB is where you stop negotiating with your own context-length settings.
Weight class 3: one big machine, 96–512 GB unified
Sparse MoE models need enormous memory but only read a fraction of it per token. That maps almost perfectly onto Apple Silicon: a huge memory pool with bandwidth that is modest by datacenter standards but irrelevant when your bottleneck is capacity.
Apple refreshed the Mac Studio on August 25: M5 Ultra, up to 36 CPU cores, 80 GPU cores, and up to 512 GB of unified memory (the 512 GB configuration ships in late October). The 96 GB base is $5,499; the 256 GB upgrade adds another $4,000.
Model
Size
Speed
Memory
DeepSeek V4 Flash (284B-A13B)
2-bit
~39 tok/s
128 GB (M5 Max)
gpt-oss-120b
~65 GB MXFP4
~43 tok/s
96 GB+
Llama 3.3 70B
~42 GB Q4
~14 tok/s
96 GB
Llama 4 Maverick 400B
~220 GB Q4
~12 tok/s
512 GB
Most of those figures are engine estimates — the machines have only just started shipping. The DeepSeek V4 Flash number is the exception: 39 tok/s on a 128 GB M5 Max is community-reported from real hardware. For a 284-billion-parameter model sitting on a desk, that is genuinely good.
Weight class 4: rack only
Kimi K3 from Moonshot AI: 2.8 trillion parameters, 896 experts with 16 activated per token, 1M context, weights published July 27, 2026. Formally, download and run. Practically:
MXFP4 weights are 1.56 TB — no single GPU and no single 8-GPU node holds them
realistic serving is a multi-node cluster with 1.6 TB+ of aggregate GPU memory; production recommendations run to 64+ accelerators
the floor for FP8 on 80 GB H100-class cards is about 40 GPUs, roughly five nodes
at $2–4 per GPU-hour, an always-on 40-GPU deployment runs $60,000–115,000 per month in compute alone, before networking, storage, staff, and power
Alongside it sit Qwen3.8-Max (2.4T total, 95B active; text weights opened August 12) and GLM-5.3 (753B total, 40B active; released August 14 — same base as GLM-5.2 with scaled-up post-training).
The sparsity trap. Kimi K3 activates 104B of 2.8T parameters per token. It is tempting to size your memory for 104B. Don't: the router can call any of the 896 experts at any moment, so the entire model must be resident. Sparsity saves compute, not memory.
What the same models cost in the cloud
Model
Provider
In / out per 1M tokens
GLM-5.3
AkashML
$1.17 / $3.96
GLM-5.3
Together
$1.40 / $4.40 (149 tok/s)
GLM-5.3
GMICloud
$1.40 / $4.40 (29 tok/s)
GLM-5.3
Venice
$1.75 / $5.50
Kimi K3
Makora
$2.55 / $12.75
Kimi K3
Together
$3.00 / $15.00
Kimi K3
Morph (Fast)
$6.00 / $22.50
Qwen3.8-Max
DeepInfra
$2.00 / $6.00
Qwen3.8-Max
Together
$2.50 / $6.25
DeepSeek V4 Pro
DigitalOcean
$0.87 / —
DeepSeek V4 Pro
Azure (US)
$1.91 / —
DeepSeek V4 Flash
DigitalOcean
$0.068 / —
DeepSeek V4 Flash
Cloudflare
$0.44 / —
Three things in that table cost real money.
Identical weights, wildly different prices. Kimi K3 spans 2.35× between cheapest and most expensive host. DeepSeek V4 Flash spans 6.5×. Same model, different hardware and margin.
Identical price, different service. Together and GMICloud both sell GLM-5.3 at $1.40/$4.40. Together delivers 149 tok/s; GMICloud delivers 29. That is 5.1× slower for the same money — the difference between "works" and "unusable" in anything interactive.
Advertised context ≠ served context. GLM-5.3 markets a 1,048,576-token window. IO Net and Reka serve it at 262,144 — a quarter. DeepInfra serves Qwen3.8-Max only in its native 262,144 window while Together offers the full 1,010,000. And DeepInfra caps DeepSeek V4 Pro output at 16,384 tokens against an advertised 384,000 — 4.3% of the marketing number. Verify the endpoint before you architect around long context.
The break-even math
A 512 GB M5 Ultra Mac Studio runs about $20,000 (one top configuration is orderable today at $18,299). Call its throughput 10–15 tok/s on a large MoE model.
GLM-5.3 costs $4.40 per million output tokens in the cloud. So $20,000 buys 4.5 billion output tokens.
At 10 tok/s running 24/7 with no idle time, the machine produces 864,000 tokens per day. To generate 4.5 billion it needs more than fourteen years. Push it to 40 tok/s and you still need three and a half years of continuous, never-idle generation.
General break-even estimates converge on 2–3 million tokens per day over a twelve-month horizon. That is a small company's sustained pipeline, not one engineer's usage.
The rule that follows: if you are calculating savings, don't buy. If you are calculating risk — data that cannot leave your perimeter, work without connectivity, the fear that the model your product depends on changes its behavior or price next quarter — buy, and stop running the spreadsheet.
Five stories from this year
The one-bit trillion-parameter model
Kimi K3 is 1.56 TB natively. Community dynamic GGUF quantizations got almost comical: the 1-bit build occupies 594 GB and retains 78.9% of quality, the 2-bit build (861 GB) reaches 90.4%, and the smallest (TQ1_0) is 509 GB. A 2.8-trillion-parameter model compressed roughly threefold and still working at four-fifths strength.
Except 509 GB is still a $20,000 machine, to run 79% of a model you can rent for $2.55 per million input tokens. Pure sport — and admirable as such.
Meta invented the strategy and lost it
Meta released Llama 5 on June 30 — 600B parameters, 5M context, genuinely competitive at the top for the first time in a while. But three months earlier, on April 8, it shipped Muse Spark, a closed model, and that is the flagship.
The lesson isn't about Meta. It's that openness stopped being anyone's strategy and became a property of the market: it is now sustained by whoever needs share, not by whoever believes in it.
"Open" covers three different legal regimes
Apache 2.0 — Qwen3.8-27B. Do anything, including commercial. No conditions.
MIT — DeepSeek V4 checkpoints. Same freedom.
Conditional — the Qwen3.8-Max License and Kimi K3's custom license carry conditions for large MaaS operators and commercial products. Read them before signing a customer contract, not after.
And universally: open weights are not open source. None of these vendors ship training data or the training recipe. You get an artifact you can run and fine-tune, not one you can reproduce.
An open model beat a closed one on a public benchmark
GLM-5.2 scored 62.1% on SWE-bench Pro against GPT-5.5's 58.6%. Not "nearly caught up" — ahead. With two caveats that matter more than the number.
Public benchmarks leak. Contamination, metric overfitting, and harness effects are all real, which is why everyone reports SWE-bench Verified (500 hand-curated tasks confirmed solvable with correct tests) and LiveCodeBench (contamination-resistant) in 2026.
And if the benchmark's harness, timeouts, repo size, and review process don't match yours, the score is a signal, not a decision. Your own eval on your own tasks remains the only ground truth.
The bottleneck turned out to be memory, not compute
The usual assumption is that compute is the constraint. In 2026 it's HBM: Chinese domestic output is capped around 250,000–300,000 high-end packages for the year. Logic manufacturing was solved; memory wasn't.
The same pattern shows up at consumer scale. What's interesting about a 512 GB Mac Studio isn't speed, it's capacity — which is exactly why MoE models landed on it so well.
And one prediction that aged badly: after DeepSeek-R1 shipped in January 2025, Nvidia dropped 17% and lost roughly $600B in market value, on the thesis that cheap models would crater compute demand. By September 2026 Nvidia is worth $5.14 trillion, up 50% year over year. Cheap tokens didn't reduce demand — they multiplied it.
Bonus: prices went up for the first time. DeepSeek raised V4 Flash from $0.14/$0.28 to $0.44/$1.32 at peak (half that off-peak). If your unit economics assume today's price, assume it can triple.
What I would pick
Laptop: Qwen3 8B or Gemma 4 E4B. Private document work, offline, anything that cannot leave the machine.
One workstation: Qwen3.8-27B in Q4_K_M. If you want a single local model for everything, this is it — Apache 2.0, multimodal, third in the world ranking.
Code and agents: GLM-5.3 via API. Best open coding results, sane price, decent throughput on Together.
High volume, low cost: DeepSeek V4 Flash. Its structural edge is cache pricing — if your pipeline re-reads the same context (RAG, an agent with a fixed system prompt) it wins by a wide margin.
Maximum quality: Qwen3.8-Max via cloud. Locally, forget it — that's a rack.
One rule on top of all of it: don't pick a model against a benchmark, pick it against a constraint. Answer honestly which binds harder — gigabytes, dollars, latency, or the requirement that data never leaves your perimeter. The model then chooses itself, and it usually isn't the one on the top line.
Open weights stopped being a compromise. They became inventory: a warehouse where everything is free, and the only question is what you can physically carry out.
All figures as of September 6, 2026. Sources: BenchLM open-weight ranking, the Digital Applied provider census for pricing and served context, Data Gravity for market share, Apple Newsroom for Mac Studio specs.
Originally published at klukyanov.ru.
Read original: https://dev.to/klukyanov/the-local-llm-weight-classes-september-2026-what-actually-fits-on-your-machine-128d
← Previous
Is History Repeating Itself? From the PC Era to the AI Era: A Developer’s Take on the Hype and Healthy Skepticism
Next →
How to monitor and maintain Laravel apps for multiple clients
Related
The Next RAG Problem Isn’t Retrieval — It’s Knowing When Not to Retrieve
AI & ML
0
Dev.to (EN Zone)
Agent Toolkit for AWS in Practice (1) - Claude Code
AI & ML
0
Dev.to (EN Zone)
RAG vs Memory vs Tools: What Information Should an AI Agent Actually Store?
AI & ML
0
Dev.to (EN Zone)
n8n: When AI Writes the Workflow, Who Reviews the Workflow?
AI & ML
0
Dev.to (EN Zone)
Comments0
No comments yet — be the first