Which Qwen should I run?
Qwen is an unusually wide open model family: real Apache-licensed weights from under one billion parameters to a 397B mixture-of-experts, plus API-only frontier tiers above that. That width is the point of this page. We run the family for real on one desktop, from the small daily drivers to the 122B and 397B, so the guidance here is what we measured, labeled as measured, next to what the vendor claims, labeled as claims. Find your hardware in the chooser, install your first one with two commands once Ollama is on your machine, take the serving profiles we actually run, and know when the honest answer is "use the API instead."
The family on one screen
Four generations matter today. The one-line version: 3.5 and 3.6 are the open ones you can run; 3.7 is API-only; 3.8 is API-only today with open weights promised imminently.
| Qwen3.5 (open) | The current big open generation: mixture-of-experts models topping out at the 397B and the 122B we serve here, Apache 2.0, with built-in speculative-decoding heads. Our measured deep-dive is the Qwen3.5 pair field card. |
|---|---|
| Qwen3.6 (open) | The newest open weights: a 27B and a 35B mixture-of-experts, Apache 2.0, files verified on the official organization. Our daily drivers below come from this generation and its predecessors. |
| Qwen3.7 (API only) | Max, Plus and Flash tiers on the official API and aggregators; the vendor states they are proprietary. No 3.7 weights have ever shipped. |
| Qwen3.8 (API today, weights promised) | The Max is live as an API. Open weights for it and a new 27B were promised for the week of 2026-08-10; as of 2026-08-09 nothing has shipped. Verified status, first API measurements, and the fake-repository warning live on our Qwen3.8 release watch page. |
Generation status checked 2026-08-09. The release-watch page carries the dated table and updates as things actually ship.
The chooser: what your machine can actually run
Rows marked measured come from recorded runs on our machine (a single RTX 5090 with 32 GB of VRAM and 188 GiB of system RAM; dates given). Rows marked arithmetic are file-size reasoning we consider sound but have not run. Rows marked vendor are the maker's sizing. Speeds are warm generation, one configuration each, not benchmarks.
| Your hardware | What we would run | Speed seen | Basis |
|---|---|---|---|
| Laptop or desktop, no real GPU | The sub-1B and 2B-class tags as utilities and pipeline testers; anything bigger belongs to the API | see basis | transferred: the 0.6B measured 857 t/s on our 32 GB GPU (July 2026); CPU-only speed untested here, but this size class runs acceptably on CPUs generally (vendor and community consensus, labeled as such) |
| 8 to 12 GB VRAM | The 4B and 9B-class tags at 4-bit | untested here | vendor sizing; we have not run these sizes and say so |
| 16 GB VRAM | A 14B-class at 4-bit comfortably; the 27B at 4-bit with partial CPU offload | untested here | arithmetic from file sizes |
| 24 GB VRAM | The Qwen3.6 27B at 4-bit (a 17 GB download) | see basis | arithmetic; measured only on our 32 GB card (72 tokens per second, July 2026), where it never had to squeeze |
| 32 GB VRAM | The sweet spot we live on: the 35B mixture-of-experts as the fast generalist, the 27B, and a 30B-class vision model, all fully on GPU | 227 / 72 / 291 t/s | measured, July 2026, Ollama serving (qwen3.6:35b / qwen3.6:27b / the 30B-class vision tag, in that order) |
| 32 GB VRAM + roughly 96 GB or more RAM | The Qwen3.5 122B: every layer on GPU, expert blocks walked to CPU, gated speculation on | 26 to 38 t/s | measured, 2026-08-05/06, llama.cpp serving on a 188 GiB machine; full story on the field card |
| 32 GB VRAM + roughly 176 GB or more RAM | The Qwen3.5 397B, same technique, more patience | 14 to 20 t/s | measured, 2026-08-05/06, same 188 GiB machine; low end is thinking replies |
| None of the above fits | Hosted Qwen: the same open weights served by API providers, or the official frontier tiers | n/a | section 06 |
The RAM rule behind those floors: at the quantizations we run, the 122B's weights are a 73 GB file and the 397B's are 150 GB, and the technique copies weights into RAM, so the weights plus operating-system headroom must fit. We measured both models only on a 188 GiB machine. With less RAM than the weights, the model streams from disk instead, which we did not measure here and which is dramatically slower. Quantization tier changes everything; ours are named on the field card.
Your first Qwen, without being an infrastructure person
The short path is Ollama. One hardware gate first: the model below is a 17 GB download that wants roughly 20 GB or more of VRAM to run fully on GPU. If your card is smaller, pick a smaller tag from the chooser above, or read section 06. Then install Ollama from the official installer for your platform, and:
ollama pull qwen3.6:27b ollama run qwen3.6:27b
Two commands, and that is genuinely it for a first conversation. Two things will save you the hours they cost us:
First, the empty-answer trap. Modern Qwens think before answering, and the thinking spends the same output budget as the visible reply. If your client caps output around 500 or 1,000 tokens, a hard question can consume the whole cap thinking and hand back nothing, which looks exactly like a broken install. It is not broken. Set the output limit to 4,096. We measured this same behavior all the way up at the API frontier (the release watch has the table).
Second, sampling. For thinking-family Qwens use temperature 0.6, top-k 20, top-p 0.95. The values are in the first profile below so you do not have to remember them.
You graduate from Ollama to llama.cpp's llama-server
when you want a model bigger than your VRAM: the technique that puts
a 122B on a 32 GB card is a llama.cpp flag set, and the second
profile below is its shape.
Known-good profiles, downloadable
Three small plain-text files, the settings we actually run, commented so you know why each value is what it is.
- ollama-qwen-thinking.txt: the parameter set for thinking-family Qwens under Ollama, including the output-budget floor that avoids the empty-answer trap.
- llamacpp-qwen35-moe.txt: the llama-server shape for big Qwen3.5 mixture-of-experts models on one GPU, including the expert-offload knob and the speculation confidence gate our field card measured (ungated speculation ran slower than none on creative text, measured on the 122B; we keep the same gate on the 397B on that evidence; the gate is one flag, off by default in the builds we used).
- opencode-local-qwen.json: a provider block that points the OpenCode coding agent at your local Qwen, either serving route.
The profiles encode our recorded configurations generalized to standard ports and paths. They are starting points, not guarantees; your quantization, context size, and card will move the numbers.
The big ones, on one desktop
The reason this page can speak about 122B and 397B models in the first person: both serve on the single 32 GB card described above, using expert offload and the models' built-in speculation heads with the confidence gate set. The 122B runs at 26 to 38 tokens per second depending on prompt shape, the 397B at 14 to 20, and the complete measured story, artifact hashes to placement arithmetic to the gate trap, is the Qwen3.5 pair field card. When the promised Qwen3.8 weights land, the same technique is the first thing we will test, and the release watch page says exactly how.
Local or hosted, honestly
Hosted Qwen is startlingly cheap. As of 2026-08-09, aggregator pricing put the same 397B we run locally at roughly fifty to sixty cents per million input tokens and about three and a half dollars per million output; a heavy personal month costs single-digit dollars. The official API's flash tier is cheaper still, with a million-token context. Local never wins that arithmetic.
What local buys instead: your prompts and documents never leave the machine; it works with the network down; nobody deprecates, throttles, or silently revises your model; and you learn more about how these systems actually behave than any API will teach you. Our rule of thumb: privacy-sensitive or high-volume-and-predictable work runs local; frontier-quality one-offs and anything needing a million-token context run hosted. A same-task local-versus-hosted study on identical weights is in progress and will get its own page.
Prices observed 2026-08-09 and they change; check the provider's live page before planning around them.
What this page does not know
- We have not run the 4B-to-14B-class sizes on small GPUs; those rows are sizing arithmetic and vendor claims, labeled as such.
- Our speed figures are one machine, one quantization tier, one serving configuration each, at the dates given. They are honest samples, not benchmarks.
- The July 2026 daily-driver numbers predate our current llama.cpp serving work; a fresh measured pass on the 27B and 35B is planned and this page will update with dated entries.
- Vision, audio, and long-context behavior are not covered here; the field card states exactly what long-context ground the big pair has actually been tested on.
Updates and sources
| 2026-08-09 | Page created. Generation status, pricing, and tag names checked that day; chooser speeds carry their own dates. |
|---|
If today is later than the newest row above, treat the generation-status and pricing lines as unverified since that date; the release watch carries the living Qwen3.8 record.
- The Qwen3.5 pair field card: dated recorded runs behind every 122B/397B number here.
- The Qwen3.8 release watch: verified generation status and the API measurements.
- July 2026 daily-driver speeds: our recorded Ollama benchmark pass (project records that can be produced if questioned).
- Official Qwen organization pages for licenses and file sizes, checked 2026-08-09; aggregator pricing pages for section 06, observed 2026-08-09.