Field card: Kimi K3

What it is

Kimi K3 is Moonshot AI's open-weight flagship, introduced 2026-07-16 with full weights published 2026-07-27. It is a mixture-of-experts model: enormous in total, comparatively small per token. Most facts in this table were read directly out of the model files on our disk, which makes them properties of the artifact rather than quotes from a press release; rows that rest on the maker's card say so.

File-derived rows were read from the pinned GGUF set on 2026-08-08 by parsing the shard headers only; no model data was loaded to produce them. Card-derived rows are the maker's claims, and the amber row is a maker's stated value we did not measure.

What we ran, exactly

A quant is a compressed build of a model's weights: lower bit counts shrink the file, at some cost to fidelity that we do not attempt to score here. The artifact we ran is Unsloth's UD-IQ1_S build of K3 from their Kimi-K3-GGUF repository: 14 files totalling 594.00 GB (553.20 GiB; the two units count the same bytes by thousands and by 1,024s). Every file's size was re-checked against the repository's published manifest, 14 of 14, on 2026-08-08; the SHA-256 checksums were verified at download against the repository's records. Despite the 1-bit name, only the routed experts are 1-bit: a census of every tensor in the set counts 276 IQ1_S tensors (the experts, 495.26 GiB of them) against 1,116 eight-bit and 1,181 full-precision tensors for everything else. The build is calibration-quantized (an importance matrix guided which precision goes where); Unsloth publishes its own quality analysis for it, which we have not reproduced and do not restate as fact.

This is the second quant generation this machine has run, and the two must not be mixed. Numbers on this page are labeled where the generation matters.

The machine: one NVIDIA GeForce RTX 5090 with 32,607 MiB of VRAM, an Intel Core Ultra 9 285K (24 cores), 188 GiB of system RAM plus 79 GiB of swap, and the weights on a consumer 4 TB NVMe SSD. The runtime: llama.cpp built from the open Kimi K3 support branch (as of 2026-08-08 no released llama.cpp supports K3; support lives in open pull request #26185), at commit 7b990cf5, CUDA 12.8. The full serving flags appear in section 07, because half of them are traps.

The fitting problem, solved on paper first

The expert pool never goes near the graphics card: 495 GiB of 1-bit experts plus 12 GiB of shared experts live on the CPU side and stream from disk on demand. What has to fit on the card is the dense spine: attention, embeddings, norms, the output head. And here the calibrated build created a problem the old one never had. The previous build's 4-bit spine fit on the card. The current build upcasts the spine to 8-bit, and that spine is 45.91 GiB, larger than the entire card. The build gained calibration and an 8-bit spine, and stopped fitting on the same day.

The fix was decided arithmetically before anything loaded. A GGUF file's headers state every tensor's type, shape, and size, so the whole placement problem can be read off the file in seconds: the spine averages 0.469 GiB per block, and the arithmetic says the card can hold the first 50 blocks plus the embeddings and output head, 26.24 GiB. So blocks 50 through 92 joined the experts on the CPU side, expressed as one tensor-override pattern:

-ot "exps=CPU,shexp=CPU,blk\.(5[0-9]|[6-8][0-9]|9[0-2])\.=CPU"

The first attempt with this placement loaded healthy: no crash, and about 29.9 GiB of VRAM in use as the load session recorded it (29,884 MiB; no telemetry capture survives for that run), leaving about 2.7 GiB of headroom. Two details make the placement less obvious than it looks. First, the cut had to come from the tail: moving the FIRST block to the CPU trips a fused operation in the linear-attention path and aborts the load outright, so offloading is by explicit tensor name, never by lowering the GPU layer count. Second, the placement guarantees that no 1-bit tensor is ever on the GPU: the card holds 453 eight-bit and 639 full-precision tensors and nothing else, which matters because of the first trap in section 07.

What actually limits it

On the previous quant generation, two fully instrumented runs showed the graphics card nearly asleep during generation: GPU utilization held a median of 1 to 3 percent, never passed 7 percent, and the card never warmed past 52 degrees. What was busy was the SSD: a median of 463 to 490 MiB/s of sustained reads, hour after hour, with peaks near 2 GB/s, while the CPU spent a median 12 percent of its time waiting on I/O. System RAM never came close to running out; the model simply flows through the page cache (the part of RAM the operating system uses to hold recently read file data), about 167 GiB of it, and the rest streams from disk every token. On this machine, with 188 GiB of RAM against 553 GiB of weights, generation is storage-bound: a storage benchmark wearing a language model's clothes.

Two honest boundaries on that finding. The current build has not been re-instrumented; it shows the same speed band, which is consistent with the same bottleneck, and no more than that. And the conclusion is about this class of machine, not a law of nature: with enough RAM to hold the whole expert pool, reports elsewhere (section 09) show several times this speed, because the bottleneck moves.

A practical corollary worth stating: the desktop stayed fully usable the whole time. The model consumes disk bandwidth and page cache, not cores and not the GPU, so the machine does not feel occupied while K3 grinds away in the background.

The speed, with nothing softened

Sources: per-token timings from the server's own logs for both generations; the four recorded probe files; one instrumented 1,315-token coding answer. Speeds are end to end as a user would experience them. The recorded completions number under ten across both generations; this is a field measurement of one configuration, not a benchmark of the model.

Three things the table understates. Prompt processing running at decode speed is the quiet killer: on ordinary local models the prompt side runs hundreds of tokens per second, and here it does not, so a long prompt costs minutes per hundred tokens before the first output token appears. That is why section 06's usage advice exists and why we cap context at 8,192. Longer answers drift toward the slow end of the band: the two-hour answer averaged 0.19 where short answers touch 0.28. And the swap to the calibrated quant cost nothing measurable in speed: on a storage-bound machine, a bigger, calibration-quantized artifact came at no speed price. What that calibration buys in answer quality we did not measure and do not claim.

One more timing that fools monitoring: the server process reports itself started within about two seconds, and the model becomes able to answer fourteen minutes later. Anything that trusts "started" will spend a quarter of an hour talking to a wall.

How you actually use a 0.2 token-per-second model

You do not chat with it. On this architecture, reusing a cached conversation prefix can corrupt the linear-attention state (a documented upstream caution we chose not to test), so every request here is sent as a fresh, self-contained, single prompt. That constraint turns out to be the right mental model anyway: using K3 at this speed is closer to posting a letter than to typing in a box. You write one complete, careful prompt, you submit it, and you come back later. The right interface is a job queue and patience, not a blinking cursor.

Two settings follow directly from measurement. First, every recorded completion produced a reasoning block before its answer, and the embedded chat template carries reasoning scaffolding, so treat thinking-first as the working default. That reasoning is paid from the same output budget the client sets, so a small budget is consumed entirely by thinking and the visible answer comes back empty. Give real jobs 2,000 to 4,096 output tokens; across the measured band the 4,096 ceiling is about four to six hours, which is the honest cost of asking this model for a long answer. Second, keep the context modest: we run 8,192, because prompt processing at 0.2 tokens per second makes large contexts cost hours before the first token, and because nothing larger has been tested on this architecture's hybrid attention here.

What is it for, at that price? One bounded question where you want what a full-topology K3 can give you from hardware you own, and you can afford to sleep on it. In recorded probes on the previous quant generation it followed a byte-exact instruction, reasoned a classic riddle to the correct answer with a correct explanation, and wrote a small function with tests that pass when executed; the current build's own recorded successes are shorter (section 08 draws that boundary). Not iteration, not agents, not a code-review loop. The plain conclusion: running a model of this scale at home is gated less by owning a datacenter than by disk capacity, RAM, and patience.

The traps, so you lose hours instead of days

Everything below was measured here or designed around from a documented report, and each one costs real time if met unprepared.

• On this GPU generation, 1-bit tensors must never touch the card. The 1-bit CUDA kernels are reported broken on this card generation (silent wrong numbers, no error). We never place them on the GPU, and the current build's design helps by accident: everything it routes to the card is 8-bit or full precision, so the broken kernels are never reached.
• --no-repack is mandatory. Without it, the loader tries to build a RAM copy of the 495 GiB expert pool and dies.
• Offload by tensor name, never by layer count. Letting the first block land on the CPU aborts the load against a fused linear-attention operation; trailing blocks move freely (43 of them, in our placement). Auto-fit can quietly lower the layer count into the same abort, so we pin -ngl 99 --fit off and control placement entirely with the override pattern in section 03.
• Do not raise the prompt batch size. A batch of 8192 asked for 9,715 MiB of extra VRAM and failed with out-of-memory twelve minutes into the load (measured on the previous generation; today's placement has only about 2.7 GiB free, far short of that buffer, so the same failure is expected and has not been re-run). The failure is at least loud.
• Ignore the loader's advice to turn off memory mapping. Every load prints a suggestion to use --no-mmap for better performance. On a machine where the model is three times the RAM, following it would mean asking for 594 GB of memory. Streaming from disk is not a compromise here; it is the entire method.
• "Started" is not "loaded". Wait on the health endpoint, not the process. The gap is about fourteen minutes.
• Verify downloads by count and checksum, not by exit status. One of our fourteen-shard downloads reported success with a shard silently missing after a transient authentication hiccup. Count files against the repository's manifest before a fifteen-minute load tells you the hard way.
• Read the headers before you load. Every tensor's size and type is in the GGUF headers, readable in seconds without touching the tensor data. The whole 45.91 GiB spine figure and the block-50 cut point came from header arithmetic, not from failed load attempts.
• One warning is noise. Every load prints an end-of-sequence token warning (special_eos_id is not in special_eog_ids); every recorded completion nonetheless terminated cleanly with finish_reason: stop.

The full serving flag set for the current placement, for anyone reproducing this class of run:

llama-server -ngl 99 --fit off -fa on --jinja --no-repack --no-warmup \
  -ot "exps=CPU,shexp=CPU,blk\.(5[0-9]|[6-8][0-9]|9[0-2])\.=CPU" \
  --ctx-size 8192 --parallel 1 --cache-reuse 0 --temp 1.0 --top-p 0.95

What we ran, and what we have not

The recorded evidence behind this page: two instrumented load-and-generate sessions with five-second telemetry (memory, VRAM, GPU utilization, disk throughput, temperatures); the completions across the two quant generations with full request, response, and server timings kept; one end-to-end job record with its server timings; the out-of-memory refutation above; and a header-level scan of all fourteen shards that re-derived the parameter counts, tensor census, and placement arithmetic on 2026-08-08.

Still not verified, stated plainly:

• Tool calling: never attempted. The embedded chat template carries extensive tool scaffolding, so the machinery exists, but there is not one recorded tool call from this machine.
• Vision: not possible here. Our file set is text-only; the multimodal claim belongs to Moonshot's card, not to anything we ran.
• Multi-turn conversation: deliberately avoided for the state-corruption caution in section 06. We have no evidence it breaks and none that it works.
• Context above 8,192: untested. The file declares a million-token window; nothing here speaks to it, for or against.
• No benchmarks and no quality verdict. The probes establish that the model generated coherent, task-correct text in those recorded cases. They do not validate the quantizer's quality claims, do not compare the two quant generations on quality, and do not rank K3 against anything.
• One backend, one machine, one request at a time. llama.cpp on the branch commit named above, a single desktop, and a single serving slot. No other runtime, no concurrency, and no power-draw measurement.
• Several previous-generation results are assumed, not re-run, on the current build: the batch-size refutation, context 4,096, the executed-code probe, and the two-hour sustained answer. The current build's own record is three completions, the longest 120 tokens, plus its healthy instrumented-free load.

Elsewhere in the field

Running K3 on modest hardware is a small genre now, and honesty about it makes our numbers more useful, not less. Public complete-topology runs we know of: a pure-C reader streaming the official weights on a GPU-less workstation (kimi-k3-in-c), expert-streaming runs on 64 GB MacBooks (Deltafin, WASTE), a Ryzen desktop with an expert cache (Colibri), a DGX Spark streaming this same Unsloth quant family (Pulsar), and notably an RTX 5090 run of the official weights in late July (Deltafin issue 6), which is why this page makes no first-on-a-32-GB-GPU claim. Each project reports its own speeds, on setups too different for the numbers to be comparable; follow the links for theirs.

What this page contributes to that genre is not the feat but the measurement: a pinned, checksum-verified artifact; placement decided from file-header arithmetic and published as one reproducible override pattern; phase-separated telemetry that locates the bottleneck; and speeds reported end to end with their failure modes. If you run one of the setups above, their numbers are theirs and ours are ours; none of them transfer across configurations, which is rather the point of measuring.

Sources

• The model: huggingface.co/moonshotai/Kimi-K3 (card, weights, and the Kimi K3 License), with the technical repository alongside. Introduced 2026-07-16; weights published 2026-07-27.
• The artifact we ran: unsloth/Kimi-K3-GGUF, the UD-IQ1_S build, 14 files, 594.00 GB (553.20 GiB). Each file's size was re-checked, 14 of 14, against the saved repository manifest on 2026-08-08; the SHA-256 checksums were verified at download against the repository's records. Unsloth's own guide and quantization analysis are the source for any quality claim about the build; none is restated here as ours.
• The runtime: llama.cpp with Kimi K3 text support from open pull request #26185, built at commit 7b990cf5 with CUDA 12.8.
• Every measured number stands on dated project records: two telemetry captures at five-second resolution, the four probe response files, the end-to-end job record, the server logs for both quant generations, the saved shard manifest, and the 2026-08-08 header-scan results. These are project records rather than public files today; they are named here so that every number on this page has a dated, recorded home that can be produced if questioned.