> ## Content Index
> Fetch the complete content index at: https://globalfeed.ai/llms.txt
> Use this file to discover other available public pages before exploring further.

# Kimi K3 ran on a single CPU in 8 GB of memory, by streaming from disk instead of fitting
- URL: https://globalfeed.ai/en/kimi-k3-ran-on-a-single-cpu-in-8-gb-of-memory-by-streaming-from-disk-instead-of-fitting/
- Published: 2026-09-10T14:07:35.000Z
- Updated: 2026-09-10T14:07:35.000Z
- Description: An open-source engine runs Kimi K3 on a single CPU in 8.24 GB of measured RAM. The model did not shrink; the way it is loaded changed. The price is 26.5 seconds per token.
- Author: GlobalFeed Editor
- Tags: elle, Kimi, open source, local ai, x-Kimi_Moonshot, dil-en, video

An inference engine has been released that runs Kimi K3 on **a single CPU, with no GPU, in 8.24 GB of measured peak memory**. The whole engine is written in portable C99 and compiles to a single 176 KB binary. No BLAS, no framework, no CUDA. It is licensed under Apache 2.0.

The number looks impossible at first. The weights of a 2.78-trillion-parameter model do not fit in 8 gigabytes, and indeed they do not. What the project changes is not the model's size but **the way it is read**.

## The memory ceiling was a property of loading, not of the model

Kimi K3 is a mixture-of-experts model: only a small share of the weights is used at each step. The engine leans directly on that. The dense trunk stays resident in memory, while **1.45 TB of routed experts remain on disk** and are read from NVMe through `O_DIRECT` calls that bypass the operating system's cache. An LRU cache sits in front of the ones used most often.

The interesting consequence is that memory stops being a threshold and becomes a dial. Whatever RAM you have, the same model returns **byte-identical answers**. Only speed moves:

- **26.5 seconds per token** in 8 GB
- **5.6 seconds per token** at 128 GB and above

That is minutes for a sentence and half an hour for a paragraph. This is a demonstration, not a product.

## Where the cost lands

What is saved in memory is paid in storage: the setup needs a **1.56 TB checkpoint plus a 109 GB packed trunk**, roughly 1.7 TB of local disk and a download measured in hours. The CPU has to support AVX2 and FMA. Linux x86-64 is the primary target, with macOS/ARM64 and Windows also supported.

There is one more limit that tends to get lost under the headline: what runs here is the **base model**, with no chat templating. It continues the text you give it rather than answering questions, so on its own it is not usable as an assistant.

## Why it matters

The value of the project is not its speed but what it makes visible. Anyone looking at running models locally arrives at the same sentence: "this model will not fit on my card." This work shows that the sentence is not always true. Not fitting is often a property of our habit of *loading everything into memory at once*, rather than of the model itself. If most experts are not used on a given step anyway, keeping all of them resident turns out to be a choice rather than a requirement.

In practice nobody will do real work at one word every 26.5 seconds. But the same idea, applied to smaller mixture-of-experts models on faster storage, lands in far more reasonable territory. Fareed Khan, who wrote the code, sums the approach up in one line in the repository: same model, same answers, different speeds depending on the memory you have.

We covered Kimi K3 before, when it [went rentable on CoreWeave](https://globalfeed.ai/en/the-worlds-largest-open-model-goes-rentable-kimi-k3-lands-on-coreweave/). The question there was who could afford to run it; this project answers the same question from an entirely different direction.