> ## Content Index
> Fetch the complete content index at: https://globalfeed.ai/llms.txt
> Use this file to discover other available public pages before exploring further.

# llama.cpp 0.4.0: initial Qwen3.8-Flash-Next and Nemotron-3-Puzzle support, plus two changes that cut memory
- URL: https://globalfeed.ai/en/llama-cpp-0-4-0-initial-qwen3-8-flash-next-and-nemotron-3-puzzle-support-plus-two-changes-that-cut-memory/
- Published: 2026-09-05T14:36:56.000Z
- Updated: 2026-09-05T14:36:56.000Z
- Description: The project most people use to run models on their own hardware has a new release. Two new model families gain initial support, but the useful part is memory: tensors are now read on demand and the quantizer is capped. On the server side, context limits are enforced per slot.
- Author: GlobalFeed Editor
- Tags: llama.cpp, local ai, open source, Qwen, Nemotron, x-ggml_org, x-Alibaba_Qwen, x-nvidia, dil-en, elle, video, x-gitti

**llama.cpp**, the de facto standard for running models locally, has [published version 0.4.0](https://github.com/ggml-org/llama.cpp/releases/tag/v0.4.0?ref=globalfeed.ai). The announcement came from the project's own account.

The release notes come down to three things: initial support for two new model families, on-demand tensor reading, and per-slot context limits on the server.

## New models

- **Qwen3.8-Flash-Next** (the `qwen4exp` architecture). The notes state this is *initial* support, with optimization work still pending.
- **NVIDIA Nemotron-3-Puzzle-75B-A9B**
- **Nemotron 3.5** DSpark variant
- **nanbeige4.2-3B**

Sparse flash attention support also landed for DeepSeek-V4, GLM and qwen4exp.

## The real story is memory

The version number is not the news; these two changes are. Both address the same problem: the memory spike you hit when running a large model on a modest machine.

**On-demand tensor reading.** Tensors are now loaded lazily, as needed, rather than all at once, which lowers the memory overhead during model initialization.

**Quantizer RAM capping and row-slab streaming.** The quantizer now has a ceiling and streams data in row slabs, to prevent memory spikes during quantization.

Together these mean room for a bigger model on the same hardware, or the same model on less memory.

## Server side: context per slot

The llama.cpp server can now enforce the context size limit **separately for each slot**. That makes resource management across concurrent requests easier: a single long-context request no longer consumes the whole server's memory.

Two more additions sit alongside it: KV-cell token tracking with n-gram history lookup, and Apple RDMA transport support for RPC. The underlying ggml dependency moves from 0.22.0 to 0.23.0.

## For the corporate reader

This release is aimed at the organization that wants to run a model on its own machine rather than send data to a cloud. The obstacle to local inference is usually not the model but the memory needed to open it, and memory is measured at the peak, not the average. On-demand tensor reading and the quantizer cap touch exactly that peak.

Practically: a model that did not fit on your server before may fit now. Measure it on your own hardware before committing, because the gain varies by model and quantization format.

## Sources

- **Primary source:** llama.cpp v0.4.0 release notes, ggml-org: [github.com/ggml-org/llama.cpp](https://github.com/ggml-org/llama.cpp/releases/tag/v0.4.0?ref=globalfeed.ai)
- **Announcement:** the release post from the project's own account, 4 September.