Ollama fixes growing memory use during long responses
According to Ollama's release notes, excessive memory growth during long responses with MLX has been fixed. The stated fix concerns generation using speculative decoding; it is not an announcement of a general speed increase across all models.
According to Ollama's release notes, excessive memory growth during long responses with MLX has been fixed. The stated fix concerns generation using speculative decoding; it is not an announcement of a general speed increase across all models. github.com
The development record explains the cause: the generation loop was supposed to release the freed-buffer pool at each 256-token boundary. Because speculative generation can emit several tokens in one round, the counter could step over the exact boundary and fail to trigger the release check. github.com
The revised check releases the pool whenever a round crosses a multiple of 256 tokens. The counter no longer has to stop exactly on the boundary; the release behavior already used for single-token rounds now also covers rounds that step past it. github.com
The same release adds a first-run setup when the Ollama command is launched, offering the choice to sign in or continue locally. Setup completion is shared with the desktop application on macOS and Windows. Continuing locally is explicitly listed as an option in the release notes. github.com
This report is based on published technical documentation. GlobalFeed has not independently benchmarked the implementation.
