A million-token memory that fits in your pocket: SparkLLM open-sources its Spark X2.5 models

The bar just rose in the on-device model race: SparkLLM has open-sourced Spark X2.5-4B and X2.5-1.7B. The company's claim is blunt: these are the only on-device models with a native context window of up to 1 million tokens. Native means the window comes from training rather than a post-hoc extension trick; the long-context capability was built in a dedicated training stage of hundreds of billions of tokens, with sequence lengths stretching to 1 million.

The architecture is notable too: a hybrid attention design pairs one full-attention layer with three sliding-window layers, which is how it carries the memory cost of long context. The models were pretrained on roughly 20 trillion tokens, cover more than 200 languages, and are tuned specifically for agentic workflows, code, math and instruction following. The weights are up on Ollama as well, one command away from a local test.

What is a million tokens on a device good for? The company's answer: the model can read a whole product manual, a full codebase or a batch of documents in one pass and keep the full picture across a multi-step task, all locally. No more chopping documents into fragments. One field number was shared too: on a smart-home command test set, the 1.7B model reaches 90.3 percent end-to-end command accuracy at 0.85 seconds average latency, and the 4B is said to rival cloud models on coding. These are the company's own measurements for now; independent evaluations should follow in the coming weeks.

The bigger picture: as we wrote covering Tencent's ContextPilot, the bottleneck of the agent race is context. SparkLLM attacks the same problem from the hardware side: instead of managing context, give the pocket-sized model an ocean of memory. Which approach wins is one of the most interesting engineering questions of the agent era.