> ## Content Index
> Fetch the complete content index at: https://globalfeed.ai/llms.txt
> Use this file to discover other available public pages before exploring further.

# Perplexity open sources Lily, a Rust and Metal inference engine for Apple Silicon
- URL: https://globalfeed.ai/en/perplexity-open-sources-lily-a-rust-and-metal-inference-engine-for-apple-silicon/
- Published: 2026-09-04T09:28:32.000Z
- Updated: 2026-09-04T09:28:32.000Z
- Description: Perplexity released Lily, the local engine running Qwen3.6-35B-A3B on Apple Silicon, reporting 1.23x faster prefill and 1.35x faster decode than MLX-LM.
- Author: GlobalFeed Editor
- Tags: Perplexity, Qwen, open source, x-perplexity_ai, x-Alibaba_Qwen, dil-en, denetim-gerekli, video, x-gitti

Perplexity has open sourced **Lily**, the local inference engine behind Hybrid Compute in Perplexity Computer. It is a single-process runtime: a Rust layer loads the checkpoint and drives the generation loop, an OpenAI-compatible chat-completions API streams tokens, and hand-written Metal kernels execute the model. Neither PyTorch nor MLX sits in the execution path.

Lily is deliberately narrow, one model (Qwen3.6-35B-A3B) on one hardware family, and Perplexity presents that narrowness as the performance argument: model structure, execution plans and kernel selection all live in the same runtime. A standalone demo is available in the pplx-garden repository, serving greedy text generation over a minimal OpenAI-compatible HTTP API.

## Requirements and model layout

The 4-bit checkpoint is 19.4 GB, making an Apple silicon Mac with 32 GB or more of unified memory the realistic floor. The shipping Hybrid Compute product lists macOS 15+, 24 GB minimum and 32 GB for best results.

Qwen3.6-35B-A3B stores 35B parameters and activates roughly 3B per token. A router scores 256 experts and picks eight, plus one shared expert that sees every token. The model mixes 10 full-attention layers using grouped-query attention (16 query heads, two KV heads) with 30 Gated DeltaNet layers. Weights use groupwise affine 4-bit quantization with every group of 64 sharing a bfloat16 scale and bias, compressing about 70 GB of bfloat16 weights to 19.4 GB.

## Measurements

In Perplexity's ablation, dequantizing one tile at a time inside the grouped GEMM raised end-to-end prefill 77.4% at a 512-token prompt, and keeping routing work inside a single GPU command buffer added 89% at the same length. Expert GEMMs account for roughly 90% of prefill time. On decode, GQA packing improved throughput 23.8% at 32K, while a fixed-block attention layout gained 40.2% at 128K.

On one 40-core, 128 GB M5 Max at batch 1, loading identical 4-bit checkpoint bytes against MLX-LM's fastest direct-generation path across ten lengths from 256 to 128K tokens, Lily averaged 4,156 prefill tokens/s versus 3,388 (1.23x) and 170.0 decode tokens/s versus 126.4 (1.35x). A teacher-forced check across 192 positions put Lily's perplexity 0.04% higher, with the same top-ranked token 96.35% of the time.

Technical details and the repository are covered in [MarkTechPost's report](https://www.marktechpost.com/2026/09/02/perplexity-open-sources-lily-a-rust-metal-inference-engine-for-qwen3-6-35b-a3b-on-apple-silicon/?ref=globalfeed.ai).