Lily is a Rust and Metal-based inference engine designed for running Qwen3.6-35B-A3B locally on Apple Silicon, powering the local side of Perplexity’s Hybrid Compute system.
Perplexity has open-sourced Lily, a specialised local AI inference engine developed for Apple Silicon. The engine powers the local computing component of Hybrid Compute in Perplexity Computer and is designed specifically to run the Qwen3.6-35B-A3B model.
Unlike general-purpose AI frameworks designed to support multiple models and hardware platforms, Lily takes a highly specialised approach. It uses a Rust runtime and hand-written Metal kernels optimised around the architecture of Qwen3.6-35B-A3B and Apple Silicon. Neither PyTorch nor Apple’s MLX framework is part of Lily’s execution path.
The engine provides an OpenAI-compatible chat-completions API for local text generation. Qwen3.6-35B-A3B is a mixture-of-experts model with 35 billion total parameters, although roughly 3 billion parameters are activated for each token. Its architecture combines mixture-of-experts processing with grouped-query attention and Gated DeltaNet layers, creating different computational workloads that Lily specifically optimises for.
According to Perplexity’s reported benchmarks, Lily achieved an average 1.23 times higher prefill throughput and 1.35 times faster decoding than MLX-LM when tested across different context lengths on a 40-core, 128 GB M5 Max system. The optimisations include keeping expert routing operations on the GPU and combining weight dequantisation with grouped matrix operations to reduce memory overhead.
By open-sourcing Lily, Perplexity is making its specialised local inference technology available to developers interested in running large AI models efficiently on Apple Silicon. The project demonstrates an alternative to general-purpose inference frameworks by focusing deeply on the requirements of a specific model and hardware architecture, while supporting Perplexity’s broader approach of combining local and cloud-based AI computing.
