Apple is boosting open source AI development on Mac by enhancing MLX and delivering 19–27% faster local LLM inference with the new M5 chip.
Apple is strengthening its position in the open source AI ecosystem, demonstrating how the new M5 chip significantly improves local inference performance for large language models when paired with its open-source MLX machine learning framework.
MLX is Apple’s open-source “array framework for efficient, flexible, and highly tuned” machine learning on Apple silicon. It enables developers to train neural networks, generate text and images, and run or fine-tune large language models directly on-device. MLX LM, a supporting package, allows downloading open models from Hugging Face and running them locally, including with quantisation for reduced memory use and faster inference.
The latest benchmarks published by Apple show that M5 GPU Neural Accelerators, equipped with dedicated matrix-multiplication engines, deliver substantial compute improvements. Tests comparing models such as Qwen 1.7B, Qwen 8B, Qwen 14B in 4-bit, Qwen 30B MoE, and GPT-OSS 20B reveal that the M5 offers a 19–27% performance boost over the M4 for time-to-first-token and sustained token generation.
This uplift is credited to an increase in memory bandwidth from 120GB/s on the M4 to 153GB/s on the M5, a 28% jump. Apple notes that a 24GB RAM MacBook Pro can comfortably handle an 8B BF16 model or a 30B MoE in 4-bit quantisation under an 18GB footprint. Image generation tasks also run up to 3.8 times faster on the M5.
