An open-source ESP32-P4 project converts spoken English into text locally, using compressed AI model weights and the chip’s own processing capabilities to enable offline voice control.
An open-source project called esp32-pie-asr brings live English speech recognition to the ESP32-P4. By without sending audio to a cloud service or relying on a separate AI accelerator. Developed by Boumedine Billal, it runs on the microcontroller itself, displaying recognised words as a person speaks. The project also includes six voice-control demonstrations, covering applications such as smart home controls, a robot vacuum and a car cabin interface.
The project uses Kroko, a 65.7-million-parameter streaming speech-recognition model developed by Banafo. Its original model weights occupy around 71MB, which is too large for the target board’s available memory in their original form. Billal has reduced them to 26.65MB using a combination of two-bit and four-bit weight quantisation. The compressed weights are stored on a microSD card and loaded into the board’s 32MB PSRAM during startup. The speech engine then uses the ESP32-P4’s dual RISC-V cores and its processor-specific PIE SIMD instructions to perform the calculations.
In tests reported by the developer, the system processes 1.28 seconds of audio in approximately 0.90 seconds, operating faster than real time. A word is typically finalised about two seconds after it is spoken. On the LibriSpeech test-clean benchmark, the implementation recorded a word error rate of 4.34 per cent using beam search with four hypotheses. These figures are the developer’s reported results and depend on the test setup and decoding configuration. The current implementation was tested on Espressif’s ESP32-P4X-Function-EV-Board, equipped with a microphone, display, microSD storage and 32MB of PSRAM.
Beyond transcription, the firmware demonstrates how recognised speech can trigger actions without a language model or internet connection. It uses configurable word and phrase rules to identify commands and their targets. Users can edit text files on the microSD card to add phrases, including alternative ways the speech model might recognise a particular word. In the developer’s tests, enabling these phrase lists improved command recognition across six demonstrations, with reported success rates ranging from 94 to 99 per cent. The demonstrations also recorded no false activations across 88 near-miss test clips per demo, although these results do not guarantee the same performance in everyday environments.
The project is released under the Apache 2.0 licence. The repository includes the speech engine, firmware, evaluation scripts, benchmarks and instructions for flashing the board, allowing developers to inspect the implementation and adapt it to their own applications. The current build supports English, while Arabic support and compatibility with the ESP32-S31 are listed as future work.
