TinyML is quietly changing what’s possible in IoT. It’s making connected devices faster, more private, and more energy efficient. And thanks to open source frameworks, you don’t need a PhD to get started.
Walk into any modern factory floor, or pick up a new wearable, and you’ll notice a quiet shift happening. Devices are no longer just collecting data and pushing it up to the cloud. Many of them are starting to make decisions on their own, right where the data is born.
That’s the promise of TinyML. Instead of sending everything to a remote server, these small but clever models run on low-power chips, sometimes the size of a coin. They decide whether a vibration in a motor is ‘normal’ or not, recognise a short wake word like ‘hey’, or monitor someone’s activity — all without the internet.
For me, this is one of the most exciting evolutions in IoT. We’ve spent years wiring up sensors everywhere; now we’re finally making those sensors smart. And what’s making this shift practical for everyday developers is, unsurprisingly, open source tools.
What is TinyML and why it matters
When people first hear the term TinyML, they often imagine some watered-down version of machine learning. But it’s not that at all. TinyML is about deploying fully functional ML models on microcontrollers — the same kind of chips that live inside Arduino boards, wearables, or small industrial sensors. These chips typically run on a few hundred kilobytes of memory and draw extremely low power.
Why bother running ML on something so small? The answer lies in a few practical benefits:
- No internet needed: Works in remote or unreliable networks.
- Instant response: On-device decisions mean no latency.
- Privacy: Sensitive data stays local.
- Efficiency: Saves bandwidth and battery.
In short, TinyML transforms ordinary sensors into decision-makers. Think of it as giving a basic ‘brain’ to the edge of the network.
The real-world challenges of TinyML
Running machine learning models on servers equipped with GPUs is straightforward compared to deploying them on tiny chips with only a few hundred kilobytes of RAM. This challenge defines the essence of TinyML, where extreme resource constraints make it a unique engineering endeavour. Most microcontrollers have only 256KB to 1MB of flash memory and even less RAM, leaving minimal space for neural networks. They also lack powerful GPUs or multi-core CPUs, relying instead on simple processors running at low clock speeds. Power efficiency becomes critical, as many devices operate on small batteries, and every extra computation reduces their lifespan. Moreover, these devices must remain functional even without constant network connectivity. Updating models across thousands of distributed devices adds further complexity. These challenges aren’t obstacles but design constraints that inspire innovation—precisely where open source frameworks empower developers to build efficient, deployable TinyML solutions.
Open source frameworks that make TinyML possible
A few years ago, if you wanted to run ML on a microcontroller, you were mostly on your own. Today, thanks to the open source community, there are some incredibly capable frameworks that take care of the heavy lifting.
TensorFlow Lite for Microcontrollers (TFLM)
TFLM is a trimmed-down version of TensorFlow Lite built specifically for microcontrollers. It runs without an OS and supports common layers like Conv2D, Fully Connected, and Softmax. Developers use it for things like wake-word detection, gesture recognition, and anomaly detection. It’s battle-tested and widely supported.
Edge Impulse
This is one of my personal favourites because it’s end-to-end. Edge Impulse gives you tools to collect and label data, train models, optimise them, and finally deploy to MCUs — all in a developer-friendly interface. It works smoothly with Arduino, ESP32, STM32, and others. For someone who wants to build fast without reinventing everything, this platform is a game changer.
MicroTVM
This one is for people who like fine-grained control. It’s part of the Apache TVM project and lets you compile neural networks into extremely efficient C code for embedded targets. It’s not plug-and-play like Edge Impulse, but the optimisation possibilities are excellent.
PyTorch Mobile / Lite Interpreter
Although not exactly ‘tiny’, PyTorch Mobile works well for Raspberry Pi or Android-based edge devices. If your project allows for a bit more processing power, this is a flexible choice.
Building a TinyML application usually follows a clear sequence. Once you do it a couple of times, it starts to feel natural. The steps are:
- Data collection and preprocessing
- Model training
- Model optimisation
- Deployment
Real-world examples that show TinyML’s power
Predictive maintenance in factories
Imagine small vibration sensors attached to motors. A TinyML model learns the ‘normal’ vibration pattern. If something deviates, the device instantly flags it — no round-trip to the cloud, no delay.
Wake-word detection
You’ve probably used this without realising it. Many smart speakers use a tiny model running locally to detect a wake word. The model’s job is simply to ‘wake up’ the larger system when needed, saving energy and ensuring privacy.
Wearable health monitoring
TinyML models can run on wristbands or patches to detect irregular motion, falls, or heart anomalies. Because everything happens on-device, user data stays private, and battery life improves dramatically.
Making TinyML devices part of the IoT network
TinyML devices use specialised IoT protocols to communicate reliably.
- MQTT is the go-to lightweight protocol. A TinyML device can simply publish a message like ‘anomaly detected’ to an MQTT broker.
- Node-RED gives you a nice flow-based way to wire these messages into dashboards, alerts, or other services.
- ThingsBoard and similar platforms act as the control centre — receiving data, visualising it, and letting you trigger actions.
TinyML’s role in the shift towards agentic AI
Here’s the interesting part: TinyML isn’t just about compressing models. It’s about giving devices autonomy.
Right now, a single TinyML model can classify or detect anomalies. But imagine a network of these devices, each running its own models, collaborating to sense, decide, and act. That’s not science fiction; it’s the foundation for agentic AI at the edge. Instead of one big brain in the cloud, you get thousands of small, distributed brains working together.
Whether you’re a hobbyist building something cool with Arduino or an engineer working on industrial systems, TinyML gives you a way to embed intelligence right where the data is born. And as these devices evolve, they’ll form the building blocks for the next wave of intelligent, agentic systems — systems that sense, plan, and act without waiting for instructions.
