We use the Arduino Portenta X8 for our examples throughout this series, we encourage you to do the same. Please note that you will be able to follow along with any Linux based device, but there are many reasons we advise aquainting yourself with the Arduino Portenta X8!
'Twas the night before Christmas, when all through the house
Not a creature was stirring, not even a mouse;
The Christmas Tree knew this and did not illuminate;
It can detect your presence, thanks to ML, webcam and Portenta X8…
As we enter the most magical time of the year, we would like to present you with something to help bring just a little bit of magic into your home. Well, it’s not really magic, it's just a self illuminating Christmas Tree which can detect and identify a human presence in the room and activate or deactivate its own lights via an MQTT switch. For some, that might as well be magic. Which is why we have put together this series of blogs to help upcoming engineers explore the basic principles of embedded development.
In this season of giving, we offer you the gift of FoundriesFactory! It’s easier than ever to develop, deploy, manage and maintain your secure IoT and Edge Linux-based devices.
The Machine Learning Christmas Tree
The goal of this series is to coach you through setting up your own light controlling Christmas Tree. To do this, we are going to connect a webcam to an Arduino Portenta X8 SoM, running a Machine Learning (ML) algorithm powered by the Edge Impulse AI Platform. We will train this algorithm to detect human presence and finally signal an MQTT switch to control the lights.
Edge Impulse are at the forefront of emerging embedded machine learning technology. With their Edge AI Platform, they provide tools for developers to empower embedded devices, implementing new techniques in machine learning on these low-power devices. With this, we can combine real-time sensor data, audio, and video together then process all this directly on the device. Using Edge Impulse’s tool set, we will be able to detect someone standing in view of our Christmas tree without having to go to the cloud or communicate with the device. It will then autonomously control its lights!
Project Overview
This series will guide you through every step; from starting your first Factory and flashing your board to deploying your finished container app. This will be a 14 part blog series detailing every step of the project, complete with code examples so you will be able to follow along and create your own Artificial Intelligence powered Christmas Tree. In the series, we will cover:
- Basic Yocto Principles
- Building and deploying a container app
- Connecting a web camera interface with a Docker Container
- Implementing an MQTT switch module
- and much more!
Arduino Portenta X8
If security concerns are not already one of your top priorities when designing a product, you will soon be left with no option but to make it so. Legislation like the European Union’s (EU) Cyber Resilience Act (CRA) highlight the increasing level of security consciousness within the community. This legislation puts the onus on the manufacturer so that their products are secure against cyber-attacks for its lifetime. Arduino Portenta X8 with X8 Board Manager software has CRA readiness built-in. The X8 Board Manager is a special version of the FoundriesFactory platform, configured for smooth integration with the Arduino EE development environment and other Arduino developer resources. You can read our thoughts on why this is so significant in our blog on the EU Cyber Resilience Act.
If you’re ready to get started, then it’s time to begin step 1 of the project: Creating Your First Factory!
Part zero: AI-Powered Machine Learning Christmas Tree
This page.
Part one: Your First FoundriesFactory AI-Powered Christmas Tree
This initial tutorial guides you through the process of starting your first Factory with Foundries.io. We discuss exactly how to get your Factory up and running, establish your credentials, build your first target and flash your device!
Part two: Configuring Network and Wi-Fi
In this tutorial, we will guide you through configuring the network on your embedded device. This will include configuring a static IP and an introduction to NetworkManager Command Line Interface (nmcli).
Part three: A Yocto & Linux Tutorial for Building An AI-Powered Christmas Tree
In this tutorial, we will be introducing you to the basic principles of the Yocto Project and its role within the Linux microPlatform. You will learn all about layers, recipes and even how to start a Yocto-based build locally.
Part four: Linux MicroPlatform (lmP) Config Tutorial for AI-Powered Christmas Tree
In this tutorial, we will be expanding on the concept introduced by the Yocto Overview and include some of our previous device network configurations to our LmP.
Part five: Modify & Extend Linux MicroPlatform for AI-Powered Christmas Tree
In this tutorial, we will be continuing to modify and extend our LmP. This time with new recipes to enable an MQTT broker.
Part six: Using a MQTT Switch Module on Arduino Portenta X8
In this tutorial, we will be configuring an MQTT switch Module to connect to our device so that we may use it to control the Christmas Tree lights.
Part seven: Creating a Shell Script Application and Adding to Linux microPlatform™ (MQTT)
In this tutorial, we will be writing a simple shell script to send MQTT messages to control the switch.
Part eight: Connecting the Web Camera Interface with Docker Container
In this tutorial, we will be guiding you through the process of interfacing your webcam with a Docker Container. We will cover two methods to connect a camera to a running container, and how to save these configurations in a Docker-compose app so that they can be deployed via your Factory.
Part nine: Using Edge Impulse AI to Recognize Human Presence With Arduino Portenta X8
In this tutorial we will be introducing you to Edge Impulse. We are going to be using their Edge AI Platform tools to create a ML algorithm. We guide you through joining the platform, building your dataset, demoing, and testing.
Part ten: Enhancing the Image Detection Capabilities of our AI Powered Christmas Tree
In this tutorial, we will be continuing our work with Edge Impulse. Now that you are acquainted with the platform, it is time to begin training the model to detect a human presence so that it is able to signal the MQTT switch for whether the lights should be on or off on your Christmas Tree! We will guide you through creating an appropriate data set for live classification.
Part eleven: Writing a Python App to Turn Our Christmas Tree On/Off Using AI
In this tutorial, we guide you through using the Edge Impulse Python-SDK to have your live detection algorithm run directly on your device. This Python app will enable communication between your webcam, the Arduino Portenta X8, and the MQTT switch, resulting in autonomous lighting control for your Christmas Tree!
Part twelve: Creating a Docker Compose App to Automatically Power Your AI Christmas Tree On/Off
In this tutorial, we cover the final steps in packaging all your individual components into a Docker Compose app which can then be deployed via your Factory.
Part thirteen: Running Docker Compose Application on a Raspberry Pi 4 to Power Our AI Christmas Tree
This tutorial showcases the same AI-powered Christmas deployment only this time on a Raspberry Pi.
Part fourteen: Managing Multiple SoM Platforms in the Same Factory
This tutorial showcases the same AI-powered Christmas deployment only this time on a TI AM62xx.