Using Edge Impulse with Tracer to build your own custom gesture/activity tracker

A project log for Tracer | A Wearable for Things!

The Tracer is an open-source ESP32-based Inertial Measurement Unit tracker

yjYJ 05/15/2022 at 09:200 Comments

I have been inspired over the last few weeks by Fabio's project with Edge Impulse and how the Tracer platform can be used to leverage the ESP32's processing power to run neural network classifiers of activities and gestures. In this example, I will go through the steps needed to train your own neural network and show it performing live classification with data streamed directly from the board. 

I wrote an Arduino sketch that would spit out the 3-axis accelerometer and 3-axis gyrometer at a frequency of 100Hz. The experiment I had in mind was basically to use Tracer as a classifier that could tell if it was moving horizontally, vertically, "caterpillar" (move in a jagged way horizontally), and waving. A dataset of 10s was collected for each of the movements.

Once I had an idea of what I wanted to build, I followed the steps shown in the video tutorial from Edge Impulse in getting a model trained up as well as a k-means-based anomaly detector.

The initial model that was trained showed inaccuracies when it came to classifying a wave and that is due to several reasons but I think it was primarily due to not having enough data. The table below shows the confusion matrix for the first training cycle with the original dataset.

Following that, I wanted to see if collecting more data would improve so I spent another 10 seconds collecting data for wave, and it improved slightly. I think more data in each category should improve the neural network's specificity and sensitivity. The accuracy improved across the board while the error rates also dropped.

Here is a short video example of how the live classification tool is being used on the data that is streaming directly from Tracer over USB (UART). The next step for me is to deploy a real-time classifier for tennis shots that will run on the ESP32 and stream the results over BLE.

And again as before, really happy to hear any feedback, ideas or suggestions!