Meltdown Detection: an Automated Approach

An experiment in Tiny ML to detect an incoming autistic meltdown and trigger mitigating strategies

Public Chat
Similar projects worth following
This experiment introduces an innovative sensor-based approach for automated meltdown prevention in individuals with autism spectrum disorder (ASD). By integrating sensor technology and Edge Machine Learning (TinyML) with inflation control mechanisms, the proposed system aims to detect early signs of distress and trigger the inflation or deflation process of a proprioceptive vest. This automated intervention also aims to alleviate the challenges associated with manual inflation, particularly during meltdowns.

Problem Statement & Project Motivation

A meltdown is an intense response to an overwhelming situation. It happens when someone becomes completely overwhelmed by their current situation and temporarily loses control of their behavior. This loss of control can be expressed in many ways. 

Meltdowns are not the only way an autistic person may express feeling overwhelmed. They may also refuse to interact, withdrawing from situations they find challenging or avoiding them altogether.

Many autistic people will show signs of distress before having a meltdown. They may start to exhibit signs of anxiety such as pacing, seek reassurance through repetitive questioning or physical signs such as rocking or becoming very still. At this stage, there may still be a chance to prevent a meltdown. 

Of the several strategies to consider, this research is based on the proprioceptive properties of a firm hug-like pressure when feeling stressed or overloaded by sensory information. In this situation, some individuals like to be held firmly, but they often dislike physical contact. An alternative is an inflating vest, like that from Squease.

To inflate the vest, specific fine motor skills are required to use the manual pump and get the vest to the correct pressure on time. This would be extremely difficult for certain autistic individuals, particularly with an incoming meltdown or during one.

A working prototype to auto inflate the vest was developed by Tevian.  It requires the intervention of the user to select actions from a touch screen. But the inflation itself is done automatically after that.  A second, more advanced version of this mechanism, proposed by Tevian, would be the core operating element. 

If a sensor, or group of sensors can detect an incoming meltdown, this apparatus could send the necessary commands to the inflating control unit to start the inflation or deflation process without user intervention, possibly providing the proprioceptive effect of a firm hug without physical contact and achieving this before the full onset of the meltdown. 

Proposed Solution

Using several biometric sensors in an Edge Machine Learning model, it is proposed that an incoming meltdown can be detected in time to provide the mitigating strategy mentioned above. 

Four types of sensors will be explored: 

  1. IMU.  Specific or anomalous movement from the individual, either in his body or some body part (i.e., arms, hands).
  2. Heart Rate.  The rate of change, rather than the actual HR, as a signature measure of an increasingly stressful situation.
  3. EMG.  Electrical activity from forearm muscles when making a fist or tensing an otherwise relaxed body.
  4. Thermal camera. Changes in face temperature as the stressful situation presents itself.

To combine these four sensors, multiple data streams will be created. These will have to be sampled at the same frequency. Then a classification, a regression and/or an anomaly detection model will be applied. Perhaps using different DSP blocks for each data stream.

To execute these experiments, Edge Impulse's platform provides several models that can serve as testing platforms. The learning blocks mentioned above include:

Of all the Edge Impulse tutorials, two could be used for base knowledge into this project:

Additionally, there are a collection of expert and community projects that may provide other insights:

Besides creating own datasets for model training, public datasets will be explored. Preliminarily, these ECG-related datasets from Kaggle will be explored.

While exploring the project data base, I came across a tutorial that uses a commercial EEG sensor (a.k.a. Muse) with ML to detect blinks and also to manipulate several computer elements via concentration tasks....

Read more »

  • Sensors

    Pedro Martin05/29/2023 at 19:26 0 comments

    • Heart rate and Pulse-Oximetry Monitor from Analog Devices.  From their website: The MAXREFDES117 reference design is a low power, optical heart-rate module complete with integrated red and IR LEDs, and a power supply. This tiny board, perfect for wearable projects, may be placed on a finger or earlobe to accurately detect heart rate. This versatile module works with both Arduino and mbed platforms for quick testing, development and system integration. A basic, open-source heart rate and SpO2 algorithm is included in the example firmware. The board features 8 sewing tap pads for attachment and quick electrical connection to a development platform.
    • Wearable EMG sensor from Ultimate Robotics. From their website: uMyo is a single channel wearable EMG sensor with wireless data transmission. Multiple sensors can be connected to one receiver. Main features of uMyo are: - It's wireless! No more mess of wires when working with EMG - Works with any Arduino via nRF24 radio module (you can use our Arduino library) - Works with ESP32 with no extra hardware (we also wrote an Arduino library) - Multiple units (up to 12 in current version) can send data to the same Arduino/ESP32 - Sends out detected muscle activity level, 4-bins spectrum; and raw EMG data (in nRF24 mode) - Can be used with a bracelet and dry electrodes, or with gel electrodes via soldered connector. 
    • MLX90640 IR Thermal Camera Breakout - 55 Degree from Adafruit. From their website:  This sensor contains a 24x32 array of IR thermal sensors. When connected to your microcontroller (or Raspberry Pi) it will return an array of 768 individual infrared temperature readings over I2C. This part will measure temperatures ranging from -40°C to 300°C with an accuracy of +- 2°C (in the 0-100°C range). With a maximum frame rate of 16 Hz.

    IMU from the Nano 33 BLE Sense Rev2 from Arduino.  From their website: The Arduino Nano 33 BLE Sense Rev2 combines a tiny form factor, different environment sensors and the possibility to run AI using TinyML and TensorFlow™ Lite. The Nano 33 BLE Sense Rev2 not only features the possibility to connect via Bluetooth Low Energy but also comes equipped with sensors to detect color, proximity, motion, temperature, humidity, audio and more.  IMU for Motion Detection. The board provides a 9-axis inertial measurement unit featuring a 3D accelerometer, gyroscope and magnetometer and allows you to detect orientation, motion or vibrations.

View project log

Enjoy this project?



Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates