The "Tingle" is a wearable device designed to combat body-focused repetitive behaviors (BFRBs), especially trichotillomania (compulsive hair pulling) and excoriation (compulsive skin picking).
Prevalence of body-focused repetitive behaviors (BFRBs)
Body-focused repetitive behaviors (BFRBs), symptoms of Obsessive Compulsive Disorder (OCD) and other conditions involving compulsions (e.g., Autism Spectrum Disorder) involve compulsively causing physical injury and/or damaging one’s physical appearance. These are among the most poorly understood symptoms; they are often misdiagnosed and undertreated. BFRBs include hitting oneself, biting, pulling out hair, skin picking and cutting, as well less severe but damaging behaviors such as nail biting, thumb sucking, and nose picking. These symptoms affect at least 5% of the population; hair pulling alone affects 1%, or about 3 million people in the US. BRFBs are highly comorbid. Studies have shown that as many as 70% of those with one BRFB will have another co-occurring BRFB. While often impairing, affecting medical health and/or disfiguring, these symptoms are frequently reported but often not observed in clinical settings. This makes diagnosis, as well as treatment planning and monitoring, exceedingly difficult. To avoid pain and disfigurement, it is imperative to identify a reliable means to automatically identify and monitor BFRBs, especially outside the clinic setting. Clinicians need data on BFRB frequency and timing for the purposes of diagnosis, treatment planning and monitoring while patients need immediate, real-time feedback to make behavioral therapies more effective.
The “Tingle” device
To address this previously unmet clinical need, I have created a wrist-worn device called the “Tingle” that can monitor and record BFRBs while also providing real-time (haptic) feedback (on the wrist) to the individual with BFRBs when they occur. The Tingle is well-positioned to address the tremendous unmet need in the care of individuals with BFRBs and other damaging and impairing compulsive behaviors. The Tingle will revolutionize the diagnosis and treatment of at least 5% of the population affected by these severe symptoms by using a convenient, automated, unobtrusive, and inexpensive device that both records behaviors for clinical assessment and provides accurate, real-time feedback to the wearer in support of therapy.
Hack Fitness Tracker Wearable Device Prototyping Platform
Although the Tingle is my primary focus in the project, my reason for sharing it on Hackaday is to open source my extensive use of hacked fitness trackers as a wearable device prototyping platform.
All are Arduino IDE compatible, completely hackable, use a fast ARM Cortex MCU and cost $20-40. For more information scroll to the bottom of this details section, see the related project log, view my other projects and check out the Hacked Fitness Tracker Wearable Device Prototyping Platform GitHub repository: https://github.com/curtpw/nRF5x-device-reverse-engineering
Now back to Tingle.
Tingle Basic Operation
The fundamental operation of the Tingle is fairly intuitive: data is gathered from a variety of sensors, sensor data is applied to a statistical model to predict whether the device is in a targeted position relative to the body, and if the model determines a sufficient likelihood that the Tingle is on target a vibration motor is activated to alert the user. Although the user trained neural network feature is really cool and makes for an impressive demo, I have deactivated it for the feasibility study. This device is being developed for use by children, and the custom model feature is not child friendly.
Tingle Mobile App
I have created a mobile app to configure and manage the Tingle. It can be found on the Google Playstore for Android based mobile devices. I plan on creating an iOS version soon, but I can configure the Tingle for test users myself at the CMI office before handing it off so lack of iOS support hasn't been much of an issue - at least as far as pilot and feasibility studies are concerned. The app is used to choose which target area will trigger haptic feedback. It also allows users to train a custom model by collecting data themselves.
Tingle Machine Learning Web Interface
This web application is LIVE: https://www.mtrlb.com/tingleneuro/
My emphasis in developing the Tingle has been on pre-trained machine learning models for a selection of target areas. This is much more user friendly than making users train their own models. In order to do this, I gather lots and lots of data for each target using the above pictured web application (you can visit, its live) which connects to the Tingle using Web Bluetooth. I've loaded a little test data for users to play around with. You can train a neural network model yourself, though you will have to use your browser's developer console to see anything happen. I use the synaptic.js machine learning library for all Tingle related machine learning code. I am using synaptic's LSTM neural network capabilities. LSTM (Long Short-Term Memory) models are recursive neural networks. This means that they incorporate time series data, not just snapshots like a simple MLP (Multi Layer Perceptron) model.
* The Tingle uses a hacked N68 generic fitness tracker as a foundation (see "Building a Tingle").
* I use blue for data and green for clock (I2C, SPI and SWD interfaces).
A: The Nordic nRF52832 (nRF52) SoC w/ ARM Cortex M4 MCU is the heart of the Tingle. For the purposes of the Hackaday project, it is programmed using Arduino syntax C with Sandeep's ArduinoCore for Nordic nRF5x SoCs. It allows for Bluetooth connectivity and can store user data in its flash memory.
B: A Kionix KX126 accelerometer is connected to the nRF52 on the N68 fitness tracker PCB using an SPI interface.
C: Four MLX90615 thermal sensors are connected to the nRF52 with an I2C interface. The VL6180 distance sensor is connected on the same I2C interface bus. Each thermal sensor must be individually programmed with its own I2C address in order for them to all function on the same I2C bus.
D: VL6180 time-of-flight ('Lidar') distance sensor. Connected using an I2C interface. The pull-up resistors on the sensor breakout support all devices on this I2C interface bus (thermal sensors).
E: Push button connected to pin 5 on the nRF52. Right now this button is just used to leave low power sleep mode. Eventually it will be used to record false positives in a clinical trial.
F: SMD LED breakout with current limiting resistor connected to pin 30 on the nRF52. Right now the LED just blinks when the device boots into low power sleep mode or is connected to the app. The LED is more about user experience than functionality.
G: Vibration motor for haptic feedback connected to pin 25 on the nRF52.
H: 80 mAh LiPo battery.
I: Battery is connected to a switch. This allows users to completely cut power to the nRF52 and reboot the device.
J: Programming and debugging connector. A 1mm pitch header pin socket exposes pin 3 on the nRF52, ground and V+ directly from the nRF52 and the two wire SWD programming interface [DIO (data) and SCK (clock)].
Tingle at the Child Mind Institute
The Tingle wearable is under development at the Child Mind Institute MATTER Lab. The Child Mind Institute is an independent, national nonprofit dedicated to transforming the lives of children and families struggling with mental health and learning disorders. The story of how the Tingle arrived at CMI is pretty cool. I was working at CMI as a web developer and ended up helping Arno Klein, the new Director of Innovative Technologies, debug some fMRI brain image visualization software he and one of his grad students were working on. He was super interested in my wearable device side projects and device development in general. Opportunity! I looked for a problem he was interested in with a solution I could build: the Tingle was born! I built the first two prototypes in my apartment during evenings and weekends. Then I wrote and filed a provisional patent application and demoed the device to Arno. He loved it and demoed it to Mike Milham, Director of Research. Mike then arranged a demo for the CEO and founder of CMI, which also went great. I signed over the patent application to CMI and was hired to do hardware full time with Arno. The rest is history. All things Tingle are designed, built, and coded entirely by yours truly.
Tingle Prototype Progression
The above prototypes are just a selection of the iterations that have taken place over the last year and a half.
- This is the very first 'secret' Tingle prototype. I say 'secret' because it was too ugly to show anyone. Built using a Teensy LC dev board and an MLX90614 thermal sensor.
- This is the second prototype which I demoed to Arno and the rest of the folks at CMI. Built using a Seeed Studio Tiny BLE dev board.
- This is one of the first prototypes I built at CMI and the very first prototype I built using a hacked fitness tracker.
- This prototype has the hardware/sensor configuration that all future prototypes make use of. This one 'did the trick', from then on it was a matter of refining.
- This is the prototype that is in current active use. It is based on the N68 fitness tracker. This will be the last prototype to make use of a hacked fitness tracker.
Usable Prototype Development: Complete
After working through more than five early prototype, a device that could be handed off to typical users was completed.
Pilot Study: Complete
50 test subjects at the CMI office were recruited to wear the Tingle prototype while undergoing numerous excersizes and movements designed to test the Tingle's effectiveness. Data from the device, cameras and research assistant observations were compiled and analyzed. A manuscript has been written for the study and publication of findings is forthcoming.
Feasibility Study: In Progress
Prototypes have been distributed to adults with BFRBs and are currently in use. Many of these individuals have fairly harmless behaviors like nail-biting, but some are in the more injurious range of BFRB. The feasibility study is intended to provide feedback necessary to improve usability and optimize performance. This includes the mobile app which controls the Tingle, training given to users and other ancillary factors in addition to the device itself.
Clinical Trial: Forthcoming
A small clinical trial with CMI patients (children and adolescents) will take place once usability and performance have been optimized. This will involve around 80 of the current prototype. Realistically this would be the first of several clinical trials. If this trial goes well, the Tingle project will receive substantial funding and we will partner with a contract manufacturer for further prototype production (in other words, no more hacked fitness trackers, in fact we might even end up hiring one of the Chinese factories that make the sort of fitness tracker I've been hacking) .
We have a patent! Or more accurately, our patent application for Tingle was approved by the patent examiners but our attorneys thought they could do better so they refiled it as a "continuance" application to see if they could get even more IP coverage. The cool thing is, this is all public record! You can go to the US Patent office PAIR (Public Patent Application Information Retrieval) site right now and see all the communications between the patent examiner and our lawyers. Pretty cool. We are application 15/816,706. What does this mean for open source? No much, patents only apply to commercial usage (ie large scale manufacture practically speaking). If you want a device to help people outside of the hacker community a patent is essential - otherwise there is no way you will get investors to pay the hundreds of thousands of dollars (if not millions) necessary for large scale production of a medical device. No patent = no real world impact for a medical device. I am listed as sole inventor on the Tingle patent application, wrote the first (provisional) version of the patent application myself and have acted as technical adviser to our patent attorney throughout the process. Its been super interesting.
A Wearable Device Prototyping Platform based on Hacked Fitness/Activity Trackers
Note: although the Tingle is this project's primary focus for the Hackaday competition, open sourcing my use of hacked fitness trackers is of more practical value as far as posting documentation on Hackaday is concerned. For that reason I have retained most of this project's original focus on hacking fitness trackers.
For the past two years I've been using hacked generic activity trackers as a rapid prototyping platform for wearable devices - primarily gesture recognition biofeedback and position tracking devices at the CMI MATTER Lab, a research institute focused on children's mental health.
My goal is to present generic nRF52832 and nRF51822 ARM Cortex activity trackers, in particular the X9 Pro, as a full fledged platform for prototyping low power devices accessible to anyone familiar with Arduino.
The $35 X9 Pro Activity Tracker is equivalent to ALL these Adafruit Products COMBINED:
- Adafruit Feather nRF52 Bluefruit LE ($24.95)
- Adafruit Vibrating Mini Motor ($1.95)
- Adafruit Pulse Sensor Amped ($24.95)
- Adafruit Triple-Axis MMA8451 Accelerometer ($7.95)
- Adafruit Lithium Ion Polymer Battery - 3.7v 100mAh ($5.95)
- Adafruit OLED Breakout Board - 16-bit Color 0.96" ($29.95)
TOTAL: $95.70 not including brushed steel enclosure, silicone strap and general miniaturization
Devices That Use This Platform (see Project Logs):
- "Tingle", a gesture recognition and biofeedback device for compulsive mental health disorders like trichotillomania and excoration.
- "Thermo", a hand position tracking device that does not require an independent camera or other external reference data (think mobile Oculus and HTC Vive controllers).
- Intraoral Respiration Monitor for Overdose Detection - a computer (hacked smartwatch) worn entirely inside your mouth with air pressure, humidity and temperature sensors.
Tutorials and Example Code:
- Blink, Button, OLED, Web Bluetooth (GATT Notifications) and Bluetooth Serial debugging Arduino example sketches that work with the X9 activity trackers (and most other activity trackers mentioned in this project with slight modification)
- Use of neural networks (LSTM MLP by way of synaptic.js) for gesture recognition using accelerometer data from the X9 streamed into a web browser over Web Bluetooth
- Accessing additional GPIO ("pins") so you can customize your device and add additional components
Neural Network Gesture Recognition with Web Bluetooth Tutorial
Stream sensor data from your hacked activity tracker into a web browser using the experimental HTML5 Web Bluetooth API. Sample data in different positions and train a neural network to distinguish position ie recognize gestures.
This is a GitHub site so all you have to do is fork the GitHub repository and you can create your own version of this site in seconds. Customize and hack it! (info on Web Bluetooth)
X9 Pro Activity Tracker Components
- nRF52832 ARM Cortex M4 SoC/MCU
- vibration motor
- 96x64 Color OLED display with SSD1331 controller IC
- Kionic KX126 Accelerometer with interrupt pin attached to MCU
- green LED for heart rate sensing
- photosensor for heart rate sensing
- 8MB SPI FLASH memory IC
- touch sense circuit
X9 Pro Activity Tracker MCU Pinout
- HR_LED_PIN 4
- HR_DETECTOR 29
- TOUCH_BUTTON 30
- VIBRATE_PIN 8
- BATTERY_PIN 28
- OLED_CS 15
- OLED_RES 14
- OLED_DC 13
- OLED_SCL 12
- OLED_SDA 11
- OLED_LED_POW 16
- OLED_IC_POW 17
- KX126_CS 24
- KX126_SDI 19
- KX126_SDO 20
- KX126_SCL 18
- KX126_INT 23
- FLASH_CS 22
- FLASH_SDI 19
- FLASH_SDO 20
- FLASH_SCL 18
X9 Pro Activity Tracker Main PCB Composite Microscope Image
Stretch Goal #1 : Adding ECG and Audio to the Platform COMPLETED!!!
I have Nordic nRF52/nRF51 based activity trackers that incorporate ECG and audio! I haven't hacked them yet but if this project generates enough interest I will document a complete hack of both devices so they can be used as prototyping platforms.
But you don't have to wait for me, take a crack at hacking them yourselves:
GETTING INVOLVED WITH THIS PROJECT: I've received a couple requests to join this project. If you want to get involved and collaborate please fork the project GitHub repository ( https://github.com/curtpw/nRF5x-device-reverse-engineering ). If you hack additional devices I would love to add them to the project on GitHub. If you'd like to help with coding I've posted a couple issues to the repo I could use help with. I could post a ton more - I only have so much free time and there are many ways this project can be expanded.