close-circle
Close
0%
0%

viEwMotion

The wearable Voight-Kampff machine.

Similar projects worth following
close
330 views
0 comments
0 followers
likes
viEwMotion's (short for View Emotion) goal is to facilitate communication for individuals who have difficulty expressing emotions due to inherent neurologic issues like Autism, Parkinsons, etc. These conditions make it difficult to outwardly express emotions, which can dramatically interfere with communication with friends, family, care givers, etc..
viEwMotion will interpret real time physiological data acquired from a wristband which is then processed by a single board computer and displayed to allow others to see the persons emotional state.
For those affected by these conditions viEwMotion will "Change their World" by profoundly improving their ability to communicate effectively with those they interact with- changing how they experience and interact with the world around them.
In addition this technology will make research on emotional response (physiological measures of emotion) more accessible to Citizen Scientists.

This project is completely Open Source (code) and Open Hardware (schematics, boards, parts, etc.)

ViEwMotion (View Emotion)

The aim of this project is to improve emotional communication between with neurological disorders (primarily those in the autistic spectrum but applies to many other areas as well) and those without neurological disorders.

To achieve this aim, what is needed is a method of Analysis and Display of Emotional response, in Real Time using minimally invasive physiological sensors.

Project Goals:

  • Increase emotional communication between individuals with neurological disorders and those without them.
  • Assess and display emotional response in real time using non-invasive physiological measurements.
  • Be a wearable device that is comfortable and does not interfere with normal activities of daily living.
  • Data is locally processed and interpreted without the use of an internet connection or cloud based service.
  • Inexpensive (ideally under $50) and built from off the shelf parts.
  • Completely Open Source (software, firmware and hardware). All portions clearly explained so that it can be altered and extended as needed.

Currently available methods for determining emotional response utilize facial expressions from video or still pictures. At present these methods are closed source (even if they are free to use) and utilize cloud based processing. In addition, these methods do not work well on individuals who have difficulty in displaying emotion. This is where viEwMotion comes in.

AdafruitOpenSourceLicense.txt

Open Source license for Adafruit's Feather Bluefruit Board and software libraries

plain - 3.42 kB - 09/06/2017 at 03:42

download-circle
Download

LICENSESparkfunMAX3010x-Software-and-Library.txt

Open Source License for software and libraries for the MAX3010x sensors

plain - 2.82 kB - 09/06/2017 at 03:33

download-circle
Download

  • 1 × Microchip MCP6002 , 1MHz, rail to rail OpAmp for EDA/GSR
  • 1 × Texas Instruments TLE2426 Virtual ground rail splitter for OpAmp - stable ground reference
  • 1 × Pulse Sensor Amped https://pulsesensor.com/products/pulse-sensor-amped
  • 1 × AD8232. (SparkFun product number SEN-12650) SparkFun Single Lead Heart Rate Monitor
  • 1 × Adafruit Feather 32u4 Bluefruit LE arduino capatible with onboard Bluethooth LE

View all 7 components

  • Proposed use of viEwMotion

    Tom Meehan09/08/2017 at 17:57 0 comments

    A small, wearable, wristband housing sensors, microcontroller and BLE transmitter.

  • New Feather Wing viEwMotion Board

    Tom Meehan09/06/2017 at 04:53 0 comments

    I successfully etched a new board (after thickening traces, etc as much as possible).  I cleaned and etched the board, re-flowed all the SMD parts, drilled holes for jumpers and header pins (and hand soldered all of that stuff) it was time to check everything under the microscope (everything looked good, including continuity across ground and power nets).

    Now to plug the board into the Adafruit Feather board... again nothing seemed to work!  The board connected to my PC without a problem, loading and running simple programs like Blink also worked fine.  Time to look deeper!  Fairly quickly I found that I was getting no voltage out at the 3.3v pin even though continuity from the board to the pin seems good.

    Just in case I re-built the circuit on a solder-less breadboard and hooked it up to an Arduino Uno (actually I tested the pulse sensor functions first, then the GSR and finally both together) thankfully my circuits and data capture software all still functions correctly.

    In the morning I'll retest the Feather Wing board by hooking it up to the Arduino Uno to see if it actually works, if not I'll need to see what's wrong and then figure out why my Adafruit Feather board is not supplying voltage through the 3.3v pin.

  • Technical References

    Tom Meehan09/05/2017 at 04:17 0 comments

    This is not an exhaustive list but I will continue to add to it.

    Emotion and Physiology

    • Neuro-Tools: Emotion Detection January 16, 2017 https://acuityets.wordpress.com/2017/01/16/neuro-tools-emotion-detection/ accessed 3/28/17
    • Broek E, Schut MH, Westerink J, Tuinenbrejier K. Unobtrusive Sensing of Emotion (USE). Journal Of Ambient Intelligence and Smart Environments (2009) 287-299.
    • Kreiberg S, Autonomic nervous system in emotion: A Review. Biological Psychology 84(2010) 394-421
    • Levenson RW, The Autonomic Nervous System and Emotion. Emotion Review, vol , no. (April 2014) 100-112.

    Heart Rate Detection (ECG and Photoplysmography)

    GSR/EDA

    Heart Rate Variability

    • Goss CF & Miller FB. Dynamic metrics of heart rate variability. August 29, 2013, 4 pages. ArXiv:1308.6018.
    • Lee C, Yoo SK, Park Y, Kim N, Jeong L, Lee B. Using Neural Network to Recognize Human Emotions from Heart Rate Variability and Skin Resistance. Proceedings of the 2005 IEEE, Engineering in Medicine and Biology 27th Annual Conference, Shanghai China, Sept. 1-4,2005. p5523-5525
    • Bailon R, Laouini G, Groa C, Orini M, Laguna P, Meste O. The Integral Pulse Frequency Modulation Model with Time Varying Threshold: Application to Heart Rate Variability Analysis During Exercise Stress Testing. IEEE Transactions on Biomedical Engineering, Vol 58, No 3 March 2011. p642-652
    • Linares L, Medez AJ, Lado MJ, Oliviera DN, Vila XA, Conde I. An open source tool for heart rate variability spectral analysis. Computer Methods and Programs in Biomedicine 103 (2011) 39-50
    • Zhao M, Adib F, Katabi D. Emotion Recognition using Wireless Signals. MobiCom'16, Oct 3-7, 2016 http://dx.doi/10.1145/29373750.2973762
    • Valderas MT, Bolea J, Laguna P, Vallverdu M, Bailon R. Human emotion recognition using heart rate variability analysis with spectral bands based on respiration.
    • Lane RD, McRae K, Reiman EM, Chen K, Athern GL, Thayer JF. Neural correlates of heart rate variability during emotion. NeroImage 44 (2009) 213-222
    • Orini M, Bailon R, Enk R, Koelsch S, Mainardi L, Laguna P. A method for continuously assessing the autonomic response to music-induced emotions through HRV analysis. Med Biol Eng Comput (2010) 48-423-433
    • Williams DP, Cash C, Rankin C, Bernardi A, Koenig J, Thayer JF. Resting heart rate variability predicts self-reported difficulties in emotion regulation: a focus on different facets of emotion regulation. Frontiers in Psychology, March 2015, vol 6, article 261
    • Valenza G, Citi L, Lanata A, Scilingo EP, Barbieri R. Revealing Real-Time Emotional Responses: a Personalized Assessment based on Heartbeat Dynamics. Scientific Reports, 4:4998, DOI: 10.1038/srep04998

  • Progress is being made!!!

    Tom Meehan08/29/2017 at 04:18 0 comments

    It's been quite a long time coming but I've finally got some updates on this project.  I've been repeatedly delayed due to a vacation, massive computer failures, waiting for a new computer (then loading all the software etc.) and finally an illness.  None of this has completely stopped me from working on this project but it all has definitely slowed me down.

    After experimenting with the PulseSensorAmped (a great project and the documentation is really great) I decided to go back to using the MAXREFDES117.  The main reasons for this decision are:

    • more pre-processing in the breakout
    • easier control of the light intensity settings, etc.

    In addition I've decided to switch to a different circuit for GSR/EDC that makes some aspects of GSR measurement and interpretation simpler:

    • self adjusting to background skin resistance (tonic?)
    • easier to use

    Circuit was pulled from research paper (link), with the addition of the precision voltage divider.  In the same paper, the authors, looked at other testing sites for GSR/EDA that were comparable to the palm and fingers but allowed the subject greater freedom of movement.  There testing revealed that the palmar wrist area proximal (near) to the hand worked as well as the fingers and palm.  They even had success with a subject using a wrist strap for a week of continuous readings while performing normal daily activities.

    In initial tests I used the above circuit along with the MAXREFDES117 breakout board, both connected to the Adafruit Feather Bluefruit, to develop a simple program to make readings of GSR and HRV and send them back to the computer for interpretation.  (The ultimate goal, obviously, is to send the information via BLE back to a processor).

    I put this together after working out a number of bugs while building the circuits on a solder-less breadboard and testing different configurations. 

    One major issue, turned out to be the lack of insulation on the face of the MAXREFDES117 board that contacts the skin (used by itself it would not be an issue, but when coupled with GSR measurements it added current and potential the the GSR circuit).  Once I insulated all the possible skin contacts in the MAXREFDES117 all the GSR data returned to normal.

    I fabricated one initial (singled sided) board  that mounts directly onto the Adafruit Feather Bluedfruit so that I could start more comprehensive testing of individual GSR and HRV changes in relation to emotional stimulation.  After designing the board, etching it, populating it, re-flowing it, drilling out holes, soldering jumper wires, soldering pins to attach to the Adafruit Feather... something went wrong!

    It turned out that either during etching, re-flow, drilling or hand soldering -  a number of thinner tracks were either cut or pulled up.  So, I had to re-etch a new board, populate it and drill out the holes for the connecting pins to the Feather board and the smaller holes for the jumper wires.  This is where I'm at right now.  I'll have more to post in the afternoon (once I have the jumpers and header pins done)!!!

  • Changes and Refinements

    Tom Meehan07/03/2017 at 01:47 0 comments

    Initially i visualized a watch like design to house the sensors etc. that could be worn but I've now moved to using a wristband instead. The reasons for this change are: comfort (a soft, mildly stretchy wrist band - think sweat band - is more comfortable than a watch band) and better sensor placement for the heart rate sensor and for the GSR sensor.

    In addition to the above changes, I've also decided to switch to the Maxim breakout board (MAXREFDES#117) due to its better stability (and my better understanding about how to use it) as well as switching to a different circuit design for the GSR sensor module.

    The above circuit is taken from: A Wearable Sensor for Unobtrusive, Long-Term Assessment of Electrodermal Activity. IEEE Transactions on Biomedical

    Engineering. Vol. 57 No.5 May 2010 (with some minor changes - primarily the addition of the MAX6520 precision voltage divider). This circuit seems to need less adjustment between individuals. I've had to play around with the code for this setup (uses 2 analog readings to then calculate GSR) to ensure that I get enough resolution in the resultant signal. I may add an additional amplifier stage to each of the analog signals.

    In the same article this circuit design came from, the researchers tested the palmar side of the wrist for GSR measurements compared to fingers and both were equally as accurate. In addition, there tests involved longer periods (one subject wore the sensor for an entire week, comfortably) and their wrist band with Ag/AgCl electrodes showed little in the way of motion artifacts, etc., basically it was able to provide a stable and consistent signal even with a subject engaging in daily activities of living.

    As for the pulse sensor, further research led me to conclude the the back of the wrist is a poor site for recording a photoplethysmography signal (due to a lower concentration of surface capillaries). Thankfully, through experimentation, I found that the radial side of the palmar portion of the wrist seems to provide consistent readings when compared to a Polar Chest strap.

    The Adafruit Feather Bluefruit seems to be a perfect choice so far. I've been able to get the BLE communication working though I am still figuring out how to send the sensor data (I can manually send data from the serial monitor and receive it on an iPhone using Adafruit's Bluefruit Application iOS AppAndroid App  

    Presently I have everything on a breadboard with leads to the sensors held in place on my wrist with a wristband.

  • Ordered New Parts for Prototype

    Tom Meehan05/13/2017 at 20:40 0 comments

    Just ordered some new parts to work on a wearable prototype to read and collect bio-metric data. These parts don't quite fit the form factor that I'm aiming for but they make it much easier to work towards a more optimal design.

    I choose the Adafruit Feather M0 Bluefruit BLE as my microcontroller because it has a number of elements included that simplify prototyping:

    1. Arduino compatible microcontroller (lots of community support)
    2. Integrated USB communication (easier debugging)
    3. On board connection for LiPo power (including voltage regulator and charging)
    4. On board Bluetooth BLE

    I decided to change my pulse (photoplethysmography) sensor from the MAXREFDES177 Pulse/Ox sensor to the Pulse Sensor Amped - for 2 reasons:

    1. Only interested in pulse data (to determine heart rate) - don't need the oxygen saturation data.
    2. The Pulse Sensor Amped can directly output IBI (inter beat interval) data directly from a microcontroller (using open source software).

    It is likely possible to get this data from the Maxim breakout but the learning curve is to steep for me now. Another plus is that the Pulse Sensor Amped uses green LED's which are likely to make data collection from the wrist easier.

    Just a quick update and now I'm getting back to working on this.

  • viEwMotion - the beginning

    Tom Meehan05/02/2017 at 02:39 0 comments

    This project is an outgrowth of my Voight-Kampff project. My goal is to use what I've learned (as well as what I'm continuing to learn) about physiological markers for emotion and focus this on better methods of facilitating communication for people who have difficulty expressing emotions though facial expressions, vocal intonation or body movements.

    Currently there are a number of applications for detecting emotions through analysis of facial expressions (Microsoft's Emotion API, Empatica's facial analysis, etc). These do a truly amazing job at detecting emotional expression and their abilities are growing daily but they only work with those who have no impediments in expressing emotions.

    The above technologies offer great promise in helping people learn to recognize the non-verbal signals of emotion (facilitate recognition of emotional states in others). This is an application that is being pursued, especially to help people who fall in the autism spectrum.

    For those who do have difficulty (due to various issues) in outwardly expressing emotion, there does not seem to be similar efforts (at least that I've found so far - please let me know if I'm incorrect!). The primary goal of viEwMotion is to address that very issue.

    There are numerous other potential applications for viEwMotion ranging from research into emotional response (especially by Citizen Scientists), game design and play (increased immersion due to game response to player emotions) and even market research (analyzing emotional response to advertisements, etc.).

View all 7 project logs

Enjoy this project?

Share

Discussions

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates