Close

We splunkified the data: internet of useful things

A project log for HeartyPatch: A single-lead ECG-HR patch with ESP32

HeartyPatch is a fully open-source, IoT connected, BLE enabled heart-rate variability & ECG patch with great accuracy

ashwin-k-whitchurchAshwin K Whitchurch 05/18/2017 at 07:280 Comments

Once we took the HeartyPatch data to Amazon Web Services IoT (https://aws.amazon.com/iot/), we wanted to visualize the data and run some analytics on it to find the heartrate "trend" throughout the monitoring period (could be a day). AWS is a managed IoT platform hosted on the AWS cloud. This makes it easier to send "thing" data to the cloud for storage and processing. The reason for choosing AWS IoT now is mainly because of security and ease-of-use reasons.

AWS IoT, however, does not store any data (not persistent). It just uses the received data to trigger actions or "rules". Internally, it uses the MQTT (http://mqtt.org/faq) to transfer to and from a thing, in this case the HeartyPatch. MQTT is very light-weight compared to HTTP or HTTPS and still can use X.509 certificates for authentication.

Enter Splunk; a "big data" platform, or in simple words, it just reads machine-generated data and stores over a long period of time. The reason we chose Splunk is actually from a suggestion from a customer of ours for another project, but we liked it so much that we decided to keep using it.

We installed Splunk on an AWS EC2 instance and used AWS Lambda (https://aws.amazon.com/lambda) to pick-up the data from AWS IoT and post it to the Splunk instance as a HTTP request. Spunk's internal HTTP Collector modules then receives the HTTP request, classifies the data and stores it. However, all said, Splunk is NOT Open-source or even free for commercial application. It is free to use for individual use, with some restrictions, such as the data transfer limit of 500 MB/day.

Now, it is possible to do all kinds of stuff with the data. In the screen-shot below, we have taken a small part of the data to analyse the heart-rate trend and calculate variability.


Discussions