Close
0%
0%

AI-driven BLE Travel Emergency Assistant

Detect keychains to inform emergency contacts via WhatsApp / SMS. Display results via BLE. Let contacts request location info from Maps API.

Similar projects worth following
481 views
Heartbreakingly and unfortunately, people with disabilities are more likely to be victims of violence, theft, verbal abuse, and neglect while traveling, especially travelers with mobility impairments. In overall crime estimations, people with disabilities have a higher risk, at least four to ten times, of being a victim[1]. Considering the discouragement of these heinous crimes hurting vacation plans of people with mobility impairments, it has become necessary to provide them with state-of-the-art assistive devices so as to preclude offenders from committing crimes against people with disabilities.

While traveling alone and feeling in danger, one of the most crucial steps to prevent a potential crime is to let your emergency contacts know your whereabouts with a brief explanation of your situation. Although smartphones provide various features regarding location tracking and wireless communication, they might still be not suitable since reaching and utilizing a smartphone may be time-consuming and arduous in a time of crisis for people with mobility impairments. In light of recent developments in machine learning and IoT, there is a surge in devices enhancing smartphone features in the form of automated notifications and additional sensors, e.g. smartwatches and fitness wearables. Thus, in this project, I focused on developing an AIoT assistive device that improves smartphone features to inform emergency contacts automatically and instantly of the user's situation.

Approximately 15% of the population is affected by some kind of disability, and this number is rising exponentially due to aging populations and the spread of chronic diseases, according to WHO reports[2]. In this regard, budget-friendly and accessible AIoT assistive devices should be versatile and provide various features in a broader spectrum, considering people with disabilities and special needs.

After inspecting recent research papers on assistive devices, I noticed there are nearly no wearable appliances focusing on detecting personalized items to execute some predefined functions covertly, such as automated notifications, by utilizing smartphone features. Therefore, I decided to build a user-friendly and accessible assistive device to detect customized keychains (tokens) with object detection and inform emergency contacts of the user's situation automatically.

Since XIAO ESP32S3 Sense is an ultra-small-sized IoT development board providing a built-in OV2640 camera and a microSD card module on its expansion board, I decided to utilize XIAO ESP32S3 in this project. Thanks to the integrated modules on the expansion board, I was able to capture images and save them to the SD card as samples without requiring any additional procedures. Furthermore, XIAO ESP32S3 comes with integrated Wi-Fi/BLE connectivity and 8MB PSRAM. Therefore, I was also able to run my object detection model consecutively. Then, I connected the XIAO round display to XIAO ESP32S3 in order to notify the user of the current saved sample numbers on the SD card and the ongoing operation by showing assigned icons.

Since I wanted to capitalize on smartphone features (e.g., GPS, GPRS, BLE) to build a capable assistive device, I decided to develop an Android application from scratch with the MIT APP Inventor. As the user interface of the assistive device, the Android application can utilize the celluar network connection to transfer data packets to a web application via GPRS, obtain precise location data via GPS, and communicate with XIAO ESP32S3 via BLE so as to get model detection results and transmit commands for data collection.

After developing my Android application, I designed various keychains (tokens) denoting different emergencies and printed them with my 3D printer. Then, I utilized the Android application to transfer commands to XIAO ESP32S3 via BLE to capture images of these customized keychains so as to construct a notable data set.

After completing my data set, I built my object detection model with Edge Impulse to detect customized keychains (tokens) denoting different emergencies. I utilized Edge Impulse FOMO (Faster Objects, More Objects) algorithm to train my model, which is a novel machine learning algorithm that brings object detection to highly constrained devices. Since Edge Impulse is nearly compatible with all microcontrollers and development boards, I have not encountered any issues while uploading and running my model on XIAO ESP32S3. As labels, I utilized the names of emergency situations represented by customized keychains:

  • Fine
  • Danger
  • Assist
  • Stolen
  • Call

After training...

Read more »

  • 1 × Seeed Studio XIAO ESP32S3 Sense
  • 1 × Seeed Studio XIAO Round Display
  • 1 × MicroSD Card
  • 1 × Anycubic Kobra 2
  • 1 × Xiaomi 10000 mAh Ultra Compact Pro 3 Powerbank

  • 1
    XIAO ESP32S3 Sense

View all instructions

Enjoy this project?

Share

Discussions

kutluhan_aktar wrote 11/16/2023 at 18:46 point

Please feel free to leave a comment here if you have any questions or concerns regarding this project 😃

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates