Close
0%
0%

PionEar: Making Roads Safer for Deaf Drivers

PionEar provides early warning to deaf drivers of an approaching emergency vehicle

Public Chat
Similar projects worth following
PionEar is a smart, small-size sensor that helps hearing-impaired drivers be alerted in time to an approaching emergency vehicle. It provides a bright visual alarm in the driver’s field of view. This helps to increase driving comfort and reduces the risk of an accident.
The PionEar sensor utilizes artificial intelligence to analyze external sound patterns. It leverages Syntiant‘s TinyML development board to run the machine learning algorithms in an extremely energy-efficient way. This makes it possible to power the sensor for long days without charging or later use the solar panel so that it does not need to be re-charged at all.
I learned about the issue of deaf drivers not hearing sirens. Contacting the hearing-impaired community showed a positive need for the device. I want to spread this idea across the maker community and get this device to real users.

The PionEar sensor concept is based on Syntiant TinyML board.  TinyML board is the ideal platform for building low-power voice, acoustic event detection (AED) and sensor ML applications. Equipped with the ultra-low-power Syntiant® NDP101 Neural Decision Processor, the TinyML board packs native neural network computation for the most demanding applications in the lowest power envelope.

The onboard microphone and BMI160 sensor enable easy configuration for any speech, AED or 6-axis motion-and vibration-related application. Trained models can be easily downloaded on the TinyML board through a micro USB connection without the need for any specialized hardware.

The below picture shows how the TinyML board is interfaced with other external circuitry. The external custom-designed PCB contains RGB LED for logo illumination, DC to DC power supply, and a phototransistor acting as a light sensor. The light sensor provides a signal (AIN) that is sensed by the TinyML board, which controls the brightness of the logo. This prevents the driver from being dazzled at night but enables good visibility in the daylight.

The li-Pol battery cell (800mAh) is connected directly to the TinyML board battery connector. It directly charges the battery when connected to a power supply via a micro USB connector. The whole system draws about 12mA while sensing and processing the sound signal so the battery can last about 3 days. I plan to significantly prolong battery life by leveraging integrated (or external) accelerometer. This will be sensing if a car is moving – if not, it will put the TinyML board into the deep sleep state where current consumption is negligible. The sensor will then operate only when car is driven.

In order to avoid charging the sensor at all, I plan to use a small solar panel and integrated module with MPPT functionality. I suppose there is plenty of time during a normal day when the sensor can be recharged unless the user is parking in the garage. The goal is to provide a device you do not need to care about at all.

Fig. 1) PionEar sensor HW block diagram

In order to deploy the machine learning model into a TinyML board the Edge Impulse platform can be preferably used. Edge Impulse provides easy to understand interface where you can create and manage your datasets, extract features, train ML models, test and finally deploy into numerous of supported hardware platforms. The TinyML board is fully supported so I have leveraged this advantage.

First, it is necessary to have a suitable dataset of emergency siren sounds and other road noises. One can find many public datasets – I have used Large-scale audio dataset for emergency vehicle sirens and road noises as a basis for my ML model. I have used other datasets of noises and speech to mix with the original dataset. My dataset including ML model is publicly available on the Edge Impulse platform – you can find it here. I expect, that I will further evolve it as I will continue with the prototype testing.

I consider the current version of the ML model sufficient for testing in real conditions. It can safely detect ambulance sirens and exhibits a low amount of false positives when exposed to road noises while driving. There are still some weak points for example music when listening to the radio in car. However, music was not part of the dataset because I expect that hearing-impaired people typically do not listen to the radio in a car 😊

Fig. 2) ML model accuracy for testing dataset

TinyML_extension_LED_board_schematic.pdf

Schematic for extension board

Adobe Portable Document Format - 126.78 kB - 05/26/2023 at 10:31

Preview
Download

TinyML_extension_LED_board_eeBOM.xlsx

BOM file for extension board

sheet - 10.89 kB - 05/26/2023 at 10:31

Download

TinyML_extension_LED_board_Assembly.pdf

PCB assembly drawing for extension board

Adobe Portable Document Format - 46.46 kB - 05/26/2023 at 10:30

Preview
Download

Gerber_TinyML_Extension_LED_board.zip

Gerber data for extension LED PCB

x-zip-compressed - 29.48 kB - 05/26/2023 at 10:30

Download

PionEar_car_holder_adapter_v3.step

3D model of adapter for standard magnetic car holder used eg. for smartphones. This is needed to avoid blocking of the microphone port.

step - 124.07 kB - 05/25/2023 at 20:30

Download

View all 6 files

  • 1 × Syntiant TinyML board
  • 1 × Lithium polymer battery 800mAh (802535)
  • 1 × Slider switch SS12D00
  • 1 × Phototransistor WP3DP3BT
  • 1 × LED extension custom board Schematic, BOM, Gerbers provided in the file list

View all 9 components

  • Solar charging test

    Jan Říha05/25/2023 at 18:47 0 comments

    In the next iteration, I would like to integrate a solar panel into the PionEar sensor to avoid the need for charging. I did some research for suitable low-power solar charger modules with MPTT functionality. I discovered AMELION from @Jasper Sikken. In combination with IXYS SLMD960H09L solar panel, it seemed to be a good starting point. AMELION module is a great solution for low-power applications. It can still provide some current even in the indoor environment. I could measure the following charging currents:

    Indoor: 20 to 50 uA

    Outdoor (cloudy/shadow): 2 to 5 mA

    Outdoor sunlight: 25 to 30 mA

    These numbers seem to be promising. I think that behind the car's front window, there should be plenty of light to keep the battery charged - unless you park in the garage.

  • Test in the car

    Jan Říha05/23/2023 at 21:10 0 comments

    Usually, as a first test for audio event detection, I use YouTube as a source for various audible events. But the real-life test is irreplaceable. In the case of PionEar sensor, it was a bit tricky to catch the ambulance with sirens on and have a camera ready. Last time I spent 2 hours driving around the nearest hospital and wasn't able to meet any car with sirens :-) Today, on my way to work I was lucky and made a shot.

View all 2 project logs

  • 1
    Light diffuser for illuminated logo

    Some of my friends have been asking, how to create an illuminated logo or symbol in 3D printed enclosure. I've been experimenting with FDM print and UV resin for SLA printers. In the end, I came up with a way that works pretty well for my project. Below are some steps to be followed:

    - Print your logo/symbol in negative. The wall thickness is 1.5mm in my case. Use only smooth print sheet (not textured one).

    - Use some tape to cover the logo from one side. I have used double side tape because it seemed to be well covered by glue and was very sticky.

    - Carefully fill the logo with white UV resin for SLA printers. Try to avoid any bubbles.

    - Put under UV light. I have used UV10A: 15W, 850lm, UVA + UVB. Put at least 10 to 15cm below, but not much closer. I have experienced that bubbles started to form in resin when I was too close to the light source. Cure about 3 to 5 min.

    - Illuminate from the other side too after peeling off the tape cover for another 3 to 5 min.

    - Peel off the rest of the tape, clean with IPA

  • 2
    Housing assembly

    Few instructions for housing assembly:

    - Insert and press-in the magnets, no glue is needed. Keep the same orientation.

    - Place rubber o-ring, insert TinyML board, and tighten the plastic crew. Check that the microphone port is visible from the backside. Sealing the microphone is important for a proper acoustic response.

    - Press-in the slide switch

    - Assembly the magnets into a magnetic holder adapter and attach the metal sheet backplate. The holder adapter ensures that the acoustic port is not blocked when sensor box is attached to a standard smartphone holder.

  • 3
    Wiring diagram

    Follow the wiring diagram below to connect the TinyML board and the LED expansion board.

View all 3 instructions

Enjoy this project?

Share

Discussions

craig wrote 05/25/2023 at 03:51 point

I had been thinking about this sort of project for people in general and more particularly that it would warn if the emergency vehicle was approaching.

I would avoid using a battery powered device as they degrade and can become dangerous in a hot vehicle. May be less of a problem in the UK, but a major issue elsewhere. I don't think there would be a problem with the device being powered via the 12 Volt system plug.

  Are you sure? yes | no

Jan Říha wrote 05/25/2023 at 19:16 point

Hi, thanks for your comment. Ideally, I would like to deliver a device without any plugs and cables. Even though I understand that in the car you have a practically infinite source of energy :-) You're right about the Li-on/Li-pol limitation. For the concept, it is the easiest solution and I can live with it for some time. In case of wider use, I would consider using eg. Lithium ion capacitors:

https://www.digikey.cz/en/blog/lithium-ion-capacitors-can-help-you-provide-high-quality-power

Although it has lower energy density, it can still be a good choice when solar charging will provide a positive energy balance for most users.

  Are you sure? yes | no

nramkarran wrote 05/24/2023 at 16:39 point

This looks amazing! My main problem though is direction. I just can never pick up the directionality of the siren. My hearing issues are age related, so not total hearing loss.

  Are you sure? yes | no

craig wrote 05/25/2023 at 03:48 point

I don't think this an age related issue. It is a function of the sound isolation in modern vehicles, which makes it difficult to hear sirens and then to determine their direction. Direction is more difficult to determine as the sound can bound off building and leak into the vehicle from a location that isn't in the direction of the emergency vehicle.

A project like this example potentially can determine if the emergency vehicle is approaching or leaving via the Doppler effect on the pitch.

  Are you sure? yes | no

TheGrim wrote 05/23/2023 at 12:29 point

It's a cool concept. I like what I see. Cheers.

  Are you sure? yes | no

Jan Říha wrote 05/23/2023 at 21:26 point

Thank you!

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates