Close
0%
0%

Alarm detection for hearing impaired

A small portable unit for detecting alarms for hearing impaired person. It flashes its screen and LED when an alarm is detected.

Similar projects worth following
One of the places I lived in years ago had no smoke detector and I was lucky to woke up in the middle of the night because it was too warm and I smell smoke. It turns out that someone was drying futon next to the heater and it caught on fire. I managed to put out the small fire on that futon. i was lucky because I wake up very easily and a few more minutes the smoke would have done me in.

A smoke detector would have helped in that situation. To a hearing impaired person, the regular alarm might not be effective. This project is small portable unit for detecting alarms. It flashes its screen to alert the user when loud alarm is detected.

It looks familiar because this is based on the some of the work in my "Automatic audio source switching" project for the spectrum analyzer, VU metering and loudness threshold detection. The spectrum analyzer can show a real time plot of the audio as well as the VU level.

From https://en.wikipedia.org/wiki/Fire_alarm_notification_appliance

The primary function of the notification appliance is to alert persons at risk. Several methods are used and documented in industry specifications published by UL.
Alerting methods include:
* Sound (audible signals)

  • ~3 kHz / ~3100 Hz tone (high frequency). Used in many current notification devices.
  • 520 Hz (low frequency). Used in newer notification devices.
  • 45 dB to 120 dB A weighted for human hearing (higher decibels, in the 100 to 120 dB range, were common with older fire alarm horns)

The idea is to detect the audio frequency used for these alarms. The unit can flash a bright LED and display the spectrum plot and VU for the volume. The user can also use the VU meter to find the source of the alarm.

Design Concept

This project is based on some of my work from my other project: Automatic audio source switching The Spectrum analyzer, loudness threshold detection are useful for detecting loud alarm.

This is what the old 3kHz smoke detector sound sample looks on the spectrum analyzer directly feed from my PC. There are some higher harmonics in the sample showing up.

The volume can be used to distinguish the difference between a beeping from a regular alarm (e.g. microwave oven) or a real alarm.

Audio is captured by the microphone and an amplifier is used to provide gain and DC offset for the ADC. The audio data is stored into a ping-pong buffer in the background by the DMA controller. When a buffer is filled, the audio processing routine is called to compute the DC offset, volume and battery level.

The volume data is plotted against time and an integer FFT is used for the spectrum plot. Frequency resolution is increased by lowering sampling rate and larger sample size for FFT.

The low power consumption of this device is achieved by:

  • Clock frequency is reduced (48MHz rated) to 8MHz. PLL is disabled to further reduce power consumption.
  • ARM core is put into sleep mode whenever possible.
  • LCD backlight is turned off automatically when there is no audio input.
  • DMA is used to transfer data, move and clear memory.
  • The operating voltage is decreased to he lowest minimum voltage of 2.7V. This reduces the power needed to be delivered by the boost converter.

Display Design

Prototype

For prototyping, I am using my spin-off sub-project LCD Backpack module along with custom PCB(s) for the analog front end microphone and I/O.

The weight of the prototype is 69 grams (2.43oz).

Current firmware power consumption is ~10mA (max). A pair of AAA NiMH batteries such as Energizer Rechargeable AAA-1000 can easily power the device for ~100 hours.

Youtube video demo for the contest


Project logs

Design:
Rough calculation
Processor load estimation

Hardware activities:

STM32F030F4 based LCD Backpack Sub-project for the processor module
LCD Backpack assembled and working
Case construction
Amplifier for the mic input
Power consumption profile

Misc:
Another "Lost in space" Github release Initial Github release
Youtube alarm collection A collection of youtube videos that can be useful for testing

Useful URL:

https://en.wikipedia.org/wiki/Fire_alarm_notification_appliance
http://www.sengpielaudio.com/calculator-soundlevel.htm

http://www.ni.com/white-paper/4278/en/
"The Fundamentals of FFT-Based Signal Analysis and Measurement in LabVIEW and LabWindows/CVI" This is background information for FFT and windowing.

https://hackaday.io/post/39702
Additional details of calculating DC offset in samples and volume calculation.

  • Where to go from here?

    K.C. Lee10/09/2016 at 00:16 2 comments

    The latest github update includes the frequency read out and all the visualization goodies. I have released a youtube demo in project detail. The project now has reach a minor milestone just in time for the HaD Contest round.

    It is one thing to help the user to recognize fire alarm by alarm frequencies and their loudness.During the course of looking at different alarms, I came to the conclusion that the alarms are different enough that a simple threshold is not sufficient. Thankfully, the human brain is pretty good learning device and should be able to figure things from some form of training.

    The next logical step is a bit more difficult. There got to be a way to automate the process at the repetitive nature, high intensity and the spectrum. This sounds like a job for some kind of neural net. (pun intended.) Unfortunately, that's a subject a bit outside of what I know currently.

    The next step

    The next step I can think of is to collect internal data - spectrum + volume and transferring to a PC for analysis/training.

    Audio data is collected in blocks about 40 times a second. The DC offset, instantaneous volume, average volume and peak volume are computed. The instantaneous volume is used for plotting the volume envelope. It is useful as a summary to show the change in volume than a meaningless jumble of raw waveform. It is stored as an 8-bit integer.

    The screen update rate currently is once every 125ms. This is a trade off between power consumption and what the LCD can handle without turning everything into a blur. For this application, it seems to be sufficient.

    The FFT is computed every 125ms. There are 128 data points for the magnitude (in log scale) of each of the FFT between 0Hz to 5120Hz in 40Hz increments. Each data point is an 8-bit integer mapped to a pixel on the display.

    Data rate

    So the bare minimal data that needed to be transfer is:

    • 1 byte of instantaneous volume every 40ms
    • 128 bytes of amplitude every 125ms.

    Or 128 + 5 = 133 bytes/125ms or 1064 bytes/sec

    The most straight forward way of transferring data to the PC is via serial port. At 115200 bps, 11520 raw data bytes/sec can be transfer. Since we got plenty of bandwidth, I can transfer the 64 bytes of current and previous instantaneous volume and make it easier for analysis.

    Serial data is a byte stream, so there need to be a way of packetizing the data so that the PC can find the starting point of a packet. I can use a single byte with MSB = '1' to mark the start of frame. Individual bytes can easily fit inside the remaining 7-bit of data.
    e.g. something like this:

    An additional CRC byte *could* be used for the receiver as a checksum. The logistic of protocol and error detection/recovery are overhead that I might not bother with.

    These data can be transfer in a raw binary format to the PC via DMA with minimal overhead in the background. This is similar to the way I am transferring bitmap graphics to the LCD via SPI using DMA without acknowledgement.

  • Visualization (in progress)

    K.C. Lee09/26/2016 at 18:20 0 comments

    This is the visualization I have at the moment. It pretty much a work in progress.

    The display is splitted into two halves. The top half shows the instantaneous volume plot. It is a bit easier to read than a raw waveform display.

    The bottom half is the spectrum display from There is battery voltage readout as well as a gauge.

    This is from my alarm clock. There are 4 peaks that corresponding to the 4 beeps it make in a row. The frequency is around 4kHz.

    It is a bit of potato quality as it is hard to take a picture of the scrolling display. I might have to slow down the rate of change.


    I did some code cleanup. Also coded up DMA version of memset() and a cut down special case of memmove() for the volume plot. New version of the code for these improvements is up on github.


    I did some more work on the visualization. Once again shown with my alarm clock as I don't want to scare my neighbors.

    Frequency readout
    The code finds the FFT bin with the peak value and prints out the frequency. The frequency resolution is 40Hz, so the tolerances could be up to +/- 40Hz.

    It is determined by the sample size for the FFT and the ADC sampling rate.

    (10240 samples/sec / 512 samples) /2 = 40Hz
    The frequency range is 120 - 5120 Hz

    Signal level
    The code also compute a RMS of the signal spectrum. This is plotted as a dash line (in log scale) between the spectrum plot and the volume. This value is used as a baseline noise level for detecting a peak frequency in a noisy environment. The readout is updated if a peak is above the threshold from this line.

    Algorithm

    Noise = Peak_Value = Bin = 0;
    for
    (i=0;i< N_FFT/2;i++) { Sp = &Plot_Data.fft_data[Tbl_brev[i]]; Magnitude = (uint32_t)(Sp->r*Sp->r) + (uint32_t)(Sp->i*Sp->i); if(i>= SPECTRUM_START) Noise += Magnitude; // Magnitude^2 to dB scale conversion *Mag++ = Mag_dB
    = Lookup(Magnitude,fft_dBScale,sizeof(fft_dBScale)/sizeof(uint32_t)-2); // Find peak frequency if((Mag_dB > Peak_Value)&&(i>=FREQ_START)) { Peak_Value = Mag_dB; Bin = i; } }
    Plot_Data.noise = Lookup(Noise/(uint32_t) SPECTRUM_BIN,fft_dBScale,sizeof(fft_dBScale)/sizeof(uint32_t)-2);

    Here is a demo on youtube:


    I use Audacity to generate 3000Hz and 520Hz sinewave and play back on my PC speaker.

  • Real life fire alarm tests

    K.C. Lee09/15/2016 at 18:42 0 comments

    My apartment building was running an annual fire alarm test last Friday. The actual test was a bit short.

    This is from the old smoke detector:

    This is the new type of centralized and addressable fire alarm (Signalink Fire-Link® II) installed in each of the apartment suite. It is 20 feet away from my detector.

    This is the fire alarm from the corridor.

    We even had a big finish with a real fire alarm later in the day. The usual garbage room fire cause by someone throwing lighted garbage into the chute. :P

    This is me standing at that sign about 15 feet from the apartment entrance next to the big fire engine parked in front of my building. There is a lot of ambient noise, but you can still see the spike of the fire alarm from the building.

    Sorry about the picture quality as it is done from screen capture of a free hand video of the unit inside a plastic bag. They are shot as video so that I have an audio sample of the alarm. I didn't want to stand around at the fire scene with a hack together circuit with mess of wires coming out of it looking at a bomb.

  • Power consumption profile

    K.C. Lee09/01/2016 at 01:32 0 comments

    I put a 10R resistor in series with the power supply. Here is a scope picture of the current consumption profile.

    Since R=10R, so 25mV corresponding to 2.5mA.

    Here is the CPU load I collected earlier (updated in this project log). The trace goes high when the CPU is busy. The cycles are 25ms corresponding to a block of data collected from the ADC. The activities are VU processing, FFT and rendering/writing to LCD. LCD data transfer is handled by DMA, so the processor take an early nap. The processor is put to sleep in between these activities.

    The power consumption can be correlated to the CPU activities. Power consumption is 10mA when CPU is in sleep mode. The current rises to 12.8mA when the processor is awake and processing the data.

    The peripherals: DMA, ADC, and timer are continuously collecting analog samples in the background.

    This is why I have decided to lower the clock speed instead of just relying on sleep mode to save power.

    The CPU is busy about 60% of the time. So average current is (60% * 12.8mA) + (40% * 10mA) = 11.68mA

    The current measured is the overall system level consumption from NiMH batteries. The current consumption at 3.3V without the boost converter is around 4mA.

    So efficiency is around 50% range. The boost mode supply I have isn't exactly efficient for very light loads. I used it out because that's what I have on hand. It is a $0.20 1.4MHz chip that I ordered from Aliexpress intended for LED.


    I did some thinking - actually more an unintended afternoon nap. It would seem that it is possible to run everything except the LCD from the battery without regulation to vastly improvely on battery life.

    STM32F030 supply is 2.4V-3.6V. Since the processor is running at 1/6 its rated speed, it might be able to handle an additional voltage drops in the supply. There might be some levels of ADC performance degrade as it is partially analog. I am really not asking much from the opamp in the mic amplifier terms of slew rate, bandwidth and output current, so I might be able to get away with that.

    The LCD is affected by supply voltage. It could be power off a low current charge pump post regulated by a LDO. The SPI driving the display is unidirectional which simplifies level translation. The low end of 2.2V VOH from the STM32F030 is 66% of the LCD supply.

    While datasheet say 70% Vcc (min) for VIH, it is more a guardband in most cases. The real logic thresholds is around at mid rail depending on the level of symmetry of the NMOS and PMOS transistor ( which should be one of the controlled parameters for production). One trick I used in the analog MUX in my automatic audio switching project is to run the MUX at 4.5V to lower the VIH to 3.15V. A 3.0 LDO or simply a schottky diode or resistor should drop the supply enough to follow the specs.

    Not sure about the LCD backlight though as it would need around 3V or so and require some switch mode supply gymnastics if I don't want to rewire the backlight connections on the LCD PCB assembly.

    Anyways, this is just a thought for future hacks/improvements. Right now battery life is decent.


    I reduce the regulator output to 2.7V by changing R2 to 100K and R3 to 8.25K. This is a compromise to keep all of the components are still within their specs. The power consumption is now dropped to 7.7mA. It also reduces the LCD backlight brightness, but it is still readable in the dark.

    Power consumption measured with Mooshimeter.

    Future Improvements:

    There is room for power supply efficiency improvement to 75-80% by using a better part. I would stay away from PFM scheme as I learnt that lesson the hard way here.

    System level power at 4mA isn't bad, but that can be improved as the STM32F030 I am using is their value line and isn't exactly designed for low power. The level of performance is pretty impressive for the price.

    ST has the STM32L0 series that are designed to be low power and runs as low as 1.65V. Their free MDK license also covers for the 32M32L0 series. :)

    STM32L031F4 is closest to the STM32F030F4.

  • Amplifier for the mic input

    K.C. Lee08/30/2016 at 23:25 4 comments

    I am going to try building this amplifier circuit. I have decided to split the 100X gain into two 10X gain stages.

    This is because of the input offset of the cheap opamp also get amplified. e.g. 3mV input offset becomes 300mV at the output which is about 10% of the full scale. For 10X gain, it is only 1% which isn't too bad. The second stage is AC coupled, so the DC offset of the first stage is eliminated.

    The bandwidth of the amplifier and the anti-alias filter is about 5.6kHz. I'll have to go to my PCB bone pile to try to find a 3.3K resistor to tweak it a bit lower (work for later).

    The anti-alias filter is on the LCD backpack PCB. Amplifier circuit is as follows.

    Here is a single side PCB layout for the amplifier circuit.

    I have decided to build this amplifier using toner transfer as it is simple enough. See my other project for an example)


    Found a mic on my old telephone. It seems like the gain of the amplifier is a bit high as there is quite a bit of noise showing up. I'll have to find some more mics to try out.

    Here is with my digital alarm clock a feet from the mic while the TV is on in the background. That distinct peak is the beep. This is the advantage of showing the frequency domain instead of the input waveform.

    Here is the background noise level while I am watching TV.

    I could use a volume control or use some adaptive algorithm. Haven't decided.

    I am hoping that a real alarm would stand out from the background noise. I'll have to talk to my rental office to find out their schedule for their monthly fire alarm testing.


    I played with 3 of the mics I found and they don't make too much of a difference.

    The gain of the amplifier is just about right for normal volume. It is too high for this application as the output will clip. I have reduced it to around 20X by changing R8 to 100K for now. The amplifier could have been implemented in the first stage amplifier.

    I have also played with the power filter on the amplifier. The old filter was more effective filter for high frequency, but most of the noise are in the lower frequency. I replaced the ferrite L1 with a 330R resistor and the 10uF (C1) cap with a 22uF electrolytic. The low pass filter is about 22Hz. This seems to drop the noise floor down a bit. Further increasing the capacitor has minimal effects.

    After looking at the Power consumption profile, I now understand the filter requirement. The CPU goes to sleep and wake up to process a block of data and as a result the power consumption fluctuates at 40Hz which can result in a change in the voltage regulation.

    The low pass filter is at 22Hz thus reducing the effects of the 40Hz noise. Further lowering the filter frequency has minimal effects.

    This is also the reason why the noise floor actually drops (instead of rises) when the LCD backlight is on as the amount of fluctuation is a much smaller percentage of the overall current consumption.

    The switching frequency of the power supply is at 1.4MHz while the analog circuits has a bandwidth of about 5kHz. The frequency plot shows that the analog circuit is insensitive to the switching harmonics. Contrary to the popular myth, it seems that I got away using switch mode supply with analog circuits.

  • Case construction

    K.C. Lee08/23/2016 at 04:10 0 comments

    This is based on some previous work here. The case is an AAA battery case from China. It is similar to Eagle Plastic Devices 12BH4441AS.

    I have rewired the holder to 2 AAA batteries and use the remaining space for the electronics.

    I removed the divider by scoring on the edges and carefully cracking the plastic with a plier. There is about 0.8" x 1.8"x 0.44" (20.32mm x 45.72mm x 11.17mm) for electronics.

    I aligned the LCD and marked the pins with a mechanical pencil. I drew an outline with pencil and slowly cut a slot in the plastic with a box cutter.

    This is how it looks inside. There'll be an analog front end amplifier driving the ADC.

    This is how it looks on the outside.

  • LCD Backpack assembled and working

    K.C. Lee08/22/2016 at 20:18 0 comments

    After a lot of waiting, the PCB for LCD backpack has finally arrived. This is used as a module for the prototype.

    The firmware is based on the Spectrum Analyzer of my Automatic Audio Source Switching. The sampling rate has been decreased while the number of data points have been increased to provide a much high resolution spectrum plot for 0-5kHz.

    Here is the familiar display after I programmed the firmware.

    The preliminary power consumption from NiMH battery is around 27mA with backlight and 14mA without.

    I have switch off the PLL and run the uC directly from a 8MHz crystal to lower the power consumption. Current dropped to 22.78mA with backlight and 9.48mA without.

    This should get some decent operating time from a pair of AAA batteries.

  • Another "Lost in space" Github release

    K.C. Lee08/18/2016 at 00:42 0 comments

    Seems like I am not having much luck with OSH Park PCB getting lost in the mail. This is the second time in a row that I have not received the shipment and likely not the last one. Ordering things from China hasn't been to reliable since the beginning of this year. I have no cash flow for a good alternative either.

    There hasn't any interest in this project, so I have decided to do a pre-emptive release of the firmware that I modified from my Automatic Audio Source Switching project.

    Github: https://github.com/FPGA-Computer/Alarm-Detection


    The PCB I ordered finally arrived a month after I ordered it.

  • Youtube alarm collection

    K.C. Lee07/20/2016 at 00:13 0 comments

    This is a collection of youtube video with alarms for testing purpose only. This is a bookmark only.


    http://soundbible.com/1656-House-Fire-Alarm.html

    Simple electronic ones (e.g. smoke detector) are easier to decode as they have a pure tone. The sweeping frequency actually looks kind of neat. The more complicated ones have a broader frequency.

  • Processor load estimation

    K.C. Lee07/19/2016 at 19:33 0 comments

    I did some modification to my audio switch code. This test give me some rough idea of CPU cycle and memory requirement.

    The sampling rate is set to 10240 samples/sec. This gives a 5kHz bandwidth which is the most sensitive hearing band for human. The raw block of ADC data consists of 256 samples arrives every 25ms.

    256 samples /10240 samples/sec = 25ms.
    FFT resolution is 10240/(2 * (256/2)) = 40Hz.

    The RAM usage is close to full. 256-point FFT is all the part can handle. On the other hand, 40Hz resolution is good enough, so this is okay. I have not done any memory optimizations yet.

    At 48MHz CPU speed, it takes about 3.14ms (3rd blip below) for the DC offset, basic volume calculation, 256-point FFT and screen update. This leaves plenty of spare CPU cycles until the next block of data. The code skips the FFT and screen update when there was no audio detected.

    The 256-point FFT code takes 1.74ms to execute.

    I can put the ARM in sleep mode to reduce a tiny bit of power. More power saving can be achieved by reducing the clock speed and disabling the PLL.


    I have reduced the CPU clock to 8MHz (external crystal = 8MHz, PLL off) to reduce power. The processing takes a bit longer, but still leaves about 40% of CPU cycles free.

    Mic, amplifier, ARM and the LCD (backlight off) draws about 4mA from 3.3V (without DC-DC converter).


    I am cleaning up the plotting routines. I am splitting off the dB look up from the plotting routine. A binary search function is used for converting the raw volume and FFT magnitudes to their dB equivalent. The new code frees up a bit more processing time and leaving ~50% free cycles.

View all 11 project logs

Enjoy this project?

Share

Discussions

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates