Close
0%
0%

Street Sense

Portable electronic device to measure air and noise pollution

Public Chat
Similar projects worth following
Street Sense is a project to build a portable, battery-powered sensor unit to measure:

Air Quality: Ozone, NO2, Particulates
Noise Pollution

There are two NGO sponsors. They require a sensor that can help them answer some important questions:
1. What air and noise pollution effects are experienced by people at the street level?
2. Can we quantify the changes in street level pollution resulting from a capital infrastructure project?

The project name "Street Sense" seems appropriate - sensing the environment at the street level in the urban landscape

A kick-off meeting with the sponsoring organizations established sensor unit requirements.

Street Sense Requirements:

  • Budget
    •  USD $225
  • Measurement sensing:
    • ozone level
    • NO2 level
    • PM2.5 particulate level
    • audible noise
  • Measurement recording
    • on-board micro SD Card media
    • continuous recording of audio stream to WAV file
    • air quality measurements logged to files
    • SD Card removable to enable file download
    • recorded data is time-stamped
  • Power
    • battery powered
    • minimum 8 hours operation time
    • rechargeable without battery removal
  • Enclosure
    • weatherproof
    • simple attachment to a street level structure such as a lighting pole
    • allows air flow thru 
  • Connectivity
    • WiFi for pushing sensor data to a cloud database
  • User Interface
    • on-board display to view readings and verify operation
    • simple operation requiring minimal training

Stretch ambitions:

  • on-board audio sample processing


traffic_and_wind.wav

10 second audio clip. Testing I2S microphone with new MicroPython I2S module. Sample rate used is 10k samples/second. Wind noise can be heard.

Waveform Audio File Format (WAV) - 195.04 kB - 11/06/2018 at 01:09

Download

ambulance.wav

10 second audio clip. Testing I2S microphone with new MicroPython I2S module. Sample rate used is 10k samples/second. A passing ambulance can be heard.

Waveform Audio File Format (WAV) - 195.04 kB - 11/06/2018 at 01:09

Download

  • Improving microphone handling with Fast IO

    Mike Teachman12/28/2018 at 17:43 0 comments

    The previous log outlined some concerns with microphone handling delays caused by delays when the loop yields to the uasyncio scheduler (e.g. the measure delay of 6ms).  I explored the use of an alternative uasyncio library called Fast IO. The Fast IO library adds a high-priority I/O queue to uasyncio.  This allows a coroutine to be given higher priority than other coroutines that are waiting to run.

    I changed the microphone coroutine to use this high-priority queue and ran some performance tests.  The change was done in one line - I used a high-priority millisecond timer in place of the the standard uasyncio sleep timer.  Using this new timer results in the microphone loop getting scheduled in a high-priority queue -- the microphone loop will run before any of the other coroutines.

    With this change the yield time in the microphone handling loop was reduced by about 20%.   More importantly, the change insures that the time-critical microphone handling loop always gets priority over other waiting coroutines.  This will greatly reduce the risk of overruns in the received DMA sample buffers.

  • Performance Concerns

    Mike Teachman12/15/2018 at 06:00 0 comments

    My previous log discussed the Asynchronous programming approach being used in this project.

    As a quick reminder, the key concept of Asynchronous programming is co-operative scheduling of coroutines that need processing time.  Each coroutine is "trusted" to only run for a minimal amount of time, then give control back to the scheduler, so that other coroutines can run... very cooperative and nice.

    The ambition for noise analysis is to record a gapless stream of audio samples to an external SD Card that can be later post-processed and characterized.  The high level functions in the audio processing loop are shown below.

    This flowchart depicts a continuous loop that is 100% dedicated to audio sample processing.  But, the sensor unit has other tasks that need to run.  For example, there are tasks that need to read the ozone, NO2, and PM2.5 particulates sensors.  There is a display task to update the OLED display.  And,  eventually there will be a MQTT task pushing sensor readings to the cloud.  If the audio loop ran continuously, no other task could ever be serviced.   

    With asynchronous programming,  a coroutine that runs in a loop must periodically give control back to the scheduler so that other tasks can run - this is called a yield.   Control is yielded to the uasyncio scheduler using a call to await asyncio.sleep(0).  Three yield points in the microphone handler are shown below.

    Problems !

    The 3 yield calls were added to the loop and an audio recording was made using a 20 kHz sampling rate.  The audio playback showed "choppy audio", indicating gaps in the recording.  What is happening?

    First, some background on constraints in the processing of audio samples

    1)  Every loop, 256 audio samples are read from the microphone.  If the sampling rate is 20kHz, the sampling period is 256/20kHz = 12.8 ms.  On average, the microphone loop needs to complete every 12.8ms.  Otherwise, samples will be missed.  

    2) Audio samples are first buffered into a chain of DMA memory blocks.  There are limitations in DMA buffering.  A total of 16 kBytes of DMA memory is used to buffer the incoming audio samples.  That amount of buffering can hold 102.4 ms of samples.  This means that the microphone loop can be blocked from running a total of 102.4 ms in the worst case.  If it is blocked for longer, then the DMA buffer will overrun and samples will be lost.

    I added some print() statements into the loop to better understand the time of each operation.  The unexpected surprise was the await asyncio.sleep(0) call.  This call gives control back to the scheduler, giving other tasks the opportunity to run (if they are ready).  I expected that the call to await asyncio.sleep(0) would return very quickly (e.g. < 1ms) when no other tasks are queued to run.  This is not the case.  It took a typical 6 ms to return control back to the audio processing loop even when no other tasks were queued to run.  What's the big deal? - 6 ms is a blink in time.  But, it represents a rather high percentage of time for the overall audio sample processing loop time (12.8ms), especially if 3 calls to asyncio.sleep(0) are made in each loop.

    Only having one call to asyncio.sleep(0) eliminated the gaps in the recording.  Still, this amount of "wasted" time in the scheduler is concerning.  When the microphone task yields to the scheduler more than one task may run.  If every task switch takes 6ms I have doubts that gapless audio recording is feasible with uasyncio.

    At this point, I'm having thoughts like "I should do this project in C/C++" which I believe would eliminate these inefficiencies.  But, I'm still fairly committed...

    Read more »

  • Recording Sensor Data to SD Card

    Mike Teachman11/27/2018 at 14:30 0 comments

    The PM2.5 particulate sensor readings are now being recorded into a text file on a SD Card, every 3 minutes.  The 3 minute interval is arbitrary at this point in the project.

    The file format is Comma-Separated Values (CSV).  Each line of the file contains two elements:

    1. Timestamp in Universal Time Coordinated (UTC) format.  UTC is an industry-standard way to represent a timestamp.  It is not affected by timezones or daylight savings time.
    2. PM2.5 Atmosphere reading from the Plantower PMS5003 particulate sensor

    Here is an example slice of data from a recording:

    1543831740, 4
    1543831920, 4
    1543832100, 2
    1543832280, 4
    1543832460, 6
    1543832640, 4
    1543832820, 4
    1543833000, 3
    1543833180, 4

    Asynchronous Programming

    The MicroPython programming piece is implemented using an asynchronous programming approach.

    Why asynchronous programming for the Street Sense project?

    Consider ... the Street Sense device has several functions that will run concurrently, for example:

    • reading data from the sensors using I2C and UART hardware interfaces
    • checking for DS3231 real time clock alarms
    • reading audio samples from the I2S microphone
    • checking for button presses
    • writing sensor data and audio samples to the SD Card
    • publishing sensor data to an internet cloud database with MQTT over a WiFi connection

    Implementing these concurrent operations is a natural fit for asynchronous programming.  The MicroPython project provides a uasyncio library for implementing a program with an asynchronous approach. 

    Getting up-to-speed

    My embedded system programming experience has been with 32-bit real-time operating systems (RTOS) with preemptive, priority based task schedulers.  In these operating systems concurrent tasks are "time-sliced" by the RTOS.  It is a very different way to program than co-operative scheduling with asynchronous programming.

    I discovered a helpful learning resource focused on MicroPython - the uasyncio tutorial by Peter Hinch.  This tutorial provides detailed examples on implementing asynchronous programming in MicroPython.   The tutorial also describes the asyn library, which provides various "primitives" to synchronize activity between the Street Sense device features.  Two synchronization primitives are particularly useful in the first implementation:  Events and Barriers

    For example, the asyn library Barrier primitive is used to align the sensor reading activity to the 3-minute interval alarm.  

    As I get better at asynchronous programming I expect to improve and refactor the code with each iteration. The first iteration of the MicroPython code is stored in a Github repository. 

    Street Sense on Github

  • Real-Time Clock and OLED Display

    Mike Teachman11/27/2018 at 14:27 0 comments

      Two new components were added to the breadboard prototype

      1. DS3231 clock module
      2. SSD1306 OLED display

      DS3231 real time clock module

      The Street Sense unit will have an onboard logging feature, where sensor data will be recorded to SD Card, every 15 minutes.  Done professionally, timestamped sample data is recorded on the 15 minute mark, 00, 15, 30, and 45 mins (rather than randomly starting at some minute value).  This requirement necessitates some sort of accurate clock source.  

      The DS3231 clock module is an ideal device for this purpose -- it includes a constantly running on-board clock that is maintained by a single coin battery.  This means the clock keeps running even when the unit is not powered.  This particular clock module was chosen for its extremely low drift specification of +- 2ppm.  Over one year the clock will drift by a maximum of ~1 minute.  

      The clock module also has 2 built-in alarms.  You can set an alarm date/time.  When the clock reaches the alarm time an on-board register flags the alarm.  The MicroPython code can detect this alarm.  The alarm feature will be used to time the 15 minute recording intervals discussed above.

      The communication interface is I2C -- my favorite.  The LoBo version of MicroPython that I'm using has excellent I2C support.

      A google hunt for "DS3231 MicroPython" revealed a few open source libraries.   I chose a library with alarm support and by an author know in the community for excellence in driver design (Radomir Dopieralski). 

      https://github.com/adafruit/Adafruit-uRTC

      https://micropython-urtc.readthedocs.io/en/latest/

      There is one small bug in the alarm implementation.  It was an easy one-line fix.  I submitted a pull request for the fix.

      https://github.com/adafruit/Adafruit-uRTC/pull/9

      The current consumption was measured at 1.2 mA.  The module has a single LED which likely accounts for most of the current.  I'll likely remove the LED as it adds little value.

      One other important note about this module.   It is recommended to remove resistor R5 to disable the recharging circuit.  The coin cell being used is not rechargeable.  R5 is circled in the schematic below.  I removed it by heating the surface mount resistor with a soldering iron.

      SSD1306 OLED display

      A display will be useful when commissioning the unit to see that the sensors are working prior to installation.  As well, there might be a need to select different operating modes during on-site commissioning.  The SSD1306 is a compact, low-cost display with 128x64 pixel resolution.  It also supports an I2C communication interface.

      The LoBo MicroPython port has SSD1306 support included, although the Framebuf module needs to be enabled in the build to avoid an error when importing the module in MicroPython.

      The photo below shows some demo readings on the display.  DS3231 clock module is on the right.

      The current consumption was measured under different usage regimes

      ConditionCurrent at Vcc [mA]
      all pixels off0.3
      all pixels on13.1
      "Hello" displayed0.44
      sleep mode0.0087  (8.7uA)

  • Powering Street Sense

    Mike Teachman11/23/2018 at 18:19 0 comments

    The Street Sense device can operate with two power sources:

    1. Stand-alone battery power
    2. USB power

    The Lolin D32 Pro board provides much of the required circuitry to manage power between the two sources.

    • When 5V USB power is present the lithium polymer battery (LiPo) battery is recharged.  
    • With stand-alone operation the LiPo battery provides power to the unit

    One feature of the Lolin board is external USB and battery pins.  When the board is plugged into a USB source, the 5V USB voltage is presented at the USB pin.  Similarly, under stand-alone battery power, the 3.7V battery voltage is presented at the BAT pin.  These external pins will be used to power the Plantower particulate sensor.

    The Plantower particulate sensor requires a 5V supply to power the internal fan.  To provide this voltage under 3.7V battery power conditions a DC-DC boost converter is used, shown in the schematic below (U2).

    The circuit shown below will block battery current when USB power is present.  The obvious benefit is that battery capacity is maintained when USB power is present.  The less obvious benefit is that it satisfies an important design constraint for the Lolin's internal battery charge management IC (TP4054) -- this IC requires that minimal load current is present on the battery during a recharge cycle.  

    How does this circuit work?   The P-Channel MOSFET (Q1) turns OFF when USB power is present at the MOSFET Gate.  This blocks the battery from supplying power to the boost converter.  The battery management IC sees minimal load.

    Note:  this circuit was taken from the battery management design shown in the Lolin D32 Pro device schematic.  The Lolin circuit has a similar objective of removing the internal 3.3V voltage regulator from the battery charging circuit when the device is plugged into USB.

    Here are some photos of the prototype circuit built up using a breadboard.

    Testing

    I copied a MicroPython program into the ESP32 that reads the particulate sensor and switches it between Active and Standby modes.  The BAT pin current was measured when the unit was under battery and USB power.  This current flows through the MOSFET and into the boost converter.  A uCurrent Gold device was used in the current path to eliminate the effects of the multimeter burden resistance.

    Results are shown in the table below.  The BAT current of 2.0mA under USB power will not interfere with the charge termination criteria of the Lolin battery management IC.

    Plantower Sensor ModePower Source BAT pin current [mA]
    Active

    Battery

    varies 95-140

    Active

    USB

    2.0

    Standby

    Battery

    12.5

    Standby

    USB

    0.0

  • First tests with the PMS5003 particulate sensor

    Mike Teachman11/20/2018 at 14:50 0 comments

    The Plantower PMS5003 device was selected for PM2.5 particulate sensing.  The Plantower particulate sensors have received favourable reviews in independent testing.

    Plantower PMS5003 particulate sensor

    The sensor is powered with 5V and has a 3.3V UART interface.  A custom cable was made to breakout the sensor ribbon cable to a breadboard-friendly 0.1" spacing.  Testing was done using a breadboard.  Prototype photo is shown at the bottom of this log.

    Google hunt for a MicroPython library

    The next step is finding some interface code.  I found a few MicroPython libraries for the PMS5003 device in Github.  After evaluation, I decided that the PMS5003 implementation by Kevin Köck is the most promising for this project. 

    Here are the compelling reasons to use this library.

    • uasyncio implementation.   I was anticipating needing to rewrite a library to work with uasyncio.  This library saves me that effort.
    • all sensor functions are implemented
    • example code
    • good documentation

    The LoBo ESP32 version of MicroPython that I am currently using does not include uasyncio as a built-in library.   This uasyncio library and the PMS5003 library need to be copied into the MicroPython filesystem on the ESP32.   The uasyncio library is found in micropython-lib.  I used the Ampy command line tool to copy these MicroPython libraries into the filesystem.  Here is the filesystem contents after copying.

    boot.py
    pms5003.py
    lib  
      | 
       uasyncio         
            __init__.py         
            core.py
    


    The PMS5003 library worked the first time using some example code.  Here is the test output showing particulate readings, sent using the ESP32 serial port.

    Measurements from PMS5003 sensor

    Current measurements

    The device has two modes of operation:  Active and Standby.  Measurement with a multimeter showed these results.

    • Active:  current varies between 50mA and 80mA
    • Standby:  7mA  

    Observation:  the measured Standby current of 7mA is considerably more than the <200uA value listed in the manufacturer's datasheet.  The measured Active current of 50mA-80mA agrees with the <100mA listed in the datasheet.

    current during active state

  • I2S Microphone

    Mike Teachman11/05/2018 at 23:45 0 comments

    Measurement  of noise pollution is one of the ambitions for this project.   Initially, the Street Sense unit will be designed to continuously record a stream of audio samples to a WAV file on a SD Card.  The WAV file will be post-analyzed to identify various audio metrics.

    The microphone selected is the Adafruit I2S MEMS microphone based on the SPH0645LM4H-B device.
    I2S MEMS microphone

    This microphone is compact, low power,  and fits the budget of this project.   The audio sampling is controlled by an I2S digital interface.  The ESP32 micro controller has an I2S interface, which will be configured in Master mode to read audio samples from the microphone.

    One challenge is MicroPython - there is no version of MicroPython that supports the I2S capabilities of the ESP32.  The Arduino core for the ESP32 does offer I2S.  However, I am fairly determined to attempt the programming in MicroPython.  This means diving into the bowels of MicroPython and adding I2S Master support.

    Using existing MicroPython module implementations as a guide, I wrote the low-level C code to add a new I2S class into the machine module of MicroPython.   Here is a simplified view of the MicroPython code used to read audio samples.

    from machine import I2S
    
    SAMPLE_BLOCK_SIZE = 2048
    samples = bytearray(SAMPLE_BLOCK_SIZE)
    audio=I2S(... initialization arguments ...)
    audio.read(samples, SAMPLE_BLOCK_SIZE)
    audio.deinit()

    Blocks of samples are read from the microphone, then written to the SD Card.  The DMA controller of the ESP32 is configured so that the audio stream is "gapless".  It should be possible to continuously write a WAV file into the SDCard at a 44.1kHz sample rate.  Initial results look promising, although some block writes to the SD Card are longer than others.  Under some conditions sample gaps may appear - more testing needs to be done to evaluate this design risk.

    The breadboard prototype is shown below

    MicroPython test code was written to capture 10 second audio clips and stream the samples to a WAV file on the SD Card.  A LiPo battery was used to power the unit and traffic sounds were captured at a local street.  Two WAV files containing traffic sounds are attached.  Wind noise is mixed with traffic sounds in the first WAV file.  Some sort of microphone covering will be needed to mitigate wind caused noise.  In the 2nd clip you can hear the sound of a passing ambulance.

    Audio clip: Traffic and Wind

    Audio clip: Ambulance

  • Programming in MicroPython

    Mike Teachman11/03/2018 at 00:53 0 comments

    For the last two years, the majority of my micro controller projects have used the ESP8266 device, programmed with the MicroPython  language, rather than C/C++.  I'm hoping to continue this approach with the ESP32 device.

    There are definitely some risks.  The ESP32 device is not well supported in the official MicroPython project.  The ESP32 has many hardware peripherals that are offered to developers with a well documented API called the ESP-IDF.  However, most of these peripherals are not offered in the official MicroPython port for the ESP32.  

    https://micropython.org/

    Fortunately, there are some derived works of MicroPython that have extended the official version, offering support for many of the valuable hardware peripherals.  A notable work is the "LoBo" version of MicroPython.  This version was developed by Boris Lovosevic from Croatia.  The LoBo port will likely be the version used in this project. 

    https://github.com/loboris/MicroPython_ESP32_psRAM_LoBo

    Even the LoBo version is not complete.  Additional low-level C coding may be needed to expose additional hardware features in MicroPython.   There are example tutorials describing how to bring unsupported hardware features into the MicroPython world. 

  • Micro controller Selection

    Mike Teachman11/02/2018 at 04:58 0 comments

    I'm a fan of the Espressif line of micros.  I was thinking that the ESP8266 would be a candidate, but the Plantower particulate sensor needs to be read with a UART.  The ESP8266 has only one UART which is pretty much dedicated to a serial-USB device.  Also, I'm planning to use an I2S MEMS microphone for audio recording - the ESP32 has a capable I2S peripheral to read samples from the microphone.  It also has multiple analog inputs which are needed to read the ozone and NO2 sensors - I'm cautiously optimistic that these  analog inputs are up to the task (whereas the single ESP8266 analog input is mostly unusable except for making crude measurements).

    The WiFi capability will be useful to push sensor data to a MQTT server and cloud database. 

    The ESP32 module which best suits this project is the Lolin D32 Pro.  This module has a built-in battery charger which I plan to use for recharging the LiPo battery. It also has a built-in micro SD Card slot - this will play into the logging of sensor data.  Lastly, it has the added advantage of 4MB of PSRAM, which will be useful for buffering audio samples.

    https://wiki.wemos.cc/products:d32:d32_pro

  • Air quality sensor selection

    Mike Teachman10/31/2018 at 04:45 0 comments

    Measuring street level pollutants that affect human health is the focus.  The sensor selection is guided by looking at the risks imposed by various pollutants.   

    Health risks, at the street level, prioritized

    • Particulate matter (PM)
    • Ozone - VOCs+NOx+Sun => Ozone
    • Nitrogen dioxide (NO2) - stress to lungs
    • Volatile Organic Compounds (VOCs) - promotes ground level ozone, “smog”
    • Nitrogen oxides (NOx) - promotes ground level ozone, “smog”
    • Carbon monoxide (CO) - toxic to humans.  Not so concerning for street level
    • Sulfur dioxide (SO2) - more an industrial pollutant

    Gasses such as CO2 are not an immediate risk to human health.

    Sources:

    https://www.ucsusa.org/clean-vehicles/vehicles-air-pollution-and-human-health/cars-trucks-air-pollution#.W9ku5ntKimx

    Based on the health risks, 3 pollutants will be measured

    • Particulate matter (PM)
    • Ozone
    • Nitrogen dioxide (NO2) 

    The criteria for sensor selection are:

    1. Demonstrated effectiveness
    2. Form factor
    3. integration with microcontroller
    4. Cost

    Particulate matter

    A Plantower 5003 unit is the pick to measure particulates.  The Plantower lineup of particulate sensors are used in many sensor units.  A tear-down of the unit shows that the design is effective and robust for particulate sensing.  The cost of the unit is around USD $15.  Measurement readings are read from the device with a UART interface.

    https://aqicn.org/sensor/pms5003-7003/
    https://ourairquality.org/

    Ozone, Nitrogen dioxide

    The Array of Things project in Chicago is the inspiration for selecting the remaining two gas sensors. 

    https://arrayofthings.github.io/

    The Array of Things node is built using gas sensors manufactured by SPEC Sensors.  The Ozone and NO2 sensors cost around USD $60 each.  Measurement readings are output as a 0-3V analog signal.

    https://www.spec-sensors.com/array-of-things/

View all 10 project logs

Enjoy this project?

Share

Discussions

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates