Street Sense is a project to build a portable, battery-powered sensor unit to measure:
Air Quality: Ozone, NO2, Particulates
There are two NGO sponsors. They require a sensor that can help them answer some important questions:
1. What air and noise pollution effects are experienced by people at the street level?
2. Can we quantify the changes in street level pollution resulting from a capital infrastructure project?
The project name "Street Sense" seems appropriate - sensing the environment at the street level in the urban landscape
Street Sense Features:
ozone, in ppb
NO2, in ppb
audio recording for post analysis
temperature and humidity
on-board micro SD Card media
continuous recording of audio stream to WAV file
air quality measurements logged to files
SD Card removable to enable file download
recorded data is time-stamped
battery or USB powered
minimum 8 hours operation time on battery
USB rechargeable without battery removal
simple attachment to a street level structure such as a lighting pole
allows air flow thru
WiFi for pushing sensor data to a cloud database
on-board display to view readings and verify operation
simple operation requiring minimal training
Open Hardware - schematics, BOMs, etc
Open Software - all firmware and software hosted on public github account
on-board audio sample processing to calculate real time dB
Version 4 of Street Sense air and noise pollution unit
3 changes to reduce noise created by boost converter
- added high side MOSFET switch
- added Schottky diode to USB/Battery select cct
- added capacitor to input of boost converter
Adobe Portable Document Format -
64.26 kB -
03/14/2019 at 23:08
The V05 schematic includes the new ADC. You will notice that the initial integration of this ADC is lacking most sensible approaches to minimize noise effects:
3.3V digital rail supplies analog power for both the ADC and Spec Sensor gas sensors.
ADC soldered to a TSSOP-16 breakout board and then plugged into a breadboard
This approach allowed me to write the MicroPython driver and get the device integrated into the rest of the MicroPython code.
Given the lack of care to deal with power supply noise, it wasn't surprising to see poor results when reading the analog values from the ozone and NO2 sensors. Results showed considerable variability in back-to-back measurements for gas concentration. An oscilloscope capture shows that the Vgas outputs from the gas sensors oscillate during the time they are being sampled by the ADC.
yellow = Vgas Ozone
aqua = Vgas NO2
purple = 3.3V rail
You can see voltage fluctuations on the 3.3V rail that are associated with the oscillations. These oscillations happen each time the ADC performs a single-shot conversion.
Next steps: Time to put on the analog design hat and investigate decoupling methods to provide low-noise analog power for the ADC and gas sensors.
The PM2.5 particulate sensor requires a 5V supply voltage. Supplying this 5V requires special consideration when the unit is powered by a 3.7V Lipo battery. Providing 5V of power with a 3.7V battery is accomplished using a type of switched-mode power supply called a boost converter. The boost converter used in this design is built around the ME2108 IC.
Switched-mode power supplies are notorious for injecting noise into circuits. I investigated this concern.
The purple trace in the scope capture below shows noise on the 3.3V rail -- measured as 132mV with a frequency of 34.7 kHz. As a quick test, I removed the boost converter -- the 3.3V rail noise dropped dramatically - this shows that the boost converter is a significant source of noise.
Many devices, including a noise-sensitive ADC is powered with the 3.3V rail. My first results with the ADC are not encouraging - I see unstable results from the ozone and NO2 sensors. I suspect that the switched-mode supply noise is contributing to the undesirable results.
The first mitigation step was to replace the 1N4001 diode in the USB/Battery selection circuit with a Schottky diode. A Schottky diode is recommended for this circuit. I found two types of Schottky diodes at our local Makerspace and chose the diode that reduced the noise the most.
Adding the Schottky diode produced measurable improvements - noise was approximately halved. Shown in the scope capture below.
Adding a 220uF electrolytic capacitor to the input of the boost converter produced more measurable noise improvements on the 3.3V rail. Apparently electrolytic capacitors are not the first choice capacitors for the input of boost converters (low ESR ceramic capacitors are preferred). However, the electrolytic capacitor reduced the noise on the 3.3V rail by an additional 50% , shown below.
Lastly, I added a high-side MOSFET switch to turn off the boost converter using a GPIO pin on the ESP32 microcontroller. The particulate sensor only needs to be powered-up on demand to make a periodic measurement. When the boost converter is switched off the noise on the 3.3V rail is reduced further.
These improvements are reflected in the latest V4 schematic release, link below.
With the I2S microphone and particulate sensors working well, the next step is integrating the ozone and NO2 sensors. These sensors are manufactured by Spec Sensor and are used in the Array of Things project in Chicago. Spec Sensor published a report showing that the Spec Sensor units perform well in side-by-side tests against calibrated industrial-grade sensors. The credibility of these sensors appears promising.
I chose the analog module versions for both the ozone and NO2 sensors. The modules include all the difficult analog amplification and biasing circuitry. I have little of that skillset - it was an easy decision to purchase units that include the analog sub-circuits.
Each sensor has an analog output, 0-3.0V range, that is proportional to the measurement of gas concentration. This analog output will be converted to digital using an analog-to-digital converter (ADC) device that will connect to the ESP32 using an I2C bus. In my parts stock I have the ADS1015 ADC by Texas Instruments. It has 12-bit resolution and can operate with a 3.3V supply. I like using ADCs having an I2C communication interface as it allows the ADC to be located in immediate proximity to the sensors This allows short analog signals runs, thereby reducing coupled noise. The ESP32 has some built-in ADCs, but reports are not flattering on the performance. I might use these ESP32 ADCs for non-critical operations like reading battery voltage.
Each sensor module has 3 outputs that can be measured. Vgas, Vref, and Vtemp. Vgas is the important one - the analog reading which represents the gas concentration. It wasn't too clear on how the other two outputs are used. I contacted the company and got a prompt response. Spec Sensor indicated that good results can be achieved using only the Vgas output. The other two outputs are high impedance outputs which are somewhat difficult to use with ADCs.
I expanded the breadboard prototype to include the two gas sensors and the ADC. The gas sensors and ADC are shown in the left side of the photo below. The V03 schematic is up-to-date with these new devices.
The ADS1015 device is quite popular and there is a MicroPython driver available. Unfortunately, the driver needed some small modifications as the I2C implementation on the Loboris MicroPython port introduces breaking changes compared to the mainline of MicroPython. It's frustrating when a fork of a project does not maintain backwards compatibility with key interfaces like I2C.
Two new co-routines were added to the Street Sense MicroPython code to manage the two sensors. The raw sensor values are displayed on the OLED display and are logged to the SD Card.
What about results? I observed that the gas values are not as stable as I expected. I modified the driver configuration to select a slower sampling rate and the values become more stable. But, still not what I need for the final unit.
From my long 25 year career in measurement at Schneider Electric Victoria I knew that this stage of the project would be toughest to crack. This is just the first step in what will be an iterative design. I fully expect to try out a few ADCs, add filter capacitors, and who knows what else -- to get a stable and accurate digital representation that fully exploits the accuracy of these gas sensors. Perhaps I'll need to seek some help from my ex-colleagues who are unbelievable world-class experts in analog design?
One area that needs work is ADC resolution. The ADS1015 devices have 12 bits of resolution. That is good enough...
The previous log outlined some concerns with microphone handling delays caused by delays when the loop yields to the uasyncio scheduler (e.g. the measure delay of 6ms). I explored the use of an alternative uasyncio library called Fast IO. The Fast IO library adds a high-priority I/O queue to uasyncio. This allows a coroutine to be given higher priority than other coroutines that are waiting to run.
I changed the microphone coroutine to use this high-priority queue and ran some performance tests. The change was done in one line - I used a high-priority millisecond timer in place of the the standard uasyncio sleep timer. Using this new timer results in the microphone loop getting scheduled in a high-priority queue -- the microphone loop will run before any of the other coroutines.
With this change the yield time in the microphone handling loop was reduced by about 20%. More importantly, the change insures that the time-critical microphone handling loop always gets priority over other waiting coroutines. This will greatly reduce the risk of overruns in the received DMA sample buffers.
My previous log discussed the Asynchronous programming approach being used in this project.
As a quick reminder, the key concept of Asynchronous programming is co-operative scheduling of coroutines that need processing time. Each coroutine is "trusted" to only run for a minimal amount of time, then give control back to the scheduler, so that other coroutines can run... very cooperative and nice.
The ambition for noise analysis is to record a gapless stream of audio samples to an external SD Card that can be later post-processed and characterized. The high level functions in the audio processing loop are shown below.
This flowchart depicts a continuous loop that is 100% dedicated to audio sample processing. But, the sensor unit has other tasks that need to run. For example, there are tasks that need to read the ozone, NO2, and PM2.5 particulates sensors. There is a display task to update the OLED display. And, eventually there will be a MQTT task pushing sensor readings to the cloud. If the audio loop ran continuously, no other task could ever be serviced.
With asynchronous programming, a coroutine that runs in a loop must periodically give control back to the scheduler so that other tasks can run - this is called a yield. Control is yielded to the uasyncio scheduler using a call to await asyncio.sleep(0). Three yield points in the microphone handler are shown below.
The 3 yield calls were added to the loop and an audio recording was made using a 20 kHz sampling rate. The audio playback showed "choppy audio", indicating gaps in the recording. What is happening?
First, some background on constraints in the processing of audio samples
1) Every loop, 256 audio samples are read from the microphone. If the sampling rate is 20kHz, the sampling period is 256/20kHz = 12.8 ms. On average, the microphone loop needs to complete every 12.8ms. Otherwise, samples will be missed.
2) Audio samples are first buffered into a chain of DMA memory blocks. There are limitations in DMA buffering. A total of 16 kBytes of DMA memory is used to buffer the incoming audio samples. That amount of buffering can hold 102.4 ms of samples. This means that the microphone loop can be blocked from running a total of 102.4 ms in the worst case. If it is blocked for longer, then the DMA buffer will overrun and samples will be lost.
I added some print() statements into the loop to better understand the time of each operation. The unexpected surprise was the await asyncio.sleep(0) call. This call gives control back to the scheduler, giving other tasks the opportunity to run (if they are ready). I expected that the call to await asyncio.sleep(0) would return very quickly (e.g. < 1ms) when no other tasks are queued to run. This is not the case. It took a typical 6 ms to return control back to the audio processing loop even when no other tasks were queued to run. What's the big deal? - 6 ms is a blink in time. But, it represents a rather high percentage of time for the overall audio sample processing loop time (12.8ms), especially if 3 calls to asyncio.sleep(0) are made in each loop.
Only having one call to asyncio.sleep(0) eliminated the gaps in the recording. Still, this amount of "wasted" time in the scheduler is concerning. When the microphone task yields to the scheduler more than one task may run. If every task switch takes 6ms I have doubts that gapless audio recording is feasible with uasyncio.
At this point, I'm having thoughts like "I should do this project in C/C++" which I believe would eliminate these inefficiencies. But, I'm still fairly committed...
Why asynchronous programming for the Street Sense project?
Consider ... the Street Sense device has several functions that will run concurrently, for example:
reading data from the sensors using I2C and UART hardware interfaces
checking for DS3231 real time clock alarms
reading audio samples from the I2S microphone
checking for button presses
writing sensor data and audio samples to the SD Card
publishing sensor data to an internet cloud database with MQTT over a WiFi connection
Implementing these concurrent operations is a natural fit for asynchronous programming. The MicroPython project provides a uasyncio library for implementing a program with an asynchronous approach.
My embedded system programming experience has been with 32-bit real-time operating systems (RTOS) with preemptive, priority based task schedulers. In these operating systems concurrent tasks are "time-sliced" by the RTOS. It is a very different way to program than co-operative scheduling with asynchronous programming.
I discovered a helpful learning resource focused on MicroPython - the uasyncio tutorial by Peter Hinch. This tutorial provides detailed examples on implementing asynchronous programming in MicroPython. The tutorial also describes the asyn library, which provides various "primitives" to synchronize activity between the Street Sense device features. Two synchronization primitives are particularly useful in the first implementation: Events and Barriers.
For example, the asyn library Barrier primitive is used to align the sensor reading activity to the 3-minute interval alarm.
As I get better at asynchronous programming I expect to improve and refactor the code with each iteration. The first iteration of the MicroPython code is stored in a Github repository.
Two new components were added to the breadboard prototype
DS3231 clock module
SSD1306 OLED display
DS3231 real time clock module
The Street Sense unit will have an onboard logging feature, where sensor data will be recorded to SD Card, every 15 minutes. Done professionally, timestamped sample data is recorded on the 15 minute mark, 00, 15, 30, and 45 mins (rather than randomly starting at some minute value). This requirement necessitates some sort of accurate clock source.
The DS3231 clock module is an ideal device for this purpose -- it includes a constantly running on-board clock that is maintained by a single coin battery. This means the clock keeps running even when the unit is not powered. This particular clock module was chosen for its extremely low drift specification of +- 2ppm. Over one year the clock will drift by a maximum of ~1 minute.
The clock module also has 2 built-in alarms. You can set an alarm date/time. When the clock reaches the alarm time an on-board register flags the alarm. The MicroPython code can detect this alarm. The alarm feature will be used to time the 15 minute recording intervals discussed above.
The communication interface is I2C -- my favorite. The LoBo version of MicroPython that I'm using has excellent I2C support.
A google hunt for "DS3231 MicroPython" revealed a few open source libraries. I chose a library with alarm support and by an author know in the community for excellence in driver design (Radomir Dopieralski).
The current consumption was measured at 1.2 mA. The module has a single LED which likely accounts for most of the current. I'll likely remove the LED as it adds little value.
One other important note about this module. It is recommended to remove resistor R5 to disable the recharging circuit. The coin cell being used is not rechargeable. R5 is circled in the schematic below. I removed it by heating the surface mount resistor with a soldering iron.
SSD1306 OLED display
A display will be useful when commissioning the unit to see that the sensors are working prior to installation. As well, there might be a need to select different operating modes during on-site commissioning. The SSD1306 is a compact, low-cost display with 128x64 pixel resolution. It also supports an I2C communication interface.
The LoBo MicroPython port has SSD1306 support included, although the Framebuf module needs to be enabled in the build to avoid an error when importing the module in MicroPython.
The photo below shows some demo readings on the display. DS3231 clock module is on the right.
The current consumption was measured under different usage regimes
The Street Sense device can operate with two power sources:
Stand-alone battery power
The Lolin D32 Pro board provides much of the required circuitry to manage power between the two sources.
When 5V USB power is present the lithium polymer battery (LiPo) battery is recharged.
With stand-alone operation the LiPo battery provides power to the unit
One feature of the Lolin board is external USB and battery pins. When the board is plugged into a USB source, the 5V USB voltage is presented at the USB pin. Similarly, under stand-alone battery power, the 3.7V battery voltage is presented at the BAT pin. These external pins will be used to power the Plantower particulate sensor.
The Plantower particulate sensor requires a 5V supply to power the internal fan. To provide this voltage under 3.7V battery power conditions a DC-DC boost converter is used, shown in the schematic below (U2).
The circuit shown below will block battery current when USB power is present. The obvious benefit is that battery capacity is maintained when USB power is present. The less obvious benefit is that it satisfies an important design constraint for the Lolin's internal battery charge management IC (TP4054) -- this IC requires that minimal load current is present on the battery during a recharge cycle.
How does this circuit work? The P-Channel MOSFET (Q1) turns OFF when USB power is present at the MOSFET Gate. This blocks the battery from supplying power to the boost converter. The battery management IC sees minimal load.
Note: this circuit was taken from the battery management design shown in the Lolin D32 Pro device schematic. The Lolin circuit has a similar objective of removing the internal 3.3V voltage regulator from the battery charging circuit when the device is plugged into USB.
Here are some photos of the prototype circuit built up using a breadboard.
I copied a MicroPython program into the ESP32 that reads the particulate sensor and switches it between Active and Standby modes. The BAT pin current was measured when the unit was under battery and USB power. This current flows through the MOSFET and into the boost converter. A uCurrent Gold device was used in the current path to eliminate the effects of the multimeter burden resistance.
Results are shown in the table below. The BAT current of 2.0mA under USB power will not interfere with the charge termination criteria of the Lolin battery management IC.
The sensor is powered with 5V and has a 3.3V UART interface. A custom cable was made to breakout the sensor ribbon cable to a breadboard-friendly 0.1" spacing. Testing was done using a breadboard. Prototype photo is shown at the bottom of this log.
Google hunt for a MicroPython library
The next step is finding some interface code. I found a few MicroPython libraries for the PMS5003 device in Github. After evaluation, I decided that the PMS5003 implementation by Kevin Köck is the most promising for this project.
Here are the compelling reasons to use this library.
uasyncio implementation. I was anticipating needing to rewrite a library to work with uasyncio. This library saves me that effort.
all sensor functions are implemented
The LoBo ESP32 version of MicroPython that I am currently using does not include uasyncio as a built-in library. This uasyncio library and the PMS5003 library need to be copied into the MicroPython filesystem on the ESP32. The uasyncio library is found in micropython-lib. I used the Ampy command line tool to copy these MicroPython libraries into the filesystem. Here is the filesystem contents after copying.
The PMS5003 library worked the first time using some example code. Here is the test output showing particulate readings, sent using the ESP32 serial port.
The device has two modes of operation: Active and Standby. Measurement with a multimeter showed these results.
Active: current varies between 50mA and 80mA
Observation: the measured Standby current of 7mA is considerably more than the <200uA value listed in the manufacturer's datasheet. The measured Active current of 50mA-80mA agrees with the <100mA listed in the datasheet.
Measurement of noise pollution is one of the ambitions for this project. Initially, the Street Sense unit will be designed to continuously record a stream of audio samples to a WAV file on a SD Card. The WAV file will be post-analyzed to identify various audio metrics.
The microphone selected is the Adafruit I2S MEMS microphone based on the SPH0645LM4H-B device. I2S MEMS microphone
This microphone is compact, low power, and fits the budget of this project. The audio sampling is controlled by an I2S digital interface. The ESP32 micro controller has an I2S interface, which will be configured in Master mode to read audio samples from the microphone.
One challenge is MicroPython - there is no version of MicroPython that supports the I2S capabilities of the ESP32. The Arduino core for the ESP32 does offer I2S. However, I am fairly determined to attempt the programming in MicroPython. This means diving into the bowels of MicroPython and adding I2S Master support.
Using existing MicroPython module implementations as a guide, I wrote the low-level C code to add a new I2S class into the machine module of MicroPython. Here is a simplified view of the MicroPython code used to read audio samples.
Blocks of samples are read from the microphone, then written to the SD Card. The DMA controller of the ESP32 is configured so that the audio stream is "gapless". It should be possible to continuously write a WAV file into the SDCard at a 44.1kHz sample rate. Initial results look promising, although some block writes to the SD Card are longer than others. Under some conditions sample gaps may appear - more testing needs to be done to evaluate this design risk.
The breadboard prototype is shown below
MicroPython test code was written to capture 10 second audio clips and stream the samples to a WAV file on the SD Card. A LiPo battery was used to power the unit and traffic sounds were captured at a local street. Two WAV files containing traffic sounds are attached. Wind noise is mixed with traffic sounds in the first WAV file. Some sort of microphone covering will be needed to mitigate wind caused noise. In the 2nd clip you can hear the sound of a passing ambulance.