07/16/2017

I finally got around to playing with the ICS43434 I2S digital microphone on the Sensor Tile. I used the I2S library that is part of the STM32L4 Arduino core and Sandeep Mistry's Arduino Sound library for the FFT analysis. Both are very easy to use. But for some reason the I2S mic did not play well with the SPIFlash.h class I created. Both work well without the other and the SPIFlash class works well all of the time but there is some conflict I haven't yet discovered so in order to make some progress, I just regressed to having the SPIFlash functions in the main code. Now all is well.

I am reading the microphone data at 8000 samples per second, rather low for CD-quality music but perfectly fine for voices and machine chatter, etc. I capture 128 sound samples, perform an FFT and then normalize the spectrum and output only those frequencies that are 50% or higher of the maximum intensity of 64 frequencies in the spectrum. So if there is nothing but low ambient sound, nothing is output onto the serial monitor. If I speak, then a handful of frequencies appear. I am only plotting the few major modes once a second just to get a feel for what the data might look like.

Here is an example of the output on the serial monitor:

This from me holding some more or less constant tone voice sound. Apparently the sound has a dominant frequency of ~250 Hz. It is interesting to see the normal modes, and it would be easy to use this capability to detect clapping, for example, or detect when someone was speaking. But I can see already that I will want to capture complete time series of the data for speech recognition. How cool would it be if the Sensor Tile could recognize and respond to the key words "Open the pod bay door HAL" or "Soylent Green is people"!

I managed to send these dominant normal modes to the iPhone via BLE so when I speak I can see the response on the UART console. I had been converting the floats to strings using dtosrtf and assembling into a packet for transmission via serial and the BLE UART bridge using sprintf but I finally realized all I need to do is a Serial2.print to send the data to the iPhone serial console! Not sure why I started with the string packaging. Not necessary in Arduino.

If I hum a constant note (as well as I can anyway) I see one or two dominant frequencies at 250 Hz or 300 Hz or whatever note I can hold (see above). I checked with a metronome tone generator set for 440 Hz and I see 437 Hz with a small side band at 500 hz. I think the way I have the FFT set up the frequency resolution is only 62.5 Hz (8000 Hz sample rate divided by 128 FFT size) so I am not discriminating very finely yet. For normal voice recognition I should probably set the sample rate to 2000 or 4000 Hz or so. Or I can increase the FFT size to 256 or 512 to get better frequency resolution, but at the cost in compute power. The STM32L432 has a Cortex M4F processor and the floating point engine is not really taxed at these kinds of rates. Not sure where the limit is though.

The bigger problem is the flash memory required to hold the FFT code. The full sketch is 214 kB of which ~175 kB is the FFT package alone; this is getting close to the ~245 kB flash memory limit on the STM32L432. I think there are a lot of lookup tables and what not driving this load and I would like to find some lighter implementation of the FFT analyzer, but as far as I can tell everyone uses this CMSIS FFT package.

The point of all this is that all of the sensors including the I2S microphone now work and work together with a lot of the relevant data being sent to the smartphone via BLE.

The smartphone is convenient as a check but I want to start using the nRF52 in central role as a point-to-point gateway so I can get the data onto the Arduino IDE serial monitor running on the laptop. I am having the nRF52 add-on I designed for the Butterfly produced in China and I expect to start using this to monitor several of these Sensor Tiles at once for real-time data analysis. This is more fully described here.

The hard part now is to start applying the Sensor Tile(s) to solve some real world problems...