Project Logs

Collapse logs

• ESP32 with SGTL5000

  Christoph Tack • 01/03/2021 at 14:36 • 0 comments

  Hardware

  1. NodeMCU-32S
  2. Adafruit 1780 : Adafruit Accessories Audio Adapter Board for Teensy

  Generating I2S

  The annoying thing about the Adafruit audio adapter is that it's not fully open source.  These are the supply voltages:

  • VDDD = 1.8V
  • VDDIO = 3.3V (powers line out)
  • VDDA = 3.3V (powers the headphone)

  On the SGTL5000 datasheet, this is the one bit delay on I2S format with respect to the left-justified format on Figure 10. I2S Port Supported Formats.  This seems to be normal I2S behavior. 
  

  The delay could be removed by setting the i2s_comm_format_t in the ESP32 to 0, but I'll just leave it to the standard setting.
  

  The SGTL5000 considers the 16bit data as signed format.  It's analog output is inverted, which actually doesn't matter much for audio.  Voutmax corresponds to 0x8000 = -32768, while Voutmin corresponds to 0x7FFF = 32767).

  The sample code to generate a 200Hz sine wave on the left channel of line-out and headphone can be found here.
  

  References

  1. SGTL5000 driver on Github (by PJRC)
  2. Interfacing an audio codec with ESP32 – Part 1 and Interfacing an Audio Codec with ESP32 – Part 2
  3. Audio Adaptor Boards for Teensy 3.x and Teensy 4.x
  4. ESP32 I2S Internet Radio (with software MP3 decoding inside ESP32)
  5. esp32_audio
  6. ESP32-2-Way-Audio-Relay

• References

  Christoph Tack • 09/16/2020 at 19:43 • 0 comments

  Commercial products

  Kiwi-tec LAP-E01

  • Works in 920.6 ~ 928MHz band, +13dBm, Japanese TELEC certification.
  • Speex codec (doesn't perform as well as Codec2 on low bit rates):
    • Speex sample : 4kbps
    • Codec2 sample : 2.4kbps
  • 125kHz bandwidth LoRa
  • Internal antenna : -0.4dBi

  Prior art

  nRF24 based

  1. Long Range Arduino Based Walkie Talkie using nRF24L01 : many similar projects, all using the RF24Audio library.

  RFM12 based

  1. Walkie Talkie Duino using RFM12B "open source" (only to the Kickstarter backers).

  RFM22 based

  Codec2WalkieTalkie

  Analog FM based

  1. DRA818V analog FM module (lots of harmonics, will need a license to operate)
  2. Auctus A1846S : HamShield (by Casey Halverson), also available in Mini version.
     1. HamShield on Tindie
     2. Kickstarter
     3. Hackaday.io
     4. Instructables
     5. Github
     6. InductiveTwig

  Comparable projects

  LoRa

  HamShield LoRa (by Casey Halverson)

  STM32

  • https://www.digikey.com/en/maker/search-results?&k=getting%20started%20with%20stm32
  • NucleoTNC

• Hardware choices

  Christoph Tack • 09/15/2020 at 18:09 • 0 comments

  Platform

  Flash size requirements

  The codec2 library needs about 87KB, while the RadioLib needs about 10KB.  Then, there's also the base Arduino libraries.  And we still need to add our own code.  To be on the safe side, a device with at least 256KB of flash will be needed.

  ESP32

  Test application built, based on Arduino codec2 library, but it crashed.  This has been solved with esp32-codec2.
  

  Some presumably also got it to work before I did, but they are unwilling to share their source code:

  
  

  STM32

  STM32F4Discovery

  • 168MHz
  • lora-codec2
  • STM32F407G-DISC1 (< €19)
  • analog IO already on board
  • supported by PlatformIO
  • using exotic audio DAC
  • rather large (66x97mm), might make integration more difficult.

  Rowetel/Dragino Tech SM1000

  • Github
  • Implements codec2 on a STM32F405 (custom board design)
  • audio : analog IO directly from STM32

  NUCLEO-L432KC

  Runs only on 80MHz, might be an option to shrink size.
  

  More info on STM32 development

  nRF52

  64MHz

  github : Implements codec2 on a Adafruit Feather nRF52 Bluefruit LE.
  

  Codec2 has been modified so that it can be built using Arduino framework.  I doubt this implementation is working correctly.
  

  • Adafruit ItsyBitsy nRF52840 Express - Bluetooth LE (Mouser & Digikey):
    • small & cheap (€15), breadboard & IC-socket compatible
    • support for Circuitpython & PlatformIO
    • debug connection, but no integrated debugger
    • as always from Adafruit : well documented
    • Application to demonstrate PWM audio functionality on nRF52

  Audio IO

  I²S

  On ESP32, using I²S is definitely advantageous because it can use DMA, which off-loads the reading and writing audio data from the processor.
  

  As we're only processing low quality 8kHz speech here, a high-end audio codec like the SGTL5000 is not necessary, but it might be a good choice after all:

  1. open source support (pjrc)
  2. I²S sink & source in a single device.
  3. High quality audio might be useful for other projects and designs.
  4. Extra features:
     1. Input: Programmable MIC gain, Auto input volume control
     2. Output: 98dB SNR output, digital volume
  5. Development board price is acceptable.

  PWM-DAC & ADC

  The SM1000 and NucleoTNC contain the analog circuitry we need.

  Adafruit Voice Changer also features some form of audio pass-through

  FSK module

  Options:
  

  • NiceRF RF4463F30
    • Bands: 142–175, 283–350, 420–525, and 850–1050 MHz
    • 64byte FIFO
    • data whitening
    • Low power 20dBm option (ebyte module)
  • HopeRF RFM23BP
    • Bands: 413-453, 848-888, 895-935 MHz
    • 64byte FIFO
    • data whitening
    • Some drawings in the datasheet are copied from the SI4463.  Is HopeRF using SI4463 ICs?

  SI4432 is obsolete.  The OnSemi AX5243-D is not available on a 500mW - 1W module.

  AX5043 looks interesting, but little tools are available.  There's a home brew radio however.
  

  LoRa module

  Low power, low cost modules

  • Ai-Thinker Ra-02 (LCSC C90039)
  • NiceRF LORA1278-C1
  • RFM96/RFM98 Seeed 109990165
  • RFM96W-433S2 : no longer available as module (except AliExpress etc.)
  • RFM98W-433S2
    • Difference with RFM96W-433S2 unclear

  High power modules

  NiceRF

  • NiceRF LORA1268F30-433
    • 33dBm
    • works on 3V3 (max. 28dBm), needs 6VDC for 33dBm
    • min. TX-pwr = 10dBm on 3V3.
    • no coax-connector
    • 38x20mm
    • 5mA RX
    • 2µA sleep
    • LoRa RX-sensitivity (BW=62.5 KHz, SF = 12 CR=4/5) = -139dBm
  • NiceRF LORA1278F30-433
    • 30dBm
    • 13mA RX
    • 10µA sleep
    • LoRa RX-sensitivity (BW=125 KHz, SF = 12 CR=4/5) = -139dBm

  EByte

  • E19-433M30S
    • 3V3 to 5V
    • 25x37mm
    • 20mA RX
    • 3µA sleep
    • has u.fl-connector
    • max. TX-power 29.5dBm
    • LoRa RX-sensitivity figures unrealistic
  • E22-400M30S
    • 2V5 to 5V5
    • 24x38.5mm
    • 14mA RX
    • 3µA sleep
    • has u.fl-connector
    • max. TX-power 21.5dBm
    • LoRa RX-sensitivity figures unrealistic

  HopeRF

  • RFM98PW
    • 5V to 6V
    • 18x35mm
    • 15mA RX
    • 5µA sleep
    • no coax connector
    • max. TX-pwr=30dBm
    • LoRa RX-sensitivity (SF12, BW=125kHz) = -136dBm
  • Seeedstudio RFM98
    • 2mm pitch, not breadboard friendly

  Microchip

  RN2483, includes an MCU so there's no direct access to the transceiver. 
  

• Speech codec

  Christoph Tack • 08/28/2020 at 19:16 • 0 comments

  Codec2

  • open source, royalty free replacement for AMBE.  At 2400bps, AMBE+ still performs better than Codec2.
  • down to 700bps
  • already implemented on embedded platforms : STM32, nRF52
  • used in FreeDV, QRadioLink

  Opus

  • open source, royalty free
  • replacement for Speex
  • down to 6kbps
  • used in VoIP-applications (e.g. WhatsApp)

  MELPe

  • NATO standard
  • licensed & copyrighted

  Speech-to-text-to-speech

  Using a speech codec, data transmission can be brought down to about 1200bps.  But how can we reduce data even further.  Let's take a 1min52s mono 8kHz speech sample as an example
  

  1. ve9qrp.wav : 1.799.212 bytes : 128000bps
  2. ve9qrp.bin : codec2 1200bps encoded : 16.866 bytes : 1200bps
  3. ve9qrp.txt : audio transcription of ve9qpr.wav : 1.178 bytes : 85bps

  Using codec2, we get a 106/1 compression ratio, using text 1527/1 compression ratio.  That's almost fifteen times better!
  

  If we use speech recognition on the transmit side, then send the transcription over and use speech synthesis on the receiving side, this might work.
  

  Speech recognition

  https://pypi.org/project/SpeechRecognition/

  Even though speech recognition engines are very good these days, they're still not as good as human beings.  The transcript text could be shown to the speaker during transmission.  In case of errors, the speaker could repeat the incorrect word or spell out its characters.

  A speech engine also requires a language preset.  That shouldn't be too much of a hurdle because we all speak the same language most of the times.
  

  Is there good speech recognition software that runs offline?

  Is speech recognition software not too power hungry?

  Speech synthesis

  Using Linux command line tools
  

• Physical layer : wireless communication

  Christoph Tack • 08/26/2020 at 19:25 • 0 comments

  Theory

  Channel capacity

  Shannon-Hartley law: C = B * log2(S/N + 1), where C is channel capacity [bps], B is bandwidth [Hz] and S/N is signal/noise ratio.

  Example: dPMR (C = 4800bps, B=6250Hz).  So S/N must be at least 0.70.

  Number of bits/symbol needed

  Nyquist's Theorem : C = 2*B *log2(N)

  Example: dPMR (C = 4800bps, B=6250Hz).  So N is 1.3bits/symbol.

  LoRa

  Why LoRa?

  Gray coding, data whitening, interleaving, forward error correcting is all done in the transceiver chip.
  

  • CSS (Chirp Spread Spectrum) :
    • Resilience to interference
    • Performance at low power
    • Resistance to multi-path fading
    • Resistance to Doppler effect
    • Easy to decode, so can also easily be done on lower power simple devices.
  • Gray coding : adds error tolerance
  • Data whitening : induces randomness
    • removing DC-bias in data
    • long bit runs become less likely
    • Whitening tends to distribute the data evenly across the radio channel’s frequency bandwidth, allowing the transmitter to run at a higher power without violating FCC regulations.
    • Helps receiver synchronization
  • Interleaving : Shuffling bits in a frame : reduces the impact of burst errors
  • Forward Error Correction : Hamming codes
  • 433MHz : less Free Space Path Loss (FSPL) than 868MHz.
    • Actually as we're on earth and not in free space, we should use the MPPL instead which takes into account the Fresnel zone due to the reflections of the earth. The losses then increase with the 4th power of the distance.
  • maximum effective bit-rate : 37.5kbps
  • 256 byte FIFO, shared for RX & TX
  • Critical parameters:
    • spreading factor (SF):
      • Each step up in SF means doubling of air time (i.e. bitrate/2).  There's always one symbol per chirp.  So increasing SF means lowering the chirp rate.
      • Each step up in SF means an extra bit per symbol (e.g. SF7 = 7bit per symbol)
      • Each step up in SF correlates to about 2.5dB extra link budget
      • TTN uses Adaptive Data Rate (ADR) to use bandwidth and power more efficiently.
    • modulation bandwidth
      • higher bandwidth has the disadvantage of higher noise floor
    • error coding rate

  Why not LoRa?

  The maximum effective bit rate according to the SX1276 datasheet is 37.5kbps, which is done at 500kHz BW, SF6. 
  Where is that legally possible (ETSI EN 300 220-2 V3.2.1 (2018-06), Annex B)?

  • Band H : 433,050 MHz to 434,790 MHz, but with following limitations: ERP = 10mW, duty cycle = 10%.  That leaves you 3.75kbps at 10mW.

  All other bands are either too narrow, have even more stringent limits on duty cycle or have stricter power limitations.

  Air time calculation : The fastest possible is SF7 with 250kHz bandwidth and a packet size of 120bytes.  Aside from the thought that such a large frame size might not be a good idea, this results to 100ms air time per packet, which is corresponds to the maximum duty cycle of 10%.  So we could send 120bytes/s = 960bits/s.  We could also send two 50ms air-time packets per second, but these would contain only 50 bytes each.

  Side note

  Polite spectrum access = listen before transmit (LBT) and adaptive frequency agility (AFA).

  As contradictory as it might seem, LBT+AFA is no benefit over 10% duty cycle.  It restricts the system to 100s per hour per 200kHz bandwidth (=2.8% duty cycle), a maximum transmission time of 4s and so on... (see ETSI EN 300 220-1 V3.1.1 (2017-02), 5.21.3.1).

  FSK

  Some basics

  • Frequency deviation (FDEV) = the difference between the minimum and maximum extent of a frequency modulated signal, and the nominal center or carrier frequency.  fcarrier - fmin = fmax - fcarrier = FDEV
  • Bandwidth is defined by Carson's Rule : BW = 2*(FDEV+BR/2) = BR+2*FDEV
  • modulation index (m) = FDEV(max) / fmodulatingMAX = FDEV/(BR/2) = FDEV*2/BR
    • MSK when m = 0.5, from FDEV*2=BR/2
    • Example MSK calculation for GSM
    • Narrowband FM when m < 1
    • For SX1276 m must be between 0.5 and 10 
  • Possible bands:
    • PMR446

  WiFi

  I'm not planning to turn this into an actual application.  It's only a proof of concept.  If you want a VoIP solution that you could...

  Read more »

• Codec2 evaluation

  Christoph Tack • 08/15/2020 at 12:01 • 0 comments

  Evaluation of Codec2 will be done using command-line tools and python tools.

  • Command line tools
  • Python

  Command line tools

  sudo apt install codec2

  Some experiments

  After installation of codec2, the raw audio file test samples are available in /usr/share/codec2/raw
  

  I tried playing with the 700bps bitrate, but that never yielded results that were easily understandable.  If you had a conversation with someone using this bitrate, you would frequently have to ask to repeat sentences.

  1200bps seems to me the minimum practically achievable bitrate.
  

  christoph@christoph-ThinkPad-L580:/usr/share/codec2/raw$ c2enc 1200 ve9qrp.raw ~/ve9qrp.bit --natural && c2dec 1200 ~/ve9qrp.bit ~/ve9qrp_codec2_1200.raw --natural && aplay -f S16_LE ~/ve9qrp_codec2_1200.raw 
  max_amp: 80 m_pitch: 320
  p_min: 20 p_max: 160
  Wo_min: 0.039270 Wo_max: 0.314159
  nw: 279 tw: 40
  max_amp: 80 m_pitch: 320
  p_min: 20 p_max: 160
  Wo_min: 0.039270 Wo_max: 0.314159
  nw: 279 tw: 40
  Playing raw data '/home/christoph/vk5qi_codec2_1200.raw' : Signed 16 bit Little Endian, Rate 8000 Hz, Mono
  christoph@christoph-ThinkPad-L580:/usr/share/codec2/raw$ 

  For your audio playback convenience, these raw files have been converted to WAV-file format using:

  sox -e signed-integer -b 16 -r 8000 -c 1 ve9qrp.raw ~/ve9qrp.wav

  File sizes:

  • ve9qrp.raw (original file) : 16bit samples, 8kHz sampling = 128ksps -> 1799168 bytes (WAV-file version)
  • ve9qrp.bit (codec2 1200 encoded) : 1.2ksps : 16866 bytes
  • ve9qrp_codec2_1200.raw (decoded) : 1799040 bytes (WAV-file version)

  So we see that codec2 achieves a 128k/1.2k = 106.7/1 compression ratio.  That's truly impressive.

  Of course, this compression ratio comes at a price : computational complexity.  There's no way you could pull this off in real time with an AVR-microcontroller.  You need at least an MCU with an FPU, such as the STM32F4.  An ESP32 doesn't seem suited either.  A Raspberry Pi could be used at the cost of higher current consumption and a lot longer startup time.  Would you want a walkie-talkie that you switch on and have to wait a minute before you can use it?  I wouldn't.
  

  TEST0: offline encoding & decoding using cli-tools

  The codec2 examples you can find on the internet are presumably specially chosen to go well with the algorithm.  Let's grab a video from youtube using a Youtube Video Downloader.  This will give you an mp4-file.  Strip the audio from that video and convert the audio to 8kHz mono and strip it down to the first two minutes using a only a single command:

  ffmpeg -i Who\ Invented\ the\ Food\ Pyramid\ and\ Why\ You\'d\ Be\ Crazy\ to\ Follow\ It.mp4 -acodec pcm_s16le -ac 1 -ar 8000 -t 00:02:00 out.wav

   Then using codec2 on that file is as simple as:
  

  c2enc 1200 ve9qrp.wav ve9qrp.bit
  c2dec 1200 ve9qrp.bit ve9qrp_decoded.raw
  ffmpeg -f s16le -ar 8k -ac 1 -i ve9qrp_decoded.raw ve9qrp_decoded.wav

  The sample from the Youtube video, after running it through codec2 sounds like this.  No, it doesn't sound great, but keep in mind that the original video has a 44.1kHz stereo signal.  Converting that to 8kHz mono already has an audible impact.  Passing it through a 1200bps codec2 tunnel is responsible for the other artifacts.

  TEST1: Sine waves

  It's easy to generate sine waves online and then downsampling them to 8kHz  (sox 440.wav -r 8000 440_8kHz.wav).  Unfortunately, pure sine waves are filtered completely out by codec2.
  

  Python

  When you're using Ubuntu, version > 19 is needed.
  

  sudo apt install python3-pip libcodec2-dev

  It might be better not to use "sudo" to avoid messing up the libraries that come with your Linux distribution. Alternatively, you can use PyCharm and use a virtual environment where all of these libraries get installed.

  sudo pip3 install Cython
  sudo pip3 install numpy
  sudo pip3 install pycodec2
  

  TEST1: offline encoding & decoding using python

  Download example.py:
  

  wget https://raw.githubusercontent.com/gregorias...

  Read more »

• Python : audio capture & playback

  Christoph Tack • 08/10/2020 at 11:32 • 0 comments

  Requirements for pyaudio:
  

  sudo apt install libportaudio2 libportaudiocpp0 python3-pip portaudio19-dev
  sudo pip3 install pyaudio
  sudo pip3 install sounddevice

  Some simple test applications:

  • play sound from WAV-file: plays 8kHz files on Wandboard, but not on my PC, as 8kHz is not supported by the hardware.
  • record sound from mic/line-in to WAV-file
  • audio pass through from mic/line-in to line-out, implemented in two ways: using pyaudio and using sounddevice library.  On the Wandboard, the sounddevice library only seems to work for 30s or so.  After that, there's sonic boom on the output.  The Wandboard needs to be power-cycled to restore sound output.  The pyaudio library doesn't seem to have that issue and keeps playing Judas Priest without problems.

  References:

  • Python audio modules
  • Playing and Recording sound in Python

  Audio resampling

  Codec2 works with a fixed sample rate of 8kHz.  The line-in and line-out work at 48kHz.  A sample rate conversion is needed.  Several options are considered:

  audioop.ratecv
  

  No libraries need to be installed.  The implemented filters are simple first order filters and it certainly sounds like that.

  This works both on the Core i7 8thGen and the Wandboard.
  

  play_sound_ratecv
  

  samplerate

  sudo pip3 install samplerate

  This library runs fine on a Core i7 8thGen and on the Wandboard.  Be sure to convert the output of this library to int16 before feeding it back to pyAudio.  Python doesn't complain about wrong types as a C++ program would, but the result sounds terrible.
  

  This "sync-best"-filter is audibly the best performing filter.
  

  play_sound_samplerate
  

  resampy
  

  Works fine on a Core i7 8thGen.  Takes more than two hours to build on a Wandboard.  This package has more than 300MB of dependencies.
  

  sudo apt install gfortran llvm-dev libblas3 liblapack3 liblapack-dev libblas-dev
  sudo pip3 install resampy

  play_sound_resampy : open a 8kHz wave file, up-sample it to 48kHz and then play it.  Works fine on a Core i7 8thGen.  On the Wandboard, it takes much longer to start and it hangs.  No sound is ever generated.

• Data layers

  Christoph Tack • 08/09/2020 at 21:15 • 0 comments

  Physical layer

  
  Physical layer : wireless communication

  • Frequency band:
    • LPD433 : 433,050 MHz to 434,790 MHz: 10mW ERP, 10% duty cycle, BWmax=the whole band.
    • PMR446 : not an option for LoRa, maybe for FSK?
    • SRD860 :
      • 863-865MHz : 25mW ERP,  no duty cycle limitation when using LBT+AFA, BWmax = 300kHz.
      • 869,400 MHz to 869,650 MHz : 500mW ERP, duty cycle : 10% or LBT+AFA, BWmax = 250kHz.

  10dBm ERP = 12.15dBm EIRP
  

  Data link layer

  LoRa

  The walkie-talkie application doesn't need that amount of overhead. We can suffice with the OSI layer 2 provided by the wireless chipset:

  Depending on the chosen Codec2 bitrate, we'll have to transfer 8bytes 50 times/s down to 6bytes 25 times/s.

  Libraries

  1. RadioLib
  2. RadioHead
  3. LoRaLayer2

• Codec2 configuration

  Christoph Tack • 08/09/2020 at 14:48 • 0 comments

   As the walkie talkie will use digital voice transmission, we need a way to digitize speech.  Several open source speech codecs are available.  We will focus on low-bitrate codecs because we want long range.  Opus and Speex won't do.  There's one codec that excels : codec2 :

  • bitrates as low as 700bps possible (but not usable, see Codec2 evaluation)
  • open source
  • existing implementation on PC, STM32 and nRF52
  • used in ham radio applications (e.g. FreeDV)

  Codec2 technical details

  Audio input format

  16bit signed integer, 8kHz sample rate, mono

  Codec2 packet details

  Reference : codec2 source code

  Encoded Data rate [bps]	Bits/packet	Bytes/packet	Time interval [ms]	Packets/s
  3200	64	8	20	50
  2400	48	6	20	50
  1600	64	8	40	25
  1400	56	7	40	25
  1200	48	6	40	25

  When using one of the lowest three data rates, there's a drawback that loosing a single packet will cost you 40ms of audio.
  

View all 9 project logs