hgSynth

ESP32 is really awesome little computer. Two CPU cores, high speed, a bunch of peripherals, wireless networking, low price. It even has a built in DAC. It's only 8-bit, but hey! It's there! No more lousy PWM filtering to get some sound from MCU.

So, an idea came to me... What would it be like to build synth based on that CPU. I was really surprised not to find existing synth designs for it so I decided to make my own. Even better - why not build it inside a bass guitar and have cool synth sounds to mix them with direct bass sound. And the best part is that it doesn't even need keyboard - pitch can be extracted from the direct bass signal!

This is the wishlist for full design:

ESP32 based (obviously)
nice, analog style oscillators
pitch tracking
integrated tuner (since we already have pitch tracking)
onboard controls to shape the sound
OLED display to show the signal (some eye candy :))
multiple patches/presets switchable during runtime
parameter configuration done over WiFi/BLE
try to keep the power consumption low
(optional) MIDI support so it can be used as "desktop" module
some better name than hgSynth

Idea is to build the first prototype using existing dev board and then do the second one on PCB with ESP32-WROOM-32U module.

Integrated DAC is only 8-bit but it will be enough for first prototype. Since ESP32 has I2S lines, external, better quality DAC, can be added easily.

Synth is classic two-oscillator design:

two oscillators (sine, triangle, sawtooth, square) with configurable note offset, detune
envelope generator (same envelope is used for filter modulation)
resonant lowpass or multimode filter
LFO which can modulate all interesting parameters

This is initial design. I might add another LFO or envelope.

Project Logs

Collapse

Video demo
Igor Brkic • 10/08/2018 at 13:10 • 0 comments

first, very lousy, demo (will upload better demo soon)
Stuffing everything in
Igor Brkic • 10/08/2018 at 01:54 • 0 comments

Like always, whenever you want to do "just this" it turns out all bunch of things get in the way.

It turns out things can't really happen as I imagined. Here I'm thinking about the blend between the bass guitar sound and synth sound. Using just a double potentiometer that was in the guitar wasn't good since the clean bass sound is needed as an input to synth (main out was "leaking" back to synth input which caused feedback). That's why I added a simplest JFET buffer before the blend control. I also added the same buffer to ADC input but with a bit higher gain (around 2).

TODO: add video and some description
After a long break...
Igor Brkic • 10/06/2018 at 21:53 • 0 comments

So, I did most of the work on the synth during two weeks in May. But then a lot of things happened at my day job and some other things came in so I haven't touched the synth since then (almost five months :)). There is a deadline for Hackaday prize in two days so I decided to revisit and finish what I started earlier.

Of course, I can't make all the features in two days (actually, one, since I have few other things to do) but I can try to make it usable.
What I will try to do is embed it into my bass guitar so I can play it. I have few unsolved things, like power supply and signal conditioning but I'll leave those for the next prototype which will be on PCB and with ESP32 module.
Pitch tracking
Igor Brkic • 05/10/2018 at 22:37 • 0 comments

There are many methods for pitch detection of a sound signal both in time and in frequency domain, from very simple to very complex. Simplest one would be to count signal zero crossings and calculate frequency from it. It would work just fine on perfect sine wave without noise and/or higher harmonics.One of the simple ones with good results in presence of noise and higher harmonics is basic autocorrelation method (and a lot of its variants) which correlates a block of signal with itself to find its periodicity. Of course, we need big enough block of data for this and sometimes it returns double frequency but it's workable. Problem with it is the required amount of calculation. Basically, you place (windowed, to avoid "sharp" edges at the beginning and the end of the block) block of signal on top of itself, multiply matching samples and sum everything. One copy of the signal is then shifted one sample to the side and the process is repeated (for all shifts where overlap exists). Some optimizations can be added by using FFT (algorithm for fast calculation of Fourier transform) but still it's a lot of calculations which isn't really suited for lower powered MCUs (even though ESP32 isn't really that low spec).

So, almost by accident, I discovered a really cool new method for very efficient pitch detection, even on MCUs. It's called Bitstream Autocorrelation and it's invented by Joel de Guzman from Cycfi Research. Check out the site for explanation and sample code. And I was amazed how great it works. Even on signals with really high amount of higher harmonics.

So, I took Joel's demo code pretty much without any changes, stuffed it into synth code and it works!

For bass guitar's signal input conditioning I just added two resistors to bias it to the middle of ESP's ADC input and added a simple RC lowpass (antialiasing :)) filter. It works but the signal is low so a proper signal conditioning which includes both lowpass filtering and amplification is required.
Bandlimited oscillators
Igor Brkic • 05/06/2018 at 22:04 • 0 comments

So, creating a good sounding square wave digital oscillator is easy — just send a bunch of zeros followed by bunch of ones (where "bunch" is related to note frequency). After all, it's just a square wave with two alternating values. Well, not exactly...

When working in digital domain (sampled signals) we have one limitation known as Nyquist-Shannon sampling theorem which states that sampling frequency has to be at least two times higher than maximum frequency contained in signal (it should be even higher for better reconstruction). And the same limitation exists both when sampling input signal and when converting digital signal to analog.

What exactly happens if signal contains frequencies higher than half of the sampling frequency? It's called aliasing — frequencies higher than half the sampling frequency appear as f-fs signal which can be used for some interesting effects (https://www.youtube.com/watch?v=uENITui5_jU) but usually we don't want it. Especially when creating oscillators.

So, what to do? Easy, remove (or don't include) everything above fs/2 before D/A or A/D conversion.

When sampling analog signal an external anti-aliasing filter is required in front of ADC.

But when synthesizing, we can immediately create signal which will not alias (in reality not that easy, especially if we have some saturators or non-linear elements).

By using Fourier analysis, every signal can be decomposed as a sum of sine and cosine signals (partials) with frequencies which are multiples of base frequency. So, if we want to create square wave signal which contains fs/2 at most then we'll sum all the partials up until that frequency. And that is called band limited synthesis.

In this project I'll be using wavetable oscillators (signal shape is saved in a table and played at the desired frequency). Good oscillator would use multiple tables (at least one per octave) but I just want to get something working so there will be only one table. So, when highest available note/frequency is played by oscillator it mustn't contain any partials above fs/2. Downside of this (using only one table)) is that for low notes we have quite a lot of "unused space" in spectrum and it sounds a bit dull without additional higher frequency components — a thing that can (will, at some point) easily be fixed with additional tables, each for one range of frequencies.
For this I've written simple Jupyter notebook which can calculate the tables and plot the signal shape (can be found in data/bandlimited_waveforms.ipynb in Git repository).
Adding partials to generate band-limited square wave
Got sounds! And GUI!
Igor Brkic • 05/06/2018 at 22:02 • 0 comments

So, I'm happy to say that progress is happening. I got the sound! And waveform on OLED. And initial web GUI.

First, a video of an OLED and the GUI (sound is really low and can be heard only at the end - it was late at night):

Now onto technical details...

Web GUI

For HTTP and websockets server I used https://github.com/me-no-dev/ESPAsyncWebServer library. It is a really nice library with one downside - it includes FS and SPIFFS ("filesystem" on ESP32's flash chip) support and when it's included code uploading takes additional 20-30 seconds (if someone knows how to turn this off please let me know). It has a support for serving files directly from SPIFFS but I decided not to use it. Using it would require additional step of uploading html/js/css files to flash which is OK but I'd like to handle everything related to uploading code in one step. So, here's my kind of hackish solution - insert the whole web page in the code as a const value (this way it will be written to flash). To minimize the code I additionally decided to use just vanilla JS without external libraries and I think the final product is quite OK - UI controls are defined in variable and dynamically built so it's easy to change UI by just changing that variable. To convert the whole html page to C header file I wrote another Python script (data/convert_to_header.py) which takes html, removes some whitespace, escapes some characters and packs it into a C header file.

ESP32's WiFi in this case connects to my local WiFi but in the real case it should start in the soft AP mode. Actually, I was thinking to first check if some predefined WiFi is available and connect to it and if it's not then start as AP.

Synth

Currently, oscillators are implemented via simple (non-bandlimited) lookup table. Filter is Paul Kellet's design (http://musicdsp.org/showone.php?id=29) - it's quite usable but I'll probably switch to some other design.
This first prototype contained components directly soldered to ESP board and a lot of flying wires so I decided to implement it on perfboard. Unfortunately, I disassembled the first prototype without taking a photo of it.
Initial software design thoughts
Igor Brkic • 05/02/2018 at 20:49 • 0 comments

ESP32 development is done using ESP-IDF (Espressif IoT Development Framework) which is official framework. But nice people also created Arduino framework for it which makes this really easy for anyone familiar with Arduino.

The standard Arduino IDE is nice and cute for small projects but the editor is really the weak spot for anything more complex so I decided to go with something else. PlatformIO is really nice tool which handles environment, compiling and uploading and it can be integrated with either Atom or VSCode as an editor. I chose to go with VSCode (the first MS product I've used in years :)) and it was a good decision.

After we've decided on dev env, let's start with code (or at least with code plans).

Components

So, from the DSP side we have pitch detection, oscillators, envelope generator and filter. Other than that, there needs to be a way to store and recall presets, there is user interface, ...

ESP32 runs FreeRTOS in the back and, even when Arduino framework is used, it's easy to create FreeRTOS tasks, use synchronization primitives, events and other goodies.

DSP

I have some previous DSP experience so this part was familiar. However, writing optimized DSP code is really hard. Luckily for me, ESP32 with its two cores and floating point support is here to help (yay, probably no need for fixed point arithmetic!). Other great thing is http://musicdsp.org/archive.php code archive - a great resource of great DSP code.

For pitch detection I decided to go with new method developed by Joel de Guzman from Cycfi Research. It is, so called, bitstream auto correlation which should be really computationally quite easy but promises exceptional results. I hope that will be the case.

For oscillators I was thinking about going with simple wavetable (actually, lookup table) design. But, since things can't really be simple, this complicates when we want nice sounding (i.e. alias free) sounds across the whole frequency range. So, ether approach with multiple wavetables is required or some band limited synthesis technique. For this I'll go with simple naive approach just to get something and later add the more advanced stuff.

I found nice series of articles on envelope generators (and a nice code) on EarLevel Engineering site: http://www.earlevel.com/main/2013/06/01/envelope-generators/. This will be more than enough for this part.

For filter I decided to start with Paul Kellet's simple design: http://musicdsp.org/showone.php?id=29 and later add better/more complex filter if needed.

GUI

First idea was to use BLE for user interface. This has a downside of requiring an app on a device so I decided that WiFi is much better choice (besides the power consumption). GUI will be a simple web app which uses websocket to communicate with backend.
Initial hardware design thoughts
Igor Brkic • 05/02/2018 at 20:41 • 0 comments

This is the first project log and will be mostly about choosing components and technical decisions.

ESP32 comes in different shapes and sizes. For prototyping it's easiest to take an existing board (even though using just ESP32 module isn't too much effort).

For this I realized I have few ESP32 based TTGO Lora boards which have small OLED display and LoRaWAN radio and which I don't really need (I bought 433Mhz version by mistake as local Lora community uses 868MHz). Positive side of this board is that it already has OLED display mounted. Negative side is that it is soldered to it and must stay near the board - I can't put board inside the bass guitar and the display on the front side. II solved this problem by attaching another similar display in parallel to this one.

I still haven't decided should I put the display on the front side of the bass guitar (for the "audience" and as a gimmick) or inside the "top side" (for me, and to be useful when used as a tuner). Also, I need to test will the overall signal be affected by adding WiFi/BLE radio near the electronics :).

Besides the ESP32 and OLED, on the hardware side we'll need a touch strip (I had one lying around) and two push buttons (one for synth control and another one for changing presets and turning WiFi on/off. Few resistors and capacitors will also be required for input/output signal conditioning.

And now to the software side...