Piano signal processor

Add metronome, reverb, compression, recording in an outboard box.

Similar projects worth following

The original instrument had analog output, metronome & lousy reverb.  The idea with this was to add a metronome from other than the front panel, which is normally covered.  The metronome would be added to the headphone signal rather than the recorded signal.  A better reverb could be added.  Rather than connect an outboard MIDI synthesizer, it would process the audio directly since the original audio was pretty good.  The original audio is a 24 bit 44.1khz I2S signal which can be captured with solder.  It would support recording, with a way to send the recording over wifi.

The original instrument exposes its digital audio on the pins of a DAC.

It received an I2S to USB converter back in 2013 so devices outside the case could record digital audio.  They didn't do anything with it besides recording, but there was always a desire to have effects.

  • Handling Device Disconnects

    lion mclionhead03/23/2021 at 18:55 0 comments

    Finally ended up with a satisfying GUI that could control line input for monitoring a microphone.  64 bit file support is still an issue with the raspberry pi's because they're 32 bit.  It may never be tested with 64 bit file support because lions have never played more than 2 hours at a time without stopping.  

    It detects disconnection & connection of the instrument but not disconnection of ALSA.  The soundcard just doesn't get disconnected.  Recovery after the loss of any peripheral would be best done by restarting the program inside a loop, but that would make it hard to debug.  If it restarted the program during a disconnect, it would stop recording.

    The final solution with the web interface was to have it read the board state once when it starts, then ignore the board state in favor of user input thereafter.  It continuously polls to get the state of the recording.  

    If another GUI starts or if the board reboots, the 1st GUI just won't show any valid board state.  There's no plan to have the board reboot on its own.  The board could send a flag requiring the GUI to reset, but it's a bit overcomplicated.

    There was some desire to have a VU meter.  The digital stage in the CP33 doesn't have variable volume.  It's calibrated for an average playing volume & just clips when it's too loud.  A VU meter would have no purpose & take up space.  

    The thought isn't lost of continuing to expand the signal processor to capture MIDI & generate human readable notation.  It could pass through MIDI to a phone or it could use its own monitor on HDMI.  All gadgets have been evolving to a dumb web based GUI with all the computation on a single board confuser.  

  • Web interface

    lion mclionhead03/21/2021 at 05:11 0 comments

    After a protracted amount of hacking, it became clear that there's a minimum window size below which the FFT causes aliasing.   2048 seemed to be the minimum before which you can apply a lowpass filter.  The window size is limited by the amount which can be processed between fragments, not the amount per second.  The pi falls over above 2048 samples.  Another 1024 samples are needed because the windows have to overlap 1/2 a window at a time rather than 1 fragment at a time.  That creates a 70ms delay, which isn't terrible because the reverb needs a delay anyways.

    In the wifi department, problems continued.  A switch to 5Ghz didn't change the traditional raspberry pi lockups.  At least it was a discovery that the raspberry pi 3 & 4 both support 5Ghz.  They can't do 2.4 & 5 simultaneously & they're not reliable enough for anything besides ssh.   The mane problem with a fully wired connection is those ancient RJ45 connectors always breaking.

    5Ghz would be useless, since the web interface is on a cheap phone & the phone only supports 2.4Ghz.  The lion kingdom may end up dedicating a phone just for a piano GUI.

    After a week of hacking, it finally had a minimally functioning web interface.  The mane challenge with a web interface has always been making the phone read the state from the board while simultaneously sending a new state from its GUI.  The latest idea was to flag every widget as write only or read only. They all start off as read only, showing the state of the board.  When the user touches a widget, it turns to write only, showing only the user input.

    If a 2nd GUI is started, the latest GUI becomes the only one with the true state of the board.  Persistent settings have to be stored in the board rather than cookies because it's a pain to lose the settings every time you change browsers.

    Helas, the web interface has to constantly poll to get the starting board state & the current recording state.  If the recording runs out of space & stops, the GUI can't update to show recording has stopped.  It'll just show no space remaneing & no advancement of the counter.  Lions don't expect to run out of space, but handling this would require the record button to have a read only, write only & a 3rd state for out of space.

    There's never been any support for playback in a lion recording device.  It's 1 of those things lions never needed, even in their days of using walkmans for portable recording.  Maybe having it permanently connected to the instrument & easy to access will make playback useful.  It would entail showing a list of recorded files & a full player interface on the phone, while the playback would still be on the pi.  The ALSA driver would have to switch between playback & recording.

    When the web interface starts polling, the wifi quickly dies while the wired connection keeps going.  Something about polling triggers a Wifi interrupt problem.  Being a stationary instrument, there's no plan to use wifi, but it is 1 more cable to manage.

  • Audio processing

    lion mclionhead03/18/2021 at 06:59 0 comments

    There was another experiment to pass analog audio from line input to phones output like a normal musician would do.  The minimum buffer size for ALSA was 64 samples & the latency was negligible.  There's little point to it, since the digital mode ended up working & it gives an exact preview of what's being recorded.

    The I2S to USB converter had an obscure parameter for adjusting the packet size.  This could go all the way down to 64 samples while ALSA bottomed out at 64 samples.  This latency was acceptable.   There's still a slight softness in the keys from latency, but it's the best lions can afford for what it is.   

    The I2S clock is slightly faster than the sound card clock, so interpolation & extra buffering was necessary.  The clock drift requires a buffer that occasionally reads 1 more sample than it writes.  The mane limitation on buffer size is ALSA reading 64 at a time & I2S writing 64 at a time.  You need 2 64 sample buffers to be sure ALSA always has data to read.  The extra input sample puts it at 129, but audio comes in blocks of 64, so 192 samples is the smallest lag before the interpolation becomes possible.

    To make recording work, the file had to be opened O_DIRECT & written with page aligned buffers.  Raw I/O has come a long way since lions last used it, 20 years ago.  There wasn't originally a way to prevent filesystem caching without using a raw partition.  SD cards go a lot faster with O_DIRECT.  After 10 years of not knowing if recording was working, there was finally a live preview of exactly what's being recorded.

    Reverb required a real short FFT & floating point.  FFT rather than FIR has proven the only way to cut off the ringing.  With recording, reverb, & metronome, the PI with ondemand frequency scaling managed to keep up with 31% of CPU usage.  The PI automatically goes to 800Mhz.  If the clockspeed is locked to 1.5Ghz, it drops to 18%.  Using doubles instead of floats kicks it up to 20% & doesn't affect the sound.  There's still an annoying artifact in the reverb which peaks at middle A.

    An ESP32 which converted directly from I2S to headphones would take several weeks to order but be the most realtime.  It would not be fast enough to do a comparable reverb.

    Another strategy was to reprogram the I2S to USB converter to also add reverb, metronome, & a DAC inside the case.  It would output audio directly to the phones while continuing to send data to USB for recording.  It might need a UART interface to control the reverb & metronome.  This too wouldn't have enough horsepower for much of a reverb.

    The worst case was to abandon audio processing, use the ALSA mixer to monitor the line input & only use software for playing metronome sounds.

  • Raspberry pi failure

    lion mclionhead03/18/2021 at 02:56 0 comments

    The 1st idea was to read audio via the I2S to USB converter of 10 years ago, then write it to a junk USB soundcard in realtime after processing.  This was the 1st time that soundcard produced any useful sound in 20 years, but it was an instant failure.  The latency was too long, even with with MMIO, 768 samples buffered.  Most of the latency appeared to be in the USB protocol.  It was so bad, it would really have to process 1 sample at a time.

    Another idea was using an ESP32 with no buffers.  There was a definite advantage in keeping the I2S to USB converter, since multiple devices could capture I2S audio from outside the case.  The ESP32 would either capture I2S directly, inside the case, or the I2S to USB converter could output something lower latency than USB.   Wifi wouldn't work very well inside the case.  The I2S could reach outside the case, with some drilling.  Then, either the USB converter or the ESP32 could plug in.

    Then of course, there's just passing analog through the soundcard for live effects & reserving the I2S just for recording.

View all 4 project logs

Enjoy this project?



ray wrote 03/31/2021 at 17:08 point

What is the original keyboard?    Parts list?

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates