Piano signal processor

Project Logs

Collapse

Channel swapping
lion mclionhead • 04/07/2024 at 22:32 • 0 comments

Since the headphone connector is on the left, the right cable is getting more strain. It's always breaking in the knot. The decision was made to swap the playback channels in software & wear the phones backwards to spread the strain. It's unlikely anyone else in the world has resorted to such drastic action to keep 38 year old phones alive.
Also tweeked the reverb again. It seems to cut completely off when reverb_level2 goes below -50. It could be a quantization problem with float32.
Android tweek
lion mclionhead • 02/22/2024 at 07:39 • 0 comments

Got it to show the status bar & use smaller buttons. Just used DOS character codes for arrows. Drawing images entails a lot of work to size the images. Being able to increment & decrement the beats per minute with buttons is a big win, now that double tapping doesn't trigger a zoom. If only there was enough room to increment & decrement the sliders.

Vintage DOS characters have a lot going for them. The original DOS character set, Code page 437, is stored in every graphics card from 1981 until this day & every phone. Lions originally thought it was cheap, upon 1st seeing it in 1985. It couldn't draw images like PETSCII. It never appealed as much as PETSCII, but it won the contest. When lions saw it in the form of Sony Fixed, in 1995, it looked a lot better.

There is a desire to configure the reverb on the phone. It doesn't really do anything useful except maybe emulate the HFAC.
Android rewrite
lion mclionhead • 02/19/2024 at 19:24 • 0 comments

The browser app has been just good enough, but it's awful. Phone browsers just can't do polling in javascript nearly as fast as commodore 64 basic. The effort to convert it to an app was stalled by the prospects of having to commute again & the longevity estimates of this project. There's no way to practice when commuting & that could happen in less than a year.

With retirement rapidly approaching, lions have pondered the future of the CP33. It's most likely going to stay the mane instrument as before & lions will never own an accoustic piano, but there's a nonzero chance it'll get replaced by an acoustic piano in less future time than it's past lifetime. It would be a waste to rewrite the signal processor if it only has 5 years to go.

Another full day rewriting most of the raspberry pi code finally pushed it to the finish line.

It's still very minimal, but it's super fast. It turned out libusb_init leaks memory if it's repeatedly called. The app more proactively disables user input until it gets a packet from the instrument containing the instrument's state. If the instrument is rebooted after the app detects it, it'll overwrite the instrument's initial state. This shouldn't be a problem, since rebooting the instrument shouldn't undo anything the app previously set.
The lion kingdom never used the reverb again, after the early days discovering it made it impossible to hear nuances.

There were ideas to require a confirm checkbox to make the update buttons do anything. There's also room for + - buttons for the metronome.

The address text is arguably useless. It could be hard coded.
Noted the app isn't showing the phone's time bar. They all seem to require a title bar before before they show the time bar, otherwise they just have a black bar on top. The truckcam app managed to show the time bar without a title bar.
Dexed on raspberry pi
lion mclionhead • 11/28/2023 at 18:45 • 0 comments

There was a guy who ported a DX7 emulator to a bare metal raspberry pi. Why the port was on bare metal is beyond lions. Maybe the pi wasn't fast enough to do all the processing. The process does seem to involve porting a VST plugin from source. There's no windows emulator for the pi which can run a headless VST plugin.

The most likely outcome for a DX7, K3, & piano emulator is going to be a dedicated windows confuser.
Phone stand
lion mclionhead • 11/13/2021 at 07:44 • 0 comments

Desperately needed a phone stand. The phone was all over the place.
True hotplug, buffer overruns
lion mclionhead • 04/06/2021 at 06:05 • 0 comments
To run it as a daemon, it needed to truly detect when the soundcard & the instrument were unplugged, then stop the signal processing when the instrument was unplugged. Detecting when ALSA is unplugged requires a complicated polling routine.
```
snd_pcm_status_t *status;
snd_pcm_status_malloc(&status);
snd_pcm_status(dsp_out, status);
int state = snd_pcm_status_get_state(status);
snd_pcm_status_free(status);

if(state == 0)
{

// disconnected

}
```
Detecting a USB disconnect requires testing for a LIBUSB_TRANSFER_STALL.

The next problem was buffer overruns when recording, despite using O_DIRECT. The activity LED tended to get stuck on.

The raspberry pi uses a swap space on the SD card which must be disabled.
```
root@piano:/root% swapon --show

NAME      TYPE SIZE USED PRIO
/var/swap file 100M   0B   -2
```
The easiest solution is renaming /usr/sbin/swapon to /usr/sbin/swapon.bak

The raspberry pi has enough RAM to record 3 hours of audio in RAM, but to simplify the pointer math, the FIFO can be expanded to just 30 minutes. The worst case would be to record to RAM & write it after the user stops, but it wouldn't be resistant to power outages.
After recording for 51 minutes,
```
F   UID   PID  PPID PRI  NI    VSZ   RSS WCHAN  STAT TTY        TIME COMMAND

4     0  1790     1  -2   - 772096 469844 hrtime Sl  ?         30:29 ./piano
```
It filled the entire allocated buffer because it's a ring buffer, but it didn't have a single overrun. It could be optimized by capturing the longest fill size, but that would be wasting the raspberry pi's 4GB of RAM. The LED only seemed to stay on for 1 minute at most.
Handling Device Disconnects
lion mclionhead • 03/23/2021 at 18:55 • 0 comments

Finally ended up with a satisfying GUI that could control line input for monitoring a microphone. 64 bit file support is still an issue with the raspberry pi's because they're 32 bit.  It may never be tested with 64 bit file support because lions have never played more than 2 hours at a time without stopping.

It detects disconnection & connection of the instrument but not disconnection of ALSA. The soundcard just doesn't get disconnected. Recovery after the loss of any peripheral would be best done by restarting the program inside a loop, but that would make it hard to debug. If it restarted the program during a disconnect, it would stop recording.

The final solution with the web interface was to have it read the board state once when it starts, then ignore the board state in favor of user input thereafter. It continuously polls to get the state of the recording.

If another GUI starts or if the board reboots, the 1st GUI just won't show any valid board state. There's no plan to have the board reboot on its own. The board could send a flag requiring the GUI to reset, but it's a bit overcomplicated.

There was some desire to have a VU meter. The digital stage in the CP33 doesn't have variable volume. It's calibrated for an average playing volume & just clips when it's too loud. A VU meter would have no purpose & take up space.

The thought isn't lost of continuing to expand the signal processor to capture MIDI & generate human readable notation. It could pass through MIDI to a phone or it could use its own monitor on HDMI. All gadgets have been evolving to a dumb web based GUI with all the computation on a single board confuser.
Web interface
lion mclionhead • 03/21/2021 at 05:11 • 0 comments

After a protracted amount of hacking, it became clear that there's a minimum window size below which the FFT causes aliasing. 2048 seemed to be the minimum before which you can apply a lowpass filter. The window size is limited by the amount which can be processed between fragments, not the amount per second. The pi falls over above 2048 samples. Another 1024 samples are needed because the windows have to overlap 1/2 a window at a time rather than 1 fragment at a time. That creates a 70ms delay, which isn't terrible because the reverb needs a delay anyways.

In the wifi department, problems continued. A switch to 5Ghz didn't change the traditional raspberry pi lockups. At least it was a discovery that the raspberry pi 3 & 4 both support 5Ghz. They can't do 2.4 & 5 simultaneously & they're not reliable enough for anything besides ssh. The mane problem with a fully wired connection is those ancient RJ45 connectors always breaking.

5Ghz would be useless, since the web interface is on a cheap phone & the phone only supports 2.4Ghz. The lion kingdom may end up dedicating a phone just for a piano GUI.
After a week of hacking, it finally had a minimally functioning web interface. The mane challenge with a web interface has always been making the phone read the state from the board while simultaneously sending a new state from its GUI. The latest idea was to flag every widget as write only or read only. They all start off as read only, showing the state of the board. When the user touches a widget, it turns to write only, showing only the user input.
If a 2nd GUI is started, the latest GUI becomes the only one with the true state of the board. Persistent settings have to be stored in the board rather than cookies because it's a pain to lose the settings every time you change browsers.
Helas, the web interface has to constantly poll to get the starting board state & the current recording state. If the recording runs out of space & stops, the GUI can't update to show recording has stopped. It'll just show no space remaneing & no advancement of the counter. Lions don't expect to run out of space, but handling this would require the record button to have a read only, write only & a 3rd state for out of space.
There's never been any support for playback in a lion recording device. It's 1 of those things lions never needed, even in their days of using walkmans for portable recording. Maybe having it permanently connected to the instrument & easy to access will make playback useful. It would entail showing a list of recorded files & a full player interface on the phone, while the playback would still be on the pi. The ALSA driver would have to switch between playback & recording.
When the web interface starts polling, the wifi quickly dies while the wired connection keeps going. Something about polling triggers a Wifi interrupt problem. Being a stationary instrument, there's no plan to use wifi, but it is 1 more cable to manage.
Audio processing
lion mclionhead • 03/18/2021 at 06:59 • 0 comments

There was another experiment to pass analog audio from line input to phones output like a normal musician would do. The minimum buffer size for ALSA was 64 samples & the latency was negligible. There's little point to it, since the digital mode ended up working & it gives an exact preview of what's being recorded.

The I2S to USB converter had an obscure parameter for adjusting the packet size. This could go all the way down to 64 samples while ALSA bottomed out at 64 samples. This latency was acceptable. There's still a slight softness in the keys from latency, but it's the best lions can afford for what it is.

The I2S clock is slightly faster than the sound card clock, so interpolation & extra buffering was necessary. The clock drift requires a buffer that occasionally reads 1 more sample than it writes. The mane limitation on buffer size is ALSA reading 64 at a time & I2S writing 64 at a time. You need 2 64 sample buffers to be sure ALSA always has data to read. The extra input sample puts it at 129, but audio comes in blocks of 64, so 192 samples is the smallest lag before the interpolation becomes possible.

To make recording work, the file had to be opened O_DIRECT & written with page aligned buffers. Raw I/O has come a long way since lions last used it, 20 years ago. There wasn't originally a way to prevent filesystem caching without using a raw partition. SD cards go a lot faster with O_DIRECT. After 10 years of not knowing if recording was working, there was finally a live preview of exactly what's being recorded.

Reverb required a real short FFT & floating point. FFT rather than FIR has proven the only way to cut off the ringing. With recording, reverb, & metronome, the PI with ondemand frequency scaling managed to keep up with 31% of CPU usage. The PI automatically goes to 800Mhz. If the clockspeed is locked to 1.5Ghz, it drops to 18%. Using doubles instead of floats kicks it up to 20% & doesn't affect the sound. There's still an annoying artifact in the reverb which peaks at middle A.

An ESP32 which converted directly from I2S to headphones would take several weeks to order but be the most realtime. It would not be fast enough to do a comparable reverb.

Another strategy was to reprogram the I2S to USB converter to also add reverb, metronome, & a DAC inside the case. It would output audio directly to the phones while continuing to send data to USB for recording. It might need a UART interface to control the reverb & metronome. This too wouldn't have enough horsepower for much of a reverb.

The worst case was to abandon audio processing, use the ALSA mixer to monitor the line input & only use software for playing metronome sounds.
Raspberry pi failure
lion mclionhead • 03/18/2021 at 02:56 • 0 comments

The 1st idea was to read audio via the I2S to USB converter of 10 years ago, then write it to a junk USB soundcard in realtime after processing. This was the 1st time that soundcard produced any useful sound in 20 years, but it was an instant failure. The latency was too long, even with with MMIO, 768 samples buffered. Most of the latency appeared to be in the USB protocol. It was so bad, it would really have to process 1 sample at a time.

Another idea was using an ESP32 with no buffers. There was a definite advantage in keeping the I2S to USB converter, since multiple devices could capture I2S audio from outside the case. The ESP32 would either capture I2S directly, inside the case, or the I2S to USB converter could output something lower latency than USB. Wifi wouldn't work very well inside the case. The I2S could reach outside the case, with some drilling. Then, either the USB converter or the ESP32 could plug in.
Then of course, there's just passing analog through the soundcard for live effects & reserving the I2S just for recording.