I initially spent some time playing with sound cards and with Fluidsynth to see if it could be used reasonably in live performance. The Pi's built-in sound jack is right out - my web research tells me the Broadcom chip downsamples audio to 11kHz, which sounds unacceptably crummy on many patches. I thought it might have lower latency, but this is not the case. With the right buffer/period settings (will provide later) on Fluidsynth the latency is not noticeable and the sound through my little USB sound card is actually pretty decent. The one horrible thing that does happen is that if you mash a lot of notes at once (10-12), Fluidsynth gets overloaded somewhere, and rather than simply dropping notes it produces horrific digitized car crash sounds until you stop pressing keys for a few seconds. Maybe some kind of memory overload? I couldn't find any help on this issue (please advise if you know what's going on) and haven't had any luck fixing it myself, but after a while I decided maybe I just don't need to hit 10 notes at once.
I did some prototyping to make sure I knew how to wire buttons and a character LCD, and that my python script didn't interfere with Fluidsynth. I had to add some bindings to pyfluidsynth to start up the MIDI driver and router - apparently it was just designed for people who want to play MIDI files. I did some jamming with my proto board synth, and everything felt and sounded good, so I proceeded to work on putting things into the stompbox.
There isn't a lot of extra space in the box with the Pi, proto hat, LCD, stompswitches, and various connectors in there. The USB-B power jack and USB-A jack for midi controller will be soldered onto the proto hat for stability. I plan to anchor the proto hat to the box with standoffs, then the Pi will be held in place with its GPIO connector and additional standoffs. I need to get USB from the Pi's ports to the sound card, and to the USB-A jack on the hat, and there's barely any space in the box over there, so I will fashion my own short right-angle USB cables by cutting down a couple USB plugs as small as possible, soldering on wires, and shrink-wrapping.