Punchline first - got everything to fit inside the box, and it appears to be working (mostly) as intended. Video evidence:

I realized I'd have to do some reconfiguring to get everything to fit inside the stompbox. I made a couple USB extender cables out of some ribbon cable to connect from the Pi's USB ports to the MIDI in port on the hat and to the little USB sound card.

I also pried the case off the sound card so it would take up less space. Wrapped some pretty orange duct tape around a couple things to prevent things from shorting to the case, and crammed everything in the box. It's a pretty tight fit:

But it works! I've got jampi.py set up as a service so it starts up quick when the box is plugged in. There's something goofy in the code where the buttons register an event on both press and release, and I have many more plans for the interface code (and for modding fluidsynth itself), but progress is being made!