So I was noticing that the board would brown out and reset occasionally; usually when writing to the SPI flash chip. I could bypass the problem by plugging into USB, so I didn't bother looking into it too much until now.

I was confused because the regulator I was using was rated for 250mA, but I was pulling much less than that. I seem to average 100mA when playing the drums, and 140mA when copying files from SD card to SPI Flash. Obviously the extra current used when copying was the problem, but how to best fix it?

It turns out that simple physics is the answer. My wall wart is 12v; I was stepping this down to 3.3v. This is 9v, times 150mA worst case scenario, which is about 1.3W dissipated in a SOT-23 package. Yeah, that's not going to happen for long. (Honestly, what I am surprised about is that the 100mA constant current draw allowed things to run at all; that is still just under 1 watt, which is more than I should be able to get from the SOT-23. I guess having a nice large ground plane as a heat sink without using the thermal relief pattern on the pins helps keep things cool.

The correct solution would be to get a 5v wall wart, but I am lazy and cheap, so instead I just threw another larger regulator with a heat sink onto the power lines before they go into the board. This steps the 12v down to 5v, which is further stepped down to 3.3v with the SOT-23. I can now load samples all day long without any issues.