I was experiencing intermittent noise from the boost converter I used to run the system off 2 AA batteries. I tried different capacitors but I just couldn't get it 100% ironed out.
Unfortunately I no longer have a really good high frequency scope to check out what was actually going on. Maybe I should have sprung for a better quality boost board with post regulation or something, but I couldn't be bothered waiting for Ali Express.
In the end I decided to just go with a whopping 6 (yes, six) rechargeable AA batteries and feed that into Vin. That way the Arduino's on-board regulator takes care of voltage regulation, and it does a pretty decent job at 7.2V (much more efficient than at 9V). I haven't done the math but these should last me more than long enough.
The hardest part was extending the housing. But that is still the same ugly-ass MDF board and not hard to do. I just bolted an extension on the back with some left over IKEA screws I salvaged from an old piece of furniture. Looks ugly, weighs a ton bot no more intermittent noise throwing off my calibration and really easy to hook up.