So I did some more playing around with the onboard regulator. The issues I was running into earlier seem to be fixed by using a higher quality wall wart. I had been using a cheap cell phone usb wall wart. Turns out it sucked. I looked at it and the ripple on it was huge. A better wart fixed it. 

This leads to a question though. The parts on my original board cost a couple pennies and clean up the power. A higher quality power supply costs maybe $15. Is an easier board to assemble worth the higher price? I think probably not.