I love coin cells, but I'm not sure they will cut it if we need to do something more current intensive (like write to SD cards).  I shopped around and found a small square LiPo battery that will fit the footprint.  With only 150 mAh, it is not going to do a whole lot for a whole while, but it is a start.

The charging circuit I borrowed from Adafruit's Trinket Backpack.  I put connectors on it for both JST PH and SH, the former seeming to be the most popular for LiPo batteries, the latter being what the battery I selected came with.   The unpopulated PH header can also act as test points or solder connections.

I released my first bit of magic smoke on this project by hooking the battery in without checking the leads match the markings on the board. The back current immediately fried the MCP73831 and burnt my finger, all in the process of about two seconds.  Hum, maybe a protection diode or PTC? I've learned that lesson the hard way more than once.

I am having an issue with the red charge light being dimly lit after disconnecting the USB power. I believe the battery diode D4 on the 32U4 board is leaking stray current back into the circuit. More on that latter.  I'm also wondering why the heck I didn't think to put a power switch on this board (or the 32U4 for that matter).  The headers and location are also not sorted yet.  I'm either going to use through-header so it will go on top of the 32U4 board, or piggy-back the 32U4 on top of it in a more permanent arrangement.