I fixed some issues with the hardware:

Buttons

the buttons reacted very badly, so I replaced them with tactile switches which have spacers glued to them. The old buttons were screwed in, the new ones were secured in true hacker fashion with excessive amounts of hot glue.

Reverse voltage protection

The schottky diode that was used to protect the circuit against reverse voltage, which can easily appear with 9V-clips dropped too much voltage under load, especially when the LCD was on, but also when sampling the battery voltage after storing the buffer to EEPROM which resulted in curiously low readings. While it was not too much of an issue for the controller, the LCDs contrast constantly had to be adjusted so I replaced the simple diode with a small MOSFET in this configuration:

Now the voltage drop is negligible, which also increases battery lifetime.

Battery Voltage Measurement

The schematic shows a voltage divider feeding into the ADC. The voltage was measured against the internal bandgap. In the new firmware I used bigjosh2's method of measuring the bandgap against the supply voltage and removed the voltage divider and the capacitor which saved me 4µA of constantly wasted power, more than the entire circuit uses with the new firmware.