I have added a diode to every button. Now I can detect more than 2 simultaneous button presses.
The circuit works pretty well on the breadboard. The multiplexed LEDs are flickering, but I think I can fix this in software.
I also ordered the PCB today, it should be here in a month or so.