With the basic princible of the cake confirmed I set to work to put the matrix together. Since the spacing of the LEDs was defined by the stripe I placed five rows of six LEDs on a cardboard box I had lying around. The box provides more than enough space to hide the Arduino and a few other accessories.
A short while later everything was hooked up and looked like this:
So the "screen" is working. Which leads me to the question what kind of gaming could be done on a screen that bad.