After intensive testing of the components (and fixing some shortcuts and other bugs) I finally soldered everything together. I was a bit nervous before powering it on, but... it works! I wrote a test program and was able to switch each LED on and off separately.
I found out that the dot matrix display is not bright enough at 5V because of the multiplexing rate of 1:16. I increased the supply voltage for the LED supply to 9V to have enough brightness.
To make this possible I had to add additional buffer ICs. Driving the rows directly with the 74595 shift registers does not work for supply voltages > 5V.The LEDs to display the seconds are a LOT brighter compared to the display modules. To dim the 60 LEDs I had to increase the appropriate row driving resistors quite a bit.
I like the look of the assembly, it is more or less like I imagined it. Mechanically it is sturdy enough. This was another lesson I learnt building this beast. I added a lot of upright wires that I glued in holes drilled in the base plate. Those upright wires serve as soldering points for horizontal wires.
The more of those "poles" I added, the better got the overall mechanical stability.
The next step is to complete the software and a video of the clock in action.