After discussing this issue with several people who want to build their own matrix, I decided to sum up the main points in a project log.
In the newer versions of the Raspberry Pi with integrated Bluetooth like Pi3B/B+, ZeroW and the new 3A+, the hardware UART is used for the Bluetooth module. The second UART, which is connected to the extension header, is a software UART implementation. This second UART seems to be limited to 250kBaud. You can read more details on this website:
This project uses 500kBaud for transferring the LED data between Pi and Arduino. I am using an old, non-Bluetooth Pi A+. The only thing, I have changed, was adding this line at the end of the /boot/config.txt :
There is an option to change the UART configuration on newer Pis by connecting the hardware UART to the extension header and the software UART to Bluetooth:
I didn't have time to test this, but it looks quite straight forward. Also if there are other solutions, I will update this post.
Of course, it is also possible to reduce display update timings to reduce the UART speed to 250k.