As I wrote before, first tests have been done with the ItsyBitsy M4 bootloader, but eventually I want to have my own. Especially since that is required for the Make Code Arcade compatibility. Fortunately there are some very detailed instructions on how to configure and compile your own. There is a pretty ingenious configuration system embedded into the bootloader itself, which lets the code figure out automatically which pins to use for the display, buttons, etc.
So I have a rudimentary bootloader working, but I need to play with the different options a little bit more: