If the system is going to boot from a cart, like the systems of old, it's going to need to boot an operating system. I've started looking at some, and have a few options I'd like to start with. I'd develop a simple and flexible OS that game developers can customize and install their games into. It would help get developers on board if they didn't have to worry much about the OS and could just install their games and go. A final version, installed on the cart, would have the GUI and all extras stripped away, and load the game immediately upon boot. Ideally, the OS would be custom built for each game, but that might be way too much work, especially if the Pi 400 has plenty of system resources to run a lightweight OS and a game. Considering the Pi 4 can run a full OS and games just fine, that should be no problem at all.
So far, I found the following options:
It turns out there there is a Ubuntu LTS for the Raspberry Pi 400. It's the first OS I've looked at, and it looks pretty good so far. Uses the Snap store, so it would be easy to add things like the game controller program.
I'm leaning towards to Gentoo the most so far. From what I gathered, you build the entire OS yourself and can start from a blank slate and add only the things you need. I like that concept. This is my top choice so far. I could offer a basic OS with no optional software, then add instructions for developers to add things like USB controller support and network support.
3) Alpine Linux
This one also looks very promising. As with Gentoo, it's a very basic Linux OS, which you build on top of. The website sure looks more friendly than Gentoo, but that doesn't mean much. I have to research Alpine to see if it really is a good choice. Seeing as it's about 130MB installed, it's a good candidate. It's still huge compared to the 8KB carts used in the C64, but I'm focusing on getting things working immediately, not in a true retro fashion.
I'll be doing some testing of the above OS choices soon, then I'll work on getting a game working on the OS of choice. Once that is finished, I can work on getting a test hardware that will allow me to load code from the SPI bus on the GPIO connector. Finally, I can focus on getting GPIO boot working to boot a system from the GPIO SPI bus, and tie it all together.
I'm aware that it may seem silly to bother including an entire OS with a game, but it would eliminate all of the various problems that come with trying to work with a user's OS and installed software. The hardware will be a fixed variable, as it will be assumed to be the Pi4 or Pi 400, so why not eliminate the OS and software variables? The goal is a super reliable game console that will just work when you plug cartridges into it. For that, elimination of variables is crucial.