I wanted to post a quick update to this. The project is still ongoing despite my silence here! I am currently working on a revision 2 PCB that simplifies things and removes several parts from the BOM, to hopefully make it cheaper and easier to assemble. There were also a few electronic issues with the first PCB, which is why I haven't posted the schematic or board files yet -- I don't want anyone to make the same mistakes I did!
In the meantime, I got LittlevGL talking to the LCD, and VGM playback is basically perfect at this point. I've also been slowly putting more code into the git repository, but my focus right now is getting this next board revision finished. Once that's done and tested, I can work on getting a UI written.
Here's a video from last night, testing the use of the ESP32 for clock generation rather than separate LTC6904 chips. This is just hacked onto my current prototype board to validate my design before I order PCBs. I hope to have the new design fabbed and tested by the end of this month.