NOT THE ESP32. I don't have those and I don't want them. Yet.
I *do* have a couple ESP-12E boards. I don't have much (okay, any) experience with them yet, but I cant imagine an 80MHz almost-ARM-ish microcontroller would let its butt be kicked by something on the order of a 1st-gen Pentium CPU -- and I have indeed run Linux on exactly that (it was an obscure low-power version of Puppy Linux, for the record... I have one picture, archived somewhere).
I'd like to see if it's doable... video would be uhm interesting, but a pair of PS/2 ports for mouse & keyboard shouldn't be too much overhead. If it could work, a video coproc consisting of an ATMega328 running the TVOut library, plus a serial-drive character ROM (SPI works for me) would probably suffice... I just want to drive an 800x480 screen... 40 rows of 100 characters each (100x40 character resolution) works if you use 8x12 px characters (8 pixels/px wide, 12 high), which is a standard character size but not a standard screen size.
All I really want is to be able to drive Links2 in a bash session, with the built-in WiFi, plus keyboard mouse and screen. The ability to use a USB keyboard/mouse combo would be nice but not strictly speaking required... I'd really like to use this in a wrist PC I'm planning, if that's possible, but that would indeed require USBHID input support for keyboard/mouse, because the one I'm planning to use there is a gutted Bluetooth mobile phone keyboard with a "nub" mouse bodged on (consisting of the four-button 'nub' joystick from a Toshiba T3400CT* and two tact-switch inputs for the left/right "click" buttons). The screen for that unit, for the record, would be a composite-in LCD intended for automotive backup camera applications from eBay. I can power it from a power bank for uhm a very long time, or a real laptop battery if someone can throw in an SMBus driver somewhere (that would be awesome!)... ;)
I'll politely note that I am by nature NOT a programmer -- my best effort to date is a four-room text adventure written in MS QuickBASIC, that I'm currently rewriting when I get in the mood, partly to exercise my Amstrad PPC640 (lol) that I got recently and partly because the original version was so awful as to essentially avoid the use of a text input parser (don't ask, you *really* don't want to know!). While I'm *technically* open to learning, my last two concentrated efforts at learning C++ ended spectacularly badly, so I hope you will forgive me for being rather hesitant!
That said, I'm more than happy to help test... ;)
* I feel the same way most folks here do about that stuff -- that's treading on hallowed ground because old PC / computer history, but I have THREE essentially complete but not-entirely-functional-at-best parts machines in a cardboard Banker's Box in addition to the one that works ;) so I think I can take the mouse 'nub' part out of one of the busted ones for this...!