I found this LCD on Aliexpress and it's about the same width as the Teensy Thumb Board:

https://www.aliexpress.com/item/20PIN-SPI-COG-12864-LCD-Module-with-Chinese-Font-ST7567R-Drive-IC-3-3V-White-Backlight/32608104073.html

I ordered a couple and tried it out with u8g2 library. It mostly worked but needed a couple tweaks. Pull request merged now and everything is working nicely. The display has support for 4 color grayscale but u8g2 only supports black and white at the moment. Anyone know a free to use driver? Maybe I'll get around to writing one that supports grayscale.

Anyway, prototype faceplate integrated with the LCD is shown below. All the keys aren't mounted yet but the screen looks great. Way better than the ti-81.

Visibility with the back light off is great too. Pictured next to a 6" phone for size comparison.

Here it is with all the keys on.

Case backs are always the hardest to design. This is a test one with a good shape. Still need to add a compartment for 4 AA NiMH batteries. 

So far the Teensy 3.2 works but I plan on switching to the Dragonfly (https://www.tindie.com/products/TleraCorp/dragonfly-stm32l47696-development-board/) for the ultra low power modes and built in 16mb flash storage.