The display that I am using is an OLED with an SSD1351 controller on a breakout board from Adafruit (here's a link: https://www.adafruit.com/product/1673). Adafruit also supply libraries and example programs to use it with an Arduino, and with a few minor tweaks these will work with a Teensy.

The libraries provide a basic set of graphical functions for drawing lines, boxes, circles etc, and these are drawn directly into to the display. For example, to draw a filled box, the library routine sets the drawing position to the top left hand corner and draws the top horizontal row of pixels. It then repositions to the left hand side of the box again, one row below the top, and draws the next row and so on until the box is complete.

The problem with using this approach with games with a lot of animation is that it would cause a lot of flicker as things are drawn and redrawn in various parts of the screen. The traditional solution to this in games is to use double buffering, where one frame of animation is shown on the screen whilst a second frame in another part of memory is updated. Once this new frame is complete the two buffers are swapped and the one that was being displayed can be redrawn before the buffers are swapped again, completing the cycle. This way, the player will never see a partially redrawn frame.

The display that I am using does not have the facility of an off-screen buffer so I can't do this, but there is an alternative that I can use which is almost as good. What I will do is instead of drawing directly to the display's memory, I will construct the frame in the SoC (Teensy's) memory. Once the frame is complete I will transfer it in one go from the SoC to the display. This will reduce flickering as much as possible albeit at the cost of some memory.

The OLED is 128x96 pixels and each pixel's colour is stored as two bytes. This means that the memory required to store a whole frame is 128 x 96 x 2 = 24576 bytes or 24Kb. Another option would be to use a palletised system where, instead of storing the colour value of each pixel, we store an index into a palette table which in turn stores the actual colour. For example, using a byte for each pixel would give us 256 colours and the memory usage would be 128 x 96 = 12288 bytes or 12Kb for the frame buffer and 256 x 2 = 512 bytes for the palette.

At this point I have not decided which approach to go for, although implementing both would be very easy. The Teensy has 64Kb of RAM, so even using the full colour frame buffer would leave plenty of memory left for other things.