[ Edit: around one year later, a new VGA generation project has started, with better specs ! ]

Trying to have the hardware simple, and keeping in mind that 'large' SRAMs are easy to obtain, the choice was made to have just a single basic video mode.

That is, 640 x 480 pixels, with 256 colors per pixel. The pixel time for VGA is 40nSec.

The hardware needs assistance from software in order to operate correctly.

The pixel counter will increment every 80nS, so the video RAM will deliver two new pixel values every 80nS. The 8 bit color values go to the 'first pixel' and 'second pixel' registers. During the next 80nS, each of the outputs of these registers will be enabled for 40 nS.

When the pixel counter has reached a certain value, an interrupt will be given to the CPU. In the interrupt, the CPU will:

• obtain exact synchronization with the pixel counter
• when the pixel counter is exact 199, reset the pixel counter to zero, to obtain 32 uSec line duration
• start the horizontal sync signal
• obtain access to the video RAM, disabling the pixel counter and the updating of the two pixel color registers, and enabling the pixel(X) register. It can now, after setting the correct pixel address in the line and pixel registers, write new data to the video RAM. The line and pixel registers may be latches, so the data can be written in the same cycle together with the address. Several locations may be written, depending on the available time during blanking.
• set the line (Y) register to the starting point of data for the next line
• when the end of a frame has been reached, do functions for frame synchronisation
• stop the access to video RAM, and stop the horizontal sync signal
• end the interrupt

The sequence of actions might be a little bit different than listed here. 

The actions might be done by microcode, and the 16 bit processor can deliver 16 bit at a time, so this will be fast enough for most operations. If there are applications (games) that need higher speed, resolution could be dropped to 320 x 240 (TBD to be determined). [ edit: Sprites were added to the video system, see here]

Since the starting point of a line is under software control, it will be easy to do fast vertical scrolling. A clear screen will also go fast, because only a single cleared line has to be present, and all other lines can point to the same cleared line.