I have been spending some time with a timing rendering Application (WaveDrom).
Here is a basic idea of the timing of the pixel renderer -
It is not very accurate as I can't seem to get it to do what I want yet but it does give a basic idea of where I am heading.
It looks like the memory access time will be over 80nS which is great as the FLASH is 70nS.
I am trying to separate the dot rendering part of the VHDL from the timing part so that I can use jumpers to select the output format: NTSC, PAL or VGA.
This approach will waste some registers for timing but there should be more than enough in the chip I am using.