The folks at VLSI tech support are great. [Panu] and [Kalle] are a big help with HW and SW. They even sent a full Arduino sketch to configure the VS23S010. It does configure the chip corretlyf̶o̶r̶ M̶C̶G̶A̶ 3̶2̶0̶x̶2̶0̶0̶ w̶h̶i̶c̶h̶ u̶n̶f̶o̶r̶t̶u̶n̶a̶t̶e̶l̶y̶ m̶y̶ T̶V̶ s̶e̶t̶ d̶o̶e̶s̶ n̶o̶t̶ f̶u̶l̶l̶y̶ ̶u̶n̶d̶e̶r̶s̶t̶a̶n̶d̶.̶ B̶u̶t̶ i̶t̶ i̶s̶ c̶l̶e̶a̶r̶l̶y̶ v̶i̶s̶i̶b̶l̶e̶ t̶h̶a̶t̶ t̶h̶e̶ v̶i̶d̶e̶o̶ o̶u̶t̶p̶u̶t̶ w̶o̶r̶k̶s̶.̶ N̶o̶w̶ ̶i̶t̶ i̶s̶ j̶u̶s̶t̶ a̶ m̶a̶t̶t̶e̶r̶ o̶f̶ t̶h̶e̶ r̶i̶g̶h̶t̶ c̶o̶n̶f̶i̶g̶u̶r̶a̶t̶i̶o̶n̶ o̶r̶ t̶h̶e̶ r̶i̶g̶h̶t̶ v̶i̶d̶e̶o̶ ̶m̶o̶n̶i̶t̶o̶r̶.
With the help of VLSI tech support I had my facepalm moment for the week. Just from this picture they could tell me, that I had the crystal for PAL video mounted and not for NTSC (displeasure for some many different standards for everything). Fixing that and getting the video out termination right got me this result. YAY. Now it's only 1s and 0s and a lot of typing to program a graphics engine.