The Weather STAR 4000: A Journey of reverse engineering the hardware to an iconic machine of the 1980s/90s.
To make the experience fit your profile, pick a username and tell us what interests you.
So now that we have basic framebuffer commands out of the way, Next thing we need to do is figure out the Memory-to-Pixel layout of the framebuffer.
I will spare you the experimentation process to get to where I have, plus its been a couple of years so I don't remember all the details anyways.
But the jest of it is this:
So now that we have this information, the next thing to do is figure out how the colors work.
So basically, each pixel is 1 byte. Since it is a 256-color system, this makes sense. However, 0 is reserved as the key color. So if there is a video signal coming into the unit, 0 gets replaced with 1 pixel of video information instead of a color.
All the other index values of 1 to 255 are physical colors. This color information is sent from the framebuffer control CPU to the RAMDAC.
There is a default palette as explained earlier, but you can also send a custom palette. We arnt quite there yet.
For now, lets see if we can get something into the framebuffer, regardless what it is. Before we can figure out exact details on some of the framebuffer control commands, we need have something on screen to use as a base of reference for experimentation.
So we have already played with colors, lines, etc... But we need to move on from that. We need to draw text and images.
But alas, all this stuff has to be recreated from scratch because none of it exists anymore. Except: the Font. For now, I was able to extract the font from the Weather STAR Jr. The font is almost 100% identical with the exception of the lower case W. it is not the same. So its from the same family, but not the same exact flavor. I am not a font/typeface expert so i dont know what all the little differences mean. but, hey. at least now we have something to start with.
There are public recreations of the WS4000 fonts, but they are far from accurate, and, they are true-type and the modern graphics suites have issues properly rendering fonts at low resolutions for bitmap use. so I abandoned that idea. Looks like garbage.
First of all we need to get the font in a format that the graphics card is going to work with. For testing, it was simple enough to just to convert the font into a simple bitmap with different values representing the different colors to be displayed on screen. Yes, this takes up a lot of memory. because now you are using 1 byte per pixel in the font table. But its good for testing.
So now I needed to write a simple program to shove data out of the UART to the Arduino for sending images/fonts.
Once I got a rough skeleton setup something like this:
'Load 18x36 Font into Pixel Memory. Dim img As Image = fx.LoadImage(File.DirAssets, "font2.gif") Dim buffer() As Byte = GetPixels(img) Dim width As Int = img.Width Dim height As Int = img.Height For x = 0 To width - 1 For y = 0 To height - 1 Dim i As Int = y * width * 4 + x * 4 Dim b As Int = Bit.And(0xFF, buffer(i)) Dim g As Int = Bit.And(0xFF, buffer(i + 1)) Dim r As Int = Bit.And(0xFF, buffer(i + 2)) Dim a As Int = Bit.And(0xFF, buffer(i + 3)) If A = 255 And R = 127 And g = 127 And b = 127 Then 'This is a Gray color (We key this out with whatever is in the background, So save as a 0) PixelArray(x, y) = 0 else if A = 255 And R = 0 And b = 0 And G = 0 Then 'We have a black color. So we make this black, or 1. PixelArray(x, y) = 1 Else if A = 255 And R = 255 And G = 255 And B = 255 Then 'We have a white color. So this is the typeface color. or, 2. PixelArray(x, y) = 2 End If Next Next 'Load 18x18 Font into Pixel Memory. Dim img As Image = fx.LoadImage(File.DirAssets, "font3.gif") Dim buffer() As Byte = GetPixels(img) Dim width As Int = img.Width Dim height As Int = img.Height For x = 0 To width - 1 For y = 0 To height - 1 Dim i As Int = y * width * 4 + x * 4 Dim b As Int = Bit.And(0xFF, buffer(i)) Dim g As Int = Bit.And(0xFF, buffer(i + 1)) Dim r As Int =...Read more »
So now we have a basic way of hacking/testing the graphics card independently from everything else, Now its time to experiment with the Framebuffer control...
To do this, we need to study the 8031 ROM in much finer detail to figure out what it all does.
So I stared at this ROM and picked it apart for hours on end, over the span of multiple days.
I toyed around with one command at a time just to try and get a handle on whats going on.
So in ROM, there is a Compare/Jump table that checks for a byte waiting in the FIFO. Once it sees a byte, it reads it and then does the "select case" to figure out what to do next based on the value of the byte sent. This is the command.
This is the command table that I have figured out and made comments on:
Now this ROM is commented out already based on what I knew, its still not 100% complete but its good enough for me to be able to do what I need to do. Commenting was easy enough to do once I figured out the address map of the 8031 on the graphics card, where everything resides. (those details can be found in the MAME driver for now, i may post that info later, but its not relevant to this article).
So at this point, its pretty much just throwing commands at it with parameters and see what sticks to the wall, or what explodes. One of my favorite cheesy movies from the mid 90s "Hackers", the quote from Joey basically. "Its like choice". Throw commands at it, and see what happens.
It isn't going to spit cash out in the middle of the street in bumsville Idaho though. Sorry. :-)
In order to do this, I had to setup my Arduino code so I could simply send serial hex data straight to the FIFO. one byte at a time, this makes it easier to experiment and just throw data at the framebuffer control to see what happens.
Now, since I never kept a blog in real-time as I was experimenting with this thing, I don't remember detail for detail what I did and what each image actually was. But I will post the images and video clips below for posterity of my primitive experiments.
This is when i drew some lines into framebuffer, and then I was able to send a control command that switches the resolution.
This thing has a default color palette built into ROM that gets loaded on reset, so all these experiments are showing off the default palette.
Here is a link to the Photos Album of the images and video clips of my experimentation process:
Basically, I determine the crawl, roll, page switching, etc commands but not yet their math or specifics. Without drawing things into framebuffer, I cant really figure out the fine details just quite yet.
That's in the next installment. Stay Tuned....
So now comes the real hacking part.
We have the basic Architecture of the graphics card now, but we have Zero idea how this card works. or even how to speak with this card, or do anything with it yet.
At the time, I did not understand 68K ASM or C that well yet, so I needed to figure out an easier way to probe this card and be able to do things.
Enter Arduino (once again).
I decided to remove the MC68010 CPU, and then use an Arduino Mega in its place to "emulate" the 68K bus cycles, this would allow me to write a much higher level program on the PC side and communicate over USB to the Arduino and be able to send commands and access addresses on the card.
This should help me figure out how to control the framebuffer, how the framebuffer is laid out, etc...
Wiring up a PicoATX Power supply. (This helps me run this card separately from the unit)
Powering it up on the bench to make sure things dont explode. Magic smoke test:
And... yeaahhhh.. :-/
Arduino is all mounted up and ready to go.
Now, I still have to write a test program that runs on the Arduino to keep the watchdog reset on this graphics board. I know once I do that, my bus emulation program is working :-)
The green LEDs on this board are the "health" indicators. They will remain green as long as the watchdog is reset.
And Success! the 68K's health light is on, and my program is running.
Now I have a good setup for probing the graphics card to learn about how it works, and its operations.
Next thing we need to do is start writing things in the Framebuffer memory location and see what shows up on screen. Also, we need to study the 8051 ROM in order to grasp how the framebuffer works.
Turns out, it took experimenting with the commands plus studying the ROM to figure out how everything works.
Aaaaand there we go! I simply drew some stuff in the framebuffer memory, and displayed its initial contents. That "checkerboard" pattern is the initial state of DRAM when first powered on, and nothing being written yet. However it still doesn't look right. instead of a clean color, its a broken alternating line. Turned out, one of my wires was loose on the Arduino. Not surprising with that ratsnest.
I am posting this completely out of order, but it's something that needs to be said. I have been seeing some comments pop up about MAME and how there is co-development on reverse engineering the 4000 going on.
This isn't true. Well, partially isn't true. I did get the ball rolling though. Getting it working in MAME vs Reverse Engineering the hardware are two very different things here.
I got a MAMEdev involved for what I thought was to help me develop software and be able to debug/trace it without trying to do something wonky on the original machine. But I was mistaken here, I actually contacted the MAMEdev to see if it was possible to make a 4000 emulation inside MAME, so I could attempt to step-through and trace the ORIGINAL ROMs. I was still very new to 68K. Kinda still am, but I have the basics down.
It was easier at the time for me to figure out what each instruction was doing in the ROM to get a better understanding of what is going on, but eventually I decided against persuing that avenue for now. Maybe I would circle back around to it later.
Plus, I still didn't know all the fine details yet to finish up the emulation driver for MAME. I still don't "entirely" know how the framebuffer works, but I know what all the commands do now. (I will get into these later).
I would actually like to circle back around eventually and get MAME working. Someday..... my projects list is getting more full by the minute :-/
I know this may sound like I am feeding the trolls, but it must be said. Here is my conversation with a MAME developer on the subject, including me sending him all my notes and schematics so he could write the MAME driver:
Sorry for all the drivel, but I had to clarify the data here, and the real information as to what went on. Why? I suppose to try and prevent as much mis-information as possible.
For accuracy purposes. I stand by my decision for letting MAME in on this as well, I think it will be a useful tool to messing with the 4000's ecosystem.
Now, back to our regularly scheduled programming...
With the architecture of the main system out of the way, my biggest focus at this point was the Graphics card itself. and figuring it out.
Since as I have stated before, if we cant figure out the graphics card, in a graphics machine, we are hosed before we even start.
So... Once again we perform what we did with the CPU card, analyze the PALs to figure out the memory addressing of the graphics card.
This is basically the memory map of the Graphics board. If you noticed from the previous articles, the main CPU does not have access to, nor can directly access the Graphics card's memory space.
The graphics card has its own memory space. So the only way to communicate between the main CPU and graphics CPU is through the doorbell interrupt to the graphics card, and shared memory space on the VME Bus. (Main CPU Bus).
The other thing to keep in mind here is how the Graphics card accesses the main system bus. it does so through a Write Only paging register, and a 128K paging buffer. Since the Paging register is write-only, it cannot be read or its state saved! Therefore, extra precaution has to be taken when using the system bus in a multitasking operating system. Ideally, a Mutex.
Interrupts are fairly simple on this card. Nothing special, there is a Vertical interrupt, and the Doorbell interrupt from the main CPU.
If you look at the memory map, this card is manually vectored. So you would load the IVEC register with the appropriate interrupt vector number, and then trigger the IRQ back to the main CPU card. with any 68K system, you could have multiple ISRs to handle different things. This is necessary for inter-process communication between the two cards.
The Framebuffer RAM is standard DRAM. This RAM's access is interleaved between the pixel clock/rasterization logic, and the main CPU. So the CPU access to this RAM is "vampire" or bottlenecked. One of the biggest bottlenecks in the WS4000 system. the VME Bus paged-access is another one.
There is another CPU on the graphics card, and that is the intel 8031 (MCS51) that sits alongside the 68K. This is the framebuffer control CPU, and the communication between the 68K and this CPU is one-way only. through a FIFO. This too has to be under a Mutex lock. Otherwise, you can/will crash the CPU if you malform the command.
Stay tuned for more graphics card details and first signs of life!
Now that we have all the PAL logic dumped in a state where we can read it to figure out what's going on, we can proceed on figuring out the architecture.
However, there are a couple registered PALs that I don't have the logic for, and there are a few PALs I didn't bother getting the logic to, as it wasn't relevant to the mission at hand. However it might not be a bad idea getting it anyways in case the PROM inside a PAL ever bit-rots. Any cases of this happening? Please comment below. (I don't know).
July-August of 2019, I focused solely on the graphics card and its logic so I could further probe the card to figure it out. I will get to this soon, but instead of going in chronological order, lets go in order of architecture.
So after studying the logic and design of the CPU card circuit, we can make these conclusions:
the CPU card is also the bus master. (of course). However, any card on the bus can request access to the bus, and the CPU card will grant it, allowing other peripheral cards to gain access to the bus. Usually this is done for DMA reasons. The cool thing? if the CPU card grants bus access, the CPU can still access its own 2MB of RAM at $0, as well as the ROM, etc... and continue to execute code as long as it doesn't need the rest of the bus! kinda neat.
However, only the graphics card can be a bus-master in the system currently. All the other cards such as the I/O, and Data/Audio are slave-only devices that require communication through vectored interrupts.
We learned that this is a VMEBus-based system, Well... a variant of it. But the primary Address/Data bus is pinned the same as VME. However IRQ lines have been routed a different way as the decode logic is in the cards themselves instead of across the bus. so the IPL lines are not exposed across the bus. Just the IRQs themselves. They instead put a voltage on the 3 IPL pins, no longer making the system 100% VME Compliant. So as I mentioned earlier, don't go plugging in VME cards!
Now that was fun! (Actually, no. it took many hours)
Once all of the dumps of all the PALs are obtained, thats when things get interesting. We have to turn those dumps into logic diagrams.
At this point, I began searching around on the internet to see if there were any projects that could decode the "dumps" back into the logic that I needed to help demystify the architecture of the system. Mainly, the memory map.
I decided to go with this project here:
In order to get the logic out of the dump, I have to convert it into a format that this program/script can understand. That's where I had a friend of mine make a utility to do this in C# as I was busy with other things and he wanted to help contribute.
There were a few bugs that needed worked out, But.... the tool work pretty well. you could specify the file, number of inputs, and then number of outputs. However, the tool still has a few bugs where it doesn't put a comma where its needed, and it orders the signal names wrong. But that's not a big deal, I just have to remember that I have to swap those around when it generates the CSV file.
Eventually, you end up with something like this:
This represents the CSV file that the ReGAL python scripts require to be able to generate the verilog output.
But, as you can see, we still have a layer of obfuscation that we need to work through. And that is the signal names. only thing it knows is i0-ix, and Q0, through Qx.
What we have to do at this point is basically rename the signals in the csv file to ones that actually make sense. And the only way to do this is by looking at the schematic, and attempt to make educated guesses based on the circuit design, and how the PAL is used. Takes pretty good critical thinking skills here.
Otherwise if you don't perform this step, you end up with this after you run the script:
Not very helpful, unless you wanna do this the hard way, of course. :-)
So, back to the schematic to see what we can decipher. There are plenty of things here we can take out of the equation that are known constants. For example, the CPU itself:
We know what all those signals are on the 68K CPU. so we can define all those net values.
For the sake of this discussion, Lets pick U48 which is an AMPAL16L8. Lets see if any of these signals match up into what is connected into U48:
Here we go. the majority of the inputs are connected into the CPU Bus. So we can go ahead and name those nets in the CSV file going into ReGAL. we can also see the R/W line connected, as well as the Address Strobe and some address lines. So this chip does address decoding. We can safely make this assumption.
Next thing is to look at what other chips are hooked into that PAL, especially on the outputs:
Ah! we see that the ROMs are connected into this IC. So we know for certain that this is an address decoding PAL, which will be critical to figure out for deciphering the architecture.
Wash... Rinse... Repeat. Given known facts that its VMEBus, and where else traces are connected, you can figure it out. Eventually, I end up with this:
Perfect! One chip down. So, we can go ahead and rename our signals in the CSV file for processing.
That's better! At least this will make the logic much easier to understand. Now, it is time to run it through the ReGAL script.
Bingo! Now, isn't that easier to read and figure out? I think so. Now, it is just a matter of going through each and every PAL and do these steps.
Note: There will be signal names that you just don't know what they are doing. And I still have that. But it is what it is. We can make educated best guesses here and still get what we need.
Eventually it starts to look more complete: (prepare to spend many more hours, as I did)
So the next battle in the journey is figuring out how to dump the PALs. So I did alot of googling and research.
With combinational-only PALs, you can easily dump them like a ROM. but you have to know the inputs/vs/outputs. Luckily by looking at the schematic, its pretty straight forward to do so.
But just dumping them as PROMs are only half the battle. You still have to figure out some way to reverse it back into logic.
Every little PAL project I ran into, had its own way and method of reading chips. Plus, I didn't have any of the hardware that was necessary to dump a PAL to run through one of these programs.
So, what I did was take a blend. of my own tools, and online tools.
First thing is first though, We need to dump the PALs like a ROM. So, I used an Arduino Nano on a breadboard, I did not have a method to dump it with an external programmer. So I improvised.
Ok, That part is done. But now, I have to write a program. Now, dont laugh! I really at this point did not know how to write code in C just yet. But I have written code in VB/BASIC all my life, and thats how I used these little microcontrollers.
Working with this machine actually taught me C. Surprisingly.
Anyways, Time to write a program!
Simple! well not really... i had to take some time to think, and then calculate how many iterations to read based on the number of inputs. i literally did this the hard way due to the reduced number of I/O on the Nano. I would read a block of bytes, then change the upper bits. i had 2 to 4 bits that the MCU didnt control. so i effectively made 16 dumps with 4 bits. Yeah, i could have used an arduino Mega and dumped the whole thing in one wack.
Then you end up with the dump, something like this:
That part is over. I dumped the rest of the chips and then stored the dumps for further processing.
The fun part is yet to come. Deciphering the PAL logic!
Remember the discussion I had earlier? Yeah, that's right. those pesky FPGAs and PALs.
So once we have the schematic drawn out, I can begin to stand back and take a big picture look at this thing and begin to study its design and architecture. However, you see things like this:
Basically, that doesn't really tell you a whole lot. Seems a lot of the logic is locked up inside those PALs. As seen above.
The one I am most afraid of, is yes. the FPGAs and PALs associated therein:
We take a look at the CPU card, and more of the same:
Especially in the Bus/RAM sections:
I suppose using PALs and other logic arrays were not that uncommon in the late 80s, probably to keep cost down or board space usage down. But then again, looking at how this unit was engineered? I don't think cost was an object.
Regardless, There isn't any way around other than figuring out a method to reverse the logic out of the PALs. Luckily? the majority of the PALs are L style, or combinational logic.
There are a couple registered logic PALs on the graphics card, But... those still remain a mystery to this day. They appear to be gating for the pixel clock and a frequency divider for the PLL.
Next step..... figure out those PALs.
So eventually I had to build my netlist on the graphics card, the CPU card, and all the other cards.
By the end of this, I was literally ready to go jump off a bridge. I had soaked up months and months of time into all of this. And this excludes work hours of course as I still have my day job.
I ended up doing the graphics card first, then working out reverse engineering it first. Once I had the graphics card done, I went back and did the rest a month or two later.
Eventually its all worth it in the end as the journey continues. It is a necessary evil if you wish to develop on a bespoke system that has ZERO documentation anywhere. (outside TWCs own archives).
After all that work, you end up with the rewards:
And by the time I got to the data card, I said screw it. I was done! I intended on replacing the data card with my own design, but we will get to that later. I documented just enough so I could figure out what to do with it.
Whew..... Well that's done! or... is it?
Become a member to follow this project and never miss any updates