As predicted, I came back to this project in December. The github code has been updated a few times over the last couple weeks, and here's what's happened.

Having left this on the shelf for so long, I'd forgotten where I'd left everything. So I started with "what do I want the most?"

Answer: hard drive support.

Back in the 80s, while I was in high school, I worked at a software store. (Babbage's, for any of those that might remember it.) While I was there the ProDOS version of Merlin (a 6502 assembler) was released. I bought myself a copy and started writing things. I noodled around with ProDOS - both the external command interface and the internal workings of its disk system. And I have images of a couple of my development floppies.

I'd love to consolidate all of that to a single hard drive image.

So, looking around for ProDOS hard drive support, I stumbled across the AppleWin implementation. One of its authors wrote a simple driver to emulate a hard drive card that ProDOS will use. So I pulled their driver and wrote code in AiiE to interface with it. All told, it took about 6 hours to get this working (3 hours to write the code, and 3 hours of constantly retracing my steps to find the typo while taking cold medicine, ugh).

Well, that was easy! What's next?

I guess I'd like to boot GEOS. Not for any particular reason other than I had run the first version of GEOS for the Apple //e back in 1988. The disks won't boot, though; they give me various system errors. Why, exactly? Well, it's all in the disk drive emulation.

The Disk ][ was a favorite research topic of mine back around 1988; I was fascinated by the encoding of data on the disk in particular. Which makes all the work on the disk emulation so much more enjoyable! Instead of being in the Apple //e and trying to read and write nibbles of disk data, I'm in a virtual Disk ][ trying to send Aiie data that /looks/ like it came from a floppy controller!

My first pass of the floppy controller code was a mishmash of approaches. I looked at how other people had implemented theirs and cobbled together something that looked like it worked. Which lead to code bits like this:

// FIXME: with this shortcut here, disk access speeds up ridiculously.
// Is this breaking anything?
  return ( (readOrWriteByte() & 0x7F) |
            (isWriteProtected() ? 0x80 : 0x00) );

Now, that piece of code totally doesn't belong there. I had it jammed in the handler for setting a disk drive to read mode. I think I'd accidentally put it in there while writing the code the first time around, noticed the performance improvement, and left it there with the comment for future me to puzzle out.

Rather than starting on this end of the thread, I figured I'd gut the rest of the disk implementation and see what could be cleaned up. First up was the stepper motor for the disk head: a simple 4-phase stepper, where each of the four phases can be energized and de-energized by accessing 8 different bytes of memory space. The drive actually steps in half-track increments, which some disks used as part of their copy-protection schemes; but that's not really useful to me, so I'm only supporting full tracks (as do all of the emulators I've looked at so far).

My first attempt kept track of the four phases and which was last energized; and then divined the direction the head was moving. If we went past "trackPos 68" (which is to say track 35, because 68/2 is 34, and tracks are 0-based) then the drive was bound at 68.

I decided to rewrite it. The first rewrite kept distinct track of the four magnets, so it could tell if something odd was happening ("why are all of these on?"). But again, that's not really useful to me, and I kept confusing myself about the track positions. So the second rewrite keeps track of the current half-track ("curHalfTrack") and only pays attention when phases are energized. It assumes that the de-energizing is being done properly. Then a two-dimensional array is consulted to see how far to move the head, and in which direction, based on the previous stepper and the current stepper energized magnets. The proper bound (at least, I'm going to say it's the proper bound) for the end of the disk is 35 * 2 - 1 = 69 half-tracks, rather than 68; and with that in place, I still got nothin'. GEOS refuses to boot.

The hint of what was wrong was right in front of me, though. Just before it triggered an error, the second disk drive lit up for a moment. Presumably it's probing to see what's in the second drive... and then it dies. Which is all because the track is cached, and my caching was buggy.

Why do I cache the track? I'm glad you asked! This gets in to that whole disk encoding thing.

The Disk ][ couldn't reliably read more than two consecutive zeros from a diskette. That means you can't just store data on it; you have to transform the data in to a stream that never contains more than two consecutive 0 bits; and then write that to the disk. So while the diskette might have a stream of $00 $00 $00 $00 bytes when it gets through the Apple RWTS disk subsystem, it's actually stored on disk as $96 $96 $96 $96 $96 $96.

And more than that: there's no physical start or end of a sector on the diskette. To read "sector 0" the head moves to track 0, and then the computer starts reading. It looks for magical headers and footers that encapsulate meta-information - specifically, what track and sector you're on. "$D5 $AA $96" <volume ID, track, sector, checksum> "$DE $AA $EB".  Then "$D5 $AA $AD" as a prolog for the data section of this sector, followed by those encoded values above; finally terminated with "$DE $AA $EB". If the header said it was, for example, sector 2 then we keep reading; we didn't find sector 0 yet! The disk spins and we find whatever's next on the diskette.

Of course you need to be careful that you don't pack the data in too tightly, or the CPU will miss some of it while it's processing. And, since drive speeds vary slightly from physical drive to drive, you could format a disk in one fast drive and then try to write a sector in a slower drive, overwriting the next sector header accidentally. So there are also junk ("gap") bytes in the stream.

And that's how a simple 256-byte sector turns in to a variably-sized mess of goop.

Looking at it from the hardware perspective: when Aiie is presenting Disk ][ data to the OS, it's being asked for one byte at a time. Which means that Aiie needs to know which encoded byte it's on. Since the sectors are in a specific order on the disk, it's easiest to prepare a whole track and then just present bytes out of that buffer.

Now you know why there's a whole track buffered. No, it's not necessary; we could do a sector at a time, and add some clever counters for gap bytes and whatnot. Maybe in the next iteration; it could save some memory. But I'm not dying for it just yet. I made various adjustments to the gap handling, though, mostly because I was troubleshooting and had no idea what I was looking for; and I removed that hacky hack shortcut code that doesn't belong there, because I was cleaning up while debugging.

Back to the problem: when you switch drives, I wasn't properly clearing the cache. We wound up switching to drive 2, which got some garbage cache because there's no disk in it; and then switching back to drive 1, where these garbage was still hanging around, but the cache flags said that drive 1's cache was still valid for the track it was on. Fixed that up, and GEOS tries to boot.

I say "tries to" because it really seemed to just hang.

So I popped back in a virtual disk image I'm very familiar with: Ali Baba, the game that inspired the name of Aiie. While loading it shows the track and sector that it's reading from onscreen. It's a great way to quickly assess how the disk drive is performing. And the answer is that it was performing *very* badly. It made progress but very slowly.

I spent a few hours reading through the code and verifying that the returned bytes were correct. And then I spent about an hour reading parts of "Understanding the Apple //e", Jim Sather's amazingly complete book - one of three serious references I turn to when I feel like I'm missing something. And there it was, hidden in chapter 9 - hidden well enough that I can't find it again as I look right now. Somewhere in the disk nibblization piece is a comment about how a byte is filled - how the disk controller might return something without the high bit set because it's still in the process of reading the data from the diskette. Along with comments about how the logic sequencer finds valid data, this got me thinking: am I returning data to the Apple too quickly?

Well, that's easy to test. I threw in a flag that would only return a valid byte on every other read attempt. And sure enough - as soon as I did that, the reads were amazingly fast. Faster than they were with the hack above.

Why the heck does the machine try to read that quickly, if it can't process that quickly? No idea. Probably something to do with the nature of speed differences in Disk ][ drives. But now, finally, I can see GEOS boot up!

Of course, I can't really use it for anything because it needs a mouse...