Project Logs

Collapse

• The SNES multiplier is free. Ternary still won by 2x.

  Scott Boudreaux • 08/19/2026 at 15:02 • 0 comments

  The Super Nintendo is the first machine in this project with a fast signed hardware multiply. The PPU's Mode 7 unit takes a 16x8 operand pair and hands back a signed product, immediately, for free. The 65816 has its own multiplier at $4202/$4203.

  So I made a prediction before writing any of the engine: this is where int8 finally beats ternary. Every earlier port in this family — Game Boy Color, Genesis, NES — runs ternary weights, and the reason I had been giving was that those CPUs have no multiplier worth using. The SNES does. It should flip.

  It did not flip. Ternary won by 2.02x, and the reason has nothing to do with multipliers.

  MEASURE THE PRIMITIVES BEFORE YOU WRITE THE ENGINE

  The one methodological rule this project runs on is that you build a validated instrument first and write the interesting code second. So before there was a transformer, there were seven multiply-accumulate primitives, each one separately proved to compute the correct sum, each one measured on the same instrument.

  Wall master clocks per multiply-accumulate, 21.477 MHz master clock, SlowROM:

  software 8x8 shift-add — 1503.9 — 13.8x

  quarter-square tables — 570.9 — 5.2x

  PPU Mode 7, textbook form — 333.8 — 3.1x

  DSP-1, bus transfer only — 323.5 and up — 3.0x

  CPU multiply $4202/$4203 — 317.3 — 2.9x

  PPU Mode 7, tuned — 220.5 — 2.0x

  ternary sign-separated gather — 109.2 — 1.00x

  The tuned Mode 7 arm is the honest best case for int8, and it is still twice the cost of ternary.

  THE MULTIPLY WAS NEVER THE VARIABLE

  Here is the part that made me rewrite the explanation I had been giving for a year.

  The Mode 7 multiply is genuinely free. It is not cheap, it is not fast, it is zero. Every single one of those 220 cycles in the best int8 arm is data movement — getting operands to the unit and getting the product back.

  Which means the rule was never about arithmetic. Ternary touches three memory locations per accumulate. int8 touches six.

  Ternary wins wherever the machine charges you per operand, and its advantage decays as the machine gets better at moving them. That is one mechanism instead of five per-platform stories, and it makes a prediction I can check:

  SNES 5A22, SlowROM — ternary 2.02x ahead

  SNES 5A22, FastROM — ternary 2.07x ahead

  SNES SuperFX GSU — ternary 1.27x ahead

  N64 RSP vector unit — inverts, int8 wins

  Monotone, and it flips exactly where you would expect it to flip: on a vector unit that fetches eight operands in one go the per-operand tax disappears, and int8 takes the lead. The sibling N64 port in this project measures that inversion directly.

  I like this result more than I would have liked being right.

  THE INSTRUMENT LIED TWICE, AND ONE OF THE LIES AGREED WITH ME

  This is the part I want to be loud about, because it is the part that would have quietly ruined the whole port.

  The measurement works by latching the PPU's H/V counters at $2137, reading $213C and $213D, running the work, and latching again. Resolution is four master clocks. Linearity holds to a worst residual of 0.056%.

  Lie one: the V-counter wrapped. With a measurement window longer than a frame the counter rolls over and the arithmetic silently reports a much smaller number. The Mode 7 arm read 46.0 cycles per MAC instead of 220.5 — wrong by 4.8x, entirely plausible on its face, and it would have confirmed my prediction. int8 at 46 cycles beats ternary at 109 comfortably. I would have published "the SNES is where int8 wins", and it would have been wrong, and it would have been perfectly reproducible. The fix is sizing every window under about 140 scanlines with a vblank guard on RDNMI.

  Lie two: the instrument was right and I was wrong. An lda abs,y measurement disagreed with my hand-derived cycle count after the refresh correction. I assumed the instrument was at fault. It was the derivation: with 16-bit index registers the 65816 always takes the extra internal cycle on absolute-indexed addressing, not only when the index crosses a page. The rule I had in my head was the 6502's.

  And a third one, which is not a timing...

  Read more »

• A three-token test will pass a broken model. Use sixteen.

  Scott Boudreaux • 08/14/2026 at 10:37 • 0 comments

  This came out of the Game Boy Color work and it applies to every autoregressive port on this project, so it gets its own entry.

  The obvious way to check that an optimization did not break the model is to generate a few tokens and compare them against a reference. Three tokens is fast, it fits in a bench ROM, and it feels like enough. It is not enough.

  The model picks each token with an argmax over 512 classes, and argmax is a step function. You can inject a large amount of numeric drift into the logits and the winning index does not move, so the output stays byte-identical while the arithmetic underneath is quietly wrong. The test passes. Nothing looks broken.

  Two genuinely broken changes sailed through a three-token gate here. First, deleting the live range checks in the fixed-point clamp — those checks demonstrably fire during normal generation, so removing them is a real behavioural change, and three tokens did not care. Second, narrowing the key-value cache entries to int16, when keys in this model peak around 110,944, far outside int16 range. Forty-four overflows occur inside the three-token run itself. Still passed.

  A sixteen-token gate catches both, and the reason is structural rather than statistical. Sixteen tokens fills the context window, so errors enter the KV cache and every later position attends over the corrupted entries. The damage compounds instead of being absorbed, and the model collapses into a repeating cycle. That repeating-cycle signature is the same failure the Genesis ROM showed when it started rambling, which is how two ports ended up sharing one diagnosis.

  There is a performance version of the same mistake. A three-token bench understates KV and attention work by roughly four to five times, because the position index never exceeds two. Real wins get discarded as noise on that bench and real regressions hide in it.

  One instrument turned out to be useless entirely: static instruction counting. It swung from minus seventeen percent to plus one percent on a reorder of lines that changed no semantics whatsoever. Only frame counts under an emulator meant anything, and those were cross-validated across PyBoy and SameBoy, which agree to within poll granularity with tokens exact.

  The general rule: a correctness gate has to run long enough for errors to feed back into whatever carries state forward. For a transformer that means filling the context window. Anything shorter is measuring whether argmax is robust, which it is, rather than whether your change is correct, which is the actual question.

• The tok/s counter was lying, and how I caught it

  Scott Boudreaux • 08/14/2026 at 10:33 • 0 comments

  The N64 port shipped with a token rate on screen, counting up live as the text generated. About 60 tokens per second. It is in the video. That number was wrong by roughly a factor of fifty. The real figure is 1.23 tokens per second.

  Here is what gave it away, eventually. The counter read essentially the same on the scalar build and on the RSP overlay build. It should not have — those two differ by a real 1.8x. A gauge that does not move when the thing it measures moves is not a gauge.

  The cause was that the rate came from a cycle-count read that does not advance against real time under emulation. Under ares this workload runs at roughly 0.35 to 0.63 times real time, drifting with load on the host GPU, so a rate computed that way tracks the host machine rather than the console.

  Fixing it meant two independent instruments that had to agree before I believed either. CP0 cycle counts, and counting vblanks. Vblanks are the honest clock here, because the console produces sixty of them a second no matter what the emulator is doing underneath. Both now put the scalar build at 1.23 tok/s and the RSP overlay at 2.19 — a 1.78x win for the RSP, and the two methods agree to within a tenth of a percent.

  A 4.769x figure also circulated for this port. That one was a ratio of CP0 cycle counts taken on an int8 blob, which is not what ships. The shipped blob is ternary, and on ternary the RSP margin drops to 1.78x, because ternary has already eliminated most of the multiply work the RSP was winning on. Two different builds, two different quantizations, one number quoted across both.

  The uncomfortable part is the part worth writing down. A broken counter made this port look roughly fifty times better than it was, and it sat there for weeks without anyone questioning it, myself very much included. Nobody audits a number that flatters them. If the counter had read 0.02 tok/s I would have found the bug that afternoon. That asymmetry is the real defect, and it is not in the code.

  Still outstanding: none of this has run on N64 silicon. Every figure above is ares. If you own the cartridge and the console, I would much rather be corrected than quoted.

View all 3 project logs