The screen works. The scoreboard barely moved.
Two pieces since the last post. The first gave the emulator a picture-processing unit, so it draws. The second put that picture in a window with buttons, so you can play it. The scoreboard:
cpu instructions 499 / 500
test roms 106 / 165Up from 85. Four groups finished outright:
| group | score |
|---|---|
| ppu timing | 12 / 12 |
| oam bug | 7 / 7 |
| cpu & interrupts | 31 / 31 |
| timer | 13 / 13 |
And one group that is the reason for this post:
| group | score |
|---|---|
| screen | 4 / 30 |
Thirty tests take a photograph of the screen and compare it, pixel for pixel, against a photograph taken from real hardware. Four of them match. That number is the most interesting thing that happened in two weeks, because of how it got there.
Nearly right is worth nothing
Partway through, the whole rendering pipeline turned out to be running seven dots ahead of where the hardware photographs place it. A dot is one four-millionth of a second; a scanline is 456 of them. Seven dots is an error you could not see by looking. It was wrong in nearly every frame the emulator had ever drawn.
Fixing it corrected about 44,000 wrong pixels. The group’s total went from 118,088 differing pixels to 73,628.
It flipped exactly one test from failing to passing.
That is not a disappointing result, it is the point. A screenshot test is pass or fail across all 23,040 pixels at once. There is no credit for being close. An image that is wrong in eight pixels and an image that is wrong in twenty thousand both score zero, and the scoreboard is right to treat them the same — a Game Boy that draws a frame nearly correctly is a Game Boy that draws the frame incorrectly.
So the emulator got dramatically more accurate and the public number went up by one. Both facts are true and the second one is the one that counts.
What a hard test suite actually looks like
The twelve PPU timing tests measure, from inside the emulated machine and to the individual machine cycle, exactly when each phase of drawing begins and ends. Passing all twelve took a task of pure debugging — nothing new was built, the whole task was finding out why six of them disagreed with reality.
One of those disagreements ended somewhere unexpected. The documentation gives an algorithm for how much extra time each sprite costs the renderer. The test insisted the real hardware charges three dots less, once per scanline, for the first sprite. All 104 of the test’s cases fit with that constant and none fit without it.
Nobody knows why it is three. It is written down as a divergence with the evidence, the arithmetic, and a plain statement that the cause is not understood — rather than quietly absorbed into the code where the next person would have to rediscover it.
There are thirteen of those entries now. Each one is a place where this emulator knowingly differs from a document or a test, with the reasoning and what would overturn it.
The part where I was wrong
The window took five tasks. It loads a ROM, runs at the Game Boy’s real 59.727 frames a second, takes keyboard input, and writes save files atomically so an interrupted write cannot destroy a save you already had.
I specified that launching it with no ROM should print an error and exit, rather than open an empty window. That is correct behaviour for a command line and it is the wrong behaviour for a person. Double-clicking gave the process no console to print into, so it flashed and vanished — indistinguishable from a crash. The first report I got was “when i open it, it does not run.”
The emulator was fine the whole time. The specification was wrong, in a way that only showed up when someone opened it the way people actually open things. It now opens a window that says Drop a Game Boy ROM here.
Tests that cannot fail
The recurring problem across both pieces was not broken code. It was tests that passed for the wrong reason.
Four separate times, a test asserted something its own setup made impossible to observe. One checked that a counter only advanced under a condition, while leaving that condition true for the entire test. One measured a dot count in buckets of four, so three different wrong answers all rounded to the right one. One read a pixel from a row that had already been overwritten.
None of them were caught by running the tests. All of them pass. They were caught by a reviewer reading the test and asking whether it could fail at all.
The fix that stuck was mutation testing: deliberately break the behaviour, rebuild, and watch the test go red. If it stays green, the test was decorative. One task applied seventeen mutations and confirmed all seventeen were caught. Another applied eight. It is slow and it is the only thing that actually works.
What is next
Two gaps stop this being an emulator you would use rather than one you would measure.
Cartridge chips. Only the simplest memory controller is implemented, which is eighteen test ROMs unclaimed and the reason Pokémon will not load — it wants a chip that is not there yet, and gets a polite refusal instead.
Sound. Nothing at all, which is twelve more.
Together those are worth more than everything the screen is still missing. The picture is the hard part and it is mostly done; what is left is breadth.
Everything is public — the commits, the specs, the plans, the thirteen divergences, and the scoreboard that generated the numbers in this post.