On October 21 the list at reversedwithai.com carried 143 projects, 50 of them marked as working. On October 28 it carried 267, with 109 working. That is 124 new entries in seven days, and the site's own search box already counts 458 submissions waiting to be sorted. If you read the Kotaku piece about browser ports of Halo and Vice City that "seem to work perfectly," this list is where the examples came from, and it is the number people now quote when they want to say the games scene has been cracked open by agents.

I have written three pieces on that scene this month: the PlayStation emulator that stopped compiled code keeping secrets, the Switch 2 jailbreak that turned a hardware lock into a two-week problem, and the decompilation leaderboard that went dark because a percentage stopped meaning anything once an agent could move it. The list is the fourth chapter, and it is the same story again. A number everyone cites, produced by the people it flatters, checked by nobody.

So I checked it.

"Works" on the list means the author said so. A build in a clean room is the lowest bar there is, and most of the list had never been asked to clear it.

What the list counts

The list is honest about what it is. Each entry, the site says, "links the author's own writeup or repo and lists the model, tools and reported outcome." It adds that "a working result does not necessarily mean a complete game; each entry records its limitations." The three outcomes are Works, Partial and Research. There is no "does not work," because nobody submits a failure.

Scraping all 267 project pages on October 28 gives the shape of it. Of the 267, 254 link a repository, all but one on GitHub. The other 13 are write-ups without code, including the first-person shooter whose author spent an estimated 600 billion tokens on it and the Snowboard Kids decompilation finished in 84 days. Of the projects that name a model, 220 say Claude and 10 say an OpenAI model; 29 do not say. Among the 109 marked as working, 95 name Claude. Windows is the most common target, named on 142 entries, then Linux on 74 and macOS on 44, with a long tail of PlayStation 2, Xbox, Android and browser ports.

None of that tells you whether the code on the other end of the link does what the entry says. The site does not claim it does. Its submission policy asks for a repository or a write-up, and the outcome field is filled in by the submitter. The scoreboard that replaced decomp.dev's percentage is a self-report.

The lowest bar there is

A reasonable person might say: of course nobody re-tests 267 hobby projects, who would. The answer is that a machine would, if you ask it the right question. The question I chose is the smallest one that still means something. Take the repository as the author left it, put it on a clean Ubuntu machine that has never seen it, follow the repository's own build files, and see whether a build finishes inside 20 minutes.

That is all the check does. It does not run the game, because running almost any of these needs the original game's files and I am not supplying them. It does not judge the code. It does not know whether a build that finishes produces something playable. What it knows is whether the repository can be built by a stranger with no help from the author, which is the thing "open source port" implies and the thing a fork or a downstream project depends on.

The method is public in the Major Labs harness, and every row, reason and log is on the dataset page. Each repository gets its own disposable runner. The harness scans the README for signs that the original game data is required, picks the build system from the files present (CMake, Make, Meson, Cargo, Go, npm, Python, Gradle), installs the common libraries a port tends to want, and runs one build command under a time cap. The result is one of a short list: built, failed with a reason mined from the log, timed out, no build system at the root, or a toolchain the runner cannot host, which in practice means Windows-only project files or Xcode. Every log is kept, so any single call can be argued with.

The reason buckets matter more than the headline. A build that fails because it needs a cross-compiler for a 1996 console is a different fact from a build that fails because the code does not compile. The first is a port that cannot be reproduced without setup the author did not document. The second is a port that cannot be reproduced at all, at least not from the commit the list points to.

What the clean room found

Of the 103 working entries that link a repository, 24 built in the clean room: 23 produced a binary from their own build files and one is a static web port with nothing to build. That is 23 percent, fewer than one in four. Across all 253 repositories the list links, working or not, 51 built, one in five.

The other 79 are the finding. Only three failed on what anyone would call a bug, a compiler rejecting the code, and one more writes its output to a folder that exists only on its author's machine. Thirty-four stopped because the runner did not have a toolchain the project assumes and does not document: 23 of them inside CMake, asking for an Xbox 360 recompiler kit, a MIPS cross-compiler or a Windows resource compiler, and 11 that ship nothing but a Windows batch file, a Visual Studio project, a Premake script or an Android project without a wrapper. Sixteen stopped at the game itself: they generate their source from a ROM, a disc image or a console BIOS at build time, so without the player's copy there is nothing to compile. Fourteen could not find a library or a sibling repository. Eleven have source files and no build system at all.

Two more things the dataset says. The list is a month old in practice: 94 of the 103 working repositories had their last commit in the 30 days before the check, which fits a wave that started with the browser-port coverage in early October, and the harness found the build files below the root in 25 of them, which is what a project looks like when it was never meant to be built by a stranger. The model named on the entry does not separate the groups: 19 of the 89 working entries that name Claude built, three of the nine that name nothing, one of the three that name an OpenAI model, and the numbers outside Claude are too small to compare. Three of the 253 repositories the list links, none of them in the working group, are already gone, returning a not-found page while their entries still sit on the list with a status attached.

Only three of the 103 "working" projects failed because the code was wrong. The rest failed because nobody wrote down what it takes to build them, or because the build starts with a file only the player has.

The result is not a verdict on the people who made these projects. Most of them are hobbyists who wrote up what they did and moved on, and a repository that needs a toolchain the README forgot to mention is an ordinary open-source failing that predates agents by decades. It is a verdict on the number. "109 working" was never a measurement. It was 109 people saying so, and the first time anyone asked a machine to check, fewer than one in four cleared the lowest bar there is.

What this means by job

If you run a list, a leaderboard or a directory. Self-reported status is fine as long as you label it, and the list does. What you cannot do is let a self-reported count travel as if it were a measurement, and the browser-port coverage shows that is exactly what happens. The fix is cheap and it is the one I just ran: a scheduled build check, published next to the self-report, with a date. The list that does that first becomes the one people cite.

If you fund, buy or depend on code an agent wrote. The decompilation scene arrived at the rule a month ago: grade the artifact, not the claim, and keep the grader where the thing being graded cannot touch it. A clean-room build is the cheapest grader there is. It costs minutes of compute per repository and it catches the most common way an agent-written project overstates itself, which is that it worked once, on one machine, with the author's setup, and nobody wrote the setup down.

If you are a publisher. The honest count of reproducible ports is 24 of the 103 working entries that link code, not 109, and every one of them still needs the player's own copy of the game to do anything. The line I keep drawing holds: decompiling or porting a game you own is one thing, and shipping the publisher's assets, piracy stores and circumvention tools is another. The list contains the first kind. A takedown campaign aimed at the headline number is aimed at a number that does not exist.

What to watch

The check reruns. I will rebuild the full list monthly and publish the delta, so the next question is whether the share that builds goes up as authors fix their instructions or down as the list fills with one-weekend projects nobody maintains. My prediction, dated so you can hold me to it: by January 31, 2027 the list will pass 500 entries, and the share of "working" entries that build in a clean room will be lower than it is today, not higher.

If the number you quote was supplied by the people it flatters, what would it cost you to check it?

Charlie Major is a Product Development Manager at Mastercard. The views and opinions expressed in Major Matters are his own and do not represent those of Mastercard.