OpenAI Published an AI-Designed Device. It Did Not Publish the Design.

On November 3 OpenAI published a research note titled "Accelerating engineering for the physical world." The hero image is captioned "TamaGPT device experiments." The experiment, in the post's words, was "seeing if we can create a device without any direct human edits." The device is a small desk unit with one push-to-talk button and a small display that lets a person talk to Codex, OpenAI's coding agent. Astra, the post says, defined the functional requirements, designed the enclosure and the internal structures that hold the components, selected the electrical components and placed and connected them on a printed circuit board sized to fit, then designed the interface and generated the firmware. Two team members worked on it as a side project for under a week. A contract manufacturer built a small batch in under a month.

Then the sentence that matters: "Getting the device ready for manufacturing still required several rounds of review and iteration. Astra did not consistently account for the manufacturer's production limits, tolerances, or details required for assembly."

I went looking for the design. In the mechanical paragraph the word "enclosure" is a link. It opens a PNG on OpenAI's image server, a labeled diagram of the case. There is no CAD file. There is no board file, no schematic, no bill of materials, no firmware, and nothing from the manufacturer about what those several rounds changed. The post that says a model designed a device without human edits publishes a photo of the device and a drawing of the box.

A design you cannot open is a claim. A design you can run through the checker is a record.

What the post says, and what the coverage added

The coverage went further than the post. The Decoder's headline says Astra designed the device "from scratch," and its summary says the model handled "PCB layout with component selection and trace routing." The post does not say trace routing. It says "component placement and connections," which could mean routed copper or could mean a netlist, and the difference is most of the work. R&D World got closer to the ground by quoting the OpenAI researcher who built it: "I told Chat what I wanted, gave it some plans, and it designed the board." Plans in, board out, and "a few rounds with the factory" in his words, "several" in the post's.

The routing question has an answer from September, when Astra's launch demo showed it laying out a board in KiCad. NextPCB, a board manufacturer, had its head of technology run two tests. Given a board with the parts already placed and no restrictions, Astra produced a routed board in about 12 minutes. The execution log showed how: it exported the design, handed it to Freerouting, an open-source autorouter, and imported the result. "Freerouting performed the core routing computation. Astra orchestrated the workflow." Told to use only KiCad's own interface, the model drew three short trace segments in an hour before the engineer stopped the run. He was careful to add that two runs on one board are not a benchmark. They are, though, the only public measurement of Astra's board work that anyone outside OpenAI has been able to make, because OpenAI has not published a board.

The post does include one number of its own. On BenchCAD 2.0, which tests whether a model can reconstruct 3D parts and identify connections on circuit boards, GPT-6 Astra at its highest reasoning setting scores 0.594. A newer, unnamed internal model scores 0.875. That is the lab telling you its shipping model gets a little over half of a benchmark it chose, and a model you cannot use gets most of it. The other number, model usage in its hardware teams up 23-fold since February, counts tokens. It measures how much the teams asked, not how often the answer was right.

So I asked for the files

The brief I wrote mirrors the four jobs OpenAI credits to Astra. A small desk device, one push-to-talk button, a small display, USB-C power, that lets me talk to the coding agent on my laptop. Set the requirements. Deliver a bill of materials. Deliver a KiCad project: a schematic that passes the electrical rules check, a board with every footprint inside a board outline and every connection routed as copper that passes the design rules check. Deliver an OpenSCAD model of the enclosure with the structures that hold the board, display and button, that renders. Deliver the firmware and the laptop-side bridge. No placeholders. The same brief went to every lane, word for word.

Seven lanes. Three are bare models through their APIs: GPT-6 Astra at high reasoning, Anthropic's Claude Fable 5.1, and Google's Gemini 3.1 Pro. Four are the coding agents built on them: Claude Code, Codex CLI on its default model, Codex CLI pinned to Astra, and Gemini CLI. The agents ran in empty folders and could write files but not run commands, so none of them could check its own work with the tools I was about to use on it. The models answered in text and I wrote their files to disk unchanged.

Then the grader, the same for everyone. KiCad 10's command line ran the electrical rules check on each schematic, the design rules check on each board with the schematic-parity test on, and a Gerber export. OpenSCAD rendered each enclosure to a mesh. Python compiled each bridge script. I compared the board outline in the board file with the board size the enclosure assumes. I did not compile the microcontroller firmware, and I did not send anything to a factory. One run per lane, with reruns only where a run was cut off or produced nothing, and every rerun is reported.

Astra through the API produced the cleanest files of the first round. Twelve of them, including a 345-line firmware, a 450-line bridge that runs local speech recognition and launches the coding agent as a read-only job by default, and a six-screen interface spec for a 64 by 48 pixel display. The schematic passed with zero errors. The board, 64 by 54 millimeters, passed the design rules with zero errors and zero unrouted connections, and exported clean Gerbers. The enclosure renders as an assembled device: a two-part case with an orange key, a display window, a microphone grille and a USB-C cutout, and the pocket inside is the size of the board. The one check it did not clear is the parity test, 26 mismatches between board and schematic, and on inspection all but four are the same thing: the schematic names a net "/GND" and the board names it "GND." A labeling convention, not a wiring fault.

Then I read its requirements file. The first section is headed "Release status" and says: "This is a generated, unverified engineering design, not a manufacturing-qualified release. KiCad ERC, KiCad DRC, compilation, hardware testing, and OpenSCAD rendering have not been executed in this environment. No passing test results are claimed." The model wrote the caveat the lab left out.

The two Codex lanes each got a second run, because the first ones ended early: my OpenAI account ran out of API credits partway through both, and the table shows the reruns. Codex running on Astra had, at the cut, a Cherry MX key, an exploded three-part printable case and a 60 by 60 board with zero electrical and zero design-rule errors. The finished rerun kept the key and the case, moved to an 80 by 70 board with zero electrical errors and nothing unrouted, and failed the design rules three times for one mistake: it put a mounting hole where the module sits, so the hole's keep-out area overlaps the module's outline and two of the module's pins fall inside it. A screw there would land on the pins. Codex on its default model, gpt-6.1-sol, had been cut off with a board and no schematic, and the board it left had the clock and data lines of the display bus shorted together in two places, a clearance violation, three solder-mask bridges and two connections never routed. The rerun finished in six minutes with 18 files: a schematic with zero errors, a 64 by 64 board with zero rule violations and nothing unrouted, a three-part case that renders, and a bridge script that compiles. Same tool, same brief, a day apart: a board a factory would reject, then a board that passes.

Gemini's files look right until you open them. The API model wrote a board file whose header declares one KiCad file version and whose fourth line uses a token from another, and KiCad refuses it: "Unknown token 'uuid'." Its schematic has six pins connected to nothing. Its enclosure renders, as a solid slab with two pockets cut into the top and no lid, no cavity and no way to put a board inside. Gemini CLI did better on the board, which opens and routes, but at least five of its tracks cross each other (KiCad counts five, six or seven on repeated runs), the schematic has ten errors, and the parity test found the real kind of mismatch: the pad the board puts on 3.3 volts is the pad the schematic puts on the data line, and the reverse. Its enclosure is an open tray with one standoff.

Claude Fable 5.1 through the API returned nothing, twice. Given 64,000 output tokens, it spent all 64,000 thinking and emitted no text. Given 128,000 with adaptive thinking at high effort, it spent all 128,000 the same way. On a third run at medium effort it thought for 75,000 tokens and then wrote, and ran out of budget in the middle of its bridge script, with the interface spec still unwritten. What it did finish is good: a 72 by 50 millimeter board with a Cherry MX key, a 1.3 inch display and an LED, zero electrical errors, zero rule violations, nothing unrouted, the same label-prefix parity mismatches as Astra's, and a two-part printable case that renders. Two runs of silence and then a board that passes, from one model and one brief. Claude Code, the agent built on the same model, used its first run to try to fetch genuine footprint files from the KiCad library through sub-agents, and was still waiting on them when it hit the 40-minute cap with nothing written. In a second run with no network and no sub-agents, its own transcript shows what happened: the model thought until it hit the output limit, was nudged by its harness to resume, listed the empty folder, thought until it hit the limit again, and was nudged again when the 40 minutes ran out. One tool call in 40 minutes, no file. The same model that eventually produced a passing board through the API, inside the product, never got as far as writing one.

Every lane that produced a board made the same engineering decision, unprompted: a Seeed XIAO ESP32-S3 module on a carrier board, with an off-the-shelf OLED and, in most cases, the microphone that comes on the module. USB-C, the voltage regulator, the radio and the antenna are bought, not designed. That is what a competent person would do for a one-off desk toy, and it is also why the boards are small: four schematic symbols, seven footprints, a few dozen track segments, no vias on Astra's board at all. Whether OpenAI's device took the same route, the post does not say. If it did, "designed the printed circuit board" means routing a carrier for a module someone else designed, which is honest work and a smaller claim.

What a pass means, and what it does not

A board that passes DRC has no copper too close to other copper, no net left unrouted and no track where the rules say a track cannot go. It says nothing about whether the microphone clock is on a pin that can supply it, whether the display initializes, or whether the enclosure's tolerances survive a printer. Astra's own requirements file lists exactly those as needing physical verification. So does OpenAI's post, in the sentence about tolerances and assembly. The difference is that my files are open and the two commands that produced the verdicts are in the repository, so anyone can rerun them, disagree with my grader, or send the Gerbers to a factory and report what comes back.

That is the standard I used on the AI-ported games list last month, where 109 entries said they worked and 24 built in a clean room. It is the standard the Bench applies when it checks what an agent says it did against its own signed record. Grade the artifact, not the claim. For a hardware design the artifact is the KiCad project, the enclosure model, the firmware, the bill of materials, and the manufacturer's notes from each round. OpenAI has every one of those. The most interesting data in the post is in the sentence about "several rounds," and it is the sentence with no data attached.

The lab reporting on its own model

Two months ago OpenAI shelved GPT-6.1 Astra because, in its own head of safety's words, the model did not meet the bar on "staying within scope and authorization, and how it communicates back to the user about the type of work it's done." The November post is a report by the lab about work by the same model family, with the work withheld. It is a self-report one level up.

I keep coming back to the same control, because it keeps being the one that is missing. The record has to live outside the agent, and when the agent belongs to a lab, outside the lab. For a payment that is the signed mandate and the rail's log. For a design it is the files and the checker's report. A reader who owns a decision an agent touches, whether the agent is routing a board or routing a refund, should be able to ask for the artifact and run the check. On November 3 the model could produce the artifact. The post chose the picture.

What would change my mind: OpenAI publishes the KiCad project, the enclosure model, the firmware, the bill of materials and the factory's notes from each revision round. Then anyone can run the two commands I ran, and the sentence "without any direct human edits" becomes something a reader can test rather than something a reader is told.

When an agent hands you a finished design, what do you run on it before you believe it?

Charlie Major is a Product Development Manager at Mastercard. The views and opinions expressed in Major Matters are his own and do not represent those of Mastercard.