Apple Makes Its Case: On-Device Is Critical AI Infrastructure

Apple has stayed out of the enterprise AI infrastructure debate while everyone else fought over it. Hyperscalers sold compute, model labs sold tokens, chipmakers sold accelerators. Apple sold laptops. This month that changed, quietly, in the form of a research report.

Omdia surveyed 1,584 enterprise technology leaders, including CIOs, CTOs, software engineers, and AI specialists, across the United States, United Kingdom, Germany, China, and India. The report, Rethinking Critical AI Infrastructure, argues that on-device infrastructure has become a serious option for enterprise AI, not just a developer convenience. Over a third of organizations plan to shift more AI workloads on-device in the next 12 months.

One thing to say plainly before the numbers: Apple paid for this. The study is commissioned research, and its conclusion points at hardware Apple sells. Read it as Apple's opening position, not a neutral audit. The reason it is worth reading anyway is that the underlying argument is one the cloud-first narrative has mostly ignored, and the data behind it is specific.

Apple is not selling a model or a cloud. It is arguing that the device you already own is enterprise AI infrastructure, and the most AI-mature teams in the survey already agree.

The argument, in three parts

The report organizes the case around three pillars: security, economics, and workload viability. Each maps to a real complaint enterprises have about cloud-first AI.

The framing matters because it reframes "on-device" away from consumer features, the Siri-on-your-phone register, and toward the boring infrastructure question a CIO actually asks: where should this workload run, and what does it cost me to run it there. Only 8 percent of organizations deploy AI exclusively on-device today. Apple's bet is that the number climbs once the trade-offs are measured rather than assumed.

Security as architecture, not paperwork

The security argument is the strongest, because it is structural rather than promotional.

A total of 99 percent of enterprises handle proprietary data in their AI workflows. To use most cloud AI, that data has to leave the building and travel to a third party. 76 percent of enterprises identify that transmission as a concern, and 87 percent say keeping sensitive data on-premises or on-device is important. The report's line is blunt: data that never transmits cannot leak.

This is where the survey's composition matters for our readers. Financial services made up 34 percent of the sample, healthcare another 33 percent. These are the industries where data residency rules, audit cycles, and contractual prohibitions on third-party access turn every cloud AI project into a compliance exercise. As we covered in the agentic AI security reckoning, the gap between what gets announced and what survives a security review is where most enterprise AI stalls.

On-device does not eliminate security work. Physical device control, access management, and audit logging still apply. What it removes is the category of risk that comes from moving regulated data across a network you do not own. That is the difference between managing a risk and not having it.

The economics nobody puts on the invoice

The cost argument has two halves: the cost of building and the cost of running.

On the build side, the report names what it calls the experimentation tax. Cloud charges accumulate per run. When a promising approach fails after 20 iterations, those 20 iterations still appear on the bill. Learning from a dead end costs the same as learning from a success. On fixed local hardware, every iteration after the initial purchase costs nothing extra, so teams stop rationing their experiments. For the 54 percent of organizations still investigating or piloting AI, that changes how freely they can work.

On the run side, the waste is more concrete. Take 1,000 cloud AI seats at $30 per user per month. If just 10 percent go unused, that is $3,000 a month spent on nothing. The report found 62 percent of organizations are only somewhat satisfied or not satisfied with getting employees to adopt AI tools, which means stranded seats are common, not hypothetical.

There is a measurement blind spot underneath all of this. Cloud compute is the top total-cost-of-ownership concern at 39 percent, and 72 percent of the firms that name it track it as a distinct line item, because the invoice arrives itemized every month. Security is the second-largest concern at 36 percent, yet fewer than half of enterprises track security costs as a discrete component. Organizations optimize what they can see. When the cloud bill is visible and the security overhead is scattered across other budgets, the math tilts toward cloud even when the full cost does not.

The workload reality check

The most useful data in the report attacks a specific assumption: that real AI needs a hyperscale data center.

It usually does not. 95 percent of organizations are not training models from scratch, they are fine-tuning and running inference on existing ones, which is far less demanding. 57 percent of the models enterprises deploy are under 10 billion parameters, a size that runs comfortably on a current MacBook Pro or Mac Studio. A 10 billion parameter model needs roughly 12GB of memory when quantized; a base laptop handles it.

The interesting part is what happens at the top end. Omdia found no meaningful relationship between model size and cloud preference. Organizations running models over 100 billion parameters were no more likely to prefer cloud than those running 10 billion ones. The real constraint is not model size, it is memory architecture. A discrete GPU offers 24GB to 48GB of fast memory, and performance falls off a cliff when a model spills past it into system RAM. Apple silicon pools memory instead: a 128GB MacBook Pro can run a 70 billion parameter model locally, and a 512GB Mac Studio can run 300 billion. That is the entire technical argument, and it is a genuine one.

This is the counterweight to the build-out story we keep covering. When OpenAI raises money at an $852 billion valuation to fund infrastructure, the implied premise is that AI demand is bottlenecked on centralized compute. Apple's data suggests a large share of enterprise AI was never going to need that compute in the first place.

The tell: who actually chooses it

The most persuasive finding is a revealed preference. Organizations that build AI in-house adopt Mac for AI workloads at 1.8 times the rate of organizations buying commercial AI solutions. The teams with the deepest technical requirements, the ones who feel iteration cost and data-access friction directly, pick on-device most often. Among firms already using on-device infrastructure, 65 percent plan to shift even more onto it.

When the people closest to the work keep choosing the same architecture, it is worth asking what they know that the procurement default does not.

Where the argument thins out

Take the frame seriously, then name what it leaves out.

This is vendor research, and it is built to flatter a conclusion. On-device is presented as a foundation that makes everything else optional, but the report's own data shows it as a complement, not a replacement: 56 percent of organizations using Mac for on-device AI also run NVIDIA GPUs locally, and large-scale training still goes to clusters. The honest reading is hybrid, where on-device joins the mix, not a migration off the cloud.

The bigger gap is what Apple is not offering. The silicon is ready, but a fleet of capable Macs is not an enterprise AI platform. There is no Apple management layer for deploying and governing models across thousands of devices, no agentic runtime, no equivalent of the orchestration and policy tooling that the cloud providers ship. Apple has made the hardware argument and stopped there. As enterprises wrestle with the same data-residency and dependency questions we explored in sovereign AI and bank dependency, the missing piece is the software and governance story, and Apple has not told it.

And the convenient subtext is hard to miss: in this telling, the answer to almost every infrastructure question is to buy more Macs.

What to watch

The data is real, the frame is sound, and the sponsor is selling something. All three can be true at once.

The question is whether Apple builds the layer that would make the hardware argument complete: management, deployment, and governance for AI on a fleet of devices. If it does, the on-device pillar becomes a platform and the survey looks like the first move in a longer strategy. If it does not, this stays a well-argued case for buying laptops, and the enterprise keeps routing its hardest workloads through the cloud regardless of where the math points.

If most enterprise AI runs on models small enough for a laptop, how much of the cloud build-out is for workloads that were never going to need it?

Charlie Major is a Product Development Manager at Mastercard. The views and opinions expressed in Major Matters are his own and do not represent those of Mastercard.