When I looked at the specs for a new laptop a few months ago, there it was: “NPU: 45 TOPS.” It made sense to use the acronym. It wasn’t clear what it meant in everyday life. It was also thought, and later proven, that almost none of the software running on that computer was touching the NPU.
That’s what really happened in 2026. The hardware has been sent. The software is still taking a deep breath.
Neural Processing Units are now built into almost all new smartphones, laptops, and Macs that come out today. They are built into the System on Chip as blocks that are only used for matrix math. At the heart of every neural network is this operation. Apply activation functions to matrices, multiply them, and do it again. NPUs can do that sequence much faster and with a lot less power than a CPU or GPU. Inference tasks that would use 150 watts or more from a discrete GPU use less than five watts. In the case of battery-powered items, that difference is completely different.
You should think about the difference between this and a GPU. GPUs were not made to work with AI. Because neural networks need the same kind of parallelism that rendering pixels does, they were changed to do it. In particular, that worked well enough for training. A GPU wasn’t made to run a model locally on a laptop all day long while the machine stays cool and the battery lasts. An NPU is more focused on that one problem, less flexible, and really better suited to it.
The fact that Apple has been in this position for so long makes it stand out. We didn’t hear about the first consumer NPU at a 2024 keynote riding the wave of generative AI. It came out in 2017 in the A11 Bionic chip of the iPhone X. It was named the Neural Engine by Apple. Two cores. Six hundred billion things happen every second. Face ID and Animoji use it. These aren’t exactly applications that will change the world, but they are the building blocks of something that will grow quietly for seven years. Before other companies tried hard to get Microsoft’s Copilot+ PC certification, which required 40+ TOPS, Apple’s M4 Neural Engine was already delivering 38 TOPS on a design that had been improved over many generations of silicon. That trend shows a level of institutional patience that simple benchmark numbers can’t fully show.
That requirement for Copilot+ needs its own time. When Microsoft set the bar at 40+ dedicated NPU TOPS in May 2024, it quickly showed something troubling: Intel’s brand-new flagship, the Core Ultra with Meteor Lake, had only about 11 TOPS. As a next-generation product, the chip was already below the threshold when it came out. The answer was Lunar Lake, which was rushed to a 48 TOPS NPU and made public just a few months later. A certification requirement from a single software company may have sped up NPU development across the PC industry more than years of engineering roadmaps had been able to do.
There’s also the matter of what TOPS really measures, which doesn’t get much attention. A chip that says it has 50 TOPS at INT4 precision is actually doing 4-bit integer operations, which cause rounding errors that lower the accuracy of the model. For the same model, a different chip that says it has 30 TOPS at INT8 might give better real-world inference results. The headline number is advertised by sellers. Most of the time, the precise context doesn’t make it onto the marketing sheet. If you compare NPU specs from AMD, Intel, Qualcomm, and Apple without checking the accuracy level, you’re comparing apples with apples that are not the same shape.
Still, the most honest observation is the one that doesn’t make the news: most of the NPU hardware is just sitting there. Standard workloads on Windows computers are sent to the CPU or a separate GPU by default. It is possible to route through Windows ML and DirectML, but application developers have to make sure they target it. There are some real exceptions, like Windows Recall, Live Captions, and some Microsoft 365 features. It’s hard not to notice that the industry has done a good job of building the hardware layer, but the software ecosystem is still running behind and catching up at its own pace.
It’s not clear whether that gap will close in a year or five. The chips are already in the machines, though. That part is at least taken care of.
