Qualcomm devotes significant time to talking about its Hexagon NPU during its new product unveilings. Keen observers will remember that this branding has been reused from the company’s line of digital signal processors (DSPs), and there’s a good reason for that.
“Our journey in AI processing started probably 15 or 20 years ago, with our first anchor point looking at signal processing,” said Vinesh Sukumar, head of AI products at Qualcomm. DSPs have similar architecture compared to NPUs, but they are much simpler with a focus on audio processing (e.g., speech recognition) and modem signals.
The NPU is one of many components in a modern SoC.
Credit: Qualcomm
The NPU is one of many components in a modern SoC.
Credit: Qualcomm
As the collection of technologies we call “artificial intelligence” evolved, engineers began using DSPs for more types of parallel processing, such as long short-term memory (LSTM). Sukumar explained that as the industry became enamored with Convolutional Neural Networks (CNN), the technology underlying applications like computer vision, DSP, etc. focused on matrix functions, which are also essential for generative AI processing.
Although there is an architectural lineage here, it is not quite right to say that NPUs are just fancy DSPs. “If you talk about DSP in the general sense of the word, yes, [an NPU] There’s a digital signal processor,” said Mark Odani, assistant vice president at MediaTek. “But it’s all come a long way and it’s much more optimized for parallelism, looking at how transformers work, and having a larger number of parameters for processing.”
Despite being so prominent in new chips, NPUs are not strictly necessary for running AI workloads at the “edge,” a term that differentiates local AI processing from cloud-based systems. CPUs are slower than NPUs but can handle some lighter workloads without using much power. Meanwhile, GPUs can often chew through more data than NPUs, but they use more power to do so. And according to Qualcomm’s Sukumar, sometimes you might want to do that. For example, running AI workloads while a game is running can benefit the GPU.
<a href