What is an NPU?

29 November 2024 AI & ML

An NPU stands for Neural Processing Unit, which is a specialised hardware accelerator designed to efficiently process tasks related to artificial intelligence, in particular deep learning models. NPUs are optimised for operations commonly found in neural networks, such as matrix multiplications, convolutions, and activation functions.

Here are some key points about NPUs:

NPUs are designed to perform the following AI tasks at maximum speed, while using the least amount of energy. NPUs are:

• Optimised for AI workloads: NPUs are designed to handle the vast number of computations involved in neural network processing more efficiently than general-purpose processors (like CPUs or GPUs).

• Power efficient: They are typically more power-efficient than using CPUs or even GPUs for AI tasks, making them ideal for mobile devices and edge computing where power consumption is critical.

• Able to perform real-time processing: NPUs can execute AI tasks, such as image recognition, speech processing, and other inference tasks, in real-time due to their high computational parallelism.

• Used in smartphones and edge devices: NPUs are increasingly common in modern smartphones (like those made by Apple, Huawei, and Google) to accelerate AI tasks such as photography enhancements, face recognition, and voice assistance.

• Integrated with SoCs (System on Chips): Many NPUs are embedded within SoCs, which are used in devices like smartphones, autonomous vehicles, IoT devices, and more, enabling AI computations directly on the device without needing to rely on cloud-based processing.

To summarise, neural processing units are crucial for enabling the growing demand for AI processing in compact, energy-efficient, and real-time environments without needing to be connected to a server to handle the AI processing.

Further reading:

AI-ready embedded SBC
AI & ML
The new Grinn GenioBoard SBC provides a production-ready implementation of a powerful eight-core MediaTek processor, backed by high-speed interfaces, a Linux distro, and CRA-ready security software.

Read more...

---

Alif Semiconductor elevates generative AI at the edge
AI & ML
Developers can now use the ExecuTorch Runtime for AI applications built to run on its Ensemble E4/E6/E8 series of MCUs and fusion processors.

Read more...

---

Questing for the quantum AI advantage
Editor's Choice AI & ML
Two quantum experts disclose high hopes and realities for this emerging space.

Read more...

---

How a vision AI platform and the STM32N6 can turn around an 80% failure rate for AI projects
Altron Arrow AI & ML
he vision AI platform, PerCV.ai, could be the secret weapon that enables a company to deploy an AI application when so many others fail.

Read more...

---

Infineon’s OPTIGA for more secure AI and ML models
Future Electronics AI & ML
Infineon Technologies provides its OPTIGA Trust M security solution to Thistle Technologies for embedded computing products.

Read more...

---

Altron Arrow introduces GX10 supercomputer
Altron Arrow AI & ML
Powered by the NVIDIA GB10 Grace Blackwell superchip, this is desktop-scale AI performance previously only available to enterprise data centres.

Read more...

---

Photonic chip to slash AI energy consumption
AI & ML
Arago, a Paris and Silicon Valley-based deeptech startup pioneering a new class of energy-efficient AI chips powered by light, has raised $26 million in seed funding to accelerate the commercialisation of its photonic processor.

Read more...

---

Exploring AI
AI & ML
Powered by an ESP32-S3 MCU with 512kB SRAM and 16MB Flash, the Unihiker K10 features a vibrant 2,8-inch colour screen, built-in Wi-Fi and Bluetooth 5.0, and a 2-megapixel camera.

Read more...

---

Empowering innovation with ST’s AI processors
Altron Arrow AI & ML
Artificial intelligence is no longer just a futuristic concept – it is here, and it is transforming industries at an unprecedented pace.

Read more...

---

Vision-language models
AI & ML
Vision-Language Models are changing the AI game again, blending image and text understanding to power everything from autonomous vehicles to robotics to AI-driven assistants.

Read more...

---