What is an NPU?
29 November 2024
AI & ML
An NPU stands for Neural Processing Unit, which is a specialised hardware accelerator designed to efficiently process tasks related to artificial intelligence, in particular deep learning models. NPUs are optimised for operations commonly found in neural networks, such as matrix multiplications, convolutions, and activation functions.
Here are some key points about NPUs:
NPUs are designed to perform the following AI tasks at maximum speed, while using the least amount of energy. NPUs are:
• Optimised for AI workloads: NPUs are designed to handle the vast number of computations involved in neural network processing more efficiently than general-purpose processors (like CPUs or GPUs).
• Power efficient: They are typically more power-efficient than using CPUs or even GPUs for AI tasks, making them ideal for mobile devices and edge computing where power consumption is critical.
• Able to perform real-time processing: NPUs can execute AI tasks, such as image recognition, speech processing, and other inference tasks, in real-time due to their high computational parallelism.
• Used in smartphones and edge devices: NPUs are increasingly common in modern smartphones (like those made by Apple, Huawei, and Google) to accelerate AI tasks such as photography enhancements, face recognition, and voice assistance.
• Integrated with SoCs (System on Chips): Many NPUs are embedded within SoCs, which are used in devices like smartphones, autonomous vehicles, IoT devices, and more, enabling AI computations directly on the device without needing to rely on cloud-based processing.
To summarise, neural processing units are crucial for enabling the growing demand for AI processing in compact, energy-efficient, and real-time environments without needing to be connected to a server to handle the AI processing.
Further reading:
Expert perspectives on robotics and AI
AI & ML
Mouser Electronics has announced a new eBook in collaboration with Analog Devices and Samtec exploring how robotics, artificial intelligence and machine learning are transforming the landscape of manufacturing, logistics, and beyond.
Read more...
Accelerating AI adoption in MCU manufacturing
Editor's Choice AI & ML
To gain the value of ML functionality, designers of MCU-based devices have to adopt a new development method and accept a new type of probabilistic rather than deterministic output.
Read more...
Altron Arrow: Empowering innovation with STMicroelectronics AI processors
Altron Arrow
Editor's Choice AI & ML
ST’s AI processors are not only smarter and faster, but also incredibly efficient, enabling a new wave of intelligent solutions across multiple industries.
Read more...
How AI is transforming software engineering
AI & ML
Artificial Intelligence is fundamentally reshaping the landscape of software engineering, particularly in South Africa, where the demand for innovative solutions is rapidly increasing.
Read more...
Quantum computing explained
AI & ML
Quantum computers are an emerging technology which has the potential to change our world, and work by harnessing quantum physics – the strange, often counterintuitive laws that govern the universe at its smallest scales and coldest temperatures.
Read more...
AI-powered weather forecasts across Africa
AI & ML
Using MetNet-3, an advanced AI weather model, precipitation is predicted with high accuracy via satellite data, which fills gaps in current radar coverage.
Read more...
From the editor's desk: Groq – the future of AI processing?
Technews Publishing
AI & ML
The introduction of Groq’s ASIC-based approach to AI inferencing marks a significant shift in the landscape of LLMs.
Read more...
Development kit for AI and edge applications
TRX Electronics
AI & ML
Mouser Electronics is now shipping the new Digi ConnectCore MP255 development kit, which boasts a versatile, secure, and cost-effective wireless system-on-module (SOM), designed for maximum power efficiency to support battery-powered and industrial AI applications.
Read more...
New platforms that deliver advanced edge AI capabilities
AI & ML
The SOM-5000, VAB-5000, and ARTiGO A5000 from VIA Technologies are powered by Mediatek Genio and designed for industrial, commercial and consumer applications.
Read more...
Ryzen-based computer on module
Altron Arrow
AI & ML
SolidRun announced the launch of its new Ryzen V3000 CX7 Com module, configurable with the eight-core/16-thread Ryzen Embedded V3C48 processor.
Read more...
printer friendly version