The market for AI servers looks bright

AI & ML

29 March 2023 AI & ML

Seeing a bright future in the development of AI technologies, Microsoft has invested $10 billion into the well-known research laboratory OpenAI, the creators of ChatGPT. On the back of this announcement, Microsoft launched an improved version of its search engine Bing. The new Bing has incorporated a large-scale language model named Prometheus and the technology that underlays ChatGPT. Prometheus is a collaboration between Microsoft and OpenAI.

Not to be left out, Baidu launched ERNIE Bot. Initially operating as a standalone software, ERNIE Bot will be integrated into Baidu’s own search engine at a later time.

Regarding the models and specifications of the computing chips used in these AI projects, ChatGPT has mainly adopted NVIDIA’s A100 and exclusively utilises the cloud-based resources and services of Microsoft Azure. If the demand from ChatGPT and Microsoft’s other applications are combined, then Microsoft’s demand for AI servers is projected to total around 25 000 units for 2023.

Turning to Baidu’s ERNIE Bot, it originally adopted NVIDIA’s A100. However, due to the export control restrictions implemented by the US Commerce Department, ERNIE Bot has now switched to the A800. If the demand from ERNIE Bot and Baidu’s other applications are combined, then Baidu’s demand for AI servers is projected to total around 2000 units for 2023. A survey by TrendForce has revealed that in the market for server GPUs used in AI-related computing, the mainstream products include the H100, A100, and A800 from NVIDIA, and the MI250 and MI250X series from AMD. It should be noted that the A800 is designed specifically for the Chinese market under the context of the latest export restrictions. In terms of the market share for server GPUs, NVIDIA now controls about 80%, whereas AMD controls about 20%.

Focusing just on the specifications of these GPUs, ones that are involved in high-bandwidth computing and thus require high-bandwidth memory (HBM), have attracted even more attention in the market. HBM currently represents about 1,5% of the entire DRAM market. The main suppliers for HBM solutions are Samsung, SK Hynix and Micron. Among them, SK Hynix is expected to become the dominant supplier for HBM3 solutions as it is only one capable of mass producing the HBM3 solution that has been adopted by NVIDIA.

Share this article:

Development kit for AI and edge applications
TRX Electronics AI & ML
Mouser Electronics is now shipping the new Digi ConnectCore MP255 development kit, which boasts a versatile, secure, and cost-effective wireless system-on-module (SOM), designed for maximum power efficiency to support battery-powered and industrial AI applications.

Read more...

New platforms that deliver advanced edge AI capabilities
AI & ML
The SOM-5000, VAB-5000, and ARTiGO A5000 from VIA Technologies are powered by Mediatek Genio and designed for industrial, commercial and consumer applications.

Read more...

Ryzen-based computer on module
Altron Arrow AI & ML
SolidRun announced the launch of its new Ryzen V3000 CX7 Com module, configurable with the eight-core/16-thread Ryzen Embedded V3C48 processor.

Read more...

What is an NPU?
AI & ML
A neural processing unit is a specialised hardware accelerator designed to efficiently process tasks related to artificial intelligence, in particular deep learning models.

Read more...

Categories

AI & ML

The market for AI servers looks bright

Further reading:

Publications by Technews