AI & ML


The market for AI servers looks bright

29 March 2023 AI & ML

Seeing a bright future in the development of AI technologies, Microsoft has invested $10 billion into the well-known research laboratory OpenAI, the creators of ChatGPT. On the back of this announcement, Microsoft launched an improved version of its search engine Bing. The new Bing has incorporated a large-scale language model named Prometheus and the technology that underlays ChatGPT. Prometheus is a collaboration between Microsoft and OpenAI.

Not to be left out, Baidu launched ERNIE Bot. Initially operating as a standalone software, ERNIE Bot will be integrated into Baidu’s own search engine at a later time.

Regarding the models and specifications of the computing chips used in these AI projects, ChatGPT has mainly adopted NVIDIA’s A100 and exclusively utilises the cloud-based resources and services of Microsoft Azure. If the demand from ChatGPT and Microsoft’s other applications are combined, then Microsoft’s demand for AI servers is projected to total around 25 000 units for 2023.

Turning to Baidu’s ERNIE Bot, it originally adopted NVIDIA’s A100. However, due to the export control restrictions implemented by the US Commerce Department, ERNIE Bot has now switched to the A800. If the demand from ERNIE Bot and Baidu’s other applications are combined, then Baidu’s demand for AI servers is projected to total around 2000 units for 2023. A survey by TrendForce has revealed that in the market for server GPUs used in AI-related computing, the mainstream products include the H100, A100, and A800 from NVIDIA, and the MI250 and MI250X series from AMD. It should be noted that the A800 is designed specifically for the Chinese market under the context of the latest export restrictions. In terms of the market share for server GPUs, NVIDIA now controls about 80%, whereas AMD controls about 20%.

Focusing just on the specifications of these GPUs, ones that are involved in high-bandwidth computing and thus require high-bandwidth memory (HBM), have attracted even more attention in the market. HBM currently represents about 1,5% of the entire DRAM market. The main suppliers for HBM solutions are Samsung, SK Hynix and Micron. Among them, SK Hynix is expected to become the dominant supplier for HBM3 solutions as it is only one capable of mass producing the HBM3 solution that has been adopted by NVIDIA.

For more information visit https://insider.trendforce.com




Share this article:
Share via emailShare via LinkedInPrint this page

Further reading:

The dream of Edge AI
Altron Arrow Editor's Choice AI & ML
AI technology carries a great promise – the idea that machines can make decisions based on the world around them, processing information like a human might. But the promise of AI is currently only being fulfilled by big machines.

Read more...
MAX78000 neural network accelerator chip
Altron Arrow AI & ML
The hardware-based convolutional neural network accelerator enables even battery-powered applications to execute AI inferences.

Read more...
Nanomaterials to build next-gen AI hardware?
AI & ML
From improving scientific analyses and imaging capabilities, to predictive maintenance and monitoring operations in industrial settings, artificial intelligence is becoming ever more present in modern-day society.

Read more...
Microchip launches MPLAB ML development suite
AI & ML
Microchip’s unique solution is first to support 8-, 16- and 32-bit MCUs and 32-MPUs for machine learning at the edge.

Read more...
ToF sensor enables AI applications
Altron Arrow AI & ML
The VL53L7CH from STMicroelectronics is the perfect Time-of-Flight sensor enabling AI applications, with ultrawide 90° diagonal FoV and low power consumption.

Read more...
Analogue compute platform to accelerate Edge AI
Altron Arrow Editor's Choice AI & ML
Microchip has teamed up with Intelligent Hardware Korea to develop an analogue compute platform to accelerate Edge AI/ML inferencing using Microchip’s memBrain non-volatile in-memory compute technology.

Read more...
World’s most powerful open LLM
AI & ML
With a staggering 180 billion parameters, and trained on 3,5 trillion tokens, Falcon 180B has soared to the top of the Hugging Face Leaderboard for pretrained LLMs.

Read more...
Advancing quality control
Avnet Silica AI & ML
As manufacturing processes continue to become more sophisticated, the importance and effectiveness of advanced DVI solutions escalate, presenting opportunities for improved quality control.

Read more...
Give your edge AI model a performance boost
AI & ML
Join this webinar from STMicroelectronics to learn how to create an edge AI application easily on an STM32 MCU using the NVIDIA TAO toolkit.

Read more...
Three reasons why AI, ML add value for SMMEs only if the basics are in place
AI & ML
There is much chatter around artificial intelligence (AI) and the subfield of machine learning (ML), which can be confusing for SMME owners who may believe that they need to climb on the bandwagon.

Read more...