The market for AI servers looks bright
29 March 2023
AI & ML
Seeing a bright future in the development of AI technologies, Microsoft has invested $10 billion into the well-known research laboratory OpenAI, the creators of ChatGPT. On the back of this announcement, Microsoft launched an improved version of its search engine Bing. The new Bing has incorporated a large-scale language model named Prometheus and the technology that underlays ChatGPT. Prometheus is a collaboration between Microsoft and OpenAI.
Not to be left out, Baidu launched ERNIE Bot. Initially operating as a standalone software, ERNIE Bot will be integrated into Baidu’s own search engine at a later time.
Regarding the models and specifications of the computing chips used in these AI projects, ChatGPT has mainly adopted NVIDIA’s A100 and exclusively utilises the cloud-based resources and services of Microsoft Azure. If the demand from ChatGPT and Microsoft’s other applications are combined, then Microsoft’s demand for AI servers is projected to total around 25 000 units for 2023.
Turning to Baidu’s ERNIE Bot, it originally adopted NVIDIA’s A100. However, due to the export control restrictions implemented by the US Commerce Department, ERNIE Bot has now switched to the A800. If the demand from ERNIE Bot and Baidu’s other applications are combined, then Baidu’s demand for AI servers is projected to total around 2000 units for 2023. A survey by TrendForce has revealed that in the market for server GPUs used in AI-related computing, the mainstream products include the H100, A100, and A800 from NVIDIA, and the MI250 and MI250X series from AMD. It should be noted that the A800 is designed specifically for the Chinese market under the context of the latest export restrictions. In terms of the market share for server GPUs, NVIDIA now controls about 80%, whereas AMD controls about 20%.
Focusing just on the specifications of these GPUs, ones that are involved in high-bandwidth computing and thus require high-bandwidth memory (HBM), have attracted even more attention in the market. HBM currently represents about 1,5% of the entire DRAM market. The main suppliers for HBM solutions are Samsung, SK Hynix and Micron. Among them, SK Hynix is expected to become the dominant supplier for HBM3 solutions as it is only one capable of mass producing the HBM3 solution that has been adopted by NVIDIA.
Further reading:
World’s most powerful open LLM
AI & ML
With a staggering 180 billion parameters, and trained on 3,5 trillion tokens, Falcon 180B has soared to the top of the Hugging Face Leaderboard for pretrained LLMs.
Read more...
Bridging the gap between MCUs and MPUs
Future Electronics
Editor's Choice AI & ML
The Renesas RA8 series microcontrollers feature Arm Helium technology, which boosts the performance of DSP functions and of AI and machine learning algorithms.
Read more...
Hardware architectural options for artificial intelligence systems
NuVision Electronics
Editor's Choice AI & ML
With smart sensors creating data at an ever-increasing rate, it is becoming exponentially more difficult to consume and make sense of the data to extract relevant insight. This is providing the impetus behind the rapidly developing field of artificial intelligence.
Read more...
xG26 sets new standard in multiprotocol wireless device performance
Altron Arrow
AI & ML
Silicon Labs has announced its new xG26 family of Wireless SoCs and MCUs, which consists of the multiprotocol MG26 SoC, the Bluetooth LE BG26 SoC, and the PG26 MCU.
Read more...
SolidRun unveils new SoM
Altron Arrow
AI & ML
SolidRun and Hailo has unveiled a game-changer for engineers and AI product developers with the launch of their market-ready SoM, which packs the cutting-edge capabilities of the Hailo-15H SoC.
Read more...
Banana Pi with NPU
CST Electronics
AI & ML
The latest Banana Pi SBC, the BPI-M7, is powered by Rockchip’s latest flagship RK3588 octa-core 64-bit processor, with a maximum frequency of 2,4 GHz.
Read more...
ESP32-P4 high-performance MCU
iCorp Technologies
AI & ML
Powered by a dual-core RISC-V CPU running up to 400 MHz, ESP32-P4 also supports single-precision FPU and AI extensions, thus providing all the necessary computational resources.
Read more...
AI-native IoT platform launched
EBV Electrolink
AI & ML
These highly-integrated Linux and Android SoCs from Synaptics are optimised for consumer, enterprise, and industrial applications and deliver an ‘out-of-the-box’ edge AI experience.
Read more...
Flash for AI
EBV Electrolink
AI & ML
SCM offers a midway latency point between DRAM and SSDs, and when coupled with the introduction of CXL, low-latency flash, such as XL-FLASH, is well-positioned to deliver improvements in price, system performance, and power consumption to everything from servers to edge devices deploying the power of AI.
Read more...
Speeding up the rollout of renewable energy with AI
AI & ML
Understanding that AI, particularly within the renewables space, will not take away jobs, but rather create them, is key to leveraging the immense power of this technology to drive South Africa forward.
Read more...
printer friendly version