13.6.2023
Every year, companies are coming up with new and unique accelerators and solutions for HPC that bring major leaps in the development, research and production of AI, ML and DL applications. That’s why we’ve summarized the most prominent ones and compared their individual parameters, performance and summarized their features.
Graphics or other computing accelerators play a huge role in HPC today. Instead of all the work being processed by traditional processors, calculations are distributed to computing units that have precisely tailored instruction sets, special cores or fast memory to process problems many times faster. Unlike processors, which currently top models with around 100 cores, graphics and other accelerators have on the order of hundreds to thousands of cores. While their cores are weaker compared to processor ones, they are precisely made for the specific operations they are supposed to perform, and they excel at it. Companies are developing better and better accelerators, better algorithms and instructions, better architectures… To help us get a better handle on the current trends and to get a good overview of the options we have right now, we’ve written this comparison of the current innovations on the market.
| Company | Intel | Intel | AMD | AMD | NVIDIA | NVIDIA | Graphcore | Cerebras |
| Accelerator | Flex 170 | Gaudi2 | MI 250X | MI 300X | A100 | H100 | C600 | WSE-2 |
| Number of cores | 32 Xe | 24 TPC | 14,080 | 12 Chiplets | 6,912 | 7,296 | 1,472 IPU | 850,000 |
| Number of transistors | 21,700 Mil. | TB/A | 58,000 Mil. | 153,000 Mil. | 54,200 Mil. | 80,000 Mil. | 59,400 Mil. | 2.600 Billion. |
| Production process | Intel 7 (10nm) | 7nm | 6nm | 5nm* | 7nm Ampere | 4nm Hopper | 7nm MK2 | 7nm TSMC N7 |
| Capacity | 16 GB | 96 GB | 128 GB | 192 GB | 80 GB | 80 GB | 900 MB | 40 GB |
| Memory type | GDDR6 | HBM2e | HBM2e | HBM3 | HBM2e | HBM2e | IPM | on-chip SRAM |
| Bus | PCIe gen. 4 | SXM5 | OAM | SXM5* | SXM4 | SXM5 | PCIe gen. 4 | prop. |
| Consumption | 150 W | 600 W | 560 W | TB/A | 500 W | 700 W | 185 W | 15 kW |
| Price | 6,400 USD (est.) | TB/A | 8,000 USD | TB/A | 32,097 USD | 36,405 USD | N/A | N/A |
*Parameters are estimates and not verified, subject to change over time
| Company | Intel | Intel | AMD | AMD | NVIDIA | NVIDIA | Graphcore | Cerebras |
| Accelerator | Xe DC GPU | Gaudi | MI 250X | MI 300X | A100 | H100 | C600 | WSE-2 |
| FP16 | 256 GFLOPs | 839 TFLOPs | 383 TFLOPs | N/A | 624 TFLOPs | 1 979 TFLOPs | 280 TFLOPs | 7 500 TFLOPs |
| TF32 | N/A | N/A | N/A | N/A | 312 TFLOPs | 989 TFLOPs | N/A | N/A |
| BFLOAT16 | 256 GFLOPs | ~ 800 TFLOPs * | 383 TFLOPs | N/A | 624 TFLOPs | 1 513 TFLOPs | 123.13 TFLOPs | ~ 7000 TFLOPs * |
| INT8 | 512 GOPs | 1678 TOPs | 383 TOPs | N/A | 1 248 TOPs | 3 026 TOPs | 123 TOPs | N/A |
Intel Flex graphics accelerators are a new line of datacenter cards from Intel. While they’re aimed mostly at cloud gaming and computing, Intel also supplies libraries for AI applications, where it doesn’t perform too badly for its price. Of course, that’s not their primary strength, as Intel Flex cards are off primarily in video operations like encoding, scaling, compression, and more.
Intel is the first company to bring a graphics accelerator to market with dedicated AV1 encoding. This is quite a breakthrough, as the AV1 video codec has many times better compression ratios than AMD and NVIDIA’s H.264. Unlike H.265 and HEVC, it is also freely available without licensing.
Intel has decided to take a slightly unconventional step and is also pushing its Intel Flex DC GPU cards in the “cloud gaming” direction in marketing. Since this business sector (streaming games to the player’s device, but all computation and rendering is done remotely on the server) has grown quite a bit in the last couple of years, it’s possible that Intel will succeed here with its cards. The main advantage for this use case is mostly the already mentioned video encoding, as this is where the biggest loss in performance and quality is, even with faster transfers. Furthermore, Intel Flex cards can offer
up to 32 Ray Tracing cores, relatively high clock speeds (2000+ MHz) per core, and plenty of GDDR6 graphics memory; 16GB for the Flex 170 and 12GB for the weaker Flex 140. Intel boasts in its specs that with this performance, a single Flex 170 card can stream up to 68 720p30 games at once.
Although Intel Flex DC GPU cards are mainly designed for cloud computing and media processing, Intel has created an environment for AI applications with several libraries and instruction sets. The Flex 140 and 170 cards support most of the most widely used AI libraries such as TensorFlow, PyTorch, and OpenVINO. Intel also mentions that AI applications should be able to scale seamlessly from Xeon processors directly to Flex cards, which could result in quite a performance lift for applications, and full Intel solutions designed for less compute-intensive computing could be a very interesting option.
Intel’s new division, Intel Habana Labs, which it bought in 2019, focuses purely on SXM cards for AI acceleration. For the latest Gaudi2, which is built on 7nm technology compared to the previous 16nm Gaudi, Intel says the card should have up to twice the throughput and performance of the NVIDIA A100.
The use of artificial intelligence has expanded into almost every industry in recent years. Habana Labs wants to make this more accessible and accelerate the process, and has therefore developed a second generation of Accelerators for a variety of AI industries, but basically for Image recognition or NLP, for example. In the mentioned areas we have a comparison of the theoretical TensorFlow performance against NVIDIA accelerators:
Habana products have a simple and effective solution for scaling training and compute capacity on Gaudi2 by integrating 24 100-Gigabit RDMA over Converged Ethernet (RoCE2) ports on the chip card, more than double the number of the first generation. 21 ports on each Gaudi2 processor are dedicated to interconnects between up to seven other processors in an all-to-all star topology across the server. The remaining three ports are for scalable output that can reach up to 2.4 Tbps for an entire eight-card system (HLS-Gaudi2). To facilitate the design of more complex solutions, Habana offers a complete eight-chip solution directly on a single board. Through the industry standard RoCE, end customers can create Gaudi2 systems exactly to their needs for a DL cluster and scale from one to thousands of Gaudi2.
This newly announced accelerator is being developed by AMD for the purpose of training artificial intelligence, as the market is currently booming in this sector. The MI 300X accelerators are intended to be a direct competitor to NVIDIA’s H100 accelerators. The MI 300X cards are composed of 12 chipsets that are built on a 5nm/6nm manufacturing process. Overall, we can notice that the MI 300X cards are essentially MI 300A APUs without the “APU” – that is, they are a standalone card. The MI 300X is also intended to compete with the H100 cards with its huge memory capacity of 192GB of HBM3. The card is expected to be launched in Q3 2023 and commercially deployed in the planned exa-scale El Capitan supercomputer.
AMD’s Instinct MI 2xx accelerators are positioned as a direct competitor to NVIDIA’s A100 SXM cards and are built on the new CDNA2 architecture, which is an enhancement to the previous, proven CDNA architecture. Compared to it, it has improved efficiency and power consumption relative to performance, but the architecture itself has also received minor and major modifications. One of them, for example, is the “Elevated Fanout Bridge 2.5D”, which is an alternative type of SMD interconnect between the accelerator core, memory and other components on the card. First of all, it saves manufacturing costs as it is made from more affordable substrates, but it is also much easier to manufacture. Furthermore, EFB2.5D makes it much easier to scale the hardware.
The world’s most powerful supercomputer, Frontier is built on 3rd generation AMD EPYC processors and AMD Instinct MI250 accelerators in an HP Enterprise chassis. With a configuration of four of these accelerators per compute node and AMD Infinity Fabric interconnects, Frontier achieves a peak performance of 1.6 ExaFLOP, making it the world’s first Exa-scale supercomputer.
With the help of several new features and technologies from AMD, the technology solutions with 3rd generation AMD EPYC processors and AMD Instinct accelerators can get the most out of the x86 architecture. 7nm SoC systems provide up to double the performance of previous generations, resulting in up to four times the TFLOPs in AI applications, so that MI250X accelerators have a theoretical point-to-point throughput of around 800 GB/s on their I/O interface.
The NVIDIA A100 TC GPU (Tensor Core GPU) accelerator provides the acceleration needed on any scaled system for flexible AI, data analytics, and HPC development and training solutions. The A100 card is built on the Ampere architecture, which delivers up to 20x more performance over the previous generation in some applications. With 80GB of HBM2e memory and a bitrate of over 2TBps, it is the ideal choice for running the largest models and datasets. In contrast, NVIDIA’s H100 is not a replacement, but a complement to NVIDIA’s version; it is built on the newer 4nm Hopper architecture versus 7nm Ampere. Performance has more than doubled, plus it has much higher efficiency thanks to a smaller manufacturing process. With the new NVIDIA NVLink Switch system, up to 256 H100 cards can also be clustered, which can bring exponential speedups in NLP or any AI application, for example.
Both cards are a great choice for neural network training solutions for any sector of AI, data analytics, data processing or other applications that work with huge data sets, for which the fastest possible interconnection between cards and high memory capacity is essential. In addition, the NVIDIA H100 features fourth-generation Tensor cores and the Transformer Engine with FP8 precision, which can achieve up to nine times the training speed on MoE models. These parameters, plus fourth-generation NVLink, which has a data throughput of up to 900 GBps for inter-GPU connections, the NVLINK Switch system that accelerates communication between each individual card in a node, PCIe 5th generation or SXM5, and NVIDIA MagnumIO software, can provide efficient scaling from smaller industries to massive, unified GPU clusters.
The Technology Research Centre of the University of West Bohemia in Pilsen together with TechSoft, supported by M Computers, carried out extensive testing of NVIDIA graphics accelerators in the Ansys Fluent R1 simulation program. Some of the comparisons and results are shown below:
We would like to thank our colleagues Jindřich Kňourek from ZČU and Richard Paulas from TechSoft engineering s.r.o. for excellent processing and presentation of the data obtained with the help of solutions supplied by M Computers s.r.o.!
The Graphcore C600 accelerator is an IPU processor PCIe card with high performance for machine learning. The card is built on the new Mk2 architecture, which adds support for FP8 precision, for example, which can further accelerate some AI calculations. The card generally serves as the aforementioned IPU, Infrastructure Processing Unit, which is a card that mainly takes care of lightening the load of the CPU and preprocessing some data that would otherwise take the CPU many times longer. “Fun fact”; if you think Graphcore’s PCIe accelerators look rather unconventional, you’re absolutely right; their design was engineered by an AI algorithm The card is made in a dual-slot, full-height format and can be deployed in standard server chassis, typically two to 2U servers and up to eight to 4U.
The IPU-Link technology was developed by Graphcore primarily for interconnecting Accelerators to each other, it is a similar principle to e.g. NVLink. However, IPU-Link has the ability to connect up to eight cards, improving the performance and scaling of solutions based on Graphcore accelerators.
Graphcore’s IPU architecture has another unique feature, which is memory integrated directly into the core of the computing unit. It can be compared to the Lx cache and also has similar advantages – great response, data flow and speed. In the Mk2 architecture, each IPU core has 900 MB of its own memory, + up to 448 GB of streaming/system memory that acts as a buffer.
In the case of Cerebras, this is completely unique – a chip with an area of 46.225 mm2, which is more than 50 times larger than the largest graphics chip. In such a large wafer, 850,000 cores have been deployed with a 7nm process, totalling over 2.6 trillion transistors. The WSE (Wafer Scale Engine) concept, more or less the entire compute unit of a supercomputer on one massive chip, has countless advantages over clustered systems, the main one being latency and data flow between cores. Since they are all printed on a single surface, the communication paths between units are completely minimal, compared to Fabric interconnects in traditional systems, which restrict the flow and slow it down slightly.
https://mcomputers.cz/wp-content/uploads/2025/02/1-scaled.jpg
1712
2560
Veronika Pantůčková
https://mcomputers.cz/wp-content/uploads/2024/12/obrazek_2024-12-03_164614578-e1733305217994.png
Veronika Pantůčková2025-02-20 13:09:552025-02-20 14:19:55We held an NVIDIA Omniverse workshop
https://mcomputers.cz/wp-content/uploads/2024/07/nvidia-h200.jpg
675
1200
Jitka Bláhová
https://mcomputers.cz/wp-content/uploads/2024/12/obrazek_2024-12-03_164614578-e1733305217994.png
Jitka Bláhová2025-02-05 14:12:082025-02-10 08:42:28Comment on the situation around NVIDIA chips in the Czech Republic
https://mcomputers.cz/wp-content/uploads/2025/01/Srdce_sek_predani.jpg
539
907
Jitka Bláhová
https://mcomputers.cz/wp-content/uploads/2024/12/obrazek_2024-12-03_164614578-e1733305217994.png
Jitka Bláhová2025-01-31 12:00:572025-01-31 14:50:26Together with our customers we help children: the Christmas encryption and support of Srdce na dlani
https://mcomputers.cz/wp-content/uploads/2025/01/07_ThinkPad_x9_14.png
986
1536
Jitka Bláhová
https://mcomputers.cz/wp-content/uploads/2024/12/obrazek_2024-12-03_164614578-e1733305217994.png
Jitka Bláhová2025-01-14 16:16:442025-01-22 10:56:46Lenovo at CES 2025: Revolutionary technology for the office and gaming
![](https://mcomputers.cz/wp-content/uploads/2024/09/Logo-Facebook-128x.png"){kind=logo}
Ansys User Conference 2023