Hopper architecture and other news from NVIDIA GTC22

31.3.2023

As usual, we’ve been keeping an eye on the Spring 2022 NVIDIA GPU Technology Conference, and now we’re bringing you a round-up of the highlights from the virtual presentations. Perhaps the most significant news was the announcement of the new Hopper architecture and the H100 GPU accelerator based on it. Find out what’s groundbreaking about the new generation and what other systems it will be used in in the article below.

hpc06-hopper-family-gtc22-spring-1600x900

NVIDIA Hopper architecture

The newly announced Hopper architecture brings many improvements over the previous generation of Ampere. Using TSMC’s cutting-edge 4N process (an improved 4nm technology), the GH100 chip is equipped with more than 80 billion transistors and extremely fast HBM3 memory with a throughput of 3 terabytes per second.

Hopper will offer several interesting innovations, whether it is Confidential Computing, which protects data and code from leaks or disruption from the outside, or improvements in computational performance of transformer models or DPX Instructions functionality for faster and more efficient dynamic programming. And, of course, the new chip is enhanced with support for 4. generation NVLink or the fifth generation PCI Express interface (PCIe Gen5).

NVIDIA H100 SXM5

The NVIDIA H100 SMX5 accelerator, built on the NVIDIA Hopper architecture, will be available in several variants. The first is the SXM5 variant, which has the highest performance but also the highest consumption. The SXM5 version will only be available as part of GPU boards with four or eight accelerators and will not be sold as a standalone card.

80 GB HBM3 memory with throughput up to 3 TB/s
16 896 CUDAand528 Tensor cores 4. generation
theoretical performance30 TFLOPS with FP64 accuracy
theoretical performance of60 TFLOPS with FP32 accuracy and up to 1000 TFLOPS for Tensor TF32 calculations using the sparsity function
FP8 computing support with theoretical performance up to 4000 TFLOPS (with sparsity function)
4th generation NVLink with transfer speeds up to900 GB/s
passive SXM card with 700W power consumption

Webinar: News from the NVIDIA GTC22 Spring Conference

The NVIDIA GTC conference traditionally features news from the world of artificial intelligence, supercomputing and visualization. And since this year’s GTC22 was packed with technology, here is a summary of the highlights. In the second part of the webinar, we showed how to quickly and efficiently deploy AI applications, frameworks and libraries in a VMware environment and introduced the NVIDIA AI Enterprise solution on Lenovocertified servers. We also addressed the issue of sharing NVIDIA GPU accelerators between multiple users and the possibilities of connecting IT infrastructure to the NVIDIA GPU Cloud(NGC).

Download the presentation

NVLink 4. generation

The fourth generation of NVLink once again delivers higher data throughput than previous generations. The latest NVLink interface can offer bi-directional throughput of up to 900GB/s over 18 links, which is 14x more than the current PCI Express 4 interface can handle. generation (64GB/s). With the advent of the planned PCIe Gen5 (128GB/s), the lead will be slightly reduced, but the difference in data throughput will still be more than seven times. In addition, NVLink now supports the so-called. in-network computing, called SHARP, which was previously only available over the Infiniband interface, and can deliver an incredible one exaFLOP performance in AI computing with FP8 accuracy (using the sparsity feature) at an All2All throughput of 57.6 TB/s. The new NVLink Switch System will support clusters of up to 256 connected H100s and provide up to 9x more bandwidth than InfiniBand HDR with Ampere generation.

18 lines per GPU with data throughput up to 900GB/s bi-directional
4x faster than PCIe Gen4 (64GB/s)
7x faster than PCIe Gen5 (128GB/s)
new support for in-network calculations
Cluster support for up to 256connected H100s

nvlink-nvswitch-lp-social-die-ig-fb-2048x2048-1

NVIDIA H100 CNX

The converged variant of the NVIDIA H100 CNX card is based on the PCIe version of the H100 accelerator, but brings something extra. Within a single board we also find the SmartNIC chip of the latest ConnectX 7network card, which supports transfer speeds of up to 400 Gbps The two chips are interconnected via an integrated PCIe Gen5 switch, and this design is particularly useful for distributed AI computing between multiple nodes. The card will be connected via the latest PCIe Gen5 interface.

80 GBHBM3memory with throughput up to 2 TB/s
14 592 CUDA and 528 Tensor cores 4. generation
Theoretical performance of 24 TFLOPS with FP64 accuracy
Theoretical performance of 48 TFLOPS with FP32 accuracy and up to 800 TFLOPS for Tensor TF32 calculations using sparsity
FP8 computing support with theoretical performance up to 3200 TFLOPS (with sparsity function
připojení přes PCIe Gen5 s přenosovou rychlostí až 128 GB/s obousměrně
dual slot card with 350 W power consumption

NVIDIA DGX H100

The NVIDIA DGX H100 supercomputer is the successor to the flagship DGX A100 computing system. Like its predecessor, the DGX H100 includes 8 interconnected NVIDIA H100 80GB accelerators. It also retained the same 6U form factor, but our consumption almost doubled to 10.2 kW.

8x H100 80 GB HBM3 memory with up to 24 TB/s throughput and GPU memory 640GB
CPU – dual x86 (to be specified)
2 TB RAM (DDR5?)
4x NVSwitch (7,2 TB/s)
OS: 2x 1.9TB NVMe M.2
DATA: 8x 3.84TB NVMe U.2
8x single-port ConnectX-7 (400Gb/s InfiniBand/Ethernet)
2x dual-port BlueField-3 DPU VPI (400Gb/s IB/Eth)
10Gb/s onboard NIC (RJ45) (optional 50Gb/s Eth)
Management (BMC) (RJ45)
2x NVIDIA BlueField-3 DPU BMC (RJ45)
Consumption max. 10,2kW
FP8 computing support with theoretical performance up to 32 petaFLOPS (with sparsity function)

Comparison of Hopper and Ampere performance

Introduction by Jensen Huang at the spring GTC22

If you missed Jensen Huang’s keynote live during GTC22, you can at least watch it in the video on the right.

His traditional keynote was literally packed with news this time, whether it was announcements in hardware and the new Hopper architecture or in the Omniverse, a digital metaverse that is gaining more and more importance.

At the end of the keynote, we also enjoyed a digital concert with a beautiful animation of the data center inside Nvidia’s California headquarters, the Endeavour Building.

NVIDIA RTX A5500

The NVIDIA RTX A5500, the latest and youngest member of the NVIDIA Ampere family, features the latest generation of NVIDIA RT, Tensor and CUDA cores for the most demanding graphics computing and AI acceleration. The RTX A5500 GPU card is ideal for running high-performance virtual workstations (vWS) with professional visualization applications or in combination with NVIDIA Virtual PC (vPC) software for multimedia virtual desktops. It also supports deep and machine learning computing, video encoding, cloud gaming, AI audio and video effects, rendering, data analysis, and many other tasks.

24 GB GDDR6 ECC memory with throughput up to 768 GB/s
10 240 CUDA and 320 Tensor cores 3. generation
80 RT cores 2. generation with 66,6 TFLOPS
Theoretical performance of 34.1 TFLOPS with FP32 accuracy and up to 272.8 TFLOPS for Tensor TF32 calculations using sparsity
PCIe Gen4connection with transfer speeds up to 64 GB/s
4x display port, vGPUsupport
active cooling with 230Wpower consumption

NVIDIA Omniverse

During his opening remarks, Jensen Huang once again showed us a small virtual replica of himself named Toy Jensen, which was created on the new Omniverse platform. Omniverse Avatar is an AI-powered platform for developing interactive avatars that combines NVIDIA’s core AI technologies – NVIDIA Riva for speech AI, NVIDIA Metropolis for computer vision, NVIDIA NeMo Megatron for spoken language understanding, and NVIDIA Merlin for best-fit answer search and recommendation. It should be used primarily in communication with the customer during the ordering of products or solving user problems.

Another part of the Omniverse virtual universe is the so-called Omniverse Replicator, which can simulate natural physical conditions to provide valid synthetic data for training artificial intelligence models. This direction will be used mainly in the segment of autonomous vehicles and robotics. However, extremely complex systems can also be simulated in the Omniverse, as demonstrated by the ability to create a realistic model of production factories or power plants, a climatic image of a particular region, and NVIDIA even plans to create a faithful digital clone of the entire planet Earth, which will be called E-2 (Earth Two).

NVIDIA Omniverse is an end-to-end team collaboration and simulation platform that revolutionizes complex development and design workflows. It’s a virtual 3D world where you can collaborate remotely with other team members in real time to easily create complex, physically accurate simulations, images or videos. NVIDIA Omniverse Enterprise enables 3D production teams – which are often large and geographically dispersed – to collaborate seamlessly on complex projects. Rather than complicated in-person meetings or the exchange and iteration of huge files, designers, artists and testers can work simultaneously in a virtual world from anywhere and on any device.

M Computers represents NVIDIA on the Czech market in the Enterprise area of computing accelerators and AI systems.

We were the first company in Central and Eastern Europe to receive the highest ELITE PARTNER status, as well as NVIDIA AI Innovator and NVIDIA AI Champion awards.

Testing

We have a wide range of NVIDIA GPU cards available for performance testing and heavy-duty processing.
If you are interested in our testing offer, please fill out this form.

Kamila Jeřábková

NVIDIA, Lenovo ISG

+420 734 161 516

kamila.jerabkova@mcomputers.cz