NVIDIA GPU Guide for Data Centres

NVIDIA introduces the most powerful graphics accelerators on the market. These cards can accelerate many massively parallel tasks
and scientific (HPC) applications or efficiently solve artificial intelligence (AI) algorithms. We’ve put together a guide to choosing NVIDIA GPU cards.

GPU	L4	A16	A40	L40S	A100 SXM4 \| PCIe	H100 PCIe	H100 \| H200 SXM5	H100 NVL	B200
Architecture	Ada Lovelace	Ampere	Ampere	Ada Lovelace	Ampere	Hopper	Hopper	Hopper	Blackwell
Card chip	AD104	GA107	GA102	AD102	GA100	GH100	GH100	GH100	B200
# CUDA cores	7 680	4x 1 280	10 752	18 176	6 912	14 592	16 896	16 896	TBA
# Tensor cores	240	4x 40	336	568?	432	456	528	528	TBA
FP64 (TFlops)	0,49	0,271	1,179	1,413?	9,69	26	34	30	TBA
FP64 Tensor (TFlops)	—	—	—	—	19,5	51	67	60	40
FP32 (TFlops)	30,3	4x 4,5	37,4	91,6	19,5	51	67	60	80
TF32 Tensor (TFlops)	120*	4x 18*	150*	366*	312*	756*	989*	835*	2 200
FP16 Tensor (TFlops)	242*	4x 35,9*	299*	733*	624*	1 513*	1 979*	1 671*	4 500
INT8 Tensor (TOPS)	FP8 485*	4x 71,8*	599*	1466*	1 248*	3 026*	3 958*	3 341*	9 000
GPU memory	24 GB	4x 16 GB	48 GB	48 GB	80 \| 40 GB	80 GB	80 \| 141 GB	94 GB	192 GB
Memory technology	GDDR6	GDDR6	GDDR6	GDDR6	HBM2	HBM3	HBM3	HBM3e	HBM3e
Memory throughput	300 GB/s	4x 200 GB/s	696 GB/s	864 GB/s	1 935 \| 2 039 GB/s	2 TB/s	3,3 \| 4.8 TB/s	3.9 TB/s	8 TB/s
Multi-Instance GPU	vGPU	vGPU	vGPU	vGPU	7 instances	7 instances	7 instances	7 instances	TBA
NVENC \| NVDEC \| JPEG engines	2 \| 4 \| 4	4 \| 8	1 \| 2	3 \| 3 \| 4	0 \| 5 \| 5	0 \| 7 \| 7	0 \| 7 \| 7	0 \| 7 \| 7	TBA
GPU link	PCIe 4	PCIe 4	NVLink 3	PCIe 4	NVLink 3	NVLink 4	NVLink 4	NVLink 4	NVLink 5
Power consumption	40-72W	250 W	300 W	350 W	400W \| 300W	350W	700W	400 W	1000W
Form factor	PCIe gen4 1-slot LP	PCIe gen4 2-slot FHFL	PCIe gen4 2-slot FHFL	PCIe gen4 2-slot FHFL	SXM4 \| PCIe gen4 2-slot FHFL	PCIe gen5 2-slot FHFL	SXM5 card	PCIe gen5 2-slot FHFL	SXM5 card
Spec sheet	spec sheet	spec sheet	spec sheet	spec sheet	spec sheet	spec sheet	spec sheet	spec sheet	N/A
Announcement	2023	2021	2020	2023	2020	2022	2022 \| 2023	2024	2024
Availability
GPU	L4	A16	A40	L40S	A100 SXM4 \| PCIe	H100 PCIe	H100 \| H200 SXM5	H100 NVL	B200

* the listed performance is for sparse matrix computations (Sparcity), for standard computations the performance is half of the listed values
** NVIDIA A100 PCIe achieves 90% of the listed computational performance

Detailed guide to choosing an NVIDIA GPU

At AIserver.eu we have created a detailed guide to NVIDIA GPUs. For a quick introduction to NVIDIA cards, the Quick guide. Expert guide contains a comparison of all card parameters. Application benchmarks or parameters of older cards are available.

Detailed guide on AIserver.eu

GPU Accelerators for data centres

NVIDIA Tesla and Ampere graphics accelerators are designed to accelerate HPC applications or deploy artificial intelligence and deep learning algorithms.

Key benefits of NVIDIA cards include dedicated Tensor cores for machine learning applications. And large memory (up to 80 GB per accelerator), secured by ECC technology. To enable the accelerators to communicate quickly with each other, NVIDIA has connected them with a special interface with huge data throughput – NVLink. NVLink achieves transfer rates of up to 600 GB/s. In addition, the NVIDIA DGX A100 offers a super-powerful NVSwitch. This provides a total throughput between eight NVIDIA Ampere A100 cards of up to 4.8 TB/s.

Intersect360 Research analysis shows that most of the most widely used HPC applications already support NVIDIA cards. These include These include GROMACS, Ansys Fluent, Gaussian, VASP, NAMD, Abaqus, OpenFoam, LS Dyna, BLAST, Amber, Gamess, ParaView, NASTRAN and many others. The large expansion of NVIDIA accelerators has been helped by support for deep learning frameworks – TensorFlow, Caffe, PyTorch, MXNET, Chainer, Keras and many more.

The graph on the right shows how fast the graphics accelerator field is evolving, with a ninefold increase in performance in just four years. The figures are based on the average of benchmark results for the most widely used AI and HPC applications (Amber, Chroma, GROMACS, MILC, NAMD, PyTorch, Quantum Espresso, TensorFlow and VASP), measured on dual-socket servers with four P100, V100 or A100 accelerators each.

NVIDIA GP GPU Performance Evolution in Time

How to choose the best GPU?

The current GP GPU cards for data centers and their typical deployments are mentioned in the infographics.

NVIDIA visualization cards

NVIDIA RTX professional cards are designed primarily for graphics processing and simulation, machine learning, data analytics, and high-performance workstation virtualization.

Comparison of Nvidia cards for visualization

Parametr	RTX A2000	RTX A4000	RTX A4500	RTX A5000	RTX A5500	RTX A6000	RX 4000 SFF Ada	RTX 4000 Ada	RTX 4500 Ada	RTX 5000 Ada	RTX 6000 Ada
Architektura	Ampere	Ampere	Ampere	Ampere	Ampere	Ampere	Lovelace	Lovelace	Lovelace	Lovelace	Lovelace
Čip karty	GA 106	GA 104	GA 102	GA 102	GA 102	GA 102	AD 103	AD 103	AD 103	AD 103	AD 102
# CUDA jader	3 328	6 144	7 168	8 192	10 240	10 752	6 144	6 144	7 680	12 800	18 176
# Tensor jader	104	192	224	256	320	336	192	192	240	400	568
FP64 (TFlops)	0,124	0,6	0,739	0,87	1,085	1,25	0.296	0.417	0.6192	1.020	1.423
FP32 (TFlops)	8	19,2	23,65	27,7	34,1	40	19.2	26.73	39.63	65.28	91.1
FP16 Tensor (TFlops)	63,9*	153,4*	189,2	222,2*	272,8*	309,7*	307	327.6*	634*	1 044.4*	1 457*
GPU paměť	6 / 12 GB	16 GB	20 GB	24 GB	24 GB	48 GB	20 GB	20 GB	24 GB	32 GB	48GB
Typ Paměti	GDDR6	GDDR6	GDDR6	GDDR6	GDDR6	GDDR6	GDDR6	GDDR6	GDDR6	GDDR6	GDDR6
Propustnost pamětí	288 GB / s	448 GB / s	640 GB / s	768 GB / s	768 GB / s	768 GB / s	280 GB/s	360 GB/s	432 GB/s	576 GB/s	960 GB/s
ECC paměti	ECC	ECC	ECC	ECC	ECC	ECC	Ano	Ano	Ano	Ano	Ano
Propojení GPU	PCIe gen4	PCIe gen4	NVLink 2-way	NVLink 2-way	NVLink 2-way	NVLink 2-way	-	-	-	-	-
Max. příkon	70 W	140 W	200 W	230 W	230 W	300 W	70 W	130 W	210 W	250 W	300 W
Provedení	PCIe gen4	PCIe gen4	PCIe gen4	PCIe gen4	PCIe gen4	PCIe gen4	PCIe 4.0	PCIe 4.0	PCIe 4.0	PCIe 4.0	PCIe 4.0
Pro datacentra**	Ano	Ano	Ano	Ano	Ano	Ano	Ano	Ano	Ano	Ano	Ano
Oznámení	2021	2021	2021	2021	2022	2020	2023	2023	2023	2023	2022
KARTA	RTX A2000	RTX A4000	RTX A4500	RTX A5000	RTX A5500	RTX A6000	RX 4000 SFF Ada	RTX 4000 Ada	RTX 4500 Ada	RTX 5000 Ada	RTX 6000 Ada
	Datasheet	Datasheet	Datasheet	Datasheet	Datasheet	Datasheet	Datasheet	Datasheet	Datasheet	Datasheet	Datasheet

* the stated power is for calculations with sparse nuts (Sparsity), for standard calculations the power is half of the specified values

** according to the Nvidia Card Driver License Agreement (EULA), GeForce graphics cards (GTX, RTX) are not intended for data centers:
“No Datacenter Deployment. The SOFTWARE is not licensed for datacenter deployment, except that blockchain processing in a datacenter is permitted.”
source: https://www.nvidia.com/content/DriverDownload-March2009/licence.php?lang=us&type=GeForce

EDU Prices

NVIDIA offers special pricing and project-specific programs on both GPU and DGX systems, plus support for educational institutions (EDUs) and start-ups.

Get an offer

NVIDIA B200

NVIDIA Blackwell chips are the next most powerful generation of accelerators for dramatically accelerating AI and HPC projects, improving efficiency and scalability. The Blackwell platform offers the largest physical chip size possible, which also leads to huge transistor density and unparalleled data throughput.

More information

DGX B200

Testing

We have a wide range of NVIDIA GPU cards available for performance testing and heavy-duty processing.
If you are interested in our testing offer, please fill out this form.