NVIDIA DGX Systems
The World’s First Portfolio of Purpose-Built AI Supercomputers
Developed to meet the demands of AI and analytics, NVIDIA DGX™ Systems are built on the revolutionary NVIDIA Volta™ GPU platform. Combined with innovative GPU-optimized software and simplified management tools, these fully-integrated solutions deliver groundbreaking performance and results. NVIDIA DGX Systems are designed to give data scientists the most powerful tools for AI exploration—from your desk to the data center to the cloud.
NVIDIA DGX A100
With the fastest I/O architecture, NVIDIA DGX A100 is the universal system for all AI infrastructure, from analytics to training to inference. It sets a new bar for compute density, packing 5 petaFLOPS of AI performance with a single, unified system that can do it all.
NVIDIA DGX Station A100
NVIDIA DGX Station A100 brings AI supercomputing to data science teams, offering data center technology without a data center or additional IT infrastructure. Designed for multiple, simultaneous users with server-grade components in an office-friendly form factor.
IS YOUR DATA CENTER READY?
Scale up with DGX SuperPOD
Get your data center infrastructure ready for AI
The NVIDIA DGX SuperPOD™ with NVIDIA DGX™ A100 systems is the next generation artificial intelligence (AI) supercomputing infrastructure, providing the computational power necessary to train today’s state-of-the-art deep learning (DL) models and to fuel innovation well into the future. The DGX SuperPOD delivers groundbreaking performance and is designed to solve the world’s most challenging computational problems.
NVIDIA DGX SuperPOD™ is a reference architecture that incorporates best practices for compute, networking, storage, power, cooling, and more, in an integrated AI infrastructure design built on NVIDIA DGX A100. Most data centers aren’t designed with the growth and unique demands of AI in mind but AMAX can customize and deploy NVIDIA DGX POD for your data center to optimize infrastructure in meeting the rising tide of AI-infused applications.
The DGX A100 system supports Multi Instance GPU (MIG) partitioning of each DGX A100 GPU. This feature can enhance the productivity of the DGX SuperPOD by:
- Providing AI research teams the ability to efficiently run thousands of smaller experiments in isolation for each other.
- Providing enhanced AI inference by supporting thousands of simultaneous inference processes.
|Compute Nodes||NVIDIA DGX A100 System||1,120 DGX A100 SXM4 GPUs|
45.6 TB of HBM2 memory
366 PFLOPS via Tensor Cores
140 TB System RAM
2.2 PB local NVMe
600 GBps NVLink bandwidth per GPU
4.8 TBps total NVSwitch bandwidth per node
|Compute Network||NVIDIA Mellanox Quantum QM8790 HDR InfiniBand Smart Switch||Full fat-tree network built with eight connections per DGX A100 system|
|Storage Network||NVIDIA Mellanox Quantum QM8790 HDR InfiniBand Smart Switch||Fat-tree network with two connections per DGX A100 system|
|In-band Management Network||NVIDIA Mellanox SN3700C switch||One connection per DGX A100 system|
|NVIDIA Mellanox AS4610 switch||One connection per DGX A100 system|
|Management Software||DeepOps DGX POD Management Software||Software tools for deployment and management of SuperPOD nodes and resource management|
|Key System Software||NVIDIA Magnum IO™ Technology|
NVIDIA CUDA-X™ technology
|A collection of libraries, tools, and technologies that maximize application performance on NVIDIA GPUs|
Suite of library technologies that optimize GPU communication performance
|User Runtime Environment||NGC|
|Containerized DL and HPC applications, optimized for performance|
Orchestration and scheduling of multi-GPU and multi-node jobs
The NVIDIA A100 Tensor Core
Delivers unprecedented acceleration at every scale for AI, data analytics, and high-performance computing (HPC) to tackle the world’s toughest computing challenges. As the engine of the NVIDIA data center platform, A100 can efficiently scale to thousands of GPUs or, with NVIDIA Multi-Instance GPU (MIG) technology, be partitioned into seven GPU instances to accelerate workloads of all sizes. And third-generation Tensor Cores accelerate every precision for diverse workloads, speeding time to insight and time to market.
NVIDIA AMPERE ARCHITECTURE
A100 accelerates workloads big and small. Whether using MIG to partition an A100 GPU into smaller instances, or NVLink to connect multiple GPUs to accelerate large-scale workloads, A100 can readily handle different-sized acceleration needs, from the smallest job to the biggest multi-node workload. A100’s versatility means IT managers can maximize the utility of every GPU in their data center around the clock.
THIRD-GENERATION TENSOR CORES
A100 delivers 312 teraFLOPS (TFLOPS) of deep learning performance. That’s 20X Tensor FLOPS for deep learning training and 20X Tensor TOPS for deep learning inference compared to NVIDIA Volta™ GPUs.
NVIDIA NVLink in A100 delivers 2X higher throughput compared to the previous generation. When combined with NVIDIA NVSwitch™, up to 16 A100 GPUs can be interconnected at up to 600 gigabytes per second (GB/sec) to unleash the highest application performance possible on a single server. NVLink is available in A100 SXM GPUs via HGX A100 server boards and in PCIe GPUs via an NVLink Bridge for up to 2 GPUs.
MULTI-INSTANCE GPU (MIG)
An A100 GPU can be partitioned into as many as seven GPU instances, fully isolated at the hardware level with their own high-bandwidth memory, cache, and compute cores. MIG gives developers access to breakthrough acceleration for all their applications, and IT administrators can offer right-sized GPU acceleration for every job, optimizing utilization and expanding access to every user and application.
With 40 gigabytes (GB) of high-bandwidth memory (HBM2), A100 delivers improved raw bandwidth of 1.6TB/sec, as well as higher dynamic random-access memory (DRAM) utilization efficiency at 95 percent. A100 delivers 1.7X higher memory bandwidth over the previous generation.
AI networks are big, having millions to billions of parameters. Not all of these parameters are needed for accurate predictions, and some can be converted to zeros to make the models “sparse” without compromising accuracy. Tensor Cores in A100 can provide up to 2X higher performance for sparse models. While the sparsity feature more readily benefits AI inference, it can also improve the performance of model training.