GPU machine types  |  Compute Engine Documentation  |  Google Cloud (2024)

You can use GPUs on Compute Engine to accelerate specific workloads onyour VMs such as machine learning (ML) and data processing. To use GPUs, youcan either deploy an accelerator-optimized VM that has attached GPUs, orattach GPUs to an N1 general-purpose VM.

Compute Engine provides GPUs for your VMs in passthrough mode so thatyour VMs have direct control over the GPUs and their associated memory.

For more information about GPUs on Compute Engine, seeAbout GPUs.

If you have graphics-intensive workloads, such as 3D visualization,3D rendering, or virtual applications, you can use NVIDIA RTX virtualworkstations (formerly known as NVIDIA GRID).

This document provides an overview of the different GPU VMs that areavailable on Compute Engine.

To view available regions and zones for GPUs on Compute Engine, seeGPUs regions and zone availability.

GPUs for compute workloads

For compute workloads, GPUs are supported for the following machine types:

  • A3 VMs: these VMs have NVIDIA H100 80GB GPUs automatically attached.
  • A2 VMs: these VMs have either NVIDIA A100 80GB or NVIDIA A100 40GBGPUs automatically attached.
  • G2 VMs: these VMs have NVIDIA L4 GPUs automatically attached.
  • N1 VMs: for these VMs, you can attach the following GPU models:NVIDIA T4, NVIDIA V100, NVIDIA P100, or NVIDIA P4.

A3 machine series

To run NVIDIA H100 80GB GPUs, you must use anA3 accelerator-optimizedmachine. Each A3 machine type has a fixed GPU count, vCPU count, and memory size.

A3 machine series are available in two types:

  • A3 High (a3-highgpu-8g): these machine types have H100 80GB GPUs(nvidia-h100-80gb) and Local SSD disks attached. In addition to the200Gbps of VM to VM network bandwidth for all A3 VMs, A3 High VMsprovide 800Gbps of GPU to GPU bandwidth, leading to a total maximumnetwork bandwidth speed of 1,000Gbps.
  • A3 Mega (a3-megagpu-8g): these machine types have H100 80GB Mega GPUs(nvidia-h100-mega-80gb) and Local SSD disks attached. In addition to the200Gbps of VM to VM network bandwidth for all A3 VMs, A3 Mega VMsprovide 1,600Gbps of GPU to GPU bandwidth, leading to a total maximumnetwork bandwidth speed of 1,800Gbps.
Machine type GPU count GPU memory*
(GB HBM3)
vCPU count VM memory (GB) Attached Local SSD (GiB) Maximum network bandwidth (Gbps)
VM to VM GPU cluster
a3-highgpu-8g 8 640 208 1,872 6,000 200 800
a3-megagpu-8g 8 640 208 1,872 6,000 200 1,600

*GPU memory is the memory that is available on a GPU devicethat can be used for temporary storage of data. It is separate from the VM'smemory and is specifically designed to handle the higher bandwidth demands ofyour graphics-intensive workloads.

A2 machine series

To use NVIDIA A100 GPUs onGoogle Cloud, you must deploy anA2 accelerator-optimizedmachine. Each A2 machine type has a fixed GPU count, vCPU count, and memory size.

A2 machine series are available in two types:

  • A2 Standard: these machine types have A100 40GB GPUs (nvidia-tesla-a100)attached.
  • A2 Ultra: these machine types have A100 80GB GPUs (nvidia-a100-80gb) andLocal SSD disks attached.

A2 Standard

Machine type GPU count GPU memory* (GB HBM2) vCPU count VM memory (GB) Local SSD supported Maximum network bandwidth (Gbps)
a2-highgpu-1g 1 40 12 85 Yes 24
a2-highgpu-2g 2 80 24 170 Yes 32
a2-highgpu-4g 4 160 48 340 Yes 50
a2-highgpu-8g 8 320 96 680 Yes 100
a2-megagpu-16g 16 640 96 1,360 Yes 100

A2 Ultra

Machine type GPU count GPU memory* (GB HBM2e) vCPU count VM memory (GB) Attached Local SSD (GiB) Maximum network bandwidth (Gbps)
a2-ultragpu-1g 1 80 12 170 375 24
a2-ultragpu-2g 2 160 24 340 750 32
a2-ultragpu-4g 4 320 48 680 1,500 50
a2-ultragpu-8g 8 640 96 1,360 3,000 100

*GPU memory is the memory that is available on a GPU devicethat can be used for temporary storage of data. It is separate from the VM'smemory and is specifically designed to handle the higher bandwidth demands ofyour graphics-intensive workloads.

G2 machine series

To use NVIDIA L4 GPUs(nvidia-l4 or nvidia-l4-vws), you must deploy aG2 accelerator-optimizedmachine.

Each G2 machine type has a fixed number of NVIDIA L4 GPUsand vCPUs attached. Each G2 machine type also has a default memory and a custommemory range. The custom memory range defines the amount of memory thatyou can allocate to your VM for each machine type. You can specify your custommemory during VM creation.

Machine type GPU count GPU memory* (GB GDDR6) vCPU count Default VM memory (GB) Custom VM memory range (GB) Max Local SSD supported (GiB) Maximum network bandwidth (Gbps)
g2-standard-4 1 24 4 16 16 to 32 375 10
g2-standard-8 1 24 8 32 32 to 54 375 16
g2-standard-12 1 24 12 48 48 to 54 375 16
g2-standard-16 1 24 16 64 54 to 64 375 32
g2-standard-24 2 48 24 96 96 to 108 750 32
g2-standard-32 1 24 32 128 96 to 128 375 32
g2-standard-48 4 96 48 192 192 to 216 1,500 50
g2-standard-96 8 192 96 384 384 to 432 3,000 100

*GPU memory is the memory that is available on a GPU devicethat can be used for temporary storage of data. It is separate from the VM'smemory and is specifically designed to handle the higher bandwidth demands ofyour graphics-intensive workloads.

N1 machine series

You can attach the following GPU models to anN1 machine type with theexception of the N1 shared-core machine type.

N1 VMs with lower numbers of GPUs are limited to a maximum number of vCPUs.In general, a higher number of GPUs lets you create VM instances with a highernumber of vCPUs and memory.

N1+T4 GPUs

You can attach NVIDIA T4GPUs to N1 general-purpose VMs with the following VM configurations.

Accelerator type GPU count GPU memory* (GB GDDR6) vCPU count VM memory (GB) Local SSD supported
nvidia-tesla-t4 or
nvidia-tesla-t4-vws
1 16 1 to 48 1 to 312 Yes
2 32 1 to 48 1 to 312 Yes
4 64 1 to 96 1 to 624 Yes

*GPU memory is the memory that is available on a GPU devicethat can be used for temporary storage of data. It is separate from the VM'smemory and is specifically designed to handle the higher bandwidth demands ofyour graphics-intensive workloads.

N1+P4 GPUs

You can attachNVIDIA P4GPUs to N1 general-purpose VMs with the following VM configurations.

Accelerator type GPU count GPU memory* (GB GDDR5) vCPU count VM memory (GB) Local SSD supported
nvidia-tesla-p4 or
nvidia-tesla-p4-vws
1 8 1 to 24 1 to 156 Yes
2 16 1 to 48 1 to 312 Yes
4 32 1 to 96 1 to 624 Yes

*GPU memory is the memory that is available on a GPU devicethat can be used for temporary storage of data. It is separate from the VM'smemory and is specifically designed to handle the higher bandwidth demands ofyour graphics-intensive workloads.

For VMs with attached NVIDIA P4 GPUs, Local SSD disksare only supported in zones us-central1-c andnorthamerica-northeast1-b.

N1+V100 GPUs

You can attachNVIDIA V100GPUs to N1 general-purpose VMs with the following VM configurations.

Accelerator type GPU count GPU memory* (GB HBM2) vCPU count VM memory (GB) Local SSD supported
nvidia-tesla-v100 1 16 1 to 12 1 to 78 Yes
2 32 1 to 24 1 to 156 Yes
4 64 1 to 48 1 to 312 Yes
8 128 1 to 96 1 to 624 Yes

*GPU memory is the memory that is available on a GPU devicethat can be used for temporary storage of data. It is separate from the VM'smemory and is specifically designed to handle the higher bandwidth demands ofyour graphics-intensive workloads.

For VMs with attached NVIDIA V100 GPUs, Local SSD disksaren't supported in us-east1-c.

N1+P100 GPUs

You can attachNVIDIA P100GPUs to N1 general-purpose VMs with the following VM configurations.

For some NVIDIA P100 GPUs, the maximum CPU and memory that is available forsome configurations is dependent on the zone in which the GPU resource is running.

Accelerator type GPU count GPU memory* (GB HBM2) vCPU count VM memory (GB) Local SSD supported
nvidia-tesla-p100 or
nvidia-tesla-p100-vws
1 16 1 to 16 1 to 104 Yes
2 32 1 to 32 1 to 208 Yes
4 64

1 to 64
(us-east1-c, europe-west1-d, europe-west1-b)

1 to 96
(all P100 zones)

1 to 208
(us-east1-c, europe-west1-d, europe-west1-b)

1 to 624
(all P100 zones)

Yes

*GPU memory is the memory that is available on a GPU devicethat can be used for temporary storage of data. It is separate from the VM'smemory and is specifically designed to handle the higher bandwidth demands ofyour graphics-intensive workloads.

NVIDIA RTX Virtual Workstations (vWS) for graphics workloads

If you have graphics-intensive workloads, such as 3D visualization, you cancreate virtual workstations that useNVIDIA RTX Virtual Workstations (vWS) (formerly known as NVIDIA GRID). When you create a virtualworkstation, an NVIDIA RTX Virtual Workstation (vWS) license is automatically addedto your VM.

For information about pricing for virtual workstations, seeGPU pricing page.

For graphics workloads, NVIDIA RTX virtual workstation (vWS) models are available:

  • G2 machine series: for G2 machine types you can enableNVIDIA L4 Virtual Workstations (vWS): nvidia-l4-vws

  • N1 machine series: for N1 machine types, you can enable the followingvirtual workstations:

    • NVIDIA T4 Virtual Workstations: nvidia-tesla-t4-vws
    • NVIDIA P100 Virtual Workstations: nvidia-tesla-p100-vws
    • NVIDIA P4 Virtual Workstations: nvidia-tesla-p4-vws

General comparison chart

The following table describes the GPU memory size, feature availability,and ideal workload types of different GPU models that are available onCompute Engine.

GPU modelGPU memoryInterconnectNVIDIA RTX Virtual Workstation (vWS) supportBest used for
H100 80GB80 GB HBM3 @ 3.35 TBpsNVLink Full Mesh @ 900 GBps Large models with massive data tables for ML Training, Inference, HPC, BERT, DLRM
A100 80GB80 GB HBM2e @ 1.9 TBpsNVLink Full Mesh @ 600 GBps Large models with massive data tables for ML Training, Inference, HPC, BERT, DLRM
A100 40GB 40 GB HBM2 @ 1.6 TBps NVLink Full Mesh @ 600 GBps ML Training, Inference, HPC
L424 GB GDDR6 @ 300 GBpsN/AML Inference, Training, Remote Visualization Workstations,Video Transcoding, HPC
T4 16 GB GDDR6 @ 320 GBps N/A ML Inference, Training, Remote Visualization Workstations, Video Transcoding
V100 16 GB HBM2 @ 900 GBps NVLink Ring @ 300 GBps ML Training, Inference, HPC
P4 8 GB GDDR5 @ 192 GBps N/A Remote Visualization Workstations, ML Inference, and Video Transcoding
P100 16 GB HBM2 @ 732 GBps N/A ML Training, Inference, HPC, Remote Visualization Workstations

To compare GPU pricing for the different GPU models and regions that areavailable on Compute Engine, see GPU pricing.

Performance comparison chart

The following table describes the performance specifications of different GPUmodels that are available on Compute Engine.

Compute performance

GPU modelFP64FP32FP16INT8
H100 80GB34 TFLOPS67 TFLOPS
A100 80GB9.7 TFLOPS19.5 TFLOPS
A100 40GB9.7 TFLOPS19.5 TFLOPS
L40.5 TFLOPS*30.3 TFLOPS
T40.25 TFLOPS*8.1 TFLOPS
V1007.8 TFLOPS15.7 TFLOPS
P40.2 TFLOPS*5.5 TFLOPS22 TOPS
P1004.7 TFLOPS9.3 TFLOPS18.7 TFLOPS

*To allow FP64 code to work correctly, a small number of FP64 hardware units are included in the T4, L4, and P4 GPU architecture.

TeraOperations per Second.

Tensor core performance

GPU modelFP64TF32Mixed-precision FP16/FP32INT8INT4FP8
H100 80GB67 TFLOPS989 TFLOPS1,979 TFLOPS*, †3,958 TOPS3,958 TFLOPS
A100 80GB19.5 TFLOPS156 TFLOPS312 TFLOPS*624 TOPS1248 TOPS
A100 40GB19.5 TFLOPS156 TFLOPS312 TFLOPS*624 TOPS1248 TOPS
L4120 TFLOPS242 TFLOPS*, †485 TOPS485 TFLOPS
T465 TFLOPS130 TOPS260 TOPS
V100125 TFLOPS
P4
P100

*For mixed precision training, NVIDIA H100, A100, and L4 GPUsalso support the bfloat16 data type.

For H100 and L4 GPUs, structural sparsity is supported which youcan use to double the performance value. The values shown are withsparsity. Specifications are one-half lower without sparsity.

What's next?

  • For more information about GPUs on Compute Engine,see About GPUs.
  • Review the GPU regions and zones availability.
  • Learn about GPU pricing.
GPU machine types  |  Compute Engine Documentation  |  Google Cloud (2024)
Top Articles
Which banks are giving the highest fixed deposit (FD) interest rates? Check here | Mint
Should you try to find out the name of your Guardian Angel? - Get Fed™
Skyward Sinton
417-990-0201
Television Archive News Search Service
O'reilly's Auto Parts Closest To My Location
Wordscapes Level 5130 Answers
Asian Feels Login
Undergraduate Programs | Webster Vienna
Chalupp's Pizza Taos Menu
35105N Sap 5 50 W Nit
Weapons Storehouse Nyt Crossword
Katie Boyle Dancer Biography
Slushy Beer Strain
Mini Handy 2024: Die besten Mini Smartphones | Purdroid.de
Patrick Bateman Notebook
Spectrum Field Tech Salary
Nesz_R Tanjiro
Nine Perfect Strangers (Miniserie, 2021)
Sprinkler Lv2
Barber Gym Quantico Hours
Scheuren maar: Ford Sierra Cosworth naar de veiling
Inbanithi Age
Lines Ac And Rs Can Best Be Described As
Disputes over ESPN, Disney and DirecTV go to the heart of TV's existential problems
Kroger Feed Login
30+ useful Dutch apps for new expats in the Netherlands
897 W Valley Blvd
Little Einsteins Transcript
Eegees Gift Card Balance
Transformers Movie Wiki
Puerto Rico Pictures and Facts
Cheap Motorcycles Craigslist
Indiana Immediate Care.webpay.md
CARLY Thank You Notes
School Tool / School Tool Parent Portal
The Transformation Of Vanessa Ray From Childhood To Blue Bloods - Looper
Pp503063
Myanswers Com Abc Resources
Bianca Belair: Age, Husband, Height & More To Know
Rs3 Bis Perks
Flags Half Staff Today Wisconsin
Qlima© Petroleumofen Elektronischer Laserofen SRE 9046 TC mit 4,7 KW CO2 Wächter • EUR 425,95
All Obituaries | Sneath Strilchuk Funeral Services | Funeral Home Roblin Dauphin Ste Rose McCreary MB
Catchvideo Chrome Extension
Vagicaine Walgreens
Stephen Dilbeck, The First Hicks Baby: 5 Fast Facts You Need to Know
Clock Batteries Perhaps Crossword Clue
Strange World Showtimes Near Atlas Cinemas Great Lakes Stadium 16
8663831604
Thrift Stores In Burlingame Ca
Latest Posts
Article information

Author: Allyn Kozey

Last Updated:

Views: 5961

Rating: 4.2 / 5 (63 voted)

Reviews: 86% of readers found this page helpful

Author information

Name: Allyn Kozey

Birthday: 1993-12-21

Address: Suite 454 40343 Larson Union, Port Melia, TX 16164

Phone: +2456904400762

Job: Investor Administrator

Hobby: Sketching, Puzzles, Pet, Mountaineering, Skydiving, Dowsing, Sports

Introduction: My name is Allyn Kozey, I am a outstanding, colorful, adventurous, encouraging, zealous, tender, helpful person who loves writing and wants to share my knowledge and understanding with you.