GPU Specifications / NVIDIA
NVIDIA V100 SXM2 32GB
Volta datacenter GPU with 32 GB of HBM2 memory, 900 GB/s of bandwidth and up to 125 TFLOPS of FP16 tensor compute.
Key Specifications
Memory
32 GB HBM2
Memory Bandwidth
900 GB/s
TDP
300 W
Architecture
Volta
Interconnect
NVLink 2.0 · 300 GB/s
Est. On-demand Price
~$2.00/h
Hourly rates are indicative on-demand estimates; actual pricing varies by provider and commitment.
Compute Performance
| Precision | Peak throughput |
|---|---|
| FP64 (double precision) | 7.8 TFLOPS |
| FP32 (single precision) | 15.7 TFLOPS |
| FP24 | 31.4 TFLOPS |
| FP16 (tensor) | 125 TFLOPS |
| INT8 (tensor) | 62.8 TFLOPS |
| INT4 (tensor) | 62.8 TFLOPS |
Tensor figures use the vendor's peak numbers (with structured sparsity where supported).
System Requirements
Recommended CPU
Intel Xeon Gold 6148 or AMD EPYC 7601
Max VRAM per node (8 GPUs)
256 GB
System RAM (min / recommended)
192 / 384 GB
Minimum PSU
1000 W
LLMs on the V100 SXM2 32GB
Number of V100 SXM2 32GB GPUs needed to serve popular models at 8-bit quantization (including 20% overhead for activations and KV cache).
| Model | Params | VRAM (8-bit) | GPUs needed |
|---|---|---|---|
| GPT-5.6 Sol | 2400B | 2682 GB | 84x V100 SXM2 32GB |
| GPT-5 Flagship | 2100B | 2347 GB | 74x V100 SXM2 32GB |
| GPT-5.6 Luna | 1400B | 1565 GB | 49x V100 SXM2 32GB |
| Kimi K3 (1.2T) | 1200B | 1341 GB | 42x V100 SXM2 32GB |
| Kimi K2.6 (1T) | 1000B | 1118 GB | 35x V100 SXM2 32GB |
| GPT-5.6 Terra | 800B | 894 GB | 28x V100 SXM2 32GB |
| DeepSeek V4 Pro (671B) | 671B | 750 GB | 24x V100 SXM2 32GB |
| Llama 4 Behemoth (500B) | 500B | 559 GB | 18x V100 SXM2 32GB |
| Claude 5 Fable (480B) | 480B | 536 GB | 17x V100 SXM2 32GB |
| GLM 5.2 (400B) | 400B | 447 GB | 14x V100 SXM2 32GB |
| Grok 4.5 | 350B | 391 GB | 13x V100 SXM2 32GB |
| Claude 4.8 Opus (300B) | 300B | 335 GB | 11x V100 SXM2 32GB |
| Muse Spark 1.1 | 300B | 335 GB | 11x V100 SXM2 32GB |
| Grok 4 | 270B | 302 GB | 10x V100 SXM2 32GB |
| Gemini 3.1 Pro | 250B | 279 GB | 9x V100 SXM2 32GB |
| Qwen 3.7 Max (235B) | 235B | 263 GB | 9x V100 SXM2 32GB |
| Mistral Large 3 (200B) | 200B | 224 GB | 7x V100 SXM2 32GB |
| Grok 3 Mini | 190B | 212 GB | 7x V100 SXM2 32GB |
| Claude 5 Sonnet (175B) | 175B | 196 GB | 7x V100 SXM2 32GB |
| Gemini 3.5 Flash | 150B | 168 GB | 6x V100 SXM2 32GB |
| Gemini 2.5 Flash | 140B | 156 GB | 5x V100 SXM2 32GB |
| Llama 4 Maverick (128B) | 128B | 143 GB | 5x V100 SXM2 32GB |
| DeepSeek V4 Flash (120B) | 120B | 134 GB | 5x V100 SXM2 32GB |
| Qwen 3.6 Plus (110B) | 110B | 123 GB | 4x V100 SXM2 32GB |
| Nova Premier (80B) | 80B | 89 GB | 3x V100 SXM2 32GB |
| Qwen 3 Coder-Next (80B) | 80B | 89 GB | 3x V100 SXM2 32GB |
| Claude 4.5 Haiku (70B) | 70B | 78 GB | 3x V100 SXM2 32GB |
| Llama 3.3 Instruct (70B) | 70B | 78 GB | 3x V100 SXM2 32GB |
| Mistral Medium 3.5 (70B) | 70B | 78 GB | 3x V100 SXM2 32GB |
| Yi 1.5 (40B) | 40B | 45 GB | 2x V100 SXM2 32GB |
| Nova Core (34B) | 34B | 38 GB | 2x V100 SXM2 32GB |
| DeepSeek V3.1 (32B) | 32B | 36 GB | 2x V100 SXM2 32GB |
| Gemma 3 (27B) | 27B | 30 GB | 1x V100 SXM2 32GB |
| Mistral Small 4 (24B) | 24B | 27 GB | 1x V100 SXM2 32GB |
| Yi 1.5 (15B) | 15B | 17 GB | 1x V100 SXM2 32GB |
| Phi 4 (14B) | 14B | 16 GB | 1x V100 SXM2 32GB |
| Nova Lite (12B) | 12B | 13 GB | 1x V100 SXM2 32GB |
| Llama 3.2 Instruct (11B) | 11B | 12 GB | 1x V100 SXM2 32GB |
| Gemma 3 (9B) | 9B | 10 GB | 1x V100 SXM2 32GB |
| Yi 1.5 Lite (9B) | 9B | 10 GB | 1x V100 SXM2 32GB |
| Phi 4 Mini (7B) | 7B | 8 GB | 1x V100 SXM2 32GB |
| Phi 3.5 (3.8B) | 3.8B | 4 GB | 1x V100 SXM2 32GB |
Frequently Asked Questions
How much VRAM does the NVIDIA V100 SXM2 32GB have?
The NVIDIA V100 SXM2 32GB has 32 GB of HBM2 memory with 900 GB/s of memory bandwidth.
Which LLMs can run on a single V100 SXM2 32GB?
At 8-bit quantization, a single V100 SXM2 32GB (32 GB) can serve models up to roughly 27B parameters, such as Gemma 3 (27B). Larger models require multiple GPUs or more aggressive quantization.
How much does it cost to rent a NVIDIA V100 SXM2 32GB?
On-demand cloud pricing for the V100 SXM2 32GB is around $2.00/hour, i.e. about $1,460/month running 24/7. Actual prices vary by provider, region, and commitment.
What are the power and system requirements of the NVIDIA V100 SXM2 32GB?
The V100 SXM2 32GB has a TDP of 300W. A power supply of at least 1000W per GPU is recommended. Recommended host CPUs: Intel Xeon Gold 6148 or AMD EPYC 7601.
Deploy on a GPU cloud
Rent the NVIDIA V100 SXM2 32GB by the hour instead of buying hardware.
