GPU Specifications / AMD
AMD Instinct MI300X
CDNA 3 datacenter GPU with 192 GB of HBM3 memory, 5300 GB/s of bandwidth and up to 2,615 TFLOPS of FP16 tensor compute.
Key Specifications
Memory
192 GB HBM3
Memory Bandwidth
5,300 GB/s
TDP
750 W
Architecture
CDNA 3
Interconnect
Infinity Fabric 3 · 896 GB/s
Est. On-demand Price
~$6.00/h
Hourly rates are indicative on-demand estimates; actual pricing varies by provider and commitment.
Compute Performance
| Precision | Peak throughput |
|---|---|
| FP64 (double precision) | 81.7 TFLOPS |
| FP32 (single precision) | 163.4 TFLOPS |
| FP24 | 326.8 TFLOPS |
| FP16 (tensor) | 2,615 TFLOPS |
| INT8 (tensor) | 5,230 TFLOPS |
| INT4 (tensor) | 5,230 TFLOPS |
Tensor figures use the vendor's peak numbers (with structured sparsity where supported).
System Requirements
Recommended CPU
AMD EPYC 9554 or Intel Xeon Platinum 8480+
Max VRAM per node (8 GPUs)
1,536 GB
System RAM (min / recommended)
512 / 1024 GB
Minimum PSU
1600 W
LLMs on the Instinct MI300X
Number of Instinct MI300X GPUs needed to serve popular models at 8-bit quantization (including 20% overhead for activations and KV cache).
| Model | Params | VRAM (8-bit) | GPUs needed |
|---|---|---|---|
| GPT-5.6 Sol | 2400B | 2682 GB | 14x Instinct MI300X |
| GPT-5 Flagship | 2100B | 2347 GB | 13x Instinct MI300X |
| GPT-5.6 Luna | 1400B | 1565 GB | 9x Instinct MI300X |
| Kimi K3 (1.2T) | 1200B | 1341 GB | 7x Instinct MI300X |
| Kimi K2.6 (1T) | 1000B | 1118 GB | 6x Instinct MI300X |
| GPT-5.6 Terra | 800B | 894 GB | 5x Instinct MI300X |
| DeepSeek V4 Pro (671B) | 671B | 750 GB | 4x Instinct MI300X |
| Llama 4 Behemoth (500B) | 500B | 559 GB | 3x Instinct MI300X |
| Claude 5 Fable (480B) | 480B | 536 GB | 3x Instinct MI300X |
| GLM 5.2 (400B) | 400B | 447 GB | 3x Instinct MI300X |
| Grok 4.5 | 350B | 391 GB | 3x Instinct MI300X |
| Claude 4.8 Opus (300B) | 300B | 335 GB | 2x Instinct MI300X |
| Muse Spark 1.1 | 300B | 335 GB | 2x Instinct MI300X |
| Grok 4 | 270B | 302 GB | 2x Instinct MI300X |
| Gemini 3.1 Pro | 250B | 279 GB | 2x Instinct MI300X |
| Qwen 3.7 Max (235B) | 235B | 263 GB | 2x Instinct MI300X |
| Mistral Large 3 (200B) | 200B | 224 GB | 2x Instinct MI300X |
| Grok 3 Mini | 190B | 212 GB | 2x Instinct MI300X |
| Claude 5 Sonnet (175B) | 175B | 196 GB | 2x Instinct MI300X |
| Gemini 3.5 Flash | 150B | 168 GB | 1x Instinct MI300X |
| Gemini 2.5 Flash | 140B | 156 GB | 1x Instinct MI300X |
| Llama 4 Maverick (128B) | 128B | 143 GB | 1x Instinct MI300X |
| DeepSeek V4 Flash (120B) | 120B | 134 GB | 1x Instinct MI300X |
| Qwen 3.6 Plus (110B) | 110B | 123 GB | 1x Instinct MI300X |
| Nova Premier (80B) | 80B | 89 GB | 1x Instinct MI300X |
| Qwen 3 Coder-Next (80B) | 80B | 89 GB | 1x Instinct MI300X |
| Claude 4.5 Haiku (70B) | 70B | 78 GB | 1x Instinct MI300X |
| Llama 3.3 Instruct (70B) | 70B | 78 GB | 1x Instinct MI300X |
| Mistral Medium 3.5 (70B) | 70B | 78 GB | 1x Instinct MI300X |
| Yi 1.5 (40B) | 40B | 45 GB | 1x Instinct MI300X |
| Nova Core (34B) | 34B | 38 GB | 1x Instinct MI300X |
| DeepSeek V3.1 (32B) | 32B | 36 GB | 1x Instinct MI300X |
| Gemma 3 (27B) | 27B | 30 GB | 1x Instinct MI300X |
| Mistral Small 4 (24B) | 24B | 27 GB | 1x Instinct MI300X |
| Yi 1.5 (15B) | 15B | 17 GB | 1x Instinct MI300X |
| Phi 4 (14B) | 14B | 16 GB | 1x Instinct MI300X |
| Nova Lite (12B) | 12B | 13 GB | 1x Instinct MI300X |
| Llama 3.2 Instruct (11B) | 11B | 12 GB | 1x Instinct MI300X |
| Gemma 3 (9B) | 9B | 10 GB | 1x Instinct MI300X |
| Yi 1.5 Lite (9B) | 9B | 10 GB | 1x Instinct MI300X |
| Phi 4 Mini (7B) | 7B | 8 GB | 1x Instinct MI300X |
| Phi 3.5 (3.8B) | 3.8B | 4 GB | 1x Instinct MI300X |
Frequently Asked Questions
How much VRAM does the AMD Instinct MI300X have?
The AMD Instinct MI300X has 192 GB of HBM3 memory with 5300 GB/s of memory bandwidth.
Which LLMs can run on a single Instinct MI300X?
At 8-bit quantization, a single Instinct MI300X (192 GB) can serve models up to roughly 150B parameters, such as Gemini 3.5 Flash. Larger models require multiple GPUs or more aggressive quantization.
How much does it cost to rent a AMD Instinct MI300X?
On-demand cloud pricing for the Instinct MI300X is around $6.00/hour, i.e. about $4,380/month running 24/7. Actual prices vary by provider, region, and commitment.
What are the power and system requirements of the AMD Instinct MI300X?
The Instinct MI300X has a TDP of 750W. A power supply of at least 1600W per GPU is recommended. Recommended host CPUs: AMD EPYC 9554 or Intel Xeon Platinum 8480+.
Deploy on a GPU cloud
Rent the AMD Instinct MI300X by the hour instead of buying hardware.
