H200 NVL vs RTX PRO 6000 Blackwell
Both run on the same nodes, the same templates and the same tiers here — so the choice comes down to memory, bandwidth and price. Every figure below is from the catalogue or the vendor datasheet.
- H200 NVL
- $0.69/h141 GB · Hopper · 16 available
- RTX PRO 6000
- $0.19/h96 GB · Blackwell · 48 available
- Price gap
- 72%On spot, per GPU-hour
- Over 100 h
- $50.00Difference for the same GPU-hours
What actually separates them.
Four differences that change a decision, computed from the numbers rather than asserted.
Bigger models fit on the H200 NVL
141 GB against 96 GB. At FP16 that is about 56B parameters on one GPU, against 38B.
The RTX PRO 6000 costs 72% less per hour
$0.19 against $0.69 on spot — $50.00 of difference over 100 GPU-hours.
Token generation is faster on the H200 NVL
4,800 GB/s against 1,597 GB/s. Inference reads the whole model from memory for every token, so bandwidth, not FLOPS, usually sets the ceiling.
More memory per dollar on the RTX PRO 6000
505 GB per dollar per hour against 204 GB. The usual tie-breaker when both cards fit the job.
Every figure we hold on both cards.
A row is marked only when both values exist and one is genuinely better. H200 NVL leads on 3 rows, RTX PRO 6000 on 6.
| H200 NVL | RTX PRO 6000 Blackwell | |
|---|---|---|
| GPU memoryDecides which models fit at all | 141 GB HBM3e | 96 GB GDDR7 |
| Memory bandwidthSets token throughput on inference | 4,800 GB/s | 1,597 GB/s |
| FP32 | 60 TFLOPS | 120 TFLOPS |
| 8-bit float (FP8)Halves the memory a model needs, where the stack supports it | Supported | Supported |
| Architecture | Hopper (2024) | Blackwell (2025) |
| Board powerLower is cheaper to run at scale — for us, not for your bill | 600 W | 600 W |
| GPU-to-GPU linkOnly matters for multi-GPU training | NVLink | PCIe only |
| Node sizes | 1×, 2×, 4×, 8× | 1×, 2×, 4×, 8× |
| Spot price | $0.69/h | $0.19/h |
| On-demand price | $0.99/h | $0.39/h |
| Memory per dollarGPU memory per $1 of spot time, per hour | 204 GB | 505 GB |
| 30-day reclaim rateShare of spot instances reclaimed | 5–10% | 5–10% |
| Available now | 16 GPUs | 48 GPUs |
Specifications come from each vendor's datasheet — linked on the H200 NVL page and the RTX PRO 6000 page. Prices are ours, per GPU-hour, billed per minute.
The same models, on one GPU of each.
Weights only, with 20% of the memory kept free for activations and the KV cache. A cross means one GPU is not enough — the model page has the multi-GPU arithmetic.
| Model | Precision | H200 NVL 141 GB | RTX PRO 6000 96 GB |
|---|---|---|---|
| Llama 3.1 8B8B parameters | FP16 · 16 GB | Fits | Fits |
| FP8 · 8 GB | Fits | Fits | |
| INT4 · 4 GB | Fits | Fits | |
| Qwen2.5 32B32B parameters | FP16 · 64 GB | Fits | Fits |
| FP8 · 32 GB | Fits | Fits | |
| INT4 · 16 GB | Fits | Fits | |
| Llama 3.3 70B70B parameters | FP16 · 140 GB | Needs 2× node | Needs 2× node |
| FP8 · 70 GB | Fits | Fits | |
| INT4 · 35 GB | Fits | Fits | |
| Mixtral 8x22B141B parameters | FP16 · 282 GB | Needs 4× node | Needs 4× node |
| FP8 · 141 GB | Needs 2× node | Needs 2× node | |
| INT4 · 71 GB | Fits | Fits |
The same 100 GPU-hours on each.
Compute only, billed per minute. Storage is $0.08 per GB-month on either card.
| H200 NVL | RTX PRO 6000 | |
|---|---|---|
| One hour, spot | $0.69 | $0.19 |
| One day, spot | $16.56 | $4.56 |
| 100 GPU-hours, spot | $69.00 | $19.00 |
| 100 GPU-hours, on-demand | $99.00 | $39.00 |
| A month, reserved | $503.70 | $211.70 |
Take the H200 NVL when…
- Training and full fine-tuning — Enough memory for optimiser states and activations on models a smaller card can only run in inference.
- Serving large models — A 70B model at FP8 on a single GPU, with room for a long context window.
- Multi-GPU jobs over NVLink — GPU-to-GPU traffic stays off the PCIe bus, which is what makes tensor and pipeline parallelism worth it.
Take the RTX PRO 6000 when…
- Training and full fine-tuning — Enough memory for optimiser states and activations on models a smaller card can only run in inference.
- Serving large models — A 70B model at FP8 across two GPUs, with room for a long context window.
- Partitioned serving (MIG) — The GPU can be split into isolated instances, each with its own memory and compute slice.
H200 NVL or RTX PRO 6000?
Which is cheaper, the H200 NVL or the RTX PRO 6000?
The RTX PRO 6000 Blackwell, at $0.19 per GPU-hour on spot against $0.69 — 72% less. On-demand: $0.39 against $0.99.
Which one should I take for inference?
Take the one that holds the model with room for the KV cache, then the one with more memory bandwidth. Here that is the H200 NVL on capacity (141 GB) and the H200 NVL on bandwidth.
Can I run both on the same job?
Not inside one instance — a node holds GPUs of a single model. You can run two instances in parallel, on the same persistent volume in turn, or split the work: a big card for the model that needs capacity, a cheap one for the parts that do not.
Other comparisons: the full list · all 47 models
Try both for less than the price of deciding.
An hour on each costs $0.88 on spot. Nothing renews, and your disk moves between them.
Billed per minute from the moment the instance is reachable. Minimum credit $40, no subscription.