Start from the job, not from the GPU.
Four workloads, each with the memory arithmetic, the cheapest model in the catalogue that fits, what an hour costs, and whether it belongs on interruptible capacity.
Each one answers the same three questions, for a different job.
Which GPU fits, what an hour costs, and whether the work belongs on interruptible capacity.
Serving a language model
Hold the weights, read them fast, answer requests. Memory capacity decides what runs; memory bandwidth decides how quickly.
Starts at $0.06/h on RTX 4090On-demand for production, spot for batch
Read the guideFine-tuning and training
Optimiser states need several times the memory of the weights. Checkpoints make the job interruptible, and interruptible is cheap.
Starts at $0.09/h on A100 PCIeSpot, with checkpoints
Read the guideImage and video generation
Diffusion models are small enough to fit almost anywhere; throughput comes from bandwidth and from keeping the pipeline resident.
Starts at $0.09/h on RTX 5090Spot for batches, on-demand for a live endpoint
Read the guideBatch and offline jobs
Embeddings, evaluation runs, transcoding, rendering, data processing: work that can stop and resume — the natural home of spot capacity.
Starts at $0.05/h on L4Spot, always
Read the guideThree questions decide every one of them.
Whatever the workload, the answer comes from the same three checks — which is why the guides all look the same underneath.
Does it fit in memory?
Weights need roughly 2 GB per billion parameters at FP16, 1 GB at FP8, 0.5 GB in 4-bit, plus about 20% for activations and cache. Every model page does the arithmetic for the exact card.
Is it bound by bandwidth or by compute?
Token generation reads the whole model from memory for every token, so bandwidth sets the ceiling. Training and image generation lean harder on compute.
Can it survive a 2-minute warning?
If yes, spot cuts the bill by 30–53%. If not, on-demand costs more and never stops. How spot works.
Your first H100 for $0.55 an hour.
Pay as you go — no contracts, no minimum commitment. Add credit, launch, stop whenever you want.
Billed per minute from the moment the instance is reachable. Minimum credit $40, no subscription.