When people compare a rented RTX 4090 with a cloud A100, they multiply the hourly GPU price by the training time. I wanted to see
what actually shows up on the Vast.ai invoice for one fixed job on different hosts.
The job:
- Qwen2.5-7B-Instruct, 4-bit QLoRA,
yahma/alpaca-cleaned, 300 steps, checkpoint every 50 steps. - Image
pytorch/pytorch:2.5.1-cuda12.4-cudnn9-runtime, dependencies installed at start. - One RTX 4090, host reliability >= 0.98, network >= 500 Mbit/s.
Three things from the invoices:
- Traffic: $0.81 against $0.19. On the first run the GPU hour looked cheap, but the bill was $0.81: $0.13 for the GPU and $0.66 for traffic, because the host charged for bandwidth and the job downloads a 15 GB model. After I started picking hosts by the full hourly price including expected traffic, the next run was billed $0.19 (0.44 h on the machine, about 19 minutes of it training).
- Same GPU, different bill: $0.21 to $0.38. On 4 November I ran the job six more times on six different 4090 hosts, with no failures. The most expensive run was on a host that trained at about 6 s per step instead of about 4, so training took 31 minutes instead of 17–19. I did not measure why that host was slower.
- Start-up is billed too. In those six runs the first training step came 4.5 to 8.5 minutes after renting. You pay for the image pull, the pip install and the model download.
For scale: the $0.19 run would be about $1.35–1.80 for the same job on one A100 at AWS on-demand rates. That is one job; I would not
generalise it.
How do you filter out slow 4090 hosts before renting? And do you bake everything into one large image to cut the start-up, or cache
the weights somewhere?
Disclosure: these runs were launched by a small supervisor service I build (NomadGPU), which is why I have the per-run billing.
Happy to share the logs.
