A Cloud Cost Comparison for Modern Machine Learning Workloads

Machine learning bills are driven by one line item more than any other: GPU hours. The three major clouds rent many of the same NVIDIA GPUs, yet the price you pay depends on the provider, the region, the size of machine the GPU comes with and how you commit. This article compares entry-level, mid-range and high-end GPUs using the prices each provider publishes for its London region, looks at alternatives to renting a GPU, and sets out what to check before you choose.

What drives the bill

For most training and inference workloads, GPU time is the largest cost, followed by storage for datasets and checkpoints, data transfer, and the engineering time spent keeping it all running. Idle GPUs are the quiet cost: an instance left running overnight costs the same as one in use. That is why the lowest hourly rate does not always give the lowest project cost.

GPUs available in London

Not every GPU is offered in every region, so we started from what each provider actually lists for London: AWS Europe (London), Azure UK South and Google Cloud europe-west2. We grouped the NVIDIA GPUs on offer into three tiers:

• Entry-level: NVIDIA T4 (16 GB). An older, widely used GPU for light inference, small models and experimentation. Offered in London by all three providers.
• Mid-range: NVIDIA L4 and A10G / A10 (24 GB). The usual starting point for serving small and medium models and fine-tuning smaller ones. AWS offers both, Azure offers the A10 in its graphics and virtual desktop range, and Google Cloud offers the L4.
• High-end: NVIDIA A100, RTX PRO 6000 and H100 (40 to 96 GB). For training and fine-tuning larger models, or serving models that need more GPU memory. The RTX PRO 6000 is offered by all three in London, while the A100 and H100 are offered by AWS and Azure but not by Google Cloud.

As a rough guide, a 24 GB GPU can typically serve a model of around 7 billion parameters at 16-bit precision, and larger models with quantisation.

Prices compared

The tables show on-demand Linux list prices per hour, in US dollars as published for each London region, excluding VAT. Each price is for the whole machine, so it includes the CPUs (vCPU) and memory listed with it.

Entry-level

Mid-range

High-end

A few points on reading them. First, prices are for the London regions specifically and can differ from US prices for the same machine. Second, the machines are not identical, as the CPU and memory figures show. Azure’s A10 machine comes from its graphics and virtual desktop range and includes an NVIDIA GRID licence, which explains much of its higher price. Microsoft documents the CUDA drivers that most AI frameworks rely on for its T4, A100 and H100 machines but not for this A10 machine, so on Azure those are the documented choices for AI work, while Microsoft positions its RTX PRO 6000 machine for both AI inference and graphics.

Third, AWS sells the A100 in London only as an eight-GPU p4d node, so the per-GPU figure assumes you need all eight, and Google Cloud’s T4 price combines the GPU with a four-vCPU N1 machine because the T4 is attached to a separate virtual machine. AWS also offers its own Inferentia2 inference chips in London, which AWS says offer up to 40 per cent better price-performance than comparable EC2 instances for supported models.

What a project costs

Take eight months of continuous running, about 5,840 hours. One T4 comes to roughly $3,600. One L4 on Google Cloud comes to roughly $4,700, and on AWS roughly $6,000. One A10G on AWS comes to roughly $7,500. These are illustrative on-demand list prices before storage, networking and VAT, and the high-end tier costs several times more again.

Running only when needed changes the picture. A GPU used for about 40 hours a week costs roughly a quarter of one left on around the clock, which is why scheduling and automatic shutdown are often the biggest single saving.

Discounts that change the answer

On-demand is the most expensive way to buy. Two levers matter, and each comes with a trade-off.

• Commitments. Reserved capacity and committed use discounts trade flexibility for price. Azure’s UK South reservations work out at about 36 per cent below pay-as-you-go for one year and 56 per cent for three years on the A10 machine, and about 35 and 63 per cent on the A100 machine. Google Cloud’s resource-based commitments on the L4 machine in London are about 37 per cent for one year and 55 per cent for three. AWS offers Savings Plans and Reserved Instances, so check the current rates for London.
• Spot capacity. Spot capacity can be far cheaper, but the discount varies by provider and the machine can be reclaimed at short notice. In London we saw about 82 per cent below pay-as-you-go for the A10 and A100 machines on Azure and about 40 per cent for the L4 machine on Google Cloud, while AWS spot prices move with demand. It suits fault-tolerant training with frequent checkpoints, not live inference.

Commitments only pay back if the GPUs are used. A one-year commitment on hardware that sits idle for half the year is not a saving.

Costs outside the GPU

Data transfer and storage are easy to overlook. All three providers charge per GB for data leaving their network to the internet, and moving data between regions costs extra, so keep training data and compute in the same region. AWS includes 100 GB of free data transfer out to the internet each month across all regions. Block storage for datasets and checkpoints adds a monthly charge per GB on top. All prices here exclude VAT, so factor it in and check whether your organisation can recover it.

GPU quotas are another practical point. New accounts often start with low or zero GPU limits, so request increases early, because approval can take time and capacity is not guaranteed. Operations matter too: automatic shutdown of idle instances, cost tagging and budget alerts are cheap to set up and often save more than switching provider.

Alternatives to renting a GPU

Renting a GPU by the hour is not the only way to run machine learning, and for some workloads it is not the cheapest.

• Pay-per-use model APIs. Amazon Bedrock, Microsoft Foundry and Google Vertex AI let you call hosted models and pay per token or per request, with nothing to pay while idle. For prototypes and low or uneven traffic, this can cost far less than keeping a GPU running. If data residency matters, check that the model you want is offered in a UK or EU region.
• Managed ML platforms. Managed platforms can add a margin over the raw machine. On Google Cloud in London, the T4 GPU element costs $0.471/hr through Vertex AI prediction against $0.41/hr on Compute Engine, about 15 per cent more. Azure Machine Learning has no extra charge beyond the compute and other Azure services you use. SageMaker has its own instance rates, so compare them with EC2 before choosing.
• CPU-only machines. Classic machine learning, such as gradient-boosted trees and linear models, often needs no GPU at all. In London, a general-purpose AWS m7i.xlarge with the same 4 vCPUs and 16 GiB as the smallest GPU machines lists at $0.23/hr, against $0.62 to $1.28/hr with a T4, L4 or A10G.

A common pattern is to prototype on a pay-per-use API, then move to rented GPUs once traffic is steady enough to keep them busy.

Choosing between them

None of the three is cheapest in every case. In London, T4 prices were close across all three providers. Google Cloud had the lowest price for an L4 and an RTX PRO 6000, AWS had the lowest price for an A10-class GPU, and Azure was the only provider offering a single A100 without taking a full eight-GPU node. The right choice depends on questions like these:

1. Training or inference? Inference often fits a single T4, L4 or A10-class GPU, or a pay-per-use API. Training and fine-tuning larger models may need the high-end tier.
2. Steady or bursty demand? Steady use justifies commitments. Bursty use favours on-demand, spot or pay-per-use APIs.
3. Where does your data live? Moving large datasets between clouds costs time and egress fees, and not every GPU is offered in London, so check availability before you plan around the UK.
4. Can the job be interrupted? If it checkpoints well, spot capacity can cut cost sharply.
5. What credits or agreements apply? Check for research, startup or enterprise credits before comparing list prices.

Run a short benchmark on your own model before committing, and measure cost per training run or per thousand inferences rather than cost per hour. If you want help comparing options for a specific workload, take a look at our cloud cost optimisation and FinOps and AI platform foundations services, or get in touch.

Sources, checked October 2026: AWS, EC2 on-demand price list for Europe (London), G7e and Inf2 instance documentation and EC2 data transfer pricing; Microsoft, Azure Retail Prices API for UK South, Azure VM size and N-series driver documentation and Azure Machine Learning pricing; Google Cloud, accelerator-optimized VM pricing for London (europe-west2), SKU price list and GPU regions and zones documentation. Prices are on-demand Linux list prices in US dollars as published for each London region. They exclude VAT and change without notice, so confirm current rates with each provider’s pricing calculator.