Fine-tune AI models on cloud GPU

If your team is planning an LLM fine-tuning project, building a RAG pipeline, or experimenting with a multimodal model that combines text and visual content, the roadblock is almost always the same: someone has asked you to justify a 100,000 to 500,000 euro server purchase before a single developer has written a line of training code or shipped an application. That conversation can take months, and while you wait for finance to sign off on the budget, your competitors have already shipped.

The alternative is GPU as a Service. You provision an H100 or A100 when you need it, run your training job, and pay only for the hours you actually use. No procurement queue, no idle hardware, no CapEx debate. OVHcloud AI Solutions provides the full stack: managed notebooks, serverless training jobs, inference endpoints, and a catalogue of pretrained open-weight foundation models — all on sovereign EU infrastructure.

This guide walks you through why cloud GPU training makes economic sense for most generative AI teams, what the OVHcloud platform looks like in practice, and exactly how to go from a dataset to a deployed model endpoint without touching a single physical server. It reflects one of the clearest trends in generative AI infrastructure today: pay for GPU-hours, not GPU hardware.

Step 1 — Spin up an AI Notebook with GPU and your ML framework

Log in to the OVHcloud Control Panel, navigate to AI Notebooks, and create a new notebook. Select your GPU flavor (H100 or V100S for demanding work, L4 for lighter experimentation), choose Jupyter or VS Code, and pick a pre-built image that matches your stack. The notebook is live in a few minutes with CUDA, your chosen framework, and a terminal ready to go.
Connect your Object Storage bucket as a data volume at launch time. This means your dataset is available as a local path inside the notebook without any manual copying, and your model checkpoints can be written directly to the bucket during training — a simple, effective pattern for any team, not just specialized ML platform engineers.

Step 2 — Stage your dataset on Object Storage

Object Storage (S3-compatible)* is the canonical data layer for AI Training jobs. It is S3-compatible, so your existing tooling (boto3, the AWS CLI, rclone) works without modification. Upload your training corpus, tokenised datasets, and any pre-processed artefacts here before submitting a training job. If your pipeline includes data augmentation steps, run them upstream and store the resulting dataset alongside the original for reproducibility and version control.

One practical advantage: there are no egress fees between OVHcloud services. Moving data from Object Storage to a GPU instance for training, reading checkpoints back to storage mid-job, or pulling model artefacts into AI Deploy after training — none of these transfers incur hidden costs. Iterating on your dataset, and on data quality generally, is cheap.

Step 3 — Submit a training job and pick the right GPU flavor

Package your training script as a Docker image (or use a pre-built OVHcloud image) and submit a job with the ovhai CLI:

ovhai job run --gpu 1 --flavor h100-1-gpu --volume my-bucket@GRA/dataset:/workspace/data:ro --volume my-bucket@GRA/output:/workspace/output:rw my-registry/my-training-image:latest

Choose your GPU flavor based on VRAM requirements: a 7B model fine-tune typically fits on a single H100 with 80 GB VRAM; larger models or larger batch sizes may require an H200, or a multi-GPU request to speed up training. The job runs serverless, monitored via live logs in the Control Panel or via the API. When it ends, your model artefacts are in Object Storage.

Step 4 — Serve your model with AI Deploy or AI Endpoints

For a custom fine-tuned model, create an AI Deploy app pointing to your model artefact in Object Storage. Configure autoscaling based on request volume and memory thresholds, and enable response caching where your traffic pattern allows it to cut inference cost further — a simple technique that pairs well with request batching for throughput-sensitive applications. Each caching layer you add reduces average latency and improves cost-performance at the same time. Your endpoint is live within minutes and scales to match traffic.

For standard open-weight models, skip the deployment step entirely and query AI Endpoints directly. The API is compatible with the OpenAI client format, so existing integrations often require only a base URL change.

The 7B fine-tune that runs on a single H100 in a few hours

The vast majority of generative AI projects do not train from scratch. They fine-tune an existing pretrained, open-weight foundation LLM — Mistral 7B, Llama 3 8B, Falcon, or a similar language model architecture — on a domain-specific dataset. Fine-tuning is itself a form of transfer learning: you start from a model that already generates fluent language broadly, and specialize it to your task with far fewer GPU-hours than training a model from zero. Using techniques like LoRA or QLoRA, and quantization to reduce memory usage during training, a 7B model can complete on a single H100 in 4 to 12 hours, depending on dataset size and epoch count. Most developer teams find this technique far more approachable on the OVHcloud platform than managing their own training architecture.

This is the scenario where pay-as-you-go compute is most economically compelling. You run your fine-tuning job, evaluate the result, adjust the dataset or hyperparameters to prevent overfitting on a small domain-specific corpus, and run again. Each iteration costs a predictable number of GPU-hours. You can run a dozen experiments for less than the monthly electricity bill on a physical server, regularly benchmark model performance, and improve model capabilities and generated content quality with every pass — then ship the result once it clears your bar.

For mid-size models in the 13B to 70B parameter range, multi-GPU jobs are possible on OVHcloud (up to 4 GPUs per job under current limits). For 70B fine-tuning, plan for H100 or H200 instances and lean on quantization to manage VRAM constraints and control layer-by-layer memory usage.

When to talk to a Solutions Architect about large-scale pre-training

If your project involves pre-training a model from scratch at tens or hundreds of billions of parameters — that is a fundamentally different architectural challenge, and a different type of client engagement. Distributed training across many GPUs, custom networking fabric, checkpoint management at petabyte scale, and long-running job orchestration are not self-serve decisions; they require dedicated tooling and a reference architecture built for your case.

For large-scale pre-training, the right first step is a conversation with an OVHcloud AI Solutions Architect. They can assess your compute requirements, discuss GPU cluster options and the right tool for each stage, and help you design a training architecture that matches your scale and your target performance improvement. The link to request that conversation is in the 'Get started' section below.

Cost comparison: on-premise H100 server vs OVHcloud pay-as-you-go

Acquisition cost, electricity, ops and amortisation
The full cost of an on-premise GPU server includes more than the sticker price. A typical 8×H100 SXM system costs between 150,000 and 250,000 euros at acquisition. Add to that: power infrastructure upgrades, cooling, rack space, a three-year maintenance contract, and the salary of the engineer responsible for keeping it running. Amortised over 36 months, the total cost of ownership breaks down to substantially more than the hardware list price alone.

On OVHcloud, there is no acquisition cost. You pay only for the GPU-hours you consume, billed per minute, with no commitment and no idle cost. If your project is cancelled, you pay nothing beyond what you have already run.

No CLOUD Act exposure on training data

The US CLOUD Act allows US authorities to compel US-headquartered companies to produce data stored anywhere in the world, including data hosted in European data centres by US cloud providers. If your training corpus contains personal data, health records, legal documents, financial transactions, or any other regulated category, training on infrastructure subject to US law creates a compliance risk that is difficult to fully mitigate through contractual measures alone.

OVHcloud is a European company with no US parent. There is no structural CLOUD Act dependency. Your training data stays in EU jurisdiction under EU law. For teams in regulated sectors — healthcare, financial services, legal tech, public sector — this is not a theoretical advantage; it is a requirement, and increasingly the key criterion procurement teams ask about upfront.

EU AI Act substrate, GDPR, HDS and ISO 27701

OVHcloud infrastructure is ISO 27001, ISO 27017, ISO 27018, and ISO 27701 certified. The HDS certification is relevant for teams training models on healthcare data in France. For GDPR, training data processed on OVHcloud is stored and processed within the EU under EU law, which simplifies Data Protection Impact Assessments significantly and helps ensure ongoing compliance.

A clarification that matters: OVHcloud provides the sovereign infrastructure substrate. Model-level EU AI Act compliance — model cards, bias detection, audit logs, transparency obligations — remains your responsibility as the AI system developer. Sovereign infrastructure supports your compliance obligations; it does not fulfil them automatically.

On the analyst side, GigaOM Radar recognised OVHcloud for interconnectivity and sovereign availability, and IDC named OVHcloud a Major Player in European Public Cloud IaaS.