Fine-tuning & training¶
Understand · 15 min · no code
Before this: Safety and responsible AI · After this: Infrastructure and operations In depth: Retrieval in depth
Prompting and RAG handle most AI use cases. But when you need a model to adopt a specific behavior, tone, or deep domain expertise that cannot be achieved through instructions alone, fine-tuning is the next step. This page explains when and how to customize AI models for your specific needs.
When to fine-tune: the decision flow¶
Before investing in fine-tuning, make sure simpler approaches will not work. Here is a decision framework:
The 80/20 rule
In practice, 80% of AI use cases can be solved with good prompt engineering and RAG. Fine-tuning is for the remaining 20% where you need the model to fundamentally change how it responds.
What is fine-tuning?¶
Fine-tuning is the process of taking a pre-trained model and further training it on a smaller, task-specific dataset. The model adjusts its internal weights to become better at the specific task while retaining its general capabilities.
Analogy¶
Think of a pre-trained model as a university graduate with broad knowledge. Fine-tuning is like that graduate doing a specialized internship — they apply their general knowledge to a specific domain and get better at it.
What fine-tuning changes¶
- Output style and tone: Write in your brand voice, match a specific format.
- Domain behavior: Respond like a medical professional, legal expert, or financial analyst.
- Task specialization: Get consistently better at a narrow task (classification, extraction, scoring).
- Reduced prompting: The model "just knows" things that previously required long prompts.
What fine-tuning does NOT do¶
- It does not give the model access to new data at inference time (that is what RAG does).
- It does not guarantee elimination of hallucinations.
- It does not change the model's architecture or fundamental capabilities.
Supervised fine-tuning (SFT)¶
Supervised Fine-Tuning is the most common approach. You provide a dataset of input-output pairs (examples of what the model should produce) and train it to replicate those patterns.
Training data format¶
Most fine-tuning APIs expect data in a conversational format:
{
"messages": [
{"role": "system", "content": "You are a medical coding assistant."},
{"role": "user", "content": "Patient presents with acute bronchitis."},
{"role": "assistant", "content": "ICD-10 Code: J20.9 - Acute bronchitis, unspecified"}
]
}
How much data do you need?¶
| Goal | Minimum Examples | Recommended |
|---|---|---|
| Style/tone adjustment | 50-100 | 200-500 |
| Task specialization | 100-500 | 500-2,000 |
| Deep domain expertise | 500-1,000 | 2,000-10,000 |
Quality over quantity
50 high-quality, carefully curated examples will outperform 5,000 noisy, inconsistent ones. Invest in data quality. Every example should be one you would be proud to show as a correct response.
LoRA and efficient fine-tuning¶
Training all parameters of a large model is expensive and slow. LoRA (Low-Rank Adaptation) and related techniques make fine-tuning practical by training only a small fraction of the model's parameters.
How LoRA works¶
Instead of updating all model weights during training, LoRA:
- Freezes the original model weights.
- Injects small, trainable matrices (adapters) into specific layers.
- Trains only these adapters, which are typically less than 1% of the total parameters.
Benefits¶
| Aspect | Full Fine-Tuning | LoRA |
|---|---|---|
| Parameters trained | All (billions) | ~0.1-1% |
| GPU memory | Very high (multi-GPU) | Moderate (single GPU possible) |
| Training time | Hours to days | Minutes to hours |
| Storage | Full model copy per task | Small adapter file per task |
| Quality | Highest | Near-equivalent for most tasks |
QLoRA¶
QLoRA combines LoRA with quantization — reducing the precision of the frozen model weights (e.g., from 16-bit to 4-bit). This further reduces memory requirements, making it possible to fine-tune large models on consumer-grade GPUs.
LoRA for experimentation
LoRA's low cost and fast iteration make it ideal for experimentation. You can quickly test whether fine-tuning will help your use case before committing to a full training run.
Transfer learning¶
Transfer learning is the broader concept that fine-tuning is built on. The idea is simple: knowledge learned from one task can be transferred to help with a related task.
Why it works¶
A model pre-trained on billions of words of text has already learned:
- Grammar and sentence structure
- Common sense reasoning
- World knowledge
- Logic and pattern recognition
When you fine-tune, you are not teaching the model language from scratch. You are redirecting its existing capabilities toward your specific domain.
The transfer learning pipeline¶
RLHF: reinforcement learning from human feedback¶
RLHF is the technique that transformed raw language models into the helpful, harmless assistants we use today. It aligns model behavior with human preferences.
How RLHF works (simplified)¶
- Supervised Fine-Tuning: Start with a base model and fine-tune it on curated examples of good responses.
- Reward Model Training: Have humans rank multiple model outputs for the same prompt. Use these rankings to train a separate "reward model" that predicts which responses humans prefer.
- Reinforcement Learning: Use the reward model to further train the language model. The language model learns to generate responses that score high on the reward model.
Why RLHF matters¶
- It is the reason models refuse harmful requests, stay on topic, and try to be helpful.
- It is how model providers align general-purpose models with safety guidelines.
- Without RLHF, models would simply predict the most statistically likely text, which often includes toxic or unhelpful content.
Alternatives to RLHF¶
- DPO (Direct Preference Optimization)
- A simpler alternative to RLHF that skips the reward model step. It directly uses human preference data to adjust model weights. Faster and more stable than RLHF.
- RLAIF (RL from AI Feedback)
- Uses an AI model (instead of humans) to provide feedback. Scales better but may inherit biases from the feedback model.
RLHF is typically done by model providers
Unless you are building a foundation model from scratch, you will likely not implement RLHF yourself. It is included here for understanding because it fundamentally shapes how all modern AI models behave.
Distillation: the technique worth knowing¶
Distillation trains a small model on a large model's outputs. You take the task you care about, generate a few thousand high-quality examples with a frontier model, and fine-tune a small open-weight model on them.
The result is a model that is far cheaper and faster than the frontier model, and much better at that one task than its size would suggest. It stays poor at everything else, which is the trade.
This is the most economically interesting form of fine-tuning right now, and it is the shape of most "we cut inference costs by 90%" stories. It needs a narrow, high-volume task, and it needs an evaluation set — otherwise you cannot tell whether the small model is close enough.
Two cautions. Check the frontier provider's terms: some restrict using their outputs to train competing models. And distillation inherits the teacher's mistakes, silently and permanently.
When fine-tuning is genuinely the answer¶
This page has spent a lot of words on when not to. Here is the positive list, because the pendulum has swung far enough that real use cases get dismissed.
A consistent output format or style you cannot get from prompting. If you need every response in a specific structure or house voice, and the system prompt is doing it in 800 tokens on every call, training that in is cheaper and more reliable. See also structured output, which solves the format half without training.
A narrow, high-volume task where cost dominates. Classification, extraction, routing, tagging. Thousands of calls a day on a small model beats a frontier model on price by orders of magnitude, and the task is narrow enough that the smaller model can match it.
Specialist vocabulary the base model handles badly. Clinical coding, legal citation formats, internal jargon and part numbers. Note the distinction: you are teaching the model to handle the vocabulary, not to memorise your facts. The facts still come from retrieval.
Behaviour that is hard to describe but easy to demonstrate. Some judgements resist being written as rules and are obvious across fifty examples. That is exactly what training is for.
A fine-tuned model is a dependency, not an artefact
The cost people miss is not the training run, it is everything after. Base models are deprecated on the provider's schedule, not yours, and a fine-tuned model does not move to a new base without retraining. Your training data becomes an asset you have to keep, version and be able to regenerate.
Budget for retraining on someone else's timetable, and keep the pipeline reproducible. If you cannot rerun the training today, you do not have a fine-tuned model — you have a fine-tuned model you are afraid to touch.
Before and after, on the same evaluation set. Fine-tuning can make a model worse at things it was previously fine at, and you will not notice unless you measure both. Build the evaluation set before you train, not to justify the result afterwards.
Cost and effort considerations¶
Fine-tuning is not free. Here is an honest look at the investment required:
Cost factors¶
| Factor | Description |
|---|---|
| Compute | GPU hours for training. LoRA reduces this significantly. |
| Data preparation | Curating, cleaning, and formatting training data is the most time-consuming step. |
| Evaluation | You need a systematic way to measure whether fine-tuning improved the model. |
| Iteration | Fine-tuning rarely works perfectly on the first try. Budget for multiple rounds. |
| Hosting | Fine-tuned models may need dedicated endpoints, increasing serving costs. |
| Maintenance | As your domain changes, training data and models need updating. |
Comparison of approaches¶
| Approach | Upfront Cost | Ongoing Cost | Time to Deploy | Flexibility |
|---|---|---|---|---|
| Prompt engineering | Very low | Per-token API costs | Hours | High (change prompts anytime) |
| RAG | Moderate (indexing pipeline) | Per-query + API costs | Days to weeks | High (update data anytime) |
| LoRA fine-tuning | Moderate (GPU + data prep) | Hosting + API costs | Weeks | Medium (retrain to update) |
| Full fine-tuning | High (multi-GPU + data prep) | Hosting + API costs | Weeks to months | Low (expensive to update) |
Do not fine-tune prematurely
Fine-tuning should be your last resort, not your first approach. Exhaust prompt engineering and RAG first. If you still cannot achieve the quality you need, then consider fine-tuning — starting with LoRA.
Go deeper¶
- Azure OpenAI fine-tuning — the managed path, including what it costs to keep a fine-tuned deployment running.
- OpenAI model optimization — when the vendor itself suggests prompting or retrieval before fine-tuning.
- Hugging Face PEFT — LoRA and QLoRA as working code, if you are tuning open-weight models yourself.
- Retrieval in depth — read this before committing to fine-tuning. For most knowledge problems, retrieval is the cheaper and more correctable answer.