Generative AI Architecture · Part 6 of 8
Fine-Tuning vs. RAG vs. Prompting
Writing a better prompt, retrieving relevant information at query time, and adapting the model's own weights through fine-tuning provide different ways to improve model performance and accuracy.
The question each one answers
| Approach | Use it when… | What it can't fix |
|---|---|---|
| Prompting | The model knows the informatino needed for the task and needs clearer instructions or examples | Information the model was never trained on |
| Retrieval-augmented generation | The model needs private or fast-changing information | A missing skill, or instructions the model keeps ignoring |
| Fine-tuning | The task needs specialized skills | Facts that change after the training data was collected |
The three also differ in what they cost to adopt. Prompting needs no training run and no data pipeline, only changed text (longer prompts do cost more per call). RAG adds a data pipeline (the ingestion, chunking, and indexing work) but no training run. Fine-tuning adds a curated dataset and a training run that computes gradients over it, which adds complexity and time to iterate on.
How they combine in practice
Many production systems combine prompting with RAG; fine-tuning is added less often, for narrow tasks. When all three are used, a model is fine-tuned to reliably produce a specific output format and tone for a narrow domain, given RAG context at query time for any fact that changes daily, and driven by a prompt that tells it how to use both. A fine-tuned customer support model might consistently produce on-brand, correctly formatted responses (the fine-tuning), while being handed the customer's order status from a database each time it answers (the retrieval), inside a prompt that specifies exactly how to phrase a refund policy (the prompting).