Prompting vs RAG vs Fine-Tuning: Which One Should You Use?

What does each approach change?#

Each approach changes a different thing. Prompting changes the words you send. RAG changes the material the model sees alongside your question, by adding passages retrieved from your documents. Fine-tuning changes the model itself, by adjusting its internal settings through further training. The deeper the change, the more effort, data and testing it needs.

A simple way to picture it is a new member of staff:

• Prompting is giving them a clear brief for today's task.
• RAG is handing them the relevant pages of the company handbook before they answer.
• Fine-tuning is sending them on a long training programme so their habits change for good.

You would not send someone on a long course to fix a vague brief, and you would not expect a good brief to teach them a handbook they have never seen. Matching the fix to the problem is the whole skill.

How does each one work?#

Prompting works by writing clearer instructions, context and examples into the prompt. RAG works by searching your documents for passages that match the question and adding them to the prompt. Fine-tuning works by training the model further on many example inputs and outputs, so it adjusts its parameters towards the behaviour you want.

Prompting#

Prompt engineering uses the model as it is. Anthropic's prompting guidance covers techniques such as being clear and direct, explaining why an instruction matters, structuring prompts with labelled sections and including a few well-chosen examples, which it says is one of the most reliable ways to steer format, tone and structure.

Including examples in the prompt is often called few-shot prompting. The 2020 research paper that showed how well this works at scale, published on arXiv, tested its model with the task and a few demonstrations specified purely as text, without any further training of the model.

RAG#

RAG was introduced in a separate 2020 research paper, also on arXiv. It combined a pre-trained language model with a searchable index of documents, accessed by a retriever. The authors noted that knowledge stored only inside a model's parameters is hard to update and hard to trace to a source. RAG keeps knowledge in documents you can read and change.

In today's tools, the retrieved passages are usually placed in the prompt, often with a request to cite them. Anthropic's documentation, for example, describes search result blocks that let the model cite your own content with the source and title you provide, for use in RAG applications. Our guide to what RAG is explains the full process.

Fine-tuning#

Fine-tuning takes a pre-trained model and trains it further on examples of the task. The few-shot paper describes traditional task-specific fine-tuning as needing datasets of thousands or tens of thousands of examples. A later paper on a method called LoRA notes that retraining all of a large model's parameters becomes less feasible as models grow. Its approach keeps the original model fixed and trains a much smaller set of added parameters instead.

Whichever method is used, fine-tuning changes how the model behaves for every future request, so the training examples need to be accurate, consistent and approved for that use.

How do they compare side by side?#

Side by side, prompting is the fastest and cheapest to change, RAG is the best fit for private or changing knowledge that needs a source, and fine-tuning is the heaviest option, suited to consistent behaviour across many cases. The table below compares them on the questions a small team usually asks first.

These approaches are layers, not rivals. Every RAG system still needs a good prompt, and a fine-tuned model can still be given retrieved documents.

How do you choose the right approach?#

Choose by first defining what a good answer looks like and how you will test it, then trying the lightest fix. Improve the prompt first. If answers fail because the model lacks your information, add RAG. Consider fine-tuning only when consistent behaviour still falls short after both, and you have the examples to train it.

Anthropic's prompt engineering overview makes the first step explicit: before refining a prompt, have clear success criteria, a way to test against them and a first draft. It also notes that not every failing test is best solved by prompting. Some problems, such as speed or cost, may be easier to fix by choosing a different model. That habit of testing first is called evaluation, and our guide on testing AI-generated outputs shows how to set it up.

A practical order of questions:

1. Is the prompt clear? Add the role, format, reasons and examples. Retest.
2. Is the model missing information? If it needs your policies, products or recent changes, add RAG.
3. Must every answer show its source? That points to RAG as well.
4. Is the behaviour still inconsistent across many cases? Only then look at fine-tuning, and only if you have enough good examples.

What does this look like in a Singapore workplace?#

Consider a small insurance brokerage in Raffles Place that wants an assistant to help staff draft replies to client questions about policy wording. This illustrative example shows how the team might work through the three options in order, without claiming any results or savings.

• Step 1: prompting. The team writes a prompt that sets the role (a client service officer), the tone (clear and careful) and the format (a short reply with a suggested next step). Drafts improve, but the assistant sometimes guesses at policy terms it has never seen.
• Step 2: RAG. The team indexes the current policy wording documents it is permitted to use. The prompt now tells the assistant to answer only from the retrieved passages, name the clause it used and say when the documents do not cover the question.
• Step 3: fine-tuning? The team asks whether anything still fails consistently. The remaining problems are occasional wrong passages, which is a retrieval issue, not a training issue. Fine-tuning would not fix that, and the team does not have a large approved set of example replies, so it stays off the list for now.

Throughout, client names and policy numbers stay out of the tools unless they are approved for personal data, and an experienced staff member reviews every reply before it goes out.

Common mistakes#

The most common mistakes are fine-tuning to teach facts that change, skipping prompt improvements and jumping straight to complex options, building without a test set, assuming RAG removes the need for checking, and using training examples or documents that contain errors or data that has not been approved for that use.

• Fine-tuning for facts. New prices or policies mean training again. RAG lets you update the documents instead.
• Skipping the prompt. Many problems that look like they need RAG or fine-tuning are vague instructions.
• No test set. Without fixed test questions and expected answers, you cannot tell whether a change helped.
• Treating RAG as foolproof. The model can still misread a passage or retrieve the wrong one.
• Training on bad examples. Errors in fine-tuning data become the model's habits.
• Ignoring data approval. Check that documents and examples are approved for the tool before they go in.

What to learn next#

The next step is to learn the building blocks behind the option you are most likely to use. For most teams that is RAG, which depends on embeddings and good retrieval. After that, practise building and testing a small app so you can judge these trade-offs from experience rather than theory.

• Read how embeddings and vector databases make retrieval work.
• Browse more guides in AI for builders, or follow the AI learning pathway.
• To plan, build, test and launch a simple AI app in a classroom, see GenAI Builder Sprint, a 2-day course. No previous AI or programming experience is needed, although formal entry requirements apply.

Sources#

1. Prompt engineering overview, Anthropic (Claude Platform Docs), accessed 8 November 2026
2. Prompting best practices, Anthropic (Claude Platform Docs), accessed 8 November 2026
3. Search results, Anthropic (Claude Platform Docs), accessed 8 November 2026
4. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, arXiv, accessed 8 November 2026
5. Language Models are Few-Shot Learners, arXiv, accessed 8 November 2026
6. LoRA: Low-Rank Adaptation of Large Language Models, arXiv, accessed 8 November 2026

Frequently asked questions#

What is the difference between prompting, RAG and fine-tuning?+

Prompting changes the instructions you give the model. RAG retrieves relevant passages from your documents and adds them to the prompt at the moment of asking. Fine-tuning further trains the model on examples, so it changes the model itself.

Which approach should I try first?+

Start with prompting. It is the quickest to try and the easiest to undo. Before you start, define what a good answer looks like and how you will test it, so you can tell whether any change is an improvement.

When is RAG the right choice?+

RAG suits knowledge that the model does not have, such as internal policies, knowledge that changes often, and cases where every answer needs to show its source. You update the knowledge by updating the documents and re-indexing them.

When does fine-tuning make sense?+

Fine-tuning can make sense when you need consistent behaviour, format or style across many cases, a good prompt with examples still falls short, and you have a large set of high-quality examples. It is a poor way to keep fast-changing facts up to date.

How much data does fine-tuning need?+

The research paper that introduced few-shot prompting at scale describes traditional task-specific fine-tuning as needing datasets of thousands or tens of thousands of examples. Few-shot prompting, by contrast, specifies the task and a handful of demonstrations purely as text.

Can I combine prompting, RAG and fine-tuning?+

Yes. They are layers, not rivals. Every RAG system still needs a good prompt, and a fine-tuned model can still be given retrieved documents. Most practical workplace systems combine a strong prompt with RAG, and many never need fine-tuning.