Students bring me a problem and a preferred solution — usually "fine-tune a model on our data" — before they have asked what kind of problem it is. I have built a production RAG platform and a fine-tuning platform, and they solve different problems. Here is how I decide.
Ask what the model is missing
- Knowledge it never saw — your documents, your prices, last week's policy. This is a retrieval problem. Fine-tuning does not reliably install facts, and it certainly does not update them.
- A behaviour or style — always answer in a strict JSON schema, speak like your brand, follow a domain's conventions. This is a fine-tuning problem, or a prompt problem if the behaviour is simple.
- A skill in a narrow domain — classify support tickets into your 40 categories, extract fields from your invoice layouts. Fine-tuning a small model often beats prompting a large one on cost and consistency.
Why RAG is usually first
RAG works with any model, updates the moment a document changes, and produces citations you can verify. Most business questions are knowledge questions. RAG.NextUpgrad exists because clients wanted answers about their own documents — no fine-tune would have helped them.
Why fine-tuning is sometimes unavoidable
When a prompt grows to two pages of rules and the model still slips, you are paying for those tokens on every request and still getting inconsistency. A QLoRA fine-tune on a few thousand examples bakes the behaviour in. FineTune Studio was built for exactly this: small models, cheap hardware, and an honest base-versus-tuned comparison so you know whether it worked (see how to evaluate a fine-tuned model honestly).
When you need both
A fine-tuned small model for format and tone, with retrieved context for facts. Support assistants often land here: the tune makes it sound like your company; the retrieval makes it correct about your products.
Three questions before spending a GPU hour
- Does the failure go away if I paste the right document into the prompt? If yes, it is retrieval.
- Do I have at least a thousand clean examples of the behaviour I want? If no, fine-tuning will disappoint.
- Will the underlying facts change monthly? If yes, they must live in retrieval regardless.
Pick the tool by the shape of the problem, not by which one sounds more impressive on a resume. Shipping the right one is what impresses.
— Pranjul Rathour, GenAI Engineer from Kanpur, India. Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at any campus: pranjulrathour41@gmail.com.