Fine-tuning with honest evals.
I built FineTune Studio, an end-to-end QLoRA platform: upload a dataset, pick a base model, launch a run from the browser, watch the loss live, and compare base against tuned before anything is called better.
- Method
- LoRA and QLoRA
- Libraries
- Transformers, PEFT, TRL
- Telemetry
- Loss, learning rate, throughput, live
- Evaluation
- Base vs tuned, same prompts, side by side
When tuning is the right tool
Most teams reach for fine-tuning too early. The honest decision tree.
Retrieval supplies knowledge at answer time. Fine-tuning changes behaviour: tone, format, a domain's vocabulary, a task the base model keeps getting slightly wrong. Prompting steers both. I have built a RAG platform and a fine-tuning platform in the same year, so the first thing I do is check which problem you actually have.
- Facts that change weekly, long documents, citations needed: that is retrieval, not tuning.
- Consistent style, structured output, a narrow task at scale: that is a fine-tune, usually a small one.
- Both at once is common: a tuned model inside a RAG pipeline.
Read the full reasoning in RAG vs fine-tuning vs prompt engineering.
How a run works
The product workflow FineTune Studio turns training into.
- 01
Dataset in, base model chosen
Upload the data, pick the base model, set LoRA or QLoRA hyper-parameters, launch from the browser.
- 02
Watch it train
Loss, learning rate and throughput stream to the UI over WebSockets, so a bad run is stopped in minutes, not discovered hours later.
- 03
Compare before you believe
Base and fine-tuned models answer the same prompts side by side. The effect is visible before deployment.
Read next
Guides from the work.
Tune a model, or find out you shouldn't.
A twenty-minute call is enough to tell which, with your data in mind.