Evaluating fine-tuning ROI: was the project actually worth the compute and time?

After a fine-tune ships, it's worth asking honestly whether it beat the simpler alternative. A framework for answering that instead of assuming success.

Written by

Pranjul Rathour

GenAI Engineer at SCULT INDIA, based in Kanpur, India. Builds production RAG systems, fine-tunes open LLMs and ships agentic AI products; has mentored 200+ college developers. Open to GenAI roles, hackathon judging, mentorship sessions and guest talks.

pranjulrathour41@gmail.com LinkedIn GitHub X Instagram Bluesky Dev.to

At VSICS, Kanpur

A fine-tuning project that ships doesn't automatically mean fine-tuning was the right call. It's worth honestly comparing the result against the simpler alternatives that were available from the start.

The comparison to actually run

Same eval set, three ways: the base model with a good prompt, the base model with few-shot examples, and the fine-tuned model.
Measure quality difference alongside cost difference — a marginal quality gain that cost forty GPU-hours may not have been worth it against a well-engineered prompt.
Factor in maintenance: a fine-tuned model needs retraining as requirements evolve; a prompt can be edited in seconds.

When the ROI question has an easy answer

If the task structurally can't be solved by prompting at all — a fixed classification head, a highly consistent generation format prompting can't reliably hit — the ROI question resolves itself. The interesting cases are the ones where prompting was actually close.

See fine-tuning vs a longer prompt: a real cost comparison.

— Pranjul Rathour, GenAI Engineer from Kanpur, India. Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at any campus: pranjulrathour41@gmail.com.