Two students both said 'I'm fine-tuning a model' — one meant a sentiment classifier, the other a support chatbot. Both are fine-tuning, but the data, model choice and evaluation are almost unrelated problems.
Classification
Needs a smaller model, a fixed label set, and a metric that's unambiguous: precision, recall, F1 per class. A confusion matrix tells you exactly which labels get confused with which — evaluation here is close to solved.
Open-ended generation
Has no fixed correct answer, so evaluation needs either human review, an LLM-as-judge rubric, or task-specific checks (does the reply contain a required disclaimer, is the format valid JSON). This is the harder, less settled half of the problem.
If your task is really classification, a smaller encoder-only model fine-tuned normally often beats fine-tuning a full generative LLM on cost and latency. See evaluating a fine-tuned model honestly.
— Pranjul Rathour, GenAI Engineer from Kanpur, India. Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at any campus: pranjulrathour41@gmail.com.