Fine-tuning checkpoints: what to actually keep, and what to safely delete

Saving every epoch's checkpoint fills a disk fast. A practical policy for what to keep, informed by what a loss curve and eval score actually tell you.

Written by

Pranjul Rathour

GenAI Engineer at SCULT INDIA, based in Kanpur, India. Builds production RAG systems, fine-tunes open LLMs and ships agentic AI products; has mentored 200+ college developers. Open to GenAI roles, hackathon judging, mentorship sessions and guest talks.

pranjulrathour41@gmail.com LinkedIn GitHub X Instagram Bluesky Dev.to

Walking a room through evaluation criteria

A student ran out of disk space on a free-tier GPU notebook because every epoch saved a full checkpoint. Most of those checkpoints were never going to be used.

A simple keep policy

Keep the checkpoint with the best validation score, not the last one — training loss can keep dropping after validation performance peaks.
Keep one early checkpoint as a sanity baseline to diff behaviour against.
Delete the rest once you've confirmed the best one still evaluates well after quantization, if you quantize.

For LoRA specifically

Save only the adapter weights, not a full merged model, at every intermediate step — adapters are a few hundred megabytes versus several gigabytes for a merged model, and you can always merge the best one at the end.

See reading a loss curve during fine-tuning for how to spot the actual best checkpoint before you delete the others.

— Pranjul Rathour, GenAI Engineer from Kanpur, India. Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at any campus: pranjulrathour41@gmail.com.