A 92% accurate classifier sounds good until you learn all the errors are on one specific class that happens to be the one your product cares about most.
Check these before shipping
- Per-class precision and recall, not just overall accuracy — a rare but important class can be badly served by a high overall score.
- Confidence calibration: does the model's confidence score actually track its real accuracy, or is it overconfident on wrong answers?
- Performance on hard cases specifically: poor lighting, unusual angles, partial occlusion — the cases your clean test set probably under-represents.
- Latency and memory at your actual deployment target, not just on the training machine.
A model that's 92% accurate but confidently wrong on the 8% is more dangerous in production than one that's 85% accurate and honestly uncertain on its misses. Design for the second kind.
Related on the text side: RAG evaluation metrics: faithfulness and recall.
— Pranjul Rathour, GenAI Engineer from Kanpur, India. Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at any campus: pranjulrathour41@gmail.com.