A year of shipping five production AI apps produced a portfolio and a long list of things I would not do again. Students hear the portfolio part in talks; here is the list, because it is the more useful half.
1. Building before measuring
The first RAG pipeline was tuned by feel for weeks. The moment I wrote fifty questions with known answers, two weeks of changes turned out to have made recall worse. Now the evaluation set is written before the pipeline; see how to evaluate a RAG system.
2. Trusting the demo corpus
Everything worked on the clean PDFs I tested with. The client's scanned, rotated, bilingual documents broke chunking, OCR and retrieval at once. Test on the ugliest real data you can get, first.
3. Hosting on hope
A free-tier server, a reranker and an embedding model do not fit together without a memory budget. The outage taught me to write the budget down — the free-tier memory-budget playbook exists because of it.
4. Under-logging
A client asked why an answer was wrong and I could not say which chunks were retrieved or which provider answered. Every request now logs an id, the model, the retrieval scores and timing — never the raw personal data.
5. Over-promising accuracy
"It will answer anything in your documents" is a sentence I said once. The confidence gate — the system saying "I don't know" — was as much a client-communication fix as a technical one.
6. Skipping the privacy conversation
Where does the data go? I learned to answer that before a client asked, after one uncomfortable meeting. FaceVision's embeddings-only design is the direct result; see privacy by design for AI apps.
What I would tell a student
Make these mistakes on a small project with a patient user, in Year 2, so that by the time a client is paying you have already stopped making them. That is what mentorship sessions are for, and why I keep telling this list at colleges.
— Pranjul Rathour, GenAI Engineer from Kanpur, India. Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at any campus: pranjulrathour41@gmail.com.