Every stage of the journey was a how does this work.

What is vector database and how it works

The first thing I had to understand to build a document assistant. A vector database stores embeddings, number lists that capture the meaning of a chunk of text, and answers one question fast: which stored items are closest to this one? It does that with an approximate nearest-neighbour index such as HNSW, a graph that lets a search hop towards the query without comparing against everything, plus metadata filters so you can restrict to one document or customer. I started with FAISS in a notebook, moved to pgvector in Postgres for the product, and learned that the store matters less than the chunking and the embedding model in front of it.

How is semantic search done?

Embed every chunk once with an embedding model; embed the query with the same model; return the chunks whose vectors are closest by cosine similarity. That is the whole mechanism, and it finds passages that mean what you asked even when they share no words with it. The surprise, building RAG.NextUpgrad, was how often it missed exact terms, names, codes, numbers, that a keyword search would catch instantly. The fix was hybrid search: run BM25 keyword retrieval alongside, fuse the two ranked lists with reciprocal-rank fusion, and re-rank. Semantic search alone is a demo; hybrid search with re-ranking is a product.

How is ranking done in RAG?

In stages, each cheaper and coarser than the next. First-stage retrieval, vector and keyword, returns a wide shortlist of maybe twenty candidates, optimised for recall. Fusion merges the two lists, rewarding passages both retrievers liked. Then a cross-encoder re-ranker reads the question and each candidate together and scores how well that passage answers it, which is far more accurate than vector distance; the top three to five go to the model. In RAG.NextUpgrad a confidence gate sits after re-ranking and refuses to answer when the best score is weak. Learning that ranking is a pipeline, not a single lookup, was the moment retrieval quality became something I could measure and improve.

How is LLM performance measured?

With a labelled set and a scoring rule, the same as any software: real inputs, expected outputs or grading criteria, run on every change. For retrieval, recall at k and mean reciprocal rank; for answers, faithfulness to the retrieved passages, relevance, format validity and correct refusals; for the system, latency and cost. Public benchmarks rank models in general and are only a shortlist tool. The habit that changed my work was building the evaluation set before the pipeline and running it in CI, because a prompt change that looked harmless once broke a case that mattered, and the test caught it before a user did. Every project since has shipped with its test suite.

How is RLHF done?

Reinforcement learning from human feedback starts from an instruction-tuned model, collects human comparisons between pairs of its responses, trains a reward model to predict which response people prefer, and then optimises the language model with reinforcement learning to produce higher-reward responses while a penalty keeps it close to where it started. It is how chat models became helpful and polite, and it is lab-scale work. What I actually use is DPO, direct preference optimisation, which reaches a similar result from preference pairs with a simple loss and no reward model, and runs in the same tools as supervised fine-tuning. Understanding RLHF explained why models flatter users; understanding DPO let me do something about it on my own data.

How is synthetic data used?

To fill gaps in a fine-tuning dataset: a strong model drafts examples from a few real seeds, a person corrects them, and the results cover edge cases, rare categories or situations where real data is private. It is also used to build evaluation sets with known answers from templates, and to distil a large model's behaviour into a small one by training on its outputs to real inputs. The rule I learned on FineTune Studio: real inputs wherever possible, synthetic outputs only after a human edit, synthetic inputs only for named gaps, and nothing synthetic in the held-out set. Synthetic data is a tool for covering what reality did not give you, not a replacement for reality.

How is Gemini API charged?

Per token, like every model API: separate rates for input and output per model, discounts for cached context, a free tier with rate limits for keys from AI Studio, and paid rates once billing is enabled on the Google Cloud project. The first thing I do with any provider now is count the tokens in a typical request, multiply by the current rates and by expected volume, and set a budget alert, because the surprise bill is always from a loop or a long prompt sent too often. Free tiers from Gemini and Groq carried most of my student projects; caching and routing by difficulty carry the production ones.

How is Groq API free?

Groq serves open-weight models on its own inference hardware and offers a free tier with per-minute and per-day limits, the usual developer on-ramp that converts some users into paying customers; open models cost nothing to license and the hardware is efficient per token, so a rate-limited free tier is sustainable. For a student it is real and enough for prototypes. DocuLens AI, my document extraction system, runs on Groq-hosted Llama because the speed makes page-by-page extraction feel instant and the cost on the paid tier stays low, with a fallback provider behind the same interface so a rate limit never stops the pipeline.

How image embeddings work

A vision model, typically a vision transformer or a convolutional network trained contrastively, maps an image to a vector such that visually or semantically similar images land close together; models like CLIP place images and text in the same space so a sentence can find a picture. In FaceVision the embedding is of a face: the browser computes a compact vector from the aligned face crop and the server stores and compares only that vector, never the image, which is what makes the system private by design. Understanding that an embedding is a comparable summary, whether of a paragraph or a face, unified the retrieval work and the vision work in my head.

How is CLIP model trained?

On hundreds of millions of image-caption pairs from the web, with a contrastive objective: an image encoder and a text encoder are trained together so that each image's vector is close to its own caption's vector and far from the other captions in the batch. No labels beyond the captions are needed. The result can classify images by comparing them with text prompts it has never seen, search images with sentences, and serve as the vision backbone inside multimodal language models. Learning how CLIP was trained was the point where generative AI and computer vision stopped being separate subjects for me and became one field with embeddings at its centre.