Embeddings are how machines compare meaning.

What are embeddings?

Embeddings are numeric representations of things, usually text, where the geometry carries meaning: two sentences about the same idea end up close together, unrelated ones far apart. A model trained on huge amounts of text learns to place inputs this way. Once everything is a vector you can measure, sort and cluster meaning with ordinary arithmetic, which computers are very good at.

What are embeddings in AI?

In AI applications embeddings are the bridge between human language and search. A language model generates text; an embedding model represents it. You embed your documents once and store the vectors; you embed each question and find the nearest documents. In RAG.NextUpgrad every chunk of every document is embedded at index time, and every question at query time, with the same model, so the distances are comparable.

What is text embedding?

A text embedding is the vector for a piece of text: a word, a sentence, a paragraph or a whole chunk. Modern embedding models are transformers trained with contrastive objectives: pairs of related texts are pulled together and unrelated ones pushed apart. Good current choices include open models from the BGE and E5 families and hosted ones from OpenAI, Google and Cohere. Check the MTEB leaderboard for your language and task before choosing.

Choosing a model comes down to four questions. Which languages do your users write in, because many strong English models degrade sharply on Hindi or mixed-script text. How long are your chunks, because each model has a maximum input length and silently truncates beyond it. How many dimensions can you afford to store and search, since 1,024 floats per chunk across a million chunks is real memory. And can you run it yourself, which matters for private data and for cost at scale. Small open models run on a laptop CPU and are good enough for most student projects; hosted models are convenient but add a network call per query and a per-token bill. Whatever you pick, use the same model for documents and queries, and re-index everything if you ever change it.

What is image embedding?

An image embedding is the same idea for pictures: a vision model maps an image to a vector so visually or semantically similar images are close. Models like CLIP go further and place images and text in the same space, so you can search photos with words. In FaceVision I use a face embedding: the browser computes a compact vector from a face and the server stores and matches only that vector, never the photo.

What are embeddings in machine learning?

In machine learning more broadly, an embedding is any learned dense representation of a discrete thing: words, users, products, categories. Word2vec in 2013 showed that vectors learned from context capture relations like king minus man plus woman landing near queen. Recommendation systems embed users and items in one space and recommend by proximity. Today's text embeddings are the transformer-era descendant of that idea.

How embeddings are generated

Text is tokenised, passed through a transformer encoder, and the token outputs are pooled, usually by averaging or by taking a special first token, into one fixed-length vector. The vector is then normalised to unit length. Training shapes the encoder so that pooled vectors of related texts have high cosine similarity. Generating an embedding for a chunk takes milliseconds on a CPU for small models; batching makes indexing a large corpus practical.

How embeddings work

Each dimension of the vector is not a human-readable feature; meaning is spread across all of them. What matters is the relative position. Cosine similarity between two embeddings is a number between minus one and one, where higher means more alike. A search computes that number between the query and every candidate (or an approximate subset through an index) and returns the highest. That is all a semantic search is.

Why are embeddings useful?

• They match meaning, not spelling, so a question phrased differently from the document still finds it.
• They work across languages when the model is multilingual.
• They compress a paragraph into a few kilobytes that can be indexed and compared at scale.
• They can be fine-tuned on your own query-document pairs, which lifts retrieval quality in a narrow domain.

Why normalize embeddings?

Normalising scales every vector to length one so that cosine similarity equals the dot product, which is faster to compute and lets the index use the cheaper operation. It also stops long or repetitive texts from scoring high merely because their vectors are larger. Most embedding APIs return normalised vectors already; if you compute your own, normalise before storing and before querying.

What is chunking and embedding?

Chunking splits a document into pieces; embedding turns each piece into a vector. They are done together at index time and the quality of one limits the other. A chunk that mixes two topics produces an embedding that sits between them and matches neither well; a chunk cut mid-sentence loses meaning. Chunk by structure, keep a small overlap, embed each chunk, and store the text and its source alongside the vector.