Embedding (AI)

An embedding is a dense numerical vector that represents the semantic content of a piece of text, image, or audio in a continuous vector space. Items with similar meaning have vectors that are close together, enabling similarity search, clustering, and classification without hand-crafted features.

Embeddings are produced by encoder models (OpenAI text-embedding-3, Cohere Embed, Google text-embedding-gecko) and stored in vector databases for retrieval. The quality of embeddings directly determines the quality of RAG retrieval: a poor embedding model returns irrelevant documents, which the LLM then reasons over incorrectly. Embedding model selection is one of the highest-leverage decisions in any RAG system design.