Day 2: Embeddings & Vector Databases — How Computers Understand Meaning

This blog post is a daily learning summary of my 40 Day RAG class from Syed Jaffer of Parotta Salna.

Why Keyword Search Isn't Enough

Your knowledge base says:

"My dog brings me so much joy."

A user asks:

"How does owning a pet improve happiness?"

Zero shared keywords. A regular Ctrl+F-style search would miss it completely. Even though the document clearly answers the question.

We need something that understands meaning, not spelling. That's what embeddings do.

What is an Embedding? (The GPS Analogy)

Imagine a giant map where every word, sentence, or paragraph has a location based on its meaning:

1. "dog" and "puppy" sit right next to each other
2. "dog" and "wolf" are nearby
3. "dog" and "spaceship" are on opposite ends

An embedding is just the coordinates of a piece of text on this meaning map.

The twist: instead of 2D (latitude, longitude), embeddings live in hundreds or thousands of dimensions. That's how they capture so much subtlety.

Embeddings let you do math on meaning:

king − man + woman ≈ queen

That's how rich the encoding is.

How Do We Create Embeddings?

You don't train your own. You use a pre-trained embedding model.

Popular ones:

1. OpenAI's text-embedding-3
2. Cohere Embed, or the free sentence-transformers from Hugging Face.

Feed text in → get a list of numbers out. That's the whole interface.

⚠️ The One Rule You Can't Break

Use the same embedding model for documents AND queries.

Different models = different coordinate systems. Mixing them is like asking for directions using Mars GPS on an Earth map. Nothing matches.

Measuring Closeness: Cosine Similarity

Once everything is on the meaning map, we need a way to ask "how close are these two things?"

The most common answer: cosine similarity.

Picture each embedding as an arrow shooting out from the centre. Cosine similarity asks: how much do these two arrows point in the same direction?

1. 1.0 → same direction (identical meaning)
2. 0.0 → perpendicular (unrelated)
3. −1.0 → opposite (rare in practice)

It cares about direction, not length, which is why it works well even when texts are different sizes.

What is a Vector Database?

We've embedded a million chunks. Now we need to store them and quickly find the ones closest to a query embedding.

Could you dump them into Postgres and brute-force compare every row? Sure, until you have 10 million chunks and queries take forever.

A vector database is purpose-built for one job:

"Given this query vector, find the top-k closest vectors fast, even at scale."

It pulls this off with a clever indexing trick called Approximate Nearest Neighbour (ANN) search — narrowing the search space dramatically with a tiny accuracy tradeoff.

Popular Vector Databases

1. Chroma: Local prototyping, beginner-friendly
2. FAISS: High-performance library from Meta
3. Pinecone: Fully managed cloud service
4. Weaviate / Qdrant: Open-source, production-ready
5. pgvector: Adds vectors to your existing Postgres

Start with Chroma or FAISS. They run on your laptop, no signup needed.

The Build Phase, Visualized

The original text is stored alongside the embedding because the LLM can't read embeddings — only actual text. The embedding is for finding; the text is for reading.

Notes:

What's an embedding, in plain English? A list of numbers representing the meaning of text, like GPS coordinates on a "meaning map" where similar meanings sit close together.

Why use embeddings instead of keyword search? They capture meaning, not just exact words. They match "my dog brings me joy" with "owning a pet improves happiness."

What's the most important embedding rule? Use the same model for documents and queries. Different models = incompatible coordinate systems.

What's cosine similarity? A score (0 to 1) for how aligned two embedding arrows are. Higher = more similar in meaning.

Why a vector database instead of SQL? Vector DBs use ANN indexing for fast nearest-neighbour search at scale. Plain SQL would brute-force compare every row.

Day 2 Takeaway

Embeddings turn text into coordinates on a "meaning map." Vector databases find the closest coordinates fast. Together, they let computers search by meaning instead of matching keywords.

Coming Up on Day 3

We've talked about embedding "chunks", but what is a chunk, and how do we split documents in the first place? Tomorrow we tackle the unsexy decision that quietly determines whether your RAG works at all:

"Bad chunking = bad RAG. Period."

See you on Day 3.