Search for “puppy” in a system that only matches keywords and a document about “dog training” will not show up, even though anyone reading both would call them closely related. Keyword search compares strings. It has no idea that “puppy” and “dog” mean nearly the same thing. Embeddings fix that. They turn a piece of text into a list of numbers, a vector, placed so that text with similar meaning lands near other text with similar meaning, whatever words it happens to use.

How an embedding model turns text into numbers

An embedding model is a neural network trained to map text to a point in a high-dimensional space: 384, 768 or 1536 dimensions are common sizes, depending on the model. It does not look words up in a dictionary. From huge amounts of text it has learned which words and phrases show up in similar contexts, and it encodes that into where the vector points.

Distance in that space tracks similarity in meaning. “King” and “queen” land close together. “King” and “golden retriever” land far apart. Nobody told the model to compare those words; the geometry comes out of training, and the comparison falls out of where each word ends up.

You can see the same shape with hand-picked numbers before any real model is involved. Give each word a 3-number vector (a real model uses hundreds):

1
2
3

king  = [0.90, 0.10, 0.85]
queen = [0.88, 0.12, 0.10]
dog   = [0.05, 0.95, 0.40]

“King” and “queen” agree closely on the first two numbers. “Dog” does not. To turn “agree closely” into a single score, you need a similarity metric.

Cosine similarity vs dot product vs Euclidean distance

Three metrics show up in embedding work, and each answers a slightly different question:

MetricWhat it measuresUse it when
Cosine similarityThe angle between two vectors, ignoring their lengthDefault for text embeddings, since most models are trained for it
Dot productAngle and length togetherThe model was explicitly trained for dot-product ranking (some retrieval models are)
Euclidean distanceStraight-line distance between the two pointsSpatial or count-based data; less common for text

Cosine similarity is the one you will reach for almost every time. Many text embedding models normalize their output to length 1, so direction is the only thing left that varies, and on normalized vectors cosine and dot product give the same ranking. The score runs from -1 (opposite) to 1 (identical), with 0 meaning unrelated. Here it is on the hand-picked vectors:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

import numpy as np

def cosine_similarity(a, b):
    a, b = np.array(a, dtype=float), np.array(b, dtype=float)
    return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))

king  = [0.90, 0.10, 0.85]
queen = [0.88, 0.12, 0.10]
dog   = [0.05, 0.95, 0.40]

print(cosine_similarity(king, queen))  # 0.801
print(cosine_similarity(king, dog))    # 0.375

0.801 against 0.375. “King” sits much closer to “queen” than to “dog”. A real model produces the same kind of gap, spread over hundreds of dimensions it learned from data instead of three picked by hand.

How to generate embeddings in Python

sentence-transformers is the standard open-source library for this. It runs locally with no API key:

1

pip install sentence-transformers

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
sentences = [
    "The weather is lovely today.",
    "It's so sunny outside!",
    "He drove to the stadium.",
]
embeddings = model.encode(sentences)          # shape: (3, 384)
print(model.similarity(embeddings, embeddings))

all-MiniLM-L6-v2 turns each sentence into a 384-number vector, and model.similarity computes cosine similarity for every pair. The two sentences about sunny weather share almost no words, yet they score high against each other and low against the one about the stadium. Same pattern as the toy example, except now the numbers come from a model that learned language.

Hosted APIs return the same thing. OpenAI’s text-embedding-3-small and Cohere’s embed endpoints both give you a list of floats per piece of text. If you already run models locally, Ollama serves embeddings too: ollama pull nomic-embed-text, then call its /api/embed endpoint (setup is covered in How to Run a Local LLM With Ollama). The library changes. The vector, and what you do with it, does not.

How vector similarity search finds the nearest match

Embedding one piece of text is the easy part. The real job is nearly always: given a query vector, find the closest matches among thousands or millions of stored ones. That is vector similarity search, and the direct version is brute force: score the query against every stored vector and keep the top results.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

import numpy as np, time

rng = np.random.default_rng(0)

for n in [10_000, 100_000, 1_000_000]:
    vectors = rng.normal(size=(n, 384)).astype(np.float32)
    query = rng.normal(size=384).astype(np.float32)

    start = time.perf_counter()
    scores = vectors @ query / (np.linalg.norm(vectors, axis=1) * np.linalg.norm(query))
    top5 = np.argsort(scores)[-5:]
    elapsed = time.perf_counter() - start
    print(f"n={n:>9,}  {elapsed*1000:.2f} ms")

On an ordinary machine, that prints something close to:

1
2
3

n=   10,000  0.70 ms
n=  100,000  7.82 ms
n=1,000,000  63.64 ms

Time grows linearly with the number of vectors, O(n) in the terms of Big O Notation Explained With Code Examples, but NumPy makes each step cheap enough that a million vectors take well under 100 ms. That stops being enough with heavy concurrent traffic, tens of millions of vectors, or both. At that point you need an approximate nearest neighbor (ANN) index, which does for vectors what a B-tree does for a WHERE clause (see How Database Indexes Work and When to Add One). Two kinds show up everywhere:

  • HNSW (Hierarchical Navigable Small World) builds a multi-layer graph linking each vector to its neighbors, so a search walks the graph toward the answer instead of touching every vector. It gives the best recall per millisecond of the common options, and pays for it with more memory and slower index builds.
  • IVFFlat groups vectors into clusters around centroids, then searches only the clusters nearest the query. It builds faster and uses less memory than HNSW, but it needs representative data loaded before you create it. Built on an empty table, it gets bad clusters and bad recall.

In Postgres with the pgvector extension, each is one statement:

1
2
3
4
5

-- HNSW: build any time, including on an empty table
CREATE INDEX ON items USING hnsw (embedding vector_cosine_ops);

-- IVFFlat: load representative data first, then build
CREATE INDEX ON items USING ivfflat (embedding vector_cosine_ops) WITH (lists = 100);

Both trade exactness for speed. They can miss the true closest vector, which is fine for search and recommendations and wrong for anything that needs the exact answer every time.

  • Querying with one model, storing with another. Vectors from two different models live in incompatible spaces. If you switch models, re-embed every stored document with the new one. A mix of old and new vectors returns garbage matches without raising any error.
  • Dot-product ranking on vectors that were never normalized. Dot product includes length, so a long vector can outrank a shorter, more relevant one. Normalize at write time, or use the cosine operator class, which ignores length.
  • Treating closeness as correctness. A high cosine score means “appears in similar contexts”, not “is the right answer”. A query about a drug’s side effects can pull back a chunk about the same drug’s dosage: related, and wrong. This is the main failure mode in What Is RAG: Retrieval-Augmented Generation Explained pipelines. Similarity narrows the candidates. It does not verify them.
  • Adding an ANN index before brute force is too slow. The benchmark above handles a million vectors in under 100 ms. HNSW or IVFFlat adds build time, parameters to tune and approximate results. Worth it past tens of millions of vectors or under heavy concurrent load, not by default.
  • Forgetting to re-embed when the source changes. An embedding is a snapshot of the text at the moment you embedded it. Edit the document without re-embedding and search keeps returning the old version with full confidence.

Do you need a vector database?

Not to start. Embed a few hundred of your real documents with sentence-transformers, keep the vectors in a NumPy array and search them with the brute-force loop above. No database, no index. Move to pgvector or a managed vector database once brute force is measurably too slow for your data size and query volume. Add an ANN index on top only when the table scan is the bottleneck you have actually measured.