Embeddings Are Not Retrieval, Where Cosine Similarity Fails

Pull-quote: “Cosine similarity tells you two passages live in the same neighborhood. Retrieval has to tell you which passage answers the question. Those are different jobs, and the space between them is where RAG systems fail without an error message.”

Why this matters

The default RAG recipe treats embedding search as if it were retrieval: chunk the corpus, embed everything, embed the query, return the nearest neighbors. Over a small, clean corpus that recipe demos well. In production it fails in a specific and dangerous way: it fails silently. Top-k always returns something. Similarity scores always look reasonable. The generator writes a fluent answer over whatever arrived. Nothing throws, and nothing is right.

The root cause is a category error. Embedding similarity measures resemblance: two texts that discuss the same topic in the same register land close together. Relevance is a different property: whether a passage contains what the question actually needs. Resemblance and relevance agree often enough to make dense retrieval look finished, and diverge often enough to hurt you where it counts.

Six places cosine quietly fails

The common thread is that every failure returns confident, on-topic, wrong context. A generator downstream will then dress wrong context in good prose, which is exactly the outcome retrieval was supposed to prevent.

The stack that catches it

            ┌──────────────────────────┐
            │ Lexical path (BM25)      │  exact terms, codes, clause numbers
   Query ──►│ Dense path (embeddings)  │  paraphrase, synonyms, topic
            │ Metadata filters         │  revision, date, document class
            └────────────┬─────────────┘
                         ▼
          Fusion (reciprocal rank fusion)
                         ▼
          Cross-encoder rerank ──► top k, with provenance

Each path covers a failure class the others miss. BM25 treats identifiers and rare terms as anchors instead of noise, which is why a clause number query comes back with the clause. Metadata filters delete superseded revisions and wrong document classes before similarity ever runs, which is the only reliable cure for version blindness. Reciprocal rank fusion combines the ranked lists without tuning drama. A cross-encoder reranker then reads query and passage together, token by token, and restores the sensitivity to negation and specificity that bi-encoder embeddings blur. None of this is exotic. It is the difference between a similarity engine and a retrieval system.

Where this shows up in the field

Aviation safety corpora make the case vividly. A collection of a few hundred thousand incident reports is dense with system names, event codes, and abbreviations that general-purpose embedding models handle poorly, so the lexical path is load-bearing, not legacy. The dense path earns its keep on paraphrase: the same failure described five different ways by five different crews. Manufacturing quality systems push from the other side: engineers and auditors cite clause numbers, and an answer sourced from a superseded clause revision is worse than no answer, so filters have to run before similarity does.

Closing

Embeddings are a component, not an architecture. Dense similarity buys real recall on paraphrase and topic, and it will happily rank resemblance over relevance on identifiers, polarity, and currency. Run the lexical path beside it, filter before you search, fuse the lists, rerank the survivors, and treat cosine score as one signal among several. Retrieval quality is a system property, and systems are what survive contact with production.