When retrieval is just a slow grep with extra steps

Embeddings help when the query and the document do not share words. They do not excuse a messy corpus.

I keep seeing teams add a vector store to a wiki that nobody has cleaned. The first demo looks magical. The second week, the assistant quotes a policy from 2023 and ignores the page that replaced it.

Before you tune a chunk size, ask whether a well-scoped lexical search already finds the right document. If it does, the work is in metadata, access control, and recency, not in a new embedding model. If it does not, look at whether the corpus is split in a way that a human would also fail to search.

A pattern that has worked for us: lexical search first, embeddings as a second lane, and a hard rule that every cited passage must include a document id and a last-updated date. If the model cannot name its source, it should say it does not know.

RAG is a document system with a language model on top. If the document system is sloppy, the model will be confidently sloppy at scale.