Semantic Search vs Keyword Search, and When to Use Both

Every AI-adjacent startup eventually rips out its keyword search and replaces it with embeddings. Then a customer types an exact SKU and gets nothing back. Semantic search vs keyword search isn't a versus at all — it's a "which for which query," and the answer usually involves both.

This is the version of the conversation we have when a team says "search is broken" and wants to know what to do about it.

What each is actually good at

Keyword search — BM25, Postgres full-text, Elastic — is built on the assumption that the words in the query appear in the document. When they do, it's fast, cheap, and hard to beat. Product SKUs, error codes, exact phrases, names: keyword wins.

Semantic search embeds the query and documents into vectors, then compares similarity. It handles paraphrases, synonyms, and intent. "How do I cancel my plan" matches a doc titled "Ending your subscription." Keyword search misses that entirely.

Why teams get burned swapping one for the other

The temptation after a bad demo is "let's just use embeddings for everything." Two weeks later, users can't find anything by ID. Ticket titles that used to work don't anymore.

The failure mode isn't semantic search being bad. It's that semantic search is optimizing for a different thing — meaning, not string match. Users mix both kinds of queries in the same session without warning.

The hybrid pattern that actually works

Run both searches. Combine the results. Rank.

The combination step is where most implementations get lazy. A naive union produces a mess. The two approaches that hold up in production:

Reciprocal rank fusion

Take each result's rank in each list, compute 1 / (k + rank) for a small constant k, and sum across lists. Sort by the sum. It's simple, parameter-light, and beats a lot of fancier schemes in practice.

Rerank with a small model

Retrieve 50 candidates from each side, then pass all 100 through a rerank model (Cohere Rerank, a small cross-encoder). Return the top 10 by rerank score. More expensive, noticeably better quality on ambiguous queries.

Where each approach costs you

Keyword search fails silently on synonyms. If your docs say "cancel" and users type "unsubscribe," you serve zero results and blame the user.

Semantic search fails on precision. Embeddings tend to return "roughly related" results even when nothing truly matches. Users typing exact identifiers get plausible-looking wrong answers, which is worse than an empty state.

Hybrid fixes most of it but adds a second index to maintain and two sets of latency to budget for.

Implementation notes from real projects

Postgres with pg_trgm plus pgvector handles both sides in one database up to a few million rows. This is our default for anything short of "search company." It removes an entire piece of infrastructure.

Beyond that scale, or when relevance really matters (marketplaces, help centers with heavy traffic), a dedicated engine like Typesense, Meilisearch, or OpenSearch pays for itself.

Embedding model choice matters less than most teams think at the start. Pick one from a stable provider, ship, and don't touch it for six months. Related reading: Choosing an embedding model.

Mistakes we keep seeing

Chunking documents into 8k-token chunks and expecting good retrieval. Semantic search rewards small, focused chunks. Related reading: Chunking strategies for RAG.

Not measuring anything. If you can't tell whether a search change made things better or worse, you're guessing. Even 20 hand-labeled queries with expected results catches regressions.

Treating the search bar as one feature. In-product search, help center search, admin search, and doc search have different query shapes. One-size-fits-all tuning is usually worse than three specialized surfaces.

Common misconceptions

"Embeddings replace BM25." They don't. They complement it.

"Larger embedding models are always better." Not for search. Latency and cost matter, and reranking often gives you more relevance improvement than a bigger base model.

"Vector search is expensive." Modern pgvector on a reasonable Postgres instance handles millions of embeddings under 100ms. Expensive is optional.

A concrete rule of thumb

• Search over user data, small scale, mostly exact terms → keyword only, ship it.
• Help center, docs, natural-language questions → hybrid.
• Product catalog with browsing and long-tail queries → hybrid with rerank.
• Internal enterprise search across mixed content → hybrid, and budget for evaluation.

FAQ

Do I need a vector database?

Only when Postgres with pgvector stops keeping up. That's later than you'd expect — usually tens of millions of vectors, or when you need multi-tenant isolation with heavy fan-out.

Is reranking worth it?

For any search where wrong answers embarrass you (docs, support), yes. For low-stakes autocomplete, no.

What about full-text search in Postgres?

Underrated. tsvector plus tsquery handles a huge amount of real-world search well, and it lives in the database you already have.

Where to go from here

If your search is missing obvious queries or returning plausible-but-wrong results, the diagnosis is usually a missing hybrid layer or bad chunking. Our AI product engineering engagements often begin with a two-week search revamp before anything else.