What's the difference between keyword and semantic search?

Keyword (lexical) search, like BM25, matches exact terms and weights rare ones — precise but blind to synonyms. Semantic (vector) search encodes meaning into embeddings and matches by similarity, so it catches paraphrase and related concepts but can miss exact tokens like IDs and error codes. They're strong in opposite situations.

What is hybrid search?

Hybrid search runs both keyword (BM25) and vector retrieval in parallel and merges their ranked results into one list with a fusion algorithm. It exists because the two methods fail differently — vector handles synonyms and concepts, BM25 nails exact terms, identifiers, and rare jargon — so combining them improves both recall and precision.

When does hybrid search beat pure vector search?

When queries mix natural language with exact identifiers, use rare or domain-specific jargon, run against out-of-domain content, or are multilingual. Pure vector search can be enough for purely conceptual questions over well-embedded, in-domain data. For technical knowledge bases full of codes and product names, hybrid is clearly better.

Why does vector search miss exact terms and IDs?

Embeddings compress text into a fixed-size vector that captures meaning, not exact spelling, so rare tokens like error codes, SKUs, and acronyms get smoothed away — a property sometimes called structural blindness. The model knows what a passage is about but not that it contains the literal string you searched for, which is exactly what BM25 is good at.

RRF or weighted fusion — which should I use?

Start with RRF: it's robust, parameter-light, and needs no per-corpus tuning. Switch to weighted (alpha) fusion when you have a clear reason to bias one retriever — a lower alpha to favor exact-term matching on identifier-heavy content, a higher alpha to favor semantics on natural-language questions.

Does hybrid search add latency or cost, and is it worth it?

It runs two retrievals plus a fusion step, so there's some overhead, and reranking adds more. For most RAG systems the recall and precision gains justify it, especially on technical or mixed-content corpora. If you're on well-embedded, purely conceptual data and latency is critical, pure vector may be enough — measure on your own queries.

← Resources

By Sora Kashiwagi — Senior Engineer, Vector Search · 2026-05-16 · TUTORIAL

Hybrid Search: Combining BM25 and Vector Search for RAG

Keyword (BM25) and vector search fail in opposite ways, which is exactly why production RAG combines them. This guide explains both methods, how fusion (RRF) merges their results, where reranking fits, and when hybrid meaningfully beats pure vector search.

Two kinds of search that fail differently

BM25 (lexical or keyword search) scores documents on exact-term overlap, weighting rare terms more and penalizing length. It's deterministic and runs over an inverted index. Vector (semantic) search encodes queries and documents into dense embeddings and retrieves by nearest-neighbor similarity, capturing meaning, synonyms, and paraphrase — even across languages.

Their weaknesses are mirror images. BM25 misses synonyms and concepts: a query for "database memory issues" won't find a page titled "cache eviction policies" if the literal words differ. Vector search misses exact tokens — acronyms, SKUs, error codes, IDs, model numbers — and suffers semantic drift, returning embedding-near-but-wrong results like "Adidas running shoes" for a "Nike running shoes" query. Because the failure modes are complementary, neither alone is enough.

What hybrid search does

Hybrid search runs lexical (BM25) and dense (vector) retrieval in parallel, then merges their two ranked result lists into one. The merge step is the interesting part, because BM25 scores and vector similarity scores live on completely different scales — you can't just add them.

That's why the dominant fusion method, Reciprocal Rank Fusion (RRF), uses only rank position, not raw scores. Each document's fused score is the sum, across both retrievers, of 1 divided by (k plus its rank), where k is a small constant (commonly 60; some studies use 20). Because it ignores the score distributions entirely, RRF is normalization-free and works "out of the box" — which is why it's the default in most vector databases.

RRF vs weighted fusion

RRF isn't the only option. Weighted (alpha) fusion normalizes the two score sets and blends them with a tunable weight: alpha of 0 is pure keyword, 1 is pure vector, 0.5 is equal, and defaults often sit around 0.75 toward semantic. Some engines also offer relative-score and distribution-based fusion.

The trade-off: RRF is robust and parameter-light, ideal when you don't want to tune per-corpus. Weighted fusion gives you a dial to favor exact-term matching (lower alpha) for identifier-heavy technical content, or semantics (higher alpha) for natural-language questions. Start with RRF; reach for weighted fusion when you have a clear reason to bias one retriever.

Where reranking fits

Hybrid retrieval is the first stage; reranking is the second. A fast retriever (BM25, vector, or hybrid) pulls a candidate set, then a cross-encoder rescores each candidate by jointly encoding the query and the document in a single pass with full cross-attention. Cross-encoders are far more accurate than the first-stage retrievers but too expensive to run over the whole corpus, so they only rescore the top-k candidates.

The canonical production pipeline is therefore: hybrid retrieval (BM25 + vector, fused with RRF) → cross-encoder or LLM reranker → top results into the model's context. Each stage does the job it's best at — recall first, precision last.

Does it actually help? And building it in osFoundry

The gains are real but depend on your data. Elastic's BEIR study found RRF hybrid lifted average NDCG@10 by about 18% over BM25 alone and a smaller margin over a strong learned encoder, and was as good or better than BM25 on every dataset tested. As dense models have improved, the marginal hybrid gain has narrowed — often a few percent on in-domain data — but it stays largest on out-of-domain corpora, rare terms, and queries that mix natural language with exact identifiers. Hybrid remains the production default.

osFoundry's custom RAG pipeline exposes this as composable stages rather than a black box: a BM25 stage and a vector stage run in parallel, a fusion stage (RRF by default, weighted as an option) merges them, and a reranking stage refines the result before it reaches the model. You tune the lexical-versus-semantic balance per knowledge base in config, matching the retrieve-fuse-rerank pattern without writing the plumbing.

Frequently asked questions

What's the difference between keyword and semantic search?: Keyword (lexical) search, like BM25, matches exact terms and weights rare ones — precise but blind to synonyms. Semantic (vector) search encodes meaning into embeddings and matches by similarity, so it catches paraphrase and related concepts but can miss exact tokens like IDs and error codes. They're strong in opposite situations.
What is hybrid search?: Hybrid search runs both keyword (BM25) and vector retrieval in parallel and merges their ranked results into one list with a fusion algorithm. It exists because the two methods fail differently — vector handles synonyms and concepts, BM25 nails exact terms, identifiers, and rare jargon — so combining them improves both recall and precision.
What is Reciprocal Rank Fusion (RRF)?: RRF merges two ranked lists using only rank position, not raw scores. Each document's fused score is the sum across retrievers of 1 divided by (k plus its rank), where k is a small constant (commonly 60). Because it ignores the incompatible score scales of BM25 and vector search, it needs no normalization and works well out of the box.
When does hybrid search beat pure vector search?: When queries mix natural language with exact identifiers, use rare or domain-specific jargon, run against out-of-domain content, or are multilingual. Pure vector search can be enough for purely conceptual questions over well-embedded, in-domain data. For technical knowledge bases full of codes and product names, hybrid is clearly better.
Why does vector search miss exact terms and IDs?: Embeddings compress text into a fixed-size vector that captures meaning, not exact spelling, so rare tokens like error codes, SKUs, and acronyms get smoothed away — a property sometimes called structural blindness. The model knows what a passage is about but not that it contains the literal string you searched for, which is exactly what BM25 is good at.
RRF or weighted fusion — which should I use?: Start with RRF: it's robust, parameter-light, and needs no per-corpus tuning. Switch to weighted (alpha) fusion when you have a clear reason to bias one retriever — a lower alpha to favor exact-term matching on identifier-heavy content, a higher alpha to favor semantics on natural-language questions.
Does hybrid search add latency or cost, and is it worth it?: It runs two retrievals plus a fusion step, so there's some overhead, and reranking adds more. For most RAG systems the recall and precision gains justify it, especially on technical or mixed-content corpora. If you're on well-embedded, purely conceptual data and latency is critical, pure vector may be enough — measure on your own queries.