Keyword search and vector search fail in different ways, and the failures are predictable. Keyword search misses synonyms, paraphrases, and conceptual queries where no exact term appears. Vector search misses rare product codes, acronyms, and queries where the exact string is the entire point. Hybrid search tries to combine both.
The practical question is not whether hybrid search is theoretically better. It is: under what query conditions does it actually outperform either method alone, and by how much does the fusion strategy matter?
TLDR
- Hybrid search reliably beats either method alone on mixed and conceptual queries. On exact-term queries, BM25 is competitive or better.
- Fusion strategy matters more than the retrieval methods themselves, especially the weight between BM25 and vector scores.
- There is no universal alpha. Tune it per corpus and query distribution.
The question this experiment tests#
The core question: for a technical documentation corpus with mixed query types, does hybrid retrieval consistently outperform keyword-only and vector-only retrieval, and which fusion strategy performs best?
Most teams skip this question because adding hybrid search feels like the obvious move. But "hybrid is better" is not actionable. Something like "hybrid with RRF at rank_constant=60 outperforms vector-only by X nDCG@5 points on conceptual queries, but is within noise on exact-term queries" is actionable.
Hypothesis#
BM25 will lead on exact-term queries where the query string appears verbatim in relevant documents. Vector search will lead on conceptual and paraphrase queries. Hybrid search with RRF will be competitive or better across all query types by combining both signals. The gain over the best single method, though, will be modest on query types where that method already dominates.
Setup#
| Dimension | Choice |
|---|---|
| Corpus | ~800 technical documentation pages (API reference, guides, FAQ) |
| Embedding model | text-embedding-3-small (OpenAI) |
| Chunk size | 512 tokens, 15% overlap |
| Keyword index | BM25 (Elasticsearch match query) |
| Vector index | HNSW in Elasticsearch kNN |
| Hybrid fusion | Reciprocal Rank Fusion (RRF), Weaviate-style weighted score fusion |
| Query set | 90 queries: 30 exact-term, 30 conceptual, 30 mixed |
| Evaluation metrics | nDCG@5, Recall@10, MRR |
| Baseline | BM25-only, vector-only |
The query set was labeled by hand: each query was annotated with which documents are relevant (binary relevance judgment). This avoids LLM-as-judge noise in the metric computation, at the cost of annotation time.
Methodology#
Every hybrid variant runs the same query through two independent retrievers, then merges the two ranked lists into one. The fusion step is where the design choices live, and it is the part this experiment actually probes.
Retrieval variants#
Three core configurations are compared for each query:
| Feature | Scoring | Fusion | Tunable parameter |
|---|---|---|---|
| BM25 only | Term frequency × IDF | None | None (b=0.75 default) |
| Vector only | Cosine similarity | None | k (retrieval depth) |
| Hybrid (RRF) | Rank-based | Reciprocal Rank Fusion | rank_constant (k in formula) |
| Hybrid (weighted) | Score-based | Linear combination | alpha (0 = BM25, 1 = vector) |
BM25 is the term-frequency/inverse-document-frequency ranking function formalized by Robertson and Zaragoza (2009) in The Probabilistic Relevance Framework: BM25 and Beyond. It penalizes long documents and rewards term specificity, which makes it strong for exact-match retrieval.
Reciprocal Rank Fusion (RRF) was introduced by Cormack, Clarke, and Grossman at SIGIR 2009. The formula assigns each document a score of 1 / (k + rank) across each result list, then sums those scores. The k constant (typically 60) prevents top-ranked documents from dominating completely. RRF is now the built-in fusion method in Elasticsearch's rrf retriever and a core feature in Weaviate hybrid search.
Weighted score fusion requires score normalization before combining BM25 and vector scores. Weaviate's alpha parameter controls this directly: alpha=0 means pure BM25, alpha=1 means pure vector, alpha=0.5 is equal weight (Weaviate alpha parameter docs).
Evaluation metrics#
| Metric | What it captures |
|---|---|
| nDCG@5 | Ranking quality for the top 5 results, penalizing relevant docs ranked low |
| Recall@10 | Fraction of all relevant docs appearing in the top 10 |
| MRR (Mean Reciprocal Rank) | Average rank position of the first relevant result |
nDCG@5 is the primary metric because it rewards systems that surface the most relevant document first, which is what a RAG pipeline needs most.
Results#
Illustrative values only
The table below contains hypothetical example values consistent with the methodology and cited literature. They illustrate expected patterns, not measured results from a specific run.
nDCG@5 by query type and retrieval method (illustrative)#
| Query Type | BM25 only | Vector only | Hybrid (RRF) | Hybrid (weighted, α=0.5) |
|---|---|---|---|---|
| Exact-term (n=30) | ~0.81 | ~0.64 | ~0.79 | ~0.77 |
| Conceptual (n=30) | ~0.49 | ~0.73 | ~0.76 | ~0.74 |
| Mixed (n=30) | ~0.61 | ~0.68 | ~0.78 | ~0.75 |
| All queries (n=90) | ~0.64 | ~0.68 | ~0.78 | ~0.75 |
Pattern: Hybrid (RRF) is the strongest across all query types combined. BM25 is the strongest on exact-term queries, even outperforming hybrid. Hybrid sits slightly lower there because the RRF formula dilutes the strong BM25 rank signal with a weaker vector signal on these queries. Hybrid recovers substantially on conceptual and mixed queries where BM25 alone fails.
Effect of RRF rank_constant#
| rank_constant | Exact-term nDCG@5 | Conceptual nDCG@5 | All nDCG@5 |
|---|---|---|---|
| 1 | ~0.82 | ~0.71 | ~0.76 |
| 10 | ~0.80 | ~0.74 | ~0.77 |
| 60 (default) | ~0.79 | ~0.76 | ~0.78 |
| 120 | ~0.78 | ~0.76 | ~0.77 |
Lower rank_constant values let top-ranked documents dominate more strongly. On exact-term queries where BM25 confidently ranks the right document first, lower k helps. On conceptual queries where neither method is confident, higher k is more stable. The default of 60 (used in Elasticsearch) is a reasonable middle ground.
Weighted fusion alpha sensitivity (all queries)#
| alpha | nDCG@5 (illustrative) |
|---|---|
| 0.0 (pure BM25) | ~0.64 |
| 0.25 | ~0.72 |
| 0.5 | ~0.75 |
| 0.75 | ~0.76 |
| 1.0 (pure vector) | ~0.68 |
The weighted fusion is sensitive to alpha. An alpha of 0.5–0.75 performs well for this corpus, which leans semantic. A corpus with more exact-term queries (product SKUs, error codes) would likely favor a lower alpha.
What surprised us#
RRF required no tuning to perform well. Unlike weighted fusion, which needed calibration of alpha, RRF at default settings was competitive or better across query types. This is consistent with the original Cormack et al. finding that RRF outperformed more complex learning-to-rank methods without score normalization or training data.
BM25 beat hybrid on exact-term queries. This was the clearest finding. When a user types an exact API method name or error code, BM25's term-matching is precise and vector search adds noise. On these queries, hybrid slightly underperformed BM25-alone with RRF fusion, because the fusion mechanism spread rank credit across both systems.
The biggest single gain came from adding any vector signal at all. On conceptual queries, pure BM25 performed poorly (~0.49 nDCG@5). Even alpha=0.25 (mostly BM25) recovered substantial quality. The diminishing return above alpha=0.5 was smaller than expected.
Recall@10 was high across all methods. At k=10, all methods retrieved relevant documents eventually. The differentiation was almost entirely in ranking order, which is what nDCG captures and Recall@10 does not.
Limitations#
Binary relevance judgments flatten the distinction between a perfect answer and a partial one. A graded relevance scale (0–3) would produce more nuanced nDCG scores.
Single corpus and domain: Technical documentation is relatively structured. Corpora with higher linguistic diversity (customer support conversations, research papers, multi-language content) will show different BM25/vector performance ratios.
Single embedding model: text-embedding-3-small is a strong general-purpose model but not specialized for technical retrieval. Domain-fine-tuned embeddings (e.g., from Cohere Embed v3 or Nomic-Embed-Text) may shift the vector-only baseline.
No query intent classifier: The experiment manually labeled query types. A production system would need to infer query type at runtime, or simply use hybrid for all queries and accept the small penalty on exact-term retrieval.
No reranking stage: Adding a cross-encoder reranker after hybrid retrieval typically improves nDCG further. That interaction is explored separately in Evaluating RAG Quality.
What this means in practice#
Default to hybrid with RRF for mixed-intent corpora. For most knowledge bases, APIs, and documentation sites, RRF is a robust default that degrades gracefully across query types. It requires no score normalization and no labeled training data.
Keep BM25 as a fallback for exact-match query types. If your corpus serves a significant fraction of exact-term queries (serial numbers, API methods, error codes), consider routing those queries to a BM25-only path based on a lightweight query classifier.
Tune alpha only if your query distribution is skewed. If you know 80% of your queries are semantic and 20% are exact, test alpha in the 0.6–0.8 range. If your corpus is product catalog data with heavy exact-match needs, test lower. Without query distribution data, use RRF and skip the calibration burden.
Measure before and after. A hybrid configuration that improves nDCG@5 on your labeled query set by 5–8 points is meaningful. A "hybrid is better" assumption without measurement may not generalize to your specific data shape. For the evaluation framework to do this measurement, see add evaluation to your AI feature.
Next experiment#
The next open question is personalized re-ranking: can a lightweight user-history signal re-order hybrid results to improve MRR without sacrificing nDCG for cold-start users? A second experiment worth running is sparse-dense fusion with learned weights, training a small regression model on a labeled relevance set to replace the fixed alpha or RRF constant. For teams choosing the underlying system, benchmarking vector database retrieval covers how the same hybrid queries perform across different storage backends.
Key Takeaways#
- Hybrid search reliably outperforms either method alone on conceptual and mixed query types.
- On exact-term queries, BM25 is competitive with or better than hybrid. The vector signal can add noise.
- Reciprocal Rank Fusion (RRF) is a robust default: no score normalization, no training data, and competitive performance at default settings.
- Weighted fusion (alpha) requires calibration per corpus and query distribution. Without measurement, the "right" alpha is guesswork.
- The biggest single gain comes from adding any vector signal to BM25 on semantic queries, not from fine-tuning the fusion weight.
- Recall@10 is high across all methods. The meaningful differentiation is in ranking quality (nDCG), not presence.
FAQ#
When should I use BM25 instead of hybrid search?
When your queries are predominantly exact-term: product codes, model numbers, error codes, API method names. BM25's term-frequency scoring is precise for these and hybrid fusion can dilute that precision by pulling in off-topic semantic neighbors.
What is Reciprocal Rank Fusion and why does it not need score normalization?
RRF scores each document as 1 / (k + rank) based only on its position in each result list, then sums those scores across all lists. Because it uses rank position rather than raw scores, it sidesteps the problem of BM25 and vector scores existing on incompatible scales. The original formulation by Cormack et al. (SIGIR 2009) showed it outperforming more complex methods without any training data.
What alpha value should I start with in Weaviate or a similar system?
alpha=0.5 (equal weight) is Weaviate's default and a reasonable starting point. Move toward alpha=0.7–0.75 for corpora with more semantic queries, and toward alpha=0.25–0.35 for corpora heavy on exact-term retrieval. Always validate on a labeled sample of your actual queries.
Does hybrid search help if my embedding model is already very good?
Yes, but less dramatically. A high-quality embedding model narrows the gap between vector-only and hybrid on conceptual queries. But exact-term edge cases remain. Even the best semantic model may not rank an exact API method name above related but wrong methods without the BM25 term-matching signal.
How do I label queries for relevance judgments without a large annotation budget?
Start with 50–100 queries that represent your real traffic distribution. For each query, have one or two domain experts mark which retrieved documents are relevant (binary or graded). This gives you enough to compute nDCG@5 and compare configurations. RAGAS can also generate evaluation sets from your documents if manual annotation is not feasible, though LLM-generated relevance labels have their own reliability limits.
