Production RAG: Hybrid Search, RRF & Rerankers

1. Why Naive RAG Fails in Production

Consider what happens in a standard tutorial RAG pipeline when an executive asks: "What were the exact Q3 net margins for Project Apollo in contract Annexure 4B?"

• Keyword Blindness of Dense Embeddings: A dense bi-encoder (e.g. text-embedding-3-small or all-MiniLM-L6-v2) maps tokens into high-dimensional latent space based on semantic concepts. However, unique strings like "Annexure 4B" or part code "APO-991" have almost no conceptual semantic weight. The vector search retrieves generic margin clauses from other projects, missing the exact contract.
• Context Fragmentation: Splitting text blindly every 500 characters breaks tabular columns and separates qualifying conditions from their governing clauses.
• The "Lost in the Middle" Effect: Research confirms that when LLMs are fed 10+ retrieved chunks, they attend strongly to the beginning and end of the context window while completely ignoring middle tokens.

2. The 4-Stage Production Retrieval Cascade

High-precision RAG pipelines decouple retrieval into four distinct computational stages:

Production 4-Stage Retrieval Cascade

• STAGE 01 Semantic Chunking Recursive markdown header and boundary splitting
• STAGE 02 Dual-Track Ingest FAISS Inner-Product dense + BM25 Okapi sparse
• STAGE 03 RRF Fusion Reciprocal Rank Fusion merging candidate pools
• STAGE 04 Cross-Encoder bge-reranker scoring top-5 chunks for LLM context

3. Hybrid Search Mechanics & Reciprocal Rank Fusion

Dense embeddings and sparse BM25 scores operate on fundamentally incompatible mathematical scales: cosine similarities range from [0.0, 1.0], while BM25 scores are unbounded positive floats dependent on term frequency.

Arbitrary linear weighted sums (e.g. 0.7 * dense + 0.3 * sparse) break down across queries because one score distribution inevitably dominates. The production solution is Reciprocal Rank Fusion (RRF), which normalizes documents based strictly on their ordinal rank:

RRF_Score(d) = sum_{m in M} ( 1 / ( k + r_m(d) ) )
where:
  M = Set of retrieval systems (Dense Vector Search, Sparse BM25)
  r_m(d) = Rank of document d in system m (1, 2, 3...)
  k = Smoothing constant (empirically set to 60 to prevent top-rank over-weighting)

4. Cross-Encoder Reranking: Bi-Encoders vs Cross-Encoders

Why do we need a reranker if we already have vector embeddings?

Bi-Encoders (standard embedding models) process the query and document independently into fixed-length vectors, computing similarity via dot product. They are fast ($O(1)$ lookup via FAISS), but cannot model complex cross-attention token interactions.

Cross-Encoders feed the query and document together into the transformer simultaneously ([CLS] Query [SEP] Document [SEP]), allowing full bidirectional attention between every query word and every document word. While too slow to run on millions of documents, running a Cross-Encoder over the top-25 RRF candidates delivers state-of-the-art precision in under 60ms.

5. Complete Production Python Implementation

Below is a complete, self-contained implementation combining FAISS, BM25, Reciprocal Rank Fusion, and Cross-Encoder reranking:

import numpy as np
from typing import List, Dict, Any
import faiss
from rank_bm25 import BM25Okapi
from sentence_transformers import SentenceTransformer, CrossEncoder

class ProductionHybridRAG:
    def __init__(self, dense_model: str = "BAAI/bge-small-en-v1.5", reranker_model: str = "BAAI/bge-reranker-base", rrf_k: int = 60):
        self.rrf_k = rrf_k
        self.embedder = SentenceTransformer(dense_model)
        self.reranker = CrossEncoder(reranker_model)
        
        self.dimension = self.embedder.get_sentence_embedding_dimension()
        # Inner Product on L2-normalized vectors == Cosine Similarity
        self.dense_index = faiss.IndexFlatIP(self.dimension)
        self.bm25: BM25Okapi = None
        self.documents: List[Dict[str, Any]] = []

    def index_corpus(self, docs: List[Dict[str, Any]]):
        """Builds dual-track dense and sparse indexes."""
        self.documents = docs
        texts = [doc["content"] for doc in docs]
        
        # 1. Build Dense FAISS Index
        embeddings = self.embedder.encode(texts, normalize_embeddings=True, show_progress_bar=False)
        self.dense_index.add(np.array(embeddings, dtype=np.float32))
        
        # 2. Build Sparse BM25 Token Corpus
        tokenized_corpus = [text.lower().split() for text in texts]
        self.bm25 = BM25Okapi(tokenized_corpus)

    def search(self, query: str, top_k: int = 3, candidate_pool: int = 25) -> List[Dict[str, Any]]:
        # 1. Dense Semantic Retrieval
        q_emb = self.embedder.encode([query], normalize_embeddings=True)
        _, dense_indices = self.dense_index.search(np.array(q_emb, dtype=np.float32), candidate_pool)
        
        # 2. Sparse BM25 Retrieval
        tokenized_query = query.lower().split()
        bm25_scores = self.bm25.get_scores(tokenized_query)
        bm25_indices = np.argsort(bm25_scores)[::-1][:candidate_pool]

        # 3. Reciprocal Rank Fusion
        rrf_table: Dict[int, float] = {}
        for rank, idx in enumerate(dense_indices[0]):
            rrf_table[idx] = rrf_table.get(idx, 0.0) + (1.0 / (self.rrf_k + rank + 1))
            
        for rank, idx in enumerate(bm25_indices):
            rrf_table[idx] = rrf_table.get(idx, 0.0) + (1.0 / (self.rrf_k + rank + 1))

        # Select top candidates for neural cross-encoder pass
        fused_candidates = sorted(rrf_table.items(), key=lambda x: x[1], reverse=True)[:candidate_pool]
        
        # 4. Cross-Encoder Full-Attention Reranking
        pairs = [[query, self.documents[idx]["content"]] for idx, _ in fused_candidates]
        cross_scores = self.reranker.predict(pairs)
        
        # Sort and return top_k verified passages
        best_ranks = np.argsort(cross_scores)[::-1][:top_k]
        final_results = []
        for rank in best_ranks:
            doc_idx = fused_candidates[rank][0]
            doc = self.documents[doc_idx]
            final_results.append({
                "doc_id": doc.get("id", doc_idx),
                "content": doc["content"],
                "cross_score": float(cross_scores[rank]),
                "source": doc.get("source", "unknown")
            })
            
        return final_results

6. Ragas Evaluation Metrics: Quantifying Hallucination

How do you prove that an upgraded RAG pipeline actually works? We use Ragas (Retrieval Augmented Generation Assessment), measuring four orthogonal metrics:

• Context Precision Measures whether all relevant chunks retrieved are ranked at the very top of context.
• Context Recall Measures whether all ground-truth facts required to answer the prompt were retrieved.
• Faithfulness (Anti-Hallucination) Verifies that every claim in the generated answer can be mathematically derived from retrieved passages.
• Answer Relevance Calculates whether the response directly addresses the user's intent without superfluous filler text.

7. Real-World Retrieval Benchmarks

We benchmarked our production hybrid pipeline against pure vector and pure BM25 baselines on a corpus of 10,000 enterprise technical documentation pages:

8. Architectural Lessons & Key Takeaways

Stack

Python 3.11, FAISS, Rank-BM25, SentenceTransformers, Ragas, LangChain, FastAPI

Key Techniques

Reciprocal Rank Fusion (RRF), Cross-Encoder Reranking, Markdown Header Chunking

Repository

github.com/kulkarnishub377/Production-RAG-Framework