DocuAI Studio v3.1: Production Document AI & Hybrid RAG

1. The Production RAG Gap: Why Naive Pipelines Fail

Before examining the code, it is critical to analyze why conventional single-vector retrieval fails on real-world enterprise corpora:

• The Semantic Blind Spot on Specific Tokens: A dense bi-encoder like all-MiniLM-L6-v2 maps semantic concepts effectively (e.g. understanding that "revenue decline" relates to "loss in earnings"), but completely fails on precise lexical identifiers like invoice code INV-2024-998A or part number X49-B.
• Loss of Tabular Geometries: Extracting text from complex multi-page financial balance sheets using standard text dumpers strips row-column spatial relationships, producing nonsensical token streams that hallucinate numerical correlations.
• Context Window Contamination: Passing 10 unranked chunks to an LLM dilutes attention. The "lost-in-the-middle" effect causes models to overlook critical clauses tucked into middle chunks.

2. Layered System Architecture

DocuAI Studio v3.1 separates concern into five strictly decoupled layers: Ingestion & Parsing, Vector & Lexical Indexing, Hybrid Retrieval Cascade, Generation & Guardrails, and WebSocket Streaming Delivery:

DocuAI Studio Ingestion to Generation Flow

• STAGE 01 Multi-Modal Parse PDF, OCR, XLSX, DOCX, Web scraper
• STAGE 02 Semantic Chunker Header-aware recursive splitting with overlap
• STAGE 03 Dual Indexing FAISS FlatL2 (384D) + BM25 Okapi corpus
• STAGE 04 RRF Fusion Reciprocal Rank Fusion k=60 merging
• STAGE 05 Cross-Encoder bge-reranker-base scoring top-5 chunks
• STAGE 06 Stream Response SSE / WebSocket with inline citation spans

3. Hybrid RRF & Cross-Encoder Implementation

The bedrock of DocuAI's precision is the hybrid retrieval coordinator. Dense search handles semantic paraphrasing, BM25 catches exact keyword matches, and Reciprocal Rank Fusion normalizes their score distributions without requiring arbitrary multiplier tuning:

import numpy as np
from typing import List, Dict, Any
from rank_bm25 import BM25Okapi
import faiss
from sentence_transformers import CrossEncoder

class ProductionHybridRetriever:
    def __init__(self, embed_dim: int = 384, rrf_k: int = 60):
        self.embed_dim = embed_dim
        self.rrf_k = rrf_k
        self.dense_index = faiss.IndexFlatIP(embed_dim)  # Inner Product for Cosine Similarity
        self.bm25: BM25Okapi = None
        self.corpus: List[Dict[str, Any]] = []
        # Local Cross-Encoder for high-precision final re-scoring
        self.reranker = CrossEncoder("BAAI/bge-reranker-base")

    def build_indexes(self, chunks: List[Dict[str, Any]], embeddings: np.ndarray):
        """Constructs both FAISS dense index and BM25 token corpus."""
        self.corpus = chunks
        faiss.normalize_L2(embeddings)
        self.dense_index.add(embeddings)
        
        tokenized_corpus = [chunk["text"].lower().split() for chunk in chunks]
        self.bm25 = BM25Okapi(tokenized_corpus)

    def retrieve(self, query: str, query_emb: np.ndarray, top_k: int = 5, candidate_pool: int = 25) -> List[Dict[str, Any]]:
        # 1. Dense Semantic Retrieval
        q_norm = query_emb.reshape(1, -1).copy()
        faiss.normalize_L2(q_norm)
        _, dense_indices = self.dense_index.search(q_norm, candidate_pool)
        
        # 2. Sparse Lexical BM25 Retrieval
        tokenized_query = query.lower().split()
        bm25_scores = self.bm25.get_scores(tokenized_query)
        bm25_indices = np.argsort(bm25_scores)[::-1][:candidate_pool]

        # 3. Reciprocal Rank Fusion (RRF)
        rrf_scores: Dict[int, float] = {}
        for rank, idx in enumerate(dense_indices[0]):
            rrf_scores[idx] = rrf_scores.get(idx, 0.0) + (1.0 / (self.rrf_k + rank + 1))
            
        for rank, idx in enumerate(bm25_indices):
            rrf_scores[idx] = rrf_scores.get(idx, 0.0) + (1.0 / (self.rrf_k + rank + 1))

        # Sort candidate pool by fused RRF rank
        fused_sorted = sorted(rrf_scores.items(), key=lambda x: x[1], reverse=True)[:candidate_pool]
        
        # 4. Cross-Encoder Deep Reranking
        pairs = [[query, self.corpus[idx]["text"]] for idx, _ in fused_sorted]
        rerank_scores = self.reranker.predict(pairs)
        
        # Assemble final results with provenance
        results = []
        ranked_order = np.argsort(rerank_scores)[::-1][:top_k]
        for r_idx in ranked_order:
            original_corpus_idx = fused_sorted[r_idx][0]
            item = self.corpus[original_corpus_idx]
            results.append({
                "chunk_id": item["id"],
                "doc_name": item["source"],
                "page_number": item["page"],
                "content": item["text"],
                "cross_encoder_score": float(rerank_scores[r_idx]),
                "rrf_score": float(fused_sorted[r_idx][1])
            })
            
        return results

4. User Interface Architecture

DocuAI Studio's frontend is designed for operational workflows rather than simplistic chat bubbles:

5. Production API Surface

DocuAI exposes 40+ REST and WebSocket endpoints organized across 6 domain routers in FastAPI:

6. Retrieval & Latency Benchmarks

We benchmarked DocuAI Studio v3.1 across an enterprise financial dataset (1,200 annual reports, 85,000 text chunks) comparing pure dense retrieval, pure BM25, and our two-stage hybrid cascade:

7. Key Engineering Challenges Solved

8. Production Takeaways & Code Access

DocuAI Studio v3.1 demonstrates that RAG system quality is won or lost in the retrieval pipeline before the LLM generates a single token. By replacing naive vector search with a principled, layered hybrid cascade and treating documents as structured data objects, teams can deploy trustworthy Document AI platforms that deliver transparent, verifiable business outcomes.

Stack

FastAPI, LangChain, FAISS, BM25, SentenceTransformers, Ollama, Docker, HTML5/CSS3

Key Techniques

Reciprocal Rank Fusion (RRF), Cross-Encoder Reranking, Markdown Table Extraction

Source Code

github.com/kulkarnishub377/Document-AI---RAG-Pipeline