RAG Has Evolved
Basic RAG — embed documents, search, inject top-k into prompt — was a breakthrough in 2023. But production systems in 2025 need much more sophisticated retrieval strategies. The gap between naive RAG and advanced RAG is enormous.
Advanced RAG addresses the fundamental weakness of basic retrieval: the quality of the generated answer is only as good as the quality of the retrieved context.
Pre-Retrieval: Query Enhancement
The user's query is often a poor search target. Pre-retrieval techniques transform it into something that retrieves better results.
Query Rewriting
async def rewrite_query(original_query: str) -> list[str]: prompt = f"""Rewrite this query in 3 different ways to improve document retrieval. Each rewrite should approach the topic from a different angle. Original: {original_query} Provide 3 rewrites, one per line:""" rewrites = await llm.generate(prompt) return [r.strip() for r in rewrites.split("\n") if r.strip()]# Example:# Original: "Why is my API slow?"# Rewrites: ["API performance bottlenecks causes", # "latency optimization techniques for REST APIs",# "troubleshooting slow API response times"]Hyde (Hypothetical Document Embeddings)
Generate a hypothetical answer to the query, then embed that answer for retrieval. The embedded answer is semantically closer to the real documents than the raw query.
async def hyde_retrieve(query: str, vector_store, k: int = 5): # Generate hypothetical answer hypothetical = await llm.generate( f"Write a detailed answer to: {query}" ) # Embed and search with hypothetical answer results = await vector_store.search( embedding=embed(hypothetical), k=k ) return resultsRetrieval: Hybrid and Multi-Stage
Hybrid Search
Combine semantic (vector) search with keyword (BM25) search. Each approach catches different relevant documents.
from rank_bm25 import BM25Okapiimport numpy as npclass HybridRetriever: def __init__(self, vector_store, documents): self.vector_store = vector_store self.bm25 = BM25Okapi([doc.tokenize() for doc in documents]) self.documents = documents async def search(self, query: str, k: int = 10): # Semantic search semantic_results = await self.vector_store.search( embed(query), k=k * 2 ) # Keyword search keyword_scores = self.bm25.get_scores(query.split()) keyword_indices = np.argsort(keyword_scores)[-k * 2:] keyword_results = [self.documents[i] for i in keyword_indices] # Reciprocal Rank Fusion return self.rrf_fusion(semantic_results, keyword_results, k=k) def rrf_fusion(self, *result_lists, k: int = 10, decay: float = 60): scores = {} for results in result_lists: for rank, doc in enumerate(results): doc_id = doc.id if doc_id not in scores: scores[doc_id] = 0 scores[doc_id] += 1 / (decay + rank) ranked = sorted(scores.items(), key=lambda x: x[1], reverse=True) return [doc_id for doc_id, _ in ranked[:k]]Cross-Encoder Reranking
After initial retrieval, use a cross-encoder to rerank results. Cross-encoders read the query and document together, providing much more accurate relevance scores than embedding similarity alone.
from sentence_transformers import CrossEncoderreranker = CrossEncoder("cross-encoder/ms-marco-MiniLM-L-6-v2")async def rerank(query: str, documents: list, top_k: int = 5): pairs = [(query, doc.text) for doc in documents] scores = reranker.predict(pairs) ranked = sorted( zip(documents, scores), key=lambda x: x[1], reverse=True ) return [doc for doc, score in ranked[:top_k]]Post-Retrieval: Context Processing
Context Compression
Don't dump entire documents into the prompt. Extract only the relevant passages to reduce token usage and improve answer quality.
Deduplication and Ordering
Remove duplicate information across retrieved chunks and order them by relevance.
Agentic RAG
Instead of a fixed retrieval pipeline, use an agent that decides what to retrieve, when to retrieve more, and how to combine results.
class AgenticRAG: def __init__(self, retriever, llm, tools): self.retriever = retriever self.llm = llm self.tools = tools async def answer(self, question: str): # Agent decides initial retrieval strategy plan = await self.llm.plan_retrieval(question) # Retrieve from appropriate sources context = [] for source in plan.sources: results = await self.retriever.search( question, source=source, k=plan.k ) context.extend(results) # Agent evaluates if more retrieval is needed needs_more = await self.llm.evaluate_context(question, context) if needs_more: # Agent reformulates and retrieves again new_query = await self.llm.reformulate(question, context) additional = await self.retriever.search(new_query, k=5) context.extend(additional) # Generate answer return await self.llm.generate(question, context)Multimodal RAG
Advanced RAG systems index and retrieve across text, images, tables, and code.
OCR and vision models extract text and describe images from PDFs
Table extraction pipelines parse structured data
Code-specific embeddings for technical documentation
Cross-modal retrieval: find images based on text queries
Evaluation
You can't improve what you don't measure. RAG evaluation requires measuring both retrieval quality and generation quality.
Retrieval metrics: precision@k, recall@k, MRR, NDCG
Generation metrics: faithfulness, answer relevance, context relevance
RAGAS framework: automated evaluation of RAG pipelines
Human evaluation: the gold standard for quality assessment
Production Architecture
Vector DB: Pinecone, Weaviate, Qdrant, or pgvector
Embedding model: text-embedding-3-large or open-source alternatives
Reranker: Cohere Rerank, cross-encoder models
Orchestration: LangChain, LlamaIndex, or custom
Monitoring: track retrieval quality and answer accuracy over time
Conclusion
Advanced RAG is about treating retrieval as a first-class problem, not a simple cosine similarity lookup. Invest in query enhancement, hybrid search, and reranking. These techniques can improve answer quality by 30-50% over naive RAG.