What is RAG?
Retrieval-Augmented Generation (RAG) is a technique that combines the power of large language models with external knowledge retrieval. Instead of relying solely on the model's training data, RAG fetches relevant documents from your own knowledge base and injects them into the prompt context.
Why RAG Over Fine-Tuning?
Fine-tuning a model is expensive and slow. RAG lets you update your knowledge base in real-time without retraining. It's also more transparent — you can trace exactly which documents influenced the output.
No GPU costs for training
Real-time knowledge updates
Transparent source attribution
Works with any LLM provider
Architecture Overview
Our pipeline has three main stages: document ingestion, vector storage, and query-time retrieval with generation.
Document Ingestion
We'll split documents into chunks using a sliding window approach. Each chunk gets embedded and stored with metadata.
import { OpenAI } from "openai";const openai = new OpenAI();async function embedChunk(text: string): Promise<number[]> { const response = await openai.embeddings.create({ model: "text-embedding-3-small", input: text, }); return response.data[0].embedding;}Vector Storage with pgvector
PostgreSQL with the pgvector extension is an excellent choice for vector storage. It lets you combine traditional SQL queries with vector similarity search.
CREATE EXTENSION IF NOT EXISTS vector;CREATE TABLE documents ( id SERIAL PRIMARY KEY, content TEXT NOT NULL, metadata JSONB, embedding VECTOR(1536));Building the Query Pipeline
At query time, we embed the user's question, find the most similar documents, and pass them as context to the LLM.
async function queryRAG(question: string) { const questionEmbedding = await embedChunk(question); const results = await prisma.$queryRaw`\n SELECT content, 1 - (embedding <=> ${questionEmbedding}::vector) as similarity\n FROM documents\n ORDER BY similarity DESC\n LIMIT 5\n `; const context = results.map(r => r.content).join('\n\n'); const response = await openai.chat.completions.create({\n model: 'gpt-4o',\n messages: [\n { role: 'system', content: 'Answer based on this context:\n' + context },\n { role: 'user', content: question },\n ],\n stream: true,\n });\n return response;}Best Practices
Chunk size of 500-1000 tokens works well for most use cases
Use overlapping chunks (50-100 token overlap) to avoid losing context
Store metadata alongside embeddings
Implement hybrid search: combine vector similarity with keyword matching
Use reranking models to improve retrieval quality
Conclusion
RAG is the most practical way to ground LLM responses in your own data. With pgvector and Next.js, you can build a complete pipeline that scales well and stays maintainable.