Build a Modular RAG Chatbot

Building production-ready Retrieval-Augmented Generation (RAG) applications requires clean architectural separation between document ingestion, vector storage, API routing, and the user interface. Single-script prototypes quickly become unmaintainable when scaling to larger documents or concurrent users.

This project outlines how to build an end-to-end, decoupled RAG PDF chatbot using LangChain, a FastAPI REST backend, a Streamlit client, ChromaDB for vector storage, and Groq (LLaMA 3) for ultra-fast generation.

System Architecture & Workflow

The system is designed with a strict frontend-backend separation to ensure maintainability and scalability:

[User: PDF Upload] ──► [Streamlit UI] ──► [FastAPI Backend]
                                                │
                                                ▼
                                   [Text Chunking & Embedding]
                                                │
                                                ▼
                                    [ChromaDB Vector Store]
                                                ▲
                                                │ (Top-k Similarity Search)
[User: Query]      ──► [Streamlit UI] ──► [FastAPI /query]
                                                │
                                                ▼
                                    [Context + Prompt Assembly]
                                                │
                                                ▼
                                      [Groq LLaMA 3 Engine]
                                                │
                                                ▼
                                    [Grounded Stream Response]

1. Document Ingestion: The user uploads a PDF via the Streamlit interface, which forwards the payload to the FastAPI document handler.

2. Chunking & Embedding: The backend splits text into manageable chunks with overlap to retain context across boundaries, computes embeddings, and indexes them into ChromaDB.

3. Semantic Retrieval: Natural language queries sent from the client trigger a cosine similarity search against ChromaDB to extract the most relevant document segments.

4. Sub-Second Generation: The retrieved context is formatted into a grounded prompt and sent to Groq's LPU inference engine running LLaMA 3, returning accurate, hallucination-resistant answers back to the UI.

Core Tech Stack

Modular Backend Structure

Rather than packing all logic into an entrypoint file, the backend isolates responsibilities into dedicated handlers:

• llm_handler.py: Manages Groq API client initialization, model configuration, and streaming parameters.

• vectorstore_handler.py: Controls ChromaDB collection creation, persistence, and similarity search queries.

• pdf_handler.py: Handles PDF file reading, validation, and recursive character text splitting.

• query_handler.py: Assembles prompt templates, injects retrieved context, and executes the LangChain RAG pipeline.

Key Takeaway

Decoupling the application logic into specialized FastAPI handlers and connecting it to Streamlit via clean REST endpoints makes it easy to test each component independently using Postman before wiring up the user interface.