RAG Chatbot System with Langchain

Inspiration

What it does

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for RAG Chatbot System with Langchain

Inspiration Useful knowledge often sits inside long documents that are difficult to search and understand. We wanted to make that information easier to access through natural conversation. That inspired us to build a document-based chatbot while exploring the engineering behind an AI application: ingestion, retrieval, streaming, caching, monitoring, and guardrails. What it does RAG Chatbot System with LangChain lets users ask questions and receive AI-generated answers informed by a document collection. The system processes documents into searchable embeddings, retrieves relevant content, and gives a language model context for answering questions. Our included dataset focuses on battery recycling and environmental information. Key capabilities include:

• Document ingestion: A pipeline for loading, splitting, and embedding documents.
• Conversational answers: Retrieval tools and conversation history support contextual responses.
• Streaming responses: Users can receive answers incrementally.
• Caching: Redis-based caching supports reuse of previous results.
• Guardrails: Input and output checks help filter disallowed content.
• Observability: Langfuse traces help inspect model interactions and troubleshoot behavior. How we built it We built the application in Python, using LangChain to connect language models, retrieval tools, and generation workflows. Apache Airflow orchestrates document ingestion. Documents are split into chunks, staged in MinIO, converted into embeddings using Sentence Transformers, and stored in ChromaDB. FastAPI exposes standard and streaming endpoints. Redis supports caching, NVIDIA NeMo Guardrails provides configurable checks, and Langfuse records traces. Infrastructure services use Docker Compose. The code separates API routes, application logic, generation services, and infrastructure integrations so each component can evolve independently. Challenges we ran into The biggest engineering challenge was connecting multiple services into a coherent workflow. Ingestion and retrieval need consistent collection names, storage paths, and embedding configuration. Streaming adds another layer of complexity: response delivery, guardrail checks, caching, and error handling must work together while an answer is being generated. Conversation-aware caching also requires care. Similar questions can need different answers depending on the user's history, making session isolation an important area for further improvement. Accomplishments that we're proud of We brought the main parts of a RAG application into one modular codebase, from document ingestion to conversational response delivery. We are particularly proud of:
• Implementing both standard and streaming response paths.
• Connecting retrieval with tool-based language model workflows.
• Integrating tracing and guardrails alongside answer generation.
• Creating a foundation for experimenting with different document collections and model providers. What we learned Building a useful AI chatbot requires more than connecting an API to a language model. Document preparation, retrieval quality, conversation context, and operational reliability all influence the experience. We also learned that adding infrastructure introduces tradeoffs. Caching needs context-aware boundaries, guardrails need validation, and monitoring needs to reveal failures clearly. These details are essential to turning a prototype into a dependable application. What's next for RAG Chatbot System with LangChain Our next priority is strengthening reliability and evaluating answer quality. Planned improvements include:
• Fixing session isolation and making caching aware of conversation context.
• Adding source citations so users can verify answers.
• Expanding tests for retrieval, streaming, and guardrail failures.
• Improving dependency health checks and timeout handling.
• Adding authentication and rate limiting to the core API.
• Creating a simpler document-upload experience.
• Measuring retrieval relevance and how closely answers follow source documents.

Built With

• apache-airflow
• chromadb
• docker
• fastapi
• hugging-face
• langchain
• langfuse
• minio
• nvidia-nemo-guardrails
• python
• redis
• sentence-transformers