ChatPDF

Inspiration

I read a lot of PDFs — research papers, docs, college notes. Ctrl+F wasn't enough, and most PDF-chat apps had a big flaw: naive RAG hallucinates when retrieval is bad.

If vector search returns irrelevant chunks, a normal RAG app still blindly answers from them. I wanted to fix that.

So I built ChatPDF with Corrective RAG (CRAG) — a pipeline that doesn't just retrieve, it questions its own retrieval before answering.

What it does

Live Demo: chatpdf-crag.streamlit.app

Upload one or more PDFs and chat with them:

• Multi-PDF ingestion — upload multiple PDFs, search across all at once
• Corrective RAG pipeline — 6-node LangGraph graph that scores every chunk 0.0 to 1.0
• Smart routing:
  • CORRECT (score >= 0.7) — answer from your documents
  • INCORRECT (score < 0.3) — rewrite query + search web via Tavily
  • AMBIGUOUS (mixed scores) — fallback to web search
• Sentence-level filtering — keeps only relevant sentences before generation
• Grounded answers — falls back to web instead of hallucinating
• Clean Streamlit chat UI — avatars, suggestion chips, custom styling

Try asking something in your PDF, then ask something unrelated — you'll see it route to web instead of making things up.

How we built it

Frontend: frontend.py with Streamlit — chat interface, sidebar uploader, session state.

Backend: backend.py with LangGraph StateGraph (6 nodes):

PDFs -> PyPDFLoader -> RecursiveCharacterTextSplitter (900 / 150)
-> HuggingFace all-MiniLM-L6-v2 -> ChromaDB
Query -> Retrieve (k=5) -> Evaluate Each Doc -> Refine OR Rewrite
-> Web Search -> Generate Answer

Stack:

• Orchestration: LangGraph + LangChain
• Vector DB: ChromaDB (persistent) + HuggingFace all-MiniLM-L6-v2
• Generation LLM: Mistral mistral-small-latest -> Groq gpt-oss-120b fallback
• Scorer LLM: Gemini gemini-flash-lite-latest with Mistral backup
• Web Search: Tavily API
• Deployment: Streamlit Community Cloud with st.secrets

I started with app.py + create_database.py (basic Groq RAG with MMR, no evaluation) as prototypes, then evolved them into the full CRAG app.

Challenges we ran into

• Scoring was flaky. Scorer returned text like fairly relevant instead of a number. Fixed with strict prompts + robust float parsing with 0.5 default.
• Rate limits. Gemini and Mistral threw 429s during demos. Built dual fallback chains: Mistral -> Groq for generation, Gemini -> Mistral Key 2 for scoring.
• ChromaDB on Streamlit Cloud. Persistence kept resetting. Solved with configurable CHROMA_DIR + os.getenv() fallback.
• Noisy chunks. Even good chunks had irrelevant sentences. Added a second sentence-level refine pass.
• Threshold tuning. 0.7 / 0.3 came from testing where naive RAG failed vs where web search was actually needed.

Accomplishments that we're proud of

• Built a real CRAG system, not just another wrapper — most apps are retrieve -> generate, mine evaluates and corrects itself
• Zero-hallucination fallback — admits when docs don't have the answer
• Resilient multi-LLM design — survives outages and rate limits
• Shipped a polished, deployed product with multi-PDF support

What we learned

• Retrieval quality matters more than model size. Small model + good context beats big model + bad context.
• LangGraph taught me to think in state + conditional edges, not linear chains.
• Evaluation is first-class in RAG — scoring chunks improved quality more than any prompt tweak.
• Practical skills: secrets management, vector persistence, lazy-loading APIs for Streamlit speed.

What's next for ChatPDF

• Citations with page numbers — click to jump to source page
• Hybrid search + reranking — BM25 + dense + cross-encoder
• Chat memory — multi-turn follow-ups
• OCR for scanned PDFs
• Eval dashboard — show scores, routing decision, latency per query
• Support for DOCX, TXT, and URLs

Built with: Python, LangChain, LangGraph, ChromaDB, Mistral, Groq, Gemini, Tavily, HuggingFace, Streamlit

Built With

• python