ai

AI Personal Knowledge Assistant — Project Retrospective

A lightweight RAG project that makes AI truly "understand you"

Inspiration 💡

The inspiration came from a very ordinary daily scenario:

I have piles of notes, papers, and course materials. Every time I want to find a specific piece of knowledge, I have to dig through folders for ages. General-purpose AI knows everything, but it doesn't know "my stuff."

One day I thought:

What if AI could answer my questions by actually reading my notes?

That thought sparked the project: A personal knowledge base AI assistant based on RAG (Retrieval-Augmented Generation) — enabling large language models to generate answers based on the user's own materials, rather than "making things up out of thin air."

What it does 🧠

This is an AI assistant that "reads your materials and answers your questions."

Core capabilities:

Feature Description

📥 Knowledge Injection Users upload Markdown / PDF / TXT notes 🔍 Semantic Retrieval Vector model finds the most relevant snippets 💬 Intelligent Q&A Generates answers based on retrieved content + LLM 📎 Source Citation Every answer cites which document it came from ❌ Hallucination Rejection Clearly says "I don't know" if the answer isn't in the materials

In one sentence:

Answer = LLM(Query Retrieve(D))

How we built it 🛠️

Tech Stack

Language: Python Vector Search: FAISS / Chroma Embedding Model: all-MiniLM-L6-v2 (Sentence Transformers) LLM: OpenAI GPT-4o-mini / Local Llama 3 Frontend: Streamlit (rapid prototyping) Data Processing: LangChain text splitters

Architecture Flow

Key Code Snippet

Challenges we ran into 🚧

Hallucination Problem Symptom: The model confidently fabricated non-existent content Cause: No constraints on "unknown" behavior Solution: Strongly constrained prompt + source citation mechanism

Hallucination Rate 1/Constraint Strength

Retrieval Noise Symptom: Retrieved snippets "looked relevant but weren't actually useful" Cause: Title similarity ≠ semantic relevance Solution: Adjusted chunk size + introduced reranking strategy

Cost vs. Speed Trade-off Local model (Llama 3 8B): slow inference, high VRAM usage Cloud model (GPT-4o-mini): fast but billed per token Compromise: Local retrieval, small questions answered locally, complex questions go to the cloud

Context Window Limitations Models have maximum token limits Can't cram entire notebooks into one prompt Solution: Only retrieve the most relevant Top-K snippets

Accomplishments that we're proud of 🏆

Zero-Hallucination Mode: With strong constraints + citation mechanism, the model no longer fabricates content End-to-End Usability: From uploading materials to Q&A, the entire pipeline works seamlessly Lightweight & Deployable: Runs on a single machine, no GPU required (pure cloud inference) Strong Explainability: Every answer comes with a source citation — users can verify Fast Iteration: From idea to MVP in just 3 days

What we learned 📚

Technical Takeaways

RAG > Fine-tuning: For scenarios with frequently updated knowledge, RAG is more practical and cost-effective than full fine-tuning Data quality determines the ceiling: No model can save bad data Prompt engineering is interface design: It's not "talking" — it's defining behavioral boundaries Evaluation first: Without an evaluation set, you can't tell if you're improving

Mindset Shift

AI Capability = Data Quality × System Design × Human Judgment

Don't worship the model — trust the system Get it working first, optimize later User trust comes from explainability, not from "looking smart"

What's next for AI 🚀

Short-term Plans

[ ] Multi-modal support: Images, scanned PDFs [ ] Conversation memory: Multi-turn context [ ] Knowledge graph: Upgrade from vector search to graph-based reasoning [ ] Web deployment: Make it accessible to more users

Long-term Vision

Personal AI → Personal Knowledge Engine

Evolve from a "Q&A tool" into a "personal knowledge engine": Auto-organize notes Discover knowledge blind spots Proactively recommend learning paths

Project Status: MVP complete, continuously iterating One-line summary: Making AI not "know everything," but "know what you know."

If you need, I can also adapt this into: 🎤 Pitch / Demo Day version 📄 Technical report / Paper version 🏅 Hackathon submission version 📱 LinkedIn / Medium blog version

Just tell me your audience!

Built With

• ai-assistant
• artificial
• chroma
• embeddings
• faiss
• hackathon
• knowledge-base
• langchain
• llama
• llm
• machine-learning
• mvp
• natural-language-processing
• open-source
• openai
• personal-notes
• prompt-engineering
• python
• q&a-system
• rag
• retrieval-augmented-generation
• sentence-transformers
• streamlit
• vector-database