-
-
A screenshot of a answer given by the assistant.
Inspiration
I wanted to build a useful AI application rather than another simple chatbot. As a student, I thought it would be interesting to create a History assistant that could answer questions from a Class IX History textbook while reducing the chances of an LLM giving answers unrelated to the study material.
This led me to build a Retrieval-Augmented Generation (RAG) chatbot that retrieves relevant passages from the textbook and uses them as context for an instruction-tuned language model.
What I Learned
Through this project, I learned how the different components of a RAG system work together. I implemented semantic search using Sentence Transformers and FAISS, and then added a Cross-Encoder to rerank the retrieved passages before sending the most relevant context to the language model.
I also learned how to work with Qwen2.5-0.5B-Instruct, design prompts for context-grounded generation, use Hugging Face Transformers, and build a user interface with Streamlit.
One of the most useful lessons was understanding that a good RAG system depends not only on the language model, but also on the quality of retrieval and the context provided to it.
How I Built It
The application follows a retrieval-augmented generation pipeline:
- The student's question is converted into an embedding using
all-MiniLM-L6-v2. - FAISS performs semantic similarity search and retrieves the top 5 candidate passages.
- A Cross-Encoder reranks those passages according to their relevance to the question.
- The top 2 passages are provided as context to Qwen2.5-0.5B-Instruct.
- A carefully designed system prompt instructs the model to answer using only the supplied context.
- The generated answer is displayed through a Streamlit interface.
I also used Streamlit's resource caching to avoid unnecessarily reloading the models during normal application use.
Challenges
One of the biggest challenges was getting the language model to follow the retrieved context instead of relying on its pretrained knowledge. I experimented with different prompts and retrieval settings to improve grounding.
I also compared retrieval configurations and found that using a larger candidate pool followed by Cross-Encoder reranking produced better results than relying only on the initial FAISS results.
Another challenge was deploying a relatively large model stack in a resource-constrained environment. This required paying attention to model size, memory usage, and efficient model loading.
Outcome
The final result is a lightweight History RAG assistant designed specifically for Class IX students. It combines semantic retrieval, reranking, and an instruction-tuned language model into a single interactive application.
