Inspiration
I wanted to explore what open-source reasoning models like gpt-oss can do in real-world applications while keeping data completely offline. Many researchers and students need AI-assisted reading and summarization but cannot rely on cloud services due to privacy concerns, so I aimed to build a local tool that makes document understanding fast and accessible.
What it does
Offline Research Assistant lets users upload PDFs and ask questions. The app retrieves relevant information from the document and provides answers along with supporting context, all running locally without internet access.
How we built it
The app splits PDFs into chunks and generates embeddings using a HuggingFace sentence-transformer model. Chunk embeddings are stored in a local Chroma database, enabling efficient retrieval. When a user asks a question, the relevant chunks are passed to gpt-oss-20b via Ollama, which generates an answer grounded in the document. The frontend is built with Streamlit, providing a clean interface for file upload, question input, and displaying answers.
Challenges we ran into
Managing inference speed and memory for gpt-oss locally was a challenge, especially balancing the size of chunks for accurate retrieval without overwhelming the model. With only two days, scoping the project to deliver a working prototype was also difficult.
What's next for Offline Research Assistant
Future improvements include supporting multiple file formats, adding more advanced reasoning features like step-by-step explanations, and optimizing inference speed for larger documents. We also plan to explore fine-tuning the model for domain-specific research tasks.
Log in or sign up for Devpost to join the conversation.