Inspiration We built GPT-OSS-Programming-Tutor to solve a critical problem: providing expert programming help where cloud-based AI can't go—secure facilities, remote areas, and for those who simply value their privacy. The release of OpenAI's GPT-OSS-20B model made this local, private future possible.
How we built it
Core: Python application using the transformers library to load the GPT-OSS-20B model and tokenizer locally.
Architecture: A hybrid Retrieval-Augmented Generation (RAG) system. It first tries to generate a novel answer with the local AI model, then falls back to a curated knowledge base for guaranteed accuracy on common questions.
Access: Built both a command-line interface and a graphical UI to cater to all users.
Challenges we ran into
Hardware Limits: Taming the 20B parameter model to run on a single laptop was the biggest hurdle, solved with quantization and smart memory management.
Accuracy vs. Speed: Balancing the model's creative power with the need for instant, correct answers led to our hybrid design.
Prompt Engineering: Getting the model to consistently output perfect code required meticulous crafting of our instructions and examples.
What we learned- We mastered running massive language models on consumer hardware through 4-bit quantization, designed a hybrid system that combines generative AI with a precise knowledge base for reliability, and discovered that advanced prompt engineering is the key to unlocking a model's potential.
Built With
- python
- transformer
Log in or sign up for Devpost to join the conversation.