Inspiration
AI has been considered an enemy of education since the advent of public LLM's like Google's Gemini and OpenAI's ChatGPT. This is because students have levered this burgeoning technology in ways that are rather academically dishonest. Therefore, our group sought to find a way of leveraging this new, powerful technology in a way that works to promote education and improve students' learning experiences while also challenging students to actively improve their conceptual understanding of lectures and course materials. We wanted to make a tool that allowed the lecture model of education to be more accessible to more students, by providing simple and easy to understand notes and practice problems that are personalized to the user and their lectures.
What it does
Our project allows users to submit an audio recording of their lectures, resulting in the following: an identification of the overall topic of the lecture, a short list of the key concepts from the lecture, an example problem from lecture (if present) with step-by-step explanation. On top of this, the program also allows for the user to generate practice problems relevant to what was learned in lecture.
How we built it
We built it by using a combination of an HTML/CSS/Javascript front-end and a Python-Flask backend. Essentially, the website front-end works to serve as an easy-to-use GUI for the user. While the Flask back-end handles user file operations, runs the OpenAI Whisper transcription software, creates and retrieves relevant prompts and responses from Google Gemini.
Challenges we ran into
One challenge we ran into was handling the actual submission of a file from the user. In particular, we had a tough time actually getting the file to be submitted via Javascript to our Flask back-end. There was a bit of confusion about how to go about packaging the request on our front-end and receiving it on the back-end. However, after consulting the documentation, we eventually managed to implement a file submission form on our website, a crucial piece of functionality for our site.
Accomplishments that we're proud of
We are proud of creating a website that is both functional and stylish. Really, we're proud to get the core functionality of our project actually working in a reliable manner. In particular, we're proud of getting a summary of the user submit lecture audio and have a dynamically generated quiz function. On top of this, we're also proud of implementing a cohesive style throughout our website, built from scratch and designed solely by us.
What we learned
First and foremost, we learned the importance of having a plan when creating a project from scratch, especially when given a time constraint. In this sense, we saw first hand how useful it was to start out by creating a simple check list of functionality and milestones to achieve for our project. Also, we grew much more familiar with actually implementing a Flask back-end, collaborating in GitHub, linking a domain to a GitHub pages site, using OpenAI Whisper to transcribe audio, and creating custom-formatted responses with Google Gemini.
What's next for Sneky.Tech
We want to gamify the learning process even more and make the entire website more personal: adding a user database, tracking users' progress across different learning areas (classes, subjects, interests), adding a virtual "pet snake" for the user to "level up" to visualize users' personal growth and development in the learning process, and creating a variety of engaging minigames using uploaded lecture audios.
Log in or sign up for Devpost to join the conversation.