Our Inspiration:
Many students rarely read their textbooks because they can be very information dense and difficult to understand. We wanted to use AI to assist students to understand and extract specific answers from their textbooks through a user friendly and intuative application. After attending the GenAI workshops we learned how to implement their innovative tools into our vision.
What it does:
StudyPal is an AI powered pdf reader that takes a user input PDF file and processes and saves it into referencable data. StudyPal then prompts the user to ask a question about the content inside the PDF. It then sends the data along with the question to VertexAI to answer and output the specific query answer. StudyPal also outputs the page numbers used to answer the question so the user can easily visit themselves!
How we built it:
We implemented lots of helpful python code from the github repository from one of the GenAI workshop speakers. We started with importing the required libraries for VertexAI, langchain, and embeddings. Then we followed the instructions on Google Cloud for authenticating our accounts. then we spent a good time resolving dependencies before testing it out with our queries. Then we worked on uploading our own PDF files, which each end up having a data base file generated and saved locally, by using the VertexAI libraries. After the first upload these files get searched with RetrievalQA chain which uses load_qa_chain behind the scenes. We finally created a simple interface with Gradio(a python library) after trying for all night to use Flutter.
Challenges we ran into:
Our front end developer ran into several walls when creating a Flutter app and stayed up all night trying to fix them, unfortunately it was too difficult to integrate Flutter with our backend design so we had to scrap Flutter and build a backup UI in our last hours. After Flutter failed, we tried using HTML, CSS, Javascript without success.
Our backend had to work through many python package dependancy errors due to the complicated AI tools we used. For several hours we debugged and fought through the errors and almost lost hope. Eventually we made a breakthrough and were able to run the first working draft of the code. It was our first time connecting our python code to Gradio.
Accomplishments that we're proud of:
We are proud that we have successfully built a working application that accomplishes our core vision.
On top of that, several of our tools were brand new to us as we learned about them through attending workshops.
We are all very proud that this working application was made during our first hackathon.
What we learned:
We learned a lot about the power of Google Cloud and VertexAI and its frameworks like LangChain. The workshops on GenAI were very helpful in how clearly they explained the tech. Some members learned how to use GitHub effectivly in a team environment.
What's next for StudyPal:
StudyPal is now missing several features we had planned. The next steps for our application would include making the following additions: Creating a cleaning and personalized GUI. Adding a username feature to store embedded data based on the specific user. Allowing the user to input photos of their notes so that the AI can reference the students work in addition to the textbook. Storing the Chroma Db (generated sqlite files) in a storage hosting service like MongoDB or Firebase. Displaying the contents of the source pages (when results show) in an optional separate content view.
Log in or sign up for Devpost to join the conversation.