Inspiration
I always hated that I'm in between classes and when I get to class I'm behind in some of the classes lectures because I was expected by the professor to have read the chapter. If my commute is an hour long, I could be listening to the lectures or the book while I'm getting to campus and be ready for the lecture for the day if I already read the book. But obviously I can't read the book if I'm driving.
What it does
Allow the user to submit a file and have it read back to the user. It uses the MathPix API in order to translate from the given file into text, and from there it converts the text into a voice over by ElevenLabs/Gemini
How we built it
Using react, Vite, TailWind, MathPix API, AI Studio, Gemini, npm, ElevenLabs API
Challenges we ran into
Confirming the API keys were a challenge and settings up the API key to be accessed because they weren't being accepted as valid keys are some points. Waiting for the conversion between the reading to audio. The larger the file, the longer the time it takes to convert all the information.
Accomplishments that we're proud of
Having a downloadable version that you could place on a phone because it's an mp3 file. As you're walking with headphones it's like you're listening to music but instead you're listening to a podcast.
What we learned
AI has mainstreamed a lot of the coding hiccups that would normally take a lot longer to figure out, especially when it comes to front and backend APIs. With AI Studio especially, if you know how to prompt the code to fix itself, you just need to test and make sure it works and give it more things to improve on rather than learning the material for oneself (obviously you have to understand the material to know where the bugs are and how to fix it)
What's next for Walk-n-Sync
It would be nice to be able to prompt Gemini to prompt the user for things that they have learned and be able to fill in holes where they seem. In addition, more examples can be used with Gemini to be able to test the user's understanding of the summary of what they heard. This would be something similar to a small pop quiz after every podcast.
Log in or sign up for Devpost to join the conversation.