Recapify

Home Page
About
Applications
Upload 1
Upload 2
Results Page (Transcript Focus)
Results Page (Summary Focus)

Inspiration

The inspiration for this project came from one of the members of the group being an international student at the University of Southern California. When she was a freshman, she had difficulty adapting to and comprehending the fast-paced classes in English as English was not her first language. And that's how Recapify was born. Recapify is a website that gives the opportunity for students to have a transcript and summary of the lecture, which can then be translated or utilized in a variety of ways.

What it does

Recapify is a website that accepts .wav audio files, transcribes them, and then offers a summary.

How we built it

The backend is in Flask and Python, and the frontend is in CSS and HTML, with the data being processed using Cloudflare Workers AI (whisper and chatgpt).

Challenges we ran into

We spent time trying to use Taipy for frontend but realized it didn't align w/ our goals, we didn't have a need to visualize data so opted for CSS and HTML instead. We started with MP3 audio files but realized that our program only accepts .wav files and had difficulty integrating the two APIs we used (whisper for audio to text, and chatgpt for summarization). Longer audio files would not run so we needed to create a for loop to split audio segments into 10 second increments. The last increment was originally excluded because it was less than 10 seconds, so we had to edit the code to include it. In our final implementation, with experimentation, we found the API could handle up to 30 second increments of audio; we upped the time interval to 30 seconds.

Accomplishments that we're proud of

We came in very calmly and were focused more on attending workshops and having a stress-free environment. We ended up designing a project that was so interesting that we got invested and were able to complete our goal in a timely manner. We were able to reach out to mentors when we ran into issues but also took the initiative to troubleshoot independently and use the Stack Overflow as needed for guidance.

What we learned

For 2 of the 3 members of our group, it was our first hackathon so the flow of the event was all new and we leaned on the support of our more experienced member. We learned about opportunities w/ Bloomberg Business over the summer, how MLH hacking works and opportunities with them shared with us by Jasmine, she has been wonderful during this hackathon. During the workshops we were able to learn basics of Figma design and how to use GitHub autopilot in order to be a helping hand during coding which one of our team members actually ended up integrating into our project to help with the backend. We learned how to use Flask, how to do frontend design, and much more, thanks to the opportunities provided to us during AthenaHacks.

What's next for Recapify

We are considering adding a feature that splits the transcript into increments, identifies which time stamp it is at, and allows users to go back into a specific section to possibly catch a word or phrase that was misunderstood by themselves or the program and to be able to go in and manually edit the transcript in order to alter what summary the program outputs.