Ossy

Ossy is a live transcription tool + AI assistant to help students A) in lectures where instructors write 10% of what is said, B) who are non-native English speakers, and C) with auditory impairments.

Comment

Inspiration

  • Being students are ourselves and experiencing fast-paced lectures where it is difficult to grasp all the information being said, we decided to build a tool to solve this problem.
  • Ossy is inspired by the word "ossicles", three bones in the ears that are the smallest in the human body

What it does

  • Ossy is an educational tool for students that actively transcribes the audio in a room and features an AI bot that can define, explain, or expand on what a lecturer has said

How we built it

  • React API to stream live audio and receive text
  • Hosted Python API that uses ChatGPT API on FastAPI to request explanation of selected text
  • React for frontend

Challenges we ran into

  • learning the IBM Watson AI streaming transcription speech-to-text API. Very new API with convoluted documentation. Even after spending hours understanding the API, the output was so difficult to use in the codebase that we opted for a simpler alternative
  • finding a way to insert punctuation in the transcribed text
  • miscellaneous minor design issues (ex: how do you evenly distribute two divs in a parent div again?)

Accomplishments that we're proud of

  • learning how to use multiple different APIs on the spot
  • learning how to create our own API and deploy it

What we learned

  • constantly reevaluate what the most important thing to be working on is
  • before the hackathon starts, learn how to use the relevant APIs. huge time saver
  • a deeper understanding of various backend concepts
  • sometimes documentation is just bad.

What's next for Ossy

  • Ossy is practical and useful, so we are going to start using it in class and rolling it out to friends

Built With

Share this project:

Updates