Livestream transcription

Stream videos @ theta network and view live transcription alongside

Comment

Inspiration

  • The potential of real-time transcription to enhance user engagement and accessibility.
  • The need for accessible transcription for live video content (for non-native speakers and the hearing impaired.

What it does

  • Provides real-time transcription of live video streams on Thetacloud.
  • Consists of a server for stream ingestion, transcription, and data storage.
  • Includes a client application to display the live video stream alongside its transcription.
  • Refetch the transcription done by plugging in m3u8 url across multiple sessions.

How we built it

  • Server: Built with FastAPI, handles stream ingestion, transcription, and data storage.
  • Client: Developed using Svelte, displays the live video stream and its transcription.
  • Transcription: Utilizes HLS (HTTP Live Streaming) for real-time video and transcription display.
  • Storage: Transcription data is stored in a filesystem, organized by unique task IDs.
  • Stream Ingestion: The server ingests live video streams using Thetacloud's streaming APIs.
  • Stream Management: Thetacloud APIs are used to start and manage new streams with account credentials.
  • Video Playback: The client application uses Thetacloud's m3u8 URLs to play live video streams.
  • Authentication: Thetacloud account credentials are used to authenticate and start new streams.

Challenges we ran into

  • Ensuring real-time transcription accuracy and speed.
  • How to code this in most efficient manner.
  • Managing multiple concurrent transcription viewers.
  • Handling asynchronous processing of video streams.

Accomplishments that we're proud of

  • Successfully implemented real-time transcription for live video streams.
  • Achieved seamless integration between the server and client applications.
  • Enabled support for multiple languages in transcription.
  • Built the architecture in an easy to extend format

What we learned

  • Deepened our understanding of FastAPI and Svelte.
  • Gained insights into real-time video streaming and transcription.
  • Learned to manage asynchronous tasks and data storage efficiently.

What's next for Livestream transcription

  • Enhance transcription accuracy and speed.
  • DEPLOY it ON GPU.
  • Add translation feature.
  • Improve the user interface for better accessibility and user experience -- put the stream initializer in the same UI as the live transcript player.
  • Add autoscaling

Built With

Share this project:

Updates

posted an update —

Few things to add:

  • Demo URL is there on the site and can be tested out. I've kept the timeout as 10minutes to prevent high server load or SPAM.
  • The inference speed/live transcription speed is dependent on the VM's memory/cpu which is 4cpu/4gb ram right now (cheap VM)

Log in or sign up for Devpost to join the conversation.

Submission history