VocalAI

Inspiration

Companies struggle with live multilingual calls, sentiment detection, and compliance checks. Inspired by this, we wanted to create an AI tool that helps transcribe speech, translate, and get real-time suggestions to the conversation.

What it does

Vocal AI performs the following tasks: Transcribes calls with Whisper Translates to any language with M2M100 Splits speaker turns in a simulated environment. Runs a rules engine for live sentiment, compliance, and escalation alerts Shows everything in a Streamlit interface.

How we built it

VocalAI was built using: FastAPI backend: Whisper( converts speech to text), M2M100(multilingual translation) and LiveCallInsights (custom python rule based insights and AI recommendations) Streamlit frontend: Upload audio, view live insights

Challenges we ran into

The challenges we faced included: Limited languages in the database & restricted suggestions based on dictionary Privacy and compliance

Accomplishments that we're proud of

We were able to build an end-to-end platform that manages call transcribing, multilingual translation while providing a seamless user experience. Furthermore, it does not rely on any paid API's eliminating the development cost and is run locally.

What we learned

We gained understanding on AI capabilities in speech transcribing, translation of models like whisper and M2M100.

What's next for VocalAI

Moving forward, VocalAI aims to expand it's language base for translation, generate entire AI powered insights and recommendations without the usage of custom python rules and deploy in a production environment instead of local deployment.

Built With

• python
• streamlit