AccentFlow: Real-Time Accent Changer

Inspiration

Communication is key in today's globalized world, yet accents often create unintended barriers. As international students ourselves, we face challenges in both understanding others and making ourselves understood. Whether in corporate meetings, classrooms, or customer service scenarios, miscommunication due to diverse accents is a common issue. Our team realized the need for a tool that could help break these barriers and make conversations clearer and more inclusive. This led to the creation of AccentFlow, designed to transform live speech into easily understandable accents.

What It Does

AccentFlow is a real-time accent conversion application that listens to live speech, transcribes it into text, and converts it into a selected accent using neural text-to-speech models. It offers the following features:

Real-Time Transcription: Converts live audio input into text using Google’s Web Speech API.
Accent Conversion: Uses AWS Polly to convert transcribed text into different accents, allowing users to choose an accent that fits their communication needs.
Intuitive Interface: A user-friendly web-based interface built with Streamlit that offers a seamless experience for starting and stopping the transcription and selecting accents.

Use cases include bridging communication in corporate meetings, enhancing academic discussions between professors and students, and enabling regional accent dubs in the movie industry.

How We Built It

We combined several key technologies to bring AccentFlow to life:

Google Web Speech API for speech recognition and live transcription.
AWS Polly to convert the transcribed text into a specified accent using its neural text-to-speech capabilities.
Streamlit for building an interactive and visually appealing web-based user interface.
PyAudio and SpeechRecognition libraries for managing live audio input and processing.

Technical Flow:

Input: Live speech/audio recording is captured.
Speech Recognition: Converts the captured audio into text using Google Web Speech API.
Text Processing: Processes and refines the transcribed text for conversion.
Text-to-Speech Conversion: Converts the text into the desired accent using AWS Polly.
Output: Plays back the converted speech in the selected accent.

Challenges We Ran Into

Building AccentFlow involved several challenges:

Real-Time Processing: Managing real-time audio capture and processing required careful synchronization of APIs and handling audio data efficiently.
Accent Consistency: Ensuring the converted speech retained natural intonation and pronunciation in different accents was a complex task.
Cross-Platform Compatibility: We needed to ensure that the application ran smoothly across different platforms and systems.

Accomplishments That We're Proud Of

Successfully Implemented Real-Time Accent Conversion: We achieved real-time audio capture, transcription, and conversion, which was our main goal.
User-Friendly Interface: We created an accessible interface that allows non-technical users to interact with the application seamlessly.
Scalable and Expandable Architecture: The modular design of AccentFlow makes it adaptable to future enhancements like multi-accent and multi-language support.

What We Learned

Our team gained valuable experience working with real-time audio data and integrating multiple APIs. We deepened our understanding of speech recognition, text-to-speech conversion, and the challenges associated with managing latency and responsiveness in such applications. Moreover, we learned the importance of designing an intuitive user experience for complex backend processes.

What's Next for AccentFlow: Real-Time Accent Changer

Multi-Accent and Multi-Language Support: We aim to expand AccentFlow to include more regional accents and languages to cater to diverse global users.
AI-Driven Accent Personalization: We plan to develop machine learning models that adapt accents based on individual preferences and speaking styles.
Accent Training: Introducing training modules to help users practice and master different accents.
Cross-Platform Integration: We plan to integrate AccentFlow with popular video conferencing tools like Zoom, Teams, and Meet for real-time accent conversion during live meetings.
Offline Mode: Building an offline mode using pre-trained local models to support low-latency conversion without relying on external APIs.

By evolving AccentFlow with these features, we aim to address a broader range of use cases and continue improving communication experiences for users worldwide.