Vocalize

Inspiration

Vocalize started from a simple idea: helping people feel more comfortable expressing themselves out loud. We initially explored ways to support people with language and communication but we quickly realized that the challenge goes beyond that.

We landed on this idea of public speaking and the struggles with it. A lot of people, including out classmates, struggle with speaking confidently, whether it’s for presentations, class discussions, or important pitches. It’s not always about knowing what to say, but about how to say it clearly and confidently.

That’s what led us to create Vocalize. We wanted to build a tool that helps people turn their ideas into clear, structured speech and feel more confident when sharing them. Our goal is to make speaking feel less stressful, more effective and accessible for everyone.

What it does

Vocalize is a platform that analyzes recorded speech, including both audio and video, to help users improve their speaking skills. Users can upload recordings and the platform processes the transcript alongside the video to evaluate their delivery.

It provides clear, actionable feedback so users can see what they did well and where they can improve. This includes a full transcript, word count, filler word detection, and insights on what to avoid. It also highlights strengths and areas for improvement based on both voice and video analysis.

In addition, Vocalize generates a “coach summary” and offers audio feedback, so users can hear guidance as if someone is personally coaching them through their performance.

The goal is to make it easier for people to practice speaking and get meaningful feedback in a simple, accessible way.

How we built it

We separated our team into different sections, with one focusing on frontend and design, one on primarily the speech functions (recording and AI coach speaking feedback), and another team working on the video analysis. After making sure each of our parts worked, we came together to integrate it into our final project. We used Vite, React, and Tailwind CSS for our frontend and Python with FastAPI for our backend, implementing 4 different routes:

analyze.py to create feedback for our audio recording based on the transcript
coach.py to score our user based on the feedback
speak.py to implement elevenlabs to have a more vocal approach to feedback
video.py to to upload the video recording to twelvelabs to rate eye contact, posture, etc We also have 2 recorders, one for audio only and one for both audio and video, while also using speech recognition to have a live-transcript in browser.

Challenges we ran into

We had issues with getting our product to work across different platforms, as we have members with different OS (including macOS and Windows 11). We also came across challenges with the versions and different API integrations for our various features, getting all of them to work together and function in our backend.

Accomplishments that we're proud of

We're proud of getting a decently viable project up and running in the limited amount of time we had, surpassing our goals and making the most of our time to deliver a functional project that embodies our original mission.

What we learned

We learned a lot about breaking down our goals into manageable steps, making it easier to divide work, help each other out, and work together as a team. In addition, we are also grateful for the opportunity to improve not only our coding skills, but planning, communicating, and managing limited resources. Many of us had limited experience with some of the coding aspects, such as parts of frontend and backend, and this hackathon has really helped us to learn new things.

What's next for Vocalize

Faster performance and smoother overall experience
Implement script generation and upload functionality
Adding a database to store past sessions and recordings
Expanding features to give more targeted insights on speaking skills like clarity, pacing, and confidence
Implementing a leaderboard to review other top public speakers and learn from them