ImPromptU

Inspiration

In the speech and debate world, Impromptu is a competitive speech event that requires speakers to think on their feet and deliver a 5-minute speech according to a given prompt with minimal preparation. They are then graded on a rubric that determines their placing in a round. Such improvisational skills require much practice. Creating a social platform that doubles as a training ground facilitates just that, combining human feedback, AI evaluation, and community. Our ImPromptU is inspired by the need for confident speakers, and more so the environment it takes for them to thrive. But this expands out into so much of our world. Public speaking has always been a valuable skill. It is not just about winning a mere competition, it is about raising a generation of world-changers.

What it does

1. Personal Portfolio

Individual accounts make it possible to store all your practice speeches in one archive page. That way you can track your improvement and keep your camera roll organized.

2. Quick Feedback

ImPromptU gives quick evaluations based on existing speech grading rubrics. Analyzing clarity, structure, pacing, movement, and other important speech components, impromptu AI turns improvement into doable steps. In addition, with each practice speech gives an opportunity to improve the AI-powered speaker score on your profile.

3. Inspire

After each practice video, users have the option to share it to the community, meaning other speakers and coaches are able to view it. We decided to have videos uploaded into a feed that allows users to scroll and like, just as your typical social media apps, designed to boost engagement and incentivize speaking. Get inspiration, inspire, and be competitive.

4. Teacher Training

ImPromptU allows the option to sign up for a Coach account, meaning people with speech experience are able to comment feedback on videos. With AI grading side by side, coaches get practice grading speeches off a rubric.

How we built it

1. The frontend was built using HTML, CSS, and vanilla JavaScript, creating pages for the Home dashboard, Practice Studio, Portfolio, Community, and Progress tracking.
2. Developed a Flask backend in Python to receive uploaded videos, manage processing, run AI analysis modules, and return structured JSON results to the frontend.
3. Used FFmpeg to extract audio from uploaded speech videos and OpenAI Whisper to generate speech transcripts for text-based analysis.
4. Implemented eye contact detection using the MediaPipe Face Landmarker Tasks API, OpenCV, and NumPy to estimate facial landmarks and analyze visual engagement frame by frame.
5. Implemented body language analysis using the MediaPipe Pose Landmarker Tasks API, OpenCV, and NumPy to evaluate posture, body alignment, movement, and gesture quality throughout the speech.
6. Created separate AI modules for Confidence, Voice Clarity, Speaking Pace, and Organization, using transcript analysis and speech features such as words per minute, filler words, etc.

Challenges we ran into

Because of us being relatively new to hackathons in general, we faced numerous technical challenges, even seemingly silly ones that stemmed from a lack of experience. One of the biggest challenges we faced while building ImPromptu was getting all of our technologies to work together. We had trouble connecting the frontend to the backend, so even when the AI generated the correct speech scores, they did not always appear on the website. We also spent a lot of time setting up MediaPipe for body language detection, using Whisper for speech transcription, and fixing problems with video uploads, file paths, and different Python libraries. Since our project used several AI tools together, fixing one problem often created another, so we had to keep testing and improving our code.

Accomplishments that we're proud of

We are proud of creating a project that solves an issue we’ve personally been having as students in speech and debate. Struggling to speak because of confidence or speech impediments, it was and still is a challenge to push ourself and speak in front of judges. Although we came up with this idea for the hackathon, we want to see it implemented in real life and benefit much more speakers such as ourselves, and that in itself is truly fulfilling. Overall we are proud of finishing this project regardless of the obstacles.

What we learned

• Throughout the hackathon, we learned a lot about building a real AI web application. We learned how the frontend and backend communicate, how to integrate AI tools like MediaPipe and Whisper, and how to debug problems when things did not work as expected. We also learned that patience, teamwork, and testing are just as important as writing code. Even when we ran into difficult problems, we kept improving our project, and by the end of the hackathon we had gained valuable experience in both AI and full-stack development.
• We also learned a good amount of video editing to make our pitch high-quality and presentable.

What's next for Impromptu

1. More accommodated speech types.
2. Teacher lecture videos: Allow teacher accounts to upload helpful speech related videos.
3. Expansion into debate: Use ImPromptU in order to help debaters time and practice their debate speeches.

Built With

• css
• gemini
• html
• javascript
• mediapipe
• python
• whisper