Inspiration
Have you ever been stuck in a situation where you didn't know what to say next? Have you ever wanted to immerse yourself in a new language, but didn't have the confidence to carry a conversation? Have you ever stood in line at the ice cream shop in Old Quebec City, sweating nervously because you didn't know how to respond when the teenager holding the scoop said "Bonjour ! Aimeriez-vous goûter notre saveur spéciale aujourd'hui ?"?
As someone who has found himself in all three positions before, I wanted to build something that could help me build my oral skills in a new language while enabling me to converse with real people in real time. That's why we made SpeakEasy. Partially inspired by dialogue systems in video games like Fallout, SpeakEasy helps you speak out without speaking over you!
What it does
SpeakEasy is designed to take in speech from someone speaking to you. It turns the speech into text and translates it into the user's native language. Then, it uses AI to generate context-sensitive, grammatically-correct response suggestions that you can then speak to the other party. SpeakEasy shows you what was just said by the other party, what you can say back to them, and what each response suggestion means in your own language.
How we built it
Back-End
The back-end was programmed in Python using FastAPI. We used Azure for services such as the Translator and Speech-to-Text functions, and Google Gemini for the AI that can generate relevant suggestions in real time.
Front-End
The front-end was built using React, Next.js, and TailwindCSS. A proof-of-concept of the UI can be seen in the attached video.
Challenges we ran into
Time Management – This was the first hackathon that three out of four of our members had ever participated in. Most of us didn’t know what to expect, so we underestimated how much time it would take to solve problems.
The Polar Vortex – The weather made things difficult for some of our members. From curfews set by parents to worried mothers, the polar vortex did not make completing this project any easier. Although the weather allowed for an extended submission deadline, the total time we could spend on hardware, coding, and collaborating was reduced. We tried to make up for it by waking up earlier on Sunday morning, but to no avail.
API Integration Between Frontend and Backend – This is ultimately what we were not able to complete. Going forward, we plan to use best practices to avoid conflicts in the frontend and backend integration process.
Accomplishments that we're proud of
The accomplishment we are most proud of is being able to collaborate and communicate together as a team working on a set goal from the get go! We loved being able to give our different insights and skill sets to work together to create something from complete scratch, and you can not forget the joy finally debugging difficult parts of the code!
What we learned
We have actually learned quite a decent bit from HackHive's Hackathon! We had a wide range of backgrounds ranging from different engineering disciplines (Software, Electrical, Mechatronics), and being able to experiment with new tools such as Microsoft Azure had taught us a lot! This was not the end, as for some of us, it was our first time working on GitHub and Visual Studio Code! Overall, HackHive 2026 was quite the learning journey from all of us!
What's next for SpeakEasy
In the future, we would like to expand SpeakEasy by incorporating several additional features:
- Direct conversation with an AI tutor using the same interface as live conversations. This would allow the AI to store all conversations in a database, enabling personalized tutoring that pulls vocabulary and patterns directly from the user’s past conversations.
- Progress-tracking database storage to monitor the user’s development and identify areas for improvement. Users could also set and achieve daily practice goals, helping them clearly see their strengths and weaknesses over time.
Log in or sign up for Devpost to join the conversation.