SpeakFlow Inspiration
Interviews are stressful because they evaluate much more than technical knowledge. Many people already use AI tools to generate answers, but very few tools help users practice speaking under pressure. We wanted to explore whether voice AI could simulate a realistic interview environment while also analyzing communication quality in real time.
The project was also inspired by the idea that active thinking becomes observable through speech patterns. During interviews, hesitation, pacing, filler words, and structure often reveal confidence, preparation, and reasoning clarity. We wanted to build a lightweight system that could provide immediate feedback on those behaviors instead of only evaluating the final answer itself.
What It Does
SpeakFlow is an AI-powered voice interview coach that simulates live interview conversations.
The system:
asks interview questions using conversational AI, listens to the user’s spoken response, analyzes communication patterns, generates spoken follow-up questions, and produces a final evaluation report.
The feedback includes:
clarity, confidence, pacing, filler word density, STAR-method structure, and response specificity. How We Built It
We designed SpeakFlow as a lightweight voice AI pipeline optimized for hackathon speed and low latency.
Frontend
We used:
Next.js TailwindCSS OpenDesign-generated UI components
The frontend manages:
microphone input, transcript display, conversation history, and feedback visualization. Speech Recognition
Instead of deploying a large speech recognition backend, we used the browser’s built-in Web Speech API:
User Voice → Browser SpeechRecognition → Transcript
This significantly reduced infrastructure complexity while keeping latency low enough for live conversation.
Conversational AI
For interview generation and conversational reasoning, we used:
gemini-2.5-flash
The model handled:
interview question generation, follow-up questioning, conversational context management, and final transcript evaluation. Voice Synthesis
For voice output, we used:
gemini-2.5-flash-preview-tts
This allowed the AI interviewer to speak naturally and maintain conversational flow during the interview session.
System Architecture User Voice ↓ Browser SpeechRecognition API ↓ Transcript ↓ Gemini 2.5 Flash (Interviewer + Reasoning) ↓ Gemini 2.5 Flash TTS ↓ Voice Playback ↓ Conversation Loop ↓ Final Transcript Evaluation ↓ Feedback Dashboard Challenges We Faced
- Voice Latency
The biggest challenge was maintaining smooth conversational timing. Even small delays between transcription, reasoning, and TTS generation made the interaction feel unnatural.
We learned that voice AI systems are extremely sensitive to latency. The difference between a 1-second and 3-second response dramatically changes how “human” the conversation feels.
- Browser Audio Constraints
Modern browsers heavily restrict autoplay and microphone permissions. We had to carefully manage:
user interaction requirements, audio playback states, and microphone lifecycle management.
A surprising amount of engineering effort went into handling browser audio behavior rather than the AI itself.
- Prompt Design
We discovered that interview simulation requires very constrained prompting. Without careful instructions, the model:
gave overly long responses, became too friendly, or provided feedback too early.
We refined prompts to make the interviewer:
concise, realistic, and slightly challenging.
- Measuring Communication Quality
Evaluating spoken communication is inherently fuzzy. Unlike coding problems, there is no single correct answer.
We experimented with lightweight heuristics such as:
Filler Density= Total Words Filler Words
and combined them with LLM-based qualitative evaluation to assess:
clarity, structure, and confidence.
This taught us how difficult it is to quantify human communication behavior reliably.
What We Learned
This project taught us that building voice AI products is not primarily about model intelligence. The hardest problems are:
latency, interaction design, conversational pacing, and creating natural user experiences.
We also learned how powerful lightweight architectures can be. By combining browser-native speech recognition with fast LLM inference, we were able to create a highly interactive AI experience without heavy infrastructure.
Most importantly, we learned that communication itself can become observable data. Speech patterns, pauses, and structure reveal much more about reasoning and confidence than text alone.
Future Improvements
Future versions of SpeakFlow could include:
real-time interruption handling, emotional tone analysis, personalized coaching history, adaptive interview difficulty, and multimodal feedback using facial expressions and eye contact analysis.
We are especially interested in exploring how voice interaction data can help measure active thinking and metacognitive behavior during high-pressure conversations.
Log in or sign up for Devpost to join the conversation.