NovaSonic AI Interviewer

NovaSonic AI Interviewer is a real-time AI agent powered by Amazon Nova Sonic on Bedrock, running end-to-end on AWS. It provides natural conversations, secure recording, and designed for enterprise.

Comment

Inspiration

We noticed how many candidates feel nervous, unprepared, or overwhelmed during interviews—especially in remote settings. Feedback is often delayed, superficial, or biased. What if there was a way to let people practise live interviews in a safe, repeatable environment, get immediate coaching, and for hiring teams to scale their interview process with consistency and data? That’s where NovaSonic AI Interviewer comes in: combining speech-to-speech, intelligent conversation, and AWS security to create an AI interviewer that helps both candidates and teams.

What it does

  • Hosts mock or real interviews via voice: candidate speaks, the AI interviewer asks questions (based on role, domain, difficulty), listens, responds, and optionally adapts follow-ups.
  • Provides live, natural speech interaction (no typing) with low latency using WebRTC + voice streaming.
  • Records interviews securely, stores audio (and optionally transcripts) for review / coaching.
  • Delivers feedback: immediate insights like pacing, clarity, filler-words, response length, gaps; later analysis or scores.
  • Allows hiring teams to standardize interview format, reuse question sets, compare across candidates, reduce interviewer bias.

How we built it

  • Backend using FastAPI on AWS (ECS Fargate) to orchestrate interviews, manage state, roles, credentials.
  • Speech model via Amazon NovaSonic (Bedrock) for converting text responses to voice, interpreting speech in real time.
  • Transport & streaming done via Daily + WebRTC: audio streaming, room management, secure token-based access.
  • Storage & infrastructure: S3 for recordings, IAM roles for access control; CloudFront and AWS CDK for deploying frontend; VPC setup for secure network; health/status endpoints.
  • Frontend: simple UI to join interviews, toggle audio/video, view transcripts, playback recordings.

Challenges we ran into

  • Latency vs Quality: balancing fast feedback vs clarity of speech (echoes, noisy inputs, variable bandwidth).
  • Context and follow-ups: deciding when the AI should branch or probe deeper vs sticking to a script, especially with free-form answers.
  • Feedback generation: what metrics to compute (e.g. pace, clarity, filler words) that are useful, fair, and interpretable.
  • User privacy & data retention: ensuring recordings are stored securely, access controlled, retention configurable, complying with GDPR / data protection best practices.
  • Voice/speech diversity: handling different accents, speech styles, background noise; ensuring the AI interviewer is fair and inclusive.

Accomplishments that we're proud of

  • Built an end-to-end working pipeline that supports real-time speech conversation with AI interviewer + voice responses.
  • Successfully recorded interviews and made playback possible, enabling review and coaching.
  • Modular architecture: easy to plug in new question sets, feedback modules, or adjust interview styles.
  • Security & infrastructure: deployed on AWS with proper IAM, roles, VPC, so it’s more than just a prototype but production-minded.

What we learned

  • Real users care not just about correctness but empathy: aspects like tone, encouragement, silence handling matter.
  • Even small delays or audio artifacts break immersion and make the candidate self-conscious.
  • There’s no one-size-fits-all in feedback: some prefer quantitative metrics, others qualitative tips. Customization is key.
  • Infrastructure and deployment concerns (security, scalability, cost) are just as important as model accuracy when building something used by real people.

What’s next for NovaSonic AI Interviewer

  • Improve feedback: more advanced metrics (sentiment, stress detection, content quality), dashboards for candidates and hiring managers.
  • Interviewer personality/styles: adapt tone (friendly, formal, technical), pacing, behavior depending on role.
  • Support multilingual interviews: candidate can speak in different languages, interviewer responds appropriately.
  • Bias detection & fairness: monitor for and mitigate any bias in follow-up questions or feedback based on accent, speech style, etc.
  • Mobile support & UX polish: better audio UIs in poor bandwidth, fallback options, more intuitive UX.
  • Experiment with real hiring process pilots: test with organizations, get live interviews, collect feedback data, improve.
Share this project:

Updates