Voca AI

Talk it through before it counts. A live AI voice interviewer that quizzes you on your own notes or job description, then scores you and breaks down every answer.

Comment

Inspiration

Interview and viva prep is silent. You reread notes, rehearse in your head, and the first time you actually say an answer out loud is when it's being graded. Every prep tool we found tested reading and typing, not the thing you're judged on, which is speaking under pressure.

What it does

Voca AI runs a real spoken interview on your own material.

  • Viva practice: upload course notes, slides, or a syllabus, and the AI quizzes you on it like an oral exam.
  • Mock interview: paste a job description and the AI runs a realistic interview for that role, with live camera and posture guidance.

The AI asks questions grounded in what you uploaded, follows up when an answer is weak, and adapts its tone to the persona you pick: Friendly, Balanced, or Tough.

When the call ends, it scores you 0 to 100 against a rubric, grades each individual answer (Strong / OK / Needs work) with a specific tip, and lets you keep chatting with the AI about how you did. The chat has your full transcript and score as context.

How we built it

The voice loop is orchestrated by a LiveKit Agents worker: Deepgram Nova-3 for speech-to-text, Groq (llama-3.3-70b-versatile) as the interviewer, Deepgram Aura-2 for speech back.

Uploaded documents are chunked and embedded into Pinecone, so every question is grounded in the user's own material rather than generic. For mock interviews, a tiered company-research step (LLM first, Tavily web search as fallback, cached in MongoDB Atlas with a 30-day TTL) weaves company-specific interview culture into the questions.

MediaPipe Face Landmarker handles attention tracking on-device in the browser, with a matching server-side check, and it factors into the final score. FastAPI serves the API; the frontend is Next.js 16 (App Router, React 19) with Tailwind v4. On call end, a shutdown callback scores the transcript against a mode-specific rubric and persists it.

Challenges we ran into

Scoring reliably on disconnect was the hard one. If the call tore down before scoring finished, the entire interview was lost. The user talked for ten minutes and got nothing back. We moved scoring into a shutdown callback so it survives teardown.

The LiveKit native layer could panic while a call ended, killing the agent worker and forcing a full restart between interviews. The dev runner now supervises the agent and auto-respawns it, so you can run one interview after another.

Beyond that: getting the voice loop to feel like a conversation rather than turn-taking, and folding camera attention into the score without it feeling arbitrary, both took real tuning.

Accomplishments that we're proud of

It's a complete loop, not a demo. Upload, live voice interview, instant scoring, per-answer breakdown, and a follow-up chat, all working end to end. The conversation genuinely feels like talking to an interviewer, and the questions are about your material, not a generic question bank.

What we learned

Real-time voice is a latency budget, not a feature list. Every component choice comes down to keeping the response gap short enough that it still feels like a conversation.

We also learned how much of "does this feel good to use" is actually failure handling: what happens when the call drops, when the agent crashes, when the camera is off. The polish that matters most is in the paths nobody demos.

What's next for Voca AI

Right now Voca AI owns one step of the job hunt: the interview itself. The roadmap is to cover the whole pipeline, from the CV that gets you the interview to the agents that find the roles worth applying to.

Resume intelligence

  • CV upload and parsing: upload your resume so interviews are grounded in your actual experience. The AI asks about your projects, your gaps, your career transitions, instead of only what the job description asks for. This slots directly into the RAG pipeline we already have.
  • ATS scoring: score a resume against a specific job description the way an applicant tracking system would. Keyword coverage, skill gaps, section structure, formatting issues that break automated parsers, and a clear score with the reasons behind it.
  • CV optimization: turn that score into edits. Rewritten bullets tailored to the target role, quantified impact suggestions, and a per-application tailored version rather than one generic resume sent everywhere.
  • Resume to interview loop: optimize the CV, then immediately practice the interview it would land you, with the AI probing exactly the claims you just made on paper.

Agentic job pipeline

  • Job discovery agents: agents that continuously scrape and match live job listings against your profile and surface the ones actually worth your time.
  • Auto-tailoring: for each matched role, generate a tailored CV and cover letter, run the ATS check, and flag what needs a human edit before sending.
  • Application tracking: one place tracking what you applied to, where each application stands, and which roles to prep for next.
  • Prep agent: ahead of a real scheduled interview, an agent researches the company and role, then runs a targeted mock interview for that specific conversation.

Platform

  • Hosted deployment so it runs with no local setup
  • Technical and coding interview rounds with live code evaluation
  • Richer cross-session progress tracking, showing which skills improve and which stay flat
  • Multi-language support via Deepgram's multilingual models

Built With

Share this project:

Updates

Submission history