🧠 InterViewAR

Empathy-driven VR interview coaching system that helps recruiters detect and respond to candidate stress signals in real time, powered by Groq Whisper + LLaMA, FastAPI, and Meta Quest 3 WebXR.

Inspiration

We’ve all seen it happen: a talented candidate freezes mid-interview, not because they’re unqualified, but because anxiety takes over.
Recruiters often don’t realize the stress levels rising in real time, and candidates leave with their best potential unseen.

InterViewAR was born to fix that.
We wanted to build a tool that empowers recruiters with empathy through data, enabling them to notice stress cues, respond calmly, and create fairer, more human interviews.

What it does

🕶️ InterViewAR is a VR-based interviewer assistant that:

Captures real-time audio from a Meta Quest 3 headset
Streams it to a FastAPI backend
Uses Groq Whisper to transcribe speech instantly
Analyzes voice patterns to compute:
- Pace (WPM)
- Tension level (0–1 scale)
- Filler count (um, uh, like…)
- Pause frequency (> 2 s gaps)
Feeds those metrics to Groq LLaMA, which generates:
- Coaching suggestions (“Nod and reassure”, “Offer a short pause”)
- Follow-up questions (“Can you elaborate on that?”)
Displays everything as a 3D heads-up display (HUD) inside the VR view, updated every 1.5 seconds
Logs the full conversation and metrics in JSON for post-session feedback

In short: it’s an AI empathy lens for interviewers, augmenting emotional awareness without bias.

How we built it

🧩 Architecture Overview

Frontend (WebXR + React.js)
- Renders the 3D HUD inside Meta Quest 3
- Connects to backend through secure WebSockets (/ws/audio, /ws/hud)
Backend (FastAPI)
- Handles audio ingestion, speech-to-text, metrics computation
- Calls Groq Whisper for transcription
- Calls Groq LLaMA for real-time suggestions & next questions
- Maintains thread-safe shared state and pushes updates every 1.5 s
- Logs sessions to JSON + ChromaDB
Networking
- Secure HTTPS + WSS over local LAN (Mac ↔ Quest 3)
- Self-signed SSL certs for in-lab demos
Stack
- FastAPI · Groq API · Three.js · WebXR · Python · JavaScript · ChromaDB

Challenges we ran into

🎧 Audio streaming in VR browsers: Quest 3 WebXR and mic permissions were tricky to align over HTTPS.
⚡ Low-latency processing: achieving < 1.5 s turnaround for transcription + AI inference required optimization and queue management.
🔐 SSL and WebSocket trust: local certificates had to be accepted manually for secure WSS communication.
🧠 Groq LLaMA prompt tuning: balancing technical insights with human coaching tone took experimentation.
🎨 HUD design in 3D space: ensuring readability while keeping immersion was a design challenge.

Accomplishments that we're proud of

Built a fully functional real-time VR pipeline: from mic capture to live HUD updates.
Designed empathy-driven AI coaching integrated into recruiter workflow.
Achieved consistent 1.5 s feedback latency using Groq inference.
Created a replayable session log system with stress-recovery tracking.
Delivered a working demo entirely over LAN on Meta Quest 3.

What we learned

How to synchronize asynchronous FastAPI tasks (audio ↔ AI ↔ broadcast).
The nuances of WebXR development and VR browser permissions.
Designing ethical AI for sensitive contexts like interviews.
Optimizing real-time data visualization inside constrained VR environments.
That small interaction cues, nodding, pausing, offering water, can profoundly change candidate experience.