Presence AI

Inspiration

Mock Interviews are either awkward text-based chatbots or video recorders that feel like talking into a void. What interviewees fear isn't the questions but presence.

Most platforms tell you if your answers was right, but they don't tell you if you looked confident, maintained eye contact, or sounded monotone. We want to build a simulator for interviews, high stakes, voice first environment that tracks not just what you say but how you show up in the room.

What it does

Presence AI is an adaptive multimodal interview AI. Presence AI engages in a live voice driven loop. Utilizing adaptive conversations and specifically tailoring it towards your resume, job description, linkedIn, and Github Repo, it digs deeper when your answer lack detail. While you talk, we analyze audio and facial features to measure eye contact, confidence and vocal volumes. We have both behavioral and technical including STAR methods and Leetcode questions. For behavioral it is designed to test the classic screening and generic questions. For technical, it has both extensive system design and conceptual questions.

How we built it

We used FASTAPI that utilizes Claude to manage interview states and checklist logics. It won't stop until it gets enough information out of you. Groq Whisper handles instant transcription while OpenAI Nova provides the TTS voice. MediaPipe runs the webcam frames, and we have react for UI.

Challenges we ran into

We ran into orchestration latency, where connecting STT, LLM, and TTS feels robotic if theres a delay. We tried optimizing most of the calls to make it more of an "interview" feel. Since HTTP is stateless, we needed to build an in-memory session that ensures that Claude remembers what was asked a couple turns ago along with the PDF, and linkedIn information. For data scrapping, we used Playwright to have headless servers to scavenge LinkedIn or other job sites to get job descriptions.

Accomplishments that we're proud of

Vision detection: We were able to offload MediaPipe to maintain a smooth feel maintaining a 60 fps front end experience while getting facial metrics.

Created an AI that "knows" when you sufficiently answered a question. Seeing a "technical depth" checklist item flip to "True" based on LLM's decision.

What we learned

Building with LLMs show that Prompt Engineering is basically State engineering. We need to structure it so that AI can act as a reliable judge designed to help the user. Value of "presence" in which social cues are just as important as code/technical quality.