Your Path

Your Path

About the project — YourPath

YourPath is an AI-driven interactive video platform built on decision-tree branching scenarios with our own SER and FER model. Users navigate high-stakes real-world moments (behavioral interviews, first dates, critical pitches) through voice-first conversation, while AI evaluates what you say, how you say it (vocal emotion), and what you show (facial expression) to determine the next story branch.

We built YourPath because many important moments in life feel one-shot: you do not get a restart button in a real interview, a big pitch, or a relationship-defining conversation. But you can rehearse them. Our goal is to turn “I wish I practiced this” into “I’ve played this scenario 30 times and I’m ready.”

At the same time, the same engine can be packaged for companies as a personalized interview simulation: candidates go through role-specific, rubric-based scenarios before (or instead of) repetitive early-stage human screens, reducing interviewer workload while keeping evaluation consistent.

Inspiration

We were inspired by two observations:

Life-changing moments are conversational. Outcomes often depend on more than the literal meaning of words — tone, confidence, hesitation, and emotional signals change how others react.
Most training and screening tools are static. Mock interview question lists, scripted role-play, and generic feedback rarely adapt in real time to the conversation.

We wanted to build something that feels like a real interaction — responsive, branching, and emotionally grounded — so people can rehearse the moments that matter, and teams can scale structured evaluations without burning human time on repetitive screens.

What it does

For individuals (B2C)

Interactive branching scenarios: each scenario is a decision tree where every response can change the storyline.
Voice-first gameplay: users speak naturally; the conversation drives progress.
Real-time AI evaluation across three signals:
- Speech content (semantic intent, relevance, reasoning)
- Vocal emotion (prosody cues like energy, pace, pitch variation)
- Facial expression (emotion cues and engagement)
Replayability: the same “one-shot” moment can be rehearsed unlimited times with different approaches.

For teams (B2B)

Role-specific interview simulations: companies create scenarios aligned with job requirements and competency rubrics.
Consistent evaluation at scale: every candidate experiences the same structured interview flow, reducing interviewer variance.
Lower screening cost: automate repetitive early-stage screens and reserve humans for high-signal final rounds.
Actionable outputs: structured scorecards (transcript highlights, rubric scores, decision paths) to support decisions and debriefs.

How we built it

1) Scenario authoring (decision tree)

We modeled each scenario as a directed graph:

Nodes = video segments + interaction prompts
Edges = conditional transitions triggered by AI evaluation

For B2C scenarios, nodes reflect narrative outcomes and coaching-style recovery paths.
For B2B interviews, nodes map to competency checkpoints (e.g., communication clarity, structured thinking, stakeholder empathy).

2) Real-time conversation loop

Capture microphone audio → speech-to-text (ElevenLabs Scribe v2)
Run analysis:
- LLM-based intent + rubric scoring for content
- Speech emotion recognition for vocal cues
- Webcam facial expression classifier for visual emotion
Decide branch → preload next video segment to minimize latency
Render next segment and continue the loop

3) Multi-signal scoring (fusion)

We used a lightweight fusion strategy:

[ Score = \alpha \cdot S_{\text{content}} + \beta \cdot S_{\text{voice}} + \gamma \cdot S_{\text{face}} ] with ( \alpha + \beta + \gamma = 1 )

(S_{\text{content}}): rubric score from LLM/heuristics (relevance, structure, clarity)
(S_{\text{voice}}): vocal emotion signal (confidence/calmness vs stress)
(S_{\text{face}}): facial expression signal (engagement/affect)

Then we apply node-specific thresholds to route the user:

Example: if Score > 0.75 → “positive branch”, else → “recovery branch”.

For B2B, we keep the rubric content-first (emotion is supportive rather than deciding), and we make the thresholds calibratable per role.

4) UX details

Because users may wait a moment while the AI decides, we added a “hold” visual: a subtle idle loop derived from the last video frame so the experience stays immersive.

Challenges we ran into

Latency and immersion: multi-model evaluation (STT + emotion + LLM) introduces delay. We optimized the pipeline, preloaded branches, and used the hold-frame loop to keep the experience smooth.
Signal reliability: emotion signals can be noisy (lighting, camera angle, mic quality). We tuned thresholds conservatively and designed branches to be forgiving rather than overly punitive.
Balancing “game” vs “realism”: too game-like loses training value; too strict becomes stressful. We focused on supportive branching where “bad” outcomes lead to coaching-style recovery paths.
Fairness & consistency (B2B): for hiring use cases, we needed to avoid over-indexing on noisy emotional cues. We structured the logic so rubric-based content remains primary, and emotion signals only provide soft context.
Scenario design effort: building realistic trees takes time. For the hackathon MVP, we shipped a small number of carefully crafted scenarios and scoped admin tooling for later.

Accomplishments that we're proud of

Built an end-to-end voice-driven branching interactive experience.
Integrated three real-time signals (content + vocal emotion + facial expression) into branch decisions.
Delivered a platform that supports both:
- B2C rehearsal (repeatable practice for high-stakes moments)
- B2B screening & training (role-specific structured simulations)
Produced a compelling demo with realistic scenarios that feel replayable and useful.

What we learned

UX is not just model accuracy — it is responsiveness + clarity. Users trust the system more when feedback is quick and the branching logic is understandable.
Multimodal signals are powerful, but should be treated as soft guidance, not absolute truth.
Scenario design is a product skill: good branches require empathy, realism, and clear success/failure states.
For B2B hiring, the value is not only automation — it is standardization + personalization: consistent rubrics with scenarios tailored to each role.

What's next for YourPath

Product

Admin scenario builder v1: generate a first-pass decision tree from a short scenario prompt + max steps, then allow review/approval.
Better personalization:
- B2C: user profiles, difficulty modes, coaching styles
- B2B: role templates, competency rubrics, scenario libraries
Richer feedback loop: transcript highlights, emotion timeline, and suggested alternative responses after each run.

Platform

Candidate / player reporting: structured scorecards + “evidence” from decision paths and key moments.
Calibration tools (B2B): tune thresholds per role, validate consistency, and run internal benchmarking.
More scenarios: negotiation, conflict resolution, leadership conversations, customer support, public speaking.

Tagline
Speak. Feel. Branch.
Rehearse life’s high-stakes moments with AI-driven, real-world simulations.