AURA

AI-Based Unified Response Assessment

Comment

Inspiration

As digital-first communication replaces face-to-face interactions, many people — especially Gen Z — struggle with confidence, clarity, and professional expression.
Traditional interview prep tools focus on what to say, not how it’s said.

We wanted to build a system that treats communication itself as a skill that can be measured, trained, and improved — just like coding or fitness.

What it does

AURA is an AI-powered interview practice and soft-skill assessment platform.

It simulates realistic interview conversations using an AI interviewer and objectively evaluates users based on how they communicate, not just their answers.

Using multimodal analysis of text, voice, and video, AURA extracts 48+ behavioral features and converts them into clear skill scores for confidence, clarity, empathy, and overall communication, along with actionable feedback and progress tracking.

Learn more about multimodal ML

How we built it

We designed AURA as a modular, production-style system:

Frontend (React + WebRTC)
Real-time interviews, AI avatar interaction, and client-side video perception

Backend (Node.js + Express + Socket.IO)
Session orchestration, authentication, and real-time communication

Perception Layer (FastAPI)
Behavioral feature extraction using NLP, audio signal processing, and computer vision

Decision Layer (FastAPI + XGBoost)
Scoring communication skills using trained ML models on a frozen 48-feature contract

LLM Integration (OpenRouter / Gemini)
Adaptive interview conversations

MongoDB + Auth0
Secure session tracking and user analytics

This separation allows the system to scale as a true human-skill intelligence pipeline, not a monolithic chatbot.

Challenges we ran into

  • Designing objective metrics for subjective human skills like confidence and empathy
  • Synchronizing real-time audio, video, and text data across services
  • Preventing LLM bias from leaking into skill evaluation
  • Maintaining consistent feature extraction across different practice modes
  • Balancing real-time performance with accurate multimodal analysis

Accomplishments that we're proud of

  • Built a 48+ feature multimodal behavioral pipeline from scratch
  • Created a clear separation between perception and judgment, improving explainability
  • Achieved real-time interview simulation with post-session ML evaluation
  • Designed a scoring system users can actually understand and act on
  • Delivered a full-stack, multi-service architecture within a limited build window

What we learned

  • Human skills can be quantified without reducing them to shallow heuristics
  • Separating perception from decision-making dramatically improves model trust
  • Feedback matters more than raw scores — explainability drives user growth
  • Building AI for humans requires as much UX thinking as ML accuracy

What's next for AURA

  • Larger, more diverse training datasets for improved model robustness
  • Personalized learning paths based on historical performance
  • Live feedback cues during interviews (posture, pace, eye contact)
  • Enterprise and campus hiring integrations
  • Expanding beyond interviews into presentations, sales calls, and leadership training

Built With

Share this project:

Updates