ReplAI

Home Page
Job Interview and Notes
Landlord Confrontation

Inspiration

Difficult conversations are hard to practice because they are high-pressure and usually happen when the stakes are already real. We wanted to build a way for people to rehearse moments like salary negotiations, job interviews, landlord disputes, HR complaints, and legal client meetings before they have to handle them in real life. For our demo, we focused on legal client intake because it adds a real professional workflow: conflict-of-interest checks. Law firms need to know whether a potential client is connected to an existing matter before they take the case, but that process is often manual and happens after the conversation. We wanted ReplAI to surface those risks while the intake is still happening.

What it does

ReplAI puts users in difficult conversations where an AI plays the other person. During the conversation, the webcam and microphone track communication signals in real time, including eye contact, fidgeting, posture, speech pace, vocal energy, and filler words. After each round, the user gets a scorecard showing how they performed and what they can improve. In the Legal Client Intake scenario, ReplAI also listens for names and companies mentioned by the client. It extracts those entities during the conversation, fuzzy-matches them against a mock law firm database, and flags potential conflicts in real time. For example, it can identify a high-risk opposing party or a related company connected to an open matter before the meeting is even over.

How we built it

We built ReplAI with Next.js, React, TypeScript, Tailwind CSS, Three.js, Gemini 2.5 Flash, LiveKit, MediaPipe, and a custom conflict-matching engine. Gemini powers the AI role-play, structured JSON output, entity extraction, coaching annotations, metrics-aware feedback, character conditioning, and the conflict analysis pipeline. LiveKit handles the echo-cancelled microphone flow, while MediaPipe tracks body-language signals from the webcam. The scoring system combines communication metrics with the conversation context to generate feedback after each round. For the legal conflict workflow, we built a custom fuzzy matching engine using Levenshtein distance and Jaccard token similarity. This lets the system catch name variations like Johnson Associates versus Johnson & Associates LLC, while also tracing indirect relationships such as subsidiaries, spouses, and business partners.

Challenges we ran into

The hardest part was making multiple real-time systems work together smoothly. The app needed to handle AI conversation, microphone input, webcam tracking, entity extraction, conflict matching, and live UI updates at the same time. Another challenge was making the legal conflict detection useful instead of noisy. Names and companies can appear in different formats, so exact matching was not enough. We needed fuzzy matching and relationship tracing to catch realistic conflicts without flagging everything as high risk. We also had to design feedback that felt actionable. A communication score is only helpful if the user understands why they received it. For example, if the attorney does not ask probing questions, the client may never mention key parties, and the system may not have enough information to detect a conflict.

Accomplishments that we're proud of

We are proud that ReplAI is more than a chatbot. It creates a full practice environment with role-play, live body-language tracking, voice analysis, real-time conflict alerts, and post-round coaching. We are especially proud of the Legal Client Intake scenario because it connects conversation practice to a concrete professional problem. The system can flag risks like David Chen as a high-risk opposing party or Apex Industries as a related entity in an open case while the attorney is still conducting the meeting. We are also proud that the app gives feedback on both communication and information-gathering. It does not just ask whether the attorney sounded confident; it also helps show whether they asked the right questions to surface all the parties involved.

What we learned

We learned that real-time AI applications need strong coordination between the model, the interface, and the supporting logic. The AI conversation is only one part of the product; the experience also depends on low-latency audio, usable visual feedback, reliable entity extraction, and clear scoring. We also learned how important structured outputs are. For the legal workflow, Gemini needed to return usable entities and analysis that could feed into the conflict engine, not just natural-language responses. Most importantly, we learned that better coaching depends on context. Feedback becomes much more useful when the system knows what happened in the conversation, what names came up, what questions the user asked, and what risks were detected.

What's next for ReplAI

Next, we want to add more realistic firm data, stronger relationship graphs, and more advanced conflict explanations. We also want to expand the scenario library beyond legal intake and improve long-term progress tracking so users can see how their communication skills change over time. We would also like to make the coaching more personalized, with follow-up drills based on what the user missed during the round.