SpeakUp

NemoTron Generated Judging Analysis
Landing Page!
What Would You Like To Practice?
PRACTICE!

Inspiration

Everyone tells you to practice speaking. Nobody tells you who you're supposed to practice on.

Between the four of us we had bombed a job interview, frozen up in a 200 person lecture hall, and accepted a salary offer without countering because asking felt rude. The advice is always the same: practice out loud, get feedback, do it again. But your roommate will not roleplay a hiring manager at 2am, your mirror does not tell you that you said "um" forty-one times, and the one place you find out you're bad at this is the place where it actually counts.

We wanted a place to fail first.

What it does

SpeakUp is a speaking practice platform that puts you in the room before you have to be in the room.

You pick a scenario. Mock job interview. Public speaking to an auditorium. Thesis defense in front of a committee. Salary negotiation with a manager who does not want to give you money. Each one comes with an AI character who has an actual personality and agenda, not a generic chatbot that agrees with everything you say. The hiring manager pushes back. The professor asks about your methodology. The manager asks for your number first and waits.

While you talk, SpeakUp reads your face through your webcam and tracks emotional signals in real time: anxiety, confidence, eye contact, engagement. When the session ends you get a full report. Filler word count, pacing, where your confidence dropped and what question caused it, and a transcript annotated with the moments you lost the room.

Then you run it again and watch the numbers move.

How we built it

Facial analysis runs [client-side with MediaPipe / face-api.js] so frames never leave the browser. We map raw landmark and expression output into a smaller set of signals that actually mean something to a user, because "brow furrow intensity 0.34" is not feedback.

Reports are generated at session end by feeding the transcript plus the time series emotion data into a scoring pass (NEMOTRON WHOOOO), so the feedback connects what you said to how you looked when you said it.

Challenges we ran into

Emotion detection is noisy. Frame by frame output jitters hard, and a raw feed made everyone look like they were having six emotions per second. We ended up smoothing over a rolling window and only surfacing sustained changes, which killed the flashy demo graph but made the data honest.

Latency was the other one. A conversation dies if the character takes four seconds to answer. We had to overlap transcription, generation, and playback instead of running them in sequence, and cut our prompts down significantly to keep responses fast enough to feel like a person.

Getting the AI characters to stay difficult was harder than expected. Left alone they drift toward being supportive and encouraging, which is the opposite of useful. Practicing a salary negotiation against someone who immediately says yes teaches you nothing. We rewrote the persona prompts several times to make them hold their position under pressure.

Accomplishments that we're proud of

It works end to end. You can sit down, pick a scenario, have a real conversation with a character who pushes back, and walk away with a report that tells you something you didn't know about yourself.

The facial analysis pipeline runs entirely in the browser, so we got the feature without asking anyone to upload video of their face to a server we built in 24 hours.

The characters are genuinely uncomfortable to talk to, which sounds like a bug and is actually the product.

What we learned

That the useful part of feedback is specificity. "You seemed nervous" is worthless. "Your confidence dropped when he asked about the gap in your resume, and you filled four seconds with um" is something you can fix. Technically, a lot about real time multimodal pipelines, and that the bottleneck in a voice app is almost never the model. It's everything you stacked around the model.

We also learned that prompt engineering for personality is closer to writing characters than writing instructions.