DeepScan Live

Catches AI deepfakes AND off-camera human proxies in live interviews — entirely in your browser, with zero data uploaded, ever.

Comment

Inspiration

Interviews used to have one failure mode: a candidate who lied on their resume. In 2026, they have two more — a face that isn't real, and a voice that isn't in the room.

Real-time face-swap tools and voice clones have gotten good enough, and cheap enough, that a fabricated candidate can now pass a live technical interview. On the other side of the same threat, a candidate can quietly have someone else off-camera feeding them every answer — no deepfake required, just a second monitor and a friend.

Enterprise tools like BrightHire already defend against this, but they're locked behind contracts that only large companies can justify. A solo recruiter, a three-person startup doing its first technical hire, a freelance recruiter running back-to-back interviews — none of them have any defense at all. That gap is what we set out to close: not a research demo, but something that actually runs, for free, in a browser tab, the moment you need it.

What it does

DeepScan Live is a browser-based dashboard that watches a live interview feed and flags synthetic media and human proxies in real time — entirely on-device. Nothing is ever uploaded.

It fuses two independent fraud signals into one live confidence score:

  • Lip-sync latency — cross-correlates the audio waveform against real-time lip landmark movement to detect the timing desync common in synthetic media, while a confidence gate prevents silence (someone just listening) from being mistaken for evidence of fakery.
  • Head-pose consistency — tracks yaw/pitch/roll frame-to-frame and flags two distinct anomaly patterns: physically implausible jumps, and unnaturally smooth motion — a subtler tell, since synthetic heads often move with a mechanical uniformity real humans don't.

Alongside that, it independently watches for a second, entirely different threat: an off-camera proxy. If a second face enters frame and stays for a sustained window, a clear alert fires — catching the "someone feeding me answers" pattern that no deepfake detector on the market addresses.

Every session builds a scrubbable confidence timeline, a live network panel that proves — visibly, in real time — that zero data ever leaves the browser, and a one-click PDF incident report a recruiter can hand straight to a hiring manager.

How we built it

Everything runs client-side: React + Vite + Tailwind on the front end, MediaPipe's Face Landmarker for 478-point real-time facial tracking, the Web Audio API for live waveform analysis, and Recharts for the timeline visualization. There is no backend. No video, audio, or derived data is ever transmitted anywhere — which we don't just claim, we prove, live, via a network activity panel that intercepts every outbound request and shows the count on screen. It's designed to be fact-checked against a judge's own browser DevTools, not taken on faith.

The two fraud-detection signals feed into a confidence-gated fusion engine: when someone isn't speaking, lip-sync data is unreliable, so the engine automatically leans on head-pose instead; when both signals agree, it weights the stronger evidence more heavily. The result is smoothed with an exponential moving average so the live meter reads as a stable, trustworthy signal instead of a flickering number — because a fraud-detection tool that looks unstable undermines its own credibility before it ever catches anything.

Challenges we ran into

Our original plan included a third signal: rPPG, extracting a heartbeat pulse from subtle skin-color variance in the face. We built it — full chrominance-based (POS) processing, bandpass filtering, honest signal-quality gating that refuses to show a fabricated BPM reading when the signal is weak. But webcam compression is genuinely hostile to that signal, and a live demo depending on a fragile heartbeat reading is a real risk we weren't willing to take into a hackathon judging round. We made the call to drop it and redirect that time into off-camera proxy detection instead — which, in hindsight, closes a real gap in our own threat model that rPPG never actually addressed.

We also caught real bugs the hard way. Our risk-alert banner once got stuck showing "High confidence" seconds after the live meter had already dropped back to "Low Risk" — a direct, credibility-damaging contradiction we found and fixed before it could surface during judging. Feeding a pre-recorded video through the exact same live-analysis pipeline (instead of faking a separate demo path) turned out to be its own rabbit hole: browsers silently throttle decode speed for video elements that exist outside the DOM tree, which quietly tanked our frame rate to 7 FPS until we found it and forced the element back on-screen (invisibly) to restore full performance.

Accomplishments that we're proud of

Every feature was built, then independently verified against adversarial inputs, not just happy-path testing: talking normally, staying silent, deliberately shaking our head as hard as possible, covering the camera mid-session, bringing a second face in and out of frame repeatedly. The system never produced a false alarm on honest, normal human behavior — and correctly paused rather than panicked the moment a face left frame. That reliability, more than any single feature, is what we're most proud of.

What we learned

The most important lesson wasn't technical — it was about restraint. The flashiest planned feature (rPPG) wasn't the right feature to ship into a live-judged demo, and choosing to cut it, honestly and early, made the rest of the product stronger rather than weaker. A fraud-detection tool's credibility depends entirely on never crying wolf, and we learned that false-positive protection has to be designed in from the first line of scoring logic, not patched on afterward.

What's next for DeepScan Live

  • A lightweight plugin wedge into ATS platforms like Greenhouse and Lever — the real path to scale, rather than a standalone tool recruiters have to remember to open
  • Packaging as a Chrome extension to remove all install friction
  • Extending the same architecture to live proctoring for online exams — an adjacent market with an identical fraud pattern

Built With

  • mediapipe
  • react
  • recharts
  • tailwind-css
  • vite
  • web-audio-api
Share this project:

Updates