Inspiration

Passwords are reused and leaked, and face unlock alone can be fooled by a photograph. We wanted a sign-in that needs only an ordinary webcam but checks two things at once: who you are (your face) and something only you know and perform (a personal hand gesture).

What it does

WaveLock is a multimodal biometric sign-in:

Face anchor: OpenCV's YuNet detector and SFace recogniser confirm the enrolled face. The face check fails closed and needs a face in most sampled frames.
Liveness challenge: the user shows a randomly chosen number of fingers and turns their head to a randomly chosen side, so a still photograph cannot pass. The challenge is bound to the enrolled face.
Gesture matching: the user's secret gesture is compared with their enrolled template through several gates: dynamic time warping (DTW) on the hand path, finger states, transition order, hand proportions and motion kinematics. The scores are then fused.
Protection around the matcher: attempt limits with a doubling lockout, template aging with a drift anchor, and encrypted templates (Windows DPAPI).

It runs as desktop programs, as a local web app, and as a mobile web app that runs the same Python decision code on the phone (Pyodide).

How we built it

Desktop and web: Python, OpenCV (camera capture, YuNet and SFace face models) and MediaPipe hand landmarks.
Mobile: Pyodide, MediaPipe Tasks Vision and ONNX Runtime Web.
Evaluation: a reproducible leave-one-out evaluation on stored enrolment recordings, an ablation layer by layer, two baselines from the literature, and automated tests (35 core, 15 web-backend, browser and mobile tests).
OpenCV 5 and AWS: [TODO: OpenCV 5 upgrade and AWS deployment, describe when done]

Results (honestly labelled)

MEASURED (nine people, single session, face stage off, zero-effort impostors): 41 of 45 genuine attempts accepted (91.1%, 95% CI 78.8–97.5%) and 0 of 360 impostor attempts accepted (95% CI 0–1.0%).
SIMULATED (synthetic attackers): the kinematic check stopped most hesitant replays, and the face stage stopped replays shown with other faces. A photo-level face combined with a perfect gesture replay was still accepted in 56 of 150 trials. The liveness challenge is meant to stop this attack, but it has not yet been tested with real photos or videos.

Challenges

A real-user study was not possible in our timeline, so we built the most rigorous evaluation we could from stored recordings and labelled every result as measured or simulated.
The frame-by-frame checks were sensitive to when the gesture started, so we added a temporal trim.
We had to run the same Python decision code identically on desktop and phone; the parity tests agreed within 1e-9.

What we learned

Extra security layers cost genuine acceptances on clean data. Their value against informed attackers has to be demonstrated, not assumed. Honest limits are part of the result.

Limitations

Small, single-session dataset; no real-world FAR/FRR is claimed.
The liveness challenge has not been tested against real photo or video attacks.
Mask use has not been tested.
The thumb is excluded from one gesture gate.
The mobile app has not been tested with real users.

What's next

A real-user study across several sessions.
Testing the liveness challenge against real photo and video attacks, following ISO/IEC 30107-3.
Stronger template protection.