• 

• 

• 

• 

• 

• 

Handshake — Verify the Person, Not the Voice

Inspiration

Voice cloning is changing what it means to recognize someone. A familiar voice is no longer reliable proof of identity, and trying to detect every synthetic voice creates a constant arms race: as generation improves, detection methods can become less reliable.

We asked a different question: What if we stopped trying to detect fake voices and instead made it easier to verify real people?

Consider a scam that targets families: "Mom, it's me. I'm in trouble. I need money right now." The voice sounds familiar, the situation feels urgent, and the victim has little time to think. By the time they question the call, the money may already be gone.

The scale of the wider fraud problem is substantial. According to the FBI's 2025 Internet Crime Report, people aged 60 and older filed 201,266 elder-fraud complaints, reporting approximately $7.748 billion in losses. These figures cover elder fraud broadly, not losses attributable exclusively to AI voice cloning. Source: FBI IC3 2025 Annual Report.

As a student building a cybersecurity project, I wanted to focus on a practical problem that ordinary people and families could face, rather than treating synthetic media as a problem only for researchers and security professionals. Handshake explores a simple principle: people should have a way to verify identity without needing to know how deepfakes work.

What it does

Handshake is an Android-first, verification-first prototype designed to help people establish trust around phone calls and messages.

• Trusted Circle: People establish a trusted relationship by pairing their phones through a short-lived QR invitation. Both participants confirm the relationship, creating a foundation for device-level trust.
• Device-level verification: The application is designed to distinguish a trusted connection from a situation requiring verification. Its intended states are Trusted connection, Verify, and Risk detected. These states must reflect available evidence rather than an assumption that a call is safe.
• Pressure Check: An LLM analyzes user-provided text, such as a transcript or message, for manipulation tactics including artificial urgency, secrecy, authority pressure, and requests for immediate payment. Its result is advisory, not a definitive verdict that someone is a scammer.
• Personalized challenge: A supporting capability can generate a verification question based on relevant private context shared between trusted people.
• First-hour response: A practical checklist guides users through actions to consider during the first 60 minutes after money has been sent to a suspected scammer.

The central idea is to reduce dependence on voice recognition. Handshake does not attempt to determine whether a voice sounds real. It uses device-level trust and contextual risk signals to help users make a more informed decision.

Important limitation: The current prototype does not access the remote audio of ordinary cellular calls or WhatsApp calls. It must not be described as a live call-audio analyzer or a cloned-voice detector.

How we built it

Device trust and pairing

We implemented the foundation for QR-based device enrollment, mutual confirmation, and server-backed trust sessions. Invitations are designed to be short-lived and single-use, reducing the need for users to exchange codes verbally or remember complicated credentials.

The current mobile direction uses device trust rather than relying on rotating codes as its primary identity mechanism. Legacy rotating-code routes remain in the backend for compatibility.

LLM layer

Handshake uses Featherless AI through an OpenAI-compatible API for its text-analysis and challenge-generation capabilities.

The backend validates structured model responses with Zod schemas. This helps prevent malformed model output from being treated as a valid application response. Pressure Check analyzes supplied text; it does not establish the identity of the speaker.

Frontend and backend

The project combines:

• Mobile: Expo and React Native, with native Android integration written in Kotlin.
• Web and API: Next.js 16, TypeScript, and the App Router.
• Validation: Zod for API inputs, outputs, and structured LLM responses.
• Styling: Tailwind CSS for the web interface.
• Hosting: Vercel for the web application and API deployment.
• Build and testing: GitHub Actions for automated checks and Android release APK generation.

Demonstration setup

A voice-clone sample can illustrate the impersonation problem, using my own voice with consent if included in the final demonstration. The sample is illustrative: it is not evidence that Handshake captured or analyzed a live conversation.

Any comparison with a commercial voice detector should report only the result actually observed during testing. Handshake's main approach is device verification, not a competition to classify synthetic audio.

Challenges we ran into

The hardest challenge was the gap between the product experience we wanted and the access Android actually provides to third-party applications.

We wanted Handshake to recognize incoming calls and display useful information automatically, including during calls made through applications such as WhatsApp. Android can expose certain call-state events, but an ordinary third-party application cannot assume access to the remote side of a cellular conversation or to WhatsApp's private two-way audio.

That distinction changed the scope of the project. A visible overlay is not proof that audio is being captured, a call is being transcribed, or a scam is being analyzed in real time. WhatsApp call-popup detection also remains best-effort in the current release and is not guaranteed to trigger on every call.

Another challenge was establishing reliable end-to-end verification. The pairing and trust-session flows are implemented in the codebase, but the complete QR invitation, scanning, acceptance, and confirmation flow has not been verified on the exact latest release APK. Automated tests passing does not establish that the complete mobile experience works on a physical phone.

We also identified backend limitations that matter beyond a hackathon. Trust state is currently held in process-local memory rather than durable shared storage. That can cause inconsistencies across serverless instances or after a process restart. Production use would require persistent storage, reviewed cryptography, stronger authentication, and a more complete device-recovery and revocation process.

These limitations are part of the project's current status, not problems we can honestly claim to have fully solved.

Accomplishments that we're proud of

• Built an Android-first mobile prototype alongside a Next.js backend and web prototype.
• Implemented the foundations of QR-based trusted-device enrollment and server-backed trust sessions.
• Developed an LLM-backed text-analysis capability that treats manipulation signals as advisory rather than proof of identity.
• Added native Android call-state and overlay integration, while documenting the platform constraints that prevent the current application from analyzing remote call audio.
• Established an automated GitHub Actions build and testing workflow.

In the latest documented automated verification on October 10, 2026, GitHub Actions successfully built an Android release APK. The automated checks reported 133 passing root tests and 9 passing native configuration-plugin tests, with no failures in those suites.

That is meaningful engineering progress, but it is not equivalent to a fully validated product. Physical-device verification of the latest APK, including the complete pairing flow and repeated call-overlay tests, remains outstanding.

The most important accomplishment is turning a different approach to voice impersonation into a concrete software prototype while identifying where the current implementation still falls short.

What we learned

Identity verification and synthetic-media detection solve different problems. A detector tries to classify the media itself. A verification system asks whether the person can establish a trusted relationship through another mechanism. The second approach can reduce reliance on voice familiarity, although it still depends on the integrity of the enrolled device, the pairing process, and the backend.

We also learned that mobile platform boundaries must shape product design from the beginning. A permission to display an overlay does not grant access to private call audio, and detecting a call event is not the same as understanding the conversation.

On the LLM side, structured output validation and clearly bounded inputs make it easier to integrate model responses into an application. Even so, an LLM's assessment of a message should remain a contextual warning rather than a definitive identity or fraud verdict.

Finally, a successful build is only one part of verification. The complete user journey must be tested on the exact release artifact, and untested behavior must remain explicitly unverified.

What's next

• Complete and verify the QR pairing flow on physical Android devices.
• Improve incoming-call event detection and test overlay reliability across cellular calls and supported third-party calling applications.
• Investigate audio analysis only through platform-supported interfaces that legitimately expose an analyzable audio stream.
• Replace process-local trust state with durable shared storage.
• Strengthen device authentication, key management, revocation, recovery, and abuse controls.
• Expand physical-device testing and document results for both successful and failed scenarios.

Our goal is to move from a hackathon prototype toward a dependable verification tool without promising protection that the implementation cannot yet demonstrate.

Ethics note

Any voice clone used in the demonstration is based on my own voice and created with my consent. No third party is impersonated without permission. The sample illustrates the impersonation threat; it is not presented as audio that Handshake has analyzed or classified in real time.

The project is a prototype, not a guarantee against fraud. Handshake does not currently detect cloned voices or provide live analysis of remote call audio. Those limitations are stated explicitly so that users and judges can distinguish implemented capabilities from future work.

Built With

• c#
• css3
• javascript
• kotlin
• powershell
• python
• typescript