ScamCheck

Inspiration

Older adults and immigrant families are frequent targets of scams, especially when messages arrive in a language they are less comfortable with or through channels they already trust (email, iMessage, phone calls). Many existing tools require the user to install an app, learn a new interface, or actively check something. We wanted the opposite: a system that lives where they already are, speaks their language, and brings a trusted family member into the loop so the decision is never left entirely to an algorithm.

What it does

ScamCheck protects older parents and immigrant families from scams in the places they already use: Gmail, iMessage, and phone calls. It does not wait to be asked.

Proactive email scanning. Watches Gmail (read-only). When a risky email arrives, it acts on its own.
Outbound warning call. For high-risk emails it calls the user and explains the danger out loud in their language, then follows up with a text.
Ask anything. Users can forward a suspicious iMessage or call the ScamCheck number and get a verdict by text and voice note.
Two-way family loop. On risky items a linked family member is alerted. They can reply (“Don’t pay, I’ll call you”) and ScamCheck relays the reply to the user, translated, as text and a voice note.
Memory across channels. It remembers cases across email, text, and calls, and across family members. During quiet hours it texts instead of calling, and “I already paid” or “I clicked” switches to an urgent path.

The AI flags the risk. The family helps decide.

How we built it

ScamCheck is a TypeScript/Node service backed by MongoDB.

Email path. Gmail API (read-only), polled every 15–20 seconds. New messages run through rule-based evidence checks, then Gemini for a structured verdict. High-risk items trigger an outbound call via ElevenLabs Agents over a Photon SMS number, followed by an iMessage and a voice note.
Inbound path. Users forward a message to the Photon iMessage number or call the ScamCheck number. The same evidence and analyze pipeline returns a text verdict and a voice note.
Family loop. On risky cases the linked family member receives a minimal alert. Their reply is translated and relayed back to the user as text and a voice note.
Voice. ElevenLabs Agents handle both inbound and outbound calls, with server tools for user context, case recording, and family alerts. Post-call webhooks continue the conversation in text.
Safety. Explicit consent before Gmail access, a banned-word filter on all outputs, prompt-injection defenses, and no raw content persisted.

All changes went through pull requests. We maintained a VEYZL audit log so findings on data minimization, human oversight, and consent could be traced to concrete fixes.

Challenges we ran into

Photon, Gmail, and ElevenLabs each have their own webhook and credential models. Keeping identity (phone number) consistent across channels took careful mapping.

The voice agent must open with the correct context on an outbound call and later reference the same case in text. Passing per-call context and writing the post-call follow-up without storing transcripts required extra design.

Avoiding false alarms on real brand emails while still catching lookalikes and hidden links meant iterating on the fixture set until the rule floor and the model agreed.

Quiet hours, “I already paid,” and the same scam hitting two family members each needed an explicit path so the system stayed helpful without becoming noisy.

Accomplishments that we're proud of

A working end-to-end demo: forwarded iMessage or risky email, then a text verdict, an optional warning call, and a family relay, all in the user’s language.
A rule-based floor the model cannot override. Clear red flags (lookalike domains, hidden links, authentication failures) always raise the risk score.
Fixture results on the rule checks: 23 scams caught, 0 missed, 0 false alarms, across 45 fixtures.
Strong data minimization: cases store only a short summary, verdict, scam type, and domain. Raw email, text, images, and audio are never persisted.
Family alerts that name who was contacted and the scam type only. The original message is never included.
A VEYZL audit that changed the build: a self-harm message no longer gets “no red flags,” card numbers and SSNs are stripped before they reach the model, and instructions aimed at the AI are refused without turning the scam check off.

What we learned

Rule-based checks (domain mismatches, authentication failures, hidden links) are more reliable than the model alone for clear red flags. The model explains. The rules set a hard floor.
Data minimization is both a privacy requirement and a practical constraint. Storing only a short summary, verdict, scam type, and domain simplified the schema and reduced risk.
Voice and text need to feel like one continuous agent. Context has to survive the handoff from an outbound call back into iMessage.
Family relay is the human-oversight mechanism. Alerts stay minimal (who and scam type only) so they do not become another copy of the scam.

What's next for ScamCheck

One calm daily summary instead of a text for every email.
Broader language coverage and more natural voice for additional languages.
An optional WhatsApp channel once Photon support is stable.
Stronger same-scam detection across larger family groups.
Android SMS and Outlook, and a wider list of official bank domains.
VEYZL on every pull request so privacy and oversight controls stay visible as the system grows.