Swivel Sentio

Inspiration

My grandmother was scammed out of $200,000. I didn't find out until it was too late. I don't talk to her every day, so I had no idea she was sending money, and by the time anyone in the family knew, it was gone. I was completely powerless.

That's the gap Swivel Sentio is built for. Scams against older adults all run on the same script: fear, urgency, and secrecy. A fake IRS agent threatens arrest. A "grandson" needs bail money and begs you not to tell his mom. A text says your package is stuck until you pay $1.99. The secrecy is the point: scammers make sure the family never hears about it. Banks catch fraud after the money moves. I wanted a calm moment of friction before it moves, and a way for family to be in the loop even when they aren't in the room.

If this had existed, my phone would have buzzed the first time she tried to send that money. The question every grandparent should ask is our domain: is this really you?

What it does

1. Reads the call or text. Google Gemini scores how likely it is to be a scam, names the type, and explains why in plain words.
   1. Notices how Granddad is reacting. An on-device camera check-in reads his expression (Calm to Scared), and an optional heart monitor reads his pulse. A scared face or racing heart raises the risk.
   2. Pauses the payment and calms him. His screen gently takes over, and an ElevenLabs voice talks him through it without repeating the scammer's threats.
   3. Asks family. His daughter's real phone buzzes with the amount, scam type, Gemini's explanation, and how he looks. Veto is one tap. Approve takes two deliberate taps and never works from the lock screen. His screen updates the instant she decides.
   4. Remembers. Every payment check, stress reading, and family decision is stored in Tiger Data.

How we built it

## How we built it

• Node.js + Express backend, plain HTML/CSS/JS frontend: a calm screen for Granddad and a phone page for his daughter.
• Google Gemini (gemini-2.5-flash) with structured JSON output for score, scam type, pressure phrases, and a plain-language explanation, fused with offline keyword rules so it works with no network.
• ElevenLabs text-to-speech with scripts written for someone already frightened; each clip is generated once and cached.
• Tiger Data (TimescaleDB on Tiger Cloud): hypertables for stress readings, scam incidents, and every payment check, a real-time continuous aggregate, and persisted family decisions.
• face-api (TensorFlow.js) for in-browser expression detection; video never leaves the device.
• Web Bluetooth heart-rate proof of concept (standard Heart Rate service) plus a labeled demo monitor.
• ntfy push notifications and a Cloudflare Tunnel on our GoDaddy Registry domain, isthisreallyyou.biz.

Challenges we ran into

• Webcam heart-rate was unrealistic and we didn't have those monitoring tools to estimate real heartrate properly, so we switched to facial expression and use that as a indicator of getting scammed instead of a heart rate monitor.

Accomplishments that we're proud of

• A real end-to-end moment: a scam call on the laptop makes a real phone buzz, one tap vetoes it, and the laptop updates instantly.
  • Safety-first choices: one-tap veto but two-tap approve, no lock-screen approvals, spoof-resistant family checks, and every service fails safe.
  • Gemini, ElevenLabs, and Tiger Data are genuinely integrated, with cost guards. We label every simulated piece honestly.

What we learned

We learned Gemini structured output, ElevenLabs voice design, TimescaleDB hypertables and continuous aggregates, Web Bluetooth, and Cloudflare tunnels this weekend. We also learned that designing for a frightened person means short sentences, reassurance first, and no alarming words, and that well-timed friction protects people more than detection alone.

What's next for Swivel Sentio

A native phone app that screens real calls and texts, Presage's camera vitals SDK for clinical-grade heart rate, real bank holds, a larger trusted circle, and Spanish-language support. heart beat monitor support.

Built With

• cloudflare
• css
• elevenlabs
• express.js
• google-gemini
• html
• javascript
• node.js
• ntfy
• postgresql
• tensorflow.js
• tiger-data
• timescaledb
• web-bluetooth