Kin

A face, a name, and a memory for every text Grandma gets.

Comment

Inspiration

In the last five years, both my great-aunt and my grandpa were scammed.

My great-aunt got a call from someone pretending to be her bank and gave them her account details. My grandpa got a text that looked like it came from a family member in trouble, and he sent money before telling anyone.

Neither of them was careless. They were kind, trusting people who answered when someone reached out. And in both cases, nobody was there in the moment to say, "Wait, let's check on this first." By the time our family found out, the money was gone, and so was some of their confidence.

They aren't alone. Americans over 60 lost $7.7 billion to fraud last year, more than any other age group (FBI IC3). Scammers look for people who are on their own, and as we get older, more of us are.

For people living with dementia or memory loss, it's even harder. A text from a granddaughter can feel like a text from a stranger, and a text from a stranger can feel like family. Many families end up locking the phone to contacts only, which keeps their loved one safe but cuts them off from the people they love.

When we thought about what could have protected my great-aunt and grandpa, it wasn't a blocked number or a locked-down phone. It was a companion: someone gentle, always there, who knew their family and could say "This isn't Nicole's number. I'm checking with Nicole" before anything went wrong.

That's Kin. It keeps older adults safe from the people who shouldn't reach them, and makes it easier to stay close to the people who should, whether they're simply getting older or living with memory loss. And it does it inside the app they already open every day.

Nobody changes how they text her. Kin just helps her answer.

What it does

Kin is a gentle iMessage companion for older adults, including people living with memory loss. It has its own contact on her phone. Her family keeps texting her exactly as before. Kin reads her real threads on the family Mac and sends her a live card in Kin's thread. Her real conversations are never touched.

Our demo family is made up: Grandma (80), her granddaughter and caregiver Nicole, her son Mike, and her daughter Linda.

Protect: a second set of eyes

  • An unknown number asking for gift cards, money or codes, or claiming "Grandma it's Nicole, I lost my phone," gets a calm red card: "This isn't Nicole's number. I'm checking with Nicole." No alarms, no blame.
  • Links are shown as text she can't tap, and Kin never replies to the sender.
  • Nicole's phone buzzes with the message. She replies SAFE or SCAM, and Grandma's card updates in place: "Nicole checked this message for you. You don't need to do anything 💙"

This is the moment we wish my great-aunt and grandpa had had someone beside them.

Recognize: who's reaching out

When family texts, a card follows within seconds: "Nicole · your granddaughter," one verified memory line ("She visited two weeks ago with mangoes"), the quoted message, and up to three one-tap replies. A tap sends the reply as Grandma, in Nicole's real thread, and the card changes to "Sent to Nicole ✓."

Names she might not place get a gentle explanation ("Biscuit is Mike's dog 🐶"). Cards are always statements, never quizzes: Kin never asks "Do you remember…?" If she taps 👍 I know Nicole, Kin steps back for Nicole, and only steps in again for questions, unfamiliar names or stressful messages.

Remind: in the family's own words

Nicole texts Kin "remind Grandma to take her 2pm pill." At 2pm Grandma gets a green card from Nicole, with ✅ Done and ⏰ In 10 minutes. One tap, or even a 👍 tapback, and Nicole gets "Grandma did it ✅." If there's no answer, Kin nudges once in the same card, then quietly lets Nicole know. A ☀️ Today card each morning shows her reminders and upcoming visits.

Listen and talk

  • 🔊 Listen reads any card aloud in a warm ElevenLabs voice, right inside the card, with the words still on screen.
  • 🎙️ Talk to Kin opens a voice conversation inside the iMessage card. Kin reads her the message, suggests a reply, reads it back, and sends only after she says "yes."
  • She can also send Kin a voice note ("Tell Nicole I can't wait to see her"). Kin turns it into a "Send to Nicole?" card, and nothing goes out until she taps Send.

Knowing when to stay quiet

No card for "ok," "👍" or people she's said she knows. A burst of texts waits and becomes one card. Optional quiet hours hold cards overnight. Family only hear about scams, finished or missed reminders, and "she'd like a call." There are no activity reports. Kin is there for her, not watching her.

Family can also add memories by text ("Remember: Mike's dog is Biscuit") or with a captioned photo. Each one appears on a live dashboard that shows Kin's thinking and the family memory map.

How we built it

Photon Spectrum is where Kin lives. One Bun process runs both providers:

  • The local provider (@spectrum-ts/imessage-local) watches Grandma's chat.db on the family Mac and sends replies as her with space.send().
  • The cloud provider is Kin's own iMessage line. Every card is an app() mini-app with a signed URL, and when anything changes (reply sent, reminder done, Nicole says it's a scam), Kin calls edit() on the same bubble, so her inbox never piles up.
  • Card taps POST back to our Bun server through a Cloudflare tunnel, with HMAC-signed URLs.

Gemini 3.5 Flash-Lite gets one structured-JSON call per family message. It returns triage flags, mentioned names, a scam score, a memory pick, up to three replies and an optional summary or calm note. It also handles vision for photo memories and audio for Grandma's voice notes. But plain code decides every action:

  • Rules for when to speak, stay quiet or protect are written in code, not left to the model.
  • verifyMemoryLine() checks that the memory Gemini picked was a real candidate and that every name in it is grounded in stored memory. It also blocks quiz words and question marks. If a check fails, the card simply shows no memory line.
  • Names in photo memories come only from the caption, never from faces.
  • If Gemini is down, rules still send cards with fixed replies, and a regex scam check still protects her.

ElevenLabs powers both voices. eleven_v4 with delivery tags ([warmly], [calmly]) makes the in-card Listen audio, prefetched so the first tap is instant. Conversational AI over WebRTC runs Talk to Kin inside the iMessage card webview. Its three webhook tools (send_reply, ask_family_to_call, mark_reminder_done) only work after a spoken yes, and our server re-checks every call: card state, text length, and no money, code or account words.

Kin has a never list, enforced in code: it never edits or hides a real message, never sends as her without her tap or spoken yes, never touches money or codes, never replies to strangers, never pretends to be family, never invents a memory, never gives medical advice, never reports her day, and never learns from anyone but enrolled family.

Challenges we ran into

  • imessage-kit 3.0.0 crashed on macOS 15's chat.db because a column it expected (ck_chat_id) doesn't exist there. We patched the library.
  • Its file watcher missed a real message. We added a polling fallback and resume-from-last-row, so Kin never loses a text after a restart.
  • Photon Pro's shared line can only message phone numbers that have texted it first. We found out late, and rebuilt our whole device plan at 11 PM.
  • The card is only 280×300 pixels when opened. We redesigned every card around one idea per screen: who and what first, replies on the next tap.
  • iOS wouldn't play audio in the card until we served it with HTTP range requests, and the microphone prompt could hang in the webview, so Talk to Kin times out and offers "Open in Safari."
  • Protecting her without isolating her. Blocking every unknown number would have been easy, and it's exactly what makes older adults lonelier. A calm warning plus a one-word check from family felt like the right middle path.

Accomplishments that we're proud of

  • A voice agent that runs inside an iMessage bubble and can reply as her, but only after she says yes, with our server checking every action.
  • Memory that can't put a made-up fact on her screen. Every line is verified or dropped.
  • It works on real devices: real threads, real replies as Grandma, real caregiver alerts.
  • Replay tests cover the whole pipeline: 34 checks with real Gemini, 19 for the Talk to Kin tools, and 13 with Gemini switched off.

What we learned

  • The hardest part was restraint. Most of Kin's rules are about when to stay quiet and which words never to use.
  • Scams and loneliness are two sides of the same problem. Every safety feature that cuts someone off makes them lonelier, and lonelier people are easier to scam. Real protection has to keep people connected.
  • Older adults don't want to be managed. Writing every card as a warm statement, never a test and never a scolding, changed nearly every design decision.
  • iMessage mini-apps are a real interface, not just a link preview. They're a surprisingly good home for a companion.

What's next for Kin

  • Joining the family group chat through Photon's Business plan, speaking only when someone addresses her.
  • A hosted version, so families don't need a Mac at home.
  • Real phone calls through Photon SIP, reusing the same ElevenLabs agent and safety checks, for "call me" and missed reminders.
  • A consent conversation at setup, where Kin asks her directly, "Is that okay?"
  • Designing with older adults, people living with memory loss and their caregivers before any real launch, starting with our own family.

Privacy: Kin runs on the family's Mac. Messages and memories stay in a local database. Only enrolled family threads are used for memories, unknown senders are only checked for scams, and our demo uses made-up data only.

Built With

Share this project:

Updates

Submission history