My Friend (Elderly Helper)

AI that helps the elderly read documents in their languages and connect them to an accountable human

Comment

Inspiration

I'm a law student. For two and a half years I've worked in social justice, and at a law firm I've helped many vulnerable, elderly clients. One came in carrying a plastic bag holding thirty years of their life — a marriage now ending in divorce, terrified of losing their home. I sat with them for ninety minutes. Not on the law — on how to open a PDF. They kept apologizing: "I'm so slow." They weren't slow. Technology was the wall between them and justice.

The hardest problem isn't a lack of technology or resources — the help already exists. The problem is that the people who need it most live outside the system that holds it. They don't know the resource exists, can't navigate to it, and won't come asking. Most can't even name their problem — they just say "I don't understand this." Generic AI assumes you'll show up and type a clear question — exactly what a marginalized, non-English-speaking elder cannot do. And for vulnerable people, an answer isn't help — it's a risk. Real help requires a responsible, accountable human behind it. That accountability is the whole point.

## What it does

My Friend triggers help the moment a problem appears in everyday life — a letter, a court form, a bill they can't read.

  • Picks their language (English, Español, 中文, Tiếng Việt, Tagalog) — the entire app, spoken and written, switches to it.
  • Points the camera at a document — it auto-magnifies and brightens so they can see it.
  • Captures it — Claude reads it aloud in their language, explaining what it is and what they need (they never have to know what to ask).
  • Surfaces the right resources — legal aid for a court form, a social worker for everyday matters, housing or health when those fit.
  • Connects them to an accountable person they choose — never automatically to a family member, always with a full audit trail.

No typing, no reading fine print: every step works by voice and big single-tap buttons. The key idea: the AI knows when not to answer — it gives information, never legal advice, never files to a court, and a real human always confirms.

## How we built it

  • TanStack Start + React, mobile-first, deployed on Lovable Cloud.
  • Claude (Anthropic) — document vision, plain-language explanation, multilingual output, and natural-language intent ("I can't see this" → magnify). Claude understands and answers in all five languages.
  • ElevenLabs (eleven_multilingual_v2) — one warm voice across all five languages, with a browser-speech fallback so audio never goes silent.
  • Supabase edge functions + realtime — handoffs appear live in the staff review queue with a tracking number and audit trail.
  • A static i18n dictionary localizes the UI instantly in five languages.

## Challenges we ran into

  • Reliable capture without a heavy scanner library: OpenCV/jscanify froze the app, so we crop after capture — Claude returns the document's bounds and we crop deterministically. Approximate but reliable.
  • End-to-end multilingual: making the UI, the spoken voice, and the AI's analysis all speak the user's language — with no leftover English or double-translation — meant cleanly separating pre-stored UI strings from AI content returned in-language.
  • Mobile audio & permissions: iOS blocks audio before a user gesture and bundles mic/camera permissions, so we unlock audio on first tap and
  • Keeping the AI safe: rules decide completeness and routing, not the model — so My Friend never gives legal advice or auto-files.

## Accomplishments that we're proud of

A fully voice-first, zero-typing, five-language app that a near-zero-literacy elder could finish alone. An accountability layer that's genuinely safe — the user picks who they trust, the handoff is logged, and a human always confirms. And most of all, reframing the problem: not "give better answers," but "reach people outside the system and connect them to a responsible human." The voice speaking warmly in someone's own language is the moment that makes it real.

## What we learned

The biggest lesson is product, not technical: the gap for vulnerable people isn't technology or resources — it's reach and accountability. They can't name their problem and can't verify an answer, so an answer alone can harm them; a responsible human is the actual deliverable. Technically: AI vision bounding boxes are approximate, not pixel-perfect; a static i18n dictionary plus AI-output-in-language beats slow runtime translation; and demo-safe fallbacks (browser speech, deterministic crops, big buttons behind every voice action) are what make an accessible app trustworthy.

## What's next for My Friend (Elderly Helper)

Every self-help court form; partnerships with legal-aid clinics and social services as grant-funded public infrastructure (e.g., the federal Technology Initiative Grant) so a real accountable institution stands behind every handoff; spoken language auto-detection so a non-English elder can simply speak and be met in their own language; and triggering help proactively — surfacing the right resource the moment a confusing document enters someone's life.

Built With

  • anthropic
  • claude
  • edgefunctions
  • elevenlabs
  • react
  • realtime
  • speech
  • start
  • supabase
  • tailwind
  • tanstack
  • typescript
  • web
Share this project:

Updates