Cadence

Inspiration

We love our grandmas. That's the whole origin story.

When we looked at the technology in their lives, almost none of it was built for them: apps with logins, tiny text, menus, notifications that vanish before you can read them. Meanwhile their kids call every evening to ask "did you take your pills?", and the visiting nurse gets updates in fragments. The person in the middle ends up doing the most work just to stay connected.

Also, honestly, this isn't the project we came to build. Halfway through the hackathon, our hardware got stolen. (Yes, really. LOL.) With nothing left to plug in, we opened an empty repo on Saturday evening, about 14 hours before submissions closed, and asked ourselves one question: what would we actually want to exist for the people we love?

The answer was Cadence, and its voice companion, Kibo.

What it does

Cadence is a bedside screen for older adults who live alone. It's always on, there's no login, and the entire screen is one button. Maggie, our 81-year-old persona, taps anywhere or says "Hey Kibo" and just talks:

"I just took my metformin": logged, and her daughter's checklist ticks on her phone.
"Play my messages" / "Tell Sarah I'll bring soup on Sunday": messages read aloud and sent, in the app or by text.
"Call Sarah": Sarah's phone buzzes and she answers a video call in her browser.
"I need a refill": Kibo walks her through it, and her doctor approves it on the care dashboard.
"I've fallen": everyone is alerted immediately, with no AI in the loop.

If she misses a dose, Kibo notices on its own and lets her family know. If she says "what?", Kibo repeats itself, slower. "I can't read that" turns on large print. "Keep this between us" takes the conversation off the record for ten minutes.

Her family gets a calm view ("Maggie's doing well"), and her care team gets the clinical detail. Maggie decides what each of them sees, by voice. Kibo speaks English, Spanish, Mandarin and Vietnamese.

How we built it

Frontend: Next.js 16 and React 19, with an anime.js sphere that ripples while it listens and spins while it thinks. Lexend type (designed for reading fluency), live captions of everything Kibo says, colours that follow the time of day, and automatic night dimming.
Voice (Cartesia): streaming ink-2 speech-to-text for live captions (batch ink-whisper for Mandarin and Vietnamese), and sonic-3.6 text-to-speech in a calm voice at 0.8× speed, mixed to mono for listeners who hear in one ear.
Reasoning (Claude): each utterance becomes at most one tool call over a set of "skills" (log_medication, send_message, call_contact, start_refill, get_help, …), or a spoken answer grounded in the household's notes.
Live state (Convex): one realtime household shared by the bedside, family and care views, plus crons that sweep for missed doses, scheduled check-ins from the nurse, and signalling for WebRTC video calls.
Plumbing: Cloudflare TURN so calls connect across networks, Twilio SMS and ntfy push notifications, deployed on Cloudflare Workers with CI on every push to main.

Our most important design decision was not letting the model do everything. The router is a priority pipeline, and the first layer that claims an utterance wins:

$$ \text{route}(u) = \text{first}\Big(\underbrace{S(u)}{\text{safety}},\ Q(u),\ R(u),\ F(u),\ T(u),\ \underbrace{C(u)}{\text{Claude}},\ \underbrace{K(u)}_{\text{offline}}\Big) $$

Emergencies $S$ are caught by deterministic pattern matching before any model runs. Quick commands $Q$ like "louder" or "say that again" never need a model. An open refill $R$, or the answer to a question Kibo just asked $F$, gets the next claim. Medicine answers $T$ are templated from the real dose log, never generated. Only then does Claude $C$ pick a skill, and if the network or the API goes down, a keyword router $K$ keeps the device from ever going mute.

The prescription refill agent started life as a separate Python prototype that took phone calls over Twilio and Cartesia, and we ported it into the main TypeScript router mid-hackathon. By the end we had around 500 unit tests, mostly on the router, because "Kibo misunderstood Grandma" is not a bug we wanted to discover on stage.

Challenges we ran into

Starting over halfway through. Losing our hardware meant losing our project, our plan and a fair bit of morale. Finding a new idea the whole team was genuinely excited about, rather than one we settled for, took time we didn't have.
Finding a challenge that fit everyone. We have different strengths, so we needed a project with enough surface area for every person to own a real piece: the bedside and router, deployment and translation, the refill agent, the family and care views.
A lot of moving parts. Voice in, a model in the middle, voice out, a realtime database, three views, video calls, SMS and four languages, built on parallel branches that all had to merge cleanly in the middle of the night (16 pull requests between 7 PM and 7 AM). Every integration point was somewhere things could break. Video calls worked great on the same network, then failed the moment one side was on cellular: public STUN alone can't connect peers across networks, so we added a TURN relay.
Simple is a lie. "I think I took my pills" must not log a dose. "Undo that" must never cancel a call for help. Older speakers often pause mid-sentence, so Kibo's listening timeout became adaptive:

$$ t_{\text{wait}} = 2.4\,\text{s} + 1.8\,\text{s} \cdot \mathbf{1}\big[\text{the last word is "um", "and", "the", } \dots\big] $$

Each of these was a small rule. There were dozens of them.

What we learned

Framing and specialization are a product. A voice assistant isn't a new idea; smart speakers have been around for a decade. But a voice assistant for an 81-year-old living alone, her family in another time zone, and her visiting nurse, with privacy she controls and safety that never depends on a model, is a different product for a different market. The core technology barely changed. Who it's for changed everything.

Simple ideas are harder to execute than they seem. "Tap and talk" sounds like a weekend project. Making it work for someone who is hard of hearing in one ear, who says "what?" a lot, who sometimes can't remember whether she took her pills, and who deserves dignity rather than surveillance, is where all the real work lives. The simpler the interface is for the user, the more the system behind it has to absorb.

On the technical side: realtime shared state makes multi-stakeholder apps feel like magic, WebRTC needs TURN in the real world, and wrapping deterministic guardrails around an LLM is what makes it trustworthy enough to sit beside someone's grandma.

What's next for Cadence

A real tablet beside a real grandma (ours first), real pharmacy and health-record integration for refills, and fall-detection hardware, which we'll be keeping a much closer eye on this time.