Cuidao

¿Ya te las tomaste? When family lives far away or caregivers can’t be there, who checks the pills? Cuidao: a simple alarm for Puerto Rico’s elders and live dose alerts for loved ones.

Comment

Inspiration

Cuidao started with my 98-year-old grandmother. Her only pill reminder is a basic iPhone alarm. When it rings, she turns it off, and later she often can't remember whether she actually took her pills. Sometimes she takes them and forgets. Sometimes she thinks she took them and didn't. The family ends up calling and asking, "¿Ya te las tomaste?"

Refills are the second problem. She takes several pills a day, so it's hard for her to notice when a bottle is running low, and she often forgets to tell us until it's empty. Then someone has to rush to the pharmacy.

Her story is common in Puerto Rico. The island is aging fast: about 199 adults aged 65+ for every 100 children under 15 (U.S. Census Bureau, 2023). 4 in 10 households include an older adult (Instituto de Estadísticas de PR). Many families live in the States or work long hours, care homes are expensive and often not covered by insurance, and the Ombudsman for Older Adults received 7,476 complaints in one year, mostly neglect and financial exploitation (OPPEA, 2024–2025).

I built Cuidao for my grandmother, and for every older adult and caregiver dealing with the same daily hassle.

What it does

Cuidao is a web app with two connected screens.

For the older adult

  • A full-screen alarm with sound and one large button at a time: "Las estoy tomando", then "Listo, ya me las tomé".
  • A camera confirmation: when they tap Listo, the camera tracks the hand and checks that the pill bottle is in it before marking the dose as taken. A manual option is always there if the camera fails.
  • Snooze (5 minutes, up to 2 times), and if an earlier dose was never confirmed, the app asks about it before the next one. No more "did I or didn't I?"

For the caregiver

  • A live feed of every step: time to take, taking, taken, snoozed, not confirmed, not taken.
  • Pill count and refill alerts: each confirmed dose subtracts from the bottle, and the caregiver is alerted before it runs out.

$$\text{pills left} = \text{pills in bottle} - \text{doses confirmed}$$

  • Medication history: Activos for current medications and Inactivos for past ones, kept for reference.
  • Three ways to add a medication: scan the bottle's barcode (name, strength in mg and pills per bottle from the FDA's public NDC Directory), take a photo of it, or type it in manually.

Everything is in Spanish by default, with English one tap away.

How we built it

  • React 19 + Vite for the interface, with a design system built around older adults: large touch targets, a high-contrast palette, the Atkinson Hyperlegible font, and a friendly coquí mascot.
  • Real-time sync between the two screens: BroadcastChannel + localStorage for a no-account demo, and optional Supabase (Postgres + Realtime) to sync across devices.
  • Computer vision in the browser: Google MediaPipe (21-point hand tracking) and YOLO11n running in ONNX Runtime Web to detect the bottle. No video ever leaves the device.
  • Barcode scanning with ZXing-C++ compiled to WebAssembly, plus image cropping and sharpening so blurry laptop-camera frames can still be read.
  • The openFDA NDC API to turn a barcode into a medication name, strength and package size.

Challenges we ran into

  • Alarms that judges can test at any time. Real schedules don't fit a 3-minute demo, so I built a test panel that rings any dose instantly without changing its scheduled time.
  • Blurry webcams. Laptop cameras can't focus up close. The first barcode reader failed on blurry frames, so I switched to a WebAssembly decoder and added crop, enlarge and sharpen steps.
  • Pharmacy labels. I learned that most pharmacy labels encode the prescription number, not the medication, so they aren't in any public database. Only manufacturer and over-the-counter barcodes map to the FDA list. For the demo, when a bottle can't be identified, the prototype registers a sample medication. This is simulated, and it's documented as such.
  • Designing for a 98-year-old. Every screen had to work with one tap, large text, and no menus, and still give the caregiver enough detail.
  • Browser limits. I fixed a bug where the "Activar" button hung waiting for a notification permission prompt, and I made the app ignore browser auto-translate, which was breaking it.

Accomplishments that we're proud of

  • A working end-to-end prototype, built solo during the hackathon.
  • On-device computer vision for privacy: no images are uploaded.
  • An honest design: Cuidao never makes medical decisions, uses only synthetic data, and its limitations are written down in the README.

What we learned

  • How to run machine-learning models (MediaPipe, YOLO) directly in the browser with WebAssembly.
  • How NDC codes, UPC barcodes and the FDA's public data work, and their real-world limits.
  • That accessibility for older adults is about removing choices, not adding features.
  • That responsible design means telling users and judges exactly what the app does and doesn't do.

What's next for Cuidao

  • Alerts when the app is closed (web push, SMS or WhatsApp) and secure accounts that link one older adult with several family members. -A cus
  • Testing with older adults in Puerto Rico, starting with my grandmother.
  • A detection model trained on real pill bottles, and weekly adherence reports to share with the doctor.

  • Working with community pharmacies so medications can be added when they're picked up.

    What's next for Cuidao

  • A personalized alarm voice: the main caregiver records a short message, like "Mami, es hora de tus pastillas", and it plays instead of the standard alarm. Hearing a familiar voice makes the reminder more personal and easier to recognize, and each family can choose whether to use it.

  • Alerts when the app is closed (web push, SMS or WhatsApp) and secure accounts that link one older adult with several family members.

  • Testing with older adults in Puerto Rico, starting with my grandmother.

  • A detection model trained on real pill bottles, and weekly adherence reports to share with the doctor.

  • Working with community pharmacies so medications can be added when they're picked up.

Built With

Share this project:

Updates

Submission history