HealthCams

An app that keeps you updated about your elder's well-being.

Comment

Inspiration

Many families cannot be in the same room as aging relatives all the time, but they still want the reassurance of a calm, continuous presence not unlike having a trusted caregiver in the home. HealthCams started from that gap: adult children and other caretakers need remote visibility into everyday health-adjacent routines (food on hand and medications taken) without adding more mental load. Built for Hack Princeton’s health track, the project also became a hands-on way to learn AWS while shipping something that feels like a real product, not just a slide deck.

What it does

HealthCams is a Caretaker Command Center with two “nanny cam” modes that run in the browser on a phone or laptop:

  1. Pantry camera — Captures periodic snapshots, compares what’s visible to a caretaker-defined inventory, and when stock looks low, creates replenishment proposals. After approval, Knot can sync a cart with a linked merchant account and drive async checkout (with webhook-driven status).

  2. Medicine camera — Compares what’s on camera to prescription schedules using multimodal understanding, and surfaces adherence signals. Important events can reach the caretaker via Photon (iMessage) in addition to the dashboard.

The caretaker dashboard ties it together: patients, cameras, inventory, proposals, and notifications—backed by Supabase and S3 for snapshots, with a web + worker split for heavy processing and integrations.

How we built it

  • Backend: A Node app (server.mjs) serves the UI and APIs; background workers handle snapshot processing and integrations (services/worker/), with PM2-style process configs for deployment.
  • Data & storage: Supabase (Postgres) models caretakers, patients, cameras, snapshots, analyses, proposals, prescriptions, and events; AWS S3 stores uploaded JPEGs with presigned URLs.
  • AI: Google Gemini (robotics-style vision prompts with structured “function call” outputs) powers pantry depletion detection and medication-scene checks (gemini-pantry.mjs, gemini-medicine.mjs).
  • Commerce: Knot shopping flows treat cart sync and checkout as asynchronous, with server-side workflow and webhooks as the source of truth for success/failure—documented and wired for realistic demos.
  • Notifications: Photon Spectrum sends iMessage-style alerts for medication-related events when configured.
  • Infrastructure: AWS scripts and a two-EC2 pattern (web + worker hosts, security groups, optional EIP, S3 bucket) support a repeatable deploy path; automated tests cover key modules (npm test).

Challenges we ran into

  • Knot is not a single “place order” button — Shopping is multi-step and async (sync cart → events → checkout → events). Designing the app around pending states and webhooks instead of instant success was essential.
  • Real-world vision is messy — Pantries and pill bottles vary wildly; the system has to gracefully skip unclear frames, log reasoning, and avoid false certainty.
  • Split architecture — Keeping the web server responsive while workers poll, call Gemini, talk to Knot, and notify via Photon added operational surface area (env per host, networking between tiers).
  • Demo vs production — Permissive RLS and rapid iteration are great for a hackathon; tightening auth and data boundaries is explicitly deferred work.

Accomplishments that we're proud of

  • An end-to-end story you can explain in one breath: camera → snapshot → analysis → proposal or alert → caretaker action → external side effects (checkout / iMessage).
  • Honest integration depth with Knot and Photon, not just mock buttons—plus internal docs that capture pitfalls and the intended async lifecycle.
  • AWS-first deployment thinking (S3, EC2 topology, scripts) alongside a simple local run (node server.mjs).
  • A tested core for critical paths (Gemini modules, Knot checkout plumbing, notifier, schema assumptions).

What we learned

  • Vision + structured outputs are powerful for household scenes, but the product layer must handle uncertainty and idempotency (duplicate snapshots, retries, partial failures).
  • Fintech-style APIs reward state machines and event logs more than synchronous request/response UI assumptions.
  • Separating web and workers early makes it easier to scale processing and isolate secrets—at the cost of more moving parts to document and deploy.

What's next for HealthCams

  • Hardening for real users: stricter auth, RLS, audit trails, and explicit consent flows for camera and messaging.
  • Richer adherence: scheduling UX, escalation rules (SMS backup), and clearer human-in-the-loop review when the model is unsure.
  • Broader fulfillment: more merchants and substitution logic when items are out of stock; optional subscriptions for staples.
  • Better device experience: dedicated mobile capture mode, offline queueing, and stronger privacy controls (on-device redaction or region-of-interest cropping where feasible).

You can paste this into Devpost as-is; tweak tone (more emotional vs more technical) or swap “HealthCams” for “Caretaker Command Center” if the submission form expects the exact on-screen product name.

Built With

Share this project:

Updates