Cuidadora+

A simulated Alexa+ that lets Spanish-speaking caregivers talk in Spanish and hand off in English: meds, vitals with safe alerts, voice notes and English shift reports.

Comment

Inspiration

I'm Ileana, a Puerto Rican LPN who works as a home health caregiver in Winter Haven, Florida. Spanish is my first language, and my English is limited. Many of my patients and their families speak Spanish. But the agency, the supervising nurses and the paperwork are all in English. I once had to ask my agency to tell clients that my English is limited.

At the end of a long visit, writing a good shift note in English is slow and hard, so important details get lost: how much the patient ate, whether she complained of pain, whether a dose was really given. When a blood pressure reading looks scary, you don't want to be searching for the right English words before you call the nurse. Thousands of caregivers in the U.S. live this every day. I wanted an assistant that speaks our language and still gives the agency what it needs.

What it does

Cuidadora+ is a simulated Alexa+ voice assistant for caregivers. You talk to it in Spanish (or English), and it acts across several tools during the shift:

  • Medication schedule: "Alexa, ¿qué medicinas le tocan a Doña Carmen a las 8?" It reads back the 8 a.m. doses with instructions.
  • Dose tracking: "Marca la metformina como dada." It remembers which time you meant, records it, tells you what's still pending, and warns you if the dose was already given.
  • Voice visit notes to English report: You dictate a note naturally in Spanish. It structures the note into English fields (meals, fluids, mood, activity, pain, falls) and keeps your original words next to it.
  • Vitals with safe-range alerts: "La presión está en 182 sobre 110." It tells you in Spanish to call the supervising nurse now, shows the number in big type, and says when to call 911. It never diagnoses.
  • Family updates: It drafts a warm message for the family in Spanish and English. It's only a draft and is never sent without your approval.
  • Close the shift: "Alexa, cierra el turno" chains two tools: the English shift report for the agency and nurse, and the family message.

The screen works like an Alexa+ device: voice chat on one side, visual cards (Patient, Meds, Vitals, Alerts, Note, Report, Family) on the other, and an agent trace showing the plan and each tool call. State is saved for the session in localStorage. It works with a microphone (Web Speech API) or by typing or tapping sample prompts.

How we built it

  • Plain HTML, CSS and JavaScript with no framework, no build step and no dependencies. It runs by opening index.html or with npx serve.
  • Web Speech API: SpeechRecognition in es-US (switchable to en-US) for input, and speechSynthesis with a Spanish voice for replies, with a typed fallback.
  • A simulated Alexa+ agent loop: understand, plan, call tools, respond. The offline planner is a bilingual rule-based router (language detection, patient, time, medication and vital-sign slot filling). It turns an utterance into an ordered list of tool calls.
  • Eight tools with JSON-schema descriptions (get_med_schedule, mark_dose_given, log_vitals, add_visit_note, generate_shift_report, draft_family_message and more). They export in Amazon Bedrock Converse toolSpec format, so an LLM can plan with the same tools. An optional adapter plus a small Node proxy for Bedrock is included as a reference. The demo runs fully offline with no API keys.
  • Safety in code, not in the model: vital-sign thresholds and the escalation message are deterministic. Family messages are drafts only.
  • A bilingual bridge: rule-based extraction maps Spanish food words, amounts, mood words, body parts, falls and negations ("no hubo caídas") into English report lines. Anything it can't classify is flagged as untranslated, and the original Spanish is attached.
  • A Node smoke test runs the full demo flow without a browser. The demo video was generated from the real app with Playwright screenshots, neural text-to-speech and ffmpeg.

Challenges we ran into

  • Spanish speech recognition is uneven. SpeechRecognition only works in Chromium browsers, needs localhost/https and mic permission, and is missing in Firefox. That's why typed input and sample prompts are first-class features.
  • Translating without an API. We wanted the demo to work offline, with no keys and no patient data leaving the device. So instead of machine translation we built targeted extraction for the things a shift report needs, and always keep the original Spanish so nothing is silently lost.
  • Being helpful without giving medical advice. We had to word alerts carefully: say whom to call and when, but never what the condition might be.
  • Time pressure: we built the whole project inside the submission window.

Accomplishments that we're proud of

  • A caregiver can go from a spoken Spanish note to a clean English shift report in one sentence: "Alexa, cierra el turno."
  • The agent is easy to inspect: the trace shows every plan and tool call, so a nurse or agency can trust what happened.
  • It runs anywhere with zero setup, including with no internet, no API keys, and no microphone.
  • Large, high-contrast, bilingual design made for tired eyes at the end of a shift.

What we learned

  • A good voice assistant for caregivers depends on context (who, what time, already given?) more than on fancy language.
  • Keeping safety rules deterministic and letting a model (optionally) handle planning is a good split for health-adjacent tools.
  • Bilingual output matters both ways: Spanish for the caregiver and family, English for the agency.

What's next for Cuidadora+

  • Run it as a real Alexa+ experience, exposing the same tools through the Alexa+ / MCP integration path.
  • Plug in Amazon Bedrock for open-ended notes and more natural translation, with the deterministic safety layer kept.
  • Agency integration: send the approved report to the agency's EVV / documentation system.
  • More languages (Haitian Creole, Portuguese) and real medication-reminder notifications.
  • Pilot with caregivers at a Florida home health agency, with proper privacy review (HIPAA) before any real patient data.

Disclaimer

Hackathon prototype, not medical advice. All patients and phone numbers are fictional demo data; no real PHI. Voices in the demo video are synthetic text-to-speech.

Built With

Share this project:

Updates

Submission history