CareBridge

Patient Side UI
Patient Side Data Display
Doctor Side UI
Voice Conversation UI
Benchtop Power supply
Dispenser
Breadboard and full set up

Inspiration

What it does## 💡 Inspiration

For many blind and low-vision individuals, taking medication independently is not as simple as reading a label and opening a bottle. Medication containers can feel identical, printed instructions may be inaccessible, and remembering whether a dose was already taken can create uncertainty. At the same time, family members and caregivers may want to support their loved ones without taking away their independence.

We wanted to build something that could bridge that gap.

CareBridge is an accessibility-first MedTech platform that connects medication management, AI assistance, and personal well-being in one system.

Instead of simply reminding someone that it is time to take medication, CareBridge connects software to the physical world. It can identify when medication is due, safely dispense from the correct container, communicate with the user through voice, keep an adherence history, and connect that information with a trusted doctor or family member.

We also realized that care should go beyond whether someone remembered their medication. After a successful dispense, CareBridge can optionally ask a short, non-clinical well-being check-in about mood, stress, sleep, and whether the user would like support.

Our goal is simple: use technology to help make everyday healthcare more accessible, independent, connected, and human.

⚙️ What it does

CareBridge is a voice-enabled smart medication and well-being assistant designed primarily for blind and low-vision users.

The system combines a physical ESP32-powered medication dispenser with an AI-powered software platform.

A typical interaction can look like this:

CareBridge determines that a scheduled medication is due.
The user interacts with the system through the accessible patient interface or AI assistant.
The backend verifies that dispensing is allowed.
The ESP32 activates the correct medication container and dispenses one demo pill.
The system records the event and updates the medication history.
The user can confirm whether the medication was taken.
CareBridge can optionally ask whether the user would like a quick well-being check-in.
The user can share how their mood, stress, and sleep have been and whether they would like support.
Saved information can be viewed through the linked doctor/family portal.

CareBridge also includes:

👤 Patient Portal — medication information, history, AI assistant, and well-being check-ins.
👨‍⚕️ Doctor / Family Portal — linked caregivers can review medication history, adherence, schedules, inventory, and saved well-being check-ins.
🤖 AI Assistant — conversational interaction powered by Gemini with deterministic safety controls and offline fallbacks.
💊 Smart Dispensing — an ESP32-controlled three-container prototype dispenses the selected demo pill.
🧠 Well-being Check-ins — optional check-ins for mood, stress, sleep, and requests for human support.
📊 Medication History & Analytics — tracks dispensing, missed events, confirmations, inventory, and adherence information.
📄 Care Reports — generates reports containing relevant medication and adherence information for care providers.
🔊 Accessible Interaction — voice, text, large controls, visual feedback, and fallback interaction methods.
🌐 Offline-first Design — core functionality can continue through local components when cloud services are unavailable.

Most importantly, the AI does not decide whether medication is safe to dispense. AI handles conversation and interpretation, while deterministic backend logic performs the actual authorization and hardware actions.

🛠️ How we built it

CareBridge combines AI, backend engineering, databases, embedded systems, accessibility design, and physical hardware into one end-to-end system.

We built the main application using Python and FastAPI, with SQLAlchemy managing operational data such as users, medication schedules, containers, dose events, dispensing history, and well-being records.

For conversational interaction, we integrated Google Gemini into an AI assistant while building deterministic fallbacks for critical functionality. The AI can understand user requests, but medication actions always pass through our dedicated safety layer before reaching the hardware.

Our hardware prototype uses an ESP32 connected to three medication containers. The backend communicates with the device and requests a specific container only after all dispensing conditions have been validated.

We designed the dispensing architecture around an important rule:

AI interprets. Deterministic code authorizes. Hardware executes.

Before a pill can drop, CareBridge verifies conditions such as device availability, the requested medication, schedule state, duplicate-dose prevention, inventory, cooldown rules, and hardware readiness.

For accessibility, we incorporated speech interaction with offline speech recognition and text-to-speech fallbacks, while also integrating ElevenLabs for more natural voice output when available.

We integrated Gemini, ElevenLabs, TiDB, and Snowflake as optional services while keeping the core architecture offline-first so that losing an internet connection does not automatically make the entire system unusable.

We also developed a separate well-being system that integrates with CareBridge without interfering with medication logic. After a successful dispense, the system can offer an optional check-in, store consented responses, and make saved information available to linked caregivers.

Finally, we built separate interfaces for the patient, doctor/family, kiosk, and hackathon demo experiences, allowing us to demonstrate the complete ecosystem rather than only the dispenser itself.

🚧 Challenges we ran into

One of our biggest challenges was connecting AI-driven interaction with safety-critical physical hardware.

LLMs are probabilistic, but medication dispensing cannot be. We therefore had to carefully separate AI interpretation from actual authorization and ensure that the AI could never directly control the ESP32.

Another major challenge was keeping the physical dispenser and software state synchronized. If communication fails after a command is sent, the system cannot simply assume whether a pill was dispensed. We designed the system to fail safely and flag uncertain outcomes instead of risking another dispense.

We also had to handle duplicate requests, medication schedules, inventory, hardware availability, concurrent requests, caregiver permissions, and failures across cloud integrations.

Adding the well-being system introduced another challenge: making it feel naturally connected to CareBridge without allowing mental well-being responses to influence medication dispensing. We kept these systems intentionally separated—the check-in reacts to care events but never controls them.

Finally, bringing hardware, AI, databases, voice interaction, accessibility, and multiple user interfaces together under hackathon time constraints required constant integration and testing.

🏆 Accomplishments we're proud of

We're especially proud that CareBridge became much more than a standalone hardware prototype.

We built a working end-to-end MedTech ecosystem where software decisions can result in controlled physical actions while maintaining clear safety boundaries.

Some of our biggest accomplishments include:

Building a functional ESP32-powered three-container medication dispenser.
Connecting physical dispensing to a full FastAPI backend.
Creating separate patient and doctor/family experiences.
Implementing medication scheduling and duplicate-dose prevention.
Building an AI assistant while keeping medication authorization deterministic.
Creating an optional well-being check-in system linked to the user's care history.
Supporting medication history, inventory, adherence information, and caregiver visibility.
Building graceful fallbacks so core functionality is not completely dependent on cloud AI.
Integrating AI, embedded hardware, databases, accessibility, and healthcare-focused UX into one prototype.

Most importantly, we built the system around the user's independence rather than around the technology itself.

📚 What we learned

CareBridge taught us that building accessible healthcare technology requires much more than adding AI to an existing workflow.

We learned how to design systems where probabilistic AI and deterministic safety logic coexist, allowing AI to make an interface more natural without giving it authority over critical physical actions.

We gained hands-on experience integrating ESP32 hardware, Python, FastAPI, databases, AI APIs, voice systems, and real-time device communication.

We also learned how important graceful failure is when software interacts with the physical world. Sometimes the safest response is not to guess, but to stop and ask for human review.

Most importantly, we learned to think about accessibility as a core design requirement rather than an additional feature.

🚀 What's next for CareBridge

CareBridge started as a hackathon prototype, but we see a much larger opportunity for accessible and connected healthcare technology.

Next, we want to explore:

Expanding beyond three medication containers.
Improving the physical dispenser into a smaller and more reliable enclosure.
Adding additional accessibility options for users with different needs.
Improving offline voice interaction and on-device intelligence.
Building richer adherence and longitudinal well-being analytics.
Giving healthcare professionals better ways to understand long-term trends.
Expanding caregiver communication and support workflows.
Exploring secure mobile access and remote monitoring.
Testing the experience directly with blind and low-vision users.
Investigating what would be required to move from a hackathon prototype toward a validated healthcare product.

Our long-term vision is for CareBridge to become more than a medication dispenser.

We want CareBridge to be a bridge between independent living and connected care—helping people manage everyday health while keeping the people they trust within reach.