Baymax

An emotionally intelligent medical application that utilizes Hume AI to provide a more empathetic user experience, while providing key insights to doctors.

Comment

Inspiration

Armaan grew up in a family of doctors. as his parents took him to work at the hospital, he gradually realized that a big part of medical work is simply talking to patients. Gregory and I both related to his experiences — doctors who took the time to understand us gave us the best experience and the most trust in the healthcare system. As a result, we intended to tackle the need for empathetic healthcare on demand — while doctors wouldn’t always be available, an AI chatbot would. Most current health AI bots only scratch the surface—they handle natural language but miss the deeper layers of human emotion and communication. Our app takes things to the next level, offering tools that truly get the context behind human interactions.

What it does

Baymax is a consumer-driven, patient-centered app that aims to supplement doctor visits with emotional support and mental health help by integrating directly with the doctor's notes. Borrowing from the beloved cartoon character, our front-facing app features a emotionally intelligent AI agent that has access to information from a patient's hospital visit. This AI can not only provide emotionally support through words, but also by adjusting and updating the user interface to accomodate the user's emotional state such as providing different backdrop gradients and guided breathing exercises. Furthermore, it can accurately scrape the web for factual details related to the patient such as medical side effects and common symptoms for their diagnosis.

How we built it

Baymax is a trifecta of three smaller projects. The first is a user-centric app built with react-native that uses Hume AI agent for UI updates, speech recognition, and human conversation. The second is a website dashboard built for doctors to enter and view patient information such as their mood when talking to the Hume AI agent. Finally, the server with MongoDB ties both frontends together, keeping data on each patient and allowing both to update the patient information.

Challenges we ran into

There were many. Firewall issues on the network preventing us from connecting to MongoDB. APIs breaking down. React native. DNS routing. Even our website getting blocked by Georgetown University because of "phising". Nonetheless, we made it out with a working product.

What we learned

We gained a lot of technical skills. But, I knew that that was going to happen. What I did not expect, however, was living in an Airbnb with my online-met teammates and creating a project that could impact real life patient safety. Our tight-knit team learned how to use AI copilots and got better at prompt-engineering, trudged through the endless barrage of issues that came with React native and hosting, and wrangled with rest api errors on the way. This hackathon made for a fruitful and enriching learning experience as a whole, and we had a lot of fun.

What's next for Baymax

  1. Embrace more UI-changing features that help with patient's mental health safety
  2. Taking user feedback to find out what works well with patients and implementing similar features

Built With

Share this project:

Updates