Despite their best efforts, doctors sometimes make mistakes. They aren’t to blame - a hospital is one of the most chaotic and high-pressure work environments possible. These problems are exacerbated by frequent shift changes and an inability to communicate without sacrificing valuable time tending to patients.

Vyapse aims to solve these problems by reducing the overhead of communication between doctors in an organization, as well as allowing doctors to engage with patients in a more personal manner.

In addition, Vyapse uses facial recognition technology to add another layer of safety to the positive patient identification process, reducing the odds of damaging and costly patient misidentification errors.

What it does

Vyapse uses facial recognition and a wearable AR device to give medical care providers a concise summary of need-to-know patient information. Due to the time constraints present while developing the application, we were only able to partially implement additional displays such as allergies, fall risk status, missed medical doses, and live vital signs. In addition, it uses speech recognition technology to reduce time spent typing notes - letting doctors spend more of their time focusing on their patients.

The app will also provide ways to access up-to-date patient data from a variety of sources, such as on-site labs, imaging facilities, and other on-site medical staff. These features were also only partially implemented due to time constraints.

How we built it

Vyapse was developed using a variety of platforms and frameworks.

On the frontend, we opted to simply use HTML5/CSS/JS as it was what our team was most proficient with.

On the backend, we used a Flask running on an EC2 server to host microservices written in Python. Facial recognition was made possible through the face_recognition framework. Websockets were used for the live sensor data feed.

Challenges we ran into

We ran into plenty of challenges while developing Vyapse. Our original vision for the app involved the integration of multiple different services, the complexity and overhead of which we underestimated.

One challenge in particular that was definitely worth the time spent to overcome it was securing our systems. While we could have saved time by not setting up HTTPS, we felt that a service that processes patient data should be well-secured.

Finally, we had initially planned on implementing our frontend as an Android application. We came to the realization (at around two in the morning!) that we didn’t have the skills to create a user-friendly Android application, and therefore we had to rewrite a substantial part of the codebase to catch up.

Accomplishments that we're proud of

We think our application is exceptional in terms of user friendliness - while most applications in the health tech space can be cumbersome to use, we think that this is an application that users will enjoy interacting with.

We’re also proud of our facial recognition system, which has been able to distinguish between registered users with minimal error.

Finally, we’re proud of all of the features that we only half-implemented. For example, we had sensors streaming live data to our server, but were unable to integrate that data with our frontend.

What we learned

We learned a lot about the overhead created by integrating multiple APIs into one product, no matter how simple it may initially seem. We also learned a lot about hosting microservices in the cloud, and about UX design.

What's next for Vyapse

Vyapse can easily be integrated into an AR platform (such as HoloLens) to further increase doctor-patient engagement.

Share this project: