Dr. DiGi

Dr. DiGi web interface

Inspiration

As international students, we all first experienced challenges in visiting doctors in US due to limited appointment availability and lack of transportation. These challenges were amplified when our grandparents needed to make a doctor’s visit, as they were physically more vulnerable. However, those with limited mobility, such as elderly and handicapped, are actually ones that would have to make the most visits, as they are more vulnerable to chronic diseases and require constant care.

As such, we believe that the most important factor of the healthcare system is accessibility, as symptoms and disease occur regardless of place and time. A healthcare platform or system that is able to provide the appropriate care and treatment as soon as the need arises is crucial.

Therefore, we wanted to build a conversational AI that is available to anyone, anywhere, and anytime. Simply put, we wanted to create a home doctor (or medical specialist) that can help better monitor patients with chronic diseases and answer basic questions regarding drugs, symptoms, and treatments.

What it does

With Dr. DiGi, you can virtually monitor your chronic disease conditions, ask about OTC drug informations and its interactions, get your symptoms checked, and schedule an doctor's appointment.

How we built it

On the frontend, we record user's voice and send the data stream through socket connection to the backend. Deepgram's STT transcribes the speech results, and this transcription is mapped to our rule-based chatbot. The rule-based chatbot outputs AI avatar generated from Synthesia.

Challenges we ran into

The accuracy of STT depends a lot on background noise and user pronunciation. Out of several STT services that we tried, Deepgram's STT seemed to have the highest transcription accuracy.
There were a lot of edge cases to consider, especially with the timing of audio recording.
As Dr. DiGi is a fully conversational AI, there were extremely many scenarios to consider depending on the user’s input, which we had to carefully plan out before starting development.

Accomplishments that we're proud of

Despite the challenges, we were able to create a fully functional conversational AI interface.
The STT functionality we initially implemented was fine, but we thought we could do better. With less than 24 hours left, we were able to successfully increase the accuracy of STT by approximately 40% using Deepgram’s STT API.

What we learned

During the development process, we learned that there were so many different ways to make this product more sophisticated. Integrating the system with NLP-based chatbot and realtime video generation would have made the interactions more natural. We couldn’t add all these features due to time and cost constraints but we definitely want to pursue this given the opportunity.

What's next for Dr. DiGi

Integrate with Datavant Switchboard so that authorized healthcare providers can provide 24/7 personalized virtual care for its patients through Dr. DiGi
Integrate NLP-based chatbot and realtime video generation for a more realistic interaction.
Integrate medical database to the chatbot so that Dr. DiGi can answer any questions about a medication.