Living well with diabetes requires patients to pay constant attention to the food they eat and to tailor their medication regimen accordingly. While smart phone food tracking apps make this task easier than it used to be, the labor required to input data means that such apps are time consuming and don't always get used.

Being able to track carbohydrate intake and calculate insulin dosage in real time has the potential to let patients take control of their diabetes management. Both of these actions can be enabled and streamlined through the use of voice recognition technology that allows patients to seamlessly record their meals with little interaction. Our proposal is to create a voice assistant that both helps to track carbohydrate intake, and, with the assistance of a patient's health care team, recommends insulin dosing.

While verbal input and confirmation is helpful for all patients, it will be especially helpful for patients facing a complication of their diabetes, such as visual impairment or peripheral neuropathy, that can impede optimal self-management.

What it does

During the early stages of diabetes, carbohydrate tracking on its own can be sufficient for disease management. In later stages, both tracking and insulin dosing assistance are needed. Our application offers both of these services by working with a patient's existing Apple or Android smart phone, and, optionally, an Amazon Echo (Alexa) or Google Home device.

The system would work as follows:

1) When sitting down to a meal, a patient can say the system activation word (e.g., Alexa, OK Google, or our application's name when running on an Android or Apple device), and then describe the components of his or her meal. The system will use natural language processing to segment a meal description, such as, "I am eating a plate of spaghetti with a side salad and a piece of bread", into the component parts and then will look up carbohydrate load using public data sources. 2) For patients who are taking insulin ahead of meals, the patient will work with their doctor to input their dosing regimen into the smart phone application. Insulin recommendations will not be available until this has occurred. Once entered, along with carbohydrate estimation, the application will output insulin dosing for the meal based on the patient's regimen. 3) An incorrectly high carbohydrate estimate could result in a dangerously high insulin dosing recommendation, so it is of paramount importance that patients' meal descriptions are accurately understood. Thus, the system will repeat back its interpretation of the meal for verbal confirmation. For example, "I understand that you're eating a 2 oz serving of white pasta, a small salad, and a single piece of white bread. I calculate that this is 30 grams of carbohydrates. Does that sound correct?" The program would only provide an insulin recommendation after this confirmation had occurred.

Who we are

Andrew is a web application developer with 10 years of consumer software experience and a background in machine learning. He has a Masters in Computer Science from Georgia Tech.

Laura is a practicing physician. She attended UCSF School of Medicine, followed by residency training in Primary Care Internal Medicine at UCSF, and fellowship training in Hospice and Palliative Medicine at Mount Sinai School of Medicine.

Share this project: