The idea to create Doctor Pepper came from the two amazing doctors we have on our team, Dr. Kate Gofman and Dr. David Moshal. Dr. Gofman is a Cardiologist from Moscow, Russia and Dr. Moshal is an Anesthesiologist with an MD in Biomedical Engineering who also serves as a true full-stack software engineer. Dr. Moshal is from South Africa and currently resides and teaches K-12 student Computer Science in Marin County.
As a team, we wanted to find a real world application for Pepper that would address a need we were not only familiar with but that we felt was underserved.
What it does
Doctor Pepper is the first real AI-powered robot that can accurately diagnose specific symptoms in a patient. Doctor Pepper has the ability to detect human heart rate using her on-board camera to detect changes in the red and green spectrum of the human face which have been proven to be accurate indicators of heart rate pulses. She can also use optical image processing to scan and recognize visible skin conditions including but not limited to chicken pocks, ring worms, bruises and pink eye. In later stages Doctor Pepper will also be able to scan for body temperature and detect fevers using her on-board infrared sensors. With the addition of simple accessories like headphones, Doctor Pepper will also be able to run hearing test to check for hearing loss or equilibrium problems.
How We built it
Doctor Pepper was built on top of the existing Pepper Robot platform provided by SoftBank Robotics along with APIs from IBM Watson, Verizon's ThinkSpace, Ziggeo Video Platform, and Radix Domain.
Challenges We ran into
To make Doctor Pepper the an amazing medical assistant we had to find and/or create the appropriate algorithms to be able to analyze the data we were able to capture. We also ran into issues loading in the Watson libraries into Pepper’s system but we were eventually able to achieve this using our own server. Of course one of our biggest challenges was working with Pepper’s existing systems and getting her to acknowledge the appropriate person and stay still so she would be able to record clean videos for proper diagnosis.
Accomplishments that We are proud of
We are very proud to have been able to complete this entire project. We truly believe it can help to provide assistance to school nurses around the world.
What's next for Doctor Pepper
In later stages Doctor Pepper will also be able to scan for body temperature and detect fevers using her on-board infrared sensors. With the addition of simple accessories like headphones, Doctor Pepper will also be able to run hearing test to check for hearing loss or equilibrium problems.