CURATOZA

Share and connect to save one more life.

Comment

Inspiration

It is estimated that nearly 100,000 people die each year because of mistakes that their hospital makes. This amounts to approximately 12 million Americans who will suffer a diagnostic error in a single year. One of the team member's close relatives passed away this summer due to this exact reason. The family had to go through so much pain and suffering and they were put into a very tough position where it was indeed the doctors’ incompetence which contributed to the death, yet they understood that the doctors simply had limited resources and information on how to treat the person. The family had to make numerous phone calls from one country to another in order to connect doctors with limited resources to those who had resources but were unable to see the patient and, therefore, could not help. This happens more frequently in under-developed countries, nevertheless, there are such cases even in the US.

What it does

We created an android app that allows doctors to input various results of tests by simply taking a picture of them and uploading them into an app, which will automatically recognize the health deviations and bring up a list of all possible diagnoses ranked by their probability. In addition, for each disease, profiles of the patients who had the same disease in the past will be attached along with the contact information of the physicians who treated them. The patient’s personal information such as name and contact details will be hidden. If needed, each doctor who has concerns about a diagnosis can get in touch with other doctors who may have already treated the disease. In the chat with other doctors, there will be a 3D model of the patient’s diseased part of the body available in order to create a deeper understanding of each individual case.

How we built it

To start, we decided to use Hololens and an android website/app connected to a cloud service which would implement machine learning. With Hololens, we used the Unity tools available and tried to build upon those. We searched for ways to render MRI images in Unity since developing such a thing in this short time span was impossible. Next, we used one of the public open source projects as the base. Although we could have added more features to the hololens, because of the instability of the system we decided to only have a proof of concept 3D MRI model there instead. On the website, we decided to make it simple and just used an online website builder. We also searched for MRI scans of sick patients and healthy patients to make a simple categorization ML algorithm and managed to make it work better than pure chance.

Challenges we ran into

There were several challenges on our way here. Making the Hololens work at all was very challenging. Because it is a new platform, there are literal code sections with comments such as “use this workaround as long as these assumptions hold” but we somehow broke the assumptions by using the MRI visualizer we found that also used a lot of workarounds. Connection was hard, building the actual app was impossible, and somehow we broke some spatial coordinate things in some of the shaders MRI visualizer was using. Still, we managed a working demo with the Hololens remote connection. Things weren’t easy on the web part either. We didn’t have anyone with experience in building a website “the quick way” so we struggled a lot. On the ML end it was hard to get MRI scan data on such short notice, as most databases worked on a “request the data and get a reply in a few days” basis. However, we did find some images to have at least proof of concept working.

Accomplishments that we're proud of

Within the time constraints of 24 hours, we as a team were able to share our own ideas and combine them into a project which can serve as an innovative opportunity to share knowledge and make more accurate diagnoses within the medical community. Each member’s skillset was diversified with the implementation of new tools such as Unity, hololens, AI, Machine learning moreover, everybody had an opportunity to expand their current knowledge in various areas as well. Overall, we are proud of making progress towards building a project which may, one day, impart the whole world and build bridges between doctors all over the world for our society.

What we learned

Through Hololens development, we learned how hard development on such cutting edge technologies can be. It was particularly hard to debug anything because there was a really good chance that we were the first people seeing these specific errors. Forum questions were nonexistent or unanswered. Thankfully we had people on Microsoft to help, so we learned how effective having mentors can be, and learned how even mentors can sometimes have no clue about projects complicated like these. With the website part, we learned that with time pressure even a simple website can be hard to build. And we learned how hard it can be to find data even on the 10s range when you run into not-so-popular territories.

What's next for CURATOZA

This application was originally planned to provide an opportunity for doctors from developing countries to have a closer insight from professionals overseas since their resources and competence level may fall short compared to innovative techniques in medicine which are now heavily integrated in practice in well-developed countries such as Japan and US. This will require the usage of Natural Language Processing in order to eliminate the language gap.

Share this project:

Updates