NOTE: Video demo also accessible at https://topdocmedicine.com/demo.

Inspiration

Malaria kills one child every 30 seconds, and over one million people die from malaria each year. It is important to note, however, that malaria can be treated if diagnosed in the appropriate timeframe. These deaths are a byproduct of disparities in the social determinants of health. Patients in under-resourced areas such as Sub-Saharan Africa and Southeast Asia often have little access to hospitals and microscopes that enable diagnosis, and even in areas with adequate medical facilities, many individuals do not have the financial resources to capitalize on those facilities. Moreover, there is currently a global shortage of 4.3 million health professionals, leading to overbooking and inadequate care for many patients, especially those in lower socioeconomic brackets. Doctors also need a faster way to diagnose diseases. To help patients and physicians alike, we set out to create Topdoc.

Topdoc uniquely integrates solutions to all of these problems, as our malaria algorithm achieves much higher accuracy than our competitors do and is the first service to apply this algorithm in a setting where it can be utilized the most. We empower patients to actively engage in their own diagnoses without having to enter a hospital and provide them with the knowledge and resources to reach out to physicians at the right time. This is especially useful for two customer channels: people living in rural or underprivileged areas where access to hospitals or microscopes is scarce, and—in these unprecedented times—patients who wish to understand their current condition but are unable or unwilling to visit a hospital due to the COVID pandemic and doctors’ extremely busy schedules.

Through Topdoc, we hope to use artificial intelligence, machine learning, and technology in general as a means of transforming healthcare and improving lives. With the rising popularity of resource stewardship and inexpensive treatment, the field of medicine has recently become a prominent AI vertical, and we hope to expand on current efforts to autonomously and intelligently facilitate diagnosis and treatment as well as health literacy and patient education.

What it does

Topdoc is a web application aimed at revolutionizing the fields of smart diagnosis, telemedicine, and healthcare in general.

Topdoc is the first service to provide fully automated diagnosis completely at home for patients with little to no medical background. Take patient Jane Doe. After obtaining a blood sample with a vaculet or similar tool, she can use a handheld microscope, such as the $1 Foldscope invented by Stanford’s Dr. Manu Prakash, to take a microscopic image that she can then upload into our malaria diagnosis convolutional neural network. Our CNN (which achieves a remarkably high accuracy of 98%) then provides her with a diagnosis that often offers higher precision than a doctor's diagnosis. If she tests positive for malaria, she can go to the patient forum and post her diagnosis for doctors and other patients to view, enabling crowdsourced patient education and treatment. She can then scroll through a list of nearby doctors registered with Topdoc and contact them to ask for virtual treatment.

Topdoc also aims to improve current telemedicine practices and foster online patient education by assisting doctors in providing virtual diagnoses and treatment to a wider array of patients. While our CNN provides consistently reliable and accurate diagnoses (often more accurate than doctors’ diagnoses), these diagnoses primarily serve as first opinions that doctors can then confirm. In addition, doctors can upload a 2D image of their patient's infected cell to our deep neural network, which converts that 2D image into a 3D model and then displays it in augmented reality using the echoAR API. By viewing 3D models of their patient’s cells in AR and also panning through the model rendered directly on our website via Sketchfab, doctors can better understand their patients’ condition and how to address them; more importantly, however, they can then use these models to explain their treatment plans to their patients. Through these educational tools, we also hope to improve health literacy, which is an aspect of care that physicians and researchers have proven to parameterize a patient’s adherence to treatment.

How we built it

After a few hours of wireframing, conceptualizing key features, and assigning tasks, we split up into 1 frontend/AI developer and 1 backend/ML/AR developer and started working.  Ayaan worked on the website’s UI while developing and training the CNN for malaria diagnosis, while Vignav developed the DNN that converted 2D cell images to 3D models visible via the echoAR platform while also constructing the backend Flask server infrastructure.

Challenges we ran into

Our biggest challenge was getting the 2D cell-level image to render into 3D. We were able to resolve this problem by combining past model conceptualization research with our Pytorch code to create a 3D canvas model and then render that within our website using Sketchfab as well as in AR using echoAR’s API. Converting our PCD file to an OBJ file and retaining/rendering the color was also a really big challenge as this was technology we weren't familiar with. We are currently displaying a single cell model generated by our DNN, and are currently in the process of integrating our DNN with our web server to enable different 3D cell models (we only partially finished this feature due to the time constraint present in the hackathon).

Accomplishments that we're proud of

We're proud of the unique way in which we were able to combine and build off of research from various sources and fuse them into one final product allowing users to walk through the entire cycle of automated diagnosis, cell-level 3D visualization, and initial treatment. It was really exciting to see all of our features come together and for us to successfully transform something that has so far been primarily an untested, theoretical concept into an innovative working product. We're really proud of our dedication to the project and determination to tackle this diagnosis problem with a revolutionary solution.

What we learned

We entered Silicon Valley Hacks with little to no experience in 3D modeling and augmented reality. By working with 3D modeling software and learning how to code an effective DNN to render 3D images is an experience that we can carry on for future projects. Furthermore, we learned how to effectively tune and deploy ML algorithms in new ways, such as using pruning algorithms to compress model weights. Working with vast amounts of AI/ML training data, 3D models, and microscopic cell images made this project really fun!

What's next for Topdoc

Our ultimate goal for Topdoc is to distribute our malaria diagnosis algorithm. We hope to build and provide kits to individuals with little access to medical facilities, such as members of homeless respite centers or residents of rural, under-resourced regions where equipment such as microscopes are scarce. We plan to include vaculet blood collection sets for patients to obtain blood samples as well as cheap handheld/smartphone microscopes such as the Foldscope, which Topdoc is compatible with.

Most importantly, Topdoc’s diagnosis and 3D modeling solutions extend far beyond malaria detection. While our work is disruptive in fully automating the diagnosis of parasitic, cell-level diseases, our technologies can be applied to various other maladies. We aim to develop AI/ML algorithms to diagnose skin cancer, glaucoma, brain tumors, and more. We believe that strengthening doctor-patient relationships, especially in virtual spaces, and empowering patients to take charge of their own health is the future of medicine.

Discord Usernames

Vignav Ramesh - rvignav#8122

Ayaan Haque - Ayaan#0434

Share this project:

Updates