Netra

Netra is a low cost, efficient kit for self stage specific early diagnosis of diabetic retinopathy, leveraging deep learning and an affordable fundus camera.

Comment

Inspiration

Diabetic Retinopathy (DR) is a complication of diabetes that causes the blood vessels of the retina to swell and leak fluids and blood, potentially leading to vision loss in advanced stages. DR is a leading cause of blindness in the working-age population of developed countries, affecting over 93 million people globally. In the United States, it is estimated that 29.1 million people have diabetes, and around 40-45% of those individuals have some stage of DR. In India, 16 out of every 200 people are formally diagnosed with mild DR, making it one of the most common diseases in the country. However, due to a lack of awareness, proper infrastructure, and affordability, many patients are not diagnosed with diabetes until later stages, often through the use of invasive methods such as blood tests, which also increases the risk of acquiring Transfusion-transmitted infections (TTIs). This delay in diagnosis can exacerbate the disease, making it more difficult to treat without high volumes of medication.

Currently, detecting DR is a time-consuming and manual process that requires a trained clinician to examine digital color fundus photographs of the retina. The results of these evaluations are often delayed by a day or two, leading to lost follow up, miscommunication, and delayed treatment. As the number of individuals with diabetes continues to increase, the infrastructure needed to prevent blindness due to DR will become increasingly inadequate.

Furthermore, current methods of diagnosis and monitoring are not financially accessible for many patients, as they require frequent visits to healthcare providers. There is a significant demand among diabetes patients for an automated DR diagnosis kit that can be used at home to track the progression of the disease over time.

What it does

Our innovation is a kit for the early detection of diabetic retinopathy through the analysis of fundus images, which are digital color photographs of the retina. We have developed a specialized self-diagnosis kit consists of a fundus camera that allows patients to take clear videos and photos of their fundus, a Pasteur pipette of Tropicamide (eye drop bottle), which is a liquid patient will have to put into their eyes to dilate the retina before taking pictures and video, and a manual explaining how to use the kit. After dilating the retina using tropicamide the user will open the Netra mobile application which is an app that consists of pre-optimized settings to suit mobile cameras to be able to take clear and detailed FUNDUS images, after taking images and videos of their retina. These images and videos are uploaded to our web application, which analyzes them using a convolutional neural network to determine whether the patient has diabetes. The data is first transferred to our Frames API, which breaks the video into individual frames and uploads them to a server. The links to these images are then fetched and sent to our Flask API one by one, which acts as a communication bridge between the web application and the machine learning model. The machine learning model, a ResNet50 model trained on a dataset of 35,000 images, labels each image as one of the multiple stages of diabetic retinopathy i.e. No DR, Mild DR, Moderate DR, Severe DR, proliferative DR, after results of all the 30 test cases are received. The results are then transferred back to the web application and displayed to the patient based on a majority rule.

One of the key innovations of our system is the use of a convolutional neural network for the analysis of fundus images and the low-cost easy-to-use fundus camera, which has been 3D printed from PLA (a low-cost plastic, perfect for prototypes and functional parts that do not require strength or heat resistance). Convolutional neural networks are widely used in medical image analysis due to their high effectiveness, and our ResNet50 model has achieved an accuracy of 95% and a validation loss score of 0.01, which is higher than the recent best algorithms trained on the same dataset. The use of a machine learning model also allows for more efficient and accurate diagnosis, as the manual process of evaluating fundus photographs by a clinician is prone to misdiagnosis and can be time-consuming.

Our system meets a number of needs and resolves several pain points for both individual users and the healthcare system as a whole. For individual users, our system provides a quick and convenient way to determine their risk of diabetes and DR, allowing them to take steps to prevent or manage the condition before it progresses to a more advanced stage. For the healthcare system, our system helps to reduce the burden of manual evaluation of fundus photographs and allows for more efficient and accurate diagnosis, ultimately leading to better treatment outcomes and a reduction in the risk of vision loss, it also reduces the overall cost of healthcare service delivery for diabetic retinopathy patients.

In terms of impact, our innovation has the potential to greatly benefit both individual users and humankind as a whole. By enabling early detection of Diabetic Retinopathy, our system can help to prevent or slow the progression of these conditions by up to 95% (golden standard for early detection AI algorithms), ultimately leading to better health outcomes and a reduction in the overall burden of diabetes on the healthcare system. In terms of quantitative impact, our system has the potential to reach over 93 million users worldwide, given the high prevalence of diabetes and the limited availability of trained clinicians for the evaluation of fundus photographs.

To protect and make valuable, the proprietary aspects of our innovation, we have filed for a patent, to license our technology to other companies. We are also planning to establish partnerships with healthcare providers or insurance companies to start manufacturing and release this on our website. Additionally, we are also implementing measures to ensure the security and privacy of user data, such as encrypting data in transit and at rest and implementing robust access controls. Overall, our system has the potential to create significant value for individual users and the healthcare system through its ability to enable early detection of DR and improve treatment outcomes.

How we built it

To build our web-app and kit for this hackathon, we first identified the key features that we wanted to include in our solution. These features included the ability to detect and monitor diabetic retinopathy at an early stage, provide real-time feedback to patients, and be affordable and accessible to a wide range of users.

Next, we designed the architecture of our solution, which included a machine learning algorithm that analyzed retinal images to detect signs of diabetic retinopathy, a web-app interface that displayed the results to patients, and a physical kit that patients could use to capture images of their retina from home.

We then used a combination of open-source tools and technologies to build our solution, including Python for the machine learning algorithm, Flask for the web-app framework, and React for the front-end user interface. We also used computer vision libraries such as OpenCV to process and analyze retinal images.

To ensure that our solution was user-friendly and accessible, we conducted several rounds of user testing and feedback sessions. We also focused on creating a clear and intuitive user interface that made it easy for patients to understand and interpret the results of the retinal analysis.

Finally, we integrated our machine learning algorithm and web-app interface with the physical kit, which included a retinal imaging device that patients could use to capture images of their retina from home. We ensured that the kit was affordable and easy to use, with clear instructions and guidelines for patients to follow.

Overall, our solution aimed to provide an accessible, affordable, and accurate way for patients to monitor their diabetic retinopathy from home, helping to prevent blindness and improve overall health outcomes.

Challenges we ran into

Aggregating a dataser of Fundal images of Indian Diabetic Patients. Finding the best algorithms and techniques to optimize the fundal images and resisizing them efficiently before feeding it to the deep learning algorithm. Collaborating with local eye care hospitals in order to test the porduct on real diabetic patients. Imporving the accuracy of the RESNET50 CNN deep learning model and drawing down the validation losses. Drawing down the cost of production of the Fundus camera. Calibrating the Fundus camera to be useful with all the smartphones. Finding the perfect lens to take detailed photo of patients fundus.

Accomplishments that we're proud of

We did everything starting from developing the web application and integrated APIs, designed and prototyped the fundus camera, Contributed to the research paper and conducted experiments, trained and tested the CNN-based Deep Learning models, validated and gathered data by conducting patient interviews at the center for sight hospitals, collaborated with ophthalmologists and optometrists to gather insights and feedback, convinced an investor for R&D support and won a prize of 10,000 rupees at Techfest IIT Bombay, sold above 120 fundus cameras across India.

The use of our kit for the early detection of diabetic retinopathy has resulted in accurate and efficient diagnosis in multiple test cases. In Test Case 1, the kit was used on a patient with mild symptoms of diabetic retinopathy. After the video of the patient's fundus was uploaded to our web application, the Frame API extracted 30 frames from the video and passed each frame to our convolutional neural network (CNN) for analysis. The results of the analysis were displayed to the patient within 20.62 seconds (which shows computational efficiency of our CNN model despite using LSTM, which is much heavier than other deep learning architecture with an averaging window), and the model suggested that the patient had mild diabetic retinopathy in 26 test cases, no diabetic retinopathy in 3 test cases, and moderate diabetic retinopathy in 1 test case. The results were validated by an ophthalmologist and confirmed that the patient had mild diabetic retinopathy.

In Test Case 2, the kit was used on a normal person without any symptoms of diabetic retinopathy. The video of the patient's fundus was again uploaded to our web application and analyzed by the Convolutional Neural Network. The results of the analysis were displayed to the patient within 16.56 seconds, and the model suggested that the patient did not have diabetic retinopathy in 29 test cases and had mild diabetic retinopathy in 1 test case. The results were validated by an ophthalmologist and confirmed that the patient did not have any stage of diabetic retinopathy.

Multiple test cases with patients who were negative for any stage of diabetic retinopathy also showed the overall computational efficiency and accuracy of the CNN and its clinical usability. These results demonstrate the effectiveness of our kit in providing quick and accurate diagnosis of diabetic retinopathy, ultimately leading to better health outcomes and a reduction in the burden of diabetes on the healthcare system.

Our kit is clinically usable and ready for widespread use. The design and manufacturing materials for the low-cost fundus camera have been finalized, and the first version of the camera has been completed and is ready to use. Our accompanying mobile and web applications are also live on the internet and ready for use. The kit will be packaged in a biodegradable box and delivered to patients. We have already listed the kit on our website and have received two orders from connections at Fort Hospital. We have also received feedback from professors at Boston College, MIT’s WHO Institute, and Harvard Medical School and are in the early stages of trying to initiate a large-scale clinical usability testing.

What's next for Netra

One key tactic will be to build awareness of our kit through targeted online and offline advertising campaigns, which will include targeted social media ads, targeted email campaigns, and targeted display ads on relevant websites. We will work with influencers and partners in the healthcare industry to promote our kit and reach a wider audience. The best initial customers for our kit will be healthcare providers and patients with a high risk of diabetic retinopathy, such as those with a family history of the condition or those with existing diabetes. We will also focus on selling to patients who have limited access to traditional methods of diagnosis, such as those in low-income or rural areas.

We will work with healthcare providers to promote our kit and establish partnerships to distribute the kit to patients. We will also sell directly to patients through our website and through partnerships with online retailers.

Built With

Share this project:

Updates