Ocular

Inspiration:

What inspired the team to create this project, was the fact that many people around the world are misdiagnosed with different types of ocular diseases, which leads to patients getting improper treatments and ultimately leading to visual impairment (blindness). With the help of Theia, we can properly diagnose patients and find out whether or not they have any sort of ocular diseases. This will also help reduce the conflicts of incorrectly diagnosed patients around the world. Our eyes are an important asset to us human beings, and with the help of Theia, we can help many individuals around the world protect their eyes and have a clear vision of the world around them. Additionally, with the rise of COVID-19, leaving the house is very difficult due to the government restrictions. We wanted to reduce constant trips between optometrists and ophthalmologists with Theia due to the diagnosis being performed at the optometrists’ eye clinic, leading to fewer people in buildings and fewer gatherings.

What it does:

Theia can analyze a fundus photograph of a patient’s eye, to see if they have an ocular disease with extremely high accuracy. By uploading a picture, Theia will be able to tell if the patient is Normal or has the following diseases: Diabetic Retinthropy, glaucoma, Cataract, Age-related Macular Degeneration, Hypertension, Pathological Myopia, or Other diseases/abnormalities. Theia then returns a bar graph with the different values that the model has predicted for the image. You can then hover over the graph to see all the prediction percentages that the model returned for each image, therefore the highest value would be the condition that the patient has. Theia will allow medical practitioners to get a second opinion on a patient's condition and decide if the patient needs further evaluation rather than sending the patient to the ophthalmologist for diagnosis if they have a concern. It also allows new optometrists to guide their patients and not be unsighted to the diseases shown in the fundus photos.

How we built it:

Theia is a tool created for optometrists to identify ocular diseases directly through a web application. So how does it work? Theia’s backend framework is designed using Flask and the front end was created using plain HTML, CSS, and JavaScript. The computer vision solution was created using TensorFlow and was exported as a TensorFlow JS file to use on the browser. When an image is uploaded to Theia, the image is converted into 224 by 224 tensor matrix. When the predict button is clicked the TensorFlow model is called with its weights, and a javascript prediction promise is returned. Which is then fetched and returned to the user in a visual bar graph format.

Challenges we ran into:

We tried to create a REST API for our model by deploying the exported TensorFlow model on google cloud. But Google has a recent user interface issue when it comes to deploying models on the cloud. So we instead had to export our TensorFlow model as a TensorFlow JS file. But why would this be a problem? Because it affects our client-side performance by predicting on the client-side. If the prediction were made on the server and returned to the client it would’ve improved the performance of the web application. We also ran into other challenges when it comes to working with promises in javascript since our team had two people that were beginners and we weren’t very experienced in working with javascript.

Accomplishments that we're proud of:

We are proud of making such an accurate model in TensorFlow. We are not very experienced with deep learning and TensorFlow, so getting a machine learning model that is accurate is a big accomplishment for us. We are also proud that we created an end to end ML solution that can help people see the world in front of them clearly. With two completely new hackers on our team, we were able to expand on our skills while still teaching the beginners something new. Using Flask as our backend was new to us and learning how to integrate it into the web app and ultimately make it work was a major accomplishment but the most important thing we learned was collaboration and how to split work among group members that may be new to this world of programming and making them feel welcomed.

What we learned:

It’s surprising how much can be learned and taught in only 48 hours. We learned how to use Flask as our backend framework which was an amazing experience since we didn’t run into too many problems. We also learned how to work with javascript and how to make a complex computer vision model with TensorFlow. Our team also learned how to use TensorFlow JS as well which means that in the future we can use TensorFlow JS to make more web-based machine learning solutions.

What's next for Theia:

We provision Theia to be more scalable and reliable. We aim to deploy the model on a cloud service like google cloud or AWS and access the model from the cloud which would ultimately increase the client-side performance. We also plan on making a database of all the images the users upload, and passing those images through a data pipeline for preprocessing the images, and then saving those images from the user into a dataset for the model to train on weekly. This allows the model to be up to date and also constantly improves the accuracy of the model and reduces the bias due to the large variety of unique fundus photos of patients. Expanding the use case of Theia to other ocular diseases like Strabismus, Amblyopia, and Keratoconus is another goal which means feeding more inputs to our neural network and making it more complex.

Built With

• css3
• flask
• html5
• javascript
• tensorflow
• tensorflowjs