Inspiration

Our team member, Chris, recently went for a checkup after experiencing constant itching and dryness with his eyes. Upon going to the Optometrist and undergoing basic tests, he was told he would have to see an Ophthalmologist for further testing due to preliminary symptoms of Glaucoma. Thankfully, upon further testing, it was determined that there were no issues and supplementary medication was provided.

What it does

ICheck is a web application that allows users to upload an image of their eye or take a picture using their device's camera.

Once an image is submitted, our application processes it through a machine learning model trained to classify the eye images into six categories (5 diseases and a healthy):

-Cataract: a cloudy or foggy area that develops in the normally clear natural lens of the eye, blocking light from reaching the retina and causing vision to become hazy or blurred.

-Bulging eyes (medically known as exophthalmos or proptosis): occurs when one or both eyeballs push forward out of their normal position in the eye socket.

-Conjunctivitis (commonly known as pink eye): is an inflammation or infection of the conjunctiva—the thin, transparent membrane that covers the white part of your eye and lines the inside of your eyelid.

-Jaundice (also known as icterus): is the yellowish or greenish pigmentation of the skin, mucous membranes, and the whites of the eyes (sclera) caused by a high level of bilirubin in the blood.

-Uveitis: is an inflammation of the uvea, the middle layer of tissue in the eyeball that contains blood vessels nourishing the eye

After this is done, our application proceeds to recommend the user to Optometrists and Ophthalmologist nearby.

How we built it

ICheck consists of three main components: a machine learning model, a FastAPI backend, and a web-based frontend.

Machine Learning — Python, FastAI, and ResNet18

This was the most challenging part of the project as our group had not worked on training neural networks previously.

We used Python, FastAI, and a ResNet18 convolutional neural network to build an image-classification model capable of distinguishing between our five supported categories. We used ResNet18 architecture rather than building a neural network entirely from scratch due to time constraints and adapted this to our classification task.

Our machine learning workflow involved preparing eye-image datasets, organizing images by category, preprocessing the data, training the model, and evaluating its performance.

We organized the processed dataset into three subsets:

Train set: Used to train the model.

Valid set: Used to monitor performance during development.

Test set: Used to evaluate the trained model on held-out images.

After training, we exported the model "eye_model.pkl", allowing our backend to load it and perform predictions without retraining datasets.

During evaluation, our model achieved approximately 85.5% classification accuracy.

While this was an encouraging result for our prototype, we recognize that performance on a relatively small dataset does not establish clinical reliability. More diverse data, rigorous evaluation, and clinical validation would be required before considering any real-world medical use.

Backend — FastAPI

We built our backend using FastAPI, a Python framework that allowed us to connect our machine learning model to the web application.

The backend serves as the bridge between the user's uploaded image and the classification model.

When a user submits an image, the backend:

-Receives the image through the /api/v1/analyze endpoint.

-Validates the uploaded file and checks basic image quality.

-Handles supported image formats and image orientation.

-Passes the image to our trained machine learning model.

-Retrieves the predicted category and classification scores.

-Associates the prediction with educational recommendations.

-Returns the results to the frontend as a structured JSON response.

We also designed the backend to load the trained model once when the API starts, avoiding the overhead of repeatedly loading it for each request.

Uploaded images are processed in memory rather than being permanently saved by the backend to ensure user discretion for photos.

Frontend — HTML, CSS, and JavaScript

For our frontend, we used HTML, CSS, and JavaScript to create an interface that makes the application straightforward to use. Our intended goal was to ensure our web application was simple and intuitive to use.

The frontend supports image uploads through file selection or drag-and-drop, as well as taking pictures using a device camera.

Once the image has been analyzed, the results are displayed alongside the predicted category, confidence score, and educational recommendations.

We also connected the frontend to our backend API, allowing users to interact with the machine learning model through a browser without needing to understand the underlying technology.

Nearby Eye-Care Provider Search

This was an added feature. We used OpenStreetMap data to retrieve information about eye-care providers near the user's location.

With the user's permission, the application accesses their approximate location through the browser and displays nearby providers on an interactive map.

Users can select a provider to view location information and open directions through Google Maps.

We felt this was an important addition because identifying a possible concern is only one part of the process. Helping users connect with professional care makes the application more practical and meaningful.

Challenges we ran into

Finding and Preparing Suitable Training Data

We had no prior experience of training a model so this felt like a worthwhile challenge for us to attempt. There is not much readily available data for eye diseases that can be detected with a camera so we had to spend a decent amount of time finding valid datasets that met our requirements. Our Sources for this included:

Building and Evaluating the Model

Like noted earlier, we had no prior experience training a model. Therefore we had to spent time watching some Youtube videos and consulting various sources of genAI for knowledge on what the best methods would be for us to structure the training of our Model.

Evaluating the model showed us why accuracy alone doesn't tell the entire story for model training. Even when a model performs reasonably well overall, it can still make incorrect predictions, sometimes with high confidence and this was a scenario we experienced a ton. This reinforced the importance of clearly communicating the limitations of our application and ensuring users understand that the results are experimental.

Image Validation and Computer Vision

Given our limited hackathon timeline, we made the decision to prioritize basic image-quality validation and a working end-to-end application rather than spending too much time on a feature that wasn't essential to our initial prototype and our goals for this initial prototype.

This was an important lesson in prioritization as this is essential in time constrained scenarios like Hackathons.

Working Under Time Constraints

Like in any hackathon, this was always going to be an issue. We considered several improvements, including more advanced image validation and further model optimization but ultimately we prioritized a working image-classification pipeline, educational recommendations, and nearby provider discovery.

Accomplishments that we're proud of

Our biggest accomplishment was successfully building an end-to-end application that brings together multiple technologies within a limited timeframe.

More than anything, we're proud that we turned an initial idea into a working application that demonstrates how machine learning and web development can be combined to explore a meaningful real-world problem in the limited amount of time we had.

What we learned

Building ICheck taught us much more than how to train a machine learning model. It taught us about: -Machine learning and computer vision -Full-stack integration -API development -Responsible AI development: Perhaps one of our most important takeaways was understanding the responsibility that comes with building technology related to healthcare and outlining that professionals always need to be consulted for healthcare decisions.

Even a promising machine learning model can make mistakes. We learned the importance of communicating uncertainty, avoiding misleading medical claims, and designing our application to encourage professional medical assessment rather than replace it.

What's next for ICheck

Although we accomplished our main goals during the hackathon, there are several ways we hope to improve ICheck in the future.

Our main priority is improving the model's accuracy and reliability by expanding our dataset with more diverse eye images and introducing better evaluation methods. We also plan to implement more advanced image validation to ensure uploaded pictures contain a human eye, as well as expand the number of eye conditions our model can recognize. Ultimately, we hope to collaborate with healthcare professionals to further evaluate and validate ICheck, making it a more reliable and accessible educational tool for eye-health awareness.