Inspiration
SightSense was inspired by the need to create an accessible and efficient tool for detecting diabetic retinopathy (DR) from retinal images. Diabetic retinopathy is a leading cause of blindness, and early detection can significantly reduce the risk of permanent vision loss. Our goal was to use modern machine learning techniques to provide a solution that is both easy to use and reliable, helping healthcare professionals and patients diagnose DR more effectively.
What it does
SightSense is a web and mobile application powered by artificial intelligence (AI) to detect diabetic retinopathy (DR) from retinal images. Users can upload images through a user-friendly interface, and the AI model will analyze them and provide a diagnosis, categorizing the level of DR from 0 (No DR) to 4 (Proliferative DR). The app aims to deliver real-time, accurate DR detection to both healthcare professionals and patients, making diabetic eye care more accessible.
How we built it
I built the core of SightSense with a deep learning model (DenseNet-121) trained on the APTOS 2019 Blindness Detection dataset. The model was fine-tuned to classify retinal images into five categories based on the severity of diabetic retinopathy.
Backend: The backend was developed using Flask, providing an API that handles image uploads, model inference, and sends back predictions. The model was trained using PyTorch, with predictions returned in JSON format.
Frontend: The frontend is a mobile app built with Expo (React Native) to ensure cross-platform compatibility. The app allows users to take or upload retinal images and display the results in a clean, user-friendly interface. The camera and upload functionalities were integrated with a modal popup showing the results immediately after analysis.
Deployment: The Flask API was deployed on a local server, with the Expo app communicating with the API to send images for predictions. I also implemented CORS (Cross-Origin Resource Sharing) for smoother communication between the frontend and backend.
Challenges we ran into
Initially, the model had lower accuracy than expected, requiring multiple iterations of tuning and data augmentation to improve its performance. I faced challenges when integrating the Flask backend with the mobile frontend. Debugging and ensuring that images were sent properly and that the Flask API handled them correctly took some time.
Accomplishments that we're proud of
Working Machine Learning Model: Despite initial challenges, we successfully trained a model that performs reliably for DR detection, achieving good accuracy on the APTOS 2019 dataset.
Full-stack Application: I built a fully functional app that combines machine learning, mobile development, and API integration, providing a complete solution for DR detection. I designed a smooth user experience for the mobile app that makes it simple for users to interact with the system, take or upload images, and receive instant diagnoses.
What we learned
Model Tuning: Deep learning models like DenseNet require careful tuning, especially for specific applications like medical image analysis. It was important to handle various image conditions to improve the model’s robustness.
Integration and Deployment: I learned a lot about deploying a model into a production environment and integrating it with a mobile app and overcoming technical issues like communication between the frontend and backend.
User-Centered Design: Building an app that’s simple, fast, and intuitive is crucial. The user interface should always prioritize ease of use, especially in healthcare applications, where users may not be tech-savvy
What's next for SightSense
Improving the Model: I plan to retrain the model with a more diverse dataset to increase accuracy and robustness across different populations and lighting conditions.
Expanding Features: In the future, I want to add more advanced features such as patient data tracking, appointment scheduling, and recommendations for next steps based on the diagnosis.
Wider Deployment: After refining the app and model, we aim to deploy SightSense on a larger scale, making it available in more regions, particularly in underserved areas where access to eye care may be limited.
Collaboration with Healthcare Providers: I plan to collaborate with healthcare providers to fine-tune the system for real-world use, making SightSense a valuable tool for doctors and clinics.
Log in or sign up for Devpost to join the conversation.