Syntoma

Syntoma: Leveraging Advanced Transfer Learning Techniques and Convolutional Neural Networks for AI-Powered Early Diagnosis of Respiratory Diseases Deployed on a Comprehensive Website

Comment

Inspiration

The main inspiration that motivated to create this app was the COVID-19 pandemic, which left a significant impact on the world just a few years ago. COVID-19, a highly widespread respiratory disease, had pneumonia as one of the prevalent symptoms. Furthermore, as we both have family in remote areas such as India, we realized that a large number of patients were often not diagnosed, due to the extensive wait times associated with diagnosing. In fact, a large number of services associated with diagnosis are expensive, instilling another roadblock in the diagnosing process. Another motivation we shared were direct family members who were severely affected by knee injuries and cases of pneumonia or other rampant chest diseases. We were further inspired by recent advancements such as early stage breast cancer detection by MIT, which confirmed the feasibility of our project and highlighted its potential for significant real world application. Overall, these factors sparked us to create our platform for early stage diagnosis detection in order to remedy these issues. Our main focus was on accessibility when developing this platform as life-saving advancements are made but not always open to remote areas. By hosting our platform on the cloud, any clinic around the world can use it.

What it does

Syntoma, an accessible website that provides quick and accurate early stage diagnosis of pneumonia, 14 chest diseases, and knee osteoarthritis within seconds, while also helping users find clinics and medical information.

First, the user must sign in or sign up through our secure authentication system. Then, the user will select their preferred model and upload their medical image. Within seconds, the model endpoints run inference on the image and output confidence intervals which are displayed on the client side. Now, the user may ask the chatbot for more information about the condition, view or delete their past scans history on Syntoma in the scans page, query for specialized clinics and schedule appointments in the clinics page, or simply diagnose another medical image. This streamlined process covers key steps in the medical diagnosis process, from initial assessment to scheduling clinic appointments, displaying its usefulness in real world applications.

How we built it

We first got started on making a picture uploading and cloud storage management platform. We then pivoted into making this project where users can get a second opinion on their disease diagnosis for common chest diseases, pneumonia, and knee osteoarthritis. One person worked on the frontend, adding login, clinics page to the preexisting image cloud storage platform. Another person worked on training the AI models and building the RESTful API. At the end we integrated the two and did some A/B testing for user experience.

Challenges we ran into

Some of the technical difficulties we faced was training the CNN models to be accurate and feasible. For example, training the chest model on all 120k images in the dataset was not feasible, as it required large amounts of compute which we did not have and the dataset was also very unbalanced. To counter this, we limited the amount of images in each category, for example, 200 images per 14 disease categories from the dataset and we decreased image resolution from 512 by 512 to 256 by 256. Another problem was overfitting done by the model, which means that the model has a high training accuracy but low test accuracy. To combat this, we used image augmentation where we apply random filters to each image so the model is better adapted. We also used a variety of metrics such as confusion matrices, loss & accuracy plots over time, ROC curves, and F1 score to verify the models’ accuracy. Another thing was high latency times in the backend, meaning the models would run inference the image but giving back the results from frontend took too long. To solve this, we switched from Flask to the FastAPI library. Furthermore, we faced issues in finding a free hosting service for our PostgreSQL database, as that was the backend we had chosen for Syntoma. Luckily, Neon was very well equipped to serve us in the situation and has a very generous free tier of 512 MiB storage, which was far more than required for our website. Another challenge we faced was with issues with sending requests to our Flask backend to access the models. Flask has been having issues lately on Mac systems where you cannot access the server on your browser with localhost, or send fetch requests via localhost, and instead need to access it with your default IP address, which is unconventional compared to other backend systems. Finally, we experienced significant issues with the Places API on the Google Cloud Platform for our automatic local clinic detection, as the script would often fail to load or complain about null pointers. Eventually, we shifted with GCP's JavaScript Google Maps API, which worked perfectly.

Accomplishments that we're proud of

We've built an end-to-end platform the PERN stack. We are proud of achieving an average of 95% grounded accuracy across all of our models. We also proud of training our CNNs on large datasets.

What we learned

We learned a lot of new technologies. We learned how to use online database services such as Neon.tech. We learned about using UI packs to make a pleasant user experience. We learned FastAPI to maintain low a latency for the REST API. We also learned to make a end-to-end ML project.

What's next for Syntoma

For the AI/machine learning side, we have several improvements planned. To increase accuracy of all of the models we will train larger image classification models such as ImageNet. We will also train the model on the entirety of the images of the datasets, such as the NIH Chest X-ray Dataset. We also plan to deploy the actual ML models using serverless methods, so any clinic around the world can access our platform without utilizing expensive GPUs. We also plan to add miscellaneous, ML backed tools such as scan quality enhancement and organ segmentation. Regarding the website, we plan to implement a method to directly communicate with professionals through the platform. This could be either in the form of scheduling appointments with a clinic, or addressing concerns about some conditions or symptoms with a human doctor. That way, our app can truly be a one-stop shop, where patients can be diagnosed, learn about their conditions, and discuss or schedule appointments with doctors. More technically focused, we would implement a more robust file upload and image hosting service, such as AWS S3, rather than our current solution: uploadthing. The tech stack utilized for the website, notably Next.js, Express.js, PostgreSQL, and Flask is quite sophisticated already and will likely not be modified largely, however, we plan on building off our website to improve the frontend user interface and invest into stronger deployment services for our backend to ensure low latency loading times.

Built With

Share this project:

Updates