SymptoAI

SymtoAI is an AI-powered tool for medical diagnostics that analyzes speech and video inputs. It offers rapid diagnostic feedback, suggests possible causes, and recommends remedies.

Comment

Inspiration

The inspiration for SymptoAI came from the fact that nearly one-third of the world's population lacks reliable internet access, yet they still need access to basic healthcare and hygiene practices. In many parts of the world, people face barriers to essential healthcare services due to distance, affordability, or lack of available medical professionals. This issue is especially prevalent in remote areas, where individuals may not be able to access healthcare facilities in time for a proper diagnosis.

We recognized that AI could bridge this gap, providing people with timely health information and remedies based on symptoms they describe through text, voice, or video, even with limited or no internet. SymptoAI was created to help provide faster, more efficient, and accessible healthcare recommendations for people in areas where medical knowledge and hygiene practices may not be widely known or easily accessible.

What it does

SymptoAI provides one-on-one support to individuals by analyzing their symptoms and offering potential diagnoses, along with hygiene practices and remedies. Users can describe their symptoms through text or images, and the AI will generate tailored suggestions based on their input. In addition, SymptoAI generates a markdown report that can be shared with healthcare professionals for further evaluation, facilitating better communication and faster diagnosis, even in resource-limited settings.

How we built it

We built SymptoAI using a combination of deep learning, natural language processing (NLP), and video and speech recognition technologies. The backend is powered by a deep learning model trained on medical data, while the frontend is a user-friendly web interface. The RESTFUL API connects the backend and frontend to facilitate smooth interactions. We used various tools and libraries for video and audio processing and NLP to ensure that the system could accurately analyze symptoms and provide relevant diagnostic suggestions.

Challenges we ran into

We ran into many technical challenges when building this project. Some of which are:

1. Training a Model Ourselves

Initially, we attempted to train our own AI model for medical diagnostics, but it failed to achieve the desired accuracy and reliability. This was due to the complexity of the medical data, lack of sufficient labeled datasets, and computational limitations. We eventually opted for a pre-trained model to ensure better performance.

2. Frontend Code Not Displaying the Markdown File

Integrating the markdown file viewer into the frontend posed several challenges. Despite correctly parsing and generating markdown on the backend, the UI often failed to render the file properly due to issues with JavaScript logic and event handling.

3. Raspberry Pi 4 Crashing with Text-to-Speech

The Raspberry Pi 4 frequently crashed when running text-to-speech functionality alongside other AI-related tasks. Its hardware limitations, particularly in terms of memory and processing power, made it unsuitable for handling heavy AI computations and multitasking.

4. Server Not Responding Properly to Image Inputs

The REST API often failed to respond accurately when sending images for AI-based diagnostics. This issue stemmed from improper server configuration and difficulties in handling large image files, leading to incomplete or delayed AI-parsed responses. Debugging and optimizing the server pipeline became a critical focus.

Accomplishments that we're proud of

We are proud of the fact that we successfully developed an AI system capable of processing both video and speech inputs and providing diagnostic suggestions. The integration of the machine learning model with the web interface was a major accomplishment, and we are especially pleased with how we could use Raspberry Pis along with web-servers and a website, all working together. We also managed to build a tool that is scalable and helpful to billions of people around world.

What we learned

Throughout this project, we learned how to apply machine learning and deep learning techniques to real-world healthcare problems. We gained experience in handling and processing video and audio data through web-servers, and we also learned about the ethical considerations involved in building AI-driven tools for healthcare. Additionally, we enhanced our skills in full-stack development, API integration, and troubleshooting issues related to Hardware system performance and how to integrate complex hardware, with complex software.

What's next for SymptoAI

In the future, we plan to enhance the AI model by expanding the dataset and improving its diagnostic capabilities. We would also like to use a custom LLM built specifically for our product. We will work on making the system more robust and reliable by testing it in real-world healthcare environments and collaborating with medical professionals. We would also like to improve its functionality with AI-purposed hardware. Additionally, we aim to add more features, such as multilingual support, integration with electronic health records (EHR), and improved video and image analysis. We also plan to explore potential collaborations with healthcare organizations to ensure the tool can truly be a real doctor for the billions of people around the world without access to healthcare or even the internet.

Built With

Share this project:

Updates