SympCare AI

Intro
Team Name
Problem Statement
Statistics
SympCare AI
Demo was being presented at this time
WorkFLow
Tech Stack
Future Possibilities
Roadmap

Inspiration

The inspiration for this project came from seeing how hard it is for people to understand their health issues without getting lost in confusing medical terms. Many people turn to the internet for answers but often end up more confused. We wanted to make health info simple, clear, and easy to access—something that feels like a natural conversation, so people can get the help they need without the frustration.

What it does

Our product is an AI-driven medical assistant that simplifies healthcare interactions by making them accessible to people who speak different languages and may not understand complex medical terminology.

Voice Input in Any Language: Users can speak to the assistant about their health concerns using their preferred language. This allows for a comfortable and natural way to communicate without the need to type or worry about language barriers.
Real-Time Translation and Diagnosis: The system translates the user’s voice input into English, allowing a sophisticated language model (LLM) to analyze the information. The LLM processes the symptoms or medical concerns provided by the user and delivers an accurate diagnosis based on its extensive medical knowledge.
Document Upload for Simplified Explanations: Users also have the option to upload medical documents, such as a doctor's diagnosis or test results. The system analyzes these documents, removes complex medical jargon, and provides a simplified explanation of the diagnosis, making it easier for users to understand their health situation.
Recommendations and Doctor Referrals: If the diagnosis suggests a serious health condition, the system advises users to see a doctor and even provides recommendations for medical professionals, making it a comprehensive health assistant.

How we built it

Our product leverages advanced technologies to create a seamless healthcare experience for users, regardless of their language or medical knowledge. Here’s how it works:

Language Input Recognition and Translation with Groq: When users describe their health concerns, we utilize Groq to accurately identify the language being spoken and translate the input into English. This ensures that the system can effectively process input from a diverse range of languages while providing immediate translation.
Diagnosis Generation with Groq: After translating the user's input into English, we use Groq to analyze the symptoms and provide a proper diagnosis. This powerful language model is trained on extensive medical data, allowing it to deliver reliable and relevant health information.
Spoken Output with Cartissa: Finally, we integrate Cartissa’s API to convert the translated text into spoken output. This feature enables users to hear their diagnosis and any necessary recommendations in their designated language, enhancing accessibility and user experience.

Challenges we ran into

Latency in Non-English Speech Processing One of the significant challenges we encountered was the latency when handling languages other than English. The speech-to-text model had difficulty processing certain languages efficiently, resulting in longer wait times. Some languages were harder for the model to recognize accurately, increasing the overall response time. For now, we had to limit speech output to English while we continue to explore ways to improve this functionality for non-English languages in future versions.
Complexity of PDF Upload and Content Extraction Implementing the feature to upload medical documents in PDF format and extract the content was more complicated than initially anticipated. The challenge was in ensuring that the model could read, understand, and simplify the dense medical language within the documents. Extracting structured and unstructured data from PDF files into a format the model could process efficiently required additional work.

Accomplishments that we're proud of

Despite facing significant challenges, we successfully completed the project, even under less-than-ideal circumstances. At the peak hour of development, one of our teammates left the team unexpectedly. The existing code they contributed was unclear, and without the ability to reach out and clarify what their code did, we had to make the tough decision to rebuild the entire product from scratch. Rebuilding everything without prior knowledge of the teammate’s work was a daunting task, but we rose to the challenge and pushed forward. The result is a fully functioning AI-powered medical chatbot, built through perseverance, collaboration, and quick adaptation to unforeseen issues. This accomplishment not only showcases our technical abilities but also our resilience as a team.

What we learned

Throughout this project, we had the opportunity to explore and learn some exciting technologies, including APIs like Groq, Deepgram, VAPI, Hume and Cartesia. Although we couldn’t incorporate all of them into this hackathon due to time constraints and project scope, we gained valuable knowledge about their capabilities. These tech stacks will definitely be part of our future projects as we continue to build innovative solutions. This hackathon helped us expand our toolbox, giving us new technologies to experiment with and enhance our future work.

What's next for SympCare AI

First, we aim to clean and optimize the codebase we’ve built. From there, the possibilities for this product are limitless. Our next goal is to reintroduce support for audio output in the same language the user spoke, overcoming the current language processing limitations. Beyond that, we plan to add features like appointment booking, allowing users to connect with doctors directly through the app. During live appointments, our chatbot would facilitate real-time translations between the patient and doctor, making healthcare more accessible, especially in multilingual settings. We also want to implement a distress signal button that would share the user’s live location with emergency responders and their emergency contact. This feature will ensure that people receive timely help in urgent situations, bringing healthcare closer and more responsive when it's needed most.