Inspiration
The inspiration behind this project stemmed from the growing need for accessible, reliable, and intelligent virtual healthcare tools. We wanted to tackle three key problems:
- Medical Advice: Bridging the gap for users seeking instant, trustworthy medical guidance.
- Mental Health Support: Providing an empathetic digital therapist powered by machine learning.
- Local Resources: Helping users locate nearby healthcare facilities with ease.
Our goal was to combine cutting-edge technologies in AI, sentiment analysis, and location services to create a seamless, user-friendly experience.
What We Learned
Throughout this project, we gained invaluable insights into:
- AI Model Training: Fine-tuning a sentiment analysis model trained on 1.6M tweets to achieve 94% accuracy with confidence-based adjustments.
- Full-Stack Development: Integrating Next.js, Supabase, Spring Boot, and Flask into a cohesive and scalable architecture.
- Speech Interaction: Implementing text-to-speech for the therapist using the Google Text-to-Speech API to provide a humanized interaction.
- Proximity Calculations: Using geolocation services to filter nearby healthcare facilities dynamically.
This project taught us the importance of modular design, cross-functional teamwork, and thorough testing for a robust final product.
How We Built It
- Frontend: Developed with Next.js (TypeScript) for a responsive and intuitive UI.
- Backend: A powerful Spring Boot server handled API requests, integrating with multiple services.
- Sentiment Analysis: Trained a machine learning model to detect happy, sad, and neutral sentiments with Flask exposing the model as an API.
- OpenAI GPT Integration: Used an OpenAI GPT wrapper to deliver accurate and conversational medical advice for the Medical Doctor AI chatbot.
- Text-to-Speech for Therapist AI: Leveraged Google Text-to-Speech to provide empathetic spoken responses, enhancing accessibility and user engagement.
- Geolocation Services: Calculated user location and filtered clinics within a specific radius.
Challenges We Faced
- Model Accuracy: Achieving high accuracy while avoiding overfitting was a challenge. Introducing confidence-based adjustments significantly improved the model’s reliability.
- System Integration: Ensuring seamless communication between Next.js, Spring Boot, and Flask required careful design and debugging.
- Real-Time Performance: Optimizing text-to-speech for the Therapist AI required efficient API calls to maintain responsiveness.
- Location Precision: Accurately determining and filtering nearby clinics required fine-tuned radius calculations.
The Result
The result is an all-in-one virtual health platform that:
- Provides instant, AI-powered medical advice.
- Offers empathetic mental health support via a digital therapist with speech communication.
- Suggests nearby clinics dynamically based on the user's location.
We’re proud of the innovation and technical depth behind this project, and beyond proud of one another for surpassing the limits of what we thought was possible for one weekend,
Built With
- flask
- google-speech-to-text
- google-text-to-speech
- next.js
- openai-api
- opencv
- spring-boot
- supabase
- tensorflow
- typescript
Log in or sign up for Devpost to join the conversation.