Health WiZ

• 

• 

• 

• 

  Architecture diagram

• 

  Is Your Food Healthy?

• 

  All Docs QA using language of choice

• 

  All Docs QA using language of choice

• 

  All Docs QA

• 

  Single doc search

• 

  Impact on retrieval of answer by changing the text chunk size

• 

  Impact on retrieval of answer by changing the temperature

• 

  Low code submission

Inspiration

The inspiration behind WellnessWiz is rooted in a vision to democratize health and wellness knowledge, leveraging technology to make personal health management accessible to everyone, regardless of language or location. It's driven by the understanding that our dietary choices have profound impacts on our overall health, and that having instant, reliable insights into the foods we eat can lead to better health outcomes. The creators of WellnessWiz recognized the challenges faced by individuals in navigating complex health information and making informed decisions about their diets and well-being. They saw the potential in harnessing AI and machine learning to analyze and interpret health data in real-time, providing users with actionable advice and support. Moreover, WellnessWiz aims to address the language barrier that often prevents non-English speaking users from accessing quality health information. By offering multilingual support, the app empowers a wider audience, ensuring that language is no longer an obstacle to understanding and improving one's health. Behind WellnessWiz lies a commitment to empathy and inclusivity, ensuring that every user feels heard and cared for. The app's intuitive interface and personalized feedback loop are designed to foster a sense of trust and encouragement, enabling users to take control of their health journey with confidence and ease. In essence, WellnessWiz is inspired by the belief that everyone deserves to live their healthiest life, supported by technology that is smart, caring, and culturally aware. It's a testament to the power of innovation to transform the way we approach health and wellness in our daily lives.

What it does

WellnessWiz goes beyond dietary insights; it's a 24/7 health assistant attuned to your personal health history, ready to guide you through any medical symptoms. Imagine the convenience and reassurance of having round-the-clock access to health advice. Whether you're facing a new health concern or managing ongoing conditions, WellnessWiz is there to support you. And for our non-English speaking users, our app breaks down language barriers by providing health information in multiple supported languages.

How we built it

Gemini LLM is the linguistic intellect of the app, a language model that not only understands the nuances of human conversation but also crafts responses that are contextually relevant and engaging. This model processes user queries with remarkable accuracy, making the interaction as natural and intuitive as speaking with a human health advisor. Complementing the linguistic capabilities of Gemini LLM is Gemini Vision Pro, our advanced image recognition model. When users upload images of their meals, Gemini Vision Pro analyzes the visual data to identify food items and provides nutritional insights. This model is essential for the app's ability to offer real-time dietary advice, transforming a simple photo into a wealth of health information. RAG, or Retrieval-Augmented Generation, is the app's research powerhouse. By combining the precision of retrieval-based models with the creativity of generative models, RAG efficiently sifts through extensive document repositories to find accurate information, which it uses to generate informed, reliable responses to user inquiries. The entire user experience is brought to life through Streamlit, the versatile Python library that allowed us to build a dynamic and interactive web interface. Streamlit empowers users to effortlessly navigate the app, engage with its features, and receive personalized health advice, all within a few clicks. Together, these technologies enable WellnessWiz to provide a comprehensive, AI-powered health advice system that is both sophisticated and accessible, ultimately aiming to enhance the well-being of its users.

Challenges we ran into

Acquiring relevant health and wellness documents: Finding trustworthy and pertinent documents proved to be difficult. To address this, we transcribed a few YouTube videos to bridge the gap and ensure a comprehensive document repository. Tuning various parameters: Determining the ideal chunk size, number of chunks, and overlap required extensive experimentation and iterative adjustments. Fine-tuning these parameters was crucial for optimal performance. Prompt tuning: Selecting the right prompt for both the Gemini Vision model and the Gemini LLM model played a vital role in achieving accurate results. Through the implementation of best practices and extensive experimentation, we were able to achieve satisfactory outcomes. Testing the application: Testing the "Is Your Food Healthy" functionality posed a challenge, specifically in obtaining royalty-free images for testing purposes. Overcoming this hurdle required meticulous sourcing and selection of appropriate images. Deployment on Google Cloud Platform (GCP): With the intention of deploying our Streamlit application on GCP, we encountered a learning curve, as GCP App Engine does not support Streamlit natively. We had to leverage the App Engine flexible environment to successfully deploy our application on GCP.

Accomplishments that we're proud of

With just a simple photo of your meal, our app instantly reveals whether you're about to indulge in a dish high in sugar or fat, or if you're set for a healthy feast. But that's just the beginning. Language is no longer a barrier to health related information 24/7 health assistant attuned to your personal health history, ready to guide you through any medical symptoms. Users can seek mental wellness related information without the fear of being judged.

What we learned

Gemini LLM: We learned the capabilities and potential of the Gemini LLM language model in understanding human conversation and generating contextually relevant responses. This model plays a crucial role in providing accurate and intuitive health advice based on user queries. Streamlit: Building the user interface using Streamlit allowed us to create a dynamic and interactive web interface for the application. We discovered the versatility and ease of use provided by this Python library in creating a seamless user experience. Langchain: Integrating the Langchain language translation tool enabled us to achieve multi-lingual capabilities in the application. We recognized the importance of language accessibility in providing health information to users around the world. Gemini Vision Pro: The utilization of Gemini Vision Pro for image recognition and analysis opened up possibilities for providing real-time dietary advice based on visual data. We harnessed the power of this model to transform uploaded food images into valuable health insights. Google Cloud platform: Deploying the application on the Google Cloud platform, specifically using the App Engine Flexible service, taught us the benefits of scalability, reliability, and easy access for users. We gained experience in leveraging cloud services for seamless application deployment. Overall, the development of this Gemini LLM based application has taught us valuable lessons in leveraging AI, machine learning, and cloud technologies to empower users with accessible and personalized health information. From understanding natural language conversations to analyzing images and translating languages, we have harnessed the capabilities of various technologies to create an intuitive, responsive, and culturally aware health assistant. Product Ideation.

What's next for Health WiZ

The Gemini LLM based application for image-based health advice has laid a strong foundation for personalized health management. Moving forward, there are several areas that can be explored to enhance the user experience and expand the application's capabilities. Some of the potential next steps include:

1. Continuous refinement of health advice accuracy: • Further training and fine-tuning of the Gemini LLM model to improve accuracy in
   providing health advice based on food images. • Integration of additional data sources and research to ensure the most up-to-date and
   reliable information is provided to users.
2. Expansion of supported languages: • Continual development of the Langchain language translation tool to expand the number of languages supported by the application. • Collaboration with language experts to ensure accurate translations and culturally sensitive health advice in different languages.
3. Integration of user feedback and preferences: • Implementation of feedback mechanisms to gather user input and improve the functionality and usability of the application. • Customization options for users, such as personalized health goals and preferences, to
   provide more tailored health advice.
4. Integration with wearable devices and health tracking platforms: • Collaboration with wearable device manufacturers and health tracking platforms to seamlessly integrate health data into the application. • This integration would enable more comprehensive and personalized health advice based on real-time data.
5. Partnerships with healthcare professionals and organizations: • Collaboration with healthcare professionals to ensure that the health advice provided by the application aligns with established medical guidelines. • Partnerships with healthcare organizations and institutions to access authoritative health information and contribute to ongoing research.
6. Expansion into other areas of health and wellness: • Exploration of additional features and functionalities related to exercise, mental health, sleep, and other aspects of holistic well-being. • Collaboration with experts and professionals in these domains to develop evidence-based advice and recommendations.
7. User education and awareness: • Development of educational materials and resources to help users understand the reasoning behind the health advice provided by the application. • A focus on promoting health literacy and empowering users to make informed decisions regarding their health and well-being.
8. Add the feature of voice enabled question /answer improving the accessibility feature of the app.

Built With

• gcp
• gemini
• langchain
• python
• streamlit