HealthEcho

Health guidance that listens first.

Comment

Inspiration

Overwhelmed by conflicting diet advice, complex fitness apps, and unclear gear recommendations, many people still want to improve their health or try a new sport. Most current solutions are highly dependent on screens, forms, and manual inputs, creating friction and discouraging consistent use. A voice-first AI system that simplifies experience by letting users speak naturally, clarify their goals and receive clear, understandable guidance without going through complicated interfaces inspired us to investigate.

What it does

HealthEcho is a voice-first AI assistant that offers personalized nutrition counseling and beginner-friendly sports gear suggestions via natural conversation. The users use only their voice to communicate their goals such as weight loss, weight gain, or starting a new sport. The system queries pertinent follow-up questions, detects user context, and provides a brief diet summary along with necessary equipment suggestions for the specified activity. Each suggestion is detailed so that users can grasp its significance, instead of getting generic or confusing recommendations.

How we built it

HealthEcho leverages ElevenLabs for conversational voice interaction and Google Cloud Vertex AI with Gemini models for intent extraction, reasoning, and response generation. The cloud-based backend processes voice input, manages prompt orchestration, and generates responses. Instead of training custom models, we created a rule-guided reasoning technique that, when combined with large language model capabilities, will produce outputs that are not only reliable but also explainable. Users of a lightweight web interface can input via microphone and playback audio, yet all core functionality remains voice-driven.

Challenges we ran into

mainly the designing of conversation flow and the information gathering for the recommendations to be useful were the two major issues that we had to deal with. Another problem was the length and the clarity of the response that had to be controlled so that the spoken answers would still be easy to follow. The smooth integration of voice interaction with the backend AI reasoning was also a challenge, as it required careful prompt design to avoid getting generic or repetitive responses.

Accomplishments that we're proud of

⦁ a complete voice-driven user flow was built with text input ⦁ ElevenLabs conversational agents were integrated with Gemini via Vertex AI ⦁ context-aware nutrition and gear recommendations were delivered along with clear explanations ⦁ a working, hosted application was deployed within the limited timeframe of a hackathon. ⦁ An image-based food recognition feature was implemented using a food image classifier to identify meals from user-uploaded images and support nutrition guidance

What we learned

voice-first design has the very different requirement of thinking which is completely opposite to the traditional UI-based applications. user trust is critically dependent on clear conversational structure, concise responses, and explainability. We also learnt the art of effectively combining the cloud-based AI services without unnecessary model training, focusing instead on system design and reasoning quality.

What's next for HealthEcho

in the future, there would be more sports and dietary preferences supported, enhancements for the long-term conversational memory would be made, and accessibility-focused features would be added. Moreover, we would like to refine the recommendation logic using user feedback but at the same time keeping a simple, voice-first experience.

Share this project:

Updates