NutriPulse AI: Smart Nutrition Coach

Inspiration

One of the biggest real-world challenges in preventative healthcare is diet adherence. Tracking calories and macronutrients is notoriously tedious, often requiring users to manually search through massive, inaccurate food databases. We were inspired to eliminate this friction by leveraging the latest advancements in Multimodal AI. We wanted to create a tool that acts not just as a tracker, but as an intelligent, privacy-first sports nutrition coach—empowering users to take control of their health simply by snapping a photo of their meal.

What it does

NutriPulse AI is an advanced, client-side web application designed to optimize personal wellness.

1. Multimodal Food Scanning: Users can drag-and-drop or upload a photo of their meal. The app uses advanced Vision AI to instantly identify the food, estimate portion sizes, and log precise caloric and macronutrient values.
2. Dynamic Onboarding & Analytics: It calculates daily caloric/macro targets using the Harris-Benedict BMR equation and tracks progress via an interactive, HTML5 Canvas-rendered 7-day trend chart.
3. AI Sports Nutrition Coach: A dedicated chat assistant that has context of your daily food logs. It acts as a personal health coach, adapting recipes, planning meals, and providing tailored health advice.
4. Privacy First: All data, including API keys and daily logs, are stored locally in the browser. No centralized servers, no tracking—putting the user's healthcare data firmly in their own hands.

How we built it

We built NutriPulse AI focusing on a seamless, client-side architecture wrapped in a premium, glassmorphism UI:

• Frontend: We used Semantic HTML5, Vanilla CSS3 (for animations and layouts), and JavaScript for state management and programmatic canvas drawing.
• The AI Engine: We integrated the Fireworks AI API to power our models.
• Vision Model: We utilized llama-v3p2-11b-vision-instruct for the multimodal food scanning, passing base64 image data to the model to extract JSON-formatted nutritional estimates.
• Coaching Model: We used llama-v3p1-70b-instruct to power the conversational sports nutrition coach, passing the user's local logs as context in the system prompt.

Challenges we ran into

Handling multimodal image processing entirely on the client-side presented unique challenges. We had to ensure that large food photos were appropriately compressed and converted to base64 before being sent to the Fireworks API to prevent latency spikes or payload rejections. Additionally, prompting the Vision model to consistently return structured, parsable nutritional data (instead of conversational text) required significant prompt engineering and negative constraints.

Accomplishments that we're proud of

We are incredibly proud of the True Multimodal Food Scan feature. Seeing the app take an unstructured, messy photo of a plate of food and accurately parse it into precise protein, carb, and fat metrics feels like magic. We're also proud of the privacy-first architecture—proving that powerful AI healthcare apps don't require sacrificing user data to cloud databases.

What we learned

We gained deep insights into the capabilities of the new Llama 3.2 Vision models. We learned how to effectively orchestrate two different LLMs (a vision model and a conversational model) within the same application, sharing context between them to create a cohesive user experience.

What's next for NutriPulse AI: Smart Nutrition Coach

We plan to expand the platform by:

• Wearable Integration: Connecting to Apple Health and Google Fit APIs to automatically adjust caloric targets based on real-time activity and heart rate data.
• Barcode Scanning: Adding a fallback traditional barcode scanner for packaged foods.
• Localized Coaching: Translating the AI coach into multiple languages to democratize access to sports nutrition globally.

Built With

• artificial-intelligence
• computer-vision
• css3
• fireworks-api
• healthcare
• html5
• javascript
• llama
• machine-learning