Spoonfed

• 

  A look of the tool

Inspiration

Inspiration

Every nutrition app I've ever tried asks me to do the same thing: manually log what I ate, search google for the exact food, estimate the portion size, and repeat three times a day. Nobody actually does this consistently. I wanted to build something that removed that friction entirely. The idea was simple: what if you just took a photo?

What it does

Spoonfed lets you photograph any meal and instantly get a full nutritional breakdown powered by Gemini Vision. Beyond meal scanning, it decodes nutrition labels to flag hidden ingredients, swaps recipe components for healthier alternatives, generates personalized 7-day meal plans based on your body metrics and dietary restrictions, and tracks your daily progress against calculated targets.

How I built it

The backend is a Flask API with four routes, each hitting the Gemini API with carefully engineered prompts that return structured JSON. The frontend is vanilla JavaScript with no framework, using HTML5 Canvas for the pixel art characters and CSS animations for the background. Every Gemini call sends the image as base64 via inline_data alongside a text prompt instructing the model to return a strict JSON schema, which the frontend parses and renders directly into the UI.

Calorie targets are calculated using the Mifflin-St Jeor equation:

$$ BMR = 10w + 6.25h - 5a + s $$

where $w$ is weight in kg, $h$ is height in cm, $a$ is age, and $s$ is +5 for males and −161 for females. This is then multiplied by an activity factor to get TDEE, and adjusted by goal:

$$ Target = TDEE \begin{cases} -500 & \text{lose weight} \ +300 & \text{build muscle} \ \pm 0 & \text{maintain} \end{cases} $$

Challenges

The biggest challenge was getting the Gemini API to return consistently structured JSON across five very different prompt types: meal images, label photos, plain text recipes, and profile data. Prompt engineering to enforce a strict schema without markdown fences took significant iteration. API quota limits on a hackathon deadline were also a real constraint, which pushed us to implement a demo mode as a fallback.

What I learned

Multimodal AI is genuinely powerful when the prompt is well-structured. The gap between "it works" and "it works reliably" in AI applications is almost entirely about how precisely you define the output format. Also: never hardcode API keys.

What it does

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for Spoonfed

Built With

• css3
• flask
• flask-cors
• gemini-vision-(multimodal)
• google-gemini-api
• html5
• javascript
• python
• python-dotenv
• tailwind-css