-
-
Homepage with no uploads
-
After an upload with the uploaded photo on left and macros on right with a chart breakdown of nutrients
-
Bottom of page where there is an option export your counted meals
Inspiration
Traditional macro tracking apps are a total chore. Nobody wants to spend ten minutes searching through endless databases trying to find the exact brand of chicken breast or guessing how many grams of rice are on their plate. We wanted to build something that feels as instant and natural as taking a snap for your story, a tool that translates visual food layouts into exact, actionable macro streams in a single tap.
What it does
PlateCheck is a high-speed vision macro orchestrator that strips away the manual data entry of fitness tracking. Users simply upload a picture of their meal or open their camera straight from the dash. Behind a ultra-clean, minimalist interface, an advanced Vision-Language Model (VLM) deconstructs the plate's architecture. It instantly isolates overlapping food items, estimates their individual weights in grams, pulls real-time nutritional profiles, and updates a live metric engine alongside a longitudinal macro vault.
How we built it
We engineered PlateCheck from the ground up to prioritize minimal friction and high UI responsiveness:
- Frontend Dashboard: Built completely using Streamlit, featuring a side-by-side control interface, custom layout state tracking, and reactive UI elements.
- Semantic Vision Logic: Integrated Google Gemini's advanced Multi-Modal capabilities via Structured Outputs to handle complex, overlapping food geometry and generate reliable weight estimates.
- Data Integration Pipeline: Developed custom ingestion modules utilizing Python pandas, mapping the visual breakdown payload to verified nutrition data engines.
- Dynamic Analytics: Integrated Plotly Graph Objects to render beautiful, interactive macro distribution profiles directly on the runtime panel.
Challenges we faced
Managing hardware and persistent data flow inside a stateless web framework was tough. Early prototypes suffered from webcam lockouts and variable scope crashes during state reruns. We overcame this by re-architecting our layout state tracking, utilizing persistent Streamlit session state buckets, and centralizing our image payload extraction so that data pipelines never break whether an image is dropped from device storage or captured raw via hardware inputs.
Accomplishments that we're proud of
- Building a responsive dual-panel interface that feels like a native mobile app.
- Adding multiple gemini keys to rotate through when too many uploads have occurred.
- Transitioning the user experience away from a dry, clinical tool into a highly engaging photo upload.
What we learned
We learned that handling unstructured visual inputs is as much about data formatting and strict typing as it is about advanced AI models. We gained deep hands-on experience structuring VLM outputs, handling active hardware state switches in Python, and building clean analytics pipelines with pandas.
What's next for PlateCheck
We want to add features that will make it more interactive. Having a leaderboard of healthy or unhealthy meals was the next feature that we wanted to add. Having a place where you can see other people's meals sounds like an interesting social feature.
Log in or sign up for Devpost to join the conversation.