VoCal

Logging food by voice
Chatting with Coach about goals

Inspiration

While talking about challenges we face in fitness, we talked about how difficult it is to track calories & macronutrients manually and the inaccuracies in AI calorie trackers. We saw an opportunity in the market to create a holistic app that allows you to efficiently and accurately track your diet. Most AI diet trackers use manual inputs which take 2-3 minutes per meal, or 2D pictures. However, meals are 3-dimensional, which means that a large aspect of most meals do not get tracked properly. Currently, AI calorie and nutrient tracking applications cannot detect foods that are completely submerged or covered, like a burrito or a chipotle bowl. For example, for a chipotle bowl, the AI could only visually see the lettuce, cheese, and sour cream but not the rice, chicken or veggies hidden underneath. Typically, you would then have to manually input all of these which would take minutes. Instead, we created VoCal, where you can snap the photo, describe your meal in detail within a couple seconds, and your calories and nutrients will be logged 4-5x faster and more accurately.

https://github.com/EricSpencer00/uncommonhacks26

What it does

VoCal is a calorie and nutrient tracking app that tracks food more accurately and much more efficiently than the current applications available. The user would take a photograph of their food/meal, describe it in detail, VoCal would scan the photograph, utilize the voice-to-text transcript, and log it. If the meal is unclear, there will be a “siri-like” voice agent that will ask the user clarifying questions to get a more holistic view of the meal as well. Additionally, there is a built-in coach that detects trends and your current goals to give you suggestions based on your inputs.

How we built it

VoCal is iOS-based first, taking advantage of all of the features that Apple provides such as widgets and HealthKit integrations. We bought a domain for $1.39 on the .best TLD and then deployed our website and backend router via Cloudflare. Our lead senior developer, named Claude Opus, worked tirelessly to bring our idea to fruition. His co-lead, Codex GPT 5.5, assisted whenever presented with a major architectural issue.

Challenges we ran into

One challenge we ran into is ensuring that our chatbot within the app stores memory about the user and previous context, so that it is well equipped to continue helping the user achieve their goals. Additionally, another challenge we faced was making sure the app's photo tracking abilities were as accurate as possible. Next, we also ran into some problems implementing the AI to ask meaningful and clarifying questions to make the tracking as accurate as possible. Finally, one last challenge we had was with our landing page on visual representation. We had different ideas on the ambience of the app, whether it should be more simplistic or visually appealing, showing actual foods, however it became too difficult to add very sophisticated designs and visuals within our landing page.

Accomplishments that we're proud of

Some accomplishments that we are proud of are actually implementing each notable feature such as the “siri-like” chatbot that asks clarifying questions and being able to grasp a more holistic view of the meal being logged. Another accomplishment is getting the AI Coach to track the users trends, habits and nutritional diets and provide feedback and encouragement for the users physical and health goals as well. One of our more unique features is the fact that after taking the picture, when you use your voice for the food you’re eating, VoCal will use this to update the nutritional information of the food that you are eating accordingly. Finally, we are just proud of the overall development of the app, the UI & UX, and the progress made in such a short time.

What we learned

During the development of VoCal, we learned how to place ourselves as a differentiated fitness app. There are currently many established apps that track calories. All of these apps are relatively useful for tracking your diet. However, VoCal differentiates itself by actually using this data to inform decisions for users. For example, when someone seems to constantly be eating similar meals each day and missing out on a certain mineral or vitamin, the coach can identify this pattern and give the user recommendations on how to fill this gap in their diet.

What's next for VoCal

In the future, VoCal plans on incorporating more health apps like Whoop and Garmin, which are more advanced fitness apps. This will allow the coach within the app to give more tailored recommendations to athletes who are more focused on their performances. In turn, VoCal can turn into a quick and reliable fitness coach for those who do not want to hire a personal trainer. Another unique feature that can also be implemented within the app are meal scores, which can help to show how healthy a meal is. This will require more precise technical analysis of a meal. This will allow users to learn more about how to properly build a healthy meal. We also want to add a feature for food catalogs, so if needed manual entry can also be possible. Over time, the app should also be able to adapt to you and learn more about your specific dietary habits, which means that food tracking will become more accurate as you continue to use it. It will also be able to remember the meals you eat over time, which will further reduce the time it takes to input your food. Another important consideration we want to implement in the future is to make VoCal’s tracking more accurate for ethnic dishes. Currently, with ethnic dishes, many apps take wild guesses at their nutrition and it’s very difficult for people to understand what they are eating. In the near future, there are also some UI features that we can improve to make the app a little more friendly.

https://youtube.com/shorts/1afIit59_ss