FastTrack: Speech to Calories

Various methods of input: Speaking, texting, photo and manual entry
A dashboard where user can see their caloric intake patterns and whether they hit their target or not.
Onboarding process helps users to generate a plan and it projects their weight loss
The fasting tracker helps the user track their fast and eating window, sends reminders and shows them if they are in ketosis etc.
Of course, a nice RevenueCat paywall
icon

Inspiration:

I've struggled with my weight for most of my life. As someone who's been trying to lose weight for nearly two decades, I've had my fair share of calorie counting. It sucks. I found myself using ChatGPT to log my meals, but it wasn't the most convenient solution. I like that I can talk to it, but having no dedicated UI and no convenient way to edit or overview data give me the idea of making an app that I'd just speak and it would log the meals.

What it does: You text or speak to the app, just like speaking to a friend, and it tracks your calories using AI. You can also take a photo of your plate to do the same. It also has some intermittent fasting and multi-day fasting related features (fasting countdown, electrolyte intake, feeding-window reminders etc.)

How we built it: I built the app in about six days using SwiftUI for the frontend and Firebase for the backend. I used Sonnet 3.5 and ChatGPT to write the most of the code. For payments and paywalls I used RevenueCat.

Challenges we ran into: The main challenge was implementing the AI features in a way that would be robust to abuse. I did not want to wake up with a 1000$ OpenAI bill. So, with the help of AI, I have implemented Firebase functions with AppCheck, DeviceCheck and a leaky bucket IP address rate limiter. Another challenge was implementing speech-to-text in a way that makes sense. Sadly, using OpenAI or AssemblyAI did not make sense, so I opted for DeepInfra to run a Whisper API for me.

Accomplishments that we're proud of: I'm honestly surprised at how quickly I was able to put this together. In just over a week, I managed to create a functional app that combines fasting tracking, calorie counting, and AI-powered food logging. But my most proud achievement is that I did not build a product that I hoped some random person would buy. I really use my app daily, and not just because it is my app, I really find it very helpful.

What we learned: As I said above, I learned a lot about implementing AI features in a safe way (hopefully) and in a way that makes economic sense. Right now, each item tracked costs a tenth of a cent. Considering further reduction in cost, I can be safely profitable even if a user uses the app aggressively.

What's next for FastTrack: The next step is to make the app more helpful for weight loss. Specifically, now that the app know when the user has overeaten (or about to overeat), it can step in and help them in a way to empower them to understand their motivations. Like why exactly do they eat, when do they make unhealthy decisions, are they binge eating and why. I have a master's at Cognitive Science, so I know how helpful timely nudges and mindfulness can be.