Inspiration
The current health and fitness market is brimming with trackers but lacks companions. People trying to get fit or stay healthy often find themselves lost in a sea of data and numbers. The emotional aspect of the journey, the personal touch, is missing. Our goal with Foodfolio is to fill this gap, offering an intelligent companion that not only provides data but also adds a personal touch, guiding users through their wellness journey, helping them anticipate pitfalls and providing insights for improvement.
What it does
Foodfolio leverages AI to transform your meal photos into powerful nutritional insights and emotional sentiment analysis. It takes into account your eating habits and combines this with your personal fitness goals to craft a custom meal plan that suits your taste preferences. Fitness does not have to be broccoli and chicken or 45 minutes on the elliptical every day. By syncing with your calendar, we intelligently identify trends in your eating habits and gym attendance, enabling us to provide proactive advice that can aid in avoiding unhealthy patterns, the guilty binges and the unwarranted cheat days.
How we built it
Foodfolio is built using React.js for the frontend, providing an intuitive and engaging user interface. The backend leverages the Passio AI for image recognition and calorie calculation from meal photos. We use Hume AI for the sentiment analysis of users' feelings towards their meals, their workouts and events throughout the day and MindsDB for predictive analytics to gauge the likelihood of meeting daily nutritional goals, likelihood of enjoying a recipe and through the Google Calendar integration predicting binge eating patterns. These technologies are seamlessly integrated through a microservice architecture to create an interactive, user-friendly platform that goes beyond simple tracking to provide personalized insights and recommendations.
Challenges we ran into
Building Foodfolio presented a set of unique challenges, both in terms of technical architecture and user experience design. As a team primarily composed of developers, our first hurdle was the lack of dedicated UX/UI designers. This led us to a steep learning curve in understanding the principles of human-computer interaction (HCI) and information architecture. Creating an intuitive and engaging user experience required us to juggle roles, quickly learn design principles, and implement them, all while maintaining focus on our technical objectives.
On the technical front, deciding how to leverage the strengths of Passio AI, OpenAI, Hume AI, and MindsDB was a considerable challenge. Each AI technology had its unique functionalities and limitations. Figuring out how to integrate these technologies and extract the maximum potential from each one, while maintaining efficient real-time performance, required a nuanced understanding of each API's intricacies.
Furthermore, the short time frame necessitated tough decisions about prioritizing features and capabilities. We had to balance our ambition to build a comprehensive, multifaceted platform with the practical constraints of time and resources. This involved rigorous brainstorming, feasibility analysis, and a precise definition of the minimum viable product (MVP) that could effectively embody the vision of Foodfolio. Despite these challenges, we are proud of the progress we have made and are excited about the journey ahead.
Accomplishments that we're proud of
We are proud of creating a unique tool that addresses problem areas many people struggle with every day. Prior to the hackathon we spoke to people at the UC Berkeley Recreational Sports Facility to understand our users pain points. The consistent theme was people feeling like they are having to do too much guess work in trying to figure out:
- what's a good meal
- what's a cheat meal
- how often should I work out
- what should I eat. Many people also emphasized that it's hard to stick to their fitness journey because they act retrospectively rather than predictively. How nice would it be if instead of feeling guilty in retrospect about a binge eating session, somebody could have warned you predictively that 'hey you seem stressed today don't deal with that through food and go down that downward spiral'. Hence, we are proud that we are able to use amazing technologies like Hume AI, Minds DB and GPT-3.5 to be that companion who helps you stay on track and takes your emotions into account to craft your goals.
What we learned
Creating Foodfolio introduced us to the unique challenges associated with integrating multiple AI technologies, coupled with creating a human-centric user interface (UI). The application combines Passio AI for image recognition and calorie counting, OpenAI for personalized meal planning, Hume AI for sentiment analysis, and MindsDB for predictive analytics.
Our journey began with integrating Passio AI’s state-of-the-art image recognition technology. Ensuring seamless interaction between our backend and Passio's API for real-time analysis presented an intriguing technical challenge. Simultaneously, we had to design UI elements that would make the image capture and analysis process intuitive for users, taking into consideration various factors such as usability heuristics, user feedback, and mobile device constraints.
Next, we integrated OpenAI for intelligent meal planning. The technical complexity here lay not only in harnessing OpenAI’s advanced algorithms for personalized recommendations but also in presenting these recommendations to users in a digestible, engaging way. We learned about applying principles of cognitive load theory in interface design to ensure that users are neither overwhelmed nor under-informed.
With Hume AI, sentiment analysis came into play. We were tasked with interpreting and analyzing unstructured user data to understand emotional responses to meals. The challenge was two-fold: developing innovative techniques for sentiment analysis and presenting this emotional data to users in a meaningful and sensitive way, a balance crucial in human-computer interaction (HCI).
MindsDB's predictive analytics capabilities were utilized to anticipate users' likelihood of meeting their nutritional goals. Interfacing with the MindsDB API taught us the intricacies of predictive modeling algorithms, while designing UX/UI elements for presenting these predictions demanded a clear understanding of information architecture and data visualization principles.
Throughout our journey, we faced the engineering challenges of integrating disparate APIs, diverse data structures, and the complex synchronization of multiple technologies. Additionally, we learned valuable lessons in user experience (UX) design, focusing on the balance between technical robustness and user-friendly interfaces. The HCI aspect further enriched our understanding of creating empathetic and effective digital health tools, reminding us to place user needs at the heart of our design and development process.
What's next for Foodfolio
Foodfolio is still in its early stages of development, and we have a promising journey of innovation and expansion ahead of us. The preliminary stage of our project was primarily focused on design, planning, and partial implementation, but there's much more to come.
Due to the limited time frame of the initial phase, we couldn't fully build out the backend to integrate all the AI technologies as planned. Going forward, one of our main objectives is to fully integrate Passio AI, OpenAI, Hume AI, and MindsDB into a unified, fully operational system. This involves working out the intricacies of each AI's API and ensuring seamless interoperability for real-time analysis. Given that we were constrained by the availability of credits for Passio AI, obtaining the necessary resources to fully exploit this technology's potential for accurate image recognition and calorie estimation is a priority.
In terms of OpenAI and personalized meal planning, we'll be enhancing our algorithms to better understand user-specific dietary patterns and nutritional needs. Similarly, with Hume AI and MindsDB, we aim to improve our sentiment analysis capabilities and predictive analytics, respectively. Each AI technology presents unique challenges, and we're eager to tackle them head-on to refine our platform's features.
In addition to these technical enhancements, we recognize the value of an intuitive, user-friendly interface and are committed to iterating on what we've learned about UI design and HCI. We'll be applying these principles to create a better user experience, with improvements planned in navigation, information architecture, and visual design. Our objective is to create an interface that is not only aesthetically pleasing but also easy to navigate and efficient to use.
We also plan to foster a stronger community within Foodfolio. By introducing more interactive features, we hope to encourage users to share their journey, connect with others, and find motivation within the app. This involves potential collaborations with nutritionists and fitness experts to offer professional insights and coaching within our platform.
The journey ahead for Foodfolio is both challenging and exciting. As we move forward, our focus will remain on continual learning, iterative improvement, and pushing the boundaries of what is technically achievable, while always prioritizing our users' needs and experiences.
Log in or sign up for Devpost to join the conversation.