Nutry

Introducing Nutry, your ultimate nutrition co-pilot! 🚀 With Nutry, decoding your food choices has never been easier.

Comment

Inspiration

We are foodies and hashtag enthusiasts! We've discovered that the majority of customers take pictures of their food before they eat. So, we thought it would be great to upload pictures of our meals and receive nutritional information. This way, we can take care of our health while enjoying the fun of hashtagging.

What it does

This tool extracts nutritional information from the pictures of food we upload and calculates the diversity of nutrients present. It then displays this information in a table format. Additionally, it identifies any nutritional gaps in our daily food intake and suggests food examples to fill these gaps. This way, we can easily access nutritional information without the hassle of googling or searching through books.

How we built it

Frontend Development

For the frontend, we harnessed the power of React, a robust and dynamic JavaScript library, to craft a seamless and intuitive user interface. React's component-based architecture allowed us to create a modular and scalable design, ensuring a delightful user experience.

Backend Infrastructure

The backbone of Nutry's functionality is supported by a Node.js backend. Leveraging the event-driven, non-blocking I/O model of Node.js, we achieved high-performance server-side execution. This backend architecture ensures swift data processing and seamless communication between the frontend and other components.

Database Management

To store and manage data efficiently, we implemented MongoDB as our database solution. MongoDB's NoSQL, document-oriented approach perfectly suits the dynamic and evolving nature of our application. This enables us to store and retrieve nutritional data with speed and flexibility.

UI Components

To enhance the visual appeal and functionality of Nutry, we integrated Shadcn Radix UI components. These components not only provide a sleek and modern aesthetic but also contribute to the overall responsiveness and user engagement of the application.

Image Recognition and Nutrition Extraction

The core of Nutry's groundbreaking image-based nutrition analysis lies in GPT-4 Vision. This advanced model accurately identifies items within uploaded images, laying the foundation for extracting precise nutritional details. By seamlessly integrating GPT-4 Vision, we ensure real-time, accurate analysis of diverse meals captured by users.

External API Integration

To complement the image recognition process, we integrated with an external API for detailed nutrient extraction. This step allows Nutry to provide users with comprehensive information about the caloric content, macronutrient breakdown, and more, contributing to an informed and personalized nutritional experience.

Personalized Suggestions

Nutry goes beyond basic nutrition information by offering personalized suggestions based on user profiles. Leveraging the GPT-4 API, we analyze user-specific details such as allergies, dietary preferences, and physical goals. This enables Nutry to generate tailored recommendations, empowering users to make choices aligned with their individual health and wellness objectives.

In summary, Nutry is a synergy of cutting-edge technologies, including React, Node.js, MongoDB, Shadcn Radix UI components, GPT-4 Vision, and external APIs. This comprehensive tech stack ensures Nutry's reliability, scalability, and, most importantly, its ability to revolutionize the way users interact with their nutritional choices.

Challenges we ran into

  • Integration frontend with backend especially using only local storage (no cloud service to upload image)
  • togther.ai's LLMs were hallucinating a lot and we spent a lot of time modifying our parameters and still the results were not beneficial.
  • Since gpt-4-vision-preview doesn't have a temperature parameter yet so we had to spent immense amount of time on engineering the prompt.

Accomplishments that we're proud of

Precision in Data Retrieval

One of Nutry's standout accomplishments is achieving an impressive 99% accuracy rate in data retrieval. This milestone was attained through the implementation of a sophisticated prompting system coupled with a meticulous chaining of prompts. By leveraging advanced techniques, we have fine-tuned the data retrieval process, ensuring users receive highly accurate and reliable nutritional information for their meals.

Sophisticated Prompting System

Nutry's prompting system stands as a testament to our commitment to excellence. We have meticulously designed a system that intelligently prompts the underlying AI models to provide precise and contextually relevant responses. This not only enhances the accuracy of nutrient extraction from images but also contributes to a smoother and more user-friendly experience.

Chain of Prompts for Scalability

We take pride in developing a robust chain of prompts that not only ensures accuracy but also lays the foundation for scalability. This chain has been strategically crafted to make the application highly scalable, allowing for easy integration of new features and adaptation to evolving technological landscapes. The result is an application that not only meets the current needs of users but is also well-positioned for future enhancements and expansions.

Ease of Maintenance

Nutry's success isn't just in its innovative features but also in its sustainability. Our team has prioritized ease of maintenance by implementing clean and modular coding practices. This approach minimizes the effort required for ongoing updates, bug fixes, and the addition of new functionalities. Nutry is designed not only to perform at its peak but to do so with minimal maintenance overhead, ensuring a reliable and efficient user experience.

In summary, Nutry's accomplishments extend beyond accuracy in data retrieval. Our sophisticated prompting system, coupled with a scalable chain of prompts, reflects a commitment to excellence in both user experience and technological innovation. Nutry is not just a snapshot of current success; it's a dynamic platform poised for continuous improvement and future growth.

What we learned

We learned about different AI API parameters such as temperatures, top_p, top_k, repetition_penalty, stream_tokens, stop_tokens, and negative prompt. We learnt a lot about LLMs in general and how we are structure our prompt in way that would sometimes eliminate the need to modify the parameters.

What's next for Nutry

  • We plan to utilize a vector database to enhance user optimization.
  • We plan on fine-tuning the model for each user's need to get personalized results.
  • We will implement a feature to filter meal information by dates.
  • We aim to visualize personalized nutritional information in various formats.
  • We intend to break down the process into concurrent API calls rather than using a single pipeline.

Built With

Share this project:

Updates