Sprig

Inspiration

Every college student knows the feeling of being so overwhelmed that cooking becomes just another task on an endless to-do list. For adults balancing careers, families, and other responsibilities, that challenge only intensifies, especially when there’s no dining halls to readily serve you. Even worse, much of contemporary food production is tied to practices that negatively affect the environment, with food production accounting for 26% of human greenhouse gas emissions. Since many people are too busy to think about the environmental consequences of their food choices, we decided to create Sprig, a website that scans users’ fridge, identifies available ingredients, and returns yummy recipes! Sprig then identifies the most environmentally friendly sources for missing ingredients a user may need.

What it does

Sprig revolutionizes sustainable cooking for the masses. All the user has to do is upload an image of the inside of their fridge. From here, Sprig analyzes the image to extract all the ingredients in the fridge, and returns a list of recipes the users can cook from these ingredients. Simultaneously, Sprig identifies any missing ingredients that would complement each recipe and suggests stores to buy these ingredients from. Each store is rated by a sustainability rating. This rating indicates the store’s environmental footprint and alignment to correct social practices. When designing this project, we prioritized user experience over all else, ensuring that Sprig simplifies the cooking experience for the user in order to make it more enjoyable.

How we built it

We used React.js paired with Vite.js and CSS for styling in order to build the frontend. We connected this frontend to a Flask backend. The image ingredient evaluation is done through a Dedalus Labs MCP server with OpenAI’s gpt, and recipe generation is handled through the Gemini API. Once we had the recipe generation, we used the brave-search MCP and exa-search MCP to get new articles relating to different stores’ environmental impact and do a sentiment analysis. The sentiment analysis decided the sustainability rating of the store. Afterward, we utilized Dedalus again to get the links to a missing ingredient’s product page so we can display it in our marketplace.

Challenges we ran into

Though our team was able to decide on a clear idea for our project, we had to go through many iterations of our website until we all agreed on a design that was functional and user-friendly. This took some time and teamwork, but we are all proud of our end product. Further, many of the prompts we inputted into Dedalus were giving unsatisfactory results, so we spent some time refining our prompt engineering skills. Another challenge we faced was our strategy for identifying the fridge contents. Our original intention was to analyze a user’s input image of fridge contents using computer vision. However, our initial model, Roboflow inference SDK, required significant image preprocessing, returned inaccurate results, and recorded extraneous information (boundary boxes, object coordinates, etc.), which reduced functionality and processing speed significantly. Consequently, we pivoted to using Dedalus Labs as an MCP server for OpenAI’s gpt-4o model, which has a more generalized knowledge base and the ability to output JSON object ingredient lists, which was perfect for our project.

Accomplishments that we're proud of

We’re proud of the sleek and modern frontend we created as a team of 4 non-design-oriented developers. We are also proud of the incredibly high accuracy of our image classification model (that surprisingly outperformed our human vision in identifying certain ingredients). Lastly, we are proud of our superb communication and teamwork that allowed us to create a functioning prototype of Sprig in such a short amount of time!

What we learned

We all learned many tips and tricks to implementing AI agents in projects. Through Dedalus Labs, we were able to implement AI deep research agents, perform sentiment analysis,and evaluate how companies impact the environment. We also learned about proper website design UI/UX concepts through discussing our React frontend product. By working collaboratively with React and Django on Sprig, we were able to enhance our knowledge of full-stack development!

What's next for Sprig

To make Sprig more versatile for users, we would love to extend Sprig to be a mobile application. This will allow users to directly take pictures of their fridge contents from their phones and get recipes and sustainable options on mobile instantly. In an ideal world, Sprig would be integrated into smart homes via cameras so that it would automatically update users with new recipes whenever they restock their fridges. Also, a feature we didn’t have time to implement is enabling the user to order missing ingredients directly from Sprig instead of navigating to an outside link first like Amazon or Walmart. By expanding Sprig into a seamless, everyday companion, we hope to empower people to cook more intentionally and live more sustainably, one meal at a time.