Cook With Me

CookWithMe makes cooking fun and interactive with a hands-free, visual, and voice-guided cooking companion. It's like cooking with your mom on a call!

Comment

🍳 CookWithMe

📊 FACTs:

  1. 81% of Consumers Cook More Than Half of Meals at Home in North America (source)
  2. 56% of people surveyed in a recent poll say they struggle with even the most basic recipes (source)
  3. CBC reports that only 35% of Canadians learned at least one new recipe between 2019-2022 (source)
  4. 63% of 2000 Americans state they need help in the kitchen (source)
  5. 74.5% of international students & immigrants surveyed were experiencing food insecurity (source)

👩‍🍳 REALITY & INSPIRATION:

We are 4 international students/immigrants who:

  1. Cook most meals ourselves, struggling to find the right taste and face health concerns.
  2. Struggle to make even the most basic recipes.
  3. Just 1 of us learned a new recipe in the past 6 months.
  4. Obviously need help in the kitchen!
  5. Face food insecurity (a challenge for every immigrant!).

CookWithMe was designed to make cooking accessible, engaging, and gamified. We allow users to follow recipes hands-free using voice commands via tedAI (we named it!). This lets users follow the steps without having to touch or scroll through the screen.

Features:

  • Hands-free cooking experience.
  • No intimidation of complicated recipes.
  • "No water, No problem!" – No ingredient? No need to rush to get it!
  • Support throughout the cooking journey.
  • Users can listen to instructions through headphones or a speaker, boosting confidence in the kitchen.
  • An engaging, interactive cooking experience (It’s like talking to your mom on how to boil an egg!).

The How:

We built CookWithMe using React for a simple and smooth interface and Flask for a lightweight backend. OpenAI’s GPT-4O-mini powers the smart recipe guidance, while the Web Speech API adds voice to transcription & voice control for hands-free cooking. Amazon EC2 hosts the platform for reliability, and Axios handles quick data exchanges. From connecting EC2 to S3 to crafting dynamic prompts for recipes & conversations.

Challenges Faced:

  • Streaming for multiple users.
  • Using React for the first time and writing 3k+ lines of code.
  • Linking EC2 with the S3 instance.
  • Formatting a smart query to generate personalized results.
  • Integrating DALL-E for generating images.

Built With

Share this project:

Updates