Remy | Devpost

mockup
Choose you recipe (or name a recipe you want)
Loading (AI process)
Ingredients list (click to choose brands)
Choose sustainable alternatives
Start!
Go one to the first step!
Continue
Finish
Design process

Inspiration

When building our app, we were driven by a straightforward idea: to stand out and spark interest. We drew inspiration from the small actions that lead to bigger changes, especially in sustainability. Our goal was to make a tool that doesn't just catch the eye but truly helps users step towards greener cooking habits.

What it does

Remy is an LLM powered voice assistant for the kitchen. The user can simply tell Remy what they're cooking (per URL to an online recipe or prompt it with an idea for a dish), Remy will interactively talk the user through every step of the recipe, allowing hand-free interaction. Additionally, as the user gathers ingredients, Remy suggests Migros’ sustainable alternatives, weaving sustainability effortlessly into the kitchen routine.

How we built it

At the core, LLM powers our back-end, deftly processing recipes, extracting ingredients, and suggesting alternatives, while adeptly navigating user prompts. Voice interaction? We leaned into the Web Speech API for flawless speech-to-text conversion, and chose Eleven-Labs to breathe life into the text, translating it back to guiding voices. The facade you interact with is sculpted using React-JS, housed comfortably on Firebase. And that touch of sustainability? It’s data-driven, with a nod to Migro's comprehensive products dataset.

Challenges we ran into

We often lost focus, wanting to do too much at once. Combining our work was challenging. Understanding and responding to natural speech, like knowing when to listen, was hard. Setting up the app for everyone to use gave us headaches. And figuring out the best way to handle recipe prompts took a lot of time and effort. Goal setting and delivering a specific result by a certain time proved to be a challenge.

Accomplishments that we're proud of

Seeing our first Proof-of-Concept come alive was a major win; it was proof our idea wasn't just a dream but could work in real life. We're incredibly proud of the originality of our concept. The way we divided tasks and maintained focus was spot-on. And bringing together everyone's contributions into one cohesive app is a feat we'll always cherish.