About a year ago, I first saw the commercials for Amazon Echo. My immediate thought was, "Oh man, that would be so cool if it had an app for doing recipes where it could speak the recipe to you. I love to cook, and I use my computer to find recipes. is my favorite website to get recipes from. However, it's always really frustrating when you're hands are messy and you're about to drop in the sriracha, but you can't remember if it was 1 tablespoon or 1 teaspoon. Now I need to wash my hands, dry them off, enter my password in (since my screen has locked again, find my place in the recipe again, and then go back to getting my cooking. Then it's rinse and repeat for the next step :)

Since then, I've looked for apps that would solve this problem. Finally, I realized I could build it myself and found the Hackathon to be the perfect opportunity.

What it does

Sous Chef solves this problem by allowing you to have a dialog with the web app. Sous Chef understands simple voice commands like, "Next Step." or "How many _____?" and can talk back to you with the proper response. Now when you're standing in front of your bowl trying to remember how much sriracha to put in, just ask Sous Chef, "How much Sriracha?" Sous Chef will vocally respond with the amount you are looking for.

Sous Chef makes the process as simple as possible by also having an optional chrome extension. Found the perfect recipe on AllRecipes? Click the Sous Chef extension app and click, "Let's Cook". You're automatically redirected to the Sous Chef site and you're recipe is automatically loaded. You are immediately ready to start interacting with Sous Chef and start going through the list of ingredients!

How I built it

Sous Chef is built with a combination of HTML, Javascript, CSS, and PHP. Sous Chef downloads the DOM of a given URL with PHP. We then use javascript to parse through the DOM to give us a list of ingredients and steps. Finally, we use the Google Web Speech API and a wrapper called Annyang to give us voice command inputs and voice response outputs to the user. Finally, a chrome extension is built using a combination of HTML and javascript to make the web app incredibly simplistic to use.

Challenges I ran into

Our biggest problems were actually pretty silly. Javascript doesn't let you to get the DOM of a cross domain, even though there are a 100 other ways you can get it. This took up a few hours. We also ran into an issue where our app was working on our "test" server but not our "production" server. A few hours of painful debugging, turns out no one ever installed php on the "production" server. Ugh...

Besides that though, we actually had a really smooth process and were able to finish extremely early. I credit this to having a good plan from the outset and constant communication between the two of us to make sure we both knew who was doing what and what we expected from the other person's code.

Accomplishments that I'm proud of

I was really proud that after a year of wanting to create this app I was finally able to do it. It actually was amazing how quickly we were able to do it and makes me want to go to more of these in the future to work on the other ideas that I have.

I'm most proud that I decided last minute to tackle an entire area that I've basically never touched before and was able to learn it extremely fast. This really made me feel like I can take on anything!

What I learned

Up until Thursday night I had envisioned this as an Android app. However, I realized the day before the Hackathon that it would be much more useful for myself, and would be more universally usable by others if it was a web app.

So with that being said, I was really learning web development on the fly as I really never did much of it before. I hadn't done javascript probably since high school. Luckily, my partner Mike was well-versed so I was able to ask him a ton of questions. I love learning new languages and APIs so it was really exciting to take on a challenge that was completely new to me and I learned a lot.

I think we learned most from the road blocks we ran into. We realized that (for some reason that makes absolutely no sense to me) you can't grab the DOM of a remote domain via Javascript due to security issues, even though there are a 100 other ways you can do it. Every issue we ran into we were able to overcome though which is a very satisfying feeling.

What's next for Sous Chef

It really needs some field testing, which I'm excited to start! I would like to make using Sous Chef feel as natural as possible. That means, users can speak to it like they would if they were talking to an actual person. This is not a complicated task, I just need to figure out what someone would like to say to Sous Chef. Once I know how people would like to speak to it, it is trivial to match input commands to functions of the web app.

I'd also like to experiment with voice modulation to see if I can find a more natural sounding voice. The standard voices sound a bit robot-y and I would like to make the experience more personable.

We could also work to include other popular recipe sites to parse.

Finally, we would just be cleaning up some of the UI. The simplification of the recipe in the web app was actually an unintended side-effect, but when discussing with my girlfriend, that was actually one of her favorite features. We also feel like it adds a nice touch and it would be nice to dress it up just a bit so that the user feels like they're still getting a 1st class experience using our web app.

With these additions, I feel like we would actually have a MVP that we could setup on a real environment and actually host the web app. The chrome extension can also be included to make Sous Chef incredibly easy to use.

Share this project: