Inspiration

Communication is the key to languages.

What is this project about?

This program is an interactive display that aids users in learning languages more easily. With hotkey training from Snowboy.ai we have created a system that gives users prompts (via socket.io) and takes in user input through the microphone to analyze the appropriate responses, thus giving language learns real time feed back in pronunciation and visual practice prompts.

How we built it

We built this as an interactive display device with a pi, pi-cam, and mic. We used magic mirror's framework system that was set up with websockets to easily communicate between different modules (video prompt, mic input, and text output).

We ran into so many challenges with this project

There has been some difficulty with connecting to firebase for a server analysis, due to magic mirrors electron wrapper for their program packaging. In the future I think I would consider just setting up my own socket rather than using this framework --further testing in this area is needed.

What we learned

We really liked the magic mirror framework because we got to use both python and javascript. Experience calling python-shell through the node server was really interesting and how to pip the installations needed for the packages was important since its behavior is a bit different than npm modules. In addition, the voice recognition training process (and the image rec) training process was really interesting and most of our first time using. Rather than using apis that have previously learned them we got to experience training our models and seeing the process of how they work.

What's next for Google2Point0

Implementing more languages but this is not going to be just used for languages, it will be used for even physical sports like Taekwondo, Gymnastics, Dancing, and more. We continue the deep learning process to implement all of these different features. This is the first step into something completely amazing and anyone no matter what walk of life, can use.

Built With

Share this project:
×

Updates