Roxy the Teaching Fox

• 

  Roxy the Fox

Inspiration

As a team of immigrants in England, we wondered about the struggles of people moving to a completely new environment and thought about how we could make their lives easier. We decided that we could really make a difference without destroying the confidence of users through actual conversational interactions with our platform, while also providing meaningful feedback on their grammar.

What it does

The platform provides the user with an interactive UI that they can talk to, as they would with a normal person, to improve their conversational skills. We used a 3D object in Unity, a fox to increase friendliness, to interact with the user. It starts by introducing itself and then goes on to ask questions with increasing levels of difficulty.

How I built it

We get voice input from the user, convert that to text, run grammatical checks on it, assign a score to the user based on their response, and return feedback along with a new question of appropriate difficulty. The fox then speaks the text that is returned by our backend server. We also made animations using Unity.

Challenges I ran into

One of the libraries that we were using (for text to speech) had certain errors in it, and we had to correct it ourselves. We will make a Pull Request on their repository to correct this later. Getting certain libraries to run in Unity was difficult since NuGet packages aren't natively supported. We ended up writing a Python server and integrating that with our app. Also, we were done with this at 5 AM, and decided it would be cool if this was in VR. Since we couldn't finish this by 10AM we reset our changes and started working on the demo instead. Given more time, this is what we would have worked on.

Accomplishments that I'm proud of

We are very proud of the entire application since none of us had worked on anything similar, and we believe that it would be actually useful.

What I learned

We learned about animations in Unity, and hacking parts of Unity to get 3rd party .NET Core packages to sort of work.

What's next for Foxy the Teaching Fox

We would add more supported languages, and create a VR version of the application.

Built With

• c#
• flask
• python
• speech-to-text
• text-to-speech
• unity