HACCS- Hackathon Application Cloud Computing System

Utilizing the Google Cloud API's we want to make a bot that would talk to you and you could talk to and play games with. The bot would detect your emotions and change it's behavior accordingly.

Comment

Inspiration

We were inspired to make this sort of robot by a funny commercial for a home assistant in Japan that acted more like a friend than a home assistant (Gatebox). The dynamic difficulty of the game being played was inspired by Resident Evil 4's dynamic difficulty system.

What it does

The bot takes in your voice as an input and utilizes the Google Cloud APIs to understand what you say and carry out a command based off of it. It will take into account the current emotions you are feeling and respond differently based on it.

How I built it

The entire bot was built in python, and it utilizes google's text to speech, speech to text and facial recognition. It reads in the audio, processes it and then selects a response that will be played back to the user.

Challenges I ran into

There were a lot of problems with the amount of time and getting the Google Cloud API set up. We have never used the Google Cloud API before and we had to learn everything, including webpage creation, audio recording, audio conversion and audio playing all from scratch at the event.

Accomplishments that I'm proud of

We are super proud of what we have made. We came in not knowing what we were going to make and now we have a finished product I am incredibly proud of. The program works very well and was very close to what we had envisioned it to be within the time we had.

What I learned

We learned an incredible amount about how to use the Google Cloud API and about python in general. I learned how to use pip to install libraries and how to utilize the command prompt to run a localhost webpage.

What's next for HACCS- Hackathon Application Cloud Computing System

The plans are immense for this project. We hope to add many more commands to make it more similar to a home assistant in nature such as being able to set timers and alarms to your google calendar and even just being able to tell the weather. We also want to implement the Google Natural Language API so we can understand the intent of the commands being said instead of looking for keywords like we do right now. We hope, however, to keep it first and foremost caring about your feelings to make the robot feel more compassionate than just a normal home assistant.

Share this project:

Updates