YuMi playing the board game
A lot of children don't go out anymore to play with friends and often both of their parents are working. But you don't want your 6-year-old child to play online games with potential threats. That's why you need our "Player2". It will play with your kid and if necessary with additional human players. No more lonely evenings, where you wish that your friends could be here to play another round of your favourite board game.
What it does
"Player2" plays snakes and ladders with you. It will roll the dice and put his token on the right place. It can pick its token or the dice from any position on the board. It recognizes ladders and moves automatically to the next position. We tell it the number it rolled and it will move the token to the appropriate position.
How we built it
Using RobotStudio from ABB, we gave life to the YuMi-System. We could teach it complex movements such as rolling the dice or picking up objects. The overall framework is written in Python. We channel voice inputs through the Bing Speech API and analyse it with the language understanding intelligence service LUIS to control the game flow. We used Azures Cognitive services emotion API to find emotions in the opponents face and encourage him if he is losing.
Challenges we ran into
Our first plan was to recognize the numbers on the dice. Unfortunately, the probability was too low for a reliable playthrough. The different light in the room and in the testing area made it even harder. That's why we switched to voice inputs. Another big problem was the amount of teams interested into the challenge. There was only one robot available and the testing time was limited. Faster feedback would've helped us a lot during the weekend.
Accomplishments that we're proud of
The "Player2" can move tokens highly reliable. The dice roll never went from the board during our testing. The process is quite fast and we didn't have problems to recognize our voice commands. We developed an efficient way to move the robots' arms.
What we learned
Using the ABB RobotStudio and remote controlling the YuMi as well as using Azure for voice recognition. And even though we failedwith it, we got many new experiences in image recognition.
What's next for Player2JoinedTheGame
Start cooking... with YuMi
Start playing drums... with YuMi
Start dealing drugs under a shady bridge... with YuMi
Start teaching skiing... with YuMi
Start procrastinating... with YuMi