Shout Out Games

Voice games for Fire TV. Scan a code and your phone becomes the microphone, then shout, sing and talk your way through twelve games.

Comment

Inspiration

Shout Out Games started out as a web project, and we wanted to give players more ways to play, and the funny, crazy gameplay style of Shout Out Games suits a TV / group setting where you want to have a bit of fun together.

What it does

Shout Out Games allows you to play voice controlled games on your Fire TV. You scan a QR from the TV and use your phone as a microphone to interact with the games. Once connected, we send audio data the user is sending over, designing different game styles, which are:

Shouty Games: These games are played by shouting. And open the door to loads of crazy game ideas, such as

  • Scream Ball: Uses the shape of your sound wave to fling a scientist into the orbs.
  • Monster Dunk: Loudness flies your monster up, quietness down to dunk them into the basketball net.
  • Chicken Run: Loudness makes your chicken jump high and being quiet makes it move slowly
  • Zig Zag Scream: Navigate a maze by using loudness. High loudness = Blob moves quickly.
  • Pop Drop: Shout at the right moment to make your rotating bottle land the right way

Harmony Games These games test your pitch and holding certain notes. We found these are fun but also have multifaceted functionality, for example people who want to train their voice and improve singing. Games include:

  • Sheep Squeeze: Sing high or low to squeeze the sheep through gaps,
  • Cyber Ball: Hold a certain pitch for a few seconds to get through the walls

Conversational Games These games simulate some offbeat conversational scenarios and cater for some unique and creative gameplay. For example:

  • Leaning Tower of Pizza: Tell lies to make Pinocchios nose grow, to press the big red off button to stop the evil pizza robot!
  • Alien Customs: Talk with an alien customs officer about your luggage while travelling into Jupiter, convince him to let you in!
  • The Imposter: Listen to 4 secret agents say a word relating to a certain topic and try blend in by saying similar words.

Quiz Games Classic quiz and memory style games.

  • Beat The Buzzer: 5 True or False questions including a daily challenge and category selection. get 5 in a row to win.
  • Bing Bong Banana: a memory game where you have to recall the sound associated with certain words.

Chicken Run, Monster Dunk, Pop Drop and Zig Zag Scream have a sensitivity setting on screen: press the minus or plus button with the remote to change how loud you need to be. The game remembers it.

The first time you open the app, you pick one of four teams: Fire, Water, Electric or Grass. The TV then makes an anonymous account for itself, with no email, password or login, and every coin you win counts for your team on the team leaderboard at shoutoutgames.com.

How we built it

We built Shout Out Games as a Fire OS web app: the lobby and every game are plain JavaScript, HTML and CSS, packaged in a Kotlin WebView. On launch, the app opens a room on our WebSocket server, which the phone uses to connect to the TV.

Players scan a QR code to open a web page on their phone. The page listens with the browser's Web Speech API and sends what the player said straight to the TV over a WebRTC data channel. On the TV, handlers pass each answer to whatever the current game is doing.

Some games use an LLM. In Leaning Tower of Pizza it judges your lies, and its reactions are generated at a high temperature so they stay funny and different every time. In Alien Customs, the alien's replies are made on the fly: an LLM decides what the alien should say, and ElevenLabs turns it into speech that plays on the TV.

We have logic to parse string input, using standard techniques such as synonyms, fuzzy match and some custom techniques to attempt to match user input to the appropriate intent.

Challenges we ran into

We found that when playing in a room with people, having the mic always open on your phone caused mishears / unintentionally triggered the game. So, we made a button to open the mic. Then we discovered that pressing and holding the mic button was annoying, because you lose focus on the TV and start looking to see if the button is held and accidentally letting go would cut off your speech. So we figured the best way was to make it so you tap the button to open mic, and then tap again to shut it off.

From the UI side, we found a few strange quirks with Fire OS Webview that we overcame. Firstly, the Fire OS WebView drew our full-screen videos at their unscaled size, ignoring the CSS transform on the page, and left a black box when a clip ended. So we removed the transform and added a css rule that gives the video a 1920x1080 box at 0,0 with no transform. We did a small friction log here: https://gist.github.com/hariz-nour/8c2f588f95a591ba70a4ddd92e4e5a95

When using LLM for reasoning in gameplay logic, we found it can produce some latency. For example in Leaning Tower of Pizza, it evaluates your lie and then produces a funny custom response based on the reply. So we got creative to "hide" the latency. In LTOP we made Pinocchios nose grow slowly before the API callback finished, to simulate instant response.

There were some occasions we were requesting audio focus from a game, while it was active on the web app already which caused microphone to stop working. We fixed by architecting the solution to request permissions at the top level.

Accomplishments that we're proud of

Not related to development exactly, but one of our team was babysitting a girl and she wasn't having a great day. She tried Shout Out Games and loved shouting into the microphone which let off a lot of steam! It took her mind off things for a while and put a smile back on her face. Hearing that meant more to us than anything else on this page.

We were really happy to get the 12 games into a single package at only 80MB size. Now we have the infrastructure in place, scaling to add more content should cause issues in terms of the size of the APK.

What we learned

We learned a lot about voice user experience when it comes to playing games on a TV. Firstly, In the loudness games, like Chicken Run, the chicken kept running for seconds after the player went quiet. The levels were queuing up. The phone kept sending audio while the link was backed up, the relay had no limit, and the TV played the backlog late.

We figured out that we had to make it so every step keeps the most recent packets: the phone skips one when the link is busy, the relay drops them when the TV falls behind, and the TV throws away anything more than 250 ms late.

What's next for Shout Out Games

We would love to investigate using some technology such as voice ID to differentiate users to prevent mishears during gameplay (ideally it would identify who is speaking).

Next is making Shout Out Games somewhere players come back to regularly and build up a profile. We would like to connect their Amazon account so they can track progress. So they can earn coins, badges, ranks and so on. We want to make the teams more present throughout Shout Out Games so you feel really motivated to play.

Team

The team for this project consisted of 3 people, with our project leaders ideating the games and then our game developer with the implementation.

Built With

Share this project:

Updates

Submission history