Hot Spots

Hot Spots is an immersive exercise game for kids, designed to make getting active fun, rewarding and engaging. Join Fit Fox for awesome daily challenges.

Comment

Inspiration

When coming up with the concept for Hot Spots, our team was looking to create an Alexa voice game that would make exercising fun and encourage kids to make exercise a part of their daily routine. We felt that by turning exercises into fun challenges, this would give kids a reason to want to return to use the Skill everyday and stay fit and healthy.

With the pandemic, so many parents have found themselves stuck at home and looking for activities to keep themselves and their children entertained. We hope that Hot Spots can make exercising in the home more fun and social and give parents something to keep their little ones busy.

Finally, we felt to make this a truly engaging experience, we wanted to utilise Alexa's multimodal functions to provide a rich screened experience for Echo Show users, with animated characters providing visual prompts for exercise.

What it does

Hot Spots provides an immersive and engaging Voice experience for kids to get fit and active.

Hot Spots allows the user to team up with the awesome workout buddy, Fit Fox, who will let you choose the workout difficulty level. Complete your warm up, main circuit and cool down with the help of Fit Fox, who will explain how to perform each exercise and offer your encouragement as you go. Each exercise is based on a 'TABATA', where the user does 20 seconds of exercise, 10 seconds of rest and repeat.

Learn how to exercise and get moving while having fun with this rich audio (and visual) experience.

How we built it

Planning

Initially, we used the white-boarding software 'Miro', to workshop ideas for the project over Zoom calls - considering the new Alexa APIs, goals and awards of the hackathon itself. Eventually, we decided to build a beyond voice skill for kids which would be an on-the-spot exercise game called Hot Spots.

We then mapped out the different states and the flow of the conversation on the whiteboard, and worked out how users would get from one state to the next (the golden path) - this formed our intents & slot values - our Alexa language model.

Developing

When the language model was planned, we created the AWS Lambda functions and DynamoDB tables needed (to see if a user was returning or not); created a code repository for the Lambda-backend part of the skill and the on-screen web app. Developers could now use IDEs to add interceptors, handlers and utilities to the skill, in CommonJs JavaScript. We used deployment pipelines from our git service (BitBucket) to deploy the development and production branches. The web app used the same method to deploy to an AWS S3 bucket. Predominantly, the skill uses the Alexa SDK for Node.Js and our framework for managing conversation state and persistent user attribute storage.

Design

More artistically-minded members of the team set about coming up with the imagery for 'Hot Spots', and red and hot pink colours where chosen for the theme. The logo was created, which is used in the skill icon and in the welcome video. The main character 'Fit Fox' was also designed in Blender and animation loops added to the model.

Organisation

Notion was used to organise the content for the skill - the different exercise movements, circuits, body regions etc. Notion was really helpful and we used this to write responses before they were translated into JSON APLA responses. Once everything agreed, the content was added over to the skill repository and utility functions created to choose an exercise routine based on users' preferences.

Web app

As we were designing for voice, the logic of the skill was placed in the Lambda function code rather than the web app so that we could create one experience for both screened and voice-only Alexa devices.

Using THREE.js, we placed the model into the web app and bootstrapped functions around it so that we could control the character's animation state. Being able to do a lot of the coding outside the Echo Show devices, and manually injecting Alexa.Presentation.HTML events into the app was beneficial as we saved time not opening the skill frequently to test a development change.

At the appropriate times, the skill, using the Alexa Web API for Games, emitted a stream of animation states to the web app, including duration and delay times. This meant we could send an event which contained all the details for one circuit - multiple animation changes after set periods of time. The actual delay values were fine tuned once the web app was running on the device. RxJs in the web app JS code was chosen to consume the events from the skill and change the animation of the model at the appropriate timestamp; it also meant we could 'cancel' future commands if the user said 'stop' - otherwise the model would remain exercising rather than standing idle. This approach also means we can send more than just the animation state data to the skill in the future - but commands to update the UI (a countdown timer for example) etc.

Challenges we ran into

As mentioned in the Alexa Web API for Games API release notes, there's a bug in the renderer in THREE.js used on Echo Show 1 and Echo Show 2. Unfortunately, our attempt to fork the open pull request didn't work on the Echo browsers in the end and we had to switch to plan B. The web app now detects if it is being displayed on an Echo Show 1 or 2, and switches off direct lighting and increases the intensity of ambient lighting. Although the picture isn't as rich as on the Echo Show 5 or 8 with direct lighting enabled, Fit Fox still looks good and is more cartoonish.

We enjoyed using APLA, but found we had to manually trim background audio to a length for every different type of response: We have a background track that plays behind Alexa's voice and that needs to end at the same time as Alexa's voice (or just before), otherwise Alexa will prompt the user for a response but the microphone on the device will not be listening yet, making a poor user experience. Where we used an APLA Selector component with randomItem strategy (to provide a variation of responses), we ended up placing the selector inside a Sequencer component with a long Silence component after (acting as padding). The whole Sequencer item was trimmed to a particular length, this way we knew that any variation spoken of a sentence would overall, take the same duration, making trimming the background audio easier and allows us to have background audio behind random variations of Alexa's voice. Having a 'receding' type of APLA component which is only playing where there is another non-Silence component playing would be really useful - or something like that.

Accomplishments that we're proud of

This is the first time any of the team has used the Alexa Web API for Games in an Alexa Skill, and the first time using a WebGL model outside of a standard Web browser. It was great to involve a diverse group of colleagues from different departments, giving exposure of Alexa development across the team. Further, it was great to build a voice app as a team and working on the same skill at the same time; development on voice apps has often been very linear in the past. Voice design and development courses have now been added to our internal learning academy.

We're proud of the visual identity of the skill and icons and we're proud of creating something that parents as well as their kids can use every day, as user retention is really important in voice apps. We're tremendously pleased about how good it looks on Fire TV, on the big screen.

What we learned

  • Work can be split up into more people - traditionally the developers were the ones creating audio responses, but as multimodal responses are now separate entities, sound designers or creatives can now generate the responses in the Alexa dev console themselves.
  • Being able to focus on visuals in skills rather than the voice design like we have done on other projects.
  • How to build voice apps with a visual HTML component.
  • Some team members also learned about RxJS, to consume the events emitted by the skill's backend code.

What's next for Hot Spots

We want to introduce rewards for completing daily challenges. The points you earn will go towards unlocking cool badges to give users an even better reason to come back and play each day.

Logging the time a child spends exercising each day could be sent to the parent's smart phone using a card in the Alexa app for future reference. With this data, we could track a user's progress and anonymously compare them against their peers.

There's also more logic we could add to the skill - for example comparing the request timestamps to see if the user has actually completed 2 minutes of exercising before rewarding them with points.

We plan to give the user more choice by introducing new characters, with Racing Rabbit and Super Squirrel set to join the Hot Spots crew soon and already modeled in Blender. Characters could also converse with the user instead of Alexa.

Along with more characters, we plan to keep adding more exercises to keep the skill fresh and interesting for the user and keep the user learning new exercises and ways to stay fit and active. This brings in the potential for ISP in the future.

Built With

Share this project:

Updates