Fireworks

Inspiration

I was surprised by some of the native apps on the Echo show 2nd Gen, like the Prime Video and Calendar and especially their touch support. I wanted to provide something more interactive as a skill that embraces sound, graphics and touch.

What it does

The skill shows a night city skyline and encourages the user to touch anywhere on the sky to set off a firework at this position. It fires of a firework targeting the position where the user touched (or very close one) and does that with visual and sound effects.

How I built it

This was one of the geekiest things I've ever done. Reading at the documentation I didn't find any components

I started with the sound. Combined couple of freely available sounds, I built what you can hear from setting off and exploding if the firework.
Then I decided to make pre-recorded fireworks. I knew that I had X number of positions I had to support, but how could I do it in a light and automated way? First I thought of editing videos of real fireworks, but that wouldn't be configurable. So, using HTML and javascript I was able to automate when and where I want my fireworks to appear and it was visually nice looking.
Then using some video editing techniques and new skills I've acquired to understand how can I squeeze the most out of the size of a video while still leaving it with fair quality.
After that the fun part started (or continued, so to say).
1. I started investigating how would be the best way to react to touch events from the echo device. I learned that the event will not tell me the coordinates where the touch happened. So I decided to split the screen into 10X5 touchable container covering the upper half of the echo device (all devices). So with that I had fairly close position of where the touch happened. I would've gone for more, but there are some limitations, and the APL renderer will complain with more than 120 touch components.
2. Now I was about to play a video of a firework for this position... but what would be the best way to display such video in Echo. Initially I split the video into 50 (10x5) shorter videos, each holding a single firework clip. But that had problems, such that between switching videos, you would see a blank screen for a fraction of the second, making it bad experience. So I went with the full video option, and using the seek and pause commands, so I could specify exactly which couple of frames/seconds of the video to play, that corresponds to the firing of the proper firework. So far, with a short delay, it turned out to be the best way to do it.

Challenges I ran into

Well, point 4. of my How I built it story

Accomplishments that I'm proud of

Well, point 4. of my How I built it story :-) Seriously though, I think the skill turned out to be nice, with some good effects. The coolest part really are the technical decisions I had to make to work with the constraints that I had.

What I learned

Making fireworks sound.
Making visual fireworks with HTML and JS. Then making super compressed video out of it.
Responding to Touch events in Alexa.
Playing media with Alexa.

What's next for Fireworks

If I can get a hold of the native engine that sits under the skills capabilities or something like a browser-compatible engine on the Echo Show, I would love to provide a native implementation for the fireworks, that would be smoother, faster and lighter.