Bad Apple but Google Calendar

Inspiration

We took the second coolest (first being anti-money laundering AI but that was invite only) Google Cloud product that we could find and tried to use it in the most unorthodox way possible. This ended up in sending hundreds of thousands of Google Calendar API requests, getting rate limited hundreds of times, but eventually finishing with probably the best animated Google Calendar in existence.

It’s also been on our bucket list to contribute to the vast array of Bad Apple’s on weird platforms and HackHer413 while aiming for the Best Use of Google Cloud prize seemed like a perfect opportunity to do so.

What it does

Transcribes an MP4 into its image frames into pixel matrices which it then scales to 49x70, which it then splits into 7 columns (for each day of the week) and optimizes into chunks (we call them blocks) to then push the columns onto a Google user’s calendar via Google Cloud’s Workspace API.

How we built it

We first showed that our process could work by writing code that could translate black and white images into ASCII art. We then sliced videos into frames, scaled each frame down while keeping its prominent characteristics, and exported it to Google Calendar.

Challenges we ran into

Black and white videos are encoded in binary matrices, and these matrices are difficult to scale down because sometimes the scale factor does not divide nicely with our non-overlapping block processing technique. Individual pixel manipulation was limited until after using the Jimp library, which is mostly asynchronous and was difficult to work with when testing.

The biggest challenge we had by far, however, was the speed at which we could upload blocks with the Calendar API. Our original code had ~2.16 million blocks, which by our calculations would have taken 180 hours. After making many, many optimizations we brought it down to ~80k which we calculated would only take us 10 hours.

This didn’t include all the rate limits, however. Had Max, who was the one running the optimized code, just let it run with the rate limits we likely wouldn’t have finished. Our solution was to quadruple our speed by having four people stay up until 7:00 in the morning instead of just 1–we had three more people running it on their own google accounts and we ended up individually running the puppeteer code and splicing our frames together to create our final video.

It worked out in the end though. We got our video before the deadline and it didn’t even take 180 hours, just like 8 with all four of us.

Accomplishments that we're proud of

The video we made! We can clearly say we got Bad Apple running on Google Calendar and if nothing else we’ve made a good youtube video (and if you think otherwise you are wrong, politely).

We were able to fully develop code that is able to take the Bad Apple video and “translate” it into “pixels” which we are able to replicate using Google Calendar events. We are also very proud of the fact that we were able to implement this fully in just 24 hours.

What we learned

Image processing with both colored and black and white images via binary matrix or 3D array. Jimp library that provided the ability to work with low level components such as manipulating each pixel of the desired image. Scaling techniques for images. Puppeteer to screenshot the Google Calendar. ffmpeg to splice the original video and process the resulting screenshots. Bash and node scripts to automate a lot of the process.

What's next for Bad Apple but Google Calendar

We hope to increase the resolution of the video and develop better algorithms that accentuate image characteristics. We have also built the entire pipeline for colored video output, as we can modify the image from SRGB to luminance scales and dither to the palette of Google Calendar colors, the process of putting the events into Google Calendar made it impossible for us to put a colored video up for demonstration. We hope to add more examples in the future.

We can drastically improve the optimization using recurring events. The only reason why we didn’t implement it is because we only realized it four hours into uploading our calendars and since while it’s certainly takes the amount of blocks into a whole lower class of complexity