A huge amount of Reels are focused on dancing. We wondered: How can we make the expression of dancing in Reels more interesting.
We were inspired by motion amplification that shows micro movements in machines. The tool finds the moving areas, the direction they're moving in and enhances those movement to become visible.
What it does
While dance movements are already visible, we enhance them by extracting their directions and create different effects that dancers can use to make their dancing look more exciting.
Arrows, the first effect we created for this Hackathon
The filter understands which part of the dancer is moving up, down, left or right and translates those movements into the separated edges of the silhouette. After the SegmentDirection Tool was created we came up with four different filters that use that new function in fun ways.
Our ideas came from manga, kirakira and... Rihanna.
0. Line Ripples
The dancer's movements are translated into graphical ripples around them. They emphasize the current dance moves and also, by slowly fading away, show the past moves.
1. Energetic Steam
This effect is created to give more energy to the dancer's performance in Reels. The divided edge area of the segmentation balances the amount of steam and its direction.
2. Kirakira Trail
The fascination of kirakira effects is undeniable. We wanted to bring kirakira to the next level by making them particles that react to your movements.
3. When the Sun Shines We Shine Together
The dance moves create water that travels, as if it got momentum. So rather than just upwards "falling" water, the movement's direction is shown in the direction the water flies away.
How we built it
The core system of all effects, the SegmentDirection Tool, is based on the extraction of the segmentation alpha channel's travel direction. This is achieved by acquiring the adjacent pixels and the delay frame. We have generated segmentation alphas for each direction of movement and created a patch asset that can retrieve them.
A Patch Asset to split the segmentation we created
The SegmentDirection Tool allows you to create more complex segmentation effects that are based on the movement's direction. We created the four different effects to show the potential of this idea and its advantage for Reels filters.
The filters will be published separately, but for demonstration purpose we combined them into one. Eventual fps lacking doesn't happen in the separated filters.
Accomplishments that we're proud of
We started our team for the first Hackathon this year. Unfortunately, our black hole filter didn't make any place at that time, but the teamwork was perfect. We appreciate each other's talents and we are proud that we were able to participate in this Hackathon again with the same team.
This time we took the hacking part more literal and "hacked" a new function into Spark AR, while also focusing on making useful and fun filters. We hope other creators benefit from the SegmentDirection Tool and create beautiful filters.
What we learned
We've gained a better understanding of new features like delay frames and render passes.
What's next for „Motion Effect Kit“
With the SegmentDirection Tool it is now possible to track motions and their direction. It's an idea that's only possible because of the latest Spark AR Studio, and we think it's going to be the helpful for other creators. We'd love to see what kind of filters other people can create by distributing the patch. We also hope to create more effects that take advantage of this feature ourselves in the future.