As part of our research we looked at world filters that were already available on Instagram, Facebook, and Snapchat for some inspiration on how to support a large social good and/ or enrich communities. We found a filter by Maximkuzlin where users can upload images into a billboard and we thought this feature had the potential to be very powerful. When we have the user insert themselves into the filter it allows them to shape what story they want to be heard and makes them more connected to their creation. This became the core to our project. While we were sketching ideas, our team imagined that users should be able to place their images on a wall in their home or blow it up to the size of a building. We thought about how it is common during a wake for the family to display photos from the loved one’s life, and how this could be presented in a collage. A memorial collage became our target goal to achieve. Starting off small we thought we should plan for the user to upload 5 images and have 3 different collage layout variants to choose from.
Our team started writing some scripts to upload images to the filter, but we couldn’t figure out how to start. On the “Facebook Online Hackathon” Group on Facebook we asked the community where would be a good place to start looking for our answer. Thankfully someone responded and they pointed us towards the Gallery Texture documentation. It seemed intimidating at first, but looking at some tutorials it was actually a lot simpler than we expected. To our surprise and a happy turn of event the Gallery Texture also supported video formats which added another layer of storytelling which is at the user’s disposal.
The biggest challenge we faced when developing this filter was with Spark AR’s native Plane Tracker, which out-of-the-box is optimized for the horizontal plane. What’s quite excellent about Spark AR is that you can extend and modify the available functionalities of most things by writing code, and so we spent some time reading the documentation and writing our own custom JavaScript for tracking vertical surfaces (in this case, walls).
To do this, we studied how the SceneModule works, what classes it ships with and the methods it works with. We were particularly interested in the PlaneTracker sub-module, along with its getters and setters. The final piece of the puzzle was the TrackingMode enums, unique keys that set the focal point of the plane object.
For our project’s purposes, we wrote a piece of JS code that looked for the plane object and found its trackingMode property. It is writable, so we set its TrackingMode enum to ‘VERTICAL_TRACKING’, allowing us to bypass the default plane tracker and map our world objects to vertical surfaces instead.
Once we had our base of uploading 1 image and our vertical plane tracking we went back to our original sketch to include multiple images on different 3D planes in a collage layout. This was a more formidable challenge however. We weren’t getting reliable results from uploading the couple images we chose and the images were not going in the planes we wanted them to be in. Unhappy with our progress and time running out we begrudgingly had to drop the idea of allowing multiple images and by proxy had to drop the idea of a collage. We still had a very valuable base, but we just needed to adapt it a bit. Looking at artwork artists had made for George Floyd we decided we should instead frame the user’s image instead. We also found some interesting silhouette artwork and thought it can also be a very compelling way to frame a photo. Once we learned more about alpha masks we were able to achieve that silhouetted looked we wanted. With both influences in mind we made a handful of frame variants for the user to choose from. We were also able to find a way to keep the 1 photo user chosen and have it easily transfer to each of the frames. In the future we want to be able to have a wide variety of frames for users to choose from that fit the memorial style they envision.
Built With
- blender
- javascript
- sketchfab
- sparkar
Log in or sign up for Devpost to join the conversation.