Greetings! I am Shubham Jain, a 22-year-old from India. I have graduated with my Bachelor’s Degree in Engineering and have been enrolled as a student for doing Masters in Computer Science at North Carolina State University, USA. I would like to thank you for providing me with this opportunity for participating in this AR-Hackathon. My passion is coming up with creative ways to express a message, and innovative ideas that people share on social media. I have a personal interest in Augmented Reality, Artificial intelligence and I like discovering about the concepts of astrophysics as a hobby.
During this pandemic, we all are stuck between four walls, missing on what lies outside. How about an idea where we could visit some place outside these walls? Well, we might be able to achieve that soon when this pandemic ends. However, how about a place that can lie only in our imagination, with no means to reach there? How about Mars?
I really got inspired by two gentlemen, Mr. Mark Zuckerberg for doing everything he has done in recent years for creating AR-AI related applications and secondly, Mr. Elon Musk, the man who is on a mission to colonize Mars by 2050.
Personally, I have always had a hobby of learning about Astrophysics. So, when I got mail from Facebook that they are conducting an AR-Based Hackathon, it really inspired me to create something that would actually transfer people into the surroundings of Mars and make Instagram Reels there. So, thus introducing "Of Course I Love Mars" Instagram filter for Instagram Reels.
What it does
The filter contains following features built in consideration for optimising the same on Instagram Reels:
Panoramic 360-Degree World Experience: This filter basically immerses the user into completely new surroundings of Mars planet thus giving its users an amazing out of the world experience. As the user moves, the background in the filter moves accordingly or in other words, it gives user a panoramic 360-degree world experience.
Astronaut Helmet: The filter augments the user’s reality by adding an astronaut helmet on the user’s face which moves and responds according to user’s head movement.
Animated Drones: If the user taps on the drones present in the scene, the drones change their location thereby creating a user-friendly experience.
Dust Particles: The filter contains dust particles throughout the screen to give users World AR immersive exposure.
Background Audio: This filter has a background audio associated with it which can be heard whenever user operates the filter.
User Voice Modulation: It has an interesting feature where the filter basically modulates the user’s voice if he/she speaks whist recording the video using the filter.
How I built it
The filter was developed in the Spark AR Studio and its whole idea of build and development for the features mentioned above are as follows:
Panoramic 360-Degree World Experience: This feature was accomplished using a 3D sphere which I created by myself using Blender Software Tool. Moreover, I added the image of Mars surroundings as textures in the sphere to create panoramic effect.
Astronaut Helmet: I used Face Tracker feature in Spark AR Studio to create this effect. As the name suggests, it basically tracks the face of the user using the filter.
Animated Drones: The drone movement was basically done in two parts. The first, where I used animation playback controller to rotate the fan of drones. Second, where I created patches for the same in the patch editor so that as soon as the user taps onto the drones, they change their respective positions.
Dust Particles: The dust particles were added into the scene using Particle System in Spark AR Studio. I also edited their birth rate, material, speed and rotation to create the effect.
Background Audio: The background audio effect was added using the Audio Playback Controller and Speaker Asset of Spark AR Library. The audio was also imported from the same library.
User Voice Modulation: This effect was added using the Patch Editor where I took the input from the user’s microphone and provided it to Distortion patch and then finally sending it to the speaker.
Challenges I ran into
The challenges I ran into are as follows:
Difficulty in Background Picture Quality: This was the major problem I faced since the texture on Spark AR’s Sphere which I used earlier was not giving a proper user experience. Thus, I resolved the same issue by creating my own 3D Sphere inside Blender Software and gave it two textures instead of one to improve quality.
Face Tracker Problem: I had a hard time while adjusting the astronaut helmet properly on the user’s face which I resolved eventually by taking help from Spark AR Tutorials.
Accomplishments that I'm proud of
The accomplishment that I’m most contended about is that I have finally learned how to immerse users into 360-degree panoramic view with face texture segmentation. And it is truly an honour to create a filter which can be used for something as good as Instagram Reels that can truly embark users to new journey and possibilities.
What I learned
There were a lot of new things that I learnt during the project:
- Learnt how to immerse users into 360-degree Panoramic experience and using Blender Software.
- How to do proper adjustment of the face tracker and face mesh.
- Animating and Moving 3D Objects.
- Adding Custom Instructions.
What's next for Of Course I Love Mars Filter
I have thought for developing the same effect filter for more different surroundings like Hilly Regions with Snowfall, Beaches, Deep Water & Sky Diving Experience. I believe that such an effect like “Of Course I love Mars” has endless possibilities if we want to create something new and innovative for Instagram Reels.
Thanks a lot for taking time to review my submission!
Try this Effect on Instagram: https://www.instagram.com/ar/1248659558804298/