WeatherHERE

You can change the weather where you are. As if you had a superpower, you can express your vibe with sunny, rainy, or snowy AR World.

Comment 2

Inspiration

When many people start a light talk, they usually mention weather right now. It's a good start to open chat with friendly feeling. Furthermore, we have accustomed to communicate with a messenger and video chat, so if it were available to share a sort of visualized weather information with Facebook friends vis messenger, it could be open another fun user experience. In addition, I would like to learn and test how to visualize Internet accessed database, so I chose GPS-detected weather visualizer with SparkAR studio. Real-time data-visualization by augmented reality technology will be more common within a coming decade, so I believe this experimental prototype could be a sort of seed to dig such a possibility.

As we all know, COVID-19 has gone viral right now, if a certain statistics of COVID-19 tracking can be connected to pretty high-performance mobile AR platform, it will be so helpful and useful AR use cases. If this WeatherHERE structure can be expanded and scaled up, this will be a promising concept to help people to live better with augmented reality.

What it does

This is basically a World AR effect, which means that it uses BACK camera.

1. Enter the effect

  • Transparent cylinder shows up

  • You can scale up and down, rotate CW and CCW, and put it on any plane by dragging the cylinder here and there.

  • A default BGM music is played

  • A dancing character starts moving

2. Click the 3D text "Click Me!" in the cylinder

  • The current city and temperature show up

  • BGM is changed to 'Sunny day ...'

3. Click through till you are back to the status #1

  • I designed 5 different weather options and cities
  • I am going to make all options detected by GPS and pulled weather data (network module is needed)
  • Currently, this is a prototype

4. Finally, a hand tracker trigger an additional users' GPS position indicator cloned from the initial cylinder.

*** The following demo video was captured from Spark AR player, because the Spark AR preview link doesn't work with my Galaxy S10. An error message popped up like this : "This effect is not supported on your device."

*** My Facebook app is the latest version. The OS version of my GalaxyS10 has also been updated in Jan 31, 2020.

*** I expect that Spark AR tech team helps me to resolve this issue.

How we built it

First of all, we conducted research about available open-source weather API or framework. These are the resources we found.

[https://openweathermap.org/current] [https://openweathermap.org] [https://www.spatialtimes.com/2019/01/Create-a-JavaScript-Weather-App-with-Location-Data-Part-1/] [https://enlight.nyc/projects/weather]/

Unfortunately, however, a developer in our team, Jong-il SEOK has come to know that there were Weather module and Networking module in Spark AR, but they are not available now. We did not know the face before starting this project. Anyway, we changed a gear to make something happen, so this version of prototype was completed.

A dancing character and fbx animation were imported from adobe mixamo. Current city and temperature were visualized by Dynamic Text. Weather vibe uses three different LightLeaks png image designed by Photoshop, and they are blended by 2D canvas in our Spark AR scene. There are two particle systems to visualize raindrops and snowflakes by setting parameters in the inspector of Spark AR. All BGM are used from Facebook creator library and edited by adobe Audition to create a short loop audio play.

Challenges we ran into

As we already mentioned above, networking module is necessary, but it has stopped to service. If the module were available, we believe that we can more robust and usable SparkAR effect by connecting a genuine weather data tracked by user's position. Please reopen the module as soon as possible. We sincerely hope to develop more tangible version in F8 2021 after winning a prize :)

Accomplishments that we're proud of

World AR effect can make a vibe more incredible and imaginary because the field of view is wider and higher. Please watch WeatherHERE instruction video ([https://vimeo.com/397917464]). As you can see, a user could experience pretty realistic rainy and snowy vibe with his/her own smartphone. Additionally, BGM and sound effect are so important to deliver invisible feeling and atmosphere. We tried to choose pretty suitable music for making multimedia experiences.

What we learned

We have never experienced to create a Spark AR effect for Facebook app. In our opinion, Spark AR has not yet been stable for an Android phone, specifically, a Facebook app installed in an Android phone. According to global market share of mobile phone, Android users are more than iOS users. The strength of Spark AR effect for Facebook is to switch users' camera to AR filter during their messenger chat easily and seamlessly. We believe this is pretty cool feature. Though Spark AR platform has been updated for Instagram much more, a sort of integration between AR effects and Messenger experience should be refined much more, so the more reliable working can be adopted to Portal as well.

We are going to hack more for such an integrated user experience from this project by focusing on Spark AR for Facebook. For this, Facebook friendly Spark AR developer tool is needed.

What's next for WeatherHERE

WeatherHERE can be developed to two kinds of ways step by step.

First, we are going to add more fun factor. Specifically, we will attach a Facemesh to the head of the 3D dancing character for a users face to be overlaid to the character. The hand tracked 3D transparent cylinder is being interacted by the user's hand even though he or she uses the front camera, so the user's facial gesture can be synced to the face of the 3D character. The user can see his/her dancing character in the front camera AR view with his/her hand tracking. In addition, we would like to add various face filters according to weather conditions. For example, a user face turns to ice when it is snowy day and to sweaty look when it's so hot.

Secondly, technical update should go on. We are supposed to attend F8 2021 onsite hackathon because it is the only chance to use resources and tools that are beta or alpha version. As a sort of exclusive opportunity to get support from Facebook/Oculus/Spark AR team, we really really hope that we can win the chance to be there next year. When it comes to Networking module, we have to say that it is necessary to connect gathered or crawled dat to a Spark AR scene. If it is possible, we can make much more usable one. In a situation COVID-19 pandemic, we hope that our friends connected via Facebook can share their safety with their own Facebook camera as like Facebook safety check feature. If we can connect global COVID-19 statistics to this AR scene, this concept might be more useful and helpful project for social good.

Built With

Share this project:

Updates