• 

  Thumbnail

• 

  Starting Screen

• 

  Built Pyramid

Inspiration

I've spent a lot of time playing with my one-year-old nephew during quarantine. If there's one thing I learned, it's that stacking things and knocking them down is fun, for anyone of any age. We decided to build an augmented reality experience to allow anyone to experience that nostalgia.

What it does

Our world effect provides an enjoyable simulator of a nearly universal childhood pastime. Build block structures that are limited only by the stretches of your imagination. Create a work of art and then throw a ball at it! Knock it to the ground! Take videos of your unique creations to share with your friends. BlockBuilder AR allows you to build towers with colorful building blocks, utilizing object physics to bring the creations to life. You can add blocks and change their colors. Grab and move them around by moving your device. When you're done with your creation, BlockBuilder features two destruction objects: a ball to throw at your tower from different angles, or a car that races toward your tower for a more dramatic destruction.

Our effect unlocks that lost childhood joy of building something simple and creative that you’re proud of, and gleefully knocking it to pieces! Whether it’s an impossibly tall tower that defies gravity or an elaborate and colorful pyramid, it happens within your real surroundings. And the best part is: there’s no cleanup!

How we built it

The glue that holds this project together is the Spark AR Physics library, which integrates Spark AR scene objects with physics objects built with the cannon.js physics library. This allowed us to provide magical AR interactions that follow the laws that govern our real world.

We started by creating our blocks in Spark AR Studio, and getting the physics library set up so that they can fall to the ground. We added our buttons to reset the scene, make new blocks, and turn on and off the gravity simulation. Since Spark AR does not include the capability to instantiate new scene objects, we created a pool of 25 blocks pre-added to the scene.

We then developed the ability to move the blocks around by selecting them. At first we attempted to do this with a pan gesture that calculated the direction in which to move them based on the angle of the camera. We changed this and opted for a physical movement of the device because reactive math made the first method difficult, and it is also more intuitive to use.

Next, we added our destruction objects. We added models for a ball and a toy car and the buttons to use them. The physics libraries allowed us to shoot them precisely and realistically. Finally, we utilized the patch editor not only for UI and audio purposes, but also for touch gesture interactions and simple animations that give our items more personality. It was an extremely useful tool due to the large scope of this project.

Challenges we ran into

A big challenge that we dealt with throughout the entirety of development was the challenge of binding all of the visible components and their invisible physics driven counterpart. It became clear very quickly that manipulating objects precisely while physics were enabled was going to be extremely difficult, so we made a design decision to do all the building before physics is enabled- which led to the implementation of the gravity button. This turned out to be a useful design change since we could make the blocks return to their pre-physics positions when the gravity button is pressed again, allowing for further editing.

Another challenge that we had to overcome was the fact that none of us had any experience with reactive programming, and figuring out how to do even simple logic with the Reactive module was a challenge.

Perhaps the most frustrating of the challenges that we encountered however was the fact that while oftentimes our code would work perfectly in the editor, it would completely break when running on our phones, much to our dismay. For example, getting the ball to stay fixed in front of the user before being thrown took several overhauls before it worked in any capacity on our phones, while almost every version worked almost flawlessly in the editor. Through many long nights of bug fixing and lots of teamwork we were able to iron out almost every bug and have the functional AR block building experience we dreamt of over a month ago!

Accomplishments that we're proud of

What we’re most proud of is the fact that we worked together as a team so effectively despite being separated due to covid. This was really the first time any one of us had worked collaboratively on a big project like this. Learning to use GitHub to make sure all of our additions merged smoothly was a huge learning curve, but we stuck it out and we were all able to flex our developer muscles.

We also worked hard doing detailed reading of the documentation, as there are few Spark AR Studio tutorials to be found online due to it being such a new technology, especially regarding advanced scripting. The Reactive paradigm was brand new to all of us, and combining our implementation with the non-reactive cannon.js was tricky. With tenacity and lots of snacks, we were able to persist and overcome.

What we learned

First and foremost we learned how to use SparkAR altogether. This was our first time using such a tool. We learned all about patch editor, the way it talks to the script, and when it should and shouldn't be used. We definitely learned a lot about reactive programming and how to make it work with the non reactive cannon.js. But most of all we learned that teamwork really does make the dream work. Or, maybe it was just tremendous amounts of caffeine.

What's next for BlockBuilder AR

When planning this project we had high ambitions for a social experience where you could build structures with your friends in real time. Due to the fact that the Networking was unavailable at the time of this Hackathon, we weren’t able to implement multiplayer, but it’s still something we’d love to add.

We’d like to add the capability to spawn different shapes, as well as new textures and even more colors. We are also planning to add an undo button and the ability to erase individual blocks. Lastly, more interactive destruction objects would make this even more fun than it already is!

Built With

• cannon.js
• javascript
• node.js
• spark-ar-physics
• sparkar
• webpack