The tech is in your hands. Now Scribble.

Graffiti has long been celebrated as the art of the people, but not all of us agree with risking law and limb to color the streets. At the same time, maturing Augmented Reality (AR) technologies have made it easier than ever for anyone to access a virtual world of infinite possibility through their smartphone (even very low-end ones).

Scribble is an experiment in wedding the tool to the need to create the next big trend in popular art: painting surfaces in the world with AR graffiti that persist forever for everyone to see. With no legal repercussions and infinite expressive value, we envision Scribble to be an opportunity to nurture creativity, foster communities, and enfranchise everyone through the power of ARt.

The Tech

Scribble's core technologies are Apple's ARKit for AR capabilities and Wix Code as a heavy-lifting backend API and database.

Apple's ARKit provides a framework for iPhones that nicely abstracts-away the difficult computer-vision challenges required to build a 3D model of the world from a video stream. In particular, it allows the extraction of Worlds, Anchors, and Scenes from each AR session which can be serialized and stored in the cloud to later reproduce the same viewing experience for an arbitrary number of users.

We used Wix Code to implement this business-logic of storing and serving AR session assets. We initially decided to adopt it because it required no additional setup of a server instance (or the tedious process of setting up boilerplate code and installing packages). We were pleased to quickly observe that Wix Code's features were sophisticated enough for our needs, and even allowed us to implement our own token-based authentication middleware for validating access from the mobile app.

The mobile app using Apple's ARKit was coded in Swift and deployed for iOS mobile, and our Wix Code API and database logic was implemented in JavaScript (ES6) using a standard REST paradigm to interact with our database objects.

Brick Walls

During our project, we ran into a number of brick walls which impeded our progress. But as any graffiti artist knows, such walls are just canvases which one can probe, freestyle, and use to grow.

  • ARKit has many features that are technically sophisticated and, being rather new, are not well-documented. To figure out exactly what we needed to do to bend ARKit to our needs, we adopted a regime of enlightened trial-and-error which eventually led to our discovery of using Worlds, Anchors, and Scenes to persist a session.

  • We were hoping to use Wix Code's wix-users or wix-users-backend libraries for user authentication, but after some experimentation and inquiry with the Wix mentors it became apparent that this was meant more as a frontend sugar than a Login API. After recognizing this, we were able to quickly pivot and implement our own authentication middleware using Wix Code's database and backend JS capabilities. We were even able to implement security measures (salted password hashes) by importing a SHA-256 library.

  • During testing, we discovered a memory leak in the Wix Database Editing UI which caused the page to consume more than 5GB of RAM and crash in the course of displaying just a few rows containing 10s of MB of data. We suspect that this may have been our use of Text fields to store large amounts of base64-encoded serialized data; there might have been a better way to store this data. In the end, we were able to workaround this problem by running the webpage on a computer with more RAM.

Masterpieces

If brick walls are canvases upon which we can shine, we think that the last 36 hours have lent us plenty of opportunities to be proud of what we've accomplished, including

  • Successfully recreating AR sessions purely from independently saved data store in the cloud.

  • Successfully implementing logic allowing users to build upon past sessions, enabling a collaborative art environment.

  • Successfully implementing business logic for an API and database after using Wix for the very first time.

  • Gaining sufficient comfort with Wix Code's features to implement a custom authentication middleware purely in Wix Code.

  • Building something cool and having a lot of fun along the way!

The Adventure

We've sure hit the streets with this Hackathon; and as street knowledge goes, we've learned a lot:

  • Learning from our teammates about the Swift and the Apple development ecosystem after coming primarily from Android and React development.

  • Discovering how to use ARKit through a hands-on, experimental approach.

  • Learning about Wix's and Wix Code's many useful features for the first time (we were impressed by how easy it was to make our cool website at https://nowscribble.com in so little time!)

  • Learning the best places to sleep in the SOCH after all the couches have been taken ;)

Scribble on the Streets?

We, the Scribble Team, feel genuinely captivated by the possibilities presented by Scribble. We feel that our proof-of-concept app already captures the fun in being able to make persistent modifications to the world in AR; with just a bit of polishing, we think our app can bring real joy, expressive enfranchisement, and community bonding as people collaborate to create a rich world of virtual ARt. With the right amount of dedication, we think we can turn Scribble from a hackathon project into a real tool for social good.

Draw anything. Anywhere. Together.

Now Scribble.

Share this project:
×

Updates