The Snakeman

• 

  modified facemesh

• 

  snakeman trio

• 

  snake mask with texture

• 

  patches for neck motion

Inspiration

I wanted to create a virtual character that could be used as a sort of puppet. The snakeman started out as a simple lizard skin mask effect but evolved into something more complex as I learned more about SparkAR.

What it does

The Snakeman's facial movements are controlled by your face. He always looks at the camera and writhes about in snakey fashion.

How I built it

I created a custom blend shape target to make the facemesh look more like a snake man. I built and rigged the neck in Maya and added the eyes and fangs. To create the texture I built a custom effect stack in Photoshop and drew scales to fit the face using the facemesh UV map as reference. I used the facetracker as the root of the skeleton to keep the face centered as the main focus of the effect. I used the patch editor to combine two types of motion for the neck, one to rotate the joints opposite to the facetracker which keeps the neck on-screen, and a second sine wave motion to give the neck some wiggly character. Finally I carefully lit the scene and added a colored rim light to accent the effect.

Challenges I ran into

It took me a while to figure out why tiling the neck texture wasn't working properly. It turned out that it had to be a power of 2 texture. It also took a fair bit of experimentation to find a setup where the neck would stay on screen when the user's head was rotating. I tried a lot of variations on textures and I wasn't able to get satisfactory results with normal maps. In the end I just used some specularity to give a more subtle effect. I was originally using eye tracking but then I realized it wasn't supported on Instagram so I cheated by making the eyes rotate opposite to the facetracker to get them to look at the camera.

Accomplishments that I'm proud of

I'm quite happy with the way it turned out. For one of my first SparkAR projects it has been very well received. It really feels good when I see videos recorded with it where people are getting into character.

What I learned

I learned a lot about the pipeline from Maya into SparkAR. There is a very specific format you need to use to transfer blendshapes correctly. The patch editor is powerful but there are still many things it can't quite do.

What's next for The Snakeman

I'd like to try a similar effect where I create an entirely custom model and drive blendshapes using the facetracker output.

Built With

• maya
• photoshop
• sparkar