My Twinning Avatar

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

Inspiration

My visual inspiration was movements of dancers in lush costumes.

An avatar mimicking a camera-detected dancer, who is virtually dressed in flamboyant costumes, which flare, flow, fold and move in reaction to both the dancer´s and avatar´s movements, creates entertaining snap opportunities.

My technical aspiration was cloth physics, especially making clothes react not only to the body of the person, who virtually wears the outfits, but also to the other objects in the scene, in this case to the avatar following the camera-detected person.

What it does

In this lens an avatar mimics a camera-detected person virtually dressed in lavish clothes.

Outfits are changed either hands-free by the camera-detected person by moving her hand approximately next to her head, or by the phone user by tapping on an outfit in the carousel.

How I built it

Lens Studio features I used include body tracking, cloth physics, physics body and colliders.

I made different flashy looking outfits with Blender using the Lens Studio provided full body mesh as the starting point for my costumes.

I used Lens Studio´s body tracking to make my outfits to be virtually worn by the camera-detected person and used the External Mesh feature to make outfits to deform to that person´s body.

I used Cloth Simulation and set it up to automatically bind Body Mesh vertices to the closest Cloth Simulation vertices.

In order to add the right amount of flare, flow, bounciness or stiffness to an outfit, I tuned the cloth simulation settings (mass, stretch, bend, gravity, repulsion, friction etc.,) of each outfit depending on the style I intended.

I chose cloth materials to match the style of the outfit.

Clothes of the camera-detected person react to the movements of that person with the use of colliders attached to her body.

In order to experiment with clothes also reacting to another object in the scene (other than the body of the camera-detected person), I created an avatar and set it up to follow the movements of the tracked person.

I created the avatar using Blender and Mixamo. I rigged the avatar´s outfits when I created the avatar.

I added Physics Body to the avatar's pants and shoes, so that when the avatar´s legs and feet are close to the costumes virtually worn by the camera-detected person, those parts of the outfits bend, bounce and fold.

Change of outfits are done either when the camera-tracked person´s hands are detected approximately next to her head, or when the phone user taps on an outfit in the carousel.

The materials of avatar outfits are changed, when the outfits of the tracked person changes.

Challenges I ran into

Making clothes on the tracked person to react not only to that person's own body movements but also to the avatar`s movements.

Making clothes on the tracked person flow, bounce and move in the style intended.

Making materials matching the style of costumes.

Accomplishments that I am proud of

Addressing the challenges described above.

What I learned

Cloth physics

What's next for My Twinning Avatar

Adding facial expressions to the avatar, improving cloth reactions, creating other stylish clothing materials and improving costume design.

Built With

• blender
• lensstudio
• mixamo