Kinetic Head

Inspiration

I was inspired by the ability for AR to make mundane experiences immersive and fun, and saw an opportunity to apply this to learning anatomy - a field difficult to comprehend through 2D mediums, that can put off young or life-long learners. This project speculates about the potential for Augmented Reality to serve as a fun, immersive and accessible medium for educational experiences.

What it does

Kinetic head provides 'layers' of cranial anatomy superimposed on the user's head, available to navigate through a slider and understand through a head tracked labelling system. For the brain and skull, it provides orbit-able 3D models for users to visualise the entire geometry.

How we built it

Kinetic head was built mainly on top of Lens Studio's head tracking, using available face masks to super impose muscle tissue and facial arteries, and a 3D model of the skull and brain that leverage head tracking. For the muscle tissue and facial arteries, I looked up medical textbooks and recreated a texture map that can be applied to Snap's face mask custom component. The 3D models were derived from open access MRI scans and in the case of the skull, modified by a script to follow the user's jaw movements. In tandem, the lens has a separate preview system, using a 3D rotation script to allow users to navigate labelled versions of the systems. I implemented each modelling and labelling incrementally, fact-checking against open source medical textbooks.

Challenges we ran into

The main challenges I ran into were creating the assets and integrating everything into a simple and coherent user interface. The face-mask assets often didn't map to the face mesh UV and took a lot of adjusting. As with the labelling system - manually applying each number or dot to a location on the mesh took a lot of time. Finally ensuring that the 3D assets were of an appropriate file size and positioning in Lens Studio required much manual adjustment.

Accomplishments that we're proud of

I am really proud of the way in which I undertook such a large project independently, applying my technical and design knowledge to take Kinetic Head from idea to a finalised lens. The project has various moving parts and attests to my effective decomposition of the technical and design problems into smaller, more achievable tasks. And then, after having a functional prototype, I am proud of going the extra mile to turn the initial project into a stylised outcome, applying a distinct visual language, graphic design and a simple user experience.

What we learned

I learnt three main lessons during this project:

1. To create more pessimistic time estimates, especially when it comes to asset preparation, in order to be able to proportionately map out time for the different tasks in a given project.
2. To spend more time at the beginning honing in a concept. Time was wasted to 'exploring through building', which could have been more efficient had I used simpler tools to ideate.
3. 'Sometimes you can solve a problem better the next morning', often in this project, I was stuck on a simple problem that merely required a bit of lateral thinking. Leaving that task to work on another or to go for a walk allowed me to more effectively tackle it later.

What's next for Kinetic Head

Kinetic head is (hopefully) a sample of the powerful potential within Snap's technology to create meaningful and memorable educational experiences for people of all ages. Given the time and resources, I would like to refine this concept and partner with interested educators and organisations to create a wider sweep of fun, immersive educational lenses to demonstrate how learning can exciting, given enough creative planning.

Built With

• bodysegmentation
• facemesh
• headtracking
• javascript