Skully

Title slide.
Problem statement and Impact statement.
Solution proposition.
Explanation of the technology.
AR Tracker integration with physical materials.
AR Tracker integration with physical materials.
AR Tracker integration with physical materials.
The interactive navigation menu that utilises gaze pointer technology to navigate through the various scenes.
Raytracing example of how hand tracking is used in an interesting way to manipulate 3D models called by target trackers.
View in the cinema.
AR Cinema scene, where users can sit back and watch video content in an AR space.
A screenshot taken of the MP4 content in the cinema.
Roadmap moving forward.

Inspiration

We wanted to create new and engaging learning materials that are complimented through immersive tech.

What it does

Utilising AR glasses and SnapDragon Spaces we created a solution that brings up 3D models when target images are located. The 3D models can be moved and manipulated with the use of hand tracking and raytracing control, allowing users to better understand the subject matter of human physiology. Through a navigation tool, you are also able to put yourself in a movie cinema to whats mp4 content (In case the idea of immersive and interactive technologies is a bit too overwhelming).

How we built it

We used SnapDragon Spaces, Unity, OpenXR plugins, visual studio, and all the VR physics required.

Challenges we ran into

Very Steep learning curve for SnapDragon spaces, fortunately the team was extremely knowledgable and very helpful. Extra user video materials: https://youtu.be/KvICL2N30wg

Accomplishments that we're proud of

That we managed to get a working project out in the time limit required. For many team members it was their first in person hackathon and/or their first time working with Unity.

What we learned

How to use SnapDragon Spaces and all the complexities that come with working with AR glasses/hardware.

What's next for Skully

Some much needed rest before we show off to all our friends and families what we managed to pull off in a weekend. It's a solid piece of work as well as a solid business case. We're all proud for pulling off this amazing achievement.

Built With

ar
glasses
handtracking
immersivetech
lenovo
qualcomm
snapdragonspaces
thinkreality
unity

Submitted to

MIT Reality Hack 2023

Created by

I worked in XR integration and Unity Development. I was the most experienced with using Unity so it was great to help my team's visions come to life.

Luke Draper
(BaPVA Hons, Wits 2015). SA Based Game developer and all round technical creative aiming to combine art and technology on all fronts.
I worked on the back-end development in Unity and contributed to the brainstorming for the project. Using my programming experience, I introduced logic to the backend algorithm to enhance the interactions with the immersive environment.

Mahdiar Edraki
I worked on the distal and proximal hand-tracking function in the project to enable button selection and object control.

Ivy Wang
I worked on the hand tracking features for manipulating 3d objects. I was not very familiar with unity before this hackathon, and this is my first hackathon, so I think I learned a lot in these 2 days.

Helen Wang
I worked on creating navigation tool with the hand tracking module.Also helped with the virtual menu bar so that user can choose options on his interest.It was my first time using Node, which was a little intimidating, but I learned a lot.

Robayth Dhrubo
Passionate XR developer skilled in C# & Python. Created Snap Dragon Space project. Eager to learn, grow, & make an impact.

Updates

Luke Draper started this project — Jan 15, 2023 12:55 PM EST

Leave feedback in the comments!

Log in or sign up for Devpost to join the conversation.