The WholeU

Inspiration

Chronic illness and autoimmune conditions are problems that we as a team have been close to or faced personally. In the diagnosis of chronic issues the medical field often lacks vision of the whole patient, only seeing individual parts of them.

What it does

Introducing The WholeU, a website where patient's can log in and track their pain or symptoms, through marking the date, symptoms, and pain level on a 3D model of themselves. Then their doctor can just use the Snapchat Spectacles to overlay the marked up image onto the patient; allowing for doctors to see the whole patient.

How we built it

We started by adapting an AI 3D model generator to use a front-facing image of the patient to then make a 3D model of them. This model was then input into an HTML and CSS based website where the patient could mark up pain points (future feature). Finally, we uploaded the 3D model into Lens Studio where we were able to send it to the Spectacles that can be worn. We have finished the ideation of features we weren't able to implement yet through Figma.

Challenges we ran into

While we had some knowledge of UnrealEngine before this project we had to learn how to use Lens Studio connected to Spectacles, which is an extremely new project. Additionally, we struggled with the connection between the website, 3D model, and Spectacles.

Accomplishments that we're proud of

We are really proud of how we were able to use entirely new technology through Spectacles, along with how we were able to adapt pre-existing AI models to our purposes.

What we learned

We learned how to work with 3D modeling and AI, along with how to create artificial reality through Lens Studio.

What's next for The WholeU

The next steps for The WholeU would be further developing the pain points feature in the website. Additionally, further touching up the HTML and CSS website to integrate JS and React to make the 'drawing' of pain feature work and making the website design match the Figma. Finally, we would like to further develop the 3D model to be interactive where the doctor could pull up a menu of the patient's concerns.