Level Up

OVERVIEW

Level Up is a online virtual service platform that provides the users the ability to utilize a facial recognition software to generate their own avatar model with a single photo's biometric data that is provided. They are provided with different selections to change any inconsistencies with their generated custom avatar. Once the user finalizes their avatar, the file is accessed onto unity by querying our SQL database, and it is applied to several different different Machine Learning models, 1 for full body in game avatar tracking, another for HMD ( Head Mounted Display) and Controller movement prediction, effectively attaching the VR rig to the avatar syncing real time VR movement with the avatar, Lastly, the avatar's mouth and eyes are also being updated as the VR player speaks and blinks in real time.

GOALS

Create a Machine Learning model to detect full body tracking Create a Machine Learning model to detect sound and trigger an animation Create a Machine Learning model to flinch/Blink a Unity Camera when a 3D model is presented too close Establish a system that can generate a 3D glb file (avatar) from a single photograph Create a system that can run our 3D file to the Machine Learning models in unity Create a website for login & implements the system of generating a 3D glb file Create an SQL database that can communicate with Unity and our Website Create a custom VR rig that can use the trained 3D file (avatar) for real time prediction and synchronicity Create a VR Unity Simulation that contains Login, and a sample world with a mirror Create a Script to query the SQL database for login verification Create a Script to query the SQL database to gather the 3D file unique of the verified user. Test the VR Simulation to ensure Avatar Movement/Lips/Eyes Tracking is working effectively

Constraints

Finding updated solutions to communicate from unity to the SQL database & from website to SQL database Implementing front end touches for beautification of the website Unity Download Software downloading issues Website Configuration: Delays in understanding Domain Website configurations and the methods required to change the settings. Webpage: Domain Page complications within the editor, the one teammate that purchased the domain is the only person that has editor privileges into the domain editing process and no one else can practically edit the website, unless they were intimate with HTML scripting. Switching Scenes in Unity Getting the right Vector Spaces for different users in the Machine Learning models to prevent overfitting in training

Solved Constraints

Unity programming was completed through one computer Switching Scenes Script was solved through the help of a mentor Vector Space Calculations were solved by using federating learning (pre-trained models)

MILESTONES

Create Machine Learning Models Implement Machine Learning Models on Unity Create Website Implement 3D Avatar generation from a single image into our website Create SQL database Query 3D files from database from unity to load 3D models Run 3D models through Machine Learning Models Create Login system/scene & a sample scene with a mirror in VR Unity Create Custom Rig to Sync everything together

Motivation

The motivation behind the idea is that there is an opportunity with, future implementations of simulation systems, to appropriately track users information without violation of private information. Using a image of a face alot of information can be gathered from a person. Ideally the image can run through many machine learning computer vision models to gather the information required for future implementations of simulation systems, such as age prediction, avatar creation, authentication, and many more points of data to identify without the use of rigorous (mostly intrusive) current outdated systems.

What we missed

We were not able to create an ideal website (login collection, front end beautification) due to lack of html knowledge. We were not able to connect the website to the SQL database. We were not able to query, the SQL database, for users authentication verification on unity.

What we got

All the machine learning models importing the models onto unity VR custom Rig VR Login Scene and Sample Scene with mirror VR realtime tracking of Lips/Eyes/ and player Jumping and movement Avatar creation from a single selfie image A database A website

Future Developments

Link our database to our website and unity for verification Create the VR simulation to enable Multiplayer by using PUN2 Photon

Built With

Share this project:

Updates