Geraldine: I was inspired to form this idea when I ran into the dilemma where I wanted to try on women's clothes but could not due to the pandemic situation. So I thought what if there was an app that allows me to look through a catalog of clothes and try them on. Then, I can determine what type of clothes fit me best.
What it does
The project is an iOS app which utilizes Apple's ARKit framework to display a 3D model using the device's camera AR capabilities. The idea behind the project is that the user can select an item from a list, a new screen pops up, and the camera is enabled which will display the item on the user once they point their device's camera to a mirror. From there, the user can determine if that item fits with their style.
How we built it
We built the application using Swift and UIKit for the user interface of the application. We utilized ARKit and RealityKit frameworks to track the user's body when the camera picks it up. From there we obtain the skeleton model of the user and map certain joints to a mesh in the 3D model. Then in real time, the item will be on the user like if the user was trying it on at a fitting room.
Challenges we ran into
The one challenge we ran into is getting the model to display properly in the AR view and also tracking the user's body. We were using relatively new technologies that Apple introduced in WWDC 2019, so there was nearly zero resources out there. We struggled trying to implement the features we wanted as a result, and we would have needed a lot more time to fully understand the API to get what we needed for this project.
Accomplishments that we're proud of
We are proud of getting the design implemented for most of the application. While we did not get every feature that we wanted implemented, it was good seeing how much we got done in the limited time frame.
What we learned
We learned about ARKit and the ARBodyTrackingConfiguration API.
What's next for VestuARio
Obviously, we plan on fully implementing the features that we needed. We were also thinking of building an API so that companies can register their products with their 3D models so that potential customers can try them on using our app. Users can then create accounts and browser the catalog like any e-commerce clothing platform would operate. From there, we may implement a way for users to pay for a certain item.