Moving to a new apartment or house, need to buy furniture.
Choosing new furniture is difficult.
Solving the problem using AR/ML and data.
What it does
- Open the ARoom app and show the room that you want to furnish
- Automatic detection of room style, colors, and size
- Furniture recommendations to best fit your room and style
- See what it looks like in your room with Augmented Reality visualization
- Choose from various pieces of furniture
- Rearrange furniture to your liking
Curated recommendations based on how your room already looks Custom recommendations about which pieces of furniture may be best View furniture as it would look in your room with Immersive experience
How we built it
Back-end: Flask, Python
Cloud & Storage: Google Cloud Compute Engine, Google Cloud Storage
AR Engine: Unity 3d(C#), AR Foundation
Machine Learning: PyTorch, Resnet50 Dilated Convolutional Neural Network, Joint Pixel Classification and Object Detection
Challenges we ran into
- Set up the server on the Google Cloud to trigger the machine learning instance.
Accomplishments that we're proud of
- We built up an iOS app as front-end to collect environment image
- We integrated Unity3d's AR function with our iOS app to display the furniture with immerse experience.
- We set up the deep learning model to run the Joint Pixel Classification and Object Detection.
- We built up the server on Google Cloud to store the image uploaded and do the recommendation using the deel learning model.
What's next for ARoom
- More precise recommendations and 3D-reconstructions although our system deploys near state-of-the-art algorithmic architecture, we recognize that rapid advancements in technology may produce better infrastructure in the near future
- Colorized 3D models store user-input images in a separate database ⇒ run matching algorithm for newly uploaded profiles ⇒ email/phone alert when new matches are found
- Recommend pieces based on budget find matches between a recent and older image of the person (when they were younger) Social Media Integration