-
Alone in the void with the telephone
-
Empty shelves (no boxes)
-
Empty shelves (still no boxes)
Inspiration
(https://www.youtube.com/watch?v=j_urZ5KDPec)[https://www.youtube.com/watch?v=j_urZ5KDPec]
What it does
An infinite exploration product list in virtual reality, using voice recognition for searching multiple datasets, and populating shelves that appear with products that the user can explore in VR.
How we built it
We use a Unity project deployed for Occulus Quest 2 Virtual Reality Headset to display shelves that appear and disappear, as well as some other widgets such as the telephone. Talking into the telephone will send a request to Google Cloud to intepret your speech, and the text is returned to Unity, where it is parsed. It is then sent to a Rust server, which interfaces with various stock APIs, and an Amazon dataset that is parsed and searched using Python. The selected results are then returned to Unity, where they are displayed on the shelves with their information.
Challenges we ran into
We had issues with exporting materials out of Blender and importing them into Unity correctly, and getting their textures to display. We also had a lot of trouble with Google Cloud authentication, and setting up the correct pipeline for accounts and billing for our agents. A big problem was Unity refusing to include internet permissions in the manifest for our android app.
Accomplishments that we're proud of
We are proud of being able to deploy the project to the VR device, and we are proud of the voice recognition working smoothly, despite manually converting audio formats according to specifications. We are also proud of being able to parse some of the badly structured datasets effectively.
What we learned
We learned many intricacies of the Google Cloud API, as well as plenty about Virtual Reality. The majority of our team also learned lots about the technologies of Unity and C#.
What's next for matrix∞
Add more data sources to the system, integrate blockchain for purchases, multiplayer shopping.
Log in or sign up for Devpost to join the conversation.