Inspiration
We love visting museums, but that is a time consuming process and their are so many museums that are fare away that you are just unable to visit. So we thought why not bring the museum to us? Also how great would it be if we could have our own personal curator that can pull from public domain works from around the world.
What it does
Just ask in natural language what you want to see and expierence the amazing collection is curated from databases around the. world.
Your natural ask is passed to a large lanugage model (LLM) that translates your ask into query for the different API endpoints and databases that contain public artworks, images, sculputers and more.
Than the returned list of images is returned and displayed into the picture frames in your own personal museum.
To support a most likely immersive and natural way of interaction we provide hand tracking only.
How we built it
The project is build in Unity usinig the Meta Presence Platform. The ask by the user is translated by a LLM into a JSON object which than serves as the input for the API calls.
Next we leverage APIs provided by museums like the MET that contain hundreds of thousand public domain artworks from which we can pick and choose and display the results for the users.
Challenges we ran into
We had some issues with initial setups in Unity and some weak hardware & internet. Render Pipeline causing unexpected Shadow Artifacts.
Accomplishments that we're proud of
This is the first time we collaborate on something like this together. For one of us it was the first time working with LLMs & his first hackathon and for the other it was the first time working within the XR space.
What we learned
We learned lots, both deepening our respective pre-existing expertise and learning from the skills of the other. We should bring more powerful hardware to develop with Unity at least half of our developing time we spent looking at loading bars.
What's next for [11] Project CurateMe
Our backlog is filled with ideas & improvements that we would like to add in the coming weeks and months. We can not wait to continue this work and make public domain works more accessible and help to find high quality real artworks to expand on your existing topics of fascination or help to find new once. Stay curious!
Some of the ideas include selecting your favorite pieces of art and then pulling them into a mixed reality space of your own flat to pin it to your own walls.
Log in or sign up for Devpost to join the conversation.