Museum Muse

• 

  Two smartphone screens displaying a museum accessibility app interface next to text about Art Accessibility On Demand.

Inspiration

Making art more accessible to everybody. Our team loves art and museums. However, we've feel that artwork can sometimes be almost sterile hanging on wall of museums with a short description from the curator. When Google released Gemini and Meta released the Rayban smart glasses, we were inspired to combine the technologies to create an accessible art app to help people from all backgrounds, physical abilities, and preferences enjoy art in museums.

What it does

The app uses the camera to understand the artwork and narrates the art work based on the selected persona (e.g. Formal art historian). The AI persona guide explains the artwork and provides additional interesting facts on the piece. There are also seven languages to choose from. The user can also review previous art works on the History and Map view. When the app detects the user is at a particular museum, it notifies the user and utilizes Retrival Augmentation Generation (RAG) of the museum's art work to help more accurately predict the art work.

How we built it

Used iOS, Meta Glasses SDK, Harvard Museum API,

Challenges we ran into

Integrating the meta glasses SDK to work with the iOS app so the camera feed streams to the app.

Accomplishments that we're proud of

Having translation services for seven languages and six different personas. Integration with Meta Rayban glasses.

What we learned

Voice is a very powerful way to interact with AI. Integrating with meta glasses makes it even more intuitive.

What's next for Museum Muse

Apply to various grants for art and AI. Try to find a museum whom will allow us to run a pilot.

Built With

• eleven
• gemini
• glasses
• ios
• labs
• meta