Concept art for ix Hololens interface
Example of heat map output the ix system would be able to help create, which curators could use to better tailor exhibits to their audience
Example of the recommended art algorithm ix will use, drawing on paths/tastes of ix users to recommend other artworks to view
This project was inspired by the desire to preserve art and cultural heritage that encourages more meaningful relationships among the artists, art institutions and viewers.
What it does
ix supports a change in the art museum ecosystem by helping artists, museums, and patrons co-create a more meaningful experience:
For Patrons The ix system uses augmented reality and RFID technology to let viewers behind the glass through artist insights, audience interactions, curator commentary, and recommendations for other artwork to view based off of where they're spending time, what they're looking at, and other visitors' experiences.
For Museums The ix system collects data about audience engagement with works, their paths through the museum, and audience comments. This will allow museum curators to better tailor exhibits and events to the interests, questions, and concerns of their audience.
For Artists The ix system allows artists to interact directly with a museum audience and gain data-driven insights into how viewers interact with and perceive their works.
How I built it
Building our product for the Hacking Arts consisted of a diverse set of skills and workflows for the final output. Through the medium of Microsoft Hololens, phones and tablets, we use Unity to overlay graphics upon the screen. Our current demo involves: 1) Video making: We shot a video and used Adobe After Effect to edit/animate the video as a prototype. 2) Technical Expertise: We used Google APIs, RFID, Data Analysis. The RFID and Image recognition links to content, where the Google Cloud API searches and recommends similar content. The data gained from the audience movement (through RFID) in the museum/exhibition allows the art curators to understand the audiences better and allows them to cater them in a more fluid way.
Challenges and accomplishments
One challenge we faced was deciding on an appropriate level of fidelity in this initial prototyping phase, especially without access to an array of augmented reality devices to experiment on. We decided to flesh out a prototype of the overall user experience using video/animation and simulated interfaces; this allowed us to think through the level of supporting technology we might need and pare down our list of technologies to a scale that makes sense in a museum. Figuring out our specific use for the Google Cloud API was also a tough, but rewarding process that gave us access to many functionalities for the experience we aim to stimulate. Simplification of technology, as well as making a usable and intuitive interface was tricky! However, by facilitating conversations in front of installations allowed us to gain insight into the natural processes that occur in a museum- which led to an intuitive interaction design.
What I learned
Working through this project made us think critically about what museums are for and what the average person can get from a museum visit. We don't believe that a transmission model of art is enough, and this project taught us more about what a viable alternative could look like.
What's next for ix
Speaking with museums to work on logistics and gather appropriate hardware to create a beta prototype is a large next step! With this in motion, the gathered data can be used to train a CNN and create an interface for curators to better optimize a layout for future exhibits. By focusing on a business model catered to exhibition management, curators, and artists, we hope to offer our services and integrate this immersive experience into exhibitions interested in furthering the art ecosystem.