We are grateful for the opportunity to participate in the Philadelphia Museum of Art (PMA) Hackathon 3.0 and create ArtMIND, a web app that works on all devices and is designed to help PMA members and visitors connect more deeply with the art in the collection. Participating in a hackathon and creating an app was a new experience for our team members, and we came together around this challenge. We took seriously the charge to give visitors a way to navigate the (sometimes) daunting experience of entering the Philadelphia Museum of Art and help them find works they can engage with and enjoy.

When PMA visitors access ArtMind, they are presented with several randomly generated artworks, representing a breadth of genres. Visitors input their likes and dislikes and ArtMIND generates suggestions that grow more intuitive as multiple visitors respond to the prompts. Behind the scenes, there is a recommendation system using a machine learning algorithm to provide these recommendations.

This algorithm takes users’ inputs regarding the initially presented artworks, and based on information about the artworks- classification, style, geography, date range, and the social tags that describe what’s actually in the work, the algorithm measures the difference between users’ preferences and the information of each work on display at the PMA. The algorithm suggests works for which this difference is minimized, as computed according to a Euclidean norm. Another metric is used to convert this difference into a corresponding match percentage, which is presented to the user.

In this way, users learn of artworks that, while possibly unfamiliar, are likely to interest and entice them to explore the museum and its galleries further.

A primary challenge the team faced was in making the available data more useful by grouping works into clusters that were easier to parse by the recommendation algorithm. For example, the Style feature in the PMA’s API contains specific information about substyles. In order to overcome this challenge, we made some creative decisions to collapse together similar styles. As a result, our overall dataset, ranging over all works on display, contains fewer unique Styles, which we then fed into the algorithm.

In addition to the basic features described here, which allow visitors to express their preferences and receive recommendations of closely matched artworks to consider, users of ArtMIND will also be able to link to the artworks’ locations in PMA’s galleries; view crowd-sourced social tags and add their own; create a user account to store their choices of favored artworks; and investigate curated lists of art based on various themes.

We are excited to conceive of what’s possible in the next iteration of ArtMIND, including a system with verified and unverified commentary to inform visitors of interesting anecdotes and background on the artworks; and the possibility of connecting to other users who share similar profiles and common preferences in art.

Video Script

Welcome to ArtMIND.

Open the webapp and create an account, and you’re presented with 10 works of art. These works are randomly generated, but within specific categories so that ArtMIND can make the best recommendation for you right off the bat across a breadth of works, being sure to include not just paintings, but sculptures, furniture, architectural fragments, and more.

Visitors are asked to rate these artworks with a thumbs up or thumbs down- based solely on what they see of the work, and not judging it based on an artist’s name, time period, or other information. We want the visitor to respond to the work alone without any preconceptions.

Once they’ve gone through this basic introduction, they’re greeted by a list of works recommended for them by ArtMIND. ArtMIND uses information about the artworks- classification, style, geography, date range, and the social tags that describe what’s actually in the work- to match artwork on view in the galleries to the visitor’s preferences. The more works that a visitor rates, the better recommendations ArtMIND’s machine-learning algorithm can provide them with.

Each artwork will have basic information about the work, where to locate it in the galleries, more in-depth knowledge on select works from publications, and the option for visitors to add their own tags describing what they see in the work, which also helps to build the recommendation system for all visitors.

All of this information is saved to each visitor’s account so that the next time they open the app, their recommendations can continue to build on themselves and only get better as they discover and rate more works.

With the help of ArtMIND, visitors can discover and engage with new work in the galleries of the Philadelphia Museum of Art.

Share this project: