pong | Devpost

logo
workflow_example
data_structure_sample

Inspiration 💡

Recommendation is part of our everyday life. Even this hackathon is found by us through the recommendation of devpost’s system. There are a lot of recommendation systems for tv shows, movies, books and songs. The museum or artwork recommendation seems to be still in the developing phase. 🚀

Museum recommendation in the offline world is pointless, since the users’ mobilities are limited. Visiting a museum far away from us costs a lot of time and money. It would be nice to taste every artwork in the world 🌍 on the spot 🌎 but there are only a few people who can do it. 🌏 On the contrary, artwork recommendation is a valid idea, because it is barrier-free. There are no distance limitations on the internet and the number of virtual exhibitions are growing day by day. 💲

What it does ❓

Pong is a brand new idea built for this hackathon. 🆕 Pong focuses on artwork level recommendation. It targets to predict the triggered emotion and the strength of the emotion related to a specific artwork. To achieve this goal we use a combination of artificial intelligence and machine learning models that get data from different sources. 🧠

Pong has 3 potential target groups: 🧑🧑‍🦰🧑‍🦱

users as part of the audience
museums (this includes the institutions and the researchers as well)
artists.

We provide an emotion based recommendation system for all groups. Users can get better and emotion oriented user-experience, while museums and artists get feedback to understand the impact of different artworks or to make scientific research.

End-users can be online or offline users. For end-users we can imagine a modal box that can be integrated in mobile applications and in websites as well. It contains a list of emojis and a dartboard only. 🎯 The user can select the emoji that best fits their feelings. If they hold the emoji for a longer time they can set the level of the feeling on a 3 step scale. 🎚 With the dartboard the user can express how relevant the recommendation is.

Our solution consists of 2 AI powered parts.

1️⃣

The first part contains a model that uses all kinds of artwork related data to predict the triggered emotions and their strengths.

2️⃣

The second part consists of multiple AI models. Their task is to automate the transformation of the museum data to data points that suit the requirements of the recommendation AI, mentioned as part one.

➕

We created the whole data structure model to feed both AI solutions.

How we built it 🔨

We did a short research about competitors on the market and we made some data analysis on the data given by the organizer team. 🔎

We searched for pain points of the existing solutions and we declared our goals with this project. 📜

We developed AI solutions that can meet the requirements of our primary goal to build an emotion focused recommendation system for artworks. 🤩

We developed the whole data structure needed for that system. 📐

We began to make the code. ⌨

Challenges we ran into 🏆

There is a large difference between the data structure required by AI and made by museum experts. Solving this situation seems to be a hard and interesting challenge. ⭐

Accomplishments that we're proud of 😎

🔷 We made a whole data structure plan.
🔷 We made more AI model architecture plans and pipelines.
🔷 We made a mock modal box that can be used as part of mobile applications or webpages.

What we learned 📘

We learned so many things about culture and data structured by culture experts that it totally widened our limits on thinking about structuring data. 😃

What's next for pong ⏱

The AI modules need to be well optimized, tested and trained based on the data that we actually have. 📗

After this phase we can go further. On the internet we can collect detailed telemetry from the beginning of the development. The collection of that kind of data is nearly impossible, when we talk about offline, real-world exhibition. That’s why we plan to make a computer vision model that can anonymously identify the audience in the exhibition area and collect the telemetry, for example how much time a user spends on a specific artwork or which is the most common user-path when they explore the exhibition. 👣

In the near future it is possible to plan an emotion path during a real-world or a virtual exhibition by planning the visit-order of the artworks. 🧭