Bella AI

Inspiration

I was being interviewed for a job by the head of a museum. I like reading about art history, but when she found out I didn't have an art history degree she said I could never successfully work in the Art World. Since then, it has fueled me to learn how to use my degree in CS to become an art historian and curator. This hackathon gave me a reason to learn LLMs and AI/ML so I built it.

What it does

You can give it a picture of some artwork and a label that tells you stuff like the artist's name, the artwork's title, the material it's made of, the date it was created, and who owns it. Then, in just a few seconds, it works its magic and gives you an artist statement and a little note about the artwork, just like an art expert would. It's like having your own art curator! Plus, it can understand the historical context and significance of the artwork and show you all the details using art world jargon.

How we built it

I have a 30GB dataset of artworks with labels. Based on that I extract the movement/style of the artwork image based on AI vision to describe the image of the artwork and a generaitve output about the artist and merge all three using Open AI to craft an accurate representation of what a catalogue note for artwork looks and sounds like.

Challenges we ran into

I needed a massive dataset to train on, so it was difficult for me to build my own LLM as it took a lot of time to do what I was trying to achieve, although I tried very hard. I was trying to use embeddings but didn't turn out to be useful in this case but has some educational benefits for people who are looking for artwork they don't recognize.

Accomplishments that we're proud of

I managed to get pretty close to what I was looking to build. I like the design and the workshops. I loved learning about embeddings and I'm proud to have built my first AI.

What we learned

Embeddings, LLMs, OpenAI API, Design, AI/ML

What's next for Bella AI

expand its capabilities and train on my own data to be more accurate and complete more tasks such as exhibition curation, and doing repetitive gallery assistant tasks. I want to also add a feature to control the readability score as to tailor the output to a certain audience whether kids or experts so that there never is a situation where someone like me can be left out of a dream job ; )

Built With

• ai
• firebase
• google
• kaggle
• machine-learning
• nextjs
• openai
• typescript