Image-Tagger

• 

  Anime Example

• 

  Landscape Example

• 

  Portrait Example

Inspiration

This project was inspired by another project a team member was working on, an art constructive criticism platform. Being able to automate image tagging for artworks posted on the site would help searching functions on the site. This project was basically a proof of concept for the system.

What it does

Automatically tags images with keywords that describe the submitted image.

How we built it

We used a library called Clip-Interrogator, which generates prompts from Stable Diffusion models. We parsed the keywords from the prompt, and used Spacey to select the most appropriate keywords for tags.

Challenges we ran into

Spacey was overly picky with words it selected for tags. We decided to use a more naive approach for selecting keywords but also left the Spacey-selected words at the bottom.

We also had trouble with package incompatibilities.

Accomplishments that we're proud of

We created an amazing landing page and user page for the system. The system is pretty accurate in choosing keywords, for example, in the Anime example, it was actually able to figure out the image was from the show "Jujutsu Kaisen". But it does occasionally select odd descriptors. For example, in the MLK portrait, it described Martin Luther King Jr. as MKBHD, an Internet personality.

What we learned and what's next

While we are happy with what we've created, we think we could've done more to make the project more interesting, such as using Segment Anything to first segment the image to get accurate tags for each object in the image.

Built With

• clip-interrogator
• flask
• next
• python
• pytorch
• spacey