Inspiration
Prompt engineering is nontrivial. It takes craftsmanship to learn how to prompt image generation models. This WeavePrompt project reverse-engineers the hidden prompt that can be used to generate any given image.
What it does
Given any image, the user can select some image generation models and compare the optimal prompt for each of them. This enables easy inspiration for future prompting and the pain of model selection.
How we built it
We built upon existing text-to-text, image-to-text, and text-to-image APIs. The first two are provided by Weave Inference, while the third one is calling external API. Everything is built entirely with Python.
Challenges we ran into
- Prompt engineering
- Image similarity metrics
- Frontend UI
Accomplishments that we're proud of
- Smooth frontend UI to visualize the intermediate results and compare different models
- Stable image similarity metric that is model-agnostic
What we learned
It takes time to build such a complex project, but once we have the correct infra and tools available, then the rest is just simple.
What's next for WeavePrompt
Once the user gets the optimal model and the prompt, there are lots of potential applications:
- Edit the prompt to get different styles of generated images
- Fine tune the model with LoRA
Log in or sign up for Devpost to join the conversation.