FusionCraft AI

Inspiration

I was inspired to create this project by my interest in both Stable Diffusion and Together.ai. I am fascinated by the ability of Stable Diffusion to generate realistic images from text prompts, and I believe that Together.ai has the potential to make Stable Diffusion even more powerful and versatile.

One of the limitations of Stable Diffusion is that it can be difficult to craft prompts that produce the desired results. Especially for non-developers who aren't familiar with the common syntax used in the LLM's, utilizing the tool can be tricky. Additionally, Stable Diffusion can be computationally expensive to train and use.

What it does

This model extracts the key elements from a given story/paragraph in order to create a proper prompt that can be depicted when entering into the stable diffusion model.

How we built it

This project was built by developing a fine-tuning python script to implement with the Together.AI model ad playground. This was done by feeding the model multiple paragraph stories and pairing it with prompts that encapsulate the key elements of it.

Accomplishments that we're proud of

The fine-tuned prompts are very accurate in terms of reflecting the paragraph that it's derived from. When I enter the prompts into the stable diffusion generator, the images are accurate.

What's next for Fine-Tune Fusion

Next would be to fully integrate the two fine-tuned models so that the AI text to image can be accessed directly.

Built With

• jupyter
• llm
• python
• stablediffusion
• together.ai