Impasto AI

• 

  Model outputs comparison

• 

  Dog finetuned impasto result

• 

  Countryside finetuned impasto result

• 

  Portrait finetuned impasto result

Inspiration

Art has always fascinated me, especially how texture and depth convey emotion.
While digital AI-generated art is impressive, I noticed a gap: few models capture the tactile feel of thick, impasto brushstrokes.
Impasto AI was born from the desire to bring traditional painting textures into AI-generated art, letting any image or prompt become a vibrant, textured masterpiece.

What I Learned

• How to finetune LoRA weights on a large base model (FLUX.1 Kontext [dev]) for a specific visual style.
  
• The nuances of image-to-image generation vs. text-to-image generation.
  
• Optimizing model inference with Diffusers pipelines in bfloat16 for GPU memory efficiency.
  
• Handling dataset limitations, ensuring enough diversity without overloading memory.
  

How I Built It

1. Base Model: Used FLUX.1 Kontext [dev] from Hugging Face as the foundation.
   
2. Dataset: Curated images and references to impasto-style art to teach the LoRA the desired texture and brushstroke patterns.
   
3. Finetuning:
   
   • Applied LoRA finetuning using the OstrisAI Toolkit on Hugging Face Spaces.
     
   • Checkpoint: Every 250 steps
   • Epochs: Finetuned until 1500 steps for best texture fidelity
   • Learning rate: 0.000005
4. Inference:
   
   • Allowed prompts like "turn this image into an impasto painting" to create expressive outputs.
     

Challenges Faced

• Memory constraints: Handling large models on limited GPUs required bfloat16 precision and LoRA to reduce footprint.
  
• Style consistency: Ensuring the impasto texture appeared consistently across different images and prompts.
  
• Hackathon time limit: Needed to balance model finetuning, testing, and repo preparation under tight deadlines.
  

Reflection

Through building Impasto AI, I learned the power of lightweight finetuning, the subtleties of artistic style transfer, and how to make AI-generated art feel tactile and expressive.
It reinforced the importance of efficient pipelines, prompt design, and dataset quality in achieving visually compelling results.

Built With

• flux.1-kontext-dev
• huggingface
• lora
• ostris-ai-toolkit