Drag

Drag is an experiement to push the boundaries of Ai content generation using OpenAi's Open source gpt-oss-120b to demonstrate how AI can help graphic designers generate and modify designs

Comment

My Story: Building Drag App

Inspiration

The inspiration behind Drag came from an experiment where i sought to push AI content generation beyond its usual boundaries. While most AI tools generate static image designs based on user prompts, I wanted to explore how far AI could go in producing editable, dynamic graphic designs. This idea was born from curiosity and the drive to give designers more control, flexibility, and creativity in their workflow.

What I Learned

Through the process, I learned how challenging it is to bridge the gap between AI-generated creativity and practical usability. I gained insights into:

  • The importance of structuring design outputs in formats that remain editable and consistent across several trials by users.
  • How design and AI can intersect to create tools that go beyond "just mere images."
  • The complexities of balancing user experience with advanced AI capabilities.
  • The importance of efficient prompt engineering for AI projects

How I Built It

I built Drag App by combining:

  • AI models specifically the open sourced open AI model (openai/gpt-oss-120b capable of generating FabricJson objects which is the underlying tech powering this innovation.
  • Editable design structures, allowing output that could be modified after generation using FabricJson canvas.
  • A workflow that converts AI creativity into layered, designer-friendly outputs instead of flattened static files.
  • The workflow is pretty simple:
  • User prompts AI
  • AI makes use of the underlying system prompt direction, choosing the best output pattern, scheme and theme for the user's design request
  • Ai outputs a structured standard FabricJson object, together with a chain of thoughts reasoning and a text answer.
  • the returned object is consumed by the frontend and smartly rendered on the canvas/app which ha already existing design tool, allowing the user to fully customize either manually or stil through prompt
  • Users are free to export designs are images (JPG, PNG or Fabric.json file) with possiblities of also extending to PDF and other formats

Challenges

Some of the main challenges I faced were:

  • Fullly utilizing the power of FabricJson in React
  • primarily it was ensuring the most optimal prompt engineering for peak performance and results from user prompts
  • Ensuring that generated designs were editable and not locked on the canvas static formats.
  • Handling complex AI outputs while keeping the design layers organized.
  • Finding the balance between automation and creative freedom for users.
  • Maintaining design principles to ensure actual usable results and not just mere shapes and texts scattered around a canvas

Built With

Share this project:

Updates