Ditto AI

Inspiration

Ditto is an AI-powered document editor for non-techies to build web apps with. We are simplifying and abstracting the app development workflow inside of a medium that non-technical users are comfortable with, a document editor. Through our experience teaching no-code tools and building apps for non-technical users, we have found that design is a big obstacle for our target audience. They usually have a rough content drafted in a Google Doc but have no knowledge nor technical ability to turn their content into a professionally designed app layouts.

Hence, over the weekend, we have built a team of AI agents with domain expertise in layout design to help our users to turn ther rough draft into professional layout blocks that best display and communicate their content.

What it does

Our team of AI agents will first receive the draft content that users have added in the editor, the team will then perform the following:

1. Our content design agent will analyse sementic meaning of the draft and plan out the best way to structure the content by topics
2. Our creative agent will take the draft, brainstorm and design as many layout blocks as possible
3. Our design agent will receive the plan provided by the content design agent and as well as all the layout blocks provided by the creative agent. It will then design the best layout combination and build it out for our users.

How we built it

• Break the layout design process down into separate parts (Planning, design, build).
• Develop an agent plan for each separate part
• Implement each chain of thought agent in langchain using OpenAI endpoints
• Connect each agent together and allow them to pass information between each other
• Create a frontend in NextJS to allow users to input text / images and see the agent output

Challenges we ran into

1. Output could be too predictive at times for our engine to render the designs
2. Agent does not extract enough information to output more creative designs
3. Hallucinations caused agents to create components not in our design system. Had to enforce this using few-shots and output validation / retries
4. Speed is slow using OpenAI gpt-4 but results were unreliable using 3.5, had to use a mixed model approach

Accomplishments that we're proud of

We have built a solid foundation of AI agent system that can be added to our editor and be further scaled with agent teams with various domain expertise.

What's next for Ditto AI

We will first add this team of layout expert agents into the editor to elevate the design quality of our users final app output. We plan to add more teams of AI agents with different domain expertise such as copywriting, graphics design, brand design into our editor. Our goal is to offer an AI product team right inside of our editor to empower non-technical users create the best app experiences without the requirement for technical know-how.

Built With

• langchain
• nextjs
• openai