What more could we accomplish if the time to test an idea was zero?
I've been in awe of what computer vision can accomplish and how it can bring about positive change in our business processes. One such area is UI prototyping. Sketch2Code made me rethink the efficiency of existing accepted process by demonstrating the utitlity of AI and computer vision in producing prototypes for user testing. I got the idea that integration with tools such as Jira and Confluence would make it readily accessible and fit seemlessly in the users' collaboration space while also making them far more efficient.
What it does
Sketch2Code Block allows one to transform a UI sketch into a working web interface within seconds. If you have sketch anywhere, just click a photo and add it to your Jira issue as attachment or add it to your confluence page by providing its URL in the configuration of the Confluence Macro.
Users viewing your Jira Issue or Confluence page can simply click on generate code to see the resulting web page of the UI Sketch(es) you added.
How I built it
- Forge SDK
- Microsoft Azure Cloud and Customvision.ai
Challenges I ran into
- Understanding how Forge UI works and figuring handling interactions on Server side was a bit of mindset change.
- Debugging was really fast paced due to
forge tunnelcommand, however error messages were sometimes cryptic.
Accomplishments that I'm proud of
- Training a custom vision model that performs well for a very small dataset of about 140 images with 10 classification labels.
- Building a easy to use extension that integrates with and enhances the design proccesses.
What I learned
- Atlassian's new Forge platform
- Azure Cloud
What's next for Sketch2Code Block
- Real time multi-party collaboration on Sketches (Google docs for UI sketches).
- Faster code generation.
- Better HTML generation with more training data.