Team Members - 755 Sarvasv, 687 Sohit, 341 Rishabh
The internet is one of the most important technological revolutions in the history of mankind, and as of today, we use it for almost everything - to remind us to wake up, to help us navigate when we travel, to keep us entertained when bored, and to connect us to others. Yet, less than 0.3% of the workforce knows how to code, and even less have actually made websites.
We took it upon ourselves to democratize web development. Instead of leaving the internet to a small group of coders, what if anyone could make a website?
Everyone can pick up a pencil and draw, what if you could turn drawings into code with a photo?
That's why we made PageBasic!
What it does
PageBasic allows you to turn drawings of your website's UI into the actual HTML and CSS markup in just a photo.
All you need to is make a freehand drawing of your UI on a piece of paper, and PageBasic can convert it into the code you need. You can also select the specific style you want, choosing between the UI design languages of AirBnB, Google, Facebook, etc.
If you want, you can submit a photo of your drawing, take a live photo, or even draw it in the browser.
How we built it
We trained a machine learning architecture that merged LSTM's and CNN's to create a model that converted images into specific tokens along a probability vector, which we then passed through a softmax function that output the most probable blocks of code for a given image.
This model was extremely computationally expensive, so we had to train it on Google Cloud Platform with two GPUs for over 7 hours. Once that was done, we integrated it into a Flask backend running locally.
Challenges we ran into
This project was the most difficult hackathon project we've ever done! The model was very difficult to put together, tuning the hyperparameters was a nightmare, and we had to look through multiple research papers and online articles to find out how to assemble it properly.
We first tried to train it on our laptop, but that required time was 57 hours (!). We had to scramble to set up a cloud GPU to train it, and finally got the model trained in 7 hours.
Integrating the backend and front end was also very annoying, we had to spend an entire hour figuring out how to work with the Base64 encoding of the images.
Accomplishments that we're proud of
Over the hackathon, we trained a custom machine-learning architecture for over seven hours, and we are proud of the fairly accurate results. We’re also particularly proud of the multiple ways to generate websites: drawing with your trackpad, uploading a file, or even taking a photo of a paper you drew on. Finally, the seamless integration between the Keras Backend and our beautiful frontend turned our project into a product.
What we learned
On the ML side, we learnt how to use GPUs to train models and fine-tune parameters to get the right model. With the flask backend, we learned to how process POST requests and integrate with other APIs. With the frontend, we learned how effective web design looked like and advanced features with SCSS and JQuery.
What's next for PageBasic
- Improving the machine learning model to identify more web components
- Adding more customizability for designs
- Increasing efficiency
- Better OCR
- Making a bigger dataset to train on
- Beta testing