The original idea was to take in a screenshot of a website and its HTML source code and return new HTML source code that could serve as a template for websites of a similar design and purpose. However, given the time constraints of this hackathon and the complexity of parsing the source code through a neural network, we decided to alter the idea to a simpler one: a program that can convert a website to its mobile counterpart.
We built a tool for expanding developers to see their web application projected in the mobile version. With smaller screens, such as smartphones and tablets, becoming more prominent due to its convenience and capabilities, web developers have been creating a special layout of their website specific for these smaller screens. However, this creates extra work for new developers, as they have to create their website and then add scripts to make it mobile compatible. This tool is aimed at helping potential developers see their website on the mobile framework and allows them to make style choices based on the example layout this tool will generate.
CycleGAN, developed by Efros’ lab earlier this month, use GANs to solve the very interesting problem of mappings between image spaces. They have shown that it can transfer style and objects extremely well, converting images with horses to images with zebras and regular images to ones with a Monet style. They do this without pairings from the two image spaces, but rather a good representation of the spaces with roughly 1000 images in each category. Using Selenium, we collect screenshots of websites, both what they would look like on a computer and on a mobile device (Nexus 5). We define our image spaces as websites on computer and website on mobile. Our goal is to find a nonlinear function approximator for F, the mapping function from computer screenshots to mobile screenshots (and its inverse G). We do so by implementing four neural networks and training them with our data.
Although the final product is not ready, as there is still more data to collect and train the network on, we hope to continue collecting data of websites and passing it into the network until it shows some sign of success and returns a potential website structure.