Authentic Clay

is our art piece AI-generated name by a GPT-2 model https://talktotransformer.com/

We wanted Ai involved in almost every dimension of our project :)

Inspiration

Saudi and Arab Heritage, Islamic typography, Holy Kabaa silk and cover writings

Link for our stylized video by our second style transfer model (LQ): https://tinyurl.com/sx32gev

-Long version https://youtu.be/xrRze0q-EZA (Not edited but combined many interpolations in a vid some are HQ some are not yet upscaled

-Proceed version of generated StyleGan images: https://youtu.be/uSoRYYut32M

-Portrait Video testing for AI Summit: https://youtu.be/BcTqdycZPIQ

-Upscaled version: https://youtu.be/DAAxwzkB_64

-This is my Favorite Video our second model (Style Transfer trained on Silk and Islamic elements data used) on video of Al-Ula: https://youtu.be/v5sKRJtCb8A

What it does

Generates new images of landscapes representing our culture and heritage.

Please note we’ve used 3 different models our main one is StyleGan for generating new images data

You may get confused between StyleGan & Style transfer

StyleGan is a copyright name for state of the art generative model by Nvidia needs 2000 images+ for training https://youtu.be/kSLJriaOumA

Style transfer is a method of transferring styles of photos on other photos or frames of a video which is way easier because you only need 1 image.

How we built it?

First, we web scraped 6000 images related to our concept with “Beautiful Soup” and other “Python library’s”

Then we manually cleaned Irrelevant photos And used Python Augmenter Library’s to augment and scale our images for our model

Then we implemented “Transfer learning” (fine-tuning) techniques on Nvidia “StyleGan” Model we taught the Neural networks to generate its own images instead of Nvidia HQfaces by showing it a-lot of landscapes data.

Next we chose a latent vector and Produced in interpolation video loop using FFMPEG library to frame and manipulate.

Finally, we trained a StyleTransfer model on our Artist art piece inspired by Kabaa, Silk, Arabic typography, and culture than we used FFMPEG again to break our video into frames. then we used our trained 2nd model to style transfer a batch of frames produced by our first generative fine-tuned StyleGan model than we made them again into a video using FFMPEG https://tinyurl.com/sx32gev

We also upscaled the resolution x4 using ESRGAN super-resolution algorithm.

The whole process took and needed a lot of computer power and time so we used 16 Amazon AWS TESLA GPU’s which we rented from AWS services and tried GCE VM 8 GPU's

Tools :

AI Models (Algorithms) :

1.Nvidia state of the art "StyleGan" https://youtu.be/kSLJriaOumA

(For generating new images and interpolation videos - we showed this model thousand of images data to understand our data and make new images based on the data it has learned ).

2.StyleTransfer VGG19 & VGG16 Models.

3.ESRGAN (Resolution Upscaling model).

Python, Tensorflow 2.0, Pytorch, FFMPEG, OpenCV, Augmentor library, Adobe tools pre-processing, rotate tools, Xnconvert Beautiful Soap, TESLA V100 8 GPU from Google Cloud, 16 TESLA k80 GPU’s AWS.

Challenges we ran into

Collecting data, Training and understanding the model, fine-tuning, Corona pandemic :(, Team being synced, time-consuming

Accomplishments that I'm proud of

Learning rate, built model from scratch in less than 24h, collected and scrapes many images, learning curve, meet a super supportive nice team Organising the Artathon

What I learned

We have learned both how to optimize our Artistic output and how to Think from both an artistic and technical perspective, we have learned how to Fine Tune our models and how Neural networks learn, and finally how to show beauty through AI to the world

-Latent space interpolation -GAN's collections -Neural networks -Fine tunning -Built a Front End for our Art

What's next for Silk NO.9 ?

We will continue Building AI art as we got in love with this kind of art which is truly artistic we will scrape and clean more data and implement more new techniques and manipulate our modeling and combine new tools and music to our art. We will build a Website like Art breeder https://www.artbreeder.com/ to publish our art

We will keep contributing to our beautiful country and embrace this opportunity to spread our cultural heritage and art to the world :).

Built With

Share this project:

Updates