Bespoke Life Using Deep Learning (RNN)

A smart blogging tool that generates blog post and comments based on images using deep learning.

Comment

Inspiration

Life can be inspirational, artistic, spiritual.
But most of the time, pictures, and simple words fail to well define those sparks of our lives.
So let us help to tell a story for them!

What it does

A web app that asks an image input from you and automatically generates fiction-like stories based on the picture using Deep Learning (RNN)
An interactive and entertaining travel blog where you can check the stories of others and share your stories to the world (with geo-visualization)

How we built it

We first train a recurrent neural network (RNN) decoder on romance novels.
Each passage from a novel is mapped to a skip-thought vector.
The RNN aims to generate the passage that it has encoded.
We use romance novels collected from the BookCorpus dataset.
Parallel to this, we train a visual-semantic embedding between COCO images and captions.
In this model, captions and images are mapped into a common vector space.
After training, we can embed new images and retrieve captions.

Challenges we ran into

Making the models work as expected.
Working on the map UI and web interface.
Difficulty with Oracle Cloud.
Handing huge amount of training data, and compiling on AWS.

Accomplishments that we're proud of

An actual functioning product.
Intuitive user interface.
Artistic story telling style.

What we learned

The mechanism of deep learning models such as RNN, encoder-decoder, and CNN.
Use of python, flask, and docker.

What's next for Bespoke Life

Better User Interface
Image Scaling Functionality.
More story style.

Share this project:

Updates