Neural Poet



The whole approach contains 4 components

  1. skip-thought vectors
  2. image-sentence embeddings
  3. conditional neural language models
  4. style shifting

The 'style-shifting' operation is what allows our model to transfer standard image captions to the style of stories from novels. The only source of supervision in our models is from Microsoft COCO captions.

That is, we did not collect any new training data to directly predict stories given images.



Clone Repository

$ git clone

Create a virtualenv

$ virtualenv -p python3 venv

Source Virtualenv

$ source venv/bin/activate

Install Python Dependencies

$ pip install --upgrade pip setuptools wheel
$ pip install -r requirements.txt

Run Server

python runserver
Running on (Press CTRL+C to quit)


Ryan Kiros, Yukun Zhu, Ruslan Salakhutdinov, Richard S. Zemel, Antonio Torralba, Raquel Urtasun, and Sanja Fidler

"Skip-Thought Vectors." arXiv preprint arXiv:1506.06726 (2015).

  title={Skip-Thought Vectors},
  author={Kiros, Ryan and Zhu, Yukun and Salakhutdinov, Ruslan and Zemel, Richard S and Torralba, Antonio and Urtasun, Raquel and Fidler, Sanja},
  journal={arXiv preprint arXiv:1506.06726},

Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew M. Dai, Rafal J{'{o}}zefowicz Samy Bengio.

Generating Sentences from a Continuous Space

      title     = {Generating Sentences from a Continuous Space},
      journal   = {CoRR},
      year      = {2015},
      biburl    = {<}>,
      bibsource = {dblp computer science bibliography, <}>

Zhang, Xingxing, and Mirella Lapata. EMNLP. 2014.

"Chinese Poetry Generation with Recurrent Neural Networks."

  title={Chinese Poetry Generation with Recurrent Neural Networks.},
  author={Zhang, Xingxing and Lapata, Mirella},

Built With

Share this project: