GPT3 became a very powerful tool for text generation, able to produce thoughtful plots with very little fine tuning. Inspired to generate entertaining and engaging stories for children, we did not want to stop at only text procedural generation, but also turn the given text into cute, simple animations, which would help toddlers and small children alike develop their reading and creative skills.
What it does
It generates short fantasy stories to text, and from the text generates an animated version of the story.
How we built it
GPT3 DaVinci for story generation, Flair for text-to-POS (Part of Sentence) feature extraction. Used to generate the animations, from the given elements within it. Flask, for back-end deployment. React, for front-end coordination. 15+ hours continuous drawing in Krita (for animated templates)
Challenges we ran into
Turning generated text into features, mainly the actors participating in the drawing, was a non-trivial challenge. Particularly when pronouns were being generated, and there was ambiguity in terms of which pronoun is used for which subject.
Placement of generated animations in the front-end in a reactive, fluid way.
Accomplishments that we're proud of
Fine-tuning DaVinci's GPT3 to generate children stories indistinguishable from human authors. Hand-crafting so many animation examples in so little time.
What we learned
React sucks. NLP is fun. Task assignment is a life saver.
What's next for StoryTelly
Use web scraping and "A neural network of artistic style" for generation of characters/monsters/environments which have not been hand-drawn yet. By training the network on the drawing style of our artist, and using search engines to retrieve drawings of new elements, we should be able to fill in the gaps which our finite human existence inherits.
Fine-tuning another NLP model to generate titles for stories automagically.
Log in or sign up for Devpost to join the conversation.