The Infinite Story

The Infinite Story
Mystical landing page with genre and language selector
Cooking up a story - loading page
Select from a list of story genres
Read and listen to the narrated stories in your language of choice
Chapter being narrated
Chapter has been narrated, user must now select an option

Inspiration

Ever since watching the Bandersnatch Black Mirror episode - an interactive choose your adventure -thriller, I've wanted to create something similar, one way or another. When this opportunity to create something with AI, I had to bring this to life.

What it does

It's a web app which generates narrated stories, in choose your own adventure style. The user, after listening to a chapter, is presented with two options to choose from - their choice determines what the next chapter will be. There's an infinite number of chapters which can be generated, hence the name "The Infinite Story"

How we built it

The frontend interface is an Angular SPA which connects to a NodeJS/Express server via Rest API. The server handles the communication with the various AI services (Vertex AI & OpenAI) for text, TTS and image generation, the results of which are sent back to the web app for UI rendering. I developed a standard prompt which gets fed to Vertex AI to generate the unique stories and chapters, including dynamic variables to allow customisation and story continuity.

Challenges we ran into

Ensuring a consistent result format from the Vertex AI was the biggest challenge, despite me designing a detailed prompt with information on how I expect the response to be formatted (pure JSON object), it often returned malformed JSON which special characters, line breaks and often contextual words outside of the {}. I overcame this as much as I could by introducing additional JSON parsing steps which cleansed the results as much as possible before attempting to parse to JSON, I also introduced a re-try mechanism which would re-generate the text up to a defined number of times if the JSON parsing failed - from my testing, the highest number of re-tries was generally 2-3. Another challenge was how much extra processing time the image generation added to the process, as such, I decided to generate an image once with the first chapter to be displayed throughout the entire story, instead of generating a new image for each chapter. There are also some quota limits with the Vertex AI API which result in errors, this could be overcome with quota increases in the future hopefully.

Accomplishments that we're proud of

I'm proud of the way in which I handled the JSON parsing and re-try functionality on the server side to ensure that the user received a successful response as quickly as possible. I'm also proud of the way I handled the loading of subsequent chapters while the current chapter is being narrated, this way the user is able to play the next chapter without seeing a loading spinner. E.g., while the user is listening to Chapter 1 which will present the user with two options which will inform Chapter 2, both of those options are being fetched in the background from the server, so that when the user selects on of the two options, the next chapter can play instantly.

What we learned

I learned that cost of AI processing is too high right now to allow for wrapper apps to exist without monetising, unless significant usage restrictions and introduced. I also learned that Google's Vertex AI is far superior at generating highly engaging and natural sounding stories than any other AI service i've used.

What's next for The Infinite Story

If I decide to continue with this project beyond the Hackathon, I would look to monetise it in some way as the AI costs would be too much to bear otherwise. I would also look to improve the UI, improve the speed and test a swipe/scroll feature to load new chapters and stories in a more interactive manner. I would also like to explore ways to incorporate different images throughout the story, without adding extra processing time and keeping costs minimal.