TellStories.ai

Inspiration

Receiving treatment at a hospital stay can be intimidating for younger patients. At the same time, reducing their stress and anxiety is also important for the healing process. How can we then help to improve their experience during their stay? This is where TellStories.ai comes in - to give them personalized and engaging stories to enhance their experience during their hospital stay.

What it does

TellStories.ai is an innovative storytelling app that helps to generate illustrated personalized stories for patients. It is powered by generative AI technologies such as LLMs and text-to-image models and is designed for children aged 14 years and below.

How we built it

We experimented with several LLMs and prompt engineering to generate suitable, personalized stories based on both instructional prompts as well as input parameters from the user. In the end, we used the Falcon 7B LLM as well as the Stable Diffusion XL model for generating short stories and their illustrations respectively.

Challenges we ran into

One of the key challenges was shaping the LLM response to generate appropriate and relevant content given the context parameters provided by the user, since it had to balance both appropriateness as well as creativity in coming up with the stories. We also encountered a similar issue when using the generated text scenes with the text-to-image model and had to experiment with different prompts and settings to ensure that the images were relevant to the scene.

Accomplishments that we're proud of

We were most excited about the ability to use the model output from LLMs for additional purposes, which is generating images in this use case, and fine-tuning the prompts in order to get appropriate output to suit that purpose.

What we learned

Through this hackathon, we have learnt much about the strengths and limitations of several LLMs and specific prompt engineering techniques to overcome some of those challenges. At the same time, we also had the opportunity to learn how to use and configure the LLMs through platforms such as HuggingFace to achieve the desired outputs. Thanks DataBricks for this opportunity!

What's next for TellStories.ai

As for the future, we hope to refine the underlying pipelines for the LLM as well as the solution to generate even more creative stories with different settings and possible multi-modal inputs to give users a more personalized experience!