Inkheart

• 

  Example use of inkheart with a description of a dragon from The Hobbit.

Inspiration

We wanted to augment the experience of storytelling by providing visual and audio aid to help engage and immerse the users of our project. We recognise that it can be harder for some children to enjoy the experience of reading, but we hope to make these challenges lesser.

What it does

As someone narrates a story, our program converts speech to text, then analyses the words to generate relevant sound and image to augment the experience.

How we built it

We built the project using a basic JS and HTML front end which would use a users microphone along with a speech to text API to send text to the backend wrote in Flask. The backend would send the text to a set of python scripts to process the text, create the images, and choose any appropriate audio files. The backend would send this back to the front end to be displayed in real time for the user.

Challenges we ran into

-Finding an appropriate sound library and API for image generation. -Dealing with async functions in JavaScript -Hosting flask programs

Accomplishments that we're proud of

-The project worked as intended!

What we learned

-Readers are leaders! - Shanai (5am) -Using flask -Using stable diffusion

What's next for Inkheart

-Spending more time on refining image generation

Built With

• ai
• flask
• html5
• javascript
• python
• stablediffusion