Yo team Narratoon

INSPIRATION

Storytelling is a powerful tradition, enhancing the minds of developing children!

We’ve all read classics like The Very Hungry Caterpillar or Good Night Moon and wished there was more to the story. The problem is, turning those ideas into illustrations isn't easy when you can’t draw!

What it does

With Narratoons, developing more story is now possible! Educators, families, and dreamers can speak their story aloud and bring it to life with personalized illustrations of their ideas!

Our project uses speech-to-text and generative AI to enhance your narrative and create matching illustrations. In just minutes, you can have your own illustrated short story.

Note* Only works if you run the backend on your side

How we built it

We built the project by combining our efforts into backend and frontend development. On the front end side, we used HTML for the interface and included simple interactions with buttons to guide the user. We included a microphone to capture the user speech, and included features to display loading and connection status. After generation, the site shows a slideshow of illustrations and story text. On the back end side, we used Flask for seamless integration, and had an API connection to handle the live data. We used speech to text to convert the user voice message and then used this text to generate the page prompts. After that, we used the page prompts to create the illustration.

Challenges we faced

Some challenges we ran into: The main challenge was with ngrok tunneling. It wasn't working so the api couldn't be hosted publicly. The temporary fix we had was to run the backend on our end, but this makes it so the server only connects when the api is ran locally. Furthermore, open-ai doesn't support giving images as a prompt to generate images, so we had to parse user input into open ai to make it into prompts for separate pages.

What we're proud of

We are proud to have connected our functions together very well with the api. The backend to frontend communication with the API was very seamless, we were able to use Python in backend and the HTML in frontend instead of having to restrict ourselves to one language, which was pretty awesome! We also collaborated really well, considering that we couldn't meet in person. In our opinion, it was a successful hackathon and a wonderful group experience.

What's next

As we begin the first semester of our senior year, we will be experiencing a lot of growth as we search the job market and improve our resumes. Team YO will be facing this head on, and our first step will be to figure out how to host our server publicly without having to run the backend, so we can present our project proudly. We aim to create a successful project where many can use to fulfill their dreams.