Our team was inspired by the never-ending strength of parents who are juggling their work and the needs of their children during the pandemic. As parents are struggling with balancing between work expectations and making sure their children have high-quality education at home, they are also concerned with their children losing out on interactive experiences. Interestingly enough in a survey from late 2020, parents are overwhelmingly accepting of using video games as a form of education and most purchased laptops for e-learning at home. Because of these insights, we created Rhyme Time, an interactive e-learning site that helps your child with reading and phonics by testing your child’s reading and pronunciation through speech-to-text.

link to video demo

What it does

Children or parents select a book. Once a book is chosen, a voice reads the first page aloud while corresponding words are shown on-screen for the child to follow along. After the audio is finished, the site stops and asks the child to repeat what was heard. The site listens to what the child says and how the child says it, noting abnormal pronunciation, and then corrects the child if any mistakes were made. This process repeats for every page in the book until the book is finished. We used all original graphics assets created by our team and copyright free music.

How we built it

We used many libraries including:

  • Google Cloud to process speech and convert it to text for photo processing. We also used the Google Cloud service to convert text to speech wherever necessary, particularly for the prompts where if a mistake was made, it could be identified, and tell the user to repeat the incorrect word.
  • Freely available libraries like OpenSeries for video.
  • Trello for project management.

Once we created an idea we got together brainstormed and created a work breakdown structure, figured out all the tasks we had to do. We used another technique called scrum to keep track of everyone’s tasks and monitor progress. We each took a task and continued doing the tasks in different categories.

We also created videos (iMovie) and graphics (Procreate on iPad and Apple Pencil) showing the interactiveness of it and royalty free music that appeals to the kids.

Non-technical flow: Intro -> Rhyme time app reads the story -> You repeat it after -> if you get it wrong, the apps repeats the wrong words ->If you get it right, you move on to the next page.

Challenges we ran into

  • Audio files and making the video work, went around it
  • Initial difficulty connecting to Google Cloud and accessing those tools
  • Reusing image assets and try to appeal to our targeted audience

Accomplishments that we're proud of

  • Working in a diverse team of different skill sets
  • Great communication and team project management (utilized Trello)

What we learned

We spoke to mentors and had to figure out the required backend tools and how to implement text to speech conversion. Also, for three of us, it was a first hackathon experience.

What's next for 1

Diversify our book collection to suit more readers’ interest. For instance, we would like to include different genres and books that are designed to aid various learning gaps (emphasizing various sounds or words, ie. long “a” vs. short “a” sounds). Also, our idea can be expanded as a supplement to a language curriculum for non-native speakers trying to learn English. We would like to suit parents’ or educators’ goals by creating an educator login to monitor their child’s or student’s progress and assign books geared to address and strengthen specific learning objectives.

Share this project: