The best way to learn a language is to practice with a native speaker. Many immigrants in the US struggle to learn English, and there is only one English as a Second Language (ESL) instructor for every 150 ESL students! Because of this, there is a growing need for conversational English education resources. Resources with live conversational practice like Omegle and chatrooms are often unsafe and full of unverified individuals. Speaking with an AI model trained to filter out toxicity is the safest way to practice speaking in the comfort of your own home, without potential embarrassment of making a mistake in public spaces.

What it does

Introducing KittyChat, an interactive website to make English conversation practice more accessible with ChatGPT-powered talking cats. Our demo scenario features Allin, a cat barista, who is taking your order at the Cat Café. You can even ask her about the soup of the day or her personal recommendations, and she will respond to you! KittyChat features:

  • Dynamic dialogue using AI
  • TTS/STT to make you feel immersed
  • Realistic scenarios to practice everyday vocabulary
  • Cute cats!

KittyChat > ChatGPT + Duolingo + Siri for language learning. KittyChat supports oral language practice, which is the most valuable and scarce resource among all aspects of language learning. Furthermore, it supports context-based learning and could maintain conversation flow according to the user's input.

How we built it

We prototyped the front-end design using Figma and Canva before converting it to HTML/CSS. Using Javascript, we added buttons to make the site interactive. On the back-end, we utilized NLP software, namely OpenAI's ChatGPT API and Whisper, to build a processing pipeline that recognizes users' voices, queries the LLM for natural dialogue text, and converts back to speech for an immersive experience.

Challenges we ran into

For most of our team, this was our first hackathon, and we had limited experience in developing a pipeline and website. We were figuring out how to use Figma for the first time, and it was difficult to translate the prototype designs to HTML and make them interactive and pretty. The ChatGPT in our program also doesn't save the context of a conversation, so we had to implement continuity by updating the prompt with the chat log. Finally, it was extremely difficult to integrate the website with the Python engine and have the user and Allin communicate with each other via Flask. We would implement this given more time, but currently, you can only chat with Allin in a terminal. In the future, we would love to unite the front and back ends.

Accomplishments that we're proud of

We're quite proud that we created a feedback loop between the user speaking, the microphone recording, and the response being played. The website also has a clean design.

What we learned

We learned how to use Figma to prototype and how to use ChatGPT in a program.

What's next for KittyChat

  • Allow the website to interface with the ChatGPT data.
  • Improve the design. We could use a variety of TTS voices and include hints such as a transcript or translation to another language.
  • Add more scenarios like running into an acquaintance at a park or conversing with a waiter at a restaurant.
  • Support conversations in other languages. Our program already supports multiple languages, so this would be more of an interface task.
Share this project: