Chat with Me

Let's learn a language together!

Comment

Inspiration

As the world becomes increasingly interconnected. More people are taking interest in learning a new language. The current language learning applications are structured like quizzes and teach users on vocabulary which isn't very beneficial in day to day conversations. Thus we decided to make an AI chatbot that will allow people from different language backgrounds to practice their conversation skills in a more natural way instead of questions structured as quizzes.

What it does

Our AI chatbot listens and takes the users speech (in any language) then convert to a text input that we feed into the ChatGPT API then once the AI returns a text file it is then transformed into a wav file with voice output of the AI's answer (in the same language that the user spoke in)

How we built it

We initially used google cloud's text-to-speech and speech-to-text API's to do the transformation of the language. However, after consulting with one of the instructors from Avanade, we decided to use OpenAI's Whisper API as well as ElevenLabs API to go between text and speech. Instructors from Avanade motivated us to build our front end with the industry standard React framework.

Challenges we ran into

For our front end we decided to build a website that would run our backend code. However, our initial HTML/CSS code didn't work with our backend python file. Speaking with an Avanade mentor, we were suggested to learn and use React to create the website. We took her advice and followed a react tutorial and created our frontend website.

Accomplishments that we're proud of

We are incredibly proud that we are able create a project with this magnitude with little to no prior experience in full-stack development. We were able to create a functional prototype of an AI chatbot with beautiful figma design for the application in a 12 hour period.

What we learned

We learned how to integrate API's into python and how to use the ChatGPT API's text answering capabilities to answer the user's questions. We also learned how to use the google cloud, ElevenLabs, and Whisper API to transform speech to text data and text to speech.

What's next for Chat with Me

We want to make the front-end design more intuitive and user friendly. We also want to store user responses and assistant responses in a database so any user can pick up a conversation where they last left off. We also hope to add more functionalities such as user auth as well as the ability for users to choose the level of conversational difficulty.

Built With

Share this project:

Updates