Inspiration
I am an avid learner of the Japanese language and I have always been interested in creating resources that help myself and anyone else also on the Japanese language learning journey, to improve in Japanese conversational skills.
What it does
It is a Japanese voice chat web application for conversation practice. It leverages Chrome’s built-in AI Prompt and Translator APIs if available, and falls back to Google's server-side Gemini API if not.
Users can speak or type in Japanese and receive real-time responses from the AI in Japanese speech and text. They can also translate the AI's responses to English for better understanding.
How we built it
The web application is built with the Next.js framework. On the frontend, users can send chat messages via text or audio recorded via the browser's Web Speech API. When a user's input is received, AI responses are generated with Chrome's built-in Prompt API if it is available, and if not, with the Google Gemini API. The AI responses are read aloud to the user using Web Speech API. The user can also translate selected AI responses to English by clicking on a response. The translation will be done via Chrome's built-in Translator API if it is available, or otherwise the Google Gemini API. For security reasons, there is a backend layer also built in Next.js, which is responsible for making calls to the Google Gemini API on behalf of the frontend.
Challenges we ran into
As Chrome's built-in APIs are still in development, there are understandably less resources and code samples available for reference.
Accomplishments that we're proud of
- Being able to leverage the built-in APIs for language learning use
- Being able to launch the project in time for the hackathon as a live web application @ www.aijapanesetutor.org so users can easily try it out
What we learned
The responses from the built-in Chrome AI are unstructured by default and occasionally not returned in the expected format. I look forward to improving the application's ability to solicit consistently structured responses or handle unexpected responses better.
What's next for AI Japanese Tutor
More Japanese learning features, such as for vocabulary, as well as greater personalization for logged-in users.
Log in or sign up for Devpost to join the conversation.