Inspiration

Frustration at the inconvenience that comes with looking up a character without knowing the English meaning or how to pronounce it. Wanting to expand on Google Translate's photo analyzing translation feature to include more details on a word's usage. Avoid having to use the writing feature for complex characters since they are finicky and take up time to write correctly.

What it does

Using a camera (webcam or Android camera), our demo recognizes a Japanese character and shows a detailed dictionary entry to the user in real time; always readable no matter what angle you hold your device because the entries always face the viewer.

How we built it

We used Unity with Vuforia to create the program, with assistance from Adobe Illustrator and Photoshop to create the image assets used.

Challenges we ran into

We had issues porting our Unity project into Android, despite help from multiple sources (mentors, peers, YouTube videos, Google, etc.) We also had issues with making sure our targets could be detected well.

Accomplishments that we're proud of

Succeeding in learning a new software for the first time in a short amount of time.

What we learned

The importance of having a game plan before the event starts. There's always unexpected issues that pop up. Persistence is key. Keeping a positive attitude and being grateful for this experience.

What's next for Kotoba - AR Japanese Character Dictionary

For users to:

1. Hear pronunciation if a sound icon is tapped
2. See a breakdown of the radicals and characters within multiple characters
3. Save terms to an in-app album to keep track of learned characters

Additional functionality:

1. Building a database of words and definitions for characters
2. Training AI to read characters in different fonts, colors, and conditions
3. Chinese character support

See an overview of our demo here!

Built With

• adobe-illustrator
• android
• photoshop
• unity
• visual-studio
• vuforia