Calligraphy AI

Inspiration

I was inspired by a very simple observation: a calligraphy notebook can contain alphabets, practice lines, and stories for many scripts. What if this notebook existed on a phone and contained every script that the learner would ever encounter? This thought of aggregating the scripts you would find in a notebook into one application is what Calligraphy AI is based on.

What it does

Calligraphy AI is a multilingual handwriting and literacy app that helps users learn to write different scripts and languages through interactive practice. The app includes all popular scripts used across the world, including Latin, Cyrillic, Greek, Arabic, Hebrew, Hiragana, Katakana, Korean, Hindi, Thai, Tamil, Bengali, Georgian, Armenian, Ethiopic, Chinese, and many more. Story-based learning is supported in 42 languages. Users learn to write by tracing on a screen or on paper. They are also able to practice their writing by listening to an AI voice. They get instant feedback on their writing through a camera-based handwriting verification system. An AI chat tutor is also available to guide users in real time. The app is gamified with badges and short stories to make learning simple and accessible to users.

How we built it

For Calligraphy AI, I used a low-code/no-code development methodology for quick prototyping and development cycles. I used Lovable AI for designing the interface and connecting interactive elements such as tracing, camera, AI narration, chat tutor, etc. I developed story packs and writing exercises for multiple scripts, and used TTS for pronunciation support as well. Camera-assisted handwriting verification was also implemented for comparing user input with reference forms, and offline content packs were also developed for users to use without requiring constant internet access.

Challenges we ran into

The biggest challenge in handling voice and accent variations for languages was handling language varieties such as English (US, UK, AU), as well as pronunciation quality for low-resource languages. Another challenge was data availability for certain scripts, which affected handwriting verification accuracy. There was also a challenge with mobile performance and offline capabilities, which affected model size and camera reliability in low-light conditions. Lastly, creating an interface that accommodates very diverse writing systems also posed a challenge, as animations for writing strokes varied for each script.

Accomplishments that we're proud of

One of the significant challenges was addressing voice and accent variability across languages, including various forms of the English language such as 'US', 'UK', 'AU', etc., and pronunciation quality for low-resource languages. Another significant challenge was the lack of data available for some of the scripts, which affected the accuracy of handwriting verification. Additionally, there were limitations on mobile performance and offline use that affected the size of the model and camera reliability, especially in poor lighting conditions. Finally, a single interface that can be effective for a wide variety of scripts requires script-specific animations for strokes.

What we learned

The experience of building the project showed me that scripts can carry history, culture, and very distinct writing logic. I learned about the strokes of scripts, families of scripts, and the need for very different affordances in some scripts. I also learned about the power of quick product development with low-code/no-code: how to prototype quickly, iterate with users, and integrate pre-built AI services with your own content.

What's next for Calligraphy AI

These challenges have helped make the product stronger, with community-sourced content, adaptive TTS, and script-specific UX as key priorities going forward. The next steps are focused pilots with partner schools, community authorship, and better local model inference to ensure that more learners benefit from using Calligraphy AI and make real literacy gains.

Built With

• ai
• lovable