Atelier⭑.ᐟ

the wand! ⸝*

Inspiration ₊ ⊹ .

Language apps make it way too easy to mindlessly tap flashcards without actually building muscle memory. We wanted a hands-on, spatial way to force our brains to actually remember complex characters and stroke orders.

What it does ₊ ⊹ .

Atelier lets you learn languages by literally drawing characters in mid-air! Using a custom ESP32 motion-tracking wand, it reconstructs your air-drawn strokes on-screen in real time, grades your stroke order and character shape using custom CNNs, and schedules your practice with an SM-2 spaced repetition loop. It supports Latin letters, Hiragana, and Kanji, features a blind audio-recall mode with ElevenLabs, and even has a real-time 1v1 Versus mode.

How we built it ₊ ⊹ .

Hardware: An ESP32 dev board wired to a 6-DOF IMU (MPU-6050/6500/9250), feedback LED, and two tactile buttons for pen-down and submit.
Trajectory Engine: Integrates gyro and accelerometer data via a Mahony complementary filter to treat the wand like a virtual laser pointer, seamlessly tracking multi-stroke characters across pen lifts.
Recognition & Scoring: Hybrid pipeline pairing geometric vector stroke-order checks with PyTorch CNNs trained on EMNIST, Kuzushiji-49, and custom CJK fonts.
Software Stack: High-speed USB serial bridge streaming 50 Hz samples to a FastAPI backend, WebSockets pushing live canvas cursors to the web frontend, and TimescaleDB handling review analytics.

Challenges we ran into ₊ ⊹ .

Venue Wi-Fi (especially enterprise networks like eduroam) was completely unviable for low-latency streaming, so we scrapped over-the-air MCU networking entirely and engineered a 921,600 baud USB serial bridge to pipe 50 Hz telemetry straight to our local API.
Air-writing doesn't translate well with simple double-integration of acceleration (gravity drift is brutal), so we had to pivot to an angular laser-pointer projection model using complementary filtering.
Handling multi-stroke validation and classifying distinct scripts—from 26 Latin letters to intricate Kanji contours.

Accomplishments that we're proud of ₊ ⊹ .

Making our grand debut into hardware hacking and walking away with a fully functional, end-to-end embedded system.
Building a custom trajectory engine from scratch that actually reconstructs readable air-written letters in real time.
Getting real-time two-player Versus mode working with independent hardware wands battling over WebSockets.

What we learned ₊ ⊹ .

Double-integrating raw IMU acceleration for position is a trap; rotational orientation projection is the way to go.
High-baud serial bridges save hackathons when event Wi-Fi decides not to cooperate.

What's next for Atelier ₊ ⊹ .

Supporting more scripts (Hangul, Cyrillic, Arabic).
Designing a custom PCB and 3D-printed ergonomic wand enclosure so it's no longer a breadboard taped to our hands.
More competitive party mini-games and deck-sharing modes!