Inspiration
My child could not retain English vocabulary from a paper word list. Even though writing the words by hand and listening to their pronunciation might help, my child did not practice that way. I also tried a multiple-choice vocabulary app, but repeated play mainly encouraged memorizing the answer choices. It did not require the learner to recall the actual meaning or spelling. My wife eventually asked me to find a better solution, so I designed Wordbook for Build Week.
What it does
Wordbook is a handwriting-first English vocabulary practice app for Japanese junior high school students. The learner sees an English word and its Japanese meaning, writes the word by hand, and receives OCR-based feedback. Correct answers are spoken aloud using text-to-speech, then the app immediately moves to the next word. The prototype includes three difficulty levels: Level 1: copy the visible word Level 2: complete a partially hidden word Level 3: write the word using only its meaning and character count Each session contains up to 100 words and uses a countdown timer. Timeouts reduce the net correct count and increase the required target. If OCR fails, the learner can use a manual pass button, but the prompt and handwriting image are saved for later parent review.
How we built it
I first designed the product behavior and compared a native iPad app with a browser prototype. I chose a static web prototype for rapid iteration and validation. The app uses plain HTML, CSS, and JavaScript. Handwriting is captured on a canvas. OCR uses Transformers.js with the Xenova/trocr-small-handwritten model. Speech uses the browser SpeechSynthesis API, and score history and audit records are stored in LocalStorage. The prototype is deployed to GitHub Pages. I used Codex and GPT-5.6 throughout the process to define the specification, make architecture decisions, implement the prototype, debug OCR, write tests and documentation, and deploy the app.
Challenges we ran into
The biggest technical challenge was browser OCR. The first implementation passed a canvas object directly to the Transformers.js image-to-text pipeline, which caused the error Unsupported input type: object. I reproduced the error, traced the expected input type, and fixed it by passing the canvas as a data URL string. I also had to design a manual-pass flow because browser OCR is not reliable enough to be the only judge of a learner’s handwriting. Another challenge was balancing strict practice rules with usability. The app must push the learner to write repeatedly without making OCR failures feel unfair.
Accomplishments that we're proud of
We turned a personal learning problem into a working prototype during Build Week. The live demo includes the core learning loop: handwriting input, OCR feedback, pronunciation, timed progression, three difficulty levels, timeout penalties, score history, and parent-auditable manual passes. We are especially proud that the app does not merely test recognition. It requires the learner to produce the spelling by hand, which is the behavior we wanted to encourage from the beginning.
What we learned
The most important lesson was how effective GPT-5.6 was at finding and fixing a concrete software bug. Other attempts had failed to resolve the OCR error, but once I asked GPT-5.6 to investigate the actual runtime failure, it quickly identified the incorrect input type and proposed a working fix. I also learned that a browser prototype is useful for validating the game loop and user experience, but it cannot fully reproduce the handwriting quality and OCR reliability of a native iPad app. Finally, long-context development is powerful, but important visual and timing results still need to be checked directly rather than assumed from a plan or script.
What's next for Wordbook
The next step is to rebuild Wordbook as a native iPad app using SwiftUI and PencilKit for a more natural Apple Pencil experience. We plan to evaluate Apple Vision OCR with real handwriting samples from Japanese junior high school students. We also want to improve the vocabulary database with grade level, frequency, and source metadata. Future versions may include richer parent and teacher reports, better OCR fallback handling, missed-word review, and more detailed learning history.
Log in or sign up for Devpost to join the conversation.