KopiTalk

• 

  This is the Mystery Island where users explore the land and play the games

• 

  This the collection page where users can all the cards they have collected

• 

  This is the capture words page where can type out the word or use the camera feature

• 

  This is the friends page where users can gift or duel with friends

• 

  This a game mode for users to play

Inspiration

The app inspiration is a combination of ideas mixed together. When we heard read about the problem statement, the first thing we thought about was Duolingo and we believed that other groups would pivot to that direction as the statement also did mention about gamify learning Mother Tongue language. We had to stand out in order to make this app into a success. The app has to be engaging but also educational. What set the foundation for our app was the island navigation. This idea was inspired by one of HunterXHunter's Arc but I did adapt it to this problem statement. This became about the user navigating on this game island to find cards to improve on their Mother Tongue language. There was another feature that got implemented with the inspiration of Pokemon Trading Card Game but with a twist, users are using the cards they earned on the island to face boss battles and there's also the ability to gift cards to friends without taking away the educational element. Another feature that stands out is our Festival Treasure Hunt where users find objects within the list and once found, it becomes a special festive card. This was implemented to give a reason for users to use this app in their daily life if they see something interesting or related. The core feature was linking it back to the cards.

What it does

KopiTalk helps young adults who already know some of their Mother Tongue to keep practicing it in their daily lives. Users can photograph or paste real test they meet, such as a textbook page, a menu or a message. They can highlight just the words they care about. Each word becomes a collectable card with its meaning, romanization, a rarity and a picture of the word. The collectable cards collected by the user can be playable in the island world. They can play mini-games and talk to NPC to practice their language. The mini-games are Pronunciation, Typing and Composition. Each cards level up from Caught to Trained to Mastered as it is used. Friends can send spare cards as gifts that the receiver must earn it by typing, saying and writing the word, once done, it is added into their collection. Users can also challenge friends to duels whereby it is a competition of speed of composition with 5 questions under a time limit. Seasonal treasure hunts, starting with Halloween, send players out to photograph real festive objects and name them in Chinese. Streaks, daily quests, badges and a profile give players reasons to come back every day.

How we built it

KoptiTalk is plain HTML, CSS and Javascript with no build setup, with a Phaser 3 Island game, hosted on Netlify and useable on a phone web app with the help of Progressive Web App (PWA). Supabase acts as the backend - provides sign-in, a Postgres database with row-level security and server-side functions for rewards, quests, streaks, friends and gifts. Claude is called through the two Supabase Edge Functions that check the player is logged in , an the API key never reaches the brower. One function called - get-word-meaning, reads text from photos, give word meanings and a rarity score, draws card art and checks practice sentences. The other function is called: festival-verify, it judges whether a photo shows a real object rather than a drawing or a screen, then checks the user's answer. If the Claude photo reading fails, Tesseract.js in the browser reads the photo instead.

Challenges we ran into

• Reading photos reliably: We made Claude the main reader with retries and a browser fallback, and added highlight boxes so users can just collect the words they want. Trusting AI output: AI-drawn card cart could carry unsafe markup, so the server rebuilds each SVG file and the app checks it again before showing it. Keeping rewards fair: We moved XP, coins, quests, badges and streaks into database functions so the browser reports what happened and never sets the amount Real objects, not drawings: The festival hunt needs a photo of a real thing, so we designed a two-step check where a word can only be named after its photo has passed. Three languages: Everyone in the group are Chinese speakers, so we cannot verify if the content for the other two Mother Tongue languages are correct and we also did not proceed further with them as well.

Accomplishments that we're proud of

We are happy that A full loop from real-life word, to card, to practice, to reward, built end to end and server-side reward system with daily caps. Additionally, the AI used specific jobs, with fallbacks so a failed AI won't block the user. We are also glad that we made a huge progress from the check-ins about 2 weeks ago.

What we learned

We learned to design row-level security and database functions so users can never edit each other’s data or their own rewards. We also learned Au features need fallbacks and clear failure messages because a silent failure disrupts the whole game.

What's next for KopiTalk

We do want to build the island, composition and typing games for Malay and Tamil, and have native speakers check the Malay and Tamil content. We also like to add more speaking features: voice notes between friends, real-world speak-out missions and speaking duels. Expand the island even more would be a good idea as well.

Built With

• anthropicapi
• css3
• html5
• javascript
• phaser.js
• postgresql
• pwa
• supabase
• supabaseedgefunctions
• tesseract.js
• typescript
• webspeechapi