AccentMirror

Inspiration

The idea for Accent Mirror came from a very personal frustration. As someone who personally wants to improve my own pronunciation in a foreign language, I found that educational apps often give generic, opaque scores—like "72% accuracy"—which offer little actionable value. A percentage doesn't tell a learner what the listener actually understood or how to physically correct the mistake.

I realized that effective communication isn't about eliminating an accent; it's about intelligibility. I wanted to build a "phonetic mirror" for myself and others that shows exactly where communication breaks down and provides an immediate, highly targeted micro-drill to fix it.

How we built it

Accent Mirror is a full-stack application built with Next.js (App Router). The core learning loop relies heavily on the official OpenAI SDK and two primary models:

1. Unbiased Listening (gpt-4o-transcribe): We capture a short audio burst (under 10 seconds) from the browser and send it to the server. We use the transcription model to interpret the raw audio without prompting it with the target text. This acts as our honest listener.
2. Pedagogical Engine (gpt-5.6-sol): If the server detects that the learner said the expected phonetic confusion (e.g., saying "sheep" instead of "ship"), we call GPT-5.6. We use strict Structured Outputs (Zod) to constrain the model to our approved curriculum. GPT-5.6 acts as an expert tutor, generating a personalized explanation in the learner's native language and a 3-step micro-drill.
3. Reference Audio (gpt-4o-mini-tts): Target phrases are pre-generated using OpenAI's TTS to provide clear, native-sounding reference audio.

We intentionally kept the architecture lean: there is no database or persistent audio storage. All progress is kept in an ephemeral React reducer state to prioritize user privacy and speed.

Challenges we faced

• Preventing Abuse: When designing the architecture for a public-facing educational tool, we needed a way to control API costs without hiding behind a login wall. We solved this by implementing a rolling in-memory rate limiter (max 8 attempts per 10 minutes per hashed client) and designing the app to be fully compatible with external Web Application Firewalls (like Vercel WAF) and strict OpenAI project spending ceilings.
• AI Hallucinations in Education: We didn't want the AI inventing incorrect phonetic rules. By forcing gpt-5.6-sol to use a strict schema and constraining its context to our curated JSON curriculum, we ensured the pedagogical feedback is always accurate and safe.
• Deterministic Testing: Testing probabilistic AI models in CI/CD is notoriously flaky. We built a deterministic server-only fixture adapter (ACCENT_MIRROR_MODEL_MODE=fixture) that allows our Playwright E2E tests to run the full browser learning loop reliably without consuming actual OpenAI API credits or exposing keys.

What we learned

• Focus beats breadth: Providing one highly targeted micro-drill on a single sound is significantly more effective than offering a broad analysis of an entire sentence.
• Structured Outputs are game-changing for EdTech: Using GPT-5.6 with strict schemas allowed us to seamlessly blend AI reasoning into our React UI without worrying about parsing errors or unexpected markdown formatting.
• Intelligibility > Perfection: AI is an incredible tool for building confidence in learners when we frame feedback around "being understood" rather than "sounding perfect."

Built With

• css
• gpt-4o
• gpt-5.6
• html5
• javascript
• next.js
• node.js
• openai
• openai-tts
• playwright
• react
• typescript
• vercel
• vitest
• zod