SpeakUp AI Tracking

Inspiration

Many English learners practice with AI, but progress feels invisible over time. We built SpeakUp AI Tracking to turn each speaking session into measurable improvement.

What it does

SpeakUp AI provides real-time voice conversation practice and tracks performance across sessions. It stores transcripts, evaluates speaking quality, and shows progress trends with actionable improvement points.

How we built it

We built a React + Vite frontend for live practice UI and a FastAPI backend for session tracking, transcript storage, and analytics. MySQL stores users, sessions, and performance history. Gemini powers live conversation and post-session evaluation.

Challenges we ran into

We faced issues with real-time session reliability, transcript quality, model endpoint/version mismatches, and schema drift while upgrading from demo data to production-style tracking. We also had to enforce strict auth, role control, and session limits without breaking existing flows.

Accomplishments that we're proud of

We moved from a demo-like prototype to a real tracked system: authenticated users, role-based controls, persistent session history, transcript files, background evaluation, and a more usable coaching dashboard.

What we learned

Real-time AI products need strong session lifecycle handling, clear save states, and resilient backend processing. We also learned that UX clarity is critical, users need explicit feedback on when data is saved and evaluated.

What's next for SpeakUp AI Tracking

Next, we plan to add deeper pronunciation analytics, richer admin/global insights, stronger retry and monitoring for background jobs, and personalized learning plans that adapt to each user’s recurring speaking patterns.

Built With

• and
• built-with:-typescript
• cloud
• deployment
• fastapi-(python)
• gemini-live-api-(`gemini-2.5-flash-native-audio-preview-12-2025`)
• gemini-text-model-for-assessment
• google
• mysql
• on
• platform
• react
• sqlalchemy
• tailwind-css
• vite