ClassClarity

Inspiration

A lecture doesn’t pause when a student encounters an unfamiliar word. For students studying in a second language, looking up one term can mean missing the next explanation. We wanted to help students follow lectures while building their understanding of the concepts and vocabulary they find difficult.

What it does

ClassClarity provides live translated captions, explanations on demand, and a personal glossary.

Our current demo transcribes English speech and displays Japanese captions. Students can stop transcription, select a confusing phrase, and request a translation and spoken explanation in their chosen help language.

ClassClarity saves those terms and explanations. When a term appears again, it highlights the phrase and brings back saved help. It can also suggest related concepts when the lecturer uses different wording.

How we built it

We built the interface with HTML, CSS, and JavaScript, supported by a Python and FastAPI backend. WebSockets carry microphone audio and caption updates.

Gemini handles live transcription, Japanese captions, and contextual explanations. ElevenLabs turns requested explanations into speech. TiDB stores each student’s glossary and uses embeddings to find concepts with similar meanings.

Exact phrase matching runs in the browser, while semantic searches run in the background so captions can continue updating.

Challenges we ran into

Frequent AI requests initially caused rate limits. We moved captions to a continuous audio connection and reused saved explanations to reduce repeated generation requests.

Another challenge was finding relevant glossary entries. Shared words could produce misleading matches, such as confusing “feedback form” with “feedback loop.” We added checks against the saved lecture context to improve relevance.

We also had to coordinate captions, database searches, and audio playback without making the interface difficult to follow.

Accomplishments that we're proud of

We connected lecture translation with a glossary that remembers the concepts a student chooses to save.

In a small browser replay using the real database, glossary matching retrieved all eight intended matches and rejected three unrelated examples. We also checked saving, refreshing, and removing entries. These checks give us a foundation for testing with real lectures.

What we learned

We learned how to connect streaming audio, AI explanations, speech synthesis, and persistent storage in one interface.

We also learned how much context matters: similar wording does not always mean the same concept. Showing the original example alongside saved help makes explanations easier to assess.