Dental Trainer

Solution Architecture

Inspiration

Students taking dental exams have issue to pass it as no reference in something like that so that's why we create that simulation.

What it does

We are developing a Live AI Agent that simulates a dental patient encounter to help postgraduate dentists prepare for fellowship and clinical examinations. The system allows dentists to practice history taking, patient assessment, clinical decision-making, consultation, and diagnosis in a realistic exam-like scenario. This interactive training environment helps clinicians strengthen their clinical reasoning and communication skills before their actual exams.

How we built it

Real-Time Multimodal Stream (ADK + Live API)

Unlike standard REST APIs that send text back and forth, architecture uses WebSockets. This allows for a continuous flow of PCM audio data.

Affective Dialog & Persona Grounding

System isn't robotic; it adapts its emotional tone. Detail: By passing "Emotional Profile" as an injected context in engine.py, the Gemini 2.5 Flash model adjusts its vocabulary (e.g., "I'm worried" if the doctor is saying something to worry about).

Silence Monitoring

After 5 sec. agent detect that no one is talking send a nudge to human if still there.

Interuption Handling

Interuppt the agent and check stopping and preserving context.

Clinical Accuracy:

Grounded in YAML knowledge bases (RAG).

Challenges we ran into

Handling VAD (Voice Activity Detection), silence monitoring, and interruption management within the Live Agent API was one of the main technical challenges.

Accomplishments that we're proud of

We successfully integrated these components so they work seamlessly together, creating a smoother and more reliable live interaction experience.

What we learned

Gemini Live API first time to use it sending request (audio, image), handling VAD Functionality from abstract level not from models like open source model.

What's next for Dental Trainer

First-time implementation of the Gemini Live API to process multimodal inputs (audio and images), with Voice Activity Detection (VAD) managed through the API’s abstraction layer instead of custom or open-source model handling.