Assistance Simulator

AI voice simulations that turn assistance procedures into measurable, adaptive coordinator training.

Comment 2

Inspiration

Assistance coordinators make high-stakes decisions while speaking with distressed clients, hospitals, family members, and service partners. Traditional onboarding relies on shadowing, static scripts, and occasional role-play. It is difficult to repeat consistently, hard to evaluate objectively, and expensive to scale across countries and departments.

Assistance Simulator turns process methodology into realistic, measurable call training.

What it does

The platform models the complete training workflow as separate, auditable entities:

Template -> Scenario -> Assignment -> Simulation -> Evaluation

  • Templates capture reusable process methodology: required information, process steps, critical errors, traps, and evaluation weights.
  • Scenarios turn that methodology into a concrete situation with a country, hospital, caller language, symptoms, incident date, hidden facts, difficulty, and emotional profile.
  • Assignments deliver scenarios to individual coordinators or organizational groups.
  • Simulations run as live voice calls. The AI caller reveals information progressively, reacts emotionally to the coordinator's behavior, and becomes calmer when the call is handled well.
  • Evaluations combine the template, scenario, transcript, required-question evidence, traps, and process rules into understandable scores and coaching guidance.

After a call, the coordinator and trainer can review the transcript timeline, recording evidence, process score, communication score, missing information, captured and missed traps, confidence warnings, and a concise coaching summary. The new adaptive coaching layer proposes the best next training scenario from the evidence instead of ending at a score.

The broader product includes scenario packs, retry comparison, skill matrices, calibration, manual trainer review, PDF reports, notifications, audit logs, AI incidents and cost monitoring, country and branch scoping, role-based dashboards, and Czech/English localization.

How I built it

The application uses Next.js, React, TypeScript, Tailwind CSS, Supabase Auth/PostgreSQL/Storage, row-level security, Google Gemini Live for the voice caller, Gemini for structured evaluation and transcript polishing, Google Translate for dynamic release-note content, and React PDF for stable reports.

Codex with GPT-5.6 was used as an engineering collaborator inside the real codebase. It inspected existing conventions, traced multi-step simulation and evaluation flows, implemented the adaptive next-training recommendation, created an isolated Build Week judge environment, checked role and country boundaries, and repeatedly validated the project with lint and production builds.

A key architectural decision was to preserve clear domain boundaries. A template is not a scenario, an assignment is not a simulation, and an evaluation remains attached to the historical attempt. This makes retry comparison, version auditing, calibration, and future Auto Assistance and Home Assistance modules possible.

GPT-5.6 and Codex contribution

The Build Week work focused on converting evaluation evidence into a useful next action. GPT-5.6 and Codex helped design and implement a structured recommendation contract containing:

  • a human-readable coaching summary and rationale;
  • focused skills and measurable objectives;
  • recommended difficulty;
  • a trainer-only follow-up scenario proposal;
  • suggested traps and hidden facts.

Codex also created repeatable judge seed data and verified that judge accounts are isolated in their own country and branch scope. The README documents the exact testing path and synthetic credentials.

The production voice and evaluation runtime remains Gemini-based. We intentionally describe that boundary accurately: GPT-5.6 and Codex accelerated product engineering, reasoning, implementation, and validation; they are not presented as the live voice provider.

Challenges

The hardest part was synchronizing a realtime voice stream, partial transcripts, recording uploads, and explicit end-and-evaluate behavior without duplicates or lost evidence. Czech speech transcripts also required multilingual sentence-level post-processing because realtime ASR can split words unpredictably.

A second challenge was trustworthy evaluation. We needed to distinguish whether a coordinator asked the right question from whether the caller actually knew the answer. Required-question tracking, evidence links, confidence flags, fallback detection, prompt/model audit data, and trainer review make the score explainable.

The third challenge was authorization. Trainers, coordinators, admins, and superadmins have different capabilities, while country, branch, department, and subgroup scopes must prevent accidental cross-organization visibility.

Accomplishments

  • A working end-to-end voice training flow for assistance coordinators.
  • Evidence-based evaluation with required-question tracking.
  • Adaptive next-training recommendations instead of score-only feedback.
  • Emotionally responsive AI callers.
  • Trainer review, retries, calibration, skill matrices, and scenario packs.
  • Searchable transcript evidence linked to recording timestamps.
  • Stable PDF reports with Czech diacritics.
  • Country-scoped RLS and isolated synthetic judge accounts.
  • Operational tooling for audit, health, AI incidents, and AI costs.

What we learned

AI training systems need more than a convincing conversation. They need explicit process methodology, evidence, versioning, confidence reporting, human review, and security boundaries. The most valuable output is not a single score; it is a defensible explanation of what happened and a concrete next exercise.

What's next

Next steps include broader validated methodology packs for Medical, Auto, and Home Assistance, longitudinal competency analytics, additional production languages, deeper calibration workflows, and controlled enterprise pilots with trainer-defined rubrics.

Built With

Share this project:

Updates