Say It First

# Say It First ### *A Neurodiversity-Affirming Conversational Sandbox Powered by Affective Computing & Bilateral Subtext Translation* **Hacklanta 2026 • Social Good Track Flagship** [![Next.js 14](https://img.shields.io/badge/Next.js-14.2-black?style=for-the-badge&logo=next.js)](https://nextjs.org/) [![TypeScript](https://img.shields.io/badge/TypeScript-5.5-3178C6?style=for-the-badge&logo=typescript&logoColor=white)](https://www.typescriptlang.org/) [![Tailwind CSS](https://img.shields.io/badge/Tailwind-3.4-38B2AC?style=for-the-badge&logo=tailwind-css&logoColor=white)](https://tailwindcss.com/) [![Anam AI WebRTC](https://img.shields.io/badge/Anam_AI-WebRTC_Avatar-6366F1?style=for-the-badge)](https://anam.ai/) [![ElevenLabs](https://img.shields.io/badge/ElevenLabs-Conversational_AI-F59E0B?style=for-the-badge)](https://elevenlabs.io/) [![Gemini 1.5](https://img.shields.io/badge/Gemini_1.5-Flash_JSON-4285F4?style=for-the-badge&logo=google&logoColor=white)](https://ai.google.dev/) [![Vitest](https://img.shields.io/badge/Vitest-221_Passed-6E9F18?style=for-the-badge&logo=vitest&logoColor=white)](https://vitest.dev/) [![Accessibility](https://img.shields.io/badge/WCAG-2.1_AAA-10B981?style=for-the-badge)](https://www.w3.org/WAI/standards-guidelines/wcag/) [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg?style=for-the-badge)](LICENSE)
> **"Communication breakdowns between neurodivergent and neurotypical people represent a bidirectional mismatch in pragmatic language styles, not a unilateral cognitive deficit."** > — *Damian Milton, The Double Empathy Problem*

💡 The Problem Space: Ending Forced Masking

For millions of autistic and neurodivergent individuals, high-stakes conversations—such as requesting workplace sensory accommodations, setting boundaries with roommates, or navigating annual performance reviews—carry intense anxiety. Traditional social skills software forces neurodivergent people to mask: suppressing authentic traits to mimic neurotypical conventions at grave psychological cost and chronic autistic burnout.

Say It First re-architects the communication sandbox into a bilateral translation engine:

Translates Pragmatic Subtext: Clarifies what conversational partners mean beneath indirect language and unspoken social subtext.
Bridges Intent vs. Impact: Shows "What you meant" alongside "How it might land" with compassionate, neutral framing ("May..." and "Might...").
Empowers Non-Masking Self-Advocacy: Synthesizes clinically grounded DBT DEAR MAN scripts that achieve accommodation needs without compromising personal boundaries.
Zero Radars or Clinical Judgment: Replaces anxiety-inducing emotion meters and clinical scores with a calm, supportive, co-regulating practice environment.

const sayItFirst = {
  mission: "Bilateral subtext translation without forced masking",
  liveAvatar: "Anam Cara 4 Real-Time WebRTC Video Stream with ElevenLabs Server-Side Native Voice",
  coreMathematics: "Mehrabian-Russell 3D VAD Vector Space [-1.0, 1.0]³",
  audioDSP: "Browser-native 44.1kHz AnalyserNode Autocorrelation F0/RMS (0 kbps cloud upload)",
  turnTaking: "Pre-stream echo protection, streaming delta accumulation, and barge-in immunity",
  safetyProtocol: "Autonomous 3-Tier Tension Index with 4-count visual box breathing reset",
  reasoningCore: "Gemini 1.5 Flash structured JSON inference (p95 < 1400ms)",
  privacy: "100% Zero-Retention ephemeral in-memory DAG session graph",
  unitTesting: "221 unit & integration tests passing across 32 test suites"
};

⚡ Flagship Capabilities

🎭 Sophia Live Avatar (Anam Cara 4 + ElevenLabs)

Photorealistic real-time video conversational partner streaming over ultra-low latency WebRTC via Anam AI, synchronized directly to ElevenLabs Conversational AI Agent voice audio for natural speech and perfect lip-sync.

Scenario-Grounded Openers: Dynamically initializes sessions directly in character with zero generic greeting fallbacks.
Single Lip-Synced Media Stream: Synchronized WebRTC media stream to HTML5 <video> with zero duplicate audio, zero PCM playback artifacts, and server-side signed token exchange.
Dynamic Status Badges: Non-verbal indicators show partner state (Speaking, Listening, Reflecting).

🌉 Two-Way Subtext Bridge (Double Empathy)

Real-time bilateral translation revealing the pragmatic social gap: maps your functional communication intent directly against how it might land emotionally for a neurotypical partner, without judgment or scoring.

Intent Formulation: "What you meant"
Impact Perception: "How it might land"
Neutral Framing: Uses respectful, non-prescriptive language.

🎚️ User-Controlled Guidance Levels

Secondary scaffolding that respects your cognitive autonomy:

Just practice (Independent): Clean, distraction-free conversational immersion.
Show cues: Posture cues and vocal tone hints displayed subtly.
Explore subtext: Full bilateral translation showing partner's unspoken perspective.

🌿 Tension Safety Gate & Co-Regulation

Autonomous mathematical safety monitoring. When conversational tension reaches a critical threshold ($T \ge 9.0$), the simulation halts character interaction to guide an accessible 30-second 4-count visual box-breathing reset (4s inhale, 4s hold, 4s exhale, 4s hold).

🔀 Non-Destructive Attempt Switching

Empowers low-stakes experimentation with "Try another version." Switch between branches, explore different wordings, and preserve draft history across attempts without penalties or progress loss.

📝 Takeaways & DBT Pocket Script

Generates an actionable 2-sentence pocket script grounded in DBT DEAR MAN principles (Describe, Express, Assert, Reinforce) that users can edit, save to their progress library, or copy to deploy in real life.

🏛️ Application Architecture & Layers

Say It First is structured into three distinct user interaction layers managed by the responsive App Shell:

┌────────────────────────────────────────────────────────────────────────┐
│                          SAY IT FIRST APP SHELL                        │
├────────────────────────────────────────────────────────────────────────┤
│                                                                        │
│  [Layer 1: Discovery & Workspace]                                      │
│  ├── Home Dashboard (Curated scenarios, Quick Start, Sensory Settings) │
│  ├── Explore Scenarios (Categorized catalog, search, tag filtering)    │
│  ├── Custom Scenario Creator (Define custom partners, goals, contexts) │
│  └── My Progress & Takeaways (Saved scripts, reflection log)           │
│                                                                        │
│  [Layer 2: Scenario Detail & Pre-Brief]                                │
│  ├── Scenario Briefing (User goal, emotional stakes, partner bio)     │
│  ├── Double Empathy Perspective (Your need vs. their perspective)      │
│  └── Recommended Wording & Practice Mode Selection                     │
│                                                                        │
│  [Layer 3: Live Practice Stage (SophiaLiveAvatar)]                     │
│  ├── WebRTC Video Stream (<video id="anam-avatar-video">)              │
│  ├── Live Dialogue Captions & Turn History                             │
│  ├── Response Dock (Natural typing + Web Audio voice input)            │
│  ├── Turn Pipeline (Delta accumulation, mic pre-mute, speech sync)     │
│  ├── Guidance Drawer (Independent / Cues / Subtext)                    │
│  ├── Calm Pause Modal & Co-Regulation Reset                            │
│  └── Post-Session Comprehensive Conversation Report                    │
│                                                                        │
└────────────────────────────────────────────────────────────────────────┘

📐 Mathematical Affect Engine

Discrete-Time Exponential Damping

Partner emotional state evolves smoothly over time in Mehrabian-Russell 3D Valence-Arousal-Dominance space, preventing abrupt visual or tone snapping:

$$\mathbf{E}(t+1) = \lambda \cdot \mathbf{E}(t) + (1 - \lambda) \cdot \Delta \mathbf{E}_{\text{input}}$$

Where:

$\mathbf{E}(t) = [V, A, D]^T$ is the current emotional coordinate vector, bounded within $[-1.0, 1.0]^3$.
$\Delta \mathbf{E}_{\text{input}}$ is the turn perturbation generated by structured LLM reasoning.
$\lambda = 0.65$ (adaptable peers) or $\lambda = 0.85$ (stressed supervisors).

Tension Index & 3-Tier Safety Gate

The Tension Index ($T$) quantifies conversational stress in real time:

$$T = \left(\frac{A - V + 2.0}{4.0}\right) \times 10.0 \in [0.0, 10.0]$$

Tension Score	Safety Tier	System Action
$T < 7.0$	Tier 1: Normal Simulation	Standard dialogue progression and subtle partner state shifts.
$7.0 \le T < 9.0$	Tier 2: Impact Notice	Emits gentle guidance notice: "Partner may feel defensive or time-pressured."
$T \ge 9.0$	Tier 3: The Curtain Drop	Intercept modal halts simulation and guides a 4-count visual box-breathing co-regulation reset.

🏗️ Hexagonal Architecture Topology

The codebase adheres strictly to Hexagonal Architecture (Ports and Adapters), separating domain logic from external frameworks, APIs, and rendering engines:

src/
├── domain/                               # Pure Enterprise Logic (Zero external dependencies)
│   ├── entities/
│   │   ├── AffectVector.ts               # 3D Vector hypercube, clamps, distance metrics
│   │   ├── ConversationNode.ts           # Timeline node, turn metadata, branching pointers
│   │   ├── ConversationTimelineGraph.ts  # In-memory DAG state machine with non-destructive rewinding
│   │   ├── SessionReport.ts              # Comprehensive debrief report entity
│   │   ├── SophiaScenarioPresentation.ts # Scenario-adaptive outfits, backgrounds, badge text
│   │   └── TwoWayBridge.ts               # Stated intent vs. received impact data structure
│   ├── scenarios/
│   │   └── topics.ts                     # Pre-configured workplace, school, and friendship scenarios
│   ├── services/
│   │   ├── AffectDynamicsService.ts      # Exponential damping differential equations
│   │   ├── ReportGeneratorService.ts     # Post-session debrief compiler
│   │   └── TensionEvaluatorService.ts    # Mathematical safety gate computation
│   └── value-objects/
│       ├── ProsodyMetrics.ts             # WPM, RMS volume, F0 pitch variance
│       └── ScaffoldingLevel.ts           # 0% (Reality), 50% (Hints), 100% (X-Ray)
├── application/                          # Orchestration & Use Cases
│   ├── ports/
│   │   ├── in/ProcessTurnCommand.ts      # Command interface for turn processing
│   │   └── out/
│   │       ├── LLMInferencePort.ts       # Structured single-shot reasoning contract
│   │       ├── SessionRepositoryPort.ts  # DAG state storage contract
│   │       └── VoiceSynthesisPort.ts     # Speech synthesis contract
│   └── use-cases/
│       ├── ProcessTurnUseCase.ts         # Coordinates input -> LLM -> damping -> safety -> DAG
│       ├── BranchConversationUseCase.ts  # Non-destructive timeline rewind & DAG forking
│       └── GeneratePocketScriptUseCase.ts# Compiles debrief & 2-sentence DBT script
├── infrastructure/                       # External Frameworks, Network & Drivers
│   ├── adapters/
│   │   ├── llm/
│   │   │   ├── GeminiStructuredAdapter.ts# Gemini 1.5 Flash JSON Mode with Zod validation
│   │   │   └── PromptTemplates.ts        # Versioned system prompt definitions
│   │   └── storage/
│   │       └── InMemorySessionRepo.ts    # Ephemeral thread-safe session map (0 cloud persistence)
│   └── web-audio/
│       ├── AudioProcessor.ts             # Web Audio API Context & AnalyserNode bindings
│       └── ProsodyExtractor.ts           # Autocorrelation F0, RMS, zero-crossing DSP (44.1kHz)
└── presentation/                         # User Interface, Hooks & API Controllers
    ├── api/routes/conversation.ts        # Turn Controller (POST /api/v1/conversation/turn)
    ├── hooks/
    │   ├── useAnamAvatarSession.ts       # Stable Anam WebRTC session lifecycle & stream hook
    │   ├── useTimelineBranching.ts       # React state interface for DAG timeline
    │   ├── useWebAudioProsody.ts         # Browser microphone capture & local DSP hook
    │   └── usePartnerVoice.ts            # Voice synthesis playback coordinator
    └── components/
        ├── AppShell.tsx                  # 3-Layer navigation and stage manager
        ├── AppSidebar.tsx                # Accessible navigation sidebar
        ├── SophiaLiveAvatar.tsx          # Real-time WebRTC avatar presentation (<video>)
        ├── PracticeStage.tsx             # Focused conversational practice stage
        ├── ConversationReportPage.tsx    # Comprehensive post-session debrief report
        ├── ReportGenerationView.tsx      # Calming report generation animation view
        ├── SubtextHud.tsx                # Two-Way Bridge, Thought Bubble & Scaffolding
        ├── CoRegulationModal.tsx         # Tier 3 4-count box-breathing overlay
        ├── InputDock.tsx                 # Push-to-Talk audio streamer & text input
        ├── ScenarioCreator.tsx           # Custom scenario builder
        ├── MyProgress.tsx                # Saved pocket scripts and takeaways library
        └── WelcomeOnboarding.tsx         # Neurodiversity-affirming onboarding modal

🚀 Quick Start

1. Prerequisites

Node.js: v18.x or v20.x
npm: v9.x+

2. Installation

git clone https://github.com/aryadevkota/say-it-first.git
cd say-it-first
npm install

3. Environment Configuration

Create .env.local in the project root:

cp .env.example .env.local

Live Video Avatar Integration:

Security: ANAM_API_KEY and ELEVENLABS_API_KEY remain strictly server-side. The client requests a short-lived ephemeral session token via POST /api/v1/avatar/session.

Fail-Fast Diagnostics: If keys are not configured, the API safely returns a descriptive 503 error envelope, and the UI gracefully offers text-mode practice.

4. Run Development Server

npm run dev

Open http://localhost:3000 in your browser.

🧪 Verification Benchmarks & Testing

Full Vitest Suite (221 / 221 Passing across 32 Test Suites)

npm test

 ✓ tests/domain/AffectDynamicsService.test.ts (25 tests)
 ✓ tests/domain/AffectVector.test.ts (37 tests)
 ✓ tests/domain/ConversationNode.test.ts (4 tests)
 ✓ tests/domain/ConversationTimelineGraph.test.ts (3 tests)
 ✓ tests/domain/ProsodyMetrics.test.ts (4 tests)
 ✓ tests/domain/ReportGeneratorService.test.ts (7 tests)
 ✓ tests/domain/ScaffoldingLevel.test.ts (4 tests)
 ✓ tests/domain/ScenarioGenerator.test.ts (4 tests)
 ✓ tests/domain/SophiaScenarioPresentation.test.ts (13 tests)
 ✓ tests/domain/TensionEvaluatorService.test.ts (10 tests)
 ✓ tests/domain/TwoWayBridge.test.ts (4 tests)
 ✓ tests/application/use-cases/BranchConversationUseCase.test.ts (2 tests)
 ✓ tests/application/use-cases/ConversationalContinuityAndSemantics.test.ts (7 tests)
 ✓ tests/application/use-cases/GeneratePocketScriptUseCase.test.ts (2 tests)
 ✓ tests/application/use-cases/ProcessTurnUseCase.test.ts (1 test)
 ✓ tests/infrastructure/AudioProsodyAnalyzer.test.ts (6 tests)
 ✓ tests/infrastructure/GeminiStructuredAdapter.test.ts (8 tests)
 ✓ tests/infrastructure/InMemorySessionRepo.test.ts (2 tests)
 ✓ tests/infrastructure/PromptTemplates.test.ts (2 tests)
 ✓ tests/presentation/AnamSessionRoute.test.ts (8 tests)
 ✓ tests/presentation/ConversationReport.test.ts (4 tests)
 ✓ tests/presentation/LatencyReduction.test.ts (13 tests)
 ✓ tests/presentation/PracticeStagePrompt.test.ts (2 tests)
 ✓ tests/presentation/SophiaLiveAvatar.test.ts (11 tests)
 ✓ tests/presentation/VoiceContractPreservation.test.ts (10 tests)
 ✓ tests/presentation/conversationRoute.test.ts (3 tests)
 ✓ tests/presentation/useAnamAvatarSession.test.ts (15 tests)
 ✓ tests/presentation/useTimelineBranching.test.ts (2 tests)
 ✓ tests/presentation/useWebAudioProsody.test.ts (2 tests)
 ✓ tests/presentation/components/CoRegulationModal.test.ts (2 tests)
 ✓ tests/presentation/components/PostSessionDebrief.test.ts (1 test)
 ✓ tests/presentation/components/SubtextHud.test.ts (3 tests)

 Test Files  32 passed (32)
      Tests  221 passed (221)

Production Build Verification

npm run build

 ✓ Compiled successfully
   Linting and checking validity of types ...
   Generating static pages (8/8) ...
   Finalizing page optimization ...

🔒 Security, Privacy & Accessibility Guarantees

0 kbps Cloud Audio Uploads for DSP: Acoustic prosody (WPM, RMS volume, F0 pitch autocorrelation) is computed purely in the browser via Web Audio API. Raw microphone audio never leaves your device for feature analysis.
Pre-Stream Microphone Echo Protection: Input audio starts pre-muted during session establishment to eliminate speaker-to-mic feedback loops and prevent ElevenLabs server-side VAD barge-in during assistant speech.
Ephemeral Session State: Conversation timeline DAGs are maintained purely in-memory (InMemorySessionRepo). Sensitive accommodations and private boundary disclosures are never persisted to databases or cloud storage.
Server-Kept Credentials: Anam AI and ElevenLabs API keys are held strictly in server-side runtime memory. Clients only receive short-lived, single-session ephemeral tokens.
WCAG 2.1 AAA Accessibility: High-contrast typography, screen-reader friendly ARIA landmarks, keyboard-navigable controls, and reduced-motion compliance.

📜 Architectural Decision Records (ADRs)

ADR	Decision	Rationale
ADR-001	Hexagonal Ports and Adapters	Decouples browser audio DSP, LLM reasoning, and WebRTC streaming, guaranteeing 100% test coverage of domain logic.
ADR-002	In-Memory Ephemeral DAG	Guarantees instant timeline branching, non-destructive attempt resets, and zero cloud persistence of personal disclosures.
ADR-003	Client-Side Web Audio DSP	Zero server bandwidth consumption for acoustic pitch and cadence extraction; eliminates audio privacy risks.
ADR-004	Anam Cara 4 Real-Time WebRTC Video & ElevenLabs Agent	Delivers photorealistic conversational partner video with synchronized lip-sync and voice over a single WebRTC stream without client-side PCM noise.
ADR-005	Gemini 1.5 Flash Structured JSON Inference	Enforces strict Zod schema validation on all LLM responses with p95 latency under 1400ms for seamless turn pacing.
ADR-006	Pre-Stream Mic Mute & Delta Chunk Accumulation	Prevents speaker-to-mic feedback loop and premature barge-in; ensures complete scenario opener delivery without mid-sentence truncation.

👥 Contributors

Arya Devkota (@aryadevkota)
Sarah Howington (@showington1)
Alpesh Patel (@AlpeshPatel1004)

📄 License

This project is licensed under the MIT License — see the LICENSE file for details. Built with empathy for Hacklanta 2026 Social Good Track.

Built With

elevenlabs
node.js
typescript