Heard

Heard: practice speaking with an AI that never judges. Get live feedback as you present, review afterward, and share with peers. Speak freely. Grow confidently. Get heard.

Comment

Heard

Problem Statement

Women have brilliant ideas. Too often, they go unspoken.

Heard is an adaptive communication coach that helps women practice speaking up — with encouragement before they start, real-time coaching while they speak, and a personal coach that remembers them and grows with them over time.

Built for the developHer track at HackHers 2026 — empowering women through education, career growth, and meaningful connections.

Why

  • 75% of female executives report imposter syndrome (KPMG Women's Leadership Study).
  • 29% of women regularly hold back in meetings; 47% cite low confidence and 52% a harsh inner critic (Women Rising).
  • 45% of female leaders find it hard to speak up in virtual meetings (Catalyst).
  • Research has also documented large gender gaps in speaking time during group decision-making.

Most tools stop at "AI grades your speech." Heard does not. It builds a longitudinal speaking profile so every session starts from where the user left off — not from zero.

The Core Idea: A Stateful Communication Coach

Heard combines three layers:

Current speech measurements
        ↓
Historical performance
        +
Persistent coaching memory
        ↓
Backboard → Gemini
        ↓
Personalized feedback

The goal is simple: if a user has already improved one weakness, the coach should stop repeating the same advice and focus on the next highest-impact skill.

Backboard: The Stateful Coaching Brain

Heard uses Backboard as the persistent coaching layer around Gemini.

Each user gets their own long-term assistant:

                    ONE USER
                       │
                       ▼
              Backboard Assistant
              long-term coach
                       │
         ┌─────────────┼─────────────┐
         ▼             ▼             ▼
     Session 1      Session 2      Session 3
      Thread A       Thread B       Thread C
Backboard concept Heard concept
Assistant One persistent coach per user
Thread One coaching/practice conversation
Memory Durable strengths, weaknesses, goals, and progress

During active coaching, Heard uses memory="Readonly" so the coach can retrieve useful past context without saving every temporary detail. At session completion, memory="Auto" allows Backboard to store durable communication patterns for future sessions.

This means Session 3 can build on what happened in Sessions 1 and 2.

TigerData: Tracking Improvement Over Time

Communication improvement is a time-series problem, so Heard uses TigerData / TimescaleDB for analytics.

The current backend stores events such as:

segment_events
- time
- speech_id
- pace_wpm
- avg_volume

speech_events
- time
- user_id
- speech_id
- overall
- clarity
- volume
- pace
- confidence
- structure

TigerData gives us the foundation to analyze questions like:

  • Is pace improving across sessions?
  • Is filler usage consistently decreasing?
  • Is an improvement sustained or just one unusually good session?
  • Which dimensions keep becoming bottlenecks?

The architecture separates responsibilities:

TigerData
= numerical performance over time

Backboard
= qualitative coaching memory

Gemini
= reasoning across both

The current repository already records TigerData analytics, while the newer longitudinal coaching pipeline reads exact historical context from Heard's application database. A key next step is using TigerData directly for rolling trend queries in that coaching context.

What Heard Does

1. Pre-Training

Users select a communication channel such as Technology, Finance, Education, Law, Healthcare, or Marketing and start a preparation conversation.

Heard loads:

  • previous sessions,
  • baseline performance,
  • recent averages,
  • recurring strengths,
  • recurring weaknesses,
  • current coaching goals.

Instead of generic advice like:

"Speak slowly and clearly."

Heard can say:

"Your overall pace has improved, but technical explanations are still where you rush. Rehearse that section in smaller components and pause between each one."

2. Live Training

The Chrome extension handles live practice.

Microphone
    ↓
ElevenLabs realtime transcription
    ↓
~60 second segments
    ↓
async coaching queue
    ↓
Backboard + Gemini
    ↓
short live coaching nudge

One worker processes each session in order, so minute 1 always reaches the coach before minute 2.

Live feedback is structured around:

{
  "focus_area": "clarity",
  "nudge": "State the main claim before the background.",
  "evidence": "The segment opened with context before the central point.",
  "next_minute_goal": "Lead the next section with one clear claim."
}

3. Speech Analysis

The backend computes measurable evidence directly rather than asking the LLM to invent metrics.

It tracks:

  • words per minute,
  • filler rate,
  • hedge rate,
  • sentence length,
  • repetition,
  • pauses,
  • silence ratio,
  • relative volume consistency,
  • pitch variation,
  • pace variability.

Heard currently evaluates six main dimensions:

Dimension What it measures
Clarity coherence and explanation quality
Conciseness repetition, fillers, hedging
Pace WPM and pacing stability
Volume relative consistency when audio exists
Confidence observable filler/hedging behavior
Structure intro, transitions, progression, conclusion

Missing data stays unavailable rather than being fabricated.

4. Post-Training

At the end of a session, Heard builds a structured context packet containing:

current transcript
current metrics
previous session
baseline
recent averages
dimension trends
recurring patterns
current goals

That context is sent through Backboard → Gemini.

The model is asked to identify:

  • what improved,
  • what regressed,
  • what keeps recurring,
  • strongest moments,
  • priority moments,
  • measurable next-session goals.

The point is not just:

"Your pace was 151 WPM."

It is:

"Your overall pace improved compared with recent Technology sessions, but your architecture section was still your fastest segment. Since filler use also improved, pacing technical sections is now the higher-value focus."

Channel-Specific Progress

Heard keeps improvement channel-specific because speaking behavior changes by context.

A user may perform differently in:

Technology
Finance
Education
Law
Healthcare
Marketing

Each user/channel profile tracks:

completed sessions
baseline score
recent score
improvement percentage
dimension trends
recurring strengths
recurring weaknesses
current goals

Improvement-Based Leaderboard

Heard does not rank users only by raw score.

The leaderboard rewards improvement relative to each user's own baseline.

User A: 55 → 74
User B: 90 → 91

User A can rank higher because they demonstrated more growth.

The leaderboard is channel-specific, requires multiple completed sessions, and excludes private profiles.

Recommended Peers and DMs

Heard also connects AI coaching with human support.

After a session, users can discover improving speakers in the same channel and start a direct message.

AI feedback
    ↓
recommended peers
    ↓
1:1 conversation

The DM system supports:

  • one-to-one conversations,
  • message history,
  • unread counts,
  • read state,
  • member-only access.

This keeps the two experiences separate:

AI COACH
User ↔ Heard

DM
User ↔ User

Architecture

User
 │
 ├── Web App
 │
 └── Chrome Extension
          │
          ▼
     FastAPI Backend
          │
   ┌──────┼───────────────┐
   ▼      ▼               ▼
Supabase  Speech Analysis  ElevenLabs
   │      │
   │      ▼
   │   Objective Metrics
   │      │
   │   TigerData
   │      │
   └──────┴──────→ Context Builder
                      │
                      ▼
                  Backboard
                      │
                      ▼
                    Gemini
                      │
                      ▼
             Personalized Feedback

Tech Stack

Layer Technology
Web app React, Vite, TypeScript
Live surface Chrome Extension
Backend Python, FastAPI
Authentication Supabase Auth
Main data store SQLModel / Postgres path
Speech-to-text ElevenLabs
Stateful memory Backboard
AI reasoning Google Gemini
Time-series analytics TigerData / TimescaleDB
Realtime transport WebSocket
Audio metrics Web Audio API

Challenges We Ran Into

  • Building a coach that remembers progress instead of resetting every session.
  • Separating qualitative memory from exact numeric history.
  • Avoiding generic AI feedback.
  • Processing live speech without blocking transcription.
  • Handling missing audio metrics honestly.
  • Designing a leaderboard around improvement instead of raw talent.

Accomplishments We're Proud Of

  • Persistent Backboard coach per user.
  • Cross-session coaching memory.
  • Realtime ~60-second coaching segments.
  • Deterministic speech and language analysis.
  • Six-dimension post-training feedback.
  • Channel-specific longitudinal profiles.
  • Improvement-based leaderboards.
  • Recommended peers and real one-to-one DMs.
  • TigerData analytics for speech and segment events.
  • A combined web app + Chrome extension experience.

What We Learned

The biggest lesson was that memory and analytics are not the same thing.

Backboard can remember:

"This user tends to rush technical explanations."

TigerData can track:

189 WPM → 178 → 174 → 159 → 151

Gemini can then reason:

"General pacing has improved, so the next goal should focus specifically on technically dense sections."

That is much more useful than repeatedly giving the same generic advice.

What's Next

  • Use TigerData directly for rolling multi-session trend queries.
  • Feed those trends into Backboard's performance-history tool.
  • Match peers based on complementary strengths and weaknesses.
  • Add context-specific modes for interviews, presentations, pitches, and research talks.
  • Build richer long-term progress visualizations.

The Vision

A good coach remembers.

A good coach notices progress.

A good coach stops correcting weaknesses you have already solved.

A good coach knows what you should work on next.

That is what Heard is trying to build.

Backboard gives Heard a memory of the person.

TigerData gives Heard a history of performance.

Gemini turns both into personalized coaching.

Heard does not start every session from zero. It starts from where you left off.

Built With

  • auth0
  • backboard
  • data
  • elevenlabs
  • gemini
  • tiger
  • tigerdata
Share this project:

Updates