The Listening Tree

• GIF

Inspiration

Most modern AI tools rely on typing, reading tiny fonts, or understanding English. When I looked at how Cantonese-speaking older adults interact with technology, I saw a huge accessibility gap. Many struggle with traditional interfaces and literacy, leaving them left behind in the AI era.

I built The Listening Tree as a solo project to flip this dynamic—creating a voice-first, highly accessible AI companion that speaks their language naturally and reduces everyday loneliness.

What it does

The Listening Tree is a voice-first web application tailored for seniors. Key features include:

• Voice-First Interaction: Powered by natural speech input and output, minimizing the need to touch or type on screens.
• High-Contrast & Accessible UI: Clean, ultra-large UI elements designed specifically for elderly visual needs.
• Bilingual AI Companion: Delivers empathetic, natural conversations in Cantonese and English using advanced LLM reasoning.

How I built it

As a solo developer, I built the system end-to-end focusing on responsive design and low-latency response:

• Frontend: Next.js / React with Tailwind CSS, optimized for high contrast, responsive viewports, and clean UI components.
• Speech Processing: Web Speech API & Whisper API for accurate Cantonese voice recognition and speech synthesis.
• AI Engine: GLM-4 model integration for contextual understanding, dialogue management, and fast response times.
• Testing & CI/CD: End-to-end browser compatibility testing using Playwright across Chromium and WebKit environments, automated via GitHub Actions.

Challenges I ran into

• Cantonese Speech Recognition: Handling Cantonese dialects, accents, and mixed speech patterns required careful prompt tuning and speech API fallback strategies.
• Voice Latency: Minimizing the turnaround time between user speech and AI response to make conversations feel natural for older users.
• Solo Execution: Balancing frontend UX design, AI integration, and robust automated E2E testing as a single builder within deadline constraints.

Accomplishments that I'm proud of

• Successfully designed and shipped a working voice-first interface specifically tailored for senior accessibility.
• Achieved high E2E browser compatibility test passes across desktop and mobile browsers using Playwright.
• Created an intuitive, warm product flow that real older adults can use without a steep learning curve.

What I learned

• Deep insights into accessible UX standards (WCAG guidelines, contrast ratios, touch targets) for senior demographics.
• Hands-on experience tuning multimodal/voice AI workflows and managing real-time speech processing pipeline latency.

What's next for The Listening Tree

• Real-user Field Testing: Conducting user testing sessions with local community seniors to gather qualitative feedback.
• Cognitive Memory & Reminders: Adding lightweight memory features and gentle daily reminder notifications.
• Offline Voice Support: Exploring local edge-speech models to improve privacy and offline capability.

Built With

• github-actions
• glm-4
• next.js
• playwright
• react
• tailwind-css
• typescript
• web-speech-api