Echo | Devpost

landing page

Project Name

Echo

Tagline

Classroom acoustic analysis from a single photo — hear what your child hears.

Inspiration

1.5 million children in the U.S. have educationally significant hearing loss and sit in classrooms designed without them in mind. Professional acoustic assessments cost $5,000+ and require specialized equipment. We wanted to give every parent the power of an acoustic consulting firm from just their phone.

What it does

Echo lets a parent photograph their child's classroom and instantly receive a full acoustic analysis. It identifies surface materials (carpet, glass, tile, whiteboard) from the photo using computer vision, maps the room geometry from 4 tapped corners, then runs 38,400 distributed acoustic ray-tracing simulations to produce a speech intelligibility heatmap — showing exactly where speech is clear and where it's lost. It identifies the best and worst seats, generates a spoken voice report, and lets the parent hear a simulation of what their child actually hears with hearing loss versus normal hearing.

How we built it

Frontend: Vite + React + TypeScript + Tailwind CSS
Auth: Auth0 for user login and saved scan history
Image Analysis: Google Cloud Vision API (LABEL_DETECTION + OBJECT_LOCALIZATION) maps detected surfaces to acoustic absorption coefficients
Distributed Compute: DCP (Distributive Compute Platform) distributes 38,400 independent ray-tracing simulations (16×12 grid × 200 rays each) across its worker network
Voice & Audio: ElevenLabs generates a spoken findings report and a clean speech sample. Web Audio API applies simulated hearing loss (high-frequency roll-off + reverb) so parents can hear the difference
3D Visualization: Three.js for acoustic particle visualization and interactive 3D heatmap
Storage: localStorage for scan history with thumbnail compression

Challenges we ran into

DCP job submission works but times out after 30 seconds due to limited available workers during development — it falls back to local compute. The job IS submitted and visible in the DCP console. Balancing visual impressiveness with acoustic accuracy in the heatmap was challenging. Compressing scan thumbnails to fit localStorage limits required careful optimization.

Accomplishments that we're proud of

Every API integration is structurally essential, not bolted on. Vision drives the simulation inputs, DCP handles the compute bottleneck, ElevenLabs creates the emotional climax of the demo (hearing the difference), and Auth0 makes it a persistent tool rather than a one-off toy. The hearing loss audio comparison is physically meaningful — it uses real frequency attenuation modeling, not just muffling.

What we learned

Acoustic ray-tracing is surprisingly parallelizable — each listener position is fully independent, making it a perfect fit for distributed computing. Speech Transmission Index (STI) is the standard metric for classroom acoustics. Material absorption coefficients vary dramatically by frequency band, and common classroom materials like glass and tile are acoustically terrible.

What's next for Echo

Real-time DCP distribution with more workers, ML-based room geometry estimation (skip the 4-corner tap), audiogram upload for personalized hearing loss simulation (not just generic high-frequency loss), and a teacher-facing mode that suggests acoustic interventions (rugs, panels, seating changes).