Sound Sense

• 

  Latency

• 

  Numbers

• 

  Teach - Flow

• 

  Felt vs. Heard

• 

  Decision

• 

  Placement

• 

  Self Test

• 

  Cover

• 

  Architecture

• 

  Teach

• 

  Alerts Timeline

• 

  More

• 

  At the Desk

• 

  On the Board

• 

  Board, Wide

Code: https://github.com/Parth-Bansal30/e8-hackathon Judged version: tag v1.2-judged (commit 0d8b0f1)

WHAT IT IS

Sound Sense is a small, always-on box for deaf and hard-of-hearing people. It sits by the door, recognises important sounds in the home, and shows a clear visual alert. It runs entirely on the Alif Ensemble E8 DevKit. Audio is never stored and never leaves the board.

WHAT IT DOES

• Recognises knocks, dog barking, baby crying, alarms, glass breaking, sirens and running water. • Learns your own sounds on the device from 8 examples, with no laptop and no retraining. They're saved to flash and restored at boot. • Uses the accelerometer to tell a knock on its own table from a knock on the TV. • Shows full-screen alerts in colour-blind-safe colours, with LED patterns and three urgency levels. Urgent alerts strobe at a photosensitivity-safe rate. • Also has a timeline, night mode, settings on the board and a boot self-test.

HOW WE BUILT IT

We trained our own PyTorch encoder, quantized it to int8 and exported it through ExecuTorch. All 14 operators run on the Ethos-U85 NPU. Audio features, matching and the UI run on the Cortex-M55. Teaching a sound just builds a new profile from the NPU's embeddings, so it takes milliseconds.

MEASURED ON THE BOARD

• 1.00 ms per run on the NPU • 437 ms median from the start of a sound to the alert on screen • 12.6% CPU active in a quiet room, down from 100% • 0 audio blocks dropped across 1,101 flash saves • 1.5 s from reset to ready

WHAT WE LEARNED

• Measure before optimising: two-thirds of the CPU time was going to a joystick read, not the DSP or the NPU. • Built-in sounds played from a laptop speaker were recognised 54% of the time live vs 88% offline. That's why teaching your own sounds became the centre of the product.

WHAT'S NEXT

An always-on listener on the low-power core, sound direction from the two mics, and an output for a bed shaker for night alerts.

TEAM

Parth Bansal: hardware, AI model, core app and testing Sanket Shah: interface, flash storage, power and submission

BUILT WITH

Alif Ensemble E8, ExecuTorch, PyTorch, Vela, CMSIS-Toolbox, SEGGER J-Link, Python

Built With

• alif-ensemble-e8
• arm-cortex-m55
• arm-ethos-u85
• bmi323
• c
• c++
• cmsis
• esc-50
• executorch
• numpy
• python
• pytorch
• segger-rtt
• speech-commands
• vela