AURA — Acoustic Anomaly Detector

AURA started with a simple question: can a phone sense something about an object that we cannot see?

Phones already have a speaker and a microphone, so we wanted to see how far we could push that hardware without adding any special sensors. AURA uses the phone to send a controlled sound sweep into an object and listens to how the object responds.

The basic idea is:

Send sound → Listen → Learn what "normal" sounds like → Compare → Detect change

How it works

First, AURA calibrates the object. It performs several sound sweeps and uses the microphone recordings to create an acoustic fingerprint.

When we scan the object again, AURA compares the new measurement with that original fingerprint.

The comparison uses Mean Squared Error (MSE):

$$ MSE = \frac{1}{K}\sum_{k=1}^{K}(S_k-B_k)^2 $$

Here, $B$ represents the original baseline measurement and $S$ represents the new measurement.

If the new response is close to the baseline, AURA reports a match. If the response changes significantly, AURA reports an acoustic deviation.

How we built it

We built AURA using Next.js and TypeScript. The Web Audio API handles the microphone and speaker, while an AudioWorklet processes the recorded audio. We also wrote our own signal-processing code to extract frequency information and calculate the difference between measurements.

The results are displayed using HTML Canvas, including the baseline spectrum, current scan, and difference between them.

What inspired us

We were interested in the amount of technology that is already sitting inside everyday devices. Instead of adding another sensor, we wanted to explore whether a normal phone could be turned into a simple sensing tool.

The goal isn't to tell the user exactly what material is inside an object. Instead, AURA asks a simpler question:

Does this object still sound like itself?

Challenges

The hardest part was dealing with real-world audio. Different phones have different microphones and speakers, background noise changes constantly, and even moving the phone slightly can change the recording.

Because of this, we chose a baseline-based approach instead of using fixed thresholds for every object. AURA learns the normal response of the specific object being tested and looks for changes relative to that baseline.

We also learned a lot about real-time audio processing, FFTs, browser audio APIs, and the difference between building something in software and making software interact with the physical world.

AURA is a prototype for acoustic anomaly detection, not a certified structural inspection device.

Our bigger idea is simple:

Give ordinary objects a sense of touch.