Resona

Screening that listens. Cough into your browser and Resona flags possible TB from the sound — no app, no hardware, no lab — then routes you to real care.

Comment

Inspiration

Tuberculosis (TB) is the world's leading cause of death from a disease. All throughout the world, it ranks among the top 10 causes of death. According to WHO, in 2025, 10.7 million fell ill from TB, yet only 8.3 million of those were actually diagnosed and notified. That is a 2.4 million-person gap that led to the deaths of 1.23 million people. This is not just a treatment failure, it's a detection failure. TB is entirely curable, yet millions worldwide suffer from it without even knowing that they have it.

Testing for TB often means reaching a clinic with sputum microscopy or a GeneXpert machine. For a lot of the world, especially in developing countries, that could lead to over a day's worth of travel and lost wages, just for a cough that you're not yet sure is serious. So, people end up not doing anything about it.

Tuberculosis measurably changes the acoustics of a cough. So, we wanted to know whether TB could be checked for before making such a trip, using a phone that most of the world already owns.

What it does

Resona screens for TB from the sound of a cough, in the browser, in a matter of seconds.

You cough two or three times. The browser detects each individual cough, cuts them into separate clips, and turns them into a frequency representation. Alongside that you fill in a short clinical intake — age, cough duration, fever, night sweats, weight loss, HIV status. A multimodal CNN scores the audio and the clinical picture together and returns a risk band, the spectrogram it actually looked at, and a concrete next step. A high signal routes you into a referral flow rather than leaving you with a frightening number and no instruction.

Everything it can't do is stated on screen. It's a screening signal, not a complete diagnosis.

How we built it

Frontend: Next.js 16 (App Router), React 19, TypeScript, Tailwind 4. Six production dependencies total, no UI framework, no component library, no charting library. One 1,500-line stylesheet where light and dark come from CSS light-dark(), so every token is declared once and the theme toggle just flips color-scheme.

Audio DSP: hand-written TypeScript, with no browser audio library. Energy-based onset detection, a short-time Fourier transform for the spectrogram, resampling to 16 kHz mono, and 16-bit WAV encoding, all in src/lib/audio-features.ts.

Model service: FastAPI + PyTorch, packaged in the repo with a Docker image. The Next.js API route proxies to it and falls back to clearly labelled demo output when it isn't connected, so the app is never broken.

Challenges we ran into

Data quality. The CODA-TB dataset mixes clean and noisy recordings, and some participants had corrupt WAV files or missing audio entirely. We had to write validation that drops unreadable clips, filter silence and low-energy segments, and normalise wildly different recording conditions before anything reached the model — otherwise the network just learned which phone recorded the cough, not whether the person had TB.

Cough segmentation. The model was trained on individually segmented coughs and only looks at the first 0.55 seconds of each clip. Hand it one long recording and you show it half a second of silence. We built energy-based onset detection in the browser that finds each cough, cuts it out with a 250 ms refractory gap so one cough yields exactly one clip, and uploads them as separate parts.

Honesty under pressure. The hardest problem wasn't technical. A screening tool that outputs a number invites people to read it as a diagnosis. We kept redesigning the result flow so a high signal routes to care instead of standing alone, labelled every demo value as demo, and refused to print an accuracy figure we haven't earned through clinical validation.

Accomplishments that we're proud of

  • A full TB screen that runs from a plain phone microphone in the browser — no app install, no clinical hardware, seconds to a result.
  • A multimodal CNN that fuses the cough acoustics with a 27-feature clinical picture, so a blank intake still works and a filled one is worth more.
  • Hand-written audio DSP — onset detection, STFT, resampling, and 16-bit WAV encoding — in TypeScript with zero browser audio libraries.
  • A client-side spectrogram computed from the exact audio we send, so the visualisation is always faithful to the model's input.
  • An app that is never broken: with no backend connected, the API route returns clearly labelled demo output instead of an error.
  • A deliberately tiny stack — six production dependencies, one stylesheet, no UI or charting library.

What we learned

  • TB is a detection problem before it's a treatment problem. A curable disease kills 1.23 million people a year largely because they're never diagnosed. Lowering the cost of that first check is a real lever.
  • Preprocessing is most of the work. More of our effort went into rejecting bad audio and segmenting coughs than into the model itself — garbage in genuinely means garbage out here.
  • Trust is a design constraint, not a footnote. How you present a risk score changes whether it helps or harms. Routing to care and stating limits on screen mattered as much as the accuracy of the number behind it.
  • You can do serious signal processing in the browser without a heavy toolchain if you understand the DSP yourself.

What's next for Resona

  • Clinical validation. Publish sensitivity and specificity against a held-out, geographically diverse set before any score is read as performance — this is the gate for everything else.
  • Real referral integration. Swap our sandbox directory for live SatuSehat (FHIR R4) endpoints so a high signal books an actual appointment.
  • On-device inference. Move the model to the phone with a quantised checkpoint, so screening works fully offline in the low-connectivity settings that need it most.
  • Field robustness. Train and test against more phones, languages, and background noise so accuracy doesn't depend on which device recorded the cough.
  • Longitudinal tracking. Let a user re-screen over time and flag a worsening acoustic signal — turning a one-shot check into a trend.

Built With

Share this project:

Updates