Neuroblast

Inspiration

Social communication can be hard when tone, sarcasm, or nonverbal cues are ambiguous. We built Neuroblast to give people a fast, practical way to understand what was likely meant in a conversation and practice real social interpretation skills in a low-pressure environment.

What it does

Neuroblast records or uploads audio/video, transcribes speech, and generates a
structured interpretation based on:

emotion
tone
likely meaning
misunderstanding risk
suggested responses
a social tip
to help the users determine how to respond.

It also includes a Social Scenarios mode, where users practice reading intent
in everyday situations with instant feedback.

How we built it

Frontend: lightweight web UI for recording, uploading, and reviewing analysis.
Transcription: whisper.cpp running in-repo for local speech-to-text.
Inference backend: Featherless API model routing with:
- Qwen as default (Faster, No Vision)
- Kaushika as vision fallback (Slower, With Vision)
API/server: Express endpoints for analysis, model target selection, and
social scenario generation.
Safety/ops: simple 4-digit access code gate to prevent abuse during demo
window.

Challenges we ran into

Originally we were using Teachable Machine and a Kaggle emotion dataset to train an image AI model. But we discovered that the Kaggle dataset was limited and was black and white. It wasn't able to differentiate between sad and angry, as in those emotions, the eyebrows and eyes are way too similar. We found another Kaggle dataset but figured that the dataset was limited and didn't work. Then we tried taking pictures of people in our group and other peers of their emotion as data, but that didn't work either.

We ended up using open-source called Qwen, which the best light weight model for our use. With using that some issues we ran into were:

Binary/runtime mismatch: a macOS whisper binary had to be rebuilt for Linux.
Multimodal tradeoffs: balancing speed vs. vision support across models.
Response reliability: enforcing structured JSON output from LLM responses.
Demo hardening under time pressure: protecting API usage from random crawler
traffic.