Inspiration
We wanted to develop a product to help parents (or anyone really) better understand how their children responds to the whatever they see around them in real time. This could help parents prevent some triggers or dangerous situations in the future based on how the developing children's brain reacts to different situations.
What it does
Archangel uses a Jetson Orin Nano with an attached Arducam camera to capture live video clips (or you can even upload your own video clips). A Python script sends those frames to a GPU workstation, where Meta’s V-JEPA2, Wav2Vec-BERT, and TRIBE v2 process the input and generate predicted brain activity, emotion-related signals, and spike alerts in real time.
How we built it
We built the system as a full pipeline starting from the Jetson, a Python script captures camera footage, encodes it, and streams it over ethernet. On the workstation, another Python backend receives the frames, runs the brain-prediction models, and sends the results through WebSockets to a live Next.js dashboard that shows real-time cortical activity, brain-region breakdowns, alerts and more details.
Challenges we ran into
One of our biggest challenges was that the original TRIBE v2 pipeline was built for slow batch research workflows, not live streaming. We had to rebuild parts of the system for real-time inference, replace inefficient frame extraction with OpenCV decoding, keep models loaded in VRAM, and get the WebSocket-based backend working smoothly with the frontend.
Accomplishments that we're proud of
We were happy to make a neuroscience research model into a real-time system. We reduced processing from minutes to under 600ms latency, built a working camera-to-dashboard pipeline, and created live visualizations of predicted brain activity and emotion-related spikes.
What we learned
We learned how hard real-time AI systems are to build across both hardware and software. We also learned a lot about edge-device streaming, GPU optimization, Python models, and how to adapt large research models into an interactive product experience.
What's next for Archangel
We really want to make Archangel into a real product with a better wearable camera setup, a more polished dashboard, and a smoother real-time pipeline. We also want to improve the camera arrangement, increase reliability, and make the system easier to use in real-world child-focused settings. We mainly want to have this for parents to access and monitor their kid's developing brains and understand how to work with kids efficiently or to alert parents of any harsh situations that the kids went through which resulted in a spike in the brain activity of the kid.
Log in or sign up for Devpost to join the conversation.