NeuroSafe

Front Page - Add File
Analyzing Normal Video
Front Page - Add YouTube Link
Analyzing Epileptic Video

Inspiration

1 in 26 people will develop epilepsy in their lifetime. Nearly 2 million of those have photosensitive epilepsy, where flashing lights and rapid visual patterns in video content can trigger seizures within seconds. In 1997, a single Pokemon episode sent 685 children to hospital in Japan. That was almost 30 years ago and the problem is still completely unsolved at scale. YouTube, TikTok, Instagram, and every major streaming platform publish millions of hours of video every day with zero automated neurological screening. The only existing tools are expensive, proprietary, and measure simple pixel-level flicker frequency. They tell you nothing about what is actually happening in the brain. We thought that was unacceptable, and we thought we could do something about it using a model that had just been released weeks before this hackathon.

What it does

NeuroSafe lets anyone upload a video file or paste a YouTube URL and receive a full brain-level seizure risk report in under 60 seconds. The moment a video is submitted, NeuroSafe runs it through a neuroscience foundation model that predicts how the human visual cortex responds to every frame. It identifies exactly which timestamps produce dangerous levels of cortical overstimulation, highlights the specific brain regions affected, and explains what the creator should do about it in plain English. The results are displayed as a synchronized three-panel dashboard: the original video with red danger markers on the scrubber timeline, a live 3D brain surface visualization showing vertex-level activation heatmaps, and a Gemini-generated clinical report with timestamped findings and specific editing recommendations. Content creators, platform trust and safety teams, broadcasters, and medical researchers can all use NeuroSafe to verify that their content is neurologically safe before it reaches an audience.

How we built it

The technical core of NeuroSafe is TRIBE v2, Meta AI's foundation model for in-silico neuroscience, trained on fMRI recordings from over 700 healthy volunteers who were exposed to images, videos, podcasts, and text. TRIBE v2 takes any video as input and returns a tensor of shape (n_timesteps, 20480 vertices), representing predicted fMRI BOLD signal activation across the entire cortical surface at every moment in the video. Each of those 20,480 vertices corresponds to a real anatomical location on the fsaverage5 cortical mesh, the standard neuroimaging surface used in clinical research.

We built a video ingestion pipeline using FFmpeg for frame extraction and yt-dlp for YouTube URL support, feeding directly into TRIBE v2's get_events_dataframe() function to construct the multimodal event timeline. After inference, we use the model's built-in utils_fmri.py neuroimaging utilities to extract the visual cortex ROIs including V1, V2, V3, V4, and MT+, the regions responsible for processing flicker, motion, luminance contrast, and color. We then run those extracted ROI activations through a custom danger scoring algorithm that computes per-ROI activation rate of change at each timestep, compares it against a clinical danger threshold calibrated against known seizure-triggering stimuli, and outputs a structured JSON array of flagged events with timestamps, ROI names, activation z-scores, and severity ratings.

The backend is built on FastAPI with WebSocket streaming, so the frontend begins receiving and rendering inference results before the full analysis is complete. The 3D brain visualization renders the cortical mesh with vertex-level activation heatmaps using the TRIBE v2 plotting module, synchronized to video playback so the brain updates in real time as the video plays. The activation timeline is built in D3.js showing five simultaneous ROI waveforms with a danger threshold line and flagged zone shading. Gemini 1.5 Flash receives the structured danger event summary and generates a natural language clinical report that any creator can understand and act on. The frontend is React with TypeScript, deployed on DigitalOcean.

Challenges we ran into

The biggest challenge was making TRIBE v2 inference fast enough to feel like a real product rather than a research experiment. The model returns a massive high-dimensional tensor and we had to build efficient streaming logic so the frontend could start rendering partial results while inference was still running, rather than waiting for the full computation to complete. Watching the brain light up progressively as inference streamed in became one of the most compelling visual moments in the demo, but it required significant work to coordinate the WebSocket layer, the brain renderer, and the D3 timeline simultaneously without race conditions.

Calibrating the danger threshold was a genuinely hard scientific problem. TRIBE v2 predicts activation levels but does not come with a built-in definition of what is clinically dangerous for photosensitive epilepsy. We had to research the literature on photosensitive seizure triggers, cross-reference WCAG 2.1 flash frequency guidelines, and design a threshold that was sensitive enough to catch real risks without generating false positives on normal video content. Getting that balance right required testing across a range of known safe and known dangerous video samples.

Rendering the cortical mesh in a browser in real time while keeping the UI performant was also a significant engineering challenge. The fsaverage5 mesh has 20,480 vertices and updating their colors every 500ms during playback without dropping frames required careful optimization of the Three.js rendering pipeline.

Accomplishments that we're proud of

We are most proud of the live brain visualization during inference. Watching the visual cortex light up vertex by vertex in real time as TRIBE v2 processes a video is something that genuinely had not existed as a consumer-facing product before this weekend. It makes the invisible visible in a way that is immediately understandable even to someone with no neuroscience background.

We are proud that NeuroSafe is not a wrapper around an existing seizure detection tool. The ROI extraction pipeline, the danger scoring algorithm, and the clinical threshold calibration are all things we built from scratch on top of a research model that was published just weeks before this hackathon. We are one of the first teams in the world to deploy TRIBE v2 in a real application.

We are also proud of the end-to-end product quality. From the landing page to the results dashboard, NeuroSafe looks and feels like a real product, not a hackathon demo. The synchronized three-panel dashboard with the video player, live brain, and Gemini report card is something we think could ship tomorrow.

What we learned

We learned that TRIBE v2 is one of the most underutilized research breakthroughs of the past year. The model can predict high-resolution fMRI responses to any audiovisual input and the applications go far beyond what any research paper has explored so far. NeuroSafe is one possible application but the same pipeline could be used for cognitive load measurement, attention research, therapy content screening, and dozens of other problems.

We also learned that the hardest part of applied neuroscience is not the model. It is the translation layer between raw activation data and human-readable insight. Turning a tensor of 20,480 floating point values per timestep into a sentence a content creator can act on is a product design problem as much as an engineering one. Gemini was genuinely useful here but the framing of the prompt and the structure of the danger event summary we passed to it made an enormous difference in output quality.

Finally, we learned that live streaming of inference results changes the entire feel of a product. The analysis page where the brain begins to activate before inference is finished is more compelling than any results screen we could have designed, because it makes the computation itself the experience.

What's next for NeuroSafe

The immediate next step is validating our danger threshold against clinical data in partnership with hospital neurology departments. Our current threshold is calibrated against published literature and known test stimuli, but clinical validation with real patient data would make NeuroSafe genuinely certifiable as a medical screening tool.

On the product side, we want to build a browser extension that screens video content before it plays, so users with photosensitive epilepsy never have to encounter dangerous content in the first place. We also want to build a platform API that content platforms like YouTube, TikTok, and Twitch can integrate directly into their upload pipelines, making neurological safety screening automatic and invisible for creators.

Longer term, the same TRIBE v2 pipeline that powers NeuroSafe could be extended to screen for other neurological risks beyond photosensitive epilepsy, including cognitive overload in educational content, stress-response triggering in therapeutic media, and attention hijacking patterns in addictive app design. The infrastructure we built this weekend is the foundation for a much broader content safety platform built on real neuroscience rather than pixel rules.