Affect

Affect is video analysis tool for continuous mental health support.

Comment

Affect - Project Story

Inspiration

The inspiration for Affect came from a very personal and real problem.
After being diagnosed as being on the bipolar spectrum, I discovered that, on average, it can take up to 15 years for bipolar disorder to be correctly identified. Even in more evident cases, such as Bipolar Type I, diagnosis often takes 8 to 10 years.

One of the main reasons for this delay is the lack of continuous, structured data. Mental health professionals usually rely on short consultations and retrospective reports, which makes it extremely hard to understand how a patient’s mood, behavior, and emotional state fluctuate over time, especially when no one is “observing” (which is usually the case).

This gap raised a question:
What if patients could capture meaningful, structured information about their daily emotional states, and make it accessible to professionals in a clear, reliable way?

That question became the foundation of Affect.

What it does

Affect is a mental health support tool designed to help patients and mental health professionals better understand mood patterns over time.

The patient records short daily videos describing how their day went and how they are feeling, following a simple and standardized structure. From these videos, the system:

  • Performs AI-based video analysis to extract emotional, behavioral, and contextual signals
  • Organizes these insights following a consistent quality standard inspired by clinical practices
  • Stores daily observations in a timeline that reflects real emotional fluctuations
  • Generates summarized reports of recent periods, highlighting patterns, changes, and relevant signals
  • Produces a downloadable report that can be shared with mental health professionals, supporting clinical decision-making

Affect is not meant to replace professionals! It is a decision-support tool that provides richer context and longitudinal data.

How we built it

The project was built using AI-first prototyping approaches, focusing on speed, experimentation, and iteration.

We used:

  • Gemini (Gemini 3 Pro) for:
    • Video analysis
    • Text analysis and summarization
  • Web-based coding tools to rapidly prototype and validate ideas, enabling fast iterations without heavy infrastructure overhead

The focus was not on building a production-ready system from day one, but on validating whether AI could meaningfully extract and organize emotional data from unstructured, real-world inputs like daily videos.

Challenges we ran into

One of the biggest challenges was prompt design and safety.

Mental health is a sensitive domain, and poorly designed prompts can lead AI systems to:

  • Jump to premature conclusions
  • Overinterpret subtle signals
  • Use overly strong clinical language (e.g., labeling something as psychosis based on limited evidence)

Ensuring that the AI remains supportive, cautious, and non-diagnostic required multiple iterations.
Even so, this is an area that still needs significant improvement.

Ideally, the next step involves direct collaboration with mental health professionals, so prompts, boundaries, and outputs align more closely with real clinical expectations and ethical standards.

Accomplishments that we're proud of

What we are most proud of is the potential real-world impact.

If a tool like Affect can help reduce the average time to identify bipolar disorder — even from several years down to one — that represents years of quality of life gained for patients.

We are also proud of:

  • Applying AI to a real, human-centered problem
  • Demonstrating that video-based emotional analysis can produce meaningful and insightful results
  • Building a system that prioritizes support, not replacement, of healthcare professionals

What we learned

Beyond technical implementation, we learned a lot about how mental health professionals think and assess patients.

One key learning was studying and incorporating concepts from the Mental Status Examination (MSE) as a reference framework for what the AI should look for. This helped align the system with real clinical reasoning rather than arbitrary emotional metrics.

We also learned how critical it is to:

  • Respect uncertainty
  • Avoid overconfidence in AI outputs
  • Design systems that acknowledge their own limitations

What's next for Affect

The next steps for Affect focus on safety, collaboration, and scalability:

  • Refining prompts to be more conservative, transparent, and clinically aligned
  • Collaborating directly with psychologists and psychiatrists to:
    • Define useful outputs
    • Establish ethical and informational boundaries
  • Improving usability and accessibility to reduce barriers for patients
  • Exploring how Affect could be safely integrated into real clinical workflows

Ultimately, the goal is to make Affect a trusted, ethical, and practical support tool that helps professionals better understand their patients and helps patients be understood sooner.

Built With

Share this project:

Updates