Cultura: AI-Powered Emotion-Based Media Recommender

• 

  it captures emotion detected from the web cam

Inspiration

Humans naturally express emotions, but machines often fall short in recognizing and responding to them meaningfully. Cultura was inspired by the idea of closing that emotional gap. I wanted to create an AI system that could go beyond processing inputs and outputs—one that could truly feel and respond in a personalized, emotionally intelligent way. Cultura became my attempt to build a private, offline AI companion that senses the user’s mood through facial expressions and delivers personalized media—whether that’s music, movies, or motivational content.

What it does

Cultura is an AI-powered emotion detection and media recommendation system. It captures a live snapshot of the user’s face through their webcam, detects facial expressions in real-time, and predicts the underlying emotion (such as happy, sad, angry, etc.). Based on this emotion, it recommends a curated set of media, including music tracks, film suggestions, or even a motivational message generated using LLaMA 3—all running completely offline for privacy and speed.

How we built it

The system starts with a CNN model trained on the FER2013 dataset for emotion recognition, integrated with OpenCV for real-time webcam access. The detected face is passed through the model, which classifies it into one of several emotional states. Based on the result, the backend logic branches into three outputs: Music recommendation Film suggestions A motivational message generated locally via LLaMA 3 using Ollama

Challenges we ran into

One of the major challenges was achieving real-time inference without noticeable lag. Optimizing the CNN model for both size and speed involved quantization and architectural tweaks. Integrating Ollama to run LLaMA 3 locally also presented difficulties, particularly in managing prompt structure and avoiding blocked or partial responses. Additionally, making the UI feel emotionally responsive and clean took several iterations in design and testing.

Accomplishments that we're proud of

We’re proud that Cultura works entirely offline, providing both privacy and speed. It combines computer vision with LLMs in a context-aware setup that actually responds meaningfully to user emotions. Successfully deploying a local LLaMA 3 model and integrating it into a real-time emotion detection system is a major achievement. Creating a seamless pipeline from facial expression to emotional understanding to actionable content is something we’re truly proud of.

What we learned

This project taught us the true potential of combining computer vision with language models to create emotionally intelligent systems. We learned how to optimize deep learning models for edge devices, run LLMs locally without relying on cloud services, and build real-time full-stack applications. We also became aware of the ethical implications of emotion AI—especially the need to respect user privacy and use these tools responsibly.

What's next for Cultura: AI-Powered Emotion-Based Media Recommender

We plan to expand Cultura by adding voice-based emotion detection, sentiment-aware journaling features, and deeper personalization based on user history. Integrating transformer-based vision models like ViT or ConvNeXt for improved emotion accuracy is also on the roadmap. In the long term, we envision Cultura as a plug-and-play emotional layer for other applications—mental health, smart homes, and emotionally aware virtual assistants—all while staying fully local and privacy-respecting.

Built With

• api
• cnn
• llama3
• ollama
• opencv
• python