Continual Entertainment Experience Generator (CEEG)

Inspiration
The CEEG (Continuity-Enhanced Emotional Guidance) project was inspired by the need for a dynamic storytelling system that adapts based on real-time emotional feedback. Traditional storytelling follows fixed narratives, limiting user interaction. Our goal was to create an AI-driven system that modifies the storyline based on user emotions, allowing for a truly interactive and personalized experience. Drawing from psychological engagement models, we designed CEEG to enhance immersion by aligning narrative progression with real-time emotional states.

What it does
CEEG integrates facial emotion recognition, satisfaction modeling, structured story prediction, and text generation to create a responsive storytelling system. It captures real-time user emotions, processes them into a satisfaction score (k_satisfaction), and then predicts the next logical story event based on historical patterns. Using a nearest-neighbor search algorithm, the system retrieves the most contextually relevant text segment, dynamically shaping the narrative to maintain engagement and user interest.

How we built it
We developed CEEG using a modular architecture, ensuring seamless interaction between its components. The facial emotion recognition model was trained on a deep learning framework to classify six core emotions. The satisfaction model converts emotion scores into a structured feedback signal. A CRF-based structured prediction model forecasts the next state of the story (S_matrix). Finally, a nearest-neighbor search selects the most relevant text snippet, allowing for coherent and contextually appropriate story progression. The system was optimized for real-time processing and GPU acceleration to improve efficiency.

Challenges we ran into
One major challenge was mapping emotions to satisfaction in a way that maintains narrative coherence. Simple linear models failed to capture the complex relationship between emotional fluctuations and engagement, so we implemented a weighted transformation function to balance satisfaction changes. Another issue was high-dimensional input in structured story modeling, which led to computational inefficiencies. We optimized feature selection and incremental training to mitigate these concerns. Lastly, ensuring semantic consistency in generated text required fine-tuning the retrieval algorithm to prioritize logical flow and contextual relevance.

Built With

• sklearn-crfsuite
• torch