Reflekt

Abstract Summary

Reflekt explores emotion as a material rather than a message. Through a private, real-time system that translates facial expression and voice into evolving generative visuals, the project captures fleeting internal states as temporary “residue.” With user consent, these residues are preserved within a shared digital archive—stripped of identity and retaining only emotional trace. Positioned between human vulnerability and machine mediation, Reflekt treats AI not as an artist, but as a medium through which feeling acquires form, persistence, and collective presence.

Inspiration

Reflekt began with a central question: what emerges when a system does not label emotion, but listens to subtle human signals and responds through art? Rather than categorizing feelings, the project explores emotion as movement and transition. Something dynamic, imprecise, and continuously shifting.

What it does

Reflekt listens to facial expression and voice in real time and transforms these inputs into a living visual system. The system does not detect, classify, or diagnose emotion. Instead, it blends multiple weak signals into a temporary emotional state that drives generative visuals as they evolve over time. At the end of each session, a single visual residue remains—an abstract trace of presence rather than an emotional verdict.

How we built it

-The backend is written in Python and processes live camera and voice input. -Facial signals and vocal features are combined into continuous valence and arousal signals, rather than discrete emotion labels. -A WebSocket bridge transmits only derived emotional state to the browser, never raw biometric data. -The frontend renders a reactive particle-based visual system and preserves a session-level visual artifact.

Challenges we ran into

-Balancing multiple weak signals without over-relying on any single input required careful calibration. -Maintaining real-time responsiveness while avoiding storage of raw biometric data posed architectural constraints. -Tuning visual behavior to feel expressive without becoming chaotic took significant iteration.

Accomplishments that we're proud of

-We built a system that reacts emotionally without claiming emotional truth.
-The visuals change smoothly over time.
-The system leaves behind art, not data

What we learned

-Simple signals can create rich experiences when combined carefully.
-Constraints can lead to more thoughtful design.

What's next for Reflekt

-Improved sensitivity to vocal rhythm and dynamics. -Deployment in live installations and interactive exhibition environments. -Adaptation of the visual archive for public-facing curation contexts.

Built With

• css
• ferret-go-sentiment-analysis
• html
• javascript
• opencv
• python
• sounddevice
• vadersentiment
• vosk
• websockets