Track B: The Emo-Agent is an integrated AI framework created by fusing the emotion-Recognization (Perceptual Module) and 007_Agent (Cognitive Module) projects. This system is designed to provide an autonomous, task-driven AI with a form of emotional intelligence by enabling it to perceive and react to a user’s real-time emotional state.
The system is architected around two primary modules communicating via an API. The Perceptual Module utilizes a vision-based Convolutional Neural Network (CNN) to capture video, detect a face, and classify the user's emotion (e.g., happy, frustrated, angry). This analysis is exposed via a simple GET /get_emotion endpoint. The Cognitive Module, the LLM-powered "brain," integrates this as a custom get_user_emotion() tool into its "utility belt," allowing its reasoning and planning loop to factor in the user’s non-textual context.
While the primary documented use case is an Empathetic Coding Tutor—where the agent modifies its instructional complexity and tone based on detected frustration—the technology has powerful applications in covert communication detection and lie analysis.
As illustrated in the "Budget War" case, the Emo-Agent’s capability lies in its power to detect a complete mismatch between the verbal transcript and the non-verbal emotional file. The system can flag "fabricated" emotions, such as hostile language coupled with a consistent 'Amusement' emotion, indicating a performance. More critically, it can pinpoint genuine emotional spikes, like 'Contempt' or 'Satisfaction,' that are not directed at the apparent subject of discussion, thereby revealing the true, covert objective and alliance between participants. This allows the agent to look past the stated purpose of an interaction and uncover the actual intent. Track C:
What it does
Runs red-team style attacks on multiple AI agents and measures prompt success rates.
How we built it
For the Red Teaming: We used Python, CSV datasets, API calls, and automated evaluation with success heuristics.
Challenges we ran into
For red teaming: The LLMs were very resistant to a range of prompts so the success rate was very low.
Accomplishments that we're proud of
At least one success when red teaming!
What we learned
Not so easy to break newer models of LLMs.
What's next for Project Emotions
We have another link:https://github.com/jason0925pig-rgb/emotion-Recognization
Built With
- vscode
Log in or sign up for Devpost to join the conversation.