Inspiration
Inspired by the unique capabilities of the Presage SDK, we developed a tool designed to assist individuals on the autism spectrum in interpreting real-time social cues during online meetings."
What it does
Our product leverages the SmartSpectra SDK alongside the EMO-AffectNet model to analyze and display a person’s attention level and emotional state in real time. Making these social cues more visible, it helps neurodivergent individuals better navigate online conversations.
How we built it
We built the system by developing a C++ application on Ubuntu that integrates the SmartSpectra SDK to capture real-time video and sensor data. We then combined neural network models, like EMO-AffectNet, with traditional computer vision algorithms to analyze attention and emotional states.
Challenges we ran into
One of the biggest challenges we faced was setting up the SmartSpectra SDK, since it only runs on Ubuntu, which required adjusting our development environment and workflows. Integrating the neural network model into a traditional computer vision algorithm also proved tricky. On top of that, tuning the emotional and attention gauges to be both responsive and reliable took a lot of experimentation and fine-tuning.
Accomplishments that we're proud of
We're especially proud that we stuck with our original idea and brought it to completion. At one point, implementing the SmartSpectra SDK felt overwhelmingly complicated, and we weren’t even sure it was possible. There were moments when we considered abandoning the project, but we pushed through and kept iterating. In the end, having a fully functioning product made all the effort and uncertainty worth it.
What we learned
We gained hands-on experience programming in C++ on Linux, implementing the SmartSpectra SDK, and integrating neural network models with traditional computer vision algorithms.
What's next for SocialLens
Next, we plan to upgrade the UI to make the visuals more aesthetically pleasing and intuitive. We also want to add real-time notifiers that prompt users to let the other person speak when the attention gauge drops below a certain threshold. Additionally, we aim to make the emotion and attention gauges more robust and accurate for a smoother, more reliable user experience.
Log in or sign up for Devpost to join the conversation.