Sentieye

Inspiration

In this digital age, our attention spans are often fragmented and scattered. We often find ourselves not fully engaged with the content in front of us. Analyzing human emotions and gaze patterns typically requires expensive and complicated head-mounted devices. However, we introduce Sentieye — an innovative solution to this problem!

What it does

Sentieye combines cutting-edge eye tracking models with hume.ai's sentiment analysis API to provide valuable insights and analytics on how users interact with your screen-shared content in virtual meetings*.* Whether you're an educator teaching students or an entrepreneur pitching ideas to potential investors, Sentieye captures and analyzes crucial information.

How we built it

We utilized the ETH-XGaze eye tracker and integrated it with Sentieye's screen-gaze alignment system. By tracking the subtle movements in your gaze through your webcam, we predict the specific pixels you are looking at on your screen. By leveraging hume.ai's sentiment analysis API, Sentieye becomes a powerful tool for observing an investor's mood and understanding what aspects of your product or presentation grab their attention the most.

Challenges we encountered

In today's digital landscape, people's attention is frequently divided, and their focus is often fragmented. It is common for individuals to not keep their gaze fixed on the screen throughout an entire meeting. To address this challenge, we implemented MinMax normalization with thresholding and calibration techniques. This allowed us to determine the boundaries of the user's screen, which is essential for accurately mapping eye movements to the viewed pixels on the screen. This approach provides a cost-effective alternative to the expensive eye-tracking systems used in psychology labs.

What we learned

Throughout the development process, we gained insights into the complexities of eye tracking, sentiment analysis, and integrating different technologies. We also realized the significance of providing affordable and accessible tools to analyze user engagement and emotional responses.

What's next for Sentieye

Our future plans for Sentieye involve incorporating hume.ai's voice analytics tool. This addition will further enhance sentiment analysis, enabling us to provide deeper insights into audience engagement and mood by analyzing vocal cues. By expanding Sentieye's capabilities, we aim to offer a comprehensive and holistic understanding of how individuals interact with content and make informed decisions based on their responses.

We also want to deploy Meta AI's Segment Anything Model to mask the objects that are viewed on-screen, simply the pixels being viewed!

Most excitingly, we'll soon host Sentieye online and get users to experiment with their own content!