DenEyes

DenEyes
Camera

Inspiration

The rising prevalence of ADHD in children, currently affecting about 7.1 million children (1 out of 9) in the United States alone, highlights the urgent need for early detection and intervention. Untreated ADHD can lead to academic, personal, and social challenges, with 23% of children diagnosed with ADHD not receiving any form of treatment. This lack of intervention significantly increases the risk of school failure, social difficulties, and even substance abuse as they grow older. This is what inspired us to create DenEyes, an automatic, mobile robot designed to assist doctors in detecting early signs of ADHD. Early intervention is crucial in helping children develop coping strategies and ensuring they don’t fall behind academically. DenEyes aims to provide parents, educators, and healthcare professionals with real-time insights into a child’s attention patterns, enabling timely and data-driven actions that can positively impact the child's development.

What We Learned

Building DenEyes taught us a lot about the complexities of both computer vision , electronics, reinforcement learning. We learned how to integrate these technologies to detect specific signs of attention in children, such as whether they were looking at the camera or if they move too much hyperactively. We also gained valuable experience working with Raspberry Pi, optimizing it for real time image processing. .

What it does

DenEyes monitors a child’s attention and behavior to detect early signs of ADHD. The robot uses computer vision to detect the child’s face, eyes, and motion. By analyzing eye gaze, distance, and movements, it determines whether the child is paying attention or becoming distracted.

How we built it

We started by selecting a Raspberry Pi as the brain of DenEyes for its portability and ability to handle real-time computing. We equipped it with a camera module to capture live video feed and integrated OpenCV for facial and motion detection, as well as MediaPipe to track eye gaze. Once the robot detects a loss of attention, it takes action. Using a reinforcement learning (RL) algorithm, DenEyes learns to adapt its responses based on the child's behavior. For example, if the child becomes distracted or moves excessively, DenEyes will perform an action designed to regain the child's focus. It may:

Call the child’s name to bring their attention back.
Mimic sounds, such as a cat’s meow, to engage the child and redirect their attention.
Make simple movements to capture visual interest.

Over time, the RL algorithm fine-tunes its responses by learning which actions are most effective in bringing the child’s focus back. If the child continues to display signs of inattention, hyperactivity, or other early ADHD indicators, DenEyes will experiment with different interventions, tracking their effectiveness. This dynamic feedback loop allows the robot to adjust its approach in real time, ensuring more personalized and effective interactions with each child.

As DenEyes interacts with the child, it compiles detailed behavioral data, including patterns of inattention or hyper-excitement. This data is analyzed and used to generate a comprehensive report that summarizes the child’s attention patterns, providing valuable insights to parents, educators, and healthcare professionals. The entire process occurs locally on the Raspberry Pi, ensuring privacy and preventing any external data collection.

Challenges we ran into

One of the major challenges we encountered was the hardware aspect of building the robot. The connections between the components, particularly the wiring, would frequently come loose, causing interruptions and forcing us to troubleshoot often. Additionally, the motor controller and the Raspberry Pi consumed a significant amount of power from the batteries, which led to frequent power shortages, making it difficult to keep the robot running smoothly during development.

Although we were not beginners, one of our team members was new to computer vision. This meant we had to quickly learn key CV concepts during the hackathon, including getting up to speed with OpenCV and MediaPipe. Figuring out how to integrate these technologies into our robot within the limited time of the hackathon was overwhelming at times, but we managed to overcome these obstacles through teamwork and determination.

Accomplishments that we're proud of

We are incredibly proud of several key achievements with DenEyes. First, we successfully managed to run the computer vision (CV) algorithms locally on the Raspberry Pi, overcoming the challenges of limited processing power and making real-time image analysis possible. Additionally, we’re proud of the fact that the robot’s movement functions worked perfectly as expected. Despite the challenges we faced with power consumption and loose connections, we managed to get the robot moving accurately and in sync with the attention-detection algorithms, making it both functional and responsive.

What we learned

Both of our team members gained new and valuable skills throughout the project. One teammate delved into learning how CV models and technologies such as OpenCV and Mediapipe works, other teammate expanded their expertise in Reinforcement Learning , learning how to make AI systems learn interactively. Together, we learned how to integrate these technologies into a cohesive system that runs efficiently on a low-power device like the Raspberry Pi.

What's next for DenEyes

Looking ahead, we plan to enhance DenEyes with new features that expand its potential applications. Our immediate goal is to adapt the robot to monitor for other diagnoses, such as Autism Spectrum Disorder (ASD)**, where attention and behavior tracking are also critical. We believe DenEyes could become a powerful tool for early detection of multiple neurodevelopmental conditions. We also see DenEyes evolving into a research project, where we can further refine its algorithms and explore its use in healthcare settings. Our plan is to reach out to professors and experts who specialize in ADHD, Autism, or related areas in child psychology and behavioral health. Their guidance will help us navigate the complexities of diagnosing conditions and ensure our technology aligns with medical standards and practices.