CradleCam

Home page of CradleCam
Notifying the user after a negative emotion is detected.
Emotion monitoring and Emotion feed.

Inspiration

The inspiration behind CradleCam comes from my (Shuvo) experience with my two nieces, Ayaat and Arisha. My family has faced numerous challenges in keeping a constant watch over them since their birth. A child needs round-the-clock attention as they are delicate beings, and even a small inconvenience can cause them great harm. It was extremely difficult to be by their side 24/7 with so many other responsibilities at hand. This project was born out of the need for a reliable solution to ensure their safety and well-being, even when we can't be physically present. As computer science students and enthusiasts, we wanted to create a smart solution that could monitor the baby on its own, a digital nanny.

What it does

Our website primarily aims to monitor babies through a live webcam feed. It uses images from the webcam to analyze the baby’s emotions. It keeps tracks of the baby’s recent emotional states and notifies the user whenever there is a negative emotion detected such as sad, angry and crying.

How we built it

We have designed a simple, user-friendly interface using React.js so that it becomes intuitive and accessible to people of any literacy level or occupation. The images are analyzed using FaceAPI to detect and interpret emotions accurately.

Challenges we ran into

Initially, we considered training a lightweight machine learning model like MobileNetV2 to analyze images and detect the baby's emotions. However, due to the lack of an existing dataset for this specific use case, we decided to develop a Flask application that used pretrained models from Python's DeepFace and MediaPipe libraries. Unfortunately, the results were not sufficiently accurate, as these models were not specifically trained to detect the emotions of babies. Thus, we ended up using JavaScript's FaceAPI, which provided more accurate results.

Accomplishments that we're proud of

• Successfully transforming our idea into a functional and realistic solution. • Providing an idea that makes a positive impact on the community. • Designing a system that provides real-time notifications to users. • Providing a unique and free solution. • Our team showed resilience and teamwork by maintaining focus despite the challenges we have faced as our country went through a massive political transition and a devastating flood. • Designing a user-friendly UI that ensures ease of use, making advanced technology accessible to all.

What we learned

Our team was composed of individuals with diverse knowledge and expertise across various topics. Throughout the development of CradleCam, our team gained knowledge and skills that extended far beyond our initial areas of expertise. We were able to deepen our understanding of computer vision and explore different related libraries, such as DeepFace, MediaPipe, and FaceAPI. We were also able to learn about frameworks, such as Next.js and React.js, which were essential in building a responsive and user-friendly web application. Except technical expertise we also learned the importance of flexibility and problem-solving when we faced technical setbacks. We learned the importance of resilience, staying focused on our goals even in the face of adversity, and maintaining our commitment to completing a task.

What's next for CradleCam

We plan to increase the functionality of CradleCam to make it even more efficient and versatile. Our vision includes integrating CradleCam with the user’s existing CCTV system, allowing it to monitor baby movements and emotions through image and audio analysis. To enhance accuracy and speed, we aim to incorporate vision transformers, training them on a custom-built dataset specifically made for our use case. Additionally, we plan to develop a mobile app, ensuring that users can easily access CradleCam from any device. To safeguard privacy, we will implement a secure login/logout system, ensuring that only authorized users can monitor their baby's status. We will also try to implement a threat detection system that will notify the user when any abnormality is detected, or any unidentified person comes near the baby. The system will also be upgraded to make emergency calls to notify the user if they are not connected to the internet.

Team Members

Member 1: Sk Shafiqul Islam Shuvo Institute/Organization: North South University, Dhaka, Bangladesh Email: [email protected]

Member 2: Hiramon Nahar Eva Institute/Organization: National University, Bangladesh Email: [email protected]

Member 3: Sayed Shabik Ashab Institute/Organization: Chongqing University Of Technology Email: [email protected]

Member 4: Yousra Amin Institute/Organization: North South University, Dhaka, Bangladesh Email: [email protected]

You can try out the app through the first link given. The first git repo consists of the project integrated with faceapi. The second one just consists of the flask backend we attempted to build, as mentioned earlier