Crib Protective Services

A digital baby monitor that analyzes baby cries and detects movements. Crib Protective Services is the CPS you call so CPS doesn't call you.

Comment

Inspiration

Although many consider it rewarding, raising a child can also be challenging– even more so for a single parent. In Dekalb County alone, more than 38% of the population, or 290,000 of Georgia’s households are run by single parents. Often as the sole financial provider for the family, it’s no easy task for a single parent to balance work, raising children, and relaxing. This challenge becomes even more evident at night when parents of young children struggle to relax, knowing their child may be at risk without supervision. In fact, during a child’s first year, 7 out of 10 parents lose an average of three hours per night. As college students already struggling to get enough sleep, we can't imagine how we'd function with three fewer hours of sleep without it compromising our mental health, academic performance, and other vital areas of our lives.

In addressing these issues, our goal was to help parents relax without undermining the important role they play in raising their children. Our commitment to creating a parental resource, rather than a replacement, led us to develop the Crib Protective Service (CPS). This system includes metrics designed to quickly identify the source of a child's discomfort when they're upset and detect when the child is in danger. With CPS, we hope to help reduce the pressure and workload on already-exhausted single parents.

What it does

We designed CPS with two core features in mind. Firstly, our AI model trained to determine the cause behind a baby's cry, and secondly, our motion detecting baby monitor to pair alongside the AI. These two features would be shown off in three primary ways on our web app. After logging in, parents would see Tear Analysis, Crib Vision, Health/History, all of which would make use of the data gathered to tell parents what their child needs most.

The Tear Analysis program would allow parents to use a microphone on their device to listen to their child's cries and give them a reported mood. Through this information parents can now best what their child may need, even when things get confusing or hectic, as parenting typically does. The next feature would be Crib Vision, which would utilize a baby monitor to not only continue determining your child's mood and feelings, but also keeping tabs on them, and letting you know when they decide it's time to move about. Lastly would be the Health and History section, where parents would be able to see a history of their child's mood based on cries, as well as keeping tabs on them at night if supported with a baby monitor.

How we built it

The development of the web app, CPS, would be developed on two fronts. The back end would be designed in Python, while the front end would be designed with React in Javascript. While CPS has many cohesive elements, the development of our ideas was non-linear.

Tear Analysis: The development of CPS initially revolved around a predictive model that analyzed the cries of young children, and its causes. A preliminary review of the dataset revealed that different cadences, pitches, and tendencies indicated a unique reason behind the child’s tears (hunger, discomfort, gassy, etc.). If the cause of a child's cries could be identified immediately, it would reduce the time parents need to stay awake (especially during common nighttime cries) and shorten the duration of the child's distress. Using tensorflow and a sound processing library librosa, we developed an AI model capable of accurately predicting crying causes.

Crib Safety Vision: If we were to integrate machine learning into a baby camera, it would be a missed opportunity to not experiment with computer vision as well. Because we had already planned to integrate our web app with a camera with a live feed, we aimed to address another problem new parents may face– their child climbing and falling out of their crib. If the program could identify when a child is moving a lot, it could help indicate the risk. In order to implement this software, we utilized OpenCV as well as streamlit to perform background subtraction of the current frame from the averages of the last frames in order to find differences to detect movement.

Web App Development: To unify all the elements together, we decided to design a web app that would enable users to easily toggle between each feature, increasing its accessibility. To do this, we took advantage of CSS and HTML programming with React to design a user-friendly interface. Colors were picked to be soothing to the eyes at a wide range of brightness, so using the app would not be overstimulating for the parents regardless if the child’s room is bright during nap time or dark during the night. To integrate backend scripts into the frontend, we used streamlit and port forwarding.

Challenges we ran into

Several challenges arose during this hackathon.

  1. While most of our team was comprised of CS students, we also didn’t have much experience developing front-end products. This led to multiple hours of trial-and-error in both the design aspect of React, and integrating our products (the tear analysis, health and history, and crib safety) required lots of discussion between the people working on the back-end products and front-end designers. Knowing how to load the altered version of the local host website instead of the blank one was one of the more notable points of confusion among our team. However, with the web app finalized, it became evident how our communication skills proved effective and efficient in sharing information.

  2. The majority of the hackathon was spent determining how to best train the AI to identify the cause of a child’s cry. Because the dataset disproportionately contained cries identified to be due to hunger, many of the preliminary iterations of the code identified new cries due to solely hunger. With the time limit presented by the hackathon, the dataset’s bias presented a new challenge, as there weren’t many ideal alternatives as comprehensive as the current dataset, meaning that we had to find a creative way to work around the heavy algorithmic bias this dataset presented. Our solution was to transform the existing data to add to the dataset and set thresholds to combat bias.

  3. One of the main functionalities of our project was analyzing baby crying sounds. Ideally, this would've been done using live microphone input. However, processing a live input in chunks proved to be challenging, as we'd need to determine the relevant chunks and divide them appropriately. Additionally, analyzing the crying sounds required us to transform audio files into images, which are then shaped into arrays. Doing so on every possible chunk in an input stream is incredibly inefficient. After many hours of iterative implementation and testing, we decided to let the user record their input instead.

  4. Our main programmer's laptop lacked an effective GPU, and the provided NVIDIA GPUs (and many other cloud computing services) did not support certain dependencies. As a result, finding the optimal number of steps and epochs needed to train the AI model was incredibly time-consuming, as each iteration takes approximately 30 minutes to complete. This meant that we had to manage our time strategically, especially considering the tight schedule and high stress hackathon environment.

Accomplishments and What We Learned

While we experienced hurdles during these thirty-six hours, we also had multiple wins. Most notably, I (Mishayla) lacked the most experience in computer science, and disproportionately so. Apart from simple statistics coding and the one intro CS class that scarred me three years ago, I lacked any genuine expertise in the discipline. Quite honestly, I was (and still am) hesitant to work with computer science, and it had initially presented itself as an obstacle to my team, all of whom already had multiple projects and classes under their belt.

Despite the intimidation, this hackathon reminded me of the power of having an interdisciplinary team. As a student studying medicine and public policy, I was able to bring my unique experiences of working with minority patients as a volunteer and draw on past health policy research on parenthood in Atlanta. At the same time, I received massive guidance on how to do front-end development with React, set up a GitHub, and differentiate between a push and pull command in repositories. I am incredibly grateful for the team I’m working with.

And together, not only were we able to learn from each other, but also became more connected to the city that ties us to Tech. Healthcare and addressing public health disparities such as childcare have always been at the forefront of Atlanta’s political discussions. One notion ingrained into every public policy student is that many of the problems that haunt us (such as the ones presented by the medical system) are often too large for a handful of people to fight against. The most long-lasting radical change we are inspired to fight for will not reveal itself until we are long gone. It’s scary, to lose that agency. However, this hackathon gives me hope that we create an impact that can immediately help people in the ways they need it– and that’s the beauty of technology.

What's next for Crib Protective Services

There were multiple features we wanted to integrate into CPS but lacked the time to do so, including:

Baby monitor integration: One of the best improvements we could make to our device would be to integrate it into baby monitor systems, so that they could work in tandem to provide parents with live updates on their child's condition.

Night vision for crib safety: Children climbing out of their cribs at night is considered an edge case of our computer vision, and an important one too, since parents cannot immediately see if their child is moving around in the dark.

Simultaneous dashboard: The original design of the web app was to have all three features (the baby monitor, tear analysis, and crib safety) all on one screen that could be displayed live whenever the parent was away from their child.

Implications for understaffed nurses, and overpopulated hospitals: The implication of a simultaneous dashboard could potentially lessen the workload of overly exhausted nurses managing multiple patients. If an AI were able to assist in monitoring the patient (assuming that no HIPPA laws are compromised), then it would reduce how often a nurse would have to cycle between patients and optimize the care they are able to provide.

Share this project:

Updates