CapaCV

• 

  An assortment of grocery stores, cafes, restaurants, and libraries in LA: the occupancy was synthetically generated

• 

  Filtering for cafes

• 

  An example of an area with 'high' occupancy: these are labelled red

• 

  An example of an area with 'low' occupancy: these are labelled green

Inspiration

Have you ever tried to meet up with friends at the local cafe before finding out that it was full? More pressingly given the need for social distancing, have you wanted to know when grocery stores and supermarkets are relatively empty? CapaCV can solve these problems by utilizing sensors and computer vision to determine the current occupancy of a given location.

What it does

CapaCV uses real-time sensor data to update the user on areas that are currently near their carrying capacity. This can help reduce transit time, as less time is wasted going from place to place in search of a cafe with a spare outlet, or aid with social distancing, as overlapping trips to areas like grocery stores could prevent us from flattening the curve.

How I built it

I used the Tensorflow object detection API to get a headcount of the number of people in a given room. I then used flutter to create a phone app that could visualize this data in real-time, while interfacing between sensor and location data via Firebase and Google Cloud Platform. While I used my laptop to process sensor data and transmit it to a central database, many devices could substitute as part of a scalable solution, such as a raspberry pi. I also generated synthetic data for both the 'current' and maximum capacities in various locations in LA, though a combination of cooperation with listed areas and crowdsourcing would be closer to production-ready. For privacy reasons, the data collected by the sensors/cameras are also never stored; all information is thrown out except for the number of people present at a given location.

Challenges I ran into

Squeezing the different facets of the problem in the tight time constraints was probably the hardest part of the project! This was also the first time I attempted to tackle the entire stack by myself, which certainly didn't help things.

What's next for CapaCV

Fine-tuning the object detection to also account for the types of open spots (ex. one 4 person table vs two 2-person tables), as well as queue length, might help reduce wasted time even further, as a group of 4 might not be content with being split up across difference tables, while someone who only wants to-go might only care about the time spent in line, and not the total occupancy in the restaurant/cafe.

Additionally, partnering with restaurants and cafes that aren't especially suited for a formal reservation system would be mutually beneficial (a la opentable), as consumers could avoid having to wait a long time, while restaurants could fill some of their otherwise empty tables. Also, akin to reservation systems, sending a notification to waiting consumers would be a neat addition, as this would help with efficiency even further.