RoomInventory

Inspiration

Volunteering at the ER, one of the tasks is to constantly restock patient rooms with linens. However, nurses and volunteers need to manually check each room whether or not each room needs to be restocked. This is an incredibly inefficient and tedious method for such a mundane task and can be optimized so that doctors and nurses can provide more attention towards patient care.

What it does

Using GCP Vision API, the image of a cabinet in a stock-room is processed to determine the content of the cabinet. First a "background" picture is sent to the API to obtain the reference "empty" cabinet. Once the background is set, the API can be used to process the images of the cabinet at regular intervals, and by comparing the contents inferred against the background, we can determine whether the cabinet is empty and require re-stocking. This project aims to reduce the manpower needed to manually check each storage room for re-stocking.

How we built it

The GCP Vision API is capable of identifying objects by reading an image. Using the corresponding client library and Python code, we make requests to the GCP Vision API for every image of the cabinet. From the information returned by the API, we build the "background" information (what kind of objects would the API identify in an empty cabinet), and then compare the data of the subsequent images of the cabinet against the "background" in order to determine whether the cabinet is empty or not.

Challenges we ran into

Coming up with ideas that can be implemented within the scope of our knowledge.
Using the GCP Vision API, the authentication thereof.
Writing code in Python language.
Installing appropriate version of GCP Vision library, its associated authentication library, the Python language, and the Python installers. ** We deeply appreciate mentors for their help and patience in guiding our team through each of the many challenges we encountered. **

Accomplishments that we're proud of

Our entire team consists of 2 biology majors and 1 international political science major and with our limited skill set we feel that we have accomplished what we believe to be a miracle during this hack. Coming in none of us had any experience with any API’s, Python or the concept of image processing. Although we are nowhere near competent hackers now, given the time constraint, our limited knowledge and resources given, we feel that we have done our best to even create a product that can help make restocking an automated process which will convenience the employees.

What we learned

We learned the concept of an API, of authentications, and of HTTP requests, the importance of version compatibility for Python, what a JSON format is, command line prompts, what a documentation (for an online API and for the Unix command line manual) is, what it consists of, and how to read it.

What's next for RoomInventory

Improvements: Currently this is applicable for only a single cabinet. We hope to expand so that we can determine the status all the cabinets in a single room and from there obtain data of all cabinets in all the rooms at a hospital. This way we can determine how much of a particular inventory is needed for hospital as a system. Another way we thought of to reduce costs is to add a motion sensor onto each cabinet door which will send a signal to the camera to take a picture and update each time the door is opened.

Using built-in cameras, is not to optimal way to achieve this process especially for confidentiality purposes. An alternative could be to use a laser to determine distance regarding the “height” of the stack of supplies. Once the distance the laser reads reaches a certain threshold, a signal for restocking will be sent. Another alternative could be to use force sensors to determine the weight of the amount of supplies left. When the compartment becomes light enough, a signal for restocking will be sent.

Alternative Uses (Locations):