We see many people lose their IDs, smartphones, car keys every day. However, the process of getting the items back is unpleasant because:
- PRIVACY: Facebook group of lost items may expose your personal items to threats such as identity theft
- TOTALLY LOST: You cannot remember where you lost the item and then get frustrated when having to go through all the lost and found location on campus
- TIME-CONSUMING: For people who manage the lost and found office, it is time-consuming to classify items and organize them into different categories
- GROSS: No one wants to go through the ominous box filled with piles of items. Sometimes clothes have been sitting there for long periods of time and have an unpleasant smell.
What it does
In order to solve three big problems above, we design a solution named Found!:
- INVENTORY: Fast and easy inventory creation for lost items using object detection, color, and textual analysis
- PROTECTION: strangers cannot see all of the items in the inventory, item description is only shown when certain level of match occurs
- ONE FOR ALL: One search site for all the lost and found offices of a campus, an airport, or even an entire city
How We built it
Azure Custom Vision API for accurate object detection Azure OCR API to detect any text on the object Azure SQL Database to store the items details Azure Blob storage to store image files Azure Search as a service to search through SQL for matches quickly Visual Studio for web app development Power BI for administration dashboard Adobe Illustrator Adobe Photoshop
Challenges We ran into
Accuracy of image of classification: ready-to-use API provide too many possibilities with different level of confidence Accuracy of object color: difficult to distinguish color of the object and color of the background Select the right database to serve searching well Connect different types of database to PowerBI for visualization Version controll across the board
Accomplishments that I'm proud of
Build a fully featured product for 3 stakeholders in a short amount of time:
- People who lost items
- People who classify items
- People who manage inventory of lost items
What I learned
THE CONNECTION: between different application of Microsoft from Azure, PowerBI, to Visual Studio Web App THE USER EXPERIENCE: the more we build it, the more we understand how an user may want to use it
What's next for Found!
ENCOURAGE "FOUND" NOT "LOST":
- IMPROVEMENT ON ITEMS CLASSIFICATION with larger dataset for increased accuracy for identifying objects
- MOBILE APP: easy drop-off for anyone that picks up items of another person. Just take a picture and drop off at a box. We make use the pictures you take for classification or take more pictures ourselves.
- NOTIFICATIONS: users won't have to continuously check the site for their item since they can now be notified when an item matches their description.