Nittany Locator

Inspiration

Regardless of items price and value, losing an item is such a disappointment. However, Team Krusty Krab identified three major problems (Uncertainties of item being picked or not) of the Pennsylvania State University’s lost and found system. Our solution is suggesting a new paradigm of the lost and found system. Our integrated systems solution will lead to an increased possibility and efficiency in finding the lost items and can contribute to the environmental development by preventing potential disposal of unclaimed items.

What it does

Let’s assume that you have lost a key or something very meaningful to you on Campus. According to the University Policy of lost items, any items under 25 dollars for 60 days will be excluded from the finding process and will be in immediate disposal. Therefore, to avoid unnecessary disposal of your precious belonging, you need to find the lost item immediately. However, there are two specific limitations to finding such items. First, you do not know if your lost item is being picked up by a person or not. Second, even if the item is picked up by a person, you do not get notified of the event. For instance, despite there being multiply lost and found offices on campus, you have limited information about where your item is being returned. Our application’s algorithm provides a matching system between the loser and finder based on the user inputs of the location, picked-up date, and item. Furthermore, not only improving the efficiency of the existing system, we can promote the supply chain and environmental development. While the existing system forced the user to buy a new one once they confirmed that the item is being lost, the solution that we suggest has integrated a used market for providing similar items to the lost item.

How we built it

We divide the users Into two groups. One is the group with people who are looking for their lost items and the other is the aggregated group of individuals who have found the items including the current lost and found offices. The users will create a post and pin on the suspected locations of the items. Using our systems algorithm, it calculates the correlation between the items. Then it will return the potential locations and matches to the two groups.

What is the detailed procedures and descriptions?

Once lost users enter the app, the app requires them to fill out the basic information about the object they lost. The information includes the item category, the model name of the item, the date when they lost, the location, and a short description of the object. For the next step, according to the information the lost users submitted, the user could get the data of similar items that have already been found by other individuals.

Name, category classification

The program implements the first data classification using item names and categories. It loops through the found item table and filters items on the database that have the same name and category.

Date, location classification

The second classification is done by using the date and location. Among the found items that we have filtered through the first classification, we compare the time and location value of the lost item to that of each of the found items. Then, it sorts out the items that are found around 1 km, registered within 3 days. Those values are adjustable. We expect that most of the items sorted at this stage are the ones that have the highest correlation of image matching, which means the ones that have the highest possibilities of what users are looking for.

Image, description classification using machine learning model

In the last stage, the data sorted goes into the AI model called “Image Classification” which recognizes the image and lists the trait as a text. By using this, we can even further boost the accuracy of the classification based on the object features such as color or shape. This data that went through three specific processes would be the final result for the data classification before matching the possible found items for users.

Challenges we ran into

The challenge that we ran into was how to provide a user with the best and most accurate information about lost items. We decided to implement machine learning techniques in the system. For instance, our first approach was to predict the possibility of the user losing an item in a specific region. However, due to the limitation of collecting valid data, it was unachievable. Secondly, we tried to send out notifications if users enter a location with a high frequency of lost items reported. It turned out to be an obvious statistical analysis that will result in those locations with concentrated populations. We could overcome the challenge by automating the system process of writing descriptions with image recognition techniques. Based on the image recognition model's return value, the image-to-text model will convert the image into a description, and finally, calculate the correlation to return the best match between items.

Accomplishments that we're proud of

We are extremely proud of our functioning prototype. Given a limited time of 24 hours, we were able to identify and analyze a problem, come up with a creative solution, implement deep learning technology to integrate image recognition, and provide the solution for the problem with a strong presentation.

What we learned

We think that the biggest takeaway from this hackathon event is the importance of a structured approach to solving problems and cooperation/communication among the team. Through consistent communication, we could distribute the work according to each team member's strength and cooperatively put all the work together with the pressure of limited time.

What's next for Nittany Locator

Our solution is suggesting a new paradigm of the lost and found system. Another feature of the solution is the integration of the Used market industry. If you are certain that you have lost the item, the system will give you suggestions on similar items to the lost item, in order to satisfy the immediate demand from the users. It will not only decrease the tremendous amount of trashes created by the unclaimed item, but also suggesting new idea of supply chain and sustainability through drawing the used market industry to the system. Therefore, Our integrated systems solution will lead to an increased possibility and efficiency in finding the lost items and can contribute to the environmental development by preventing potential disposal of unclaimed items.