FINDER

Inspiration

Members of our team have had times where we could not find j-cards, keys, or other small gadgets in our rooms, or having trouble locating a particular book on our shelves. Elders in our families have talked about the difficulty of searching for certain things at home due to degrading eyesights or poorer memory.

What it does

FINDER is an application that helps busy college students, households and old people with possible dementia with locating their lost objects in the house. The application takes in keywords of lost items, uses an input video and analyzes it frame by frame. Then it highlights the identified object with boxes and prints labels beside them. In addition, FINDER also allows users to input book titles or part titles on an interface and search the book among rows of books on the shelf, and saving time for the user.

How we built it

We brainstormed on the difficulties and needs we faced in quickly locating and recognizing objects in our houses from videos and pictures as students and from the perspective of our grandparents. We learned to use google cloud vision API in video and image analysis, and text dictation. We also used open CV to build image analysis results with annotations, and Tkinter for the graphic user interface construction.

Challenges we ran into

We had trouble setting up the environment. Some members had problems running API on their laptops and some had issues with CPR. The debugging process took hours, but we could not find the bugs in locating only on the box of the target title in a bookshelf picture.

Accomplishments that we're proud of

We were able to anticipate problems the target group may face and write up solutions. We also learned skills to quickly read and understand previously unseen codes.

What we learned

We learned to select and utilize the most suitable API and libraries to realize our designed functions. Python is also a new language for our members, and we have strengthened our skills in writing codes with Python.

What's next for Finder

In the future, we hope to train the application to recognize more labels with more accuracy. We also hope to store in databases the common places an object usually appears at, thereby informing users where they can look for it. We hope to extend the shelf-searching function to more languages, and with a mix of languages.

Built With

• cv2
• google-cloud
• numpy
• opencv
• python
• tkinter