-
Canny Edge Detection
-
Original Image of Books
-
Hough Lines, and grouping of lines
-
Lines over the original image
-
Web app submission form
Inspiration
We were inspired by our interest in computer vision and google's machine learning APIs. Bookshelves seemed like an interesting problem, because there's a wealth of information, but it's non trivial to separate it out, and it's hard for humans to organize, too! I think a lot of people hardly know what books they own, and it's not easy to have that information easily available if you have to individually catalog each book. Something that presented that information in an easily accessible format seemed like a really interesting project, and I think anyone with a large book collection might find it useful.
What it does
LibGarden is a web app that allows you to submit one or more images of your bookshelves, and give you information about the books that you scanned. It reads the spines of the books to get information about them, displays a list of the images next to the text read from that spine.
How I built it
The web app portion uses PHP, Bootstrap and HTML, and allows the user to submit images and view their book collection. Upon submission, the image is run through a Python program which first performs edge finding to identify areas of high contrast, then fits lines to the edges that make up the sides of book spines, using a variety of filters and operations in OpenCV (Canny edge finding and Hough filter among them). Then, the image is sliced along the lines, and each slice is sent to google for OCR (optical character recognition). The resulting text is stored in the SQL database and displayed to the user on the web app.
Challenges I ran into
One of the biggest challenges of the project was edge finding. I had very limited prior experience with OpenCV in Python, and it was fun, but difficult work to go from an image to a bite-sized piece that means something all on its own, which is an individual book. There was a lot of trouble with a filter that finds lines in images, because it would give four or five line segments that really made up one line but were registered as different due to things not being perfectly straight. I couldn't figure out how to solve it from google searches, so I had to develop my own method of grouping nearby clusters of lines into one.
Accomplishments that I'm proud of
I'm really proud of figuring out that tricky edge finding problem was really satisfying. I am really proud of my whole team for working together and getting a product with both a clean user interface and a complex back end to work really well.
What I learned
A whole lot about Computer Vision and a little bit about Google's API.
What's next for LibGarden
There are a lot of different ways this could be extended. The Computer Vision could always be improved, and we could use the information that we parse from the text to learn more about the books.
Log in or sign up for Devpost to join the conversation.