Inspiration

Being frustrated at looking through tables of contents, indexes, when technology provides easier, more functional searching.

What it does

An iOS app that enables you to search real life documents by simply taking a picture of them. If the picture is of a page of a book, we also tell you other places within that book where your query matches! Whether you're a recruiter looking for candidates with specific skills, or simply a student trying to find which page talks about method overriding, this app is for you.

How I built it

Using Swift 2, iOS 8.4, we made an iOS app (the first app for most of us on the team). We used tesseract, an open-source library for OCR, to extract text from pictures. We also use google books API to find book matches and to search within a book.

Challenges I ran into

Making api calls in Swift (and using the JSON it returns) is not exactly the most intuitive task, but thankfully we found a library (SwiftJSON) that helps a lot. We also found that tesseract does not expose each character's enclosing rectangles in easy ways, so the task of searching within the recognized string, and only highlighting the matched characters is still a bit buggy. We are often able to highlight characters close to the match, but a few characters offset. Given more time, (and we will definitely continue developing this), we could probably improve the accuracy of the highlighting.

Accomplishments that I'm proud of

Creating and running my first iOS app. Being able to click a button and debug on my own phone was quite exciting. I also finally got to learn Swift 2 and put it to practice.

What I learned

Basic (if messy) iOS app organization. How to use the iOS frameworks to build dynamic apps, etc.

What's next for Highlight

Highlight still needs a better connection to google books. Right now, the api doesn't expose the actual page numbers where our query matches, so we linked to a UIWebView of the online google book viewer. Not only that, but enabling the google books api section of the app makes it a little unstable, so that is something we still need to work on.

Share this project:
×

Updates

Loren Heubert-Aubry posted an update

Wield the powers of Command + F in real life! Using only your phone, watch as Highlight discovers your search terms by looking at a physical page, and then highlights them on the screen! Coming soon: refined Google Books implementation to not only learn the source you're searching text from, but where else in the document your search terms appear!

Log in or sign up for Devpost to join the conversation.