We wanted to make physical mail as convenient as email.
What it does
When mail arrives at a dorm or apartment building, it is dropped off into our machine. Using a webcam and optical character recognition, we parse the text on the outside of the envelope and look for a relevant name. If we find a match in our database, we send that person an email notifying them that a letter has arrived for them.
========= DETECTED TEXT ========= ["CHASE o\n\n, 6 p0 BOX17198 | Wilmington, DE 1985077198\n\n901'2n7oc'5405‘55501m'zu 1\nDELIAN ASPAROUHOV\n\nNIGHTINGALE APP INC\n487 COMMONWEALTH AVE\nBOSTON. MA 02215-2204", 'PRSR‘T STD\nUS. POSTAGE\nPAID\nCARD CENTER', 'EARN 80,000 3‘\n\nwith the Chase Ink Busi', "ONUS POINTS\n\nness Preferred 5'“ Card", '90 1\'2327ow5405-555mm-zn\nDELIAN ASPAROUHOV\n\nNIGHTINGALE APP INC\n487 COMMONWEALTH AVE.\n\nBOSTON. MA 02215-2204\n"-I-I"III\'I"u-l\'l\'u"II\'III-III-II"II""II\'II\'III"I"\'III\n\n \n\nI on nnn nnkll IC D“'] ========== DATABASE MATCHING ========== Name: Delian Asparouhov Score: 0.0 Match found! Sending firstname.lastname@example.org an email.
Challenges we ran into
Optical Character Recognition (OCR) is a really difficult computer vision problem! Even one of the best open-source libraries, Google Tesseract, has a lot of trouble on handwritten mail and requires substantial preprocessing to images.
Accomplishments that we're proud of
Under the right conditions, text is properly recognized! Also tying together hardware and many different software components.
What's next for Automated Mail Recognition and Notification
We're hoping to add spam mail detection, and more helpful emails to our users (i.e sending them a picture of both sides of the envelope).