Automated Mail Recognition and Notification

Inspiration

We wanted to make physical mail as convenient as email.

What it does

When mail arrives at a dorm or apartment building, it is dropped off into our machine. Using a webcam and optical character recognition, we parse the text on the outside of the envelope and look for a relevant name. If we find a match in our database, we send that person an email notifying them that a letter has arrived for them.

For example:

========= DETECTED TEXT =========
["CHASE o\n\n, 6 p0 BOX17198 | Wilmington, DE 1985077198\n\n901'2n7oc'5405‘55501m'zu 1\nDELIAN ASPAROUHOV\n\nNIGHTINGALE APP INC\n487 COMMONWEALTH AVE\nBOSTON. MA 02215-2204", 'PRSR‘T STD\nUS. POSTAGE\nPAID\nCARD CENTER',
'EARN 80,000 3‘\n\nwith the Chase Ink Busi',
"ONUS POINTS\n\nness Preferred 5'“ Card", '90 1\'2327ow5405-555mm-zn\nDELIAN ASPAROUHOV\n\nNIGHTINGALE APP INC\n487 COMMONWEALTH AVE.\n\nBOSTON. MA 02215-2204\n"-I-I"III\'I"u-l\'l\'u"II\'III-III-II"II""II\'II\'III"I"\'III\n\n \n\nI on nnn nnkll IC D“']

========== DATABASE MATCHING ==========
Name: Delian Asparouhov Score: 0.0
Match found! Sending iperper@mit.edu an email.

Challenges we ran into

Optical Character Recognition (OCR) is a really difficult computer vision problem! Even one of the best open-source libraries, Google Tesseract, has a lot of trouble on handwritten mail and requires substantial preprocessing to images.