Snail Mail

• 

  Example Email, Summarizing Mail Content and Ordering by Priority

Inspiration

Everything about physical mail sucks. It requires your attention, every piece demands context switching, and all that really matters it the occasional signal buried within a ton of noise.

What it does

Snail Mail users advanced OSS technologies to turn your physical mail into e-mail. Your mail is organized by priority and summarized, so you can focus on more important things.

How we built it

Our mail is full of dark data; content shows up in all kinds of formats, fonts, and arrangements. We use:

• A 3rd party partnership to receive and digitize physical mail
• PaddlePaddle's OCRv3 to extract data from images
• OpenAI's API to classify mail type and priority, while also summarizing contents
• Sendgrid to send a tailored summary of mail pieces
• TBD: Will need some infra to support the whole thing in production. Will need to host inference for PP-OCR, and will need something like AWS Lambda functions for the execution and AWS S3 bucket for securely uploading the digitized documents so the end user could also have private access to the original digital documents.

Challenges we ran into

While existing services with worldwide facilities are available to open mail, partnership is required. We are not yet sure what options they exist for securely providing images to our endpoint, but if necessary we can upload the digitized images. Getting the prompt right is tricky, especially with different models. It is likely that multiple models and/or multiple passes on the data may be necessary to improve output.

Accomplishments that we're proud of

The POC works, and shockingly well. Out of the first ~15 documents we tried all but 2 produced results which were very reasonable and exactly what we were looking for. The sample email in the example images shows actual output!

What we learned

It's possible to do absolutely amazing things with modern tech, and fast. This AI/ML thing is bananas.

What's next for Snail Mail

I'm thinking about renting my house and travelling, and legit don't know where to send my mail. Am a bit nervous about the overhead for doing this privately and securely while accessing my mail overseas, but it's compelling to think about.

Built With

• gradio
• huggingface
• openai
• paddlepaddle
• python
• sendgrid