Agent of Death

GIF
A GIF showing the browsing agent going through forgot password flow

Inspiration

When someone passes away, unfortunately the burden of managing their digital real estate falls upon their loved ones. We wanted to build a product that tackles this stigmatised and unaddressed market, where LLMs can make a significant impact in the quality of everyday people's lives.

The number of digital subscriptions that people have is growing year on year, research from barclaycard shows that the average British person spends £552 per year on subscriptions. Meanwhile research from Nordpass shows that the average person has about 100 passwords. Closing these accounts is an onerous process on a good day, but especially challenging while you are grieving the loss of a friend or a close relative.

LLMs have shown incredible potential to act autonomously in pursuit of achieving a certain objective. We strongly believe that the process of managing an individual's digital estate is a net negative to society and therefore a perfect use case for using LLMs to automate it away.

What it does

Once a user (grieving family) signs up to our product, the first step is to understand what subscriptions the deceased has. We do this in two ways, either parsing a bank statement to extract any subscriptions from there, or by allowing the user to manually enter subscriptions.

Once we have a list of subscriptions, the process of closing accounts begins. To do this, the product will either: AUTOMATIC CANCELLATION - Use an agent to browse the web, login to an account and cancel the subscription (where account details are known). Also if the login details are unknown the agent could go through the reset password process. This agent is still super experimental and is not live yet.

CANCEL VIA EMAIL - The agent will search for a customer support email for the service and draft an email from the user to begin closing the account for the deceased. Where an email is not found, the llm will use online information about how to cancel an account for that specific service and return these steps to the user.

How we built it

We started by thinking about the flow of the product that a user would have to go through. Roughly, it follows the steps of onboarding to collect relevant details, finding subscriptions that need cancelling and then finally going through the cancellation process.

Frontend

NextJS app with React
Tailwind CSS

Bank statement extraction:

Read the pdf using PDFJs
Use claude-2 api to understand the bank statement provided by the user and extract any potential subscriptions.
Store the subscriptions in the backend DB

Customer support email extraction

Use the Brave Search API to get relevant search results for finding customer support email for a particular service
Use claude-2 to parse the results and extract the email (being careful to avoid hallucinations, if it can't see the email it returns None)

Email generation

Where a customer support email is found, draft the email from the user to the service to begin the cancellation process (using claude-2)

Finding out how to cancel (if customer support email is not found)

Use Brave Search API to get search results for how to cancel that service for a deceased person
Use claude-2 to parse the results and return the easiest set of steps to follow to cancel the account

Browsing agent (experimental)

Built a crawler using playwright that can extract interactable elements on a web page and also take actions like clicking / typing on the page
Built an LLM agent that, given the browser content and an objective (e.g. reset password), will decide what action to take next on the webpage. The possible actions are TYPE, CLICK, SCROLL.
The combination of these enables to browse the web to accomplish either the goal of (a) logging in and cancelling the account if we have the account details OR (b) going through the reset password steps if the user has access to the deceased person's email (so they can take control of that account)

Challenges we ran into

Every service provider offers a very different way of cancelling an account (an online form, customer service email, live chat)
It's still difficult to get an LLM to browse the web by parsing web content, although it shows promising signs
claude-2 is very good at hallucinating emails! We had to be careful to handle the case where the email is not in the search results

Accomplishments that we're proud of

Building a thin version of the product that could already be used to support people during this process
The browsing agent can reason well over actions on the web which means in theory it could accomplish many tasks
Bank statement parser shows that we could theoretically use utility bills / mortgage statements or other letters to keep automating more of this process

What's next for Agent of Death

Continue building out the browsing agent to manage the whole process. Have an agent that automatically carries out the cancellation process via forms / email / direct cancellation through the account
Where a user has access to the deceased person's email, we can also use this email to find more subscriptions that need to be closed
Use more input documents to find out more subscriptions or accounts that we can automate the cancellation process for
Offer a much more empathic user experience, using the power of LLMs to make it conversational and make the user feel supported throughout the process