Inspiration

Applications that utilise the AWS SQS service, will generally have infrastructure setup to handle fail over of records i.e. [D]ead [L]tter [Q]ueues (using the acronynm from here-in).

DLQs are used to persist records that fail to get processed, for example because of network connection timeouts, etc.

With featured updates in 2021 users have the ability to re-drive dead letter messages to their sibling source queue.

However, with the new release of the StartMessageMoveTask aws sdk function, we can pragmatically re-drive DLQs without an engineer having to do this manually via the AWS Console.

Using AWS Lambda and Cloudwatch Events, we can programmatically trigger a DLQ re-drive. This is done simply by providing the relevant DLQ and source queue ARN.

What it does

I have setup a AWS Lambda to be triggered by a Cloudwatch Event trigger to periodically trigger a Lambda function to re-drive configured DLQs to their sources.

How I built it

  • AWS Lambda
  • AWS SQS
  • Python

Challenges we ran into

  • IAM permissions

Accomplishments that we're proud of

  • An Automated AWS SQS DLQ Redriver, that can run outside of daily business operational hours

Built With

Share this project:

Updates