Live Migration of DynamoDB Tables

Recently I was faced with the challenge of having to migrate a set of AWS DynamoDB tables to completely new tables. We wanted to achieve this without affecting any of our users, and without having a maintenance window while migrating data from the old table to the new ones.

The following will be a very high-level overview of how you:

  • Get all your DynamoDB events onto a queue
  • Replicate your DynamoDB tables to the new tables (or even a different region)
  • Continuously synchronize your original tables to your new tables
    • Restart the migration if you made an error
  • Complete the switchover after validating everything looks OK

Read more →

Setting up UnrealIRCd and Anope IRC Services on EC2

Having recently discovered sameroom.io I wanted to update the codetalk IRC server to be compliant with their authentication method. This basically just meant enabling SASL support, but while I was tinkering with stuff anyways, I thought I might as well streamline the setup process for the IRC server. In short, everything is fully automated and set up on AWS using EC2 and S3.

This will go through the process of doing exactly that, by talking about:

Read more →

S3 bucket specific policy

I have recently started caring a bit more about security on my AWS applications, and to this end Identity & Access Management (IAM) users are a great way to limit access to a need-to-use-only basis.

Recently I set up my IRC server to download its configuration and install files from an S3 bucket. This meant that it needed to have read access to a specific bucket, and for this an IAM role was created.

There are two ways to generate policies:

I will generally advise to either use the generator completely or at least use it for the basis of the policy you want to create.

Read more →