Remote Terraform Backends: Terraform Cloud and AWS S3

Remote backends are essential for managing your Terraform state files in a collaborative environment. We'll discuss the two primary options for remote backends: Terraform Cloud and self-managed backends, with a focus on AWS S3.

Terraform Cloud:

• A managed offering from HashiCorp
• Specify a backend type of "remote" with organization and workspace names in the Terraform configuration
• Web UI allows you to interact with your account, organization, and workspaces
• Free up to five users within an organization, but costs $20 per user per month for more than five users

Self-managed Backend (AWS S3):

• Specify an S3 bucket and a DynamoDB table in the Terraform configuration
• S3 bucket stores the state file, while the DynamoDB table prevents multiple concurrent apply commands
• Requires a bootstrapping process to provision the S3 bucket and DynamoDB table

Bootstrapping Process for AWS S3 Backend:

1. Create a Terraform configuration without a remote backend (defaults to a local backend)
2. Define the necessary AWS resources: S3 bucket and DynamoDB table with a hash key named "LockID"
3. Run terraform apply to create the S3 bucket and DynamoDB table
4. Update the Terraform configuration to use the remote backend with the S3 bucket and DynamoDB table
5. Re-run terraform init to import the state into the new remote backend

terraform {
  #############################################################
  ## AFTER RUNNING TERRAFORM APPLY (WITH LOCAL BACKEND)
  ## YOU WILL UNCOMMENT THIS CODE THEN RERUN TERRAFORM INIT
  ## TO SWITCH FROM LOCAL BACKEND TO REMOTE AWS BACKEND
  #############################################################
  # backend "s3" {
  #   bucket         = "devops-directive-tf-state" # REPLACE WITH YOUR BUCKET NAME
  #   key            = "03-basics/import-bootstrap/terraform.tfstate"
  #   region         = "us-east-1"
  #   dynamodb_table = "terraform-state-locking"
  #   encrypt        = true
  # }

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.0"
    }
  }
}

provider "aws" {
  region = "us-east-1"
}

resource "aws_s3_bucket" "terraform_state" {
  bucket        = "devops-directive-tf-state" # REPLACE WITH YOUR BUCKET NAME
  force_destroy = true
}

resource "aws_s3_bucket_versioning" "terraform_bucket_versioning" {
  bucket = aws_s3_bucket.terraform_state.id
  versioning_configuration {
    status = "Enabled"
  }
}

resource "aws_s3_bucket_server_side_encryption_configuration" "terraform_state_crypto_conf" {
  bucket        = aws_s3_bucket.terraform_state.bucket
  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}

resource "aws_dynamodb_table" "terraform_locks" {
  name         = "terraform-state-locking"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "LockID"
  attribute {
    name = "LockID"
    type = "S"
  }
}

Terraform Cloud offers a managed solution, while self-managed backends like AWS S3 provide more control and customization.