Terraform for Teams: State Management, Modules, and CI/CD Integration

Editorial Team · · 4 min read

Terraform Gets Complicated When There's More Than One of You

Running Terraform solo is straightforward. You write HCL, run terraform apply, and your infrastructure exists. Add a second person and suddenly you've got state conflicts, resource duplication, and the very real risk of someone running terraform destroy against production because they forgot to switch workspaces.

Here's how to set up Terraform for team use without losing your mind or your production database.

Remote State: Stop Committing terraform.tfstate

Terraform's state file tracks the mapping between your HCL resources and real infrastructure. By default, it's a local file. Committing it to git is dangerous — it contains sensitive values (database passwords, API keys) in plain text, and simultaneous edits create merge conflicts that corrupt the state.

S3 + DynamoDB Backend (AWS)

# backend.tf
terraform {
  backend "s3" {
    bucket         = "mycompany-terraform-state"
    key            = "prod/networking/terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true
    dynamodb_table = "terraform-locks"
  }
}

The S3 bucket stores the state file with encryption at rest. The DynamoDB table provides state locking — when someone runs terraform apply, it acquires a lock preventing anyone else from modifying state simultaneously.

Create the S3 bucket and DynamoDB table manually (or with a one-time bootstrap script) before using them as a backend. Don't manage the state backend with Terraform itself — that's a chicken-and-egg problem.

GCS Backend (Google Cloud)

terraform {
  backend "gcs" {
    bucket = "mycompany-terraform-state"
    prefix = "prod/networking"
  }
}

GCS provides built-in locking without needing a separate lock table. One less component to manage.

Terraform Cloud

HashiCorp's hosted service handles state storage, locking, and run execution. Free for up to 5 users. It's the lowest-friction option if you don't mind the dependency on HashiCorp's infrastructure. The remote execution model means terraform apply runs on their servers, not your laptop — consistent environment, but slower feedback loop.

Module Design

Modules are Terraform's reuse mechanism. A well-designed module encapsulates a logical infrastructure component (a VPC, an ECS service, an RDS cluster) with configurable inputs and useful outputs.

Module Structure

modules/
  vpc/
    main.tf          # Resources
    variables.tf     # Input variables
    outputs.tf       # Output values
    versions.tf      # Provider version constraints
    README.md

  rds-cluster/
    main.tf
    variables.tf
    outputs.tf
    versions.tf

Module Versioning

For shared modules used across teams, version them. If modules live in a separate repo, use git tags:

module "vpc" {
  source  = "git::https://github.com/myorg/terraform-modules.git//vpc?ref=v2.1.0"

  cidr_block       = "10.0.0.0/16"
  availability_zones = ["us-east-1a", "us-east-1b", "us-east-1c"]
  environment      = "production"
}

Pinning to a specific version (ref=v2.1.0) means a module update in the shared repo doesn't automatically affect your stack. You upgrade deliberately by bumping the version — same principle as pinning library versions.

What Makes a Good Module

A module should represent one logical thing. A "vpc" module creates a VPC with subnets, route tables, and NAT gateways. An "ecs-service" module creates a task definition, service, target group, and listener rule. Don't make a module that creates an entire application stack — that's too coarse to reuse.

Expose enough variables for configuration but not so many that using the module is as complex as writing the raw resources. Default values should cover 80% of use cases. I aim for 5-10 input variables per module; more than 20 is a smell that the module is doing too much.

CI/CD Integration

Plan on PR, Apply on Merge

The most common CI/CD pattern for Terraform:

  1. Developer opens a PR with infrastructure changes
  2. CI runs terraform plan and posts the plan output as a PR comment
  3. Reviewer reads the plan to verify what will change
  4. On merge to main, CI runs terraform apply with auto-approve
# GitHub Actions example
name: Terraform
on:
  pull_request:
    paths: ['infra/**']
  push:
    branches: [main]
    paths: ['infra/**']

jobs:
  terraform:
    runs-on: ubuntu-latest
    defaults:
      run:
        working-directory: infra/prod
    steps:
      - uses: actions/checkout@v4
      - uses: hashicorp/setup-terraform@v3
        with:
          terraform_version: 1.7.4

      - run: terraform init
      - run: terraform validate

      - name: Plan
        if: github.event_name == 'pull_request'
        run: terraform plan -no-color -out=tfplan
        continue-on-error: true

      - name: Apply
        if: github.ref == 'refs/heads/main'
        run: terraform apply -auto-approve

Atlantis

Atlantis is an open-source tool purpose-built for Terraform CI/CD. It runs as a webhook server that listens for PR events, runs plan/apply, and posts results as PR comments. It supports locking at the directory level — if someone has an open PR changing the networking module, nobody else can plan against it until that PR is merged or closed.

I've used Atlantis on teams of 15+ and it works well. The main advantage over generic CI is the built-in workflow awareness — it understands Terraform-specific concepts like workspaces, state locking, and plan-then-apply ordering.

State Organization

Don't put everything in one state file. A single state for your entire infrastructure means every terraform plan checks every resource, which gets slow (minutes for 500+ resources) and every change carries blast radius risk — a typo could theoretically affect any resource.

Split by blast radius and change frequency:

infra/
  networking/          # VPC, subnets, NAT — rarely changes
    main.tf
    backend.tf         # s3://state/prod/networking/
  database/            # RDS, ElastiCache — changes occasionally
    main.tf
    backend.tf         # s3://state/prod/database/
  application/         # ECS, ALB — changes frequently
    main.tf
    backend.tf         # s3://state/prod/application/

Use terraform_remote_state data sources or SSM Parameter Store to share values between states. The VPC module outputs its VPC ID and subnet IDs; the application module reads them:

data "terraform_remote_state" "networking" {
  backend = "s3"
  config = {
    bucket = "mycompany-terraform-state"
    key    = "prod/networking/terraform.tfstate"
    region = "us-east-1"
  }
}

resource "aws_ecs_service" "app" {
  # ...
  network_configuration {
    subnets = data.terraform_remote_state.networking.outputs.private_subnet_ids
  }
}

Drift Detection

Infrastructure drift happens when someone modifies resources outside of Terraform — through the AWS console, a CLI command, or another tool. Terraform doesn't know about these changes until you run terraform plan.

Run terraform plan on a schedule (daily or weekly) in CI and alert if drift is detected. If the plan shows changes you didn't expect, someone modified infrastructure manually. Either import the change into Terraform or revert it.

AWS Config, GCP's Cloud Asset Inventory, and tools like driftctl can also detect drift independently from Terraform. They're worth setting up as a second line of defense.