Definition
Terraform is an infrastructure-as-code tool with a declarative plan-then-apply workflow.
- Configuration – HCL files declaring the desired end state of cloud resources.
- State file – a JSON record mapping each declared resource to the real cloud resource by ID and attributes.
- Why it matters – Terraform diffs config against state against reality to compute what to create, change, or destroy.
- Remote state – stores the state in a shared backend (S3, Terraform Cloud) so every team member and CI pipeline reads the same copy.
- State locking – DynamoDB lock (S3 backend) or built-in locking (Terraform Cloud) ensures only one apply runs at a time.
At the National Bank of India, Terraform is a master plan saying exactly how the branch should be built: three counters, one vault, two ATMs. The state file is the official inventory ledger recording which specific vault and ATMs were actually installed, by serial number. Remote state keeps that ledger in the head-office safe so every engineer reads the same copy. State locking is the rule that only one person may write in the ledger at a time.
Understanding the Concepts
What is Terraform?
Terraform is an infrastructure-as-code tool that lets you declare cloud resources in configuration files and provisions them to match. You write what you want, not how to get there.
# main.tf - declare a VPC and an EKS cluster for the banking platform
resource "aws_vpc" "banking" {
cidr_block = "10.0.0.0/16"
tags = {
Name = "banking-vpc"
Environment = "production"
}
}
resource "aws_eks_cluster" "banking" {
name = "banking-cluster"
role_arn = aws_iam_role.eks.arn
vpc_config {
subnet_ids = aws_subnet.private[*].id
}
}
# The plan-then-apply workflow
terraform init # download providers and initialize backend
terraform plan # preview what will change
terraform apply # execute the changes
What is the state file?
The state file is a JSON record that maps every resource in your configuration to the real resource that exists in the cloud. It stores resource identifiers (instance IDs, ARNs), attribute values, dependency ordering, and sometimes sensitive outputs.
# The state file after applying
terraform show
# aws_vpc.banking:
# id = "vpc-0abc123def456"
# cidr = "10.0.0.0/16"
# tags = {Name = "banking-vpc"}
# Without the state, Terraform would not know vpc-0abc123def456 exists
# and would try to create a duplicate VPC
Why does the state file matter?
Terraform computes changes by diffing the desired configuration against the state, then checking the state against reality. Without state, Terraform has no memory of what it created.
# With state: Terraform knows the VPC exists and only updates tags
terraform plan
# ~ aws_vpc.banking
# ~ tags = { Name = "banking-vpc" -> "banking-vpc-v2" }
# Plan: 0 to add, 1 to change, 0 to destroy.
# Without state: Terraform thinks nothing exists
terraform plan
# + aws_vpc.banking
# Plan: 1 to add, 0 to change, 0 to destroy.
# This would create a DUPLICATE VPC!
How do you set up remote state?
A local state file on one laptop is unshared, unencrypted, and easily lost. Remote state puts the file in a shared backend.
# backend.tf - S3 remote state with encryption and versioning
terraform {
backend "s3" {
bucket = "nbi-terraform-state"
key = "prod/banking-cluster/terraform.tfstate"
region = "ap-south-1"
encrypt = true
dynamodb_table = "nbi-terraform-locks"
}
}
# Create the S3 bucket with versioning and encryption
aws s3api create-bucket \
--bucket nbi-terraform-state \
--region ap-south-1 \
--create-bucket-configuration LocationConstraint=ap-south-1
aws s3api put-bucket-versioning \
--bucket nbi-terraform-state \
--versioning-configuration Status=Enabled
aws s3api put-bucket-encryption \
--bucket nbi-terraform-state \
--server-side-encryption-configuration '{
"Rules": [{"ApplyServerSideEncryptionByDefault": {"SSEAlgorithm": "aws:kms"}}]
}'
How does state locking work?
State locking prevents the most dangerous failure: two applies running at once. Without locking, two engineers can read the same starting state, make conflicting changes, and write back a state that no longer matches reality.
# Create the DynamoDB lock table
aws dynamodb create-table \
--table-name nbi-terraform-locks \
--attribute-definitions AttributeName=LockID,AttributeType=S \
--key-schema AttributeName=LockID,KeyType=HASH \
--billing-mode PAY_PER_REQUEST
# When terraform apply runs, it writes a lock item to DynamoDB
# A second apply sees the lock and fails fast:
# Error: Error acquiring the state lock
# Lock Info:
# ID: abc-123-def-456
# Who: rajesh@ci-runner
# Operation: OperationTypeApply
How does local state differ from remote state?
| Aspect | Local state | Remote state (S3 + DynamoDB) |
|---|---|---|
| Shared | No (one laptop) | Yes (whole team + CI) |
| Encrypted | No | Yes (KMS) |
| Versioned | No | Yes (S3 versioning) |
| Locking | No | Yes (DynamoDB) |
| Recoverable | Easily lost | Restore from version history |
| Suitable for production | No | Yes |
What about state containing secrets?
The state file can contain sensitive values like a generated database password or a TLS private key. It must be encrypted at rest and access-restricted.
# A generated password ends up in the state file
resource "random_password" "db_password" {
length = 32
special = true
}
resource "aws_db_instance" "ledger" {
engine = "postgres"
password = random_password.db_password.result
# The password is now stored in the state file in plain text
}
# WRONG - state file committed to Git
git add terraform.tfstate
# Anyone with repo access can read the database password
# CORRECT - state in S3 with KMS encryption and restricted IAM
# Only the CI pipeline and senior engineers can read the state
What are common mistakes?
The most common mistake is not using remote state with locking from the start. Two engineers applying at the same time can orphan or duplicate production resources.
# WRONG - local state, no locking
# Engineer A: terraform apply (reads state, starts creating)
# Engineer B: terraform apply (reads same state, starts creating)
# Both create the same resource -> duplicate production database!
# CORRECT - remote state with DynamoDB locking
# Engineer A: terraform apply (acquires lock)
# Engineer B: terraform apply
# Error: Error acquiring the state lock
# Someone else is running apply. Wait and retry.
What should a senior engineer know?
Keep separate state files per environment so a mistake in dev can never touch production state. Use different S3 keys or entirely different backends.
# Dev state
backend "s3" {
key = "dev/banking-cluster/terraform.tfstate"
}
# Production state (completely separate)
backend "s3" {
key = "prod/banking-cluster/terraform.tfstate"
}
Run applies only through a controlled CI pipeline. This ensures the lock, plan review, and audit trail are consistent, and no one accidentally applies from their laptop against production.
# CI pipeline flow
terraform init -backend-config=prod.hcl
terraform plan -out=tfplan
# Human reviews the plan
terraform apply tfplan
Visual Overview
Terraform Plan-Apply Workflow with Remote State
=================================================
terraform plan
│
├─→ Read desired config (main.tf)
├─→ Read state (S3 bucket)
├─→ Query providers for reality (AWS API)
├─→ Diff: config vs state vs reality
▼
Plan output: "1 to add, 2 to change, 0 to destroy"
terraform apply
│
├─→ Acquire lock (DynamoDB)
├─→ Execute the plan (create/update/destroy)
├─→ Write updated state (S3)
├─→ Release lock (DynamoDB)
▼
Done.
State Locking (DynamoDB)
=========================
Engineer A: terraform apply
│
├─→ Writes lock item: {LockID: "prod/banking", Who: "rajesh"}
├─→ Applies changes...
│
Engineer B: terraform apply (concurrent)
│
├─→ Sees lock item exists
▼
Error: "Error acquiring the state lock"
"Lock held by: rajesh"
Key Takeaway
Terraform is an infrastructure-as-code tool where you declare the desired end state and Terraform provisions reality to match. The engine of that workflow is the state file, which maps every declared resource to the real cloud resource by ID and attributes. Without state, Terraform cannot tell an update from a new creation and would try to duplicate production infrastructure.
Remote state stores the file in a shared, versioned, encrypted S3 bucket so the whole team and the CI pipeline work from the same source of truth. State locking with DynamoDB ensures only one apply runs at a time, preventing two engineers from reading the same starting state and making conflicting changes that corrupt it.
The state can contain sensitive values like generated passwords, so it must be encrypted at rest and access-restricted. Keep separate state files per environment, and run applies through a controlled CI pipeline for audit and safety.
Remember one sentence: the state file is Terraform's memory, so store it remotely with versioning, encryption, and locking, and never apply without it.
---