Introduction:
Terraform state serves as the foundation of your infrastructure provisioning endeavors, crucial for managing cloud resources efficiently. In collaborative environments where multiple developers or teams contribute to a Terraform project, challenges emerge when updating the shared state file stored remotely, often in an S3 Bucket.
In an ideal scenario, updates to the state file should be carried out incrementally to ensure seamless integration of changes. However, in agile work settings, this sequential approach isn’t always feasible. Developers may need to update the Terraform state file independently and concurrently, leading to potential conflicts.
To mitigate these challenges and prevent conflicts, it’s essential to implement mechanisms that restrict developers from modifying the Terraform state file while it’s being accessed or updated by another developer. This ensures data integrity and fosters smooth collaboration within the team.
What is state and why is it important in Terraform?
“Terraform relies on maintaining state information regarding your provisioned infrastructure and its configuration. This state serves multiple crucial purposes, including mapping the resources defined in your configuration to real-world infrastructure, tracking metadata associated with these resources, and optimizing performance, especially in large-scale infrastructures. The state file acts as a vital reference, linking resource metadata to their corresponding IDs, enabling Terraform to accurately manage and track the infrastructure. It’s imperative to securely store and share this state file with anyone involved in running Terraform to ensure consistent and effective infrastructure management.”
Prerequisites:
1. Basic Understanding of Terraform
2. Familiarity with AWS Services (DynamoDB, S3, IAM)
3. Understanding of Infrastructure as Code (IaC) principles
4. Experience with Distributed Systems concepts
5. Access to an AWS Account
6. Basic Understanding of Database Concepts
7. Terraform Installed on Local Environment
What is a Remote State?
By default, Terraform stores state files locally under the name terraform.tfstate. However, in team environments, managing Terraform state becomes intricate due to the need to ensure that each user has the latest state data and that simultaneous Terraform runs are avoided.
Using a remote state, Terraform stores state data in a centralized location accessible to all team members, facilitating collaboration and avoiding concurrency issues.
What is a State Locking?
When your backend supports it, Terraform automatically implements state locking for any operations that involve writing state. This mechanism safeguards your state from potential corruption by preventing others from acquiring the lock. While state locking occurs seamlessly without any visible indication, Terraform halts operations if locking fails. Although you have the option to disable state locking for most commands using the -lock flag, doing so is generally discouraged.
STEP 1: Creating S3 Bucket using Terraform.
Please create a new file named “S3.tf” in your current working directory. Copy the provided Terraform configuration below into your text editor. Ensure to select a unique name for your S3 bucket in place of the given placeholder.
STEP 2: Setting our S3 Backend.
Please create a new file named “Backend.tf” in your current working directory. Copy and paste the provided configuration into your source code editor within the “Backend.tf” file.
STEP 3: Setting our DynamoDB Table.
Please create a new file named “dynamo.tf” in your current working directory. Then, copy and paste the provided configuration into your source code editor within the “dynamo.tf” file.
STEP 4: Let’s run commands.
Execute the command “terraform init” to initialize our backend configuration. Following that, proceed with “terraform apply” to apply the configuration changes and observe the outcomes.
STEP 5: State Locking. Our system employs DynamoDB to prevent concurrent modifications. When Person 1 initiates changes and endeavors to deploy the infrastructure, they will acquire a lock on the DynamoDB table(state file). Meanwhile, if Person 2 attempts to make alterations concurrently, they will encounter an error indicating that Person 1 has already obtained the lock, ensuring consistency and preventing conflicting modifications.
“When your backend supports state locking, Terraform automatically applies state locking to prevent concurrent writes that could corrupt the state. This locking occurs seamlessly during operations without any explicit notification. If state locking fails, Terraform halts further execution to prevent potential data corruption.”
STEP 6: Testing and validating our state-locking mechanism using DynamoDB.
Thank you for reading! If you have any additional thoughts or questions, please feel free to respond or leave a note. Your feedback is valuable to us!
Author: Shilpa Joshi