Late Night Terraform: The Assembly Line

🎙️ Opening Monologue

My configuration file has officially crossed the line from helpful to hostile.

I’ve been scrolling for minutes just to find a single security group. It feels like my living room floor after the kids leave their Legos everywhere — chaotic, disconnected, and painful to step through. Everything technically works, but nothing feels manageable.

Late nights make this kind of sprawl impossible to ignore. Terraform was never meant to be a monolith. It was meant to be composed — built from parts that can be understood, reused, and trusted.

Tonight is about breaking the mess into meaningful pieces. About boxing chaos into clean units that scale without collapsing under their own weight.

If we’re going to build a cathedral, we need better bricks.

It’s time to think in modules.

🎯Episode Objective

This episode aligns with the Terraform Associate (004) exam objectives listed below.

• Explain how Terraform sources modules
• Describe variable scope within modules
• Use modules in configuration
• Manage module versions

The Blueprint Revolution: Why Modules are the Unit of Scale

In simple terms, a Terraform module is a container for multiple resources that are used together. Think of it like a function in programming or a Lego set: instead of building every individual piece from scratch every time, you package a group of components into a single, reusable unit.

Why Use Modules?

Without modules, your Terraform code can become a “wall of text” that is hard to manage. Modules solve this by providing:

• Reusability: Write the code once (e.g., for a standard web server setup) and deploy it across Dev, Staging, and Production environments.
• Organization: Group related resources together so your main configuration stays clean and readable.
• Consistency: Ensure that every VPC or Database created in your organization follows the same security and tagging standards.

Types of Terraform Modules

1. Root Module

• Every Terraform project has exactly one root module.
• The directory where you run terraform init / terraform apply
• Contains .tf files directly executed by Terraform

1. Child Module

• Any module called by another module is a child module.
• Used to encapsulate reusable infrastructure logic
• Can be local, from Git, or from a registry

The Interface: Understanding the module Block

A Terraform module block is a parameterized instantiation of reusable infrastructure, resolved at init time and executed as part of the dependency graph.

module "<LABEL>" {
  source = "<module-source>"
  # Optional arguments
  version    = "<version-constraint>"    # Registry modules only
  count      = <number>                  # Mutually exclusive with for_each
  for_each   = <map | set(string)>       # Mutually exclusive with count
  providers  = {                         # Provider aliases
    <PROVIDER> = <provider.alias>
  }
  depends_on = [<resource.address>]
  # Module input variables
  <input_name> = <value>
}

label (Required): A local name for the module block. Used to reference module outputs.

source (Required): Specifies where Terraform retrieves the module source code. Must be a literal string (no interpolation or expressions).

version (Optional): Specifies the module version constraint. Only valid for registry modules (public or private). Requires terraform init after modification.

count (Optional): Creates multiple identical instances of the same module. Each instance is indexed numerically. Mutually exclusive with for_each.

for_each (Optional): Creates multiple module instances based on: A map or A set of strings. Mutually exclusive with count.Preferred over count when identity matters.

providers (Optional): Overrides provider configurations inside the module. Required when using provider aliases. Note: The argument name is providers, not provider.

depends_on (Optional): Explicitly declares dependencies for the module. Terraform will complete all actions on the dependency before applying the module. Use sparingly — prefer implicit dependencies via inputs.

The Global Warehouse: Sourcing Code from Local Paths to the Registry

Terraform can load modules from multiple source types, allowing teams to balance reuse, governance, and flexibility.

1. Local Paths: The “Internal” Reference

Local paths are the easiest to start with. They refer to a directory on your local disk, usually within the same Git repository as your main configuration.

• Syntax: Always starts with ./ or ../.
• Best For: Development, testing, or modules that are highly specific to a single project and don’t need to be shared across the company.
• Pro-Tip: Avoid absolute paths (e.g., /Users/dev/module). They make your code non-portable, meaning it will break the moment a teammate or a CI/CD runner tries to use it.
• Example:

source = "./modules/network"
source = "../shared/rg"

2. The Terraform Registry: The “App Store” for Cloud

The Terraform Registry is a massive public library of modules maintained by HashiCorp, cloud providers (AWS, Azure, Google), and the community.

• Syntax: <NAMESPACE>/<NAME>/<PROVIDER>
• Official registry: registry.terraform.io
• Best For: Standard infrastructure like VPCs, EKS clusters, or S3 buckets where you don’t want to reinvent the wheel.
• Key Feature: Supports the version argument, allowing you to “pin” your infrastructure to a specific release (e.g., version = "5.0.0").

3. The Private Module Registry: Your Internal Service Catalog

While the public registry is great for generic tools, most companies have “secret sauce” or specific compliance requirements. This is where HCP Terraform (formerly Terraform Cloud) or Terraform Enterprise comes in.

Why go private?

• Curation: You only show modules that have been vetted by your security and DevOps teams.
• Semantic Versioning: Unlike Git tags which can be deleted or moved, a registry enforces strict versioning.
• Enhanced Documentation: The registry automatically generates documentation, input/output lists, and usage examples from your code.

The Source Syntax

Depending on where your registry is hosted, the source string changes slightly:

• HCP Terraform (SaaS): source = "app.terraform.io/<ORGANIZATION>/<NAME>/<PROVIDER>"
• Terraform Enterprise (Self-hosted): source = "<YOUR-HOSTNAME>/<ORGANIZATION>/<NAME>/<PROVIDER>"
• The “Local” Shortcut: source = "localterraform.com/<ORGANIZATION>/<NAME>/<PROVIDER>"
• (Note: Using _localterraform.com_ is a clever trick—it tells Terraform to look for the module on whichever platform is currently running the plan, making your code more portable between different Enterprise instances.)

4. Version Control Repositories: The DevOps Workhorse

Terraform can load modules directly from version control systems (such as Git). A module may reside at the root of the repository or within a subdirectory of the repository package.

• If the module is located in a subdirectory, use // in the source path to indicate the module’s location within the repository.
• Any query parameters (such as ref or depth) must be specified after the subdirectory path.
• Terraform downloads the entire repository to local disk, but evaluates only the specified module subdirectory.
• Because the full repository is available locally, modules in subdirectories can reference other modules in the same repository using relative local paths.

Authentication and Access

• Terraform installs VCS-based modules by running git clone.
• It relies on the Git configuration and credentials available on the local system.
• For private repositories, appropriate Git credentials must be configured in advance.
• When using SSH, Terraform automatically uses the configured SSH keys, making this approach ideal for CI/CD pipelines and other non-interactive environments.

Query Parameters for VCS Sources

Terraform supports additional query parameters to control how repositories are cloned:

• ref: Specifies the branch name, tag, or commit SHA to check out.
• depth:Performs a shallow clone and limits the commit history depth (default: 1).

Example: source = "git::https://github.com/org/repo.git?ref=v1.2.0&depth=1"

General Git Repository Sources (Private or Public)

To install a module from any Git repository, use the git:: prefix followed by a valid Git URL.

• SSH format: source = "git::ssh://[<user>@]<host>[:<port>]/<path-to-git-repo>"
• SCP-style SSH format: source = "git::[<USER>@]<HOST>/<PATH-TO-GIT-REPO>"
• HTTP / HTTPS / FTP / FTPS / Git protocol: source = "git::<protocol>://<host>[:<port>]/<path-to-git-repo>"

To wrap up your blog guide, these two sources cover the “Advanced” and “Enterprise” tiers of module management. They are perfect for organizations that want custom branding (Vanity URLs) or high-security, air-gapped storage (Cloud Buckets).

5. HTTP URLs: The “Vanity” Redirect

Think of HTTP sources as a URL shortener for your Terraform modules. Instead of forcing developers to remember a long, complex Git string, you provide a clean, internal URL.

How it Works

When you provide an HTTP/HTTPS URL, Terraform sends a GET request to that address. It expects the server to respond with a special header (X-Terraform-Get) that contains the actual source location (like a Git or S3 link).

• Syntax: source = "https://modules.acme.corp/vpc"
• The Benefit: If you ever move your code from GitHub to GitLab, you only have to update your redirect server. Your developers don’t have to change a single line of their Terraform code.
• Pro-Tip: This is how many “Vanity URLs” for open-source projects work behind the scenes.

6. Cloud Backends: S3 & GCS Buckets

For organizations that want to avoid Git entirely for production artifacts — or those operating in highly restricted cloud environments — storing modules as Versioned Archives in S3 or Google Cloud Storage is the way to go.

S3 Buckets (AWS)

Terraform uses your AWS credentials (environment variables or IAM roles) to pull these files.

• Syntax: source = "s3::https://s3-eu-west-1.amazonaws.com/my-terraform-modules/vpc.zip"
• Requirement: The source must be a .zip, .tar.gz, or .tgz archive.

GCS Buckets (Google Cloud)

Similar to S3, but uses Google’s storage API.

• Syntax: source = "gcs::https://www.googleapis.com/storage/v1/BUCKET_NAME/PATH_TO_MODULE"
• Why use this? It’s incredibly fast if your infrastructure is also running on GCP, and it allows you to use fine-grained IAM permissions to control who can “see” your infrastructure blueprints.

The Standard Pattern: Structuring Professional Terraform Modules

Terraform recommends a standard file and directory layout for reusable modules, especially those distributed via separate repositories. Terraform tooling understands this structure and uses it to:

• Generate documentation automatically
• Index modules in public and private registries
• Improve discoverability and usability

While the structure may appear extensive, only the root module is mandatory. Everything else is optional but strongly recommended.

Root Module (Required)

The root module is the primary entry point of the repository.

• Terraform configuration files (.tf) must exist at the root of the repository.
• This module should be opinionated, defining the intended and supported usage of the module.
• All consumers interact with the module through this root interface.

README

Both the root module and any nested modules should include a README or README.md file.

Recommended contents:

• Clear description of what the module does
• Intended use cases
• High-level design considerations

Best practices:

• Place usage examples in the examples/ directory rather than embedding them directly in the README
• Consider including an architecture diagram showing resources and relationships
• Inputs and outputs do not need to be manually documented — Terraform tooling can generate these automatically
• When linking to files or images within the repository, use commit-specific absolute URLs to prevent broken or incorrect references in future versions

LICENSE

A LICENSE file defines the terms under which the module can be used.

• Strongly recommended for all modules, including internal ones
• Required for public adoption in many organizations
• Can be open-source or proprietary

Recommended File Names

Terraform recommends the following filenames, even if some are initially empty:

• main.tf – Primary entry point; contains resource definitions and nested module calls
• variables.tf – Input variable declarations
• outputs.tf – Output value declarations

Guidelines:

• Simple modules may define all resources in main.tf
• Complex modules may split resources across multiple files
• Calls to nested modules should typically remain in main.tf

Variables and Outputs

• Every variable and output should include a clear description
• Descriptions should be one or two sentences, explaining purpose and usage
• These descriptions are consumed by documentation generators and registries

Nested Modules

Nested modules live under the modules/ directory.

Rules and conventions:

• Any nested module with a README.md is considered publicly consumable
• Nested modules without a README are treated as internal-only (advisory, not enforced)
• Use nested modules to decompose complex logic into small, reusable building blocks

Best practices:

• Root modules should reference nested modules using relative paths: source = "./modules/consul-cluster"
• Avoid deeply nested module chains
• Prefer composable modules that callers can assemble as needed

Examples

Usage examples should be placed under the examples/ directory at the repository root.

Guidelines:

• Each example may include its own README explaining intent and usage
• Examples for nested modules should also live in the root examples/ directory
• Since examples are often copied into other repositories:
• Module source values must use the external module address, not relative paths

Example Directory Layout

complete-module/
.
├── README.md
├── main.tf
├── variables.tf
├── outputs.tf
├── modules/
│   ├── nestedA/
│   │   ├── README.md
│   │   ├── main.tf
│   │   ├── variables.tf
│   │   └── outputs.tf
│   ├── nestedB/
│   └── ...
├── examples/
│   ├── exampleA/
│   │   └── main.tf
│   ├── exampleB/
│   └── ...

Nested Architecture: Composing Infrastructure with Child Modules

Introducing module blocks transforms a Terraform configuration from a flat list of resources into a hierarchical structure. Each module owns its own resources and may itself call child modules, which can quickly result in a deep and complex module tree.

Terraform strongly recommends keeping the module tree shallow, ideally with only one level of child modules, and modeling relationships between modules using expressions and outputs, rather than nesting modules inside one another.

Example: Flat Module Composition

module "network" {
  source = "./modules/aws-network"
  base_cidr_block = "10.0.0.0/8"
}

module "consul_cluster" {
  source = "./modules/aws-consul-cluster"
  vpc_id     = module.network.vpc_id
  subnet_ids = module.network.subnet_ids
}

This approach is called module composition. Instead of embedding dependencies inside modules, the root module assembles multiple small, composable building blocks to construct a complete system.

Why Module Composition Matters

With composition:

• Modules remain focused and reusable
• Dependencies are explicit and visible
• The same modules can be connected in different ways to produce different architectures

Rather than each module creating and managing its own dependencies, dependencies are passed in from the root module, which acts as the system integrator.

Dependency Inversion in Terraform Modules

Terraform module design benefits from applying the dependency inversion principle: Modules should depend on inputs, not on how those inputs are produced.

In the earlier example, the consul_cluster module does not know or care whether the VPC and subnets were:

• Created by another module
• Retrieved via data sources
• Provided by an entirely separate Terraform configuration

This decoupling improves:

• Reusability
• Testability
• Future refactoring flexibility

For example, a future refactor might replace the network module with data sources, without requiring any changes to the consul_cluster module interface.

Conditional Creation of Objects

Instead of designing modules that:

• Detect whether a resource exists
• Conditionally create or skip it internally

Terraform recommends dependency inversion again:

• Make the module accept the required object (ID, ARN, name, etc.) as an input variable
• Let the calling configuration decide whether and how that object is created

This keeps modules:

• Predictable
• Easier to reason about
• Free from environment-specific logic

Assumptions and Guarantees

Every Terraform module implicitly defines a contract with its consumers, made up of assumptions and guarantees.

Assumptions

Conditions that must be true for the module to function correctly.

Example:

• An AMI provided to an EC2 module must support the x86_64 architecture.

Guarantees

Properties or behaviors that consumers can safely rely on.

Example:

• An EC2 instance created by the module will have a private DNS record.

Terraform recommends explicitly validating assumptions and guarantees using:

• Input validation
• Preconditions and postconditions

This:

• Surfaces errors earlier
• Improves diagnostics
• Communicates design intent to future maintainers

Multi-Cloud Abstractions

Terraform intentionally avoids providing built-in abstractions across cloud providers. Attempting to unify different platforms often results in lowest-common-denominator designs that hide important provider-specific capabilities.

However, module composition enables teams to create their own lightweight multi-cloud abstractions, making deliberate trade-offs about:

• Which features matter
• Which differences are acceptable

This keeps abstraction intentional, rather than implicit.

Data-Only Modules

Not all modules need to manage infrastructure.

A data-only module:

• Contains no resource blocks
• Uses data sources to retrieve information about existing infrastructure
• Raises the level of abstraction by hiding how the data is obtained

Common Use Case

In systems split into multiple subsystem configurations, certain infrastructure (such as a shared network) may already exist. A data-only module (e.g., join-network-aws) can provide:

• Network IDs
• Subnet details
• Shared metadata

The benefits:

• Centralized logic for data retrieval
• Ability to change the data source later without updating all consumers
• Easier refactoring when paired with a similarly shaped “management” module

The Hand-off: Managing Provider Inheritance Across Modules

Every resource in a Terraform configuration must be associated with a provider configuration. Unlike most Terraform constructs, provider configurations are global to the entire configuration and can be shared across module boundaries.

Provider configurations can be defined only in the root module.

Child modules consume provider configurations but must not define them.

Provider Propagation Across Modules

Provider configurations can be made available to child modules in two ways:

1. Implicit inheritance (default behavior)
2. Explicit passing using the providers argument in a module block

Provider Blocks and Reusable Modules

A module that is intended to be reused must not contain provider blocks.

Why:

• Provider configurations must outlive the resources they manage
• Terraform v0.13 introduced for_each, count, and depends_on on modules
• Modules with embedded provider blocks are incompatible with these features

🚫 A module that defines its own provider configuration cannot be used with:

• for_each
• count
• depends_on

Terraform will raise an error if you attempt to combine them.

Provider Lifecycle Constraint (Important)

Provider configurations must remain present as long as any resource managed by them exists.

If Terraform detects a resource in state whose provider configuration no longer exists:

• terraform plan will fail
• Terraform will instruct you to reintroduce the missing provider configuration

This is why providers must be:

• Defined in the root module
• Removed only after all dependent resources are destroyed

Provider Version Constraints in Modules

Although provider configurations are global, each module must declare its own provider requirements.

Purpose:

• Ensure a single provider version satisfies all modules
• Declare the source address for the provider

Example:

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = ">= 3.0.0"
    }
  }
}

Best Practice for Shared Modules

• Use a minimum version constraint (>=)
• Specify the minimum version that provides required features
• Allow the root module to select newer versions if needed

✅ Good:

version = ">= 3.50.0"

❌ Avoid:

version = "~> 3.50.0"

Provider Aliases Within Modules

When a module needs to support multiple configurations of the same provider, it must declare acceptable aliases using configuration_aliases.

Example:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 2.7.0"
      configuration_aliases = [ aws.alternate ]
    }
  }
}

This allows resources in the module to reference:

provider = aws.alternate

Implicit Provider Inheritance

By default, child modules automatically inherit the default provider configurations from their parent module.

Implications:

• Provider blocks appear only in the root module
• Child modules can declare resources without referencing providers explicitly

✅ Recommended when:

• A single provider configuration is sufficient for the entire system

Passing Providers Explicitly

When a child module requires:

• A different provider configuration
• A provider alias
• Multiple configurations of the same provider

You must pass providers explicitly using the providers argument.

Example:

module "network" {
  source = "./modules/network"
  providers = {
    azurerm = azurerm.hub
  }
}

How the providers Map Works

• Keys → provider names expected by the child module
• Values → provider configurations defined in the current module

Important rules:

• Default providers are inherited unless overridden
• Aliased providers are never inherited automatically
• Aliases must always be passed explicitly

Legacy Modules with Embedded Provider Configurations

Before Terraform v0.13, modules often defined their own provider blocks to work around limitations.

Problems with this approach:

• Provider and resources were tightly coupled
• Removing a module also removed its provider configuration
• Violated provider lifecycle requirements

Terraform v0.13 retained limited backward compatibility:

• Legacy modules may still work
• But cannot be used with for_each, count, or depends_on

If different instances of a module require different provider configurations:

• You must use separate module blocks
• You cannot vary provider configurations per instance using count or for_each

Design Rules Summary (Standards)

• Providers are defined only in the root module
• Child modules must never define provider blocks
• Modules must declare required_providers
• Use >= version constraints in shared modules
• Prefer implicit inheritance for simplicity
• Use explicit provider passing for aliases or multiple configurations
• Never combine embedded providers with count, for_each, or depends_on

🌙 Late-Night Reflection

I used to think that ‘Modules’ were just for massive enterprises. I was wrong. Tonight I realized that Modules are for anyone who values their own time. By boxing up my VPC logic, I’m not just organizing code; I’m writing a letter to my future self saying, ‘I’ve already solved this problem, so you don’t have to.’ Good code is a gift you give yourself.

This final section of your blog is a lifesaver for anyone managing production infrastructure. In the past, refactoring Terraform code was a terrifying process of manual terraform state mv commands that were prone to human error.

The moved block changed the game by allowing you to record refactors directly in your code. Here is how to present this “State Management as Code” concept to your readers.

Refactoring Without the Fear: The moved Block

As your infrastructure evolves, you will eventually need to rename a resource, split a large module into two, or convert a single resource into a collection using for_each.

Normally, Terraform would see a name change and think: “Okay, delete the old resource and build a new one from scratch.” If that resource is a database or a critical network, that’s a disaster. The moved block prevents this by telling Terraform: “The resource is still there; it just has a new name.”

When Should You Use a moved Block?

• Renaming: Changing resource "aws_instance" "web_server" to "web".
• Module Splitting: Moving an S3 bucket from your main module into a specialized storage sub-module.
• Scaling: Moving from a single module call to a count or for_each loop.

The Syntax: Mapping the History

The syntax is simple but powerful. It lives in your .tf files (usually a moved.tf file to keep things tidy) and looks like this:

moved {
  from = aws_instance.old_name
  to   = aws_instance.new_name
}

When you run terraform plan, instead of seeing “1 to add, 1 to destroy,” you will see a much friendlier message: “1 to move.”

Why You Should Keep Your History (The “Upgrade Path”)

If you are writing modules for others (either in the public registry or a private company registry), do not delete your old moved blocks.

Think of them as a “migration script.” If a user hasn’t updated their infrastructure in six months, they need every moved block in the chain to get from their old version to your latest one safely.

Pro-Tip: Chaining Moves

If you rename something twice, chain the blocks together. Terraform will follow the trail from A $\rightarrow$ B $\rightarrow$ C automatically.

# The original move
moved {
  from = module.network.aws_vpc.main
  to   = module.vpc.aws_vpc.this
}

# A subsequent move during a later refactor
moved {
  from = module.vpc.aws_vpc.this
  to   = module.vpc.aws_vpc.primary
}

Summary Checklist for a Safe Refactor

• Draft the code changes: Rename your resources or modules in your .tf files.
• Add the moved block: Map the from (old address) to the to (new address).
• Run terraform plan:** Verify that Terraform shows “move” operations and no “destroy” operations for the resources you intended to keep.
• Apply and Commit: Once the apply is successful, commit both the code changes and the moved blocks to your repository.

🌙 Late-Night Reflection

In a world where everything is transparent, your secrets are the only things that can’t be recovered once they’re lost. Security isn’t a feature you add at the end; it’s the foundation you build on from the very first line of code. If you don’t respect the sensitivity of your data, the machine will eventually broadcast your mistakes to the world.

✅ Key Takeaways

• Modules are the unit of scale in Terraform Terraform is designed to be composed, not written as a monolith. Modules allow infrastructure to grow without becoming unmanageable.
• Every Terraform configuration has exactly one root module The directory where terraform init is run is the root module. All other modules are child modules, regardless of where they are sourced from.
• Child modules encapsulate reusable infrastructure logic Modules package related resources into a single, reusable unit with a defined interface of inputs and outputs.
• The module block is a declarative instantiation, not a copy-paste Modules are resolved during terraform init and executed as part of the dependency graph, just like resources.
• Registry modules support version constraints Always pin module versions using version to avoid breaking changes. Prefer explicit versions over floating references.
• Composition beats deep nesting Prefer flat module composition in the root module. Pass outputs from one module as inputs to another instead of nesting modules inside modules.
• Modules should depend on inputs, not on how inputs are created Dependency inversion improves reusability and refactoring safety. Modules should accept IDs, ARNs, and values — not create or discover them internally.
• Avoid conditional resource creation inside reusable modules Let the caller decide whether something exists. Modules should define how something is created, not if it should be.
• Data-only modules are valid and powerful Modules can contain only data sources. These raise abstraction by hiding how infrastructure is discovered while exposing consistent outputs.
• The moved block makes refactoring safe and declarative It records resource and module address changes in code, preserving state and preventing destructive operations.
• moved blocks are upgrade paths, not temporary fixes Keep old moved blocks to allow users to upgrade safely across versions. Terraform can follow chained moves automatically.
• State refactoring belongs in version control moved blocks turn state surgery into auditable, reviewable infrastructure code—eliminating risky CLI-only operations.

📚 Further Reading

• Modules overview documentation
• Use modules in your configuration documentation
• Standard module structure documentation
• Providers within modules documentation
• Manage values in modules documentation
• Best practices for composing modules

🎬 What’s Next

The factory is running — but how do we know what’s really been built? Without memory, automation is guesswork.

We’ll dive into how Terraform tracks reality and what happens when it drifts.