How to Use Terraform’s ‘for_each’, with Examples
Terraform is one of the most popular infrastructure as code (IaC) tools. With Terraform, you can write code that defines the infrastructure components you want and the configuration for them. You then execute that code, and Terraform will make sure that your infrastructure is set up the way you defined it. This means either creating new components or responding with “All of these are already created.”
Sounds easy, right?
In reality, Terraform code can be quite complicated. That’s especially likely when you want to do some advanced stuff, like iterating over a certain number of resources.
For example, let’s say you want to create multiple virtual machines with the same configuration. For cases like that, in Terraform you can use for_each. In this post, you’ll learn what that is as well as how and when to use it.
Infrastructure as Code with Terraform
If you’re new to Terraform, before we move on, you need to understand what it actually is.
Modern environments are quite complex, and you have a few options when you want to add, change or destroy some components of your infrastructure. If you’re on the cloud, you can go to your cloud provider web UI and execute any necessary action from there. You can also use CLI or even write some scripts yourself and call your cloud provider API.
There are, however, some limitations to all these options. Clicking in the web UI doesn’t scale. And you don’t want to create dozens of different services every time you need a new environment.
Using CLI or writing your own scripts is a step forward. But why not take two steps forward instead and use a dedicated infrastructure-as-code tool?
One of the biggest advantages of using Terraform is that it will keep the state of your infrastructure saved.
Terraform is one such tool. You write Terraform-specific code defining how you want your infrastructure to look. Then you execute Terraform and everything is taken care of for you. It’s a highly efficient and scalable way of creating infrastructure.
Also, one of the biggest advantages of using Terraform is that it will keep the state of your infrastructure saved. Therefore, it will always try to have your infrastructure in sync. So once you execute Terraform, it will only create, change or destroy resources that aren’t in sync with the saved state.
Terraform Meta-Arguments
Before we dive into explaining how for_each works, let’s briefly talk about what it actually is.
In Terraform, we mainly talk about resource blocks and module blocks. For example, when you want to create a virtual machine, you need to define a resource block with configuration details of that machine. Within the resource and module block, you can also use one of the five so-called meta-arguments. These are special instructions that aren’t part of the resource configuration per se, but they instruct Terraform to do some action in relation to that resource. And one of these instructions is for_each.
As I already mentioned, the main purpose of the for_each meta-argument is to create multiple instances of a resource. So, as you can imagine, it’s quite useful to know.
It’s also worth mentioning that for_each has been added to Terraform in version 0.12. But I hope you’ve already upgraded to Terraform 1.x anyway.
Multiple Resources
To understand better what purpose for_each serves, let’s see how you could achieve the same outcome in Terraform without using for_each.
The outcome we’re talking about is deploying multiple copies of the same resource. So, let’s take virtual machines, for example.
Normally, to deploy more than one virtual machine, you’d have to specify multiple resource blocks, like this:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
resource "google_compute_instance" "vm1" { name = "vm1" machine_type = "e2-small" zone = "us-central1-a" (...) } resource "google_compute_instance" "vm2" { name = "vm2" machine_type = "e2-medium" zone = "us-central1-a" (...) } resource "google_compute_instance" "vm3" { name = "vm3" machine_type = "f1-micro" zone = "us-central1-a" (...) } |
Seems like a lot of duplicated code, doesn’t it? That’s exactly where for_each can help.
Instead of duplicating all that code for each virtual machine, you can define your resource once and provide a map or a set of strings to iterate over.
Take a look at the example. This is how achieving the same results as above would look with for_each:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
resource "google_compute_instance" "vm" { for_each = { "vm1" = "e2-small" "vm2" = "e2-medium" "vm3" = "f1-micro" } name = each.key machine_type = each.value zone = "us-central1-a" (...) } |
As you can see, we defined the configuration parameters that differ per virtual machine as key-value pairs in the for_each block and left the parameters that are the same for each VM in the resource block. Then, we accessed the key-value pair by special keywords each.key and each.value.
What if you want to pass more than just two (key and value) parameters? For example, what if you want to also parameterize the zone in the above example? You can simply change the value to a map, as follows:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
resource "google_compute_instance" "vm" { for_each = { "vm1" = { vm_size = "e2-small", zone = "us-central1-a" } "vm2" = { vm_size = "e2-medium", zone = "us-central1-b" } "vm3" = { vm_size = "f1-micro", zone = "us-central1-c" } } name = each.key machine_type = each.value.vm_size zone = each.value.zone (...) } |
You can pass as many parameters in the value as you want. Then in the actual resource configuration, you can reference them with each.value.<parameter_key>.
To keep your code clean and have the ability to reuse values for different resources, you can even extract the actual parameters into a variable:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
locals { virtual_machines = { "vm1" = { vm_size = "e2-small", zone = "us-central1-a" }, "vm2" = { vm_size = "e2-medium", zone = "us-central1-b" }, "vm3" = { vm_size = "f1-micro", zone = "us-central1-c" } } } resource "google_compute_instance" "vm" { for_each = local.virtual_machines name = each.key machine_type = each.value.vm_size zone = each.value.zone (...) } |
‘for_each’ Versus ‘count’
If you’re not new to Terraform, you may have used another meta-argument that seems like the same thing: count. And while count also lets you create multiple instances of the same resource, there’s a difference between count and for_each. The latter isn’t sensitive to changes in the order of resources.
A common issue with count is that once you delete any resource other than the last one, Terraform will try to force replace all resources that the index doesn’t match.
You don’t have that problem with for_each because it uses the key of a map as an index. You can’t use both count and for_each on the same resources, but why would you anyway?
Are there any drawbacks to for_each? Yes.
Limitations of ‘for_each’
While for_each is pretty straightforward to use, there are some limitations you should be aware of. First of all, the keys in your for_each map block must have a known value. Therefore, for example, they can’t be generated on the fly by functions (like bcrypt or timestamp). They also can’t refer to resource-specific attributes that are provided by a cloud provider, like a cloud resource ID. Another limitation is the fact that you can’t use sensitive values as arguments for for_each. Basically, when using for_each, you need to directly specify the values.
Using ‘for_each’ is relatively easy, but you need a solid understanding of how it works to get the most benefits from it.
Summing Up and Learning More
for_each is probably one of the most commonly used Terraform meta-arguments. Modern environments usually consist of multiple instances of resources for high-availability and scalability reasons. Using for_each is relatively easy, but you need a solid understanding of how it works to get the most benefits from it. It also has its own limitations.
In this article, you learned how for_each works and got some tips on how to use it efficiently. Now, you can try to play around with it yourself or look into other meta-arguments.
Terraform at Scale
ReleaseHub’s environment as a service is an easy-to-use, highly scalable service that leverages Terraform to create snapshots of even the most complex environments and automatically manages their creation and teardown as part of your development lifecycle.
