How to Manage a Home Network with Infrastructure as Code
I decided to take advantage of some downtime I had during the holiday break to overhaul my home network. As I began my project, I bought a Unifi Dream Machine home management device because, in addition to my personal and guest SSIDs, there is an apartment in my house for which I wanted to segment traffic. I also wanted to add an extra layer of security around some of the home automation and IoT devices that were being added to our home network with a fourth SSID.
I started to configure the new network, I had started a spreadsheet of VLANs, subnet CIDRs and mappings of those to SSIDs. Additionally, I needed to track firewall rules, port forwards and other settings and configurations. Needless to say, this was a lot of information to maintain and manage.
My day job is working on the Infrastructure-as Code (IaC) product HashiCorp Terraform, an open source tool for automating and codifying infrastructure provisioning. Syncing a spreadsheet of data to a web UI by clicking around in my browser just felt tedious and error-prone — I was bound to make some mistakes. There had to be a better way, so I decided that this was an area where the automation and versioning features of Terraform could help.
My Search for Terraform Integrations
My first step was just to see what solutions were out there. Others have to have this problem, right? How do they solve it? Maybe there already was a Terraform provider out there!
My first search for “terraform unifi” brought a few potentially promising results. A module for spinning up a controller on Google cloud and a repository called “terraform-provider-unifi.” Here we go! Unfortunately, once I clicked through, it looked like the repository only had README and LICENSE files, no code yet, so someone else had the same idea but hadn’t made much progress yet.
If no Terraform provider existed, maybe at least there was a Go SDK, which would provide a way for me to talk to the API in the Go programming language. Go is the language Terraform providers are typically implemented in and the one I have been primarily using since joining HashiCorp, I thought. I did find a few on GitHub, but they were entirely read-only, while others seemed to just not have many create, update or delete methods or were otherwise lacking in functionality.
There were SDKs in other languages, namely a PHP one that seemed to be the most popular. The Ubiquiti Community Wiki also has some documentation on the Unifi controller API, but without links to existing SDK packages, so it looked like I was going to have to write an SDK myself.
Writing an SDK
While investigating the existing SDKs, I did find some information that would be useful for writing my own. Specifically, I found a precedent for the authentication flow, but in order to find how the create, update and delete methods worked, I needed to start collecting examples via the development tools in my browser.
I started with three of the Unifi data types I wanted to manage in the provider:
- Wireless networks (referred to as WLANs in the API).
- Networks (used to implement VLANs).
- Clients (referred to as users in the API.)
Once I grabbed the example list, get, create, update and delete methods, I was able to find the structure of the JSON messages, the HTTP methods and the URL patterns for the requests. With those pieces of data, I could write the SDK in Go using the standard library’s net/http package.
However, I soon realized doing this manually for every API endpoint would be extremely tedious. In a previous job, I had experimented with extracting message-type information from a .jar file and a Java .class file and then generating Go structs for the JSON serialization. Since the Unifi controller is written in Java, I was curious if this strategy would work.
Unfortunately, the .jar file is obfuscated and I have not found a way to get the information I need from its .class files. However, inside the .jar file there were .json files (here is an example for WLANs) that had all the fields listed for every endpoint. I could thus use that to at least help ensure my SDK implementation had all the data elements the server could accept.
Writing the Provider
Now that I had an SDK, it was time to write (and test) the provider. A Terraform provider is the integration between Terraform and another library or application, most commonly this is a cloud API like AWS, Azure, or Google Cloud, but it can be written for any API (even for ordering pizza). The code was relatively straightforward — I work on Terraform in my role at HashiCorp and have created multiple integrations in the past (like the New Relic one).
I started with the template repository that HashiCorp maintains. In addition to the template, there is a large section of the Terraform docs site dedicated to writing custom Terraform providers. We also have some videos and other resources on our site that talk through the process.
I typically start by creating Terraform config files that are examples of how I’d like to interact with the API and then from there create the resources to support those files, but there are many ways to approach this. In some cases, where we are granularly exposing an upstream API, we start from the API definitions and try to translate those as faithfully as possible.
Testing the Provider
Once I had a provider, I needed to be able to test it end to end. In the beginning, I just tested it live on my own network controller. However, this had a number of downsides:
- Provisioning to some equipment on my own network caused the Access Points to reset their radios, which caused my laptop to disconnect from WiFi, interrupting the tests and forcing me to switch to hard-wired.
- If you counted above, you’ll notice I have four SSIDs and at the time I thought the max per controller was four (I later found out this is the max per WLAN group), but for a while, I just deleted one and made sure my test concurrency didn’t go above one.
- It was tested in production, what could go wrong?
Testing against my own network also had some other downsides that we experience in other Terraform providers, for example, it would make it very difficult to run tests against pull requests or difficult for contributors to run their own tests. I dug around a little and found some docker containers with the controller inside that I could swap to and after setting up a GitHub action, I now get end to end testing on all my code on GitHub.
Writing the Configuration for My Network
After I had a functional, tested provider, it was relatively straightforward to describe my VLANs, SSIDs, fixed IPs and other networking configuration options in Terraform. This is what it looks like to create a VLAN and map it to an SSID for your Unifi controller:
This was a good opportunity to take advantage of several new features released in last year’s Terraform 0.12 version. There were two specifically that I wanted to try out: csvdecode and resource for _each. When mapping MAC address to notes and fixed IPs, I thought this would be much easier to describe in a CSV:
To map this into the unifi_client resource, the code would look like:
You can see I use locals to simplify the CSV decoding and convert the data to map with the MAC address as the key. This makes iterating with a for_each more straightforward as each item has a unique key to identify its resource.
Importing the Existing Infrastructure
Now that I had written all the config that described my network, I didn’t want to just create it all again from scratch. I instead wanted to test it against my existing infrastructure to make sure I captured everything necessary. When I implemented the provider, I also implemented the terraform import functionality for each resource if you specify the ID from Unifi. This gets a little tricky though as the IDs are sometimes not apparent in the web UI. Occasionally you can grab them from a URL. For example in this URL:
https://unifi.ubnt.com/5.12.22.0/unifi/site/default/v2/settings/wifi/manage/edit/700cc880b9d3a0b50aab0636
The ID is the last path segment that looks like a 24-digit hexadecimal number. If they are not available in the URL, you have to grab them directly from the API requests in dev tools.
What’s Next?
So what’s next for the provider and my home network? One thing I’d like to do is to add support for WebRTC to the provider. This is how the Unifi web portal is able to communicate with your local controller without opening a port in your firewall. It may not be possible, but it’s something I’d like to investigate. Additionally, I’d like to add full coverage to the provider to manage all the pieces of the Unifi controller via Terraform. Of course, I’m also going to need to buy a lot more Unifi networking gear to test it all.
