Infrastructure as Configuration (IAC) is a massive boon for anyone who remembers the frustration of manually configuring servers, forgetting to install a crucial library, and spending weekends resolving trivial issues. Docker brought us a long way in standardizing our execution environments, but it was never meant to be the full solution. We still need a way to provision and manage resources such as Docker orchestration, network and DNS configuration, SSL certificates, and more. This is where Terraform IAC setup comes in. Tools like Terraform allow us to define the ideal state of our cloud resources through configuration files, while Terraform ensures the current state matches this desired configuration.

In this blog, I’ll share my experience migrating an application from Azure to Google Cloud Platform (GCP) and developing a new app natively on GCP. Our goal was to embrace Terraform as much as possible and avoid the inefficiencies of ClickOps (aka manually configuring resources through a cloud provider’s console). Along the way, one of the biggest challenges I faced was configuring Identity Aware Proxy (IAP) on GCP—a process that took longer than expected. I’ll walk you through what IAP is, how it fits into the larger cloud infrastructure picture, and the configuration hurdles I encountered, along with the solution that ultimately worked.

IAP – Identity Aware Proxy

Prior to coming to GCP, I had been on Azure, and used MSAL extensively. This allowed me to set up an app registration, perform an OAuth (OpenIDConnect) flow with the end user, and receive an ID token of the authenticated user. This flow was built into my application, and simply hosted on Docker containers in Azure.

GCP allows for a slightly different model. Rather than enforcing authentication, and to some extent authorization, at the app level, it can be done at a higher level than the application. An application is usually composed of multiple Docker containers, one that is responsible for the UI, and another (or a service mesh) that is responsible for the API. These containers need to be fronted with some load balancing layer that also provides a consistent facade for hostname/port/protocol to avoid CORS round trips. A typical application may look like this:

On GCP, the Application Load Balancer allows for IAP to be attached to it, allowing for an OAuth flow as well as enforcing any authorization requirements. In my case, the only users in the domain that should be able to access the application are ones that belong to a particular AD group. Rather than having to enforce that within the application code itself, it can be done through the IAP:

IAP Terraform Configuration

One of the first pieces to getting IAP setup on GCP is to have a “brand”, which is roughly equivalent to an Azure App Registration. It’s the “name” of the application, which scopes are needed by the application that the user needs to give permission to, and eventually the domain that end users will expect to have the application used within.

My first attempt to set up the IAP brand did not go according to plan. Although I read through the documentation, I wasn’t able to get my IAP brand to be automatically created. I verified my TF service account had the appropriate IAM roles, but still nothing. My configuration looked something like this:

resource "google_iap_brand" "my-app" {
  support_email     = "[email protected]"
  application_title = "My APP"
  project           = var.project_id

And then I got this wonderful error:

Error: Error creating Brand: googleapi: Error 400: Request contains an invalid argument.
with google_iap_brand.my-app
on 005-application_load_balancer.tf line 102, in resource "google_iap_brand" "my-app":

No information about what I did wrong, about what I could do to fix it, nothing. Note that my project_id variable was the name of my project, because that will come in later.

Just as I figured, this was a limitation of TF GCP, so I decided to create the brand myself. I had to give myself OAuth editor permission:

resource "google_project_iam_member" "oauthconfig-editor" { 
  project = var.project_id 
  member  = "user:[email protected]"
  role    = "roles/oauthconfig.editor"

But even after that was created, I had to figure out a way to import it into the Terraform state.

The Fix – GCP APIs To The Rescue

Whenever I tried to import the IAP brand into the TF state, like this:

import {
  to = google_iap_brand.my-app
  id = "projects/1234567890/brands/my-app"

I kept getting the same error:

Error: Error when reading or editing IapBrand "projects/1234567890/brands/my-app": googleapi: Error 400: Unable to parse project number and brand. Use following format: projects/{ProjectNumber|ProjectId}/brands/{brand}

No matter what combination I tried, I kept getting the same error. I had to then go back to those docs that I was referencing earlier to call the GCP API to get a list of IAP brands. From that I saw the following:

{
  "brands": [
    {
      "name": "projects/1234567890/brands/1234567890",
      "supportEmail": "[email protected]",
      "applicationTitle": "my-app",
      "orgInternalOnly": true
    }
  ]
}

Therefore, I was able to change my import to be:

import {
  to = google_iap_brand.my-app
  id = "projects/1234567890/brands/1234567890"

And it worked. Nowhere on the IAP Brand UI was there any indication that was how to uniquely identify the brand I had created.

After doing a Terraform plan, there was one more error that I was getting:

Error: Error creating Brand: googleapi: Error 409: Requested entity already exists
with google_iap_brand.my-app
on 005-application_load_balancer.tf line 172, in resource "google_iap_brand" "my-app":

I had to do a little connecting the dots, but I noticed that in my google_iap_brand configuration, I was using the project id (e.g. “my-project”) rather than the project number (e.g. 1234567890). After changing the configuration to use the number, I got this:

resource "google_iap_brand" "my-app" {
  support_email     = "[email protected]"
  application_title = "My APP"
  project           = var.project_number

And presto blamo, everything worked. I was able to have an IAP brand that actually worked, imported into my Terraform state, so I could proceed with creating the IAP.

Possible Solution

One of my colleagues pointed out after the fact, that I may have been able to get this to work by using the same email address as the service account my Terraform Cloud for Business was using to manage my GCP resources for the email address on the iap_brand. I don’t think this is an ideal solution, as it’s not a true “support” email address. Sure, it’s an email address, but it doesn’t have an actual email inbox to receive support requests.

The API on the GCP side also doesn’t support deletes for this resource type. So there are some definite limitations on both the Terraform as well as GCP side for this resource.

Conclusion

Although I wasn’t able to get everything into IAC, having one or two ClickOps steps seems to be a necessary evil. This setup allows me to automate almost everything around the scaffolding of my environment, and makes it consistent as I promote changes up the pipeline. There are still some growing pains, such as certain places it’s called “region” or “location” when they are the same thing, or when I need to use the project number rather than project id. Overall though, it’s been a positive experience, and I’m excited to continue using Terraform on GCP.

For more insights and practical tips on cloud technologies, I encourage you to explore the Keyhole Software blog, where we share more content like this to help you tackle your next project with confidence.