
## About Terraform with GCP

Terraform provide infrastructure automation to provision resources to many cloud provider, GCP is one of them, this article shows how to use Terraform to deploy a simple Big Query Schedule Job to the GCP Cloud.

## Preparation

## Create terraform files

### Create `provider.tf` file

Here start with the terraform provider, google credentials and the GCP project.

```
terraform {
  required_providers {
    google = {
      source = "hashicorp/google"
      version = "5.15.0"
    }
  }
}

provider "google" {
  credentials = file("{this is option but your can create GCP service account with BigQuery.job.create role}")
  project = "{your gcp project id}"
}
```

### Create service account

For the above credentials, below are few steps to create a service account has minimal role to perform below GCP operation.

- go to the GCP Roles page, create a new custom role with `bigquery.job.create` permission.
- go to the IAM service account page, create new service account and then assign the previous custom role.
- once the service account created, go to the KEYS tab and click `ADD KEY` to create new JSON key and save locally so above credentials can lookup

### Create `backend.tf` file

Here, we're using the GCP cloud bucket to store the terraform state file, this is similar to share the code within the team so we can deploy same infrastructure no matter who runs it.

```
terraform {
 backend "gcs" {
   bucket  = "{create a GCP cloud bucket}"
   prefix = "test"
 }
}
```

### Create `schedule-job.tf`

Now, finally here come to the main code we're trying to write a simple query and deploy to GCP BQ Schedule job and runs every sunday.

```
resource "google_bigquery_data_transfer_config" "job" {
    display_name = "test-query"
    project = "{your gcp project}"
    location = "us"
    data_source_id = "scheduled_query"
    schedule = "first sunday of quarter 00:00"
    params = {
      query = "select 1 as ID;"
    }
}
```

### Start Deployment

First, go to the folder has above terraform files, run this command to initialize terraform from gcp cloud bucket to download state files.

```
terraform init
```

This command will start a new workspace with `test`, if not exist then will save the state to the cloud bucket.

```
terraform workspace new test
```

Terraform command to run the plan, to check what resources will generate to the GCP.

```
terraform plan
```

If you confirmed the resource, run this command to deploy the changes to the GCP.

```
terraform apply
```

## Conclusion

Use the terraform is great to deploy infrastructure resource to the GCP for productions.


This article demonstrate how to terraform to deploy GCP Schedule JOB

How to Create GCP Schedule Job with Terraform


### Switch to the correct workspace environments

For example, we can run below command to create qa workspace environment.

```
terraform workspace new qa
or
terraform workspace select qa
```

### pull latest state file

Run this command to get latest state file for the environment from storage.

```
terraform state pull
```

This is what I get when I run the `terraform state pull`, basically nothing saved to the current workspace state.

```
{
  "version": 4,
  "terraform_version": "1.4.6",
  "serial": 1,
  "lineage": "59eb4435-e702-29bf-71de-1231231",
  "outputs": {},
  "resources": [],
  "check_results": null
}

```

### test by run the `terraform plan`

if at this point run the `terraform plan`, you'll see some changes if you have active resources at local file, which is wrong if the same resource already exist at the GCP, and you don't want to run `terraform apply` which will end up create duplicate resources or throw errors.

### run import to update state file

For instance, I have current create GCP BQ schedule query, you can run below command to sync the state file.

```
terraform import google_bigquery_data_transfer_config.job projects/1233/locations/us/transferConfigs/12312312
```

Above command should update the state file, and if you run again with `terraform plan` you should not see any changes.


This blog shows how to run terraform to create state file from the existing resources.

How to Create Terraform State file for current Resources


### Sample Terraform Code

This blog is using below terraform code to get started.

**_provider.tf_**

```
terraform {
  required_providers {
    google = {
      source = "hashicorp/google"
      version = "5.15.0"
    }
  }
}

provider "google" {
  # Configuration options
  credentials = file("your google credential file path")
  project = "{gcp project id}"
}
```

**_backend.tf_**

```
terraform {
 backend "gcs" {
   bucket  = "{gcp bucket for terraform}"
   prefix = "test"
 }
}
```

**_schedule-job.tf_**

```
resource "google_bigquery_data_transfer_config" "job" {
    display_name = "my_test_sj"
    project = "{your project id}"
    location = "us"
    data_source_id = "scheduled_query"
    schedule = "first sunday of quarter 00:00"
    params = {
      query = "call spTest();"
    }
}
```

### Run the sample Terraform

Run the above sample terraform should create a schedule job.

### Use Variable

Now, let's say we want to move the **_display_name_** to the variable, because we want to re-use the same name later.

Let's create a new **\*variable.tf**, and if you run `terraform plan`, you can see the display name as **_my_test_sj_**.

#### variable with default

```
variable "bq_name" {
  type = string
  nullable = false
  default = "my_test_sj"
}
```

#### with out default variable

let's say we want to enter the name every time when we run the terraform, we can remove the default so every time run the **_terraform plan_** if will ask the input for the variable. The example here might not be idea, but there might be some area can be use this.

```
variable "bq_name" {
  type = string
  nullable = false
}
```

### different value for each environments

Assume we have 2 different workspace **_dev_** and **_qa_**.

For this case, we define the Variables as below.

##### variable.tf

```
variable "bq_name" {
  type = string
  nullable = false
}
```

##### dev.tfvars

create a **_dev.tfvars_** and add the below key values.

```
name = "dev_job"
```

##### qa.tfvars

create a **_qa.tfvars_** and add the below key values.

```
name = "qa_job"
```

So far at this point, nothing need to modify at the schedule job terraform file, but we'll run the **_terraform plan_** with this option **_--var-file=dev.tfvars_**, this will run the terraform with the key/value environments files defined at **_dev.tfvars_**.

```
terraform plan --var-file=dev.tfvars
```

### Conclusion

Use the terraform **_workspace_** with **_var-file_** can write the terraform for different environments.


This blog shows how to use Terraform variable for move environment variables out of the code, so you can run the terraform for different environment and inject the variables for each environments.

Category Terraform

How to Create GCP Schedule Job with Terraform

How to Create Terraform State file for current Resources

How to run Terraform with Input Variable