I like the concept of terraform operator. I had the following questions:

• How can terrafrom modules pickup its configuration values from Consul when used via the operator way?

Similar to the example defined here

https://github.com/ned1313/terraform-tuesdays/tree/017f5bd976f91e33513ed2c885866d6f654a29f1/2020-06-02-ConsulDataSource

• Also i am using vault and terraform. How can i use a vault token in the crd to contact vault via terraform ?

Looking into the possiblity of PoC'ing this terraform operator I noticed that there's no way to load additional terraform context beyond the module.

I have a situation where there are modules available for use but these require references to existing AWS shared resources loaded via terraform datasources.

Example:

data "aws_sns_topic" "alerts" {
  name = "slack-feed-product_alerts"
}

data "aws_security_group" "rds_default" {
  tags = {
    Name = "rds-shared-security-group"
  }
}

module "db" {
  source         = "[email protected]/terraform-modules.git//aws/postgres"

  vpc_security_group_ids              = [data.aws_security_group.rds_default.id]
  alerts_sns_topic_arn                = data.aws_sns_topic.alerts.arn
}

Describe the solution you'd like Provide an alterative field to inject custom terraform hcl code to be able to able to overcome such situations. Not sure if there's a better way to handle this as these data sources are specific to the resources, but an hcl field of sorts would allow us to cater to it.

Describe alternatives you've considered Looking at the source code for the operator, we can probably use the providersConfig field, as a workaround, to inject any terraform configuration and it would likely work (as it parses directly into the template), but it is not intuitive and can be a bit confusing given the name.

Additional context Add any other context or screenshots about the feature request here.

Terraform objects own resources like (ConfigMap, and Secrets). With each run, a new secret/configmap will be created. A proper cleanup needs to be in place

Maybe consider keeping only the last run/workflow

Role binding is not created if the service account already exist

To Reproduce Create a Terraform object in non default namespace. There is no error, there is a condition check that results in this bug and suppresses the error. Code is here

Expected behavior A role binding and a service account should be created if both don't exist

Versions

• Operator: 0.1.1
• Runner: 0.4.0

When no backend is specified, Terraform operator will add the Kubernetes as a backend by default. However, the namespace field is missing from the configuration

To Reproduce When a Terraform object is created in any other namespace than the default, the terraform-runner will try to list secrets from the default namespace due to the missing namespace config, which will cause the runner pod to fail with RBAC permissions to list secrets from the default namespace

Expected behavior Backend secret should be created in the namespace where the CR was created in

Versions

• Operator: 0.1.2
• Runner: 0.4.0

Given Go 1.18 has been out for some time now, and seems to be stable expect for some minor issues with the performance of the newly introduced Generics, we should start moving to it.

It would be nice to control the requeue intervals for job watch and dependency

Implementation We can pass these values as flags when running the operator as flag.DurationVar with default values

When a Terraform kind is being deleted, the following line is logged in the controller

{"level":"info","ts":1657826774.588966,"logger":"controllers.TerraformController","msg":"Terraform run is being deleted"}

Expected behavior It's not clear from the logs which object is being deleted, the name of the object needs to be added to the log line

Versions

• Operator: 0.1.1

Proper testing for error handling on client sets needs to be added.

We can leverage the Fake library which is already being used. Here is an example for simulating a failure in creating a service account

import (
  fakecorev1 "k8s.io/client-go/kubernetes/typed/core/v1/fake"
)

....

kube.ClientSet.CoreV1().(*fakecorev1.FakeCoreV1).PrependReactor("create", "serviceaccounts", func(action testing.Action) (handled bool, ret runtime.Object, err error) {
  return true, &v1.ServiceAccount{}, errors.New("Error creating service account")
})

Unit tests needs to be adjusted to create additional workflows with error

• Remove terraform client implementing the Kube interface and rely on the reconciler to query objects for dependency
• Lose the suffix Spec in go Struct definitions
• Move pkg/kube & pkg/utils to /internal

If the pod fails to mount the secret based on a variable that have dependencyRef on a key that does not exist, it will cause the pod to be in CreateContainerConfigError state with error couldn't find key number in Secret ....

To Reproduce The following will produce the issue

apiVersion: run.terraform-operator.io/v1alpha1
kind: Terraform
metadata:
  name: terraform-run1
spec:
  terraformVersion: 1.1.7

  module:
    source: IbraheemAlSaady/test/module
    version: 0.0.3

  variables:
    - key: length
      value: "4"

  outputs:
    - key: result
      moduleOutputName: result
---
apiVersion: run.terraform-operator.io/v1alpha1
kind: Terraform
metadata:
  name: terraform-run2
spec:
  terraformVersion: 1.1.7

  module:
    source: IbraheemAlSaady/test/module
    version: 0.0.3

  dependsOn:
    - name: terraform-run1

  variables:
    - key: length
      dependencyRef:
        name: terraform-run1
        key: invalid-key

Expected behavior If the pod hit an error, we need to update the Terraform status to Failed

Current evaluations are only happening on the job, the job is not aware that pod is in that state. We probably need to validate the pod status as well

Possibly we need to do the pod evaluation when the job is in a running state

Versions

• Operator: 0.1.1
• Runner: 0.4.0

The terraform runner Dockerfile does not have user set. Check here

The reason for that decision came when support for private SSH keys was introduced as there were issues adding the SSH key with the ssh-agent command. (Ref)

We need to investigate how to make this work while the container is running in a non-root user

Versions

• Runner: 0.4.0

When the terraform operator was created with Kubebuilder, it was already created with Ginkgo V1.

Ginkgo V2 has been out for some time. There is an open issue here on Kubebuilder's repo to upgrade to V2 #2532

The terraform runner pod is logging the following

time="2022-07-14T19:19:56Z" level=error msg="failed to list files in the var files path" error="open /tmp/tfvars: no such file or directory"

To Reproduce Apply any Terraform kind

Expected behavior Check if the directory exist, and log a warn message

Versions

• Operator: 0.1.1
• Runner: 0.4.0

Describe the solution you'd like A way to schedule Terraform workflows and run either plan or apply

Additional context The configuration could look something like this

apiVersion: run.terraform-operator.io/v1alpha1
kind: ScheduledWorkflow
metadata:
  name: terraform-aws-s3
spec:
  schedule: 0 * * * *
  withApply: true
  terraformRef:
    name: terraform-aws-s3

By default, it will schedule a run to execute a plan, we can specify the withApply flag to run Terraform apply as well