Today we have no mechanism to tie components to collections and so are enforcing one collection per cluster.

It will be a common requirement to support multiple collections so we'll need to add something like the collectionRef to the components:

apiVersion: ingress.acme.com/v1alpha1
kind: Contour
metadata:
  name: contour-sample
spec:
  EnvoyImage: nginx:1.17
  namespace: ingress-system
  contourReplicas: 3
  contourImage: nginx:1.17
  collectionRef:
    kind: CloudNativePlatform
    apiVersion: platforms.acme.com/v1alpha2
    name: cloudnativeplatform-sample

Consider the following custom resource. When this resource is created, the controller would create two CronJobs to run at the scheduled time and scale the sample-deploy Deployment to the given number of replicas:

apiVersion: autoscaling.containers.myorg.com/v1alpha1
kind: PodScalingSchedule
metadata:
  name: sample
  namespace: pss
spec:
  targetReference:
    name: sample-deploy
    kind: Deployment
    apiVersion: apps/v1
  scaleOperations:
  - schedule: "54 12 * * *"
    replicas: "2"
  - schedule: "55 12 * * *"
    replicas: "5"

So we will need to create a CronJob resource for each object in spec.scaleOperations array.

We could possibly use a source manifest with a marker that indicates that field name scaleOperations needs to be an array. See example of proposed marker on line one:

# +operator-builder:field:name=scaleOperations,contains=array
apiVersion: batch/v1
kind: CronJob
metadata:
  name: sample-scaling-schedule  # +operator-builder:field:name=targetReference.name,type=string
spec:
  schedule: "27 16 * * *"  # +operator-builder:field:name=scaleOperations.schedule,type=string
  jobTemplate:
    spec:
      template:
        spec:
          serviceAccount: sample-scaling-schedule  # +operator-builder:field:name=targetReference.name,type=string,replace="sample"
          containers:
          - name: sample-scaling-schedule  # +operator-builder:field:name=targetReference.name,type=string,replace="sample"
            image: lander2k2/deploy-update:0.1.0
            imagePullPolicy: IfNotPresent
            command:
            - bash
            args:
            - /update-deployment.sh
            - deployment  # +operator-builder:field:name=targetReference.kind,type=string
            - curfew-sample  # +operator-builder:field:name=targetReference.name,type=string
            - "2"  # +operator-builder:field:name=scaleOperations.replicas,type=string

And then we would need to generate CreateFunc that returns an array of client.Object and looks something like this:

// CreateCronJobTargetReferenceName creates the CronJob resources.
func CreateCronJobTargetReferenceName(
	parent *autoscalingv1alpha1.PodScalingSchedule,
	transferLabels map[string]string,
	updateImage string,
) ([]client.Object, error) {
	resourceObjects := []client.Object{}

	for n, scaleOp := range parent.Spec.ScaleOperations {
		var resourceObj = &unstructured.Unstructured{
			Object: map[string]interface{}{
				"apiVersion": "batch/v1beta1",
				"kind":       "CronJob",
				"metadata": map[string]interface{}{
					"name":   "" + parent.Spec.TargetReference.Name + "-scaling-schedule-" + strconv.Itoa(n),
					"labels": transferLabels,
				},
				"spec": map[string]interface{}{
					"schedule": scaleOp.Schedule,
					"jobTemplate": map[string]interface{}{
						"metadata": map[string]interface{}{
							"labels": transferLabels,
						},
						"spec": map[string]interface{}{
							"template": map[string]interface{}{
								"metadata": map[string]interface{}{
									"labels": transferLabels,
								},
								"spec": map[string]interface{}{
									"restartPolicy":  "OnFailure",
									"serviceAccount": "" + parent.Spec.TargetReference.Name + "-scaling-schedule",
									"containers": []interface{}{
										map[string]interface{}{
											"name":            "" + parent.Spec.TargetReference.Name + "-scaling-schedule",
											"image":           updateImage,
											"imagePullPolicy": "IfNotPresent",
											"command": []interface{}{
												"bash",
											},
											"args": []interface{}{
												"/update-replicas.sh",
												strings.ToLower(parent.Spec.TargetReference.Kind),
												parent.Spec.TargetReference.Name,
												scaleOp.Replicas,
											},
										},
									},
								},
							},
						},
					},
				},
			},
		}

		resourceObj.SetNamespace(parent.Namespace)

		resourceObjects = append(resourceObjects, resourceObj)
	}

	return resourceObjects, nil
}

It appears that namespaced resources (any resources that are set to clusterScoped: false) are not properly inheriting their namespace from the parent resource.

We are overriding the namespace for child resources when the parent has its namespace set at: https://github.com/vmware-tanzu-labs/operator-builder/blob/main/internal/plugins/workload/v1/scaffolds/templates/api/resources/definition.go#L77

However, we are seeing errors in the logs:

2021-11-07T08:54:02.263Z        ERROR   controller.webstore     Reconciler error        {"reconciler group": "apps.acme.com", "reconciler kind": "WebStore", "name": "webstore-sample", "namespace": "test-create-parent", "error": "cross-namespace owner references are disallowed, owner's namespace test-create-parent, obj's namespace default"}

To duplicate:

kubectl create ns test
kubectl apply -f config/samples/* -n test

You can also see this error on the resource which you've created:

kubectl get <myresourcetype> -n test -o yaml

...
status:
  conditions:
  - lastModified: 2021-11-07 09:02:46.729276 +0000 UTC
    message: Successfully Completed Phase
    phase: DependencyPhase
    state: Complete
  - lastModified: 2021-11-07 09:02:46.7349354 +0000 UTC
    message: Successfully Completed Phase
    phase: PreFlightPhase
    state: Complete
  - lastModified: 2021-11-07 09:02:46.7438233 +0000 UTC
    message: Failed Phase with Error; cross-namespace owner references are disallowed,
      owner's namespace test-create-parent, obj's namespace default
    phase: CreateResourcesPhase
    state: Failed
  resources:
  - condition:
      created: false
      lastModified: 2021-11-07 09:02:46.7395297 +0000 UTC
      lastResourcePhase: WaitForResourcePhase

Main.go is not currently updated (overwritten) with changes specified however, we are asking to overwrite the file at https://github.com/vmware-tanzu-labs/operator-builder/blob/main/internal/plugins/workload/v1/scaffolds/templates/main.go#L60.

Add the following stub-phases to controller reconciliation:

Controller Phases:

• PostFlight - the very last thing that runs

Resource Phases:

• PreCreateResources - the ResourcePhase that runs before doing anything else
• PostCreateResources - the last ResourcePhase that runs before doing anything else

This will address the internal conversation that we had regarding "how do I make my controller more extensible".

The generated config/samples come with no metadata.namespace. It would be nice to supply a default one when using CRDs that are able to be namespaced. That way they don't just get created in the default k8s namespace.

When initializing a project with operator-builder, the domain spec field should allow the operator-builder command to be run without the --domain flag. However, it fails with a missing go.mod.

Spec:

name: cloud-native-platform
kind: WorkloadCollection
spec:
  domain: acme.com
  apiGroup: platforms
  apiVersion: v1alpha1
  apiKind: CloudNativePlatform
  clusterScoped: true
  companionCliRootcmd:
    name: cnpctl
    description: Manage platform stuff like a boss
  componentFiles:
  - ns-operator-component.yaml
  - contour-component.yaml

Error:

TEST_PATH=/tmp/test TEST_SCRIPT=platform.sh make test
go build -o bin/operator-builder cmd/main.go
cp bin/operator-builder /usr/local/bin/operator-builder
mkdir /tmp/test/.test
cp test/platform.sh /tmp/test/.test/
(cd /tmp/test; ./.test/platform.sh)
Error: failed to initialize project: unable to inject the configuration to "base.go.kubebuilder.io/v3": error finding current repository: could not determine repository path from module data, package data, or by initializing a module: go: cannot determine module path for source directory /tmp/test (outside GOPATH, module path must be specified)

Example usage:
        'go mod init example.com/m' to initialize a v0 or v1 module
        'go mod init example.com/m/v2' to initialize a v2 module

Run 'go help mod init' for more information.

Only way around that is to manually initialize the go.mod.

The following is changed with this PR:

• Fixes #234 which fixes a bug in the companion CLI which will not allow a file to be both overwritten and updated at the same time.
• Move logic from the above into the apis/ folder
• Refactor API scaffolding to be more readable (from 360 lines of code down to 281 - also allows for lots of unnecessary logic to be removed from scaffold files as well)

This PR addresses the following:

• Restructures the companion CLI to use the same folder structure as that of the companion CLI command structure to make it easier to follow
• Moves the subcommands within their own group folder, much like the controller folder to avoid collisions with the same kind existing across multiple groups
• Adds in the version subcommand to allow for users to list versions of their components as well as their own CLI version
• Adds in the ability to generate manifests for multiple different api versions
• Various refactors for inefficiencies

Currently if we create two different instances of a standalone workload in a single cluster, we get the following error:

2021-11-11T14:44:52.861-0500    INFO    controllers.apps.WebStore       expected only 1 resource of kind: [WebStore]; found 2

I think the standalone workloads accidentally inherited this limitation from the workload collections

Given this tree:

operator-builder/
├── resource3.yaml
└── resources
    ├── resource1.yaml
    └── resource2.yaml

And this configruation:

name: webstore
kind: StandaloneWorkload
spec:
  api:
    domain: acme.com
    group: apps
    version: v1alpha1
    kind: WebStore
    clusterScoped: false
  companionCliRootcmd:
    name: webstorectl
    description: Manage webstore stuff like a boss
  resources:
  - operator-builder/resources
  - operator-builder/resource3.yaml

We would consume the following files:

operator-builder/resources/resource1.yaml
operator-builder/resources/resource2.yaml
operator-builder/resource3.yaml

When using the replace function of a marker, with the missing replace string, the values become hard-coded.

Given the following markers:

    # +operator-builder:field:name=google.region,type=string,replace=GOOGLE_LOCATION,default=us-east1
    # +operator-builder:field:name=clusterName,type=string,replace=GOOGLE_KMS_KEY_RING,default=cluster-key-ring
    # +operator-builder:field:name=clusterName,type=string,replace=GOOGLE_KMS_KEY,default=cluster-key
    # +operator-builder:collection:field:name=google.project,type=string,replace=GOOGLE_PROJECT,default=my-project
    keyName: projects/my-project/locations/us-east1/keyRings/cluster-key-ring/cryptoKeys/cluster-key
    state: ENCRYPTED

Would result in:

    keyName: "projects/my-project/locations/us-east1/keyRings/cluster-key-ring/cryptoKeys/cluster-key",
    state: "ENCRYPTED"

We should error out and let the user know that they have made a mistake with their markers.

Consider the following marker:

  # +operator-builder:collection:field:name=google.projectNumber,type=int,default=12345,replace=PROJECT_NUMBER,description=`
  # +kubebuilder:validation:Required
  # Project number for the google.project that is being used.`
  member: serviceAccount:[email protected]

Results in the following error:

/home/scottd018/VSCode/github/project/operator/bin/controller-gen "crd:preserveUnknownFields=false,crdVersions=v1,trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
/home/scottd018/VSCode/github/project/operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
go fmt ./...
go vet ./...
# github.com/project/operator/apis/infra/v1alpha1/cluster
apis/infra/v1alpha1/cluster/google_kms_key.go:141:41: invalid operation: "serviceAccount:service-" + collection.Spec.Google.ProjectNumber (mismatched types untyped string and int)
make: *** [Makefile:50: vet] Error 2

Situation:

• I have a working operator with a collection/component configuration
• I want to add a standalone workload to this operator
• I run the operator-builder create api command to add the standalone workload
• Because the CLI is initially scaffolded during init, it errors out
• Making the companionCliRootcmd match the existing command with work, but the pathing is wonky (mycmd generate generate because a standalone workload expects a root command and not a subcommand)

We should be able to add a StandaloneWorkload to an existing operator without requiring that it exists on its own.

Not all resources are core Kubernetes resources, and because we manage dependencies, we need to know when certain conditions of a resource are met. Because of this, we should allow users to specify ready conditions for their resources. A good example of this is within the operator-builder project itself using the status.ready = True concept.

Loose requirements are:

1. This is a marker for each resource.
2. Might look something like this:

+operator-builder:resource:readyCondition:field=status.ready,condition=true

Add the ability to specify resources inline with the operator-builder workload configuration. This obviously is not to encourage someone to type out an entire deployment in their workload config, rather it would be useful for something like a Namespace resource where one may need to be created, but it is not desirable to manage a separate file for that resource. That would change the spec for resources to:

resources:
  - inline: |
      apiVersion: v1
      kind: Namespace
      metadata:
        name: my-namespace # +operator-builder:field:name=myField,default=my-namespace,type=string
  - file: path/to/another/resource.yaml

We should also maintain backwards compatibility and assume that the absence of a file or inline key would default to it being a file. E.g. this would still be valid:

resources:
  - path/to/legacy/resource.yaml