This issue tracks support for kubernetes v1.25.

So far these are the features/components that make ktf not fully support k8s v1.25:

• metallb addon using version v0.12.1 which uses PodSecurityPolicy which was removed after being deprecated v1.21

  Error: tests failed: failed to deploy addon metallb: resource mapping not found for name: "controller" namespace: "" from "https://raw.githubusercontent.com/metallb/metallb/v0.12.1/manifests/metallb.yaml": no matches for kind "PodSecurityPolicy" in version "policy/v1beta1"
  ensure CRDs are installed first
  resource mapping not found for name: "speaker" namespace: "" from "https://raw.githubusercontent.com/metallb/metallb/v0.12.1/manifests/metallb.yaml": no matches for kind "PodSecurityPolicy" in version "policy/v1beta1"
  

Problem: Fix issue https://github.com/Kong/kubernetes-ingress-controller/issues/1279 Solution: Bump Enterprise Postgres 2.5 using secret enterprise-license and superuser-password. W/ this image, workspace, rbac admin API is enabled, which provides a test-bed that KIC can interact using admin-token, non-admin-token, non-default workspace.

Bumps google.golang.org/api from 0.102.0 to 0.103.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Is there an existing issue for this?

• [X] I have searched the existing issues

Current Behavior

setup-kong-license silently fails. Possibly being a reason for failure of TestKongEnterprisePostgres test.

From the pipeline run https://github.com/Kong/kubernetes-testing-framework/actions/runs/3299432277/jobs/5454573296

echo "encrypted=$(gpg --symmetric --batch --passphrase  --output - <(echo ${KONG_LICENSE_DATA}) | base64 -w0)" >> $GITHUB_OUTPUT
  shell: /usr/bin/bash -e {0}
  env:
    KONG_LICENSE_DATA: {"license":{"payload":{"admin_seats":"5","customer":"automation","dataplanes":"100","license_creation_date":"[2](https://github.com/Kong/kubernetes-testing-framework/actions/runs/3299432277/jobs/5454573296#step:3:2)022-10-10","license_expiration_date":"2022-11-20","license_key":"***","product_subscription":"Konnect Enterprise","support_plan":"None"},"signature":"***","version":"1"}}
gpg: directory '/home/runner/.gnupg' created
gpg: keybox '/home/runner/.gnupg/pubring.kbx' created
gpg: error creating passphrase: Invalid passphrase
gpg: symmetric encryption of '/dev/fd/6[3](https://github.com/Kong/kubernetes-testing-framework/actions/runs/3299432277/jobs/5454573296#step:3:3)' failed: Invalid passphrase

Expected Behavior

Should work

Steps To Reproduce

Detected during pipeline run

Kong Kubernetes Testing Framework Version

No response

Kubernetes version

No response

Anything else?

No response

Is there an existing issue for this?

• [X] I have searched the existing issues

Problem Statement

When we deploy some addons or services, we need a tool to detect or wait for the service to be ready to serve. For example, when we deploy kuma addon in the cluster, we need to wait for kuma's webhook service to be ready before we create CRDs in kuma. So, we need a method to test whether a service could be accessed inside the testing kubernetes cluster.

Proposed Solution

• provide a solution that runs a Job with some images to test the TCP connection periodically, and complete the job when the TCP connection is created.

Additional information

No response

Acceptance Criteria

could call a method in KTF to test if a service/address could be accessed in the test cluster by:

• [x] (At least) TCP connection
• [ ] GET/POST to an HTTP URL
• [ ] TLS connection (with CA/client cert)
• [ ] GET/POST by an HTTPS URL (with a CA/client cert)

Back when we first made KTF we added a check to wait for the service account that would be used for testing until it's Secret was generated. Modern versions of Kubernetes since v1.24.0 have a different implementation for ServiceAccounts which doesn't make this secret available like it used to be. As such this patch removes the check as it is irrelevant going forward and is breaking recent deployments.

This PR removed the udp hack and instead uses udpProxy defined in our helm charts. Support for udpProxy in helm charts have been added by https://github.com/Kong/charts/commit/016ace388a56fe64ee9f3975b7507d289a9ebbd3

Fixes https://github.com/Kong/kubernetes-testing-framework/issues/392

Bumps github.com/docker/docker from 20.10.19+incompatible to 20.10.20+incompatible.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps github.com/kong/deck from 1.14.0 to 1.15.1.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Try to add a job to test connectivity to a TCP service inside a k8s cluster to solve #331. DONE in this PR:

• create a image retries to connect to a TCP address and push it to Kong repo
• KTF provide a method to wait for a TCP address to be connected by the image above

TODO:

• support more parameters (connect timeout/retry interval or retry times)
• discuss about other kinds of "tests", http/https/tls/udp ?

Problem Statement

When tests using KTF fail (particularly E2E tests in KIC) on conditions that aren't met currently the way to investigate this is to reproduce the problem locally and investigate the cluster there. This however is problematic when we get a transient failure or flake. The purpose of this issue is to enable the creation of diagnostic dumps for KTF clusters so that users like KIC can take and collect a dump before the cluster is cleaned up as a part of the CI run.

Additional information

Potentially taking a backup of the entire API with something like velero might be viable. I do remember in the past seeing tools that will let you host a dump of the API server and interrogate it with kubectl, though I've forgotten the name of that tool and can't find it. Suffice to say when we implement this we should be certain to check the open source ecosystem for existing tools.

Acceptance Criteria

• [x] when a KTF environment fails to create a diagnostic bundle is automatically generated
• [x] as a user of the go library I can trigger a diagnostic dump for the cluster which returns the file(s) to me to collect as an artifact in CI
• [x] diagnostic dumps should contain as much API state as possible
• [x] diagnostic dumps should contain pod logs

Is there an existing issue for this?

• [X] I have searched the existing issues

Problem Statement

As described in https://github.com/Kong/kubernetes-ingress-controller/issues/3326, gcloud limited number ofIP ranges in a subnet. To prevent failures caused by the limit of IP range, we should confirm that occupied ranges is released after the Cleanup of cluster is finished,

Proposed Solution

• Verify whether occupied IP ranges is released after successfully cleaned up the cluster in GKE
• If not guaranteed that occupied IP ranges is released after cleanup of cluster, we add poll and wait step in cluster cleanup after calling API to delete cluster.

Additional information

No response

Acceptance Criteria

• [ ] For GKE cluster, IP ranges (and possibly other resource with limitations) are released after clean up of cluester succeeded.

Problem statement

Whenever addons that utilize helm for installation (kong and kuma) they are prone to failure due to hosting services that host the helm charts.

Exemplar failure:

=== FAIL: test/integration  (0.00s)
INFO: setting up test environment
INFO: configuring cluster for testing environment
INFO: no existing cluster found, deploying using Kubernetes In Docker (KIND)
INFO: build a new KIND cluster with version 1.25.3
INFO: building test environment
WARNING: failure occurred, performing test cleanup
Error: tests failed: failed to deploy addon kong: Error: INSTALLATION FAILED: failed to fetch https://github.com/Kong/charts/releases/download/kong-2.13.1/kong-2.13.1.tgz : 503 Service Unavailable

https://github.com/Kong/kubernetes-ingress-controller/actions/runs/3691517677/jobs/6249752001#step:6:48

This causes the tests to fail.

Proposed solution

Introduce a (configurable?) retry mechanism which will retry whenever an error like this is received during helm chart installation.

Problem statement

Since we do not want to expose our secrets to outside world and make them available in CI workflows that run for outside contributions we need a mechanism/structure to adhere to, thanks to which we'll be able to separate enterprise and non-enterprise tests which in turn will allow us to make ee tests conditional (similarly to what has been done for go-kong: https://github.com/Kong/go-kong/pull/247)

Problem statement

Sometimes the framework can receive a 4xx or 5xx HTTP error code when requesting a file (like a manifest) in order to deploy/teardown an add-on. These in most cases are intermittent and should be retried since as a developer I do not want to have my full suite of tests to fail because Github returned a 503.

Example of a failed CI: https://github.com/Kong/kubernetes-testing-framework/actions/runs/3647089054/jobs/6158901046

=== RUN   TestKongWithNodePort
    kongaddon_test.go:243: configuring the testing environment, with the kong addon using NodePort service type for proxy
    kongaddon_test.go:248: building the testing environment and Kubernetes cluster
20[22](https://github.com/Kong/kubernetes-testing-framework/actions/runs/3647089054/jobs/6158901046#step:6:23)/12/08 09:48:52 well-defined vars that were never replaced: SERVICE_NAME,SERVICE_NAMESPACE
    kongaddon_test.go:250: 
        	Error Trace:	/home/runner/work/kubernetes-testing-framework/kubernetes-testing-framework/test/integration/kongaddon_test.go:[25](https://github.com/Kong/kubernetes-testing-framework/actions/runs/3647089054/jobs/6158901046#step:6:26)0
        	Error:      	Received unexpected error:
        	            	failed to deploy addon kong: Error: INSTALLATION FAILED: failed to fetch https://github.com/Kong/charts/releases/download/kong-2.13.1/kong-2.13.1.tgz : 503 Service Unavailable
        	            	: exit status 1
        	Test:       	TestKongWithNodePort
--- FAIL: TestKongWithNodePort (138.36s)

Bumps sigs.k8s.io/kustomize/kyaml from 0.13.9 to 0.13.10.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Release workflow uses an unsupported release action: actions/create-release@v1.

Because it uses the deprecated set-output command, we'll have to approach this sooner or later.

Exemplar CI failure: https://github.com/Kong/kubernetes-testing-framework/actions/runs/3628481705/jobs/6119549896#step:5:12