Kubernetes Operator to sync secrets between different secret backends and Kubernetes

Vals-Operator

Here at Digitalis we love vals, it's a tool we use daily to keep secrets stored securely. We also use secrets-manager on the Kubernetes deployment we manage. Inspired by these two wonderful tools we have created this operator.

vals-operator syncs secrets from any secrets store supported by vals into Kubernetes. It works very similarly to secrets-manager and the code is actually based on it. Where they differ is that it not just supports HashiCorp Vault but many other secrets stores.

Installation

You can use the helm chart to install vals-operator. You will need to provide the configuration to access the secrets store you decided on via either environment variables pre existing secrets.

# Example for Vault
helm upgrade --install vals-operator --create-namespace -n vals-operator \
  --set "env[0].name=VAULT_ROLE_ID,env[0].value="vals-operator"" \
  --set "env[1].name=VAULT_SECRET_ID,env[1].value="my-secret-id"" \
  --set "env[2].name=VAULT_ADDR,env[2].value=https://vault:8200"
  charts/vals-operator

# Example for AWS using a secret
kubectl create secret generic -n vals-operator aws-creds \
  --from-literal=AWS_ACCESS_KEY_ID=foo \
  --from-literal=AWS_SECRET_ACCESS_KEY=bar \
  --from-literal=AWS_DEFAULT_REGION=us-west-2

helm upgrade --install vals-operator --create-namespace -n vals-operator \
  --set "secretEnv[0].secretRef.name=aws-creds"  \
  charts/vals-operator

Usage

apiVersion: digitalis.io/v1
kind: ValsSecret
metadata:
  name: vals-secret-sample
  labels:
    owner: digitalis.io
spec:
  name: my-secret # Optional, default is the resource name
  ttl: 3600       # Optional, default is 0. The secret will be checked at every "reconcile period". See below.
  type: Opaque    # Default type, others supported
  data:
    username:
      ref: ref+vault://secret/database/username
      encoding: text
    password:
      ref: ref+vault://secret/database/password
      encoding: text
    ssh:
      ref: ref+vault://secret/database/ssh-private-key
      encoding: base64
    aws-user:
      ref: ref+awssecrets://kube/test#username
    aws-pass:
      ref: ref+awssecrets://kube/test#password

The example above will create a secret named my-secret and get the values from the different sources. The secret will be kept in sync against the backed secrets store.

The TTL is optional and used to decrease the number of times the operator calls the backend secrets store as some of them such as AWS Secrets Manager will incur a cost.

The default encoding is text but you can change it to base64 per secret reference. This way you can, for example, base64 encode large configuration files.

Options

The following options are available. See the helm chart documentation for more information on adding them to your deployment configuration.

  -exclude-namespaces string
    	Comma separated list of namespaces to ignore.
  -health-probe-bind-address string
    	The address the probe endpoint binds to. (default ":8081")
  -kubeconfig string
    	Paths to a kubeconfig. Only required if out-of-cluster.
  -leader-elect
    	Enable leader election for controller manager. Enabling this will ensure there is only one active controller manager.
  -metrics-bind-address string
    	The address the metric endpoint binds to. (default ":8080")
  -reconcile-period duration
    	How often the controller will re-queue secretdefinition events (default 5s)
  -record-changes
    	Records every time a secret has been updated. You can view them with kubectl describe (default true)
  -watch-namespaces string
    	Comma separated list of namespaces that vals-operator will watch.
  -zap-devel
    	Development Mode defaults(encoder=consoleEncoder,logLevel=Debug,stackTraceLevel=Warn). Production Mode defaults(encoder=jsonEncoder,logLevel=Info,stackTraceLevel=Error) (default true)
  -zap-encoder value
    	Zap log encoding (one of 'json' or 'console')
  -zap-log-level value
    	Zap Level to configure the verbosity of logging. Can be one of 'debug', 'info', 'error', or any integer value > 0 which corresponds to custom debug levels of increasing verbosity
  -zap-stacktrace-level value
    	Zap Level at and above which stacktraces are captured (one of 'info', 'error', 'panic').
Comments
  • Fix Helm chart kubeVersion comparison error

    Fix Helm chart kubeVersion comparison error

    Fixes this error when trying to install the chart:

    $ helm upgrade --install vals-operator ./vals-operator-0.5.0.tgz Release "vals-operator" does not exist. Installing it now. Error: chart requires kubeVersion: >= 1.19 which is incompatible with Kubernetes v1.21.5-eks-bc4871b

    The issue and solution is described here: https://github.com/helm/helm/issues/3810

    In addition, the duplicate kubeVersion field was removed.

  • Permit combing secrets with raw data

    Permit combing secrets with raw data

    For example,

    ---
    apiVersion: digitalis.io/v1
    kind: ValsSecret
    metadata:
      name: my-secret
    spec:
      ttl: 3600       # Optional, default is 0. The secret will be checked at every "reconcile period". See below.
      type: Opaque    # Default type, others supported
      data:
        username:
          ref: ref+vault://secret/username
          encoding: text
        password:
          ref: ref+vault://secret/password
          encoding: text
        server:
          plain: my-server.com
    
  • Don't log

    Don't log "updated" messages or create Events if the secret has not changed

    Currently whenever we check a secret we log the message "Secret created or updated" and we write an Event to K8s even if the secret did not change. We should stop logging that message (possibly replaced with a "secret has not changed" message) and only create the Event if the secret was changed.

  • get parameter: ResourceNotFoundException: Secrets Manager can't find the specified secret

    get parameter: ResourceNotFoundException: Secrets Manager can't find the specified secret

    Hi,

    I am testing vals-operator to see it can fit in our current setup. I was able to install the helm charts using the instructions. Created a secret to be used by helm charts.

    When I was trying to create using object kind: ValsSecret, get the above error, even though the secret exit. I have created a secret kube.test#username and kube.test#password in our region. FYI, my secret name in AWS secret manager is kube.test having two fields username and password.

    Default AWS region is same for helm secrets and the following secret.

    apiVersion: digitalis.io/v1
    kind: ValsSecret
    metadata:
      name: vals-secret-sample
      labels:
        owner: digitalis.io
    spec:
      name: my-secret # Optional, default is the resource name
      #ttl: 3600       # Optional, default is 0. The secret will be checked at every "reconcile period". See below.
      type: Opaque    # Default type, others supported
      data:
        aws-user:
          ref: ref+awssecrets://kube.test#username
        aws-pass:
          ref: ref+awssecrets://kube.test#password
    

    Logs from the vals-operator pod,

    1.6590061832090268e+09	ERROR	controllers.vals-operator	Failed to get secrets from secrets store	{"name": "vals-secret-sample", "error": "expand awssecrets://kube.test#username: get parameter: ResourceNotFoundException: Secrets Manager can't find the specified secret."}
    sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
    	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:121
    sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
    	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:320
    sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
    	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:273
    sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
    	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:234
    1.6590061832092552e+09	INFO	controllers.vals-operator	errorBackoff: 5.47083928s  (jitter=470.83928ms)
    1.6590061832097049e+09	DEBUG	events	Normal	{"object": {"kind":"ValsSecret","namespace":"default","name":"vals-secret-sample","uid":"74c0c393-67c5-4736-bd98-3dd0d031ab28","apiVersion":"digitalis.io/v1","resourceVersion":"100782694"}, "reason": "Failed", "message": "Failed to get secrets from secrets store expand awssecrets://kube.test#username: get parameter: ResourceNotFoundException: Secrets Manager can't find the specified secret."}
    

    Can someone please shed some light, I might be missing something here? Thanks

    Regards, Abdul

  • Template field missing in helmchart CRDs

    Template field missing in helmchart CRDs

    I tried using the templating for ValsSecret when I noticed that my CRDs which was installed with helm hasn't been updated. And after looking at the chart in this repo it seems that the CRD might not have been regenerated after templating was added. Would it be possible to regenerate the CRD for the helmchart, as I would very much like to try out the new templating feature.

  • Issue with digitalisdocker/vals-operator:v0.6.0

    Issue with digitalisdocker/vals-operator:v0.6.0

    Hi, trying install latest version with helm get an issue bellow:

    Failed to pull image "digitalisdocker/vals-operator:v0.6.0": rpc error: code = Unknown desc = Error response from daemon: manifest for digitalisdocker/vals-operator:v0.6.0 not found: manifest unknown: manifest unknown

  • feat: links vals-secret to created secret

    feat: links vals-secret to created secret

    This feature ensures the secrets created and managed by Vals-Operator have the ownerReferences attribute to ensure they can be tracked. Example:

      ownerReferences:
      - apiVersion: digitalis.io/v1
        blockOwnerDeletion: true
        controller: true
        kind: ValsSecret
        name: my-secret
    
  • feat: Support for templated secrets

    feat: Support for templated secrets

    Templates are not supported to allow users to craft for example complex configuration files:

      template:
        config.yaml: |
          # Config generated by Vals-Operator on {{ now | date "2006-01-02" }}
          username: {{.username}}
          password: {{.password}}
          {{- if .url }}
          url: {{ .url | lower }}
          {{ end }} 
    
  • feat: update core libraries to latest versions

    feat: update core libraries to latest versions

    Updates core libraries to latest versions fixing two High security alerts:

    Total: 2 (UNKNOWN: 0, LOW: 0, MEDIUM: 0, HIGH: 2, CRITICAL: 0)
    
    ┌──────────────────────────┬────────────────┬──────────┬───────────────────┬───────────────┬────────────────────────────────────────────────────────────┐
    │         Library          │ Vulnerability  │ Severity │ Installed Version │ Fixed Version │                           Title                            │
    ├──────────────────────────┼────────────────┼──────────┼───────────────────┼───────────────┼────────────────────────────────────────────────────────────┤
    │ github.com/gogo/protobuf │ CVE-2021-3121  │ HIGH     │ v1.3.1            │ 1.3.2         │ gogo/protobuf: plugin/unmarshal/unmarshal.go lacks certain │
    │                          │                │          │                   │               │ index validation                                           │
    │                          │                │          │                   │               │ https://avd.aquasec.com/nvd/cve-2021-3121                  │
    ├──────────────────────────┼────────────────┤          ├───────────────────┼───────────────┼────────────────────────────────────────────────────────────┤
    │ golang.org/x/text        │ CVE-2021-38561 │          │ v0.3.6            │ 0.3.7         │ golang: out-of-bounds read in golang.org/x/text/language   │
    │                          │                │          │                   │               │ leads to DoS                                               │
    │                          │                │          │                   │               │ https://avd.aquasec.com/nvd/cve-2021-38561                 │
    └──────────────────────────┴────────────────┴──────────┴───────────────────┴───────────────┴────────────────────────────────────────────────────────────┘
    
  • Bugfix: kubernetes login creates too many tokens

    Bugfix: kubernetes login creates too many tokens

    The environment variables VAULT_AUTH_METHOD conflicts with the underlying vals library and we were doing login all the time instead of login once and renewing the token when its about to expire.

  • Implement a better Vault token management

    Implement a better Vault token management

    Following this example, there is a better way of managing tokens:

    https://www.vaultproject.io/docs/concepts/auth#code-example

    Edit: reclassified as bug as it appears it creates new vault tokens all the time when using Kubernetes login.

  • TLS connection to vault server doesn't work

    TLS connection to vault server doesn't work

    Hi everyone,

    I've tried to configure vals-operator to use TLS authentication for connecting to Vault server but it doesn't seem to be working.

    Here is the piece of my values.yaml that I use to configure TLS:

    env:
      - name: VAULT_ROLE_ID
        value: vals-operator
      - name: VAULT_ADDR
        value: https://vault.vault:8200
      - name: VAULT_CACERT
        value: /vault/userconfig/vault-server-tls/vault.ca
      - name: VAULT_CLIENT_CERT
        value: /vault/userconfig/vault-server-tls/vault.crt
      - name: VAULT_CLIENT_KEY
        value: /vault/userconfig/vault-server-tls/vault.key
      - name: VAULT_TLS_SERVER_NAME
        value: vault
    
    volumes:
      - name: vault-server-tls
        secret:
          secretName: vault-server-tls
          defaultMode: 420
    volumeMounts:
      - name: vault-server-tls
        mountPath: /vault/userconfig/vault-server-tls
        readOnly: true
    

    Connection without TLS works fine if I set VAULT_SKIP_VERIFY to true. I've also tested TLS connection manually using the same certificates stored in vault-server-tls secret and it works properly.

    Unfortunately I didn't find a way to properly troubleshoot it because vals-operator is using distroless Docker image with no capabilities for using shell.

    Error that I get in vals-operator logs is the following:

    ERROR   vault   unable to authenticate to Vault {"error": "unable to login to kubernetes auth method: unable to log in to auth method: unable to log in with Kubernetes auth: Put \"https://vault.vault:8200/v1/auth/kubernetes/login\": x509: certificate signed by unknown authority"}
    

    I would very appreciate your assistance with resolving this issue.

    Thanks.

    Best regards, Roman Timoshevskii.

  • Allow a secret to exist on multiple namespaces

    Allow a secret to exist on multiple namespaces

    The change would allow the same secret to being synced to multiple namespaces. This can be very useful for secrets such as private docker registries. The new schema could look like the example below:

    ---
    apiVersion: digitalis.io/v1
    kind: ValsSecret
    metadata:
      name: private-registry
    spec:
      type: kubernetes.io/dockerconfigjson
      namespaces:
        - one
        - two
        - three
      data:
        auth:
          ref: "ref+vault://secret/registry/dockerconfigjson"
          encoding: text
    
Basic Kubernetes operator that have multiple versions in CRD. This operator can be used to experiment and understand Operator/CRD behaviors.

add-operator Basic Kubernetes operator that have multiple versions in CRD. This operator can be used to experiment and understand Operator/CRD behavio

Dec 15, 2021
A k8s vault webhook is a Kubernetes webhook that can inject secrets into Kubernetes resources by connecting to multiple secret managers
A k8s vault webhook is a Kubernetes webhook that can inject secrets into Kubernetes resources by connecting to multiple secret managers

k8s-vault-webhook is a Kubernetes admission webhook which listen for the events related to Kubernetes resources for injecting secret directly from sec

Oct 15, 2022
An operator which complements grafana-operator for custom features which are not feasible to be merged into core operator

Grafana Complementary Operator A grafana which complements grafana-operator for custom features which are not feasible to be merged into core operator

Aug 16, 2022
A CLI to sync configmaps and secrets in a kubernetes cluster

kube-sync Kube Sync is a CLI application to copy/sync configmaps and secrets from one namespace to another. Motivation While working with kubernetes,

Oct 15, 2022
Secret - Prevent your secrets from leaking into logs and std*

secret - Prevent your secrets from leaking into logs and std* The package provid

Dec 30, 2022
This repo contains example on how to consume secrets from Google Secret Manager from GKE

GKE Secret Manager. Environment setup This repo contains examples of how to consume secrets from Google Secret Manager (GSM) from Google Kubernetes En

Dec 5, 2022
To copy a secret to another namespace and sync it up-to-date

Secret Mirror Operator This kubebuilder-based Kubernetes operator copies a Secret to another namespace and synchronizes it with the custom resource Se

Jul 20, 2022
A component for sync services between Nacos and Kubernetes.

简介 该项目用于同步Kubernetes和Nacos之间的服务信息。 目前该项目仅支持 Kubernetes Service -> Nacos Service 的同步 TODO 增加高性能zap的logger 增加 Nacos Service -> Kubernetes Service 的同步 监听

May 16, 2022
Kubernetes Operator Samples using Go, the Operator SDK and OLM
Kubernetes Operator Samples using Go, the Operator SDK and OLM

Kubernetes Operator Patterns and Best Practises This project contains Kubernetes operator samples that demonstrate best practices how to develop opera

Nov 24, 2022
The Elastalert Operator is an implementation of a Kubernetes Operator, to easily integrate elastalert with gitops.

Elastalert Operator for Kubernetes The Elastalert Operator is an implementation of a Kubernetes Operator. Getting started Firstly, learn How to use el

Jun 28, 2022
Minecraft-operator - A Kubernetes operator for Minecraft Java Edition servers

Minecraft Operator A Kubernetes operator for dedicated servers of the video game

Dec 15, 2022
K8s-network-config-operator - Kubernetes network config operator to push network config to switches

Kubernetes Network operator Will add more to the readme later :D Operations The

May 16, 2022
Pulumi-k8s-operator-example - OpenGitOps Compliant Pulumi Kubernetes Operator Example

Pulumi GitOps Example OpenGitOps Compliant Pulumi Kubernetes Operator Example Pr

May 6, 2022
grafana-sync Keep your grafana dashboards in sync.

grafana-sync Keep your grafana dashboards in sync. Table of Contents grafana-sync Table of Contents Installing Getting Started Pull Save all dashboard

Dec 14, 2022
Help developer to sync between local file and remote apollo portal web since portal web is so messy to use

apollo-synchronizer Help developer to sync between local file and remote apollo portal web since portal web is so messy to use Features download names

Oct 27, 2022
Operator Permissions Advisor is a CLI tool that will take a catalog image and statically parse it to determine what permissions an Operator will request of OLM during an install

Operator Permissions Advisor is a CLI tool that will take a catalog image and statically parse it to determine what permissions an Operator will request of OLM during an install. The permissions are aggregated from the following sources:

Apr 22, 2022
Test Operator using operator-sdk 1.15

test-operator Test Operator using operator-sdk 1.15 operator-sdk init --domain rbt.com --repo github.com/ravitri/test-operator Writing kustomize manif

Dec 28, 2021
a k8s operator 、operator-sdk

helloworld-operator a k8s operator 、operator-sdk Operator 参考 https://jicki.cn/kubernetes-operator/ https://learnku.com/articles/60683 https://opensour

Jan 27, 2022