cert-manager webhook & CoreDNS plugin

This repo exists for a niche case scenario in which we are running cert-manager on one or multiple Kubernetes clusters to issue/manage certificates but also running our own authoritative DNS using CoreDNS and need to get DNS01 challenges solved from it.

Contents:

1. Overview of the repository
2. Certificates & mTLS
   • Minica
3. CoreDNS Plugin
   • Pre-built Docker image
   • Build CoreDNS with the plugin
4. cert-manager's webhook
   • Add this repo
   • Manually install webhook
5. Limitations
6. Contributing
7. Resources

Overview

architecture - monorepo - not typical go structure

go modules

ci - releases/packages

Certificates

Authentication/encryption between webhook and CoreDNS is achieved via mTLS.

For this purpose we need to 3 things:

1. Certificate Authority (CA) key pair PEM encoded

2. Server certificate PEM encoded (signed by CA) and key

3. Client certificate PEM encoded (signed by CA) and key

There are many ways to achieve this. For this guide we are using Minica to keep the whole process as pure Go as possible.

Note: the rest of the certificate section exist as demonstration, for production use you probably want a much more secure key strategy (rolling keys etc).

Minica

The simplest way acquiring Minica is:

go install github.com/jsha/[email protected]

But you can also directly download a binary from project's latest releases.

Now generate server and client cert/key pairs:

minica -domains server,localhost
minica -domains client

Note: replace localhost with your actual FQDN (ie. example.org) when not developing locally.

The above commands should generate 6 files. A key pair on top level named minica.pem & minica-key.pem, and the equivalent client & server key pairs.

Transfer cert files to DNS machines

We need to copy to all our DNS servers a. the CA certificate and b. server's cert/key pair.

scp minica.pem yourusername@yourserver:/home/yourusername/CAcerts/
scp -r server yourusername@yourserver:/home/yourusername/server

Create Kubernetes secrets

We also need to create 2 Kubernetes tls secrets, one with the CA key pair and a second with client's key pair.

kubectl create secret tls dns01-ca --cert=minica.pem --key=minica-key.pem
kubectl create secret tls dns01-client --cert=client/cert.pem --key=client/key.pem

Note: create the above secrets in the same namespace cert-manager lives in.

CoreDNS

plugin

Corefile example configuration:

.:1053 {
        reload
        selfhosted 4771 /path/to/ca/folder /path/to/server/cert.pem /path/to/server/key.pem
        records {
            @   60  IN SOA ns.example-acme-webook.example.org ns.example-acme-webook.example.org. 20 5 5 5 5
            @   60  IN NS ns.example-acme-webook.example.org.
            @   60  IN A  127.0.0.1
        }
}

Security note: a second gRPC server will start at the port number given + 1. So make sure the port is available and behind firewall as it is meant for internal communication only. In the future I plan to add authentication for internal communications but until then you have to secure this port manually from the outside world.

Docker

docker run -it --rm --volume "$(pwd):/conf" -p 1053:53 -p 4771:4771 delete:latest -conf /conf/corefile

Build

You may opt to build CoreDNS. For complete instructions read the official documentation first.

To add the plugin you have to:

cd path/to/coredns/plugin

ln -s path/to/this/repo/selfhosted .

cd ..
nano plugin.cfg # add selfhosted:selfhosted

selfhosted:github.com/bh90210/selfhosted/selfhosted
go get github.com/bh90210/selfhosted

go generate
go build

webhook

intro

Repo

helm repo add selfhosted-webhook https://bh90210.github.io/selfhosted
helm repo update
# Replace the groupName value with your desired domain
helm install --namespace cert-manager selfhosted-webhook selfhosted-webhook/selfhosted-webhook --set groupName=acme.example.org

helm uninstall --namespace cert-manager selfhosted-webhook
helm repo remove selfhosted-webhook

Manual

helm install --namespace cert-manager selfhosted-webhook deploy/selfhosted-webhook

configuration

Limitations

Due to the fact that each time you make use of the selfhosted plugin in Corefile’s blocks a new gRPC server is spawned my solution will not scale well for all scenarios. If you use multiple blocks to organise/separate zones it will add unnecessary overhead. I tend to use a few blocks (3-4) and group together many zones on each block. While this works very good for me, given my hardware, use-case and design choices, it might be a no-go solution for you. If you have an idea to fix this limitation consider opening a ticket to discuss about it.

Contributing

All contributions small or big are welcome. Just open an issue to discuss or directly open a PR against master branch.

Resources

https://codeburst.io/mutual-tls-authentication-mtls-de-mystified-11fa2a52e9cf

https://dev.to/techschoolguru/how-to-secure-grpc-connection-with-ssl-tls-in-go-4ph

https://coredns.io/2016/12/19/writing-plugins-for-coredns

https://coredns.io/manual/setups

https://cert-manager.io/docs/configuration/acme/dns01/#webhook