cluster-machine-approver
This controller provides the following functionality:
- Watch the CSR endpoint for CSR requests
- Decide if the CSR should be allowed or denied
- Approve or deny and update CSR status
Introduction
Kubernetes includes support for TLS bootstrapping for Nodes, which OpenShift makes use of.
Kubelet needs two certificates for its normal operation:
- Client certificate - for securely communicating with the Kubernetes API server
- Server certificate - for use in its own local https endpoint, used by the API server to talk back to kubelet
When a new host is provisioned, kubelet will start and communicates to the CSR (Certificate Signing Request) API endpoint to request signed client and server certificates. It issues this request using bootstrap credentials that it finds on its local filesystem.
At this point, these CSRs must be approved. They can be manually approved through the API using kubectl, or kube-controller-manager can be configured to approve them. Alternatively, some custom component could be built to approve CSRs through the API, which is what OpenShift has done.
OpenShift and CSRs
OpenShift includes a custom component to approve CSRs: the cluster-machine-approver. The cluster-machine-approver is used to automatically approve CSRs, but with more strict criteria than what was supported in kube-controller-manager.
Note that the cluster-machine-approver only kicks in post-install. During the cluster bootstrapping phase, the approve-csr service on the bootstrap node automatically approves all CSRs. This bootstrap service will end up approving the CSRs for the control plane nodes, while cluster-machine-approver will take over for future new CSRs from worker nodes.
Disabling Node Client CSR Approvals
It is possible to disable node client CSR approvals completely. This is done using a ConfigMap resource, as shown in this PR comment.
apiVersion: v1
kind: ConfigMap
metadata:
name: machine-approver-config
namespace: machine-approver
data:
config.yaml: |-
nodeClientCert:
disabled: true
This may be useful if you explicitly want to only allow manual CSR approvals for new nodes.
Node Client CSR Approval Workflow
CSR approval details can be found in csr_check.go. Assuming this has not been disabled, the following criteria must be met for the client CSR to be approved:
- The user must be the node bootstrapper
- The username in the CSR must be
system:serviceaccount:openshift-machine-config-operator:node-bootstrapper - The groups in the CSR must be
system:serviceaccounts:openshift-machine-config-operator,system:serviceaccounts, andsystem:authenticated.
- The username in the CSR must be
- A
Nodeobject must not yet exist for the node that created the CSR. - The
MachineAPI is used to do a sanity check. AMachinemust exist with aNodeInternalDNSaddress in itsStatusthat matches the future name of theNode, as found in the CSR. - This
Machinemust not have aNodeRefset. - The CSR creation timestamp must be close to the
Machinecreation timestamp (currently within 2 hours) - The CSR is for node client auth.
Node Server CSR Approval Workflow
Details of this workflow can be found in the same file as the client workflow, csr_check.go.
For this workflow, it is assumed that the Node is now up and running, and the Node object exists in the API. Validation for the server CSR is different than the client case and is based primarily on matching addresses between associated Node and Machine objects.
First, there must be a Machine object with a NodeRef field set to the Node that sent this CSR. The NodeRef is set by a Node controller under the machine-api-operator.
Once a Node-Machine pair has been identified, validation is done on all of the Addresses in the Status field of the Machine. The CSR requests a certificate with the SAN (Subject Alternate Names) extension. The resulting certificate will be valid for every address or hostname listed on the Node resource to validate this request, the cluster-machine-approver ensures that every DNS name or IP address in the CSR matches a (NodeInternalDNS, NodeExternalDNS, NodeHostName) or (NodeInternalIP, NodeExternalIP) address on the corresponding Machine object.
Requirements for Cluster API Providers
As discussed in previous sections, cluster-machine-approver imposes some requirements on each Cluster API provider used with the machine-api-operator. This section serves as a summary of those requirements.
- A
Machinemust have aNodeInternalDNSset inStatus.Addressesthat matches the name of theNode. TheNodeInternalDNSentry must be present, even before theNoderesource is created. - A
Machinemust also have matchingNodeInternalDNS,NodeExternalDNS,NodeHostName,NodeInternalIP, andNodeExternalIPaddresses as those listed on theNoderesource. All of these addresses are placed in the CSR and are validated against the addresses on theMachineobject.
