Self-signed certificates with k3s and cert-manager

At least for now, my homelab cluster (4x Raspberry Pi, k3s, etc) is available only for the devices on my local network, inside a custom DNS zone k8s.pi.home. I don’t think there are practical reasons to run anything with HTTPS in that setup, but there are cases, like browser extensions, where it’s required.

Turned out, in 2021, it’s fairly straight forward to set up a Certificate Authority (CA), that will issue TLS certificates to “secure” the ingress resources. At least, it’s way simpler comparing to how I remember it was back in the days. All thanks to cert-manager and some YAML.

First thing is to install cert-manager to the cluster. k3s comes with helm-controller, that gives us a way to manage helm charts with Custom Resource Definitions (CRD). The following manifest defines a new namespace, and a resource of a kind HelmChart, to install cert-manager inside this namespace:

apiVersion: v1
kind: Namespace
metadata:
  name: cert-manager

---
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: cert-manager
  namespace: kube-system
spec:
  chart: cert-manager
  repo: https://charts.jetstack.io
  targetNamespace: cert-manager
  valuesContent: |-
    installCRDs: true
    prometheus:
      enabled: true
      servicemonitor:
        enabled: true

After applying the manifest above — kubectl apply -f cert-manager.yml — define a self-signed certificate, which is used to bootstrap a CA:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: selfsigned-cluster-issuer
spec:
  selfSigned: {}

---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: selfsigned-ca
spec:
  isCA: true
  commonName: selfsigned-ca
  secretName: selfsigned-ca-root-secret
  privateKey:
    algorithm: ECDSA
    size: 256
  issuerRef:
    name: selfsigned-cluster-issuer
    kind: ClusterIssuer
    group: cert-manager.io

---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: selfsigned-issuer
spec:
  ca:
    secretName: selfsigned-ca-root-secret

And now, I can use selfsigned-issuer to issue TLS certificates for the ingress resources (Traefik ingress in the k3s’s case). E.g. to play around, I run an open-source version of LanguageTool server. The ingress manifests for the server looks like as following:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: languagetool-server
  annotations:
    kubernetes.io/ingress.class: traefik
    cert-manager.io/issuer: selfsigned-issuer
spec:
  rules:
  - host: languagetool.k8s.pi.home
    http:
      paths:
      - path: /
        pathType: ImplementationSpecific
        backend:
          service:
            name: languagetool-server
            port:
              number: 8010
  tls:
  - hosts: [languagetool.k8s.pi.home]
    secretName: languagetool-server-cert

Of course, any certificate signed by my CA won’t be automatically trusted by anyone, including my own system. If I try to access https://languagetool.k8s.pi.home, any HTTP client will raise a “failed to verify the legitimacy of the server” issue. I don’t know if there is a better way to solve that, but I can hack that around by installing the cluster’s root CA certificate into the system’s keychain, and telling the system, that it should “trust” the certificate:

$ kubectl get secret/selfsigned-ca-root-secret -o json \
  | jq -r '.data["ca.crt"]' \
  | base64 -D > ~/tmp/selfsigned-root-ca.crt
$ open ~/tmp/selfsigned-root-ca.crt

Choose “Always trust” the certificate in the keychain’s certificate settings. The server looks legitimate now!