Using Cloudflare - This article is part of a series.
I’ve been running Kubernetes in my Home Lab for a long time now. There have been some rough patches, but on the whole it works well for me, and through trying to figure out how to best self host a variety of software I’ve learned a lot about how things work.
This post isn’t specifically about Kubernetes, but understanding how I have things set up is useful context. I have two
nginx ingress controllers running. One is labeled
internal, and the other
external. If it’s not obvious, one is for services that I access from my local LAN, and the other is for services that I expose to the internet, with the internal one being the default. Until recently that meant that I had ports 80 & 443 forwarded via my firewall to the
LoadBalancer IP of the external ingress controller. To put some guardrails around that, I restricted access to the
Cloudflare IPs, and set the Cloudflare DNS record to
Proxied. My external IP rarely changes, so that worked fine. However, I’ve recently discovered Cloudflare Tunnels (formerly called Argo Tunnels, and now called Cloudflare Zero Trust Access Tunnels … yeah, I don’t understand it either). This post is about setting up a Cloudflare Tunnel in a Kubernetes environment to work with a specific ingress controller.
So, what’s the advantage of using a Cloudflare Tunnel? In short, it means you don’t need to port forward 80 & 443 on your firewall. The way it works is there’s a
cloudflared process that runs locally that connects, via configuration, to a named Cloudflare Tunnel (yes, you can have more than one). One can think of it as a specialized VPN connection. The
cloudflared configuration contains rules that can direct traffic to different targets. In my case this is simplied since I just need one rule to send everything to the ingress controller designated for external services. Once the
cloudflared client is connected, the DNS for the service needs to be updated to point to the generated FQDN of the Cloudflare Tunnel. Clients get directed to Cloudflare’s external endpoint, traverse the tunnel, and ultimately connect to the locally running service.
While a tunnel can be created via the
Create a tunnel button at
Cloudflare Zero Trust -> Access -> Tunnels, I find it easier to use the
cloudflared CLI. It is available via the
cloudflared GitHub repository or via various package management systems. I’m on a Mac, so I used
brew install cloudflare/cloudflare/cloudflared.
cloudflared login will launch a web browser so you can provide your Cloudflare credentials. When complete, creating a new tunnel is as simple as:
cloudflare tunnel create NAME
The important thing here is a credentials JSON file will be created:
along with a certifictate:
The JSON file contains the token information to allow
cloudflared to connect to the named tunnel. Since it will be running in Kubernetes, the file will need to be made availabile to the Pod at a mount point of
/etc/cloudflared/creds/credentials.json. There are many ways to achieve this. In my case, I just made the file a Secret in the namespace that
cloudflared will be running in:
kubectl create secret generic -n cloudflared tunnel-credentials --from-file=credentials.json=$HOME/.cloudflared/CloudflareTunnelGUID.json
I do the same with the cert file:
kubectl create secret generic -n cloudflared tunnel-cert --from-file=$HOME/.cloudflared/cert.pem
When it comes to the rest of the Kubernetes setup, there are only two components, the actual Deployment, and a ConfigMap.
The ConfigMap, which defines a
config.yaml for cloudflared, is straight forward, and contains the following elements:
- This is the NAME of the tunnel from where it was created.
- The full path to the credentials.json file.
- This is set to
truesince it doesn’t make sense for the software to try to update itself in the context of a container.
- This is set to
- This is the cloudflared ingress configuration section. While multiple rules can be placed here, I want it to send all traffic to a specific endpoint.
- This specifies the default endpoint, and I point it to the internal cluster name of my external ingress controller.
My ConfigMap file can be seen here.
The Deployment is also straight forward. I don’t guarentee any uptimes for anything I self host, so I’m less concerned about HA. Therefore, I’m fine with a single tunnel, and a single
The Secrets for the
cert.pem are mounted into the Pod, as is the
config.yaml ConfigMap. The container is given run arguments of
tunnel --config /etc/cloudflared/config/config.yaml run , and that’s all there is to it.
My Deployment file can be seen here
Post Setup #
Once it’s up and running, verification of the tunnel connection can be done via
kubectl logs inspection of the running Pod and/or via the Cloudlfare Tunnel web UI. Assuming everything is connected, the next bit is to update the DNS records for the services that need to be accessible via the internet.
The records need to be of type
CloudflareTunnelGUID.cfargotunnel.com set as the target, and the
Proxy status enabled. Once that’s done, external clients will be able to connect to any Ingresses that are configure with the
IngressClass of the external ingress controller via the
cloudflared tunnel. Also, if you’re using
Let's Encrypt certs (which you should be doing, by the way … I suppose I should write something up about how to setup
cert-manager), then you can (and should) also enable the
Full (strict) setting on the domain’s
SSL/TLS Cloudflare configuration.
Final Notes #
Another advantage to this approach is the Cloudflare Tunnel will act as an IPv6 gateway of sorts. My Kubernetes cluster is IPv4 only, but an IPv6 client will get an AAAA record that resolves to the Cloudflare Tunnel endpoint. In the access logs of the service pod, the client’s IPv6 IP will be recorded.
While this approach doesn’t guarentee more security than port forwarding ports 80 & 443, I do like that I no longer need to do that. And in the event that my public IP ends up changing, that has no bearing on any of the configuration since the local
cloudflared client is the one initiating the connection.