Kubernetes expose TCP service with nginx ingress controller

Background

In our case we build a kubernetes cluster with some linux virtual machines. Some of services in our cluster need layer 4 transport access from outside of cluster such as database and redis etc. However the default kubernetes ingress dose not support this. So we need use other ingress controller to achieve it. One of the famous ingress controll is nginx ingress controll. Here we will introduce a way to expose a tcp service with nginx ingress controller.

Requirements

• Nginx Ingress Controller
  Before expose your tcp service you need install the nginx ingress controller. Nginx Ingress Controller provided a helm chart, with the helm chart we can easily install the ingress controller, for the detailed information, please refer the official document here.

• Open the Port
  You need to ensure the exposed port should be abled to be accessed outside from the cluster.

Steps to setup

Create ConfigMap

As the official document metioned, we can not directly setup from ingress, we need specify a config map and let the ingress controller refer the config map. So we need create a config map and specify the port which will be exposed and the backend service.

Ingress does not support TCP or UDP services. For this reason this Ingress controller uses the flags --tcp-services-configmap and --udp-services-configmap to point to an existing config map where the key is the external port to use and the value indicates the service to expose using the format: ::[PROXY]:[PROXY]

apiVersion: v1
kind: ConfigMap
metadata:
  name: tcp-services
  namespace: ingress-nginx
data:
  3306: "mynamespace/mysql:3306"

Add args from ingress controller

Confirm if your ingress controller pod has --tcp-service-configmap argument and the value is pointed to the config map you just created. If No you need to edit the ingress controller deployment and add it and deploy the ingress controller.

apiVersion: v1
kind: Pod
spec:
  containers:
  - args:
    - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services

If the argument --tcp-services-configmap has been configured before. When you create/edit the config map. The ingress controller will load it automatically. And you will see some logs like this from ingress controller pods.

I0404 06:40:50.714239       7 controller.go:168] "Configuration changes detected, backend reload required"
I0404 06:40:50.818643       7 controller.go:185] "Backend successfully reloaded"

References

https://docs.nginx.com/nginx-ingress-controller/installation/installation-with-helm/
https://lokie.wang/article/126