ingress-nginx导出TCP端口

  • Exposing TCP and UDP services – Ingress-Nginx Controller
  • https://github.com/kubernetes/ingress-nginx/blob/main/docs/user-guide/exposing-tcp-udp-services.md

  • helm upgrade ingress-nginx导出redis 6379端口（这种方式最简单，接下为我们研究一下主要的流程原理）

    • K8s and Redis; a tale of Layer 4 Ingress
helm upgrade ingress-nginx ingress-nginx/ingress-nginx -n ingress-nginx \--values values.yaml \--set tcp.26379="develop/redis-cluster:6379" --debug
  • nginx-ingress 已经能够完成 7/4 层的代理功能（4 层代理基于 ConfigMap）
  • 流程图，请参考：Kubernetes K8S 1.20部署Ingress nginx 0.30_k8s1.20.9部署ingress-nginx-CSDN博客

  • 整体流程大概是这样子的

    • SVC(NodePort:30009映射到后台26379端口，26379端口转发到26379-tcp)
      • DaemonSet/ingress-nginx-controller(26379-tcp转到controller容器的端口containerPort: 26379)
        • ingress-nginx-controller容器中有nginx，nginx监听26379端口
          • 从configMap中取的26379端口要转发的地址
            • develop/redis-cluster:6379
    • 用户请求流程
      • 访问30009NodePort端口
        • 映射到26379端口
          • ingress-nginx-controller中的nginx监听26379端口
            • 将26379端口转发至develop/redis-cluster:6379

  • 添加configMap

vi configMap.yamlapiVersion: v1kind: ConfigMapmetadata:name: tcp-servicesnamespace: ingress-nginxdata:26379: "develop/redis-cluster:6379" #从26379的TCP端口转发到Redis集群的6379端口，至于nodePort是哪个还未定kubectl apply -f configMap.yaml -n ingress-nginx

也可以自己配置SVC

编辑service, 即ClusterIPkubectl edit svc -n ingress-nginx ingress-nginx-controller- name: 26379-tcp #配置一个新的TCP端口，取名为26379-tcpnodePort: 30009 #导出对外的端口port: 26379 #监听26379端口protocol: TCPtargetPort: 26379-tcp #转发到ds上的26379-tcp端口

配置DaemonSet

  • ingress-nginx-controller pod实际是Controlled By: DaemonSet/ingress-nginx-controller
  • 修改DaemonSet/ingress-nginx-controller
    • 将svc上监听到的26379端口转发到DaemonSet/ingress-nginx-controller上
  • DaemonSet/ingress-nginx-controller连接到ingress-nginx-controller对应的pod容器的26379端口上
 kubectl edit ds -n ingress-nginx ingress-nginx-controller - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-servicesports:- containerPort: 80name: httpprotocol: TCP- containerPort: 443name: httpsprotocol: TCP- containerPort: 8443name: webhookprotocol: TCP- containerPort: 26379name: 26379-tcpprotocol: TCP#查看SVC，已经有了26379端口，并绑定了nodePort端口30009[root@k8s-master01 ingress-nginx]# kubectl get svc -n ingress-nginxNAME TYPE CLUSTER-IPEXTERNAL-IP PORT(S) AGEingress-nginx-controller LoadBalancer 10.99.163.44 80:31268/TCP,443:31052/TCP,26379:30009/TCP 3d11hingress-nginx-controller-admission ClusterIP10.100.131.12 443/TCP

查看nginx.conf代理配置

  • kubectl exec -it ingress-nginx-controller-lkzzh /bin/sh -n ingress-nginx
    • /etc/nginx
    • $ ls -l
    • cat nginx.conf
  • 监听26379端口，应该是configMap中的data的26379
    • 从configMap中取出26379端口要转发的地址
  • 转发到tcp-develop-redis-cluster-6379
# TCP servicesserver {preread_by_lua_block {ngx.var.proxy_upstream_name="tcp-develop-redis-cluster-6379";}listen26379;listen[::]:26379;proxy_timeout 600s;proxy_next_upstream on;proxy_next_upstream_timeout 600s;proxy_next_upstream_tries 3;proxy_passupstream_balancer;}

测试

  • telnet可以通过
  • telnet 192.168.221.131 30009