Nginx 四层负载均衡

四层代理介绍

四层负载均衡工作在OSI模型中的四层，即传输层。四层负载均衡只能根据报文中目标地址和源地址对请求进行转发，而无法修改或判断所请求资源的具体类型，然后经过负载均衡内部的调度算法转发至要处理请求的服务器。四层负载均衡单纯的提供了终端到终端的可靠连接，并将请求转发至后端，连接至始至终都是同一个。LVS就是很典型的四层负载均衡。

需求

当我们阿里云或者腾讯云需要将3306、6379等中间件服务映射到公网进行本地调试时，如果使用默认端口会被人扫描。除了添加密码外，还需要将默认的端口修改，程序连接的端口不变，本地开发调试跨公网使用非默认端口号

安装Nginx

Nginx 4层代理基于--with-stream模块，非内置模块，需要在安装时编译进去

yum install pcre pcre-devel openssl openssl-devel gcc gcc-c++ automake autoconf libtool make wget vim lrzsz -y
cd /opt/
wget https://nginx.org/download/nginx-1.20.2.tar.gz
tar nginx-1.20.2.tar.gz
cd nginx-1.20.2
useradd nginx -s /sbin/nologin -M
./configure --prefix=/opt/nginx-1.20 --with-pcre --with-http_ssl_module --with-http_stub_status_module --with-stream --with-http_stub_status_module --with-http_gzip_static_module && make && make install 

官方文档给的参考案例

stream需要配置在http标签，也就是不可以通过域名或者location访问，只能用ip+端口做转发

worker_processes auto;

error_log /var/log/nginx/error.log info;

events {
    worker_connections  1024;
}

stream {
    upstream backend {
        hash $remote_addr consistent;

        server backend1.example.com:12345 weight=5;
        server 127.0.0.1:12345            max_fails=3 fail_timeout=30s;
        server unix:/tmp/backend3;
    }

    upstream dns {
       server 192.168.0.1:53535;
       server dns.example.com:53;
    }

    server {
        listen 12345;
        proxy_connect_timeout 1s;
        proxy_timeout 3s;
        proxy_pass backend;
    }

    server {
        listen 127.0.0.1:53 udp reuseport;
        proxy_timeout 20s;
        proxy_pass dns;
    }

    server {
        listen [::1]:12345;
        proxy_pass unix:/tmp/stream.socket;
    }
}

我们开始配置Nginx

我这里采用stream代理redis

cat >/opt/nginx-1.20/conf/nginx.conf<<EOF
# TCP
worker_processes  1;
events {
    worker_connections  1024;
}

# TCP Redis
stream {
    server {
        listen 9990;
        proxy_pass redis;
        proxy_connect_timeout 60s;
        proxy_timeout 60m;
    }

    upstream redis {
        server 127.0.0.1:6379 max_fails=3 fail_timeout=10;
    }
}

http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;

    server {
        listen       80;
        server_name  localhost;
        location / {
            root   html;
            index  index.html index.htm;
        }
    }

}
EOF

重载配置

[root@abcdocker conf]# /opt/nginx-1.20/sbin/nginx  -t
nginx: the configuration file /opt/nginx-1.20/conf/nginx.conf syntax is ok
nginx: configuration file /opt/nginx-1.20/conf/nginx.conf test is successful
[root@abcdocker conf]# /opt/nginx-1.20/sbin/nginx
[root@abcdocker conf]# /opt/nginx-1.20/sbin/nginx  -s reload

查看tcp端口

[root@abcdocker conf]# lsof -i:9990
COMMAND   PID   USER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
nginx   22469   root    6u  IPv4 8188127      0t0  TCP *:osm-appsrvr (LISTEN)
nginx   22472 nobody    6u  IPv4 8188127      0t0  TCP *:osm-appsrvr (LISTEN)

steam需要配置在http同级，因为4层代理是属于端口转发，所以不可以配置在http标签下

我这里安装一个redis测试一下

yum install redis -y
sed -i 's#^notify-keyspace-events.*#notify-keyspace-events Ex#g' /etc/redis.conf
sed -i 's/^bind 127.0.0.1/#bind 127.0.0.1/g' /etc/redis.conf
sed -i 's#^protected-mode.*#protected-mode no#g' /etc/redis.conf
systemctl restart redis
systemctl enable redis

访问测试

[root@k8s-01 ~]# telnet 192.168.31.98 9990
Trying 192.168.31.98...
Connected to 192.168.31.98.
Escape character is '^]'.