文章
文章用户商铺文档圈子

Kubernetes 二进制添加扩容Node节点

释放双眼，带上耳机，听听看~！
前面一直介绍了kubernetes各个版本的二进制安装，但是很多情况是我们需要进行扩展。 随着业务的扩大，node节点的扩展也是必不可少的。这篇文章就给大家带来机遇kubernetes 1.14安装的一个扩展教程，其他集群安装的扩展也大概相同。可能相关的路径不对而已
🤖 由 ChatGPT 生成的文章摘要

Kubernetes 1.14 二进制集群安装

新闻联播老司机
  • 喜欢：1
  • 浏览：20.1k

Prometheus 原理介绍

新闻联播老司机
  • 喜欢：0
  • 浏览：6.8k

本次环境这里添加了一个k8s-05 ip为192.168.0.14的新node节点，添加多个node节点的步骤和方式相同，这里不一一举例

没有特殊说明的都在k8s-01上进行操作

首先我们需要修改host主机，添加k8s-05

#在之前的master节点添加
echo "192.168.0.14  k8s-05" >>/etc/hosts


#在k8s-05操作
cat >>/etc/hosts<<EOF
192.168.0.10  k8s-01
192.168.0.11  k8s-02
192.168.0.12  k8s-03
192.168.0.13  k8s-04
EOF

检查host解析

[root@k8s-01 ~]# ping -c 1 k8s-05
PING k8s-05 (192.168.0.14) 56(84) bytes of data.
64 bytes from k8s-05 (192.168.0.14): icmp_seq=1 ttl=64 time=0.910 ms

--- k8s-05 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.910/0.910/0.910/0.000 ms

分发秘钥

cd 
ssh-copy-id -i ~/.ssh/id_rsa.pub k8s-05
#为了后面拷贝证书等文件方便快捷，这里继续分发秘钥

在k8s-05节点创建K8s相关目录

mkdir -p  /opt/k8s/{bin,work} /etc/{kubernetes,etcd}/cert

推送CA证书

cd /etc/kubernetes/cert
scp ca.pem ca-config.json k8s-05:/etc/kubernetes/cert/

flanneld部署

cd /opt/k8s/work/flannel
scp flanneld mk-docker-opts.sh k8s-05:/opt/k8s/bin/

拷贝flanneld密钥

ssh k8s-05 "mkdir -p /etc/flanneld/cert"
scp /etc/flanneld/cert/flanneld*.pem k8s-05:/etc/flanneld/cert

拷贝flannel启动文件

scp /etc/systemd/system/flanneld.service k8s-05:/etc/systemd/system/

#启动flannel
ssh k8s-05 "systemctl daemon-reload && systemctl enable flanneld && systemctl restart flanneld"

#检查是否启动成功
ssh k8s-05 "systemctl status flanneld|grep Active"

查看etcd网络数据

新增flannel会注册到etcd

source /opt/k8s/bin/environment.sh
etcdctl \
  --endpoints=${ETCD_ENDPOINTS} \
  --ca-file=/etc/kubernetes/cert/ca.pem \
  --cert-file=/etc/flanneld/cert/flanneld.pem \
  --key-file=/etc/flanneld/cert/flanneld-key.pem \
  ls ${FLANNEL_ETCD_PREFIX}/subnets

正常结果如下

Kubernetes 二进制添加扩容Node节点

上面的步骤结束后我们flannel网络就设置完毕

Kubernetes Node 节点安装Docker

这里我们直接在k8s-05节点安装docker

yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum install -y conntrack ntpdate ntp ipvsadm ipset jq iptables curl sysstat libseccomp wget
yum makecache fast
yum -y install docker-ce

创建配置文件

mkdir -p /etc/docker/
cat > /etc/docker/daemon.json <<EOF
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "registry-mirrors": ["https://hjvrgh7a.mirror.aliyuncs.com"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2"
}
EOF

修改配置文件

vim /usr/lib/systemd/system/docker.service

Kubernetes 二进制添加扩容Node节点

在配置文件14行删除原来并添加下面的参数

EnvironmentFile=-/run/flannel/docker
ExecStart=/usr/bin/dockerd $DOCKER_NETWORK_OPTIONS -H fd:// --containerd=/run/containerd/containerd.sock

启动docker并检查服务状态

systemctl daemon-reload && systemctl enable docker && systemctl restart docker

systemctl status docker|grep Active

查看docke0网桥是否正常

[root@k8s-05 ~]# ip addr show flannel.1 && /usr/sbin/ip addr show docker0
3: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default
    link/ether 1a:c7:47:3a:f5:4e brd ff:ff:ff:ff:ff:ff
    inet 172.30.64.0/32 scope global flannel.1
       valid_lft forever preferred_lft forever
    inet6 fe80::18c7:47ff:fe3a:f54e/64 scope link
       valid_lft forever preferred_lft forever
4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:99:c3:03:9d brd ff:ff:ff:ff:ff:ff
    inet 172.30.64.1/21 brd 172.30.71.255 scope global docker0
       valid_lft forever preferred_lft forever

安装kubelet

创建kubelet bootstrap kubeconfig文件

#k8s-01操作

cd /opt/k8s/work
export BOOTSTRAP_TOKEN=$(kubeadm token create \
  --description kubelet-bootstrap-token \
  --groups system:bootstrappers:k8s-05 \
  --kubeconfig ~/.kube/config)
# 设置集群参数
kubectl config set-cluster kubernetes \
  --certificate-authority=/etc/kubernetes/cert/ca.pem \
  --embed-certs=true \
  --server=https://192.168.0.100:8443 \
  --kubeconfig=kubelet-bootstrap-k8s-05.kubeconfig
# 设置客户端认证参数
kubectl config set-credentials kubelet-bootstrap \
  --token=${BOOTSTRAP_TOKEN} \
  --kubeconfig=kubelet-bootstrap-k8s-05.kubeconfig
# 设置上下文参数
kubectl config set-context default \
  --cluster=kubernetes \
  --user=kubelet-bootstrap \
  --kubeconfig=kubelet-bootstrap-k8s-05.kubeconfig
# 设置默认上下文
kubectl config use-context default --kubeconfig=kubelet-bootstrap-k8s-05.kubeconfig

分发kubeconfig

cd /opt/k8s/work
scp kubelet-bootstrap-k8s-05.kubeconfig k8s-05:/etc/kubernetes/kubelet-bootstrap.kubeconfig

查看kubeadm为各个节点创建的token

$ kubeadm token list --kubeconfig ~/.kube/config
TOKEN                     TTL       EXPIRES                     USAGES                   DESCRIPTION               EXTRA GROUPS
2bd2l8.48aqiyi70ilmyapd   23h       2020-02-15T15:18:04-05:00   authentication,signing   kubelet-bootstrap-token   system:bootstrappers:k8s-03
2juok7.m3ovzxlplynkidg2   23h       2020-02-15T15:18:04-05:00   authentication,signing   kubelet-bootstrap-token   system:bootstrappers:k8s-02
4tco8p.fnzj1yfvsx5hkf2e   23h       2020-02-15T15:18:05-05:00   authentication,signing   kubelet-bootstrap-token   system:bootstrappers:k8s-04
8kw20m.ehj3git0b2e1bwkc   23h       2020-02-15T15:17:56-05:00   authentication,signing   kubelet-bootstrap-token   system:bootstrappers:k8s-01
j1olh5.qzktcctz5kcaywqk   23h       2020-02-15T16:02:01-05:00   authentication,signing   kubelet-bootstrap-token   system:bootstrappers:k8s-05
kktiu3.3a1adkatjo4zjuqh   23h       2020-02-15T15:17:57-05:00   authentication,signing   kubelet-bootstrap-token   system:bootstrappers:k8s-02
nzkb4r.g63nm9qqbq2e474q   23h       2020-02-15T15:18:04-05:00   authentication,signing   kubelet-bootstrap-token   system:bootstrappers:k8s-01

#目前这里我们已经可以看到K8s-05节点的信息

查看各token关联的Secret

$ kubectl get secrets  -n kube-system|grep bootstrap-token
bootstrap-token-2bd2l8                           bootstrap.kubernetes.io/token         7      47m
bootstrap-token-2juok7                           bootstrap.kubernetes.io/token         7      47m
bootstrap-token-4tco8p                           bootstrap.kubernetes.io/token         7      47m
bootstrap-token-8kw20m                           bootstrap.kubernetes.io/token         7      47m
bootstrap-token-j1olh5                           bootstrap.kubernetes.io/token         7      3m29s
bootstrap-token-kktiu3                           bootstrap.kubernetes.io/token         7      47m
bootstrap-token-nzkb4r                           bootstrap.kubernetes.io/token         7      47m

#可以看到已经有一个新建的

创建和分发kubelet参数配置

cd /opt/k8s/work
sed -e "s/##NODE_IP##/192.168.0.14/" kubelet-config.yaml.template > kubelet-config-192.168.0.14.yaml.template
scp kubelet-config-192.168.0.14.yaml.template root@k8s-05:/etc/kubernetes/kubelet-config.yaml

拷贝kubelet启动文件

cd /opt/k8s/work
source /opt/k8s/bin/environment.sh
sed -e "s/##NODE_NAME##/k8s-05/" kubelet.service.template > kubelet-k8s-05.service
scp kubelet-k8s-05.service root@k8s-05:/etc/systemd/system/kubelet.service

拷贝kubelet命令

scp /opt/k8s/bin/kubelet k8s-05:/opt/k8s/bin/

启动kubelet

cd /opt/k8s/work
source /opt/k8s/bin/environment.sh
ssh root@k8s-05 "mkdir -p ${K8S_DIR}/kubelet/kubelet-plugins/volume/exec/"
ssh root@k8s-05 "/usr/sbin/swapoff -a"
ssh root@k8s-05 "systemctl daemon-reload && systemctl enable kubelet && systemctl restart kubelet"

手动approve server cert csr

稍等片刻后，需要手动通过证书请求

基于安全考虑，CSR approving controllers不会自动approve kubelet server证书签名请求，需要手动approve

kubectl get csr | grep Pending | awk '{print $1}' | xargs kubectl certificate approve

当我们再次查看节点

[root@k8s-01 work]# kubectl get node
NAME     STATUS   ROLES    AGE   VERSION
k8s-01   Ready       57m   v1.14.2
k8s-02   Ready       57m   v1.14.2
k8s-03   Ready       57m   v1.14.2
k8s-04   Ready       57m   v1.14.2
k8s-05   Ready       51s   v1.14.2

安装kube-proxy

此处均在 node01 上执行

推送kube-proxy二进制启动文件

cd /opt/k8s/work/
scp kubernetes/server/bin/kube-proxy k8s-05:/opt/k8s/bin/

分发kubeconfig文件

cd /opt/k8s/work/
scp kube-proxy.kubeconfig root@k8s-05:/etc/kubernetes/

分发和创建kube-proxy配置文件

cd /opt/k8s/work/
sed -e "s/##NODE_NAME##/k8s-05/" -e "s/##NODE_IP##/10.0.20.15/" kube-proxy-config.yaml.template > kube-proxy-config-k8s-05.yaml.template
scp kube-proxy-config-k8s-05.yaml.template root@k8s-05:/etc/kubernetes/kube-proxy-config.yaml

分发kube-proxy systemd unit文件

scp kube-proxy.service root@k8s-05:/etc/systemd/system/

启动kube-proxy服务

cd /opt/k8s/work
source /opt/k8s/bin/environment.sh
ssh root@k8s-05 "mkdir -p ${K8S_DIR}/kube-proxy"
ssh root@k8s-05 "modprobe ip_vs_rr"
ssh root@k8s-05 "systemctl daemon-reload && systemctl enable kube-proxy && systemctl restart kube-proxy"

检查启动结果

ssh root@k8s-05 "systemctl status kube-proxy|grep Active"
ssh root@k8s-05 "netstat -lnpt|grep kube-prox"

检查ipvs路由规则

ssh root@k8s-05 "/usr/sbin/ipvsadm -ln"

这里所有节点安装完毕，可以创建Pod进行测试

cd /opt/k8s/work
cat > nginx-ds.yml <<EOF
apiVersion: v1
kind: Service
metadata:
  name: nginx-ds
  labels:
    app: nginx-ds
spec:
  type: NodePort
  selector:
    app: nginx-ds
  ports:
  - name: http
    port: 80
    targetPort: 80
---
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: nginx-ds
  labels:
    addonmanager.kubernetes.io/mode: Reconcile
spec:
  template:
    metadata:
      labels:
        app: nginx-ds
    spec:
      containers:
      - name: my-nginx
        image: daocloud.io/library/nginx:1.13.0-alpine
        ports:
        - containerPort: 80
EOF

kubectl apply -f /opt/k8s/work/nginx-ds.yml

检查结果

kubectl get pod -o wide
NAME             READY   STATUS    RESTARTS   AGE   IP             NODE     NOMINATED NODE   READINESS GATES
busybox          1/1     Running   1          63m   172.30.88.4    k8s-01              
nginx-ds-25w2v   1/1     Running   0          65m   172.30.104.2   k8s-03              
nginx-ds-2b6mn   1/1     Running   0          65m   172.30.240.2   k8s-04              
nginx-ds-6rsm4   1/1     Running   0          12m   172.30.64.2    k8s-05              
nginx-ds-n58rv   1/1     Running   0          65m   172.30.88.2    k8s-01              
nginx-ds-zvnx2   1/1     Running   0          65m   172.30.184.2   k8s-02

相关文章：

  1. Kubernetes 1.14 二进制集群安装
  2. Kubenetes 1.13.5 集群二进制安装
  3. Kuerbernetes 1.11 集群二进制安装
  4. Kubeadm搭建高可用(k8s)Kubernetes v1.24.0集群

给TA打赏
共{{data.count}}人
人已打赏
Kubernetes
Kubernetes

Kubernetes Traefik Ingress

2020-2-5 20:32:04

Kubernetes

Kubernetes 亲和性调度

2020-2-25 5:08:01

猜你喜欢

Kubernetes 1.14 二进制集群安装

11220.1k
0 条回复 A文章作者 M管理员
    暂无讨论，说说你的看法吧
❯

解锁会员权限

开通会员

解锁海量优质VIP资源

立刻开通
个人中心
购物车
优惠劵
今日签到
有新私信 私信列表
搜索

  • 扫码打开当前页

  • 运维博客公众号

返回顶部