我是靠谱客的博主 沉静口红，最近开发中收集的这篇文章主要介绍k8s多节点二进制部署（单节点基础上新加master2、两台LB安装nginx和keepalived实现负载均衡及高可用！）一、master02部署二、配置两台LB，觉得挺不错的，现在分享给大家，希望可以做个参考。
概述
文章目录
- 一、master02部署
- 1、首先关闭防火墙和selinux服务
- 2、复制kubernetes目录到master02
- 3、修改配置文件 kube-apiserver 中的IP
- 4、拷贝master1上已有的etcd证书给master2使用
- 5、启动master2中的三个组件服务
- 6、增加环境变量
- 二、配置两台LB
- 1、安装nginx服务，把nginx.sh拷贝到家目录
- 2、添加四层转发
- 3、启动nginx服务
- 4、部署keepalived服务
- 5、修改 keepalived 配置文件
- 6、编辑检测脚本
- 7、赋予检测脚本执行权限并启动 keepalived
- 8、验证地址漂移
- 9、恢复操作
- 10、开始修改node节点配置文件统一VIP（bootstrap.kubeconfig,kubelet.kubeconfig）
- 11、重启kubelet和kube-proxy服务
- 12、在lb01上查看nginx的k8s日志
- 13、测试创建pod
- 1、基于nginx镜像创建pod
- 1）注意日志问题
- 2）创建并绑定认证匿名用户，并赋予其管理员权限
- 2、查看pod网络
- 3、在对应网段的node节点上操作可以直接访问 pod 业务
- 4、访问就会产生日志，查看访问日志
Master1:192.168.2.3/24 kube-apiserver、kube-controller-manager、kube-scheduler、etcd
Master2:192.168.2.3/24 kube-apiserver、kube-controller-manager、kube-scheduler、etcd
Node01:192.168.2.5/24 kubelet、kube-proxy、docker、flannel、etcd
Node02:192.168.2.6/24 kubelet、kube-proxy、docker、flannel、etcd
LB01:192.168.2.7/24 nginx、keepalived
LB01:192.168.2.8/24 nginx、keepalived
注：先具备单master单节点部署环境
一、master02部署
1、首先关闭防火墙和selinux服务
Master2:192.168.2.4/24
systemctl stop firewalld
systemctl disable firewalld
setenforce 0
2、复制kubernetes目录到master02
Master1:192.168.2.3/24
//在master1上操作
scp -r /opt/kubernetes/ root@192.168.2.4:/opt
//复制master1中的三个组件启动脚本kube-apiserver.service kube-controller-manager.service kube-scheduler.service
scp /usr/lib/systemd/system/{kube-apiserver,kube-controller-manager,kube-scheduler}.service root@192.168.195.131:/usr/lib/systemd/system/
3、修改配置文件 kube-apiserver 中的IP
Master2:192.168.2.4/24
//master2上操作
cd /opt/kubernetes/cfg/
vim kube-apiserver
KUBE_APISERVER_OPTS="--logtostderr=true
--v=4
--etcd-servers=https://192.168.195.149:2379,https://192.168.195.150:2379,https://192.168.195.151:2379
--bind-address=192.168.2.4
--secure-port=6443
--advertise-address=192.168.2.4
--allow-privileged=true
--service-cluster-ip-range=10.0.0.0/24
--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction
--authorization-mode=RBAC,Node
--kubelet-https=true
--enable-bootstrap-token-auth
--token-auth-file=/opt/kubernetes/cfg/token.csv
--service-node-port-range=30000-50000
--tls-cert-file=/opt/kubernetes/ssl/server.pem
--tls-private-key-file=/opt/kubernetes/ssl/server-key.pem
--client-ca-file=/opt/kubernetes/ssl/ca.pem
--service-account-key-file=/opt/kubernetes/ssl/ca-key.pem
--etcd-cafile=/opt/etcd/ssl/ca.pem
--etcd-certfile=/opt/etcd/ssl/server.pem
--etcd-keyfile=/opt/etcd/ssl/server-key.pem"
注意：master02一定要有etcd证书
4、拷贝master1上已有的etcd证书给master2使用
Master1:192.168.2.3/24
scp -r /opt/etcd/ root@192.168.2.4:/opt/
5、启动master2中的三个组件服务
Master2:192.168.2.4/24
systemctl start kube-apiserver.service
systemctl enable kube-apiserver.service
systemctl start kube-controller-manager.service
systemctl enable kube-controller-manager.service
systemctl start kube-scheduler.service
systemctl enable kube-scheduler.service
6、增加环境变量
Master2:192.168.2.4/24
vim /etc/profile
export PATH=$PATH:/opt/kubernetes/bin/ #末尾添加
source /etc/profile
[root@localhost cfg]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
192.168.2.5 Ready <none> 2d12h v1.12.3
192.168.2.6 Ready <none> 38h v1.12.3
二、配置两台LB
LB01:192.168.2.7/24
LB02:192.168.2.8/24
//lb01 lb02操作
1、安装nginx服务，把nginx.sh拷贝到家目录
systemctl stop firewalld.service
systemctl disable firewalld.service
setenforce 0
vim /etc/yum.repos.d/nginx.repo
[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/centos/7/$basearch/
gpgcheck=0
yum install nginx -y
2、添加四层转发
vim /etc/nginx/nginx.conf
events {
worker_connections 1024;
}
stream {
log_format main '$remote_addr $upstream_addr - [$time_local] $status $upstream_bytes_sent';
access_log /var/log/nginx/k8s-access.log main;
upstream k8s-apiserver {
server 192.168.2.3:6443;
server 192.168.2.4:6443;
}
server {
listen 6443;
proxy_pass k8s-apiserver;
}
}
http {
3、启动nginx服务
systemctl start nginx
netstat -natp | grep nginx
4、部署keepalived服务
LB01:192.168.2.7/24
LB02:192.168.2.8/24
yum install keepalived -y
5、修改 keepalived 配置文件
LB01:192.168.2.7/24
vim /etc/keepalived/keepalived.conf
cp：是否覆盖"/etc/keepalived/keepalived.conf"？ yes
//注意：lb01是Mster配置如下：
! Configuration File for keepalived
global_defs {
# 接收邮件地址
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
# 邮件发送地址
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id NGINX_MASTER
}
vrrp_script check_nginx {
script "/etc/nginx/check_nginx.sh"
}
vrrp_instance VI_1 {
state MASTER
interface ens33
virtual_router_id 51 # VRRP 路由 ID实例，每个实例是唯一的
priority 100 # 优先级，备服务器设置 90
advert_int 1 # 指定VRRP 心跳包通告间隔时间，默认1秒
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.2.100/24
}
track_script {
check_nginx
}
}
LB02:192.168.2.8/24
//注意：lb02是Backup配置如下：
! Configuration File for keepalived
global_defs {
# 接收邮件地址
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
# 邮件发送地址
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id NGINX_MASTER
}
vrrp_script check_nginx {
script "/etc/nginx/check_nginx.sh"
}
vrrp_instance VI_1 {
state BACKUP
interface ens33
virtual_router_id 51 # VRRP 路由 ID实例，每个实例是唯一的
priority 90 # 优先级，备服务器设置 90
advert_int 1 # 指定VRRP 心跳包通告间隔时间，默认1秒
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.2.100/24
}
track_script {
check_nginx
}
}
6、编辑检测脚本
LB01:192.168.2.7/24
LB02:192.168.2.8/24
[root@localhost ~]# vim /etc/nginx/check_nginx.sh
count=$(ps -ef |grep nginx |egrep -cv "grep|$$")
if [ "$count" -eq 0 ];then
systemctl stop keepalived
fi
7、赋予检测脚本执行权限并启动 keepalived
LB01:192.168.2.7/24
LB02:192.168.2.8/24
[root@localhost ~]# chmod +x /etc/nginx/check_nginx.sh
[root@localhost ~]# systemctl start keepalived
LB01:192.168.2.7/24
//查看lb01地址信息
[root@localhost ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:5b:e8:41 brd ff:ff:ff:ff:ff:ff
inet 192.168.2.7/24 brd 192.168.2.255 scope global ens33
valid_lft forever preferred_lft forever
inet 192.168.2.100/24 scope global secondary ens33
valid_lft forever preferred_lft forever
inet6 fe80::d013:857:6073:7325/64 scope link
valid_lft forever preferred_lft forever
LB02:192.168.2.8/24
//查看lb02地址信息
[root@localhost nginx]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:03:42:13 brd ff:ff:ff:ff:ff:ff
inet 192.168.2.8/24 brd 192.168.2.255 scope global ens33
valid_lft forever preferred_lft forever
inet6 fe80::35f1:eade:78d5:8644/64 scope link
valid_lft forever preferred_lft forever
8、验证地址漂移
（lb01中使用pkill nginx，再在lb02中使用ip a 查看）
LB01:192.168.2.7/24
ip a
pkill nginx
ip a
LB02:192.168.2.8/24
ip a
9、恢复操作
LB01:192.168.2.7/24
（在lb01中先启动nginx服务，再启动keepalived服务）
//nginx站点/usr/share/nginx/html
systemctl restart nginx
systemctl restart keepalived.service
ip a
10、开始修改node节点配置文件统一VIP（bootstrap.kubeconfig,kubelet.kubeconfig）
Node1:192.168.2.5/24
Node2:192.168.2.6/24
[root@localhost cfg]# vim /opt/kubernetes/cfg/bootstrap.kubeconfig
[root@localhost cfg]# vim /opt/kubernetes/cfg/kubelet.kubeconfig
[root@localhost cfg]# vim /opt/kubernetes/cfg/kube-proxy.kubeconfig
//统统修改为VIP
server: https://192.168.2.100:6443
11、重启kubelet和kube-proxy服务
Node1:192.168.2.5/24
Node2:192.168.2.6/24
[root@localhost cfg]# systemctl restart kubelet.service
[root@localhost cfg]# systemctl restart kube-proxy.service
//替换完成直接自检
[root@localhost cfg]# grep 100 *
bootstrap.kubeconfig: server: https://192.168.2.100:6443
kubelet.kubeconfig: server: https://192.168.2.100:6443
kube-proxy.kubeconfig: server: https://192.168.2.100:6443
12、在lb01上查看nginx的k8s日志
LB01:192.168.2.7/24
[root@localhost ~]# tail /var/log/nginx/k8s-access.log
192.168.2.6 192.168.2.3:6443 - [15/Apr/2021:12:25:32 +0800] 200 1117
192.168.2.5 192.168.2.4:6443 - [15/Apr/2021:12:25:32 +0800] 200 1117
192.168.2.5 192.168.2.4:6443 - [15/Apr/2021:12:25:32 +0800] 200 1117
192.168.2.6 192.168.2.4:6443 - [15/Apr/2021:12:25:32 +0800] 200 1118
13、测试创建pod
Master1:192.168.2.3/24
//在master1上操作
1、基于nginx镜像创建pod
[root@localhost ~]# kubectl run nginx --image=nginx
kubectl run --generator=deployment/apps.v1beta1 is DEPRECATED and will be removed in a future version. Use kubectl create instead.
deployment.apps/nginx created
//查看状态
[root@localhost ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-dbddb74b8-d68t4 0/1 ContainerCreating 0 30s //正在创建中
[root@localhost ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-dbddb74b8-d68t4 1/1 Running 0 51s //创建完成，运行中
1）注意日志问题
Master1:192.168.2.3/24
[root@localhost ~]# kubectl logs nginx-dbddb74b8-d68t4
Error from server (Forbidden): Forbidden (user=system:anonymous, verb=get, resource=nodes, subresource=proxy) ( pods/log nginx-dbddb74b8-d68t4)
2）创建并绑定认证匿名用户，并赋予其管理员权限
Master1:192.168.2.3/24
[root@localhost ~]# kubectl create clusterrolebinding cluster-system-anonymous --clusterrole=cluster-admin --user=system:anonymous
clusterrolebinding.rbac.authorization.k8s.io/cluster-system-anonymous created
2、查看pod网络
Master1:192.168.2.3/24
[root@localhost ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE
nginx-dbddb74b8-d68t4 1/1 Running 0 5m48s 172.17.3.2 192.168.2.5 <none>
3、在对应网段的node节点上操作可以直接访问 pod 业务
根据IP地址192.168.2.5可知pod被分配到了，node01上
Node1:192.168.2.5/24
[root@localhost cfg]# curl 172.17.3.2
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
4、访问就会产生日志，查看访问日志
Master1:192.168.2.3/24
//回到master1操作
[root@localhost ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE
nginx-dbddb74b8-d68t4 1/1 Running 0 5m48s 172.17.3.2 192.168.2.5 <none>
[root@localhost ~]# kubectl logs nginx-dbddb74b8-d68t4
172.17.3.1 - - [15/Apr/2021:05:10:35 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"
最后
以上就是沉静口红为你收集整理的k8s多节点二进制部署（单节点基础上新加master2、两台LB安装nginx和keepalived实现负载均衡及高可用！）一、master02部署二、配置两台LB的全部内容，希望文章能够帮你解决k8s多节点二进制部署（单节点基础上新加master2、两台LB安装nginx和keepalived实现负载均衡及高可用！）一、master02部署二、配置两台LB所遇到的程序开发问题。
如果觉得靠谱客网站的内容还不错，欢迎将靠谱客网站推荐给程序员好友。
本图文内容来源于网友提供，作为学习参考使用，或来自网络收集整理，版权属于原作者所有。
发表评论 取消回复