keepalived集群搭建

keepalived是通过vrrp协议保证系统高可用的解决方案，一般作为一个高可用系统的入口，提供虚拟ip（vip）供用户访问系统，keepavlived集群的下一个中间件往往为haproxy或者nginx。

一. 搭建过程

• vip: 192.168.56.107

0. 预先准备

   # vim /etc/sysctl.conf
   net.ipv4.ip_nonlocal_bind=1 # linux绑定非本机ip(需要绑定vip)
   # 执行sysctl -p使其生效
   

1. 三台主机上创建相同的docker-compose.yml文件

version: "3"
services:
    keepalived:
      image: arcts/keepalived
      container_name: keepalived
      environment:
              KEEPALIVED_AUTOCONF: "true"
              KEEPALIVED_STATE: "${KEEPALIVED_STATE}"
              KEEPALIVED_INTERFACE: "${INTERFACE_NAME}"
              KEEPALIVED_PRIORITY: "${KEEPALIVED_PRIORITY}"
              KEEPALIVED_VIRTUAL_ROUTER_ID: "${KEEPALIVED_VIRTUAL_ROUTER_ID}"
              KEEPALIVED_VIRTUAL_IPADDRESS_1: "${VIP}"
              TZ: Asia/Shanghai
      network_mode: "host"
      restart: always
      privileged: true

详细的环境变量配置可以参考：https://hub.docker.com/r/arcts/keepalived

2. 三台主机docker- compose.yml文件同级目录创建不同的.env文件
   .env

VIP=192.168.56.107  #提供服务的vip
KEEPALIVED_STATE=MASTER # 该主机的状态: 104:MASTER 105/106:BACKUP
INTERFACE_NAME=enp0s8 # 通过ip addr命令查看该ip所使用的网卡名
KEEPALIVED_PRIORITY=200 # 该主机vrrp包的优先级 104:200 105/106:100
KEEPALIVED_VIRTUAL_ROUTER_ID=123 #该集群的唯一标识，同一集群内的keepalived该值必须相同（0-255）

3. 三台主机docker- compose.yml文件同级目录执行启动命令

docker-compose up -d

❤️ tips ❤️:
此启动方式是通过传入docker容器的环境变量配置的keepalived，非容器化启动keepalived一般通过配置/etc/keepalived/keeplived.conf文件来进行配置，进入容器也可以看到此配置文件

/etc/keepalived/keeplived.conf

global_defs {
router_id LVS_MAIN
}
vrrp_instance MAIN {
  state MASTER
  interface enp0s8
  virtual_router_id 123
  priority 200
  advert_int 1
  unicast_src_ip 192.168.56.104
  unicast_peer {
  }
  authentication {
    auth_type PASS
    auth_pass pwd123
  }
  virtual_ipaddress {
    192.168.56.107
  }
  track_interface {
    enp0s8
}
}

注意:
大括号前面有一个空格(之前老是提示我配置文件格式错误????)
可以通过keepalived -t来检查配置文件是否有格式错误

二. 验证keepalived集群

1. 集群正常工作

Master主机104网卡下产生虚拟ip(vip)107.此时访问107等同于访问104
另外两台Backup主机上无法看到vip

2. Master主机宕机

Slave1或者Slave2其中的一台产生vip，此时访问vip等于访问该Slave主机，以此来保证服务的高可用，此过程被称为ip漂移

如果三台宕机任意两台，这vip一定漂移到剩余的那台机器。

3. Master主机恢复

Master宕机，vip漂移到某台Slave机器，但Master主机恢复后，vip又会漂回到Master主机

注意:

1. 同一keepalived集群内的所有机器的virtual_router_id应该相同，但是同一局域网内的多个keepalived集群的virtual_router_id应该不同，不然会报以下错误.
   
   ip address associated with VRID not present in received packet
   one or more VIP associated with VRID mismatch actual MASTER advert

2. vip应该是局域网内dhcp未分配的ip，最好先ping一下你想要设置的vip，ping不通即可.

三. keepalived集群中的每台机器都出现了vip的解决办法(组播VS单播)

先说结论: 将keepalived组播改为单播

之前开发阶段在用于开发的三台机器上搭建了keepalived集群，一点问题没有，完全符合预期。后来在QA测试阶段，他们在虚拟机上搭建了keepalived集群，集群内的每台机器上都可以看到vip,虽然vip仍然可以访问，但是完成不了ip漂移，所以无法保证高可用，等于没有搭建keepalived集群…????????????

我的第一反应是让QA把三台机器的防火墙都关闭了，怀疑是不是slave机器未收到master发送的vrrp包。结果关闭防火墙后还是此现象…

最后好在公司里另外一个项目组有人在公司里搭建过keepalived集群，向其请教才知道，我们公司里keepalived需要使用单播，不能使用组播。

组播:

Master向组播地址224.0.0.18 发送vrrp包，Slave1和Slave2收到优先级高（200）的消息，因为自己的消息优先级低(100),就不会对外发送消息。

正常情况下在三台机器上执行 tcpdump -i enp0s8 vrrp -n 都能看到相同的Master向组播地址发送的vrrp包信息。

三台机器都显示:23:52:08.012282 IP 192.168.56.104 > 224.0.0.18: VRRPv2, Advertisement, vrid 123, prio 200, authtype simple, in tvl 1s, length 20

但是我在QA搭建的keepalived集群上执行上面命令，发现每台机器上都是看到的自己向组播地址发送的vrrp包.
ubuntu01: 23:52:08.012282 IP 192.168.56.104 > 224.0.0.18: VRRPv2, Advertisement, vrid 123, prio 200, authtype simple, in tvl 1s, length 20
ubuntu02: 23:52:08.012282 IP 192.168.56.105 > 224.0.0.18: VRRPv2, Advertisement, vrid 123, prio 100, authtype simple, in tvl 1s, length 20
ubuntu03: 23:52:08.012282 IP 192.168.56.106 > 224.0.0.18: VRRPv2, Advertisement, vrid 123, prio 100, authtype simple, in tvl 1s, length 20

可能是上层交换机禁用了arp的广播限制，造成keepalive无法通过广播通信。总之就是Slave没有收到Master发给组播地址的高优先级的vrrp包，导致Slave自己也给组播地址发送了vrrp包，所以每台主机都产生了vip。

单播:

单播即各个主机不再给组播地址发送vrrp包，而是各个主机之前端对端通信，能够降低干扰和冲突。

这里以Master机器上的keepalived启动为例,主要是增加了三个环境变量

• KEEPALIVED_UNICAST_PEER_1
• KEEPALIVED_UNICAST_PEER_2
• KEEPALIVED_UNICAST_SRC_IP

docker-compose.yml

version: "3"
services:
    keepalived:
      image: arcts/keepalived
      container_name: keepalived
      environment:
              KEEPALIVED_AUTOCONF: "true"
              KEEPALIVED_STATE: "${KEEPALIVED_STATE}"
              KEEPALIVED_INTERFACE: "${INTERFACE_NAME}"
              KEEPALIVED_PRIORITY: "${KEEPALIVED_PRIORITY}"
              KEEPALIVED_VIRTUAL_ROUTER_ID: "${KEEPALIVED_VIRTUAL_ROUTER_ID}"
              KEEPALIVED_VIRTUAL_IPADDRESS_1: "${VIP}"
              KEEPALIVED_UNICAST_PEER_1: "${SLAVE1_IP}" # 集群中另外机器的ip
              KEEPALIVED_UNICAST_PEER_2: "${SLAVE2_IP}" # 集群中另外机器的ip
              KEEPALIVED_UNICAST_SRC_IP: "${MASTER_IP}" #自己的机器的ip
              TZ: Asia/Shanghai
      network_mode: "host"
      restart: always
      privileged: true

.env

MASTER_IP=192.168.56.104
SLAVE1_IP=192.168.56.105
SLAVE2_IP=192.168.56.106
VIP=192.168.56.107
KEEPALIVED_STATE=MASTER
INTERFACE_NAME=enp0s8
KEEPALIVED_PRIORITY=200
KEEPALIVED_VIRTUAL_ROUTER_ID=123

docker-compose up -d

此时容器内的/etc/keepalived/keepalived.conf文件内容为:

global_defs {
router_id LVS_MAIN
}
vrrp_instance MAIN {
  state MASTER
  interface enp0s8
  virtual_router_id 123
  priority 200
  advert_int 1
  unicast_src_ip 192.168.56.104
  unicast_peer {
    192.168.56.105
    192.168.56.106
  }
  authentication {
    auth_type PASS
    auth_pass pwd123
  }
  virtual_ipaddress {
    192.168.56.107
  }
  track_interface {
    enp0s8
}
}

最后

以上就是体贴火龙果为你收集整理的keepalived集群搭建的全部内容，希望文章能够帮你解决keepalived集群搭建所遇到的程序开发问题。

如果觉得靠谱客网站的内容还不错，欢迎将靠谱客网站推荐给程序员好友。