k8s二进制安装篇2-部署etcd集群

104 阅读 0 评论 69 点赞

我是靠谱客的博主坦率小白菜，这篇文章主要介绍k8s二进制安装篇2-部署etcd集群，现在分享给大家，希望可以做个参考。

etcd 的leader选举机制，要求至少为3台或以上的奇数台。本次安装涉及：192.168.1.246，192.168.1.247，192.168.1.248

一，签发etcd证书

证书签发服务器192.168.1.245:
1，创建ca的json配置: /opt/certs/ca-config.json
oserver 表示服务端连接客户端时携带的证书，用于客户端验证服务端身份
oclient 表示客户端连接服务端时携带的证书，用于服务端验证客户端身份
opeer 表示相互之间连接时使用的证书，如etcd节点之间验证
“expiry”: “175200h” 证书有效期十年如果这里是一年的话到期后集群会立宕掉

复制代码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
vim
/opt/certs/ca-config.json
{
"signing": {
"default": {
"expiry": "175200h"
},
"profiles": {
"server": {
"expiry": "175200h",
"usages": [
"signing",
"key encipherment",
"server auth"
]
},
"client": {
"expiry": "175200h",
"usages": [
"signing",
"key encipherment",
"client auth"
]
},
"peer": {
"expiry": "175200h",
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
]
}
}
}
}

2，创建etcd证书配置：/opt/certs/etcd-peer-csr.json
重点在hosts上，将所有可能的etcd服务器添加到host列表，不能使用网段，新增etcd服务器需要重新签发证书

复制代码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
vim /opt/certs/etcd-peer-csr.json
{
"CN": "k8s-etcd",
"hosts": [
"192.168.1.245",
"192.168.1.246",
"192.168.1.247",
"192.168.1.248"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "guangdong",
"L": "guangzhou",
"O": "zz",
"OU": "ops"
}
]
}

3，签发证书

复制代码

1
2
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=peer etcd-peer-csr.json |cfssljson -bare etcd-peer

二，安装etcd

etcd地址：https://github.com/etcd-io/etcd/
下载地址：https://github.com/etcd-io/etcd/releases/download/v3.4.0/etcd-v3.4.0-linux-amd64.tar.gz
本次安装涉及：192.168.1.246，192.168.1.247，192.168.1.248
1，下载etcd（192.168.1.246，192.168.1.247，192.168.1.248）

复制代码

1
2
3
4
5
6
7
8
9
10
11
12
cd /usr/local/src/
wget https://github.com/etcd-io/etcd/releases/download/v3.4.0/etcd-v3.4.0-linux-amd64.tar.gz
useradd -s /sbin/nologin -M etcd
tar -xf etcd-v3.4.0-linux-amd64.tar.gz
mv etcd-v3.4.0-linux-amd64 /opt/etcd-v3.4.0
cd /opt/etcd-v3.4.0
ln -s /opt/etcd-v3.4.0/
/opt/etcd
mkdir -p /opt/etcd/certs
/data/etcd
/data/logs/etcd-server

2，下发证书到各个etcd上（192.168.1.245）

复制代码

1
2
3
cd /opt/certs/
for i in 246 247 248;do scp ca.pem etcd-peer.pem etcd-peer-key.pem 192.168.1.${i}:/opt/etcd/certs/ ;done

3，创建启动脚本(部分参数每台机器不同)（192.168.1.246，192.168.1.247，192.168.1.248）

复制代码

vim /lib/systemd/system/etcd.service
[Unit]
Description=etcd
Documentation=https://github.com/coreos
[Service]
Type=notify
ExecStart=/opt/etcd/etcd 
--name=etcd-server-246 
--data-dir=/data/etcd/etcd-server 
--listen-peer-urls https://192.168.1.246:2380 
--listen-client-urls https://192.168.1.246:2379,http://127.0.0.1:2379 
--quota-backend-bytes 8000000000 
--initial-advertise-peer-urls https://192.168.1.246:2380 
--advertise-client-urls https://192.168.1.246:2379,http://127.0.0.1:2379 
--initial-cluster
etcd-server-246=https://192.168.1.246:2380,etcd-server-247=https://192.168.1.247:2380,etcd-server-248=https://192.168.1.248:2380 
--initial-cluster-token etcd-cluster-0 
--cert-file=/opt/etcd/certs/etcd-peer.pem 
--key-file=/opt/etcd/certs/etcd-peer-key.pem 
--client-cert-auth

--peer-client-cert-auth 
--trusted-ca-file=/opt/etcd/certs/ca.pem 
--peer-cert-file=/opt/etcd/certs/etcd-peer.pem 
--peer-key-file=/opt/etcd/certs/etcd-peer-key.pem 
--peer-client-cert-auth 
--peer-trusted-ca-file=/opt/etcd/certs/ca.pem 
--enable-v2  # 由于flannel目前确实不能与etcdV3直接交互，开启etcd 支持V2api功能，在etcd启动参数中加入 --enable-v2参数
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
vim /lib/systemd/system/etcd.service
[Unit]
Description=etcd
Documentation=https://github.com/coreos
[Service]
Type=notify
ExecStart=/opt/etcd/etcd 
--name=etcd-server-246 
--data-dir=/data/etcd/etcd-server 
--listen-peer-urls https://192.168.1.246:2380 
--listen-client-urls https://192.168.1.246:2379,http://127.0.0.1:2379 
--quota-backend-bytes 8000000000 
--initial-advertise-peer-urls https://192.168.1.246:2380 
--advertise-client-urls https://192.168.1.246:2379,http://127.0.0.1:2379 
--initial-cluster
etcd-server-246=https://192.168.1.246:2380,etcd-server-247=https://192.168.1.247:2380,etcd-server-248=https://192.168.1.248:2380 
--initial-cluster-token etcd-cluster-0 
--cert-file=/opt/etcd/certs/etcd-peer.pem 
--key-file=/opt/etcd/certs/etcd-peer-key.pem 
--client-cert-auth

--peer-client-cert-auth 
--trusted-ca-file=/opt/etcd/certs/ca.pem 
--peer-cert-file=/opt/etcd/certs/etcd-peer.pem 
--peer-key-file=/opt/etcd/certs/etcd-peer-key.pem 
--peer-client-cert-auth 
--peer-trusted-ca-file=/opt/etcd/certs/ca.pem 
--enable-v2  # 由于flannel目前确实不能与etcdV3直接交互，开启etcd 支持V2api功能，在etcd启动参数中加入 --enable-v2参数
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target

启动etcd

复制代码

1
2
3
4
systemctl daemon-reload
systemctl start etcd.service
systemctl enable etcd.service

复制代码

1
2
3
4
5
6
7
8
9
10
11
查看集群的状态
./etcdctl
--cacert=/opt/etcd/certs/ca.pem --cert=/opt/etcd/certs/etcd-peer.pem
--key=/opt/etcd/certs/etcd-peer-key.pem
member list --endpoints=https://192.168.1.246:2379,https://192.168.1.247:2379,https://192.168.1.248:2379
查看leader
etcd-v3.4.0]# curl http://127.0.0.1:2379/v2/stats/leader
{"leader":"e87617ab250d1acb","followers":{"7e037e71de645e58":{"latency":{"current":0.002364,"average":0.005725999999999999,"standardDeviation":0.0031717232434960887,"minimum":0.002347,"maximum":0.010384},"counts":{"fail":0,"success":6}},"dae8c96b50d21d09":{"latency":{"current":0.002043,"average":0.0039885,"standardDeviation":0.0016390976938547633,"minimum":0.002043,"maximum":0.00654},"counts":{"fail":0,"success":6}}}}
etcd-v3.4.0]# curl http://127.0.0.1:2379/v2/stats/leader
{"message":"not current leader"}