k8s集群Canal的网络控制

71 阅读 0 评论 47 点赞

我是靠谱客的博主文艺香水，最近开发中收集的这篇文章主要介绍k8s集群Canal的网络控制，觉得挺不错的，现在分享给大家，希望可以做个参考。

概述

2019独角兽企业重金招聘Python工程师标准>>>

1 简介

Canal 是calico和flannel的结合，我们因为Calico不仅能够提供很好的网络性能，还能有网络规则控制，但是我们很多时候使用的Flannel网络，他的缺点就是没有网络控制只能提供网络服务，所以Canal诞生了，为Falanel提供了网络控制。

网络策略控制出站： egress 控制进站： ingress 可以以ip地址块定义规则，也可以基于命名空间规则定义，还可以对pod定义规则

2 部署Canal

curl 
https://docs.projectcalico.org/v3.6/getting-started/kubernetes/installation/hosted/canal/canal.yaml 
-O
kubectl apply -f canal.yaml
1）canal默认控制的是10.244.0.0/16 ，如果你在安装k8s集群是设置的pod网络不是此段网络，则你需要修改配置文件
2）我们采用的和k8s集群公用etcd的方式创建

查看创建的pod
# kubectl get pods -n kube-system 
NAME                                    READY   STATUS    RESTARTS   AGE
canal-9lmw8                             2/2     Running   0          4m50s
canal-ln6kg                             2/2     Running   0          4m50s
canal-mx6tn                             2/2     Running   0          4m50s
coredns-fb8b8dccf-lfbkh                 1/1     Running   2          10d
coredns-fb8b8dccf-t2kdz                 1/1     Running   2          10d
etcd-master-1                           1/1     Running   2          10d
kube-apiserver-master-1                 1/1     Running   2          10d
kube-controller-manager-master-1        1/1     Running   2          10d
kube-flannel-ds-amd64-df7gk             1/1     Running   3          9d
kube-flannel-ds-amd64-dzxfd             1/1     Running   3          9d
kube-flannel-ds-amd64-mgw2m             1/1     Running   2          9d
kube-proxy-47d6q                        1/1     Running   2          10d
kube-proxy-jztrs                        1/1     Running   3          10d
kube-proxy-rt4xx                        1/1     Running   3          10d
kube-scheduler-master-1                 1/1     Running   2          10d
kubernetes-dashboard-5f7b999d65-8h79h   1/1     Running   0          21h

3 创建两个实验的空间

kubectl create namespace shengchan
kubectl create namespace test

4 创建网络规则测试

默认ingress 和 engress可以一起使用，当这两规则同时存在那么将使用policyTypes做具体的规则指向
例如：创建一个policy默认拒绝所有在空间shengchan的入站规则


# cat ingress-deny.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-all-ingress
  namespace: shengchan
spec: 
  podSelector: {}
  policyTypes:
  - Ingress


kubectl apply ingress-deny.yaml

创建一组属于shengchan空间的pod
# cat pod-networkpolicy1.yaml 
apiVersion: v1
kind: Pod
metadata:
   name: pod-test
spec:
 containers:
 - name: pod-test-policy
   image: nginx:1.10

kubectl apply -f pod-networkpolicy1.yaml  -n shengchan

创建一组属于test空间的pod
kubectl apply -f pod-networkpolicy1.yaml  -n test

分别查看两组pod的IP地址

# kubectl get pods -o wide  -n shengchan
NAME       READY   STATUS    RESTARTS   AGE   IP           NODE    NOMINATED NODE   READINESS GATES
pod-test   1/1     Running   0          16s   10.244.1.2   node2   <none>           <none>

[root@master-1 ~]# kubectl get pods -o wide  -n test
NAME       READY   STATUS    RESTARTS   AGE   IP           NODE    NOMINATED NODE   READINESS GATES
pod-test   1/1     Running   0          9s    10.244.2.2   node1   <none>           <none>


在宿主机上面分别ping这两个ip
# ping 10.244.1.2 -c 1
PING 10.244.1.2 (10.244.1.2) 56(84) bytes of data.
^C
--- 10.244.1.2 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

# ping 10.244.2.2 -c 1
PING 10.244.2.2 (10.244.2.2) 56(84) bytes of data.
64 bytes from 10.244.2.2: icmp_seq=1 ttl=63 time=0.660 ms

--- 10.244.2.2 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.660/0.660/0.660/0.000 ms

此时发现属于shengchan空间的pod是被拒绝访问的，属于test空间的pod正常访问
此时我们是基于名称空间级别的入站网络访问策略

5 将上面的默认拒绝策略放行

修改配置文件
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-all-ingress
  namespace: shengchan
spec: 
  podSelector: {}
  ingress:
  - {}
  policyTypes:
  - Ingress
kubectl apply -f ingress-deny.yaml

再次测试10.244.1.2

[root@master-1 ~]# curl -I 10.244.1.2
HTTP/1.1 200 OK
Server: nginx/1.10.3
Date: Sat, 27 Apr 2019 16:58:04 GMT
Content-Type: text/html
Content-Length: 612
Last-Modified: Tue, 31 Jan 2017 15:01:11 GMT
Connection: keep-alive
ETag: "5890a6b7-264"
Accept-Ranges: bytes

[root@master-1 ~]# curl -I 10.244.2.2
HTTP/1.1 200 OK
Server: nginx/1.10.3
Date: Sat, 27 Apr 2019 16:58:07 GMT
Content-Type: text/html
Content-Length: 612
Last-Modified: Tue, 31 Jan 2017 15:01:11 GMT
Connection: keep-alive
ETag: "5890a6b7-264"
Accept-Ranges: bytes

说明我们的网络策略放行成功

6 放行特定的入站

以标签的形式进行划分pod入站规则是否被放行

1）回复默认拒绝所有
[root@master-1 ~]# cat ingress-deny.yaml 
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-all-ingress
  namespace: shengchan
spec: 
  podSelector: {}
  policyTypes:
  - Ingress

kubectl apply -f ingress-deny.yaml

2)给shengchan pod打一个标签
kubectl label pods pod-test app=hello -n shengchan
3）创建一个新的规则
# vim ingress-allow.yml 

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-hello-ingress
  namespace: shengchan
spec:
  podSelector:
    matchLabels:
      app: hello
  ingress:
  - from:
    - ipBlock:
        cidr: 10.244.0.0/16
        except:
        - 10.244.2.2/32
    ports:
    - protocol: TCP
      port: 80


解释： 我们重新定义了一个policy，通过标签匹配到我们已有的pod（我们一已有的pod是拒绝所有访问的），定义ingres规则，如果不通过from方式定义默认是所有网段都可以访问到拥有这个标签的pod为了测试我们使用from方式去拒绝10.244.0.0但是会放行10.244.2.2因为我们有这个ip的pod创建万郴更可以进行测试ports就是指定特定开放的端口
4）创建之前进行访问测试
# kubectl get pods -n shengchan -o wide
NAME       READY   STATUS    RESTARTS   AGE   IP           NODE    NOMINATED NODE   READINESS GATES
pod-test   1/1     Running   0          62m   10.244.1.2   node2   <none>           <none>
# curl 10.244.1.2
说明无法访问
5）创建之后测试
kubectl apply -f ingress-allow.yml 

[root@master-1 ~]# curl -I 10.244.1.2
HTTP/1.1 200 OK
Server: nginx/1.10.3
Date: Sat, 27 Apr 2019 17:32:15 GMT
Content-Type: text/html
Content-Length: 612
Last-Modified: Tue, 31 Jan 2017 15:01:11 GMT
Connection: keep-alive
ETag: "5890a6b7-264"
Accept-Ranges: bytes
说明已经放行
至于对于10.244.0.0网段的限制有兴趣可以测试一下

6 egress出站规则

1）设置一个拒绝所有出站policy
cat egress.yml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-all-egress
  namespace: test
spec: 
  podSelector: {}
  policyTypes:
  - Egress

kubectl apply -f egress.yml

2）创建一个test命名空间下的pods
kubectl apply -f pod-networkpolicy1.yaml -n test
3)查看podIP并进去容器去测试ping一个正常的podip
[root@master-1 ~]# kubectl get pods -o wide
NAME                        READY   STATUS    RESTARTS   AGE    IP            NODE    NOMINATED NODE   READINESS GATES
net-test-5764c456cb-2c9df   1/1     Running   24         24h    10.244.2.22   node1   <none>           <none>
net-test-5764c456cb-ng6vh   1/1     Running   46         6d5h   10.244.1.21   node2   <none>           <none>

[root@master-1 ~]# kubectl get pods -o wide -n test
NAME       READY   STATUS    RESTARTS   AGE   IP           NODE    NOMINATED NODE   READINESS GATES
pod-test   1/1     Running   0          86m   10.244.2.2   node1   <none>           <none>

kubectl exec -it pod-test -n test -- /bin/bash

root@pod-test:/# ping 10.244.1.21
PING 10.244.1.21 (10.244.1.21): 56 data bytes
^C--- 10.244.1.21 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss

说明此时已经应用策略
4）修改配置文件放行所有出站
# cat egress.yml 
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-all-egress
  namespace: test
spec: 
  podSelector: {}
  egress:
  - {}
  policyTypes:
  - Egress

kubectl apply -f egress.yml 
测试：
root@pod-test:/# ping 10.244.1.21 -c 1
PING 10.244.1.21 (10.244.1.21): 56 data bytes
64 bytes from 10.244.1.21: icmp_seq=0 ttl=62 time=0.939 ms
--- 10.244.1.21 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.939/0.939/0.939/0.000 ms
此时说明已经放行成功

7 更加复杂的policy规则需要根据不同的环境规划进行深入探究，其所有都规则都体现在配置文件中

转载于:https://my.oschina.net/54188zz/blog/3043290