canal 高可用配置

57 阅读 0 评论 38 点赞

我是靠谱客的博主优美镜子，最近开发中收集的这篇文章主要介绍canal 高可用配置，觉得挺不错的，现在分享给大家，希望可以做个参考。

概述

如果 WINDOWS 同机器部署多个 Canal 服务，注意，不是 Canal 实例，请修改端口参数：
1. 修改 canal.propeties 文件中的 canal.port、canal.metrics.pull.port、canal.admin.jmx.port 各配置项，指向其它端口
2. 修改 startup.bat 中 -Xrunjdwp:transport=dt_socket,address=9099 中的 9099 为其它端口

1. 修改 canal.properties

canal.zkServers=120.78.200.102:2181,120.78.200.102:2182,120.78.200.102:2183 # zookeeper IP 及端口列表
canal.instance.global.spring.xml = classpath:spring/default-instance.xml # 只能是 default-instance.xml，其它不支持 HA

2. 修改instance.properties，两台机器上的 instance 目录的名字需要保证完全一致，HA模式是依赖于instance name进行管理

# canal.instance.mysql.slaveId = 1234 ## v1.0.26+ will autoGen
canal.instance.master.address = 120.78.200.102:3306 # 数据库地址，不同实例可不一样

3. 启动两台机器的 Canal

sh bin/startup.sh

注意，只会有一台机器提示启动成功

4. 查看 zookeeper 节点信息

bin/zkCli.sh -server 127.0.0.1:2181 # 连接 Zookeeper
[zk: 127.0.0.1:2181(CONNECTED) 56] ls /otter/canal/destinations/zyloops/cluster
[172.19.0.1:22222, 192.168.31.13:11111, 192.168.31.13:22222]

可以看到 zookeeper 中已经存在三个 Canal 服务地址，因为我在本机启动了两个 Canal，在云服务器启动了一个，所以这里显示了三个，继续查看当前是哪个在提供服务

[zk: 127.0.0.1:2181(CONNECTED) 58] get /otter/canal/destinations/zyloops/running
{"active":true,"address":"172.19.0.1:22222","cid":2}

可以看到，当前工作的是云服务上的 Canal

为了确定当前服务确实在正确地提供服务，可以查看数据推送进度，本次部署采用 Kafka 的方式，所以我们去查看一下 Kafka 中的消息是否正确发布了

[root@ii ~]# docker exec -it 20 bash
bash-4.4# /opt/kafka/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic zyloops

打开消费端，等待消息到来。然后去数据数据进行数据更新操作，比如更新操作

UPDATE t_clz SET `name` = '55 grade3383' WHERE user_id = 1;

可以看到消息正常产出

[root@ii ~]# docker exec -it 20 bash
bash-4.4# /opt/kafka/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic zyloops
{"data":[{"id":"11613","user_id":"1","name":"55 grade3383"}],"database":"x1","es":1564044392000,"id":3,"isDdl":false,"mysqlType":{"id":"int(11)","user_id":"int(11)","name":"varchar(50)"},"old":[{"name":"55 grade83"}],"pkNames":["id"],"sql":"","sqlType":{"id":4,"user_id":4,"name":12},"table":"t_clz","ts":1564044392179,"type":"UPDATE"}

说明服务运行正常，现在我们停掉云服务器上的 Canal

sh bin/stop.sh

重新查看 zookeeper

[zk: 127.0.0.1:2181(CONNECTED) 59] get /otter/canal/destinations/zyloops/running
{"active":true,"address":"192.168.31.13:11111","cid":2}
[zk: 127.0.0.1:2181(CONNECTED) 60]

可以看到，已经自动切换到本机监听端口为 11111 的 Canal，继续进行数据更新操作

UPDATE t_clz SET `name` = '我是谁' WHERE user_id = 1;

Kafka 输出

[root@ii ~]# docker exec -it 20 bash
bash-4.4# /opt/kafka/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic zyloops
{"data":[{"id":"11613","user_id":"1","name":"55 grade3383"}],"database":"x1","es":1564044392000,"id":3,"isDdl":false,"mysqlType":{"id":"int(11)","user_id":"int(11)","name":"varchar(50)"},"old":[{"name":"55 grade83"}],"pkNames":["id"],"sql":"","sqlType":{"id":4,"user_id":4,"name":12},"table":"t_clz","ts":1564044392179,"type":"UPDATE"}
{"data":[{"id":"11613","user_id":"1","name":"55 grade3383"}],"database":"x1","es":1564044392000,"id":1,"isDdl":false,"mysqlType":{"id":"int(11)","user_id":"int(11)","name":"varchar(50)"},"old":[{"name":"55 grade83"}],"pkNames":["id"],"sql":"","sqlType":{"id":4,"user_id":4,"name":12},"table":"t_clz","ts":1564044517759,"type":"UPDATE"}
{"data":[{"id":"11613","user_id":"1","name":"我是谁"}],"database":"x1","es":1564044595000,"id":2,"isDdl":false,"mysqlType":{"id":"int(11)","user_id":"int(11)","name":"varchar(50)"},"old":[{"name":"55 grade3383"}],"pkNames":["id"],"sql":"","sqlType":{"id":4,"user_id":4,"name":12},"table":"t_clz","ts":1564044594437,"type":"UPDATE"}

说明 HA 配置正确