Flume安装配置

1.上传文件配置

上传到master,然后解压

2.配置环境变量

vi /etc/profile

export FLUME_HOME=/bigdata/flume
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HIVE_HOME/bin:$SCALA_HOME/bin:$SPARK_HOME/bin:$FLINK_HOME/bin:$ZOOKEEPER_HOME/bin:$KAFKA_HOME/bin:$FLUME_HOME/bin:

保存然后刷新

source /etc/profile

3.配置flume-env.sh

进入到flume下的conf目录

mv flume-env.sh.template flume-env.sh
vi flume-env.sh
#加上
export JAVA_HOME=/bigdata/jdk
export HADOOP_HOME=/bigdata/hadoop

4.配置master.conf

这个文件没有我们在conf下创建一个

a1.sources=r1
a1.sinks=k1
a1.channels=c1
a1.sources.r1.type=http
a1.sources.r1.port=9000
a1.sources.r1.bind=0.0.0.0
a1.sources.r1.positionFile=/bigdata/flume/flume.json
a1.sources.r1.filegroups=f1 f2
a1.sources.r1.filegroups.f1=/bigdata/flume/.*file.*
a1.sources.r1.filegroups.f2=/bigdata/flume/logs/.*log.*
a1.sinks.k1.channel = c1
a1.sinks.k1.type = org.apache.flume.sink.kafka.KafkaSink
a1.sinks.k1.kafka.topic = mytopic
a1.sinks.k1.kafka.bootstrap.servers = master:9092,slave1:9092,slave2:9092
a1.sinks.k1.kafka.flumeBatchSize = 20
a1.sinks.k1.kafka.producer.acks = 1
a1.sinks.k1.kafka.producer.linger.ms = 1
a1.sinks.ki.kafka.producer.compression.type = snappy
a1.sources.r1.handler=org.example.rest.RestHandler
a1.sources.r1.handler.nickname=random props
a1.channels.c1.type=memory
a1.channels.c1.capacity=1000
a1.channels.c1.transactionCapacity=100
a1.sources.r1.channels=c1
a1.sinks.k1.channel=c1

5.测试

flume-ng agent -n a1 -c ./conf -f ./conf/master.conf -Dflume.root.logger=DEBUG,console &

成功之后就分发

最后

以上就是怡然电脑为你收集整理的8.Flume安装配置(大数据集群)Flume安装配置的全部内容，希望文章能够帮你解决8.Flume安装配置(大数据集群)Flume安装配置所遇到的程序开发问题。

如果觉得靠谱客网站的内容还不错，欢迎将靠谱客网站推荐给程序员好友。