flume sink hive 启动命令中常见异常

常见问题1

Failed to start agent because dependencies were not found in classpath. Error follows.
java.lang.NoClassDefFoundError: org/apache/hive/hcatalog/streaming/RecordWriter

解决办法一:

将hive中相关的jar复制到$FLUME_HOME/lib下

cp /home/xiaobin/soft/apache-hive-0.14.0-bin/hcatalog/share/hcatalog/*.jar
$FLUME_HOME/lib/
cp /home/xiaobin/soft/apache-hive-0.14.0-bin/hive-*.jar $FLUME_HOME/lib/
cp antlr-2.7.7.jar ~/soft/apache-flume-1.7.0-bin/lib/
cp antlr-runtime-3.4.jar ~/soft/apache-flume-1.7.0-bin/lib/

解决办法二:

指定classpath

--classpath "/home/work/soft/apache-hive-0.14.0-bin/hcatalog/share/hcatalog/*":"/home/work/soft/apache-hive-0.14.0-bin/lib/*"

常见问题二

Caused by: org.apache.flume.sink.hive.HiveWriter$ConnectException: Failed connecting to EndPoint {metaStoreUri='thrift://192.168.1.115:9083', database='default', table='kafkauser', partitionVals=[20170826] }
at org.apache.flume.sink.hive.HiveWriter.<init>(HiveWriter.java:99)
at org.apache.flume.sink.hive.HiveSink.getOrCreateWriter(HiveSink.java:343)
at org.apache.flume.sink.hive.HiveSink.drainOneBatch(HiveSink.java:295)
at org.apache.flume.sink.hive.HiveSink.process(HiveSink.java:253)
... 3 more
Caused by: org.apache.flume.sink.hive.HiveWriter$ConnectException: Failed connecting to EndPoint {metaStoreUri='thrift://192.168.1.115:9083', database='default', table='kafkauser', partitionVals=[20170826] }
at org.apache.flume.sink.hive.HiveWriter.newConnection(HiveWriter.java:383)
at org.apache.flume.sink.hive.HiveWriter.<init>(HiveWriter.java:86)
... 6 more
Caused by: java.util.concurrent.TimeoutException
at java.util.concurrent.FutureTask.get(FutureTask.java:205)
at org.apache.flume.sink.hive.HiveWriter.timedCall(HiveWriter.java:434)
at org.apache.flume.sink.hive.HiveWriter.newConnection(HiveWriter.java:376)
... 7 more

这个问题是目录权限问题,解决办法如下

hadoop dfs -chmod 777 /tmp/hive
chmod 777 /tmp/hive

常见问题三

NullPointerException Non-local session path expected to be non-null

配置文件错误（仔细检查）

最后

以上就是成就柠檬为你收集整理的flume sink hive 启动命令中常见异常的全部内容，希望文章能够帮你解决flume sink hive 启动命令中常见异常所遇到的程序开发问题。

如果觉得靠谱客网站的内容还不错，欢迎将靠谱客网站推荐给程序员好友。