flume报错信息收集

53 阅读 0 评论 35 点赞

我是靠谱客的博主年轻小蝴蝶，最近开发中收集的这篇文章主要介绍flume报错信息收集，觉得挺不错的，现在分享给大家，希望可以做个参考。

概述

1.当http sink的端口，有异常的时候，sink不成功，会报transaction的提交错误，如下：

2019-02-27 16:13:24,313 (lifecycleSupervisor-1-0) [INFO - org.apache.flume.instrumentation.MonitoredCounterGroup.start(MonitoredCounterGroup.java:95)] Component type: SOURCE, name: r3 started

2019-02-27 16:14:06,117 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - com.didichuxing.sts.HttpSinks.postJson(HttpSinks.java:116)] request was error!

2019-02-27 16:14:06,118 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:158)] Unable to deliver event. Exception follows.

java.lang.IllegalStateException: close() called when transaction is OPEN - you must either commit or rollback first

at com.google.common.base.Preconditions.checkState(Preconditions.java:172)

at org.apache.flume.channel.BasicTransactionSemantics.close(BasicTransactionSemantics.java:179)

at com.didichuxing.sts.HttpSinks.process(HttpSinks.java:92)

at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67)

at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145)

at java.lang.Thread.run(Thread.java:748)

2019-02-27 16:14:11,124 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - com.didichuxing.sts.HttpSinks.process(HttpSinks.java:87)] Failed to commit transaction.java.lang.IllegalStateException: begin() called when transaction is OPEN!

2019-02-27 16:14:11,125 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:158)] Unable to deliver event. Exception follows.

java.lang.IllegalStateException: begin() called when transaction is OPEN!

at com.google.common.base.Preconditions.checkState(Preconditions.java:145)

at org.apache.flume.channel.BasicTransactionSemantics.begin(BasicTransactionSemantics.java:131)

at com.didichuxing.sts.HttpSinks.process(HttpSinks.java:49)

at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67)

at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145)

at java.lang.Thread.run(Thread.java:748)

2019-02-27 16:14:19,144 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - com.didichuxing.sts.HttpSinks.postJson(HttpSinks.java:116)] request was error!

2019-02-27 16:14:19,144 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:158)] Unable to deliver event. Exception follows.

java.lang.IllegalStateException: close() called when transaction is OPEN - you must either commit or rollback first

2.flume-ng-sql-source的c3p0连接池，允许链接不操作时一直不断开链接。而mysql、oracle数据库在客户端不操作8小时左右的时候会自动断开链接，报下边这个错误

[c3p0] A PooledConnection that has already signalled a Connection error is still in use!

2019/07/30 10:28:57,096

[c3p0] Another error has occurred [ java.sql.SQLRecoverableException: Closed Connection ] which will not be reported to listeners!

java.sql.SQLRecoverableException: Closed Connection

at oracle.jdbc.driver.OracleStatement.ensureOpen(OracleStatement.java:4061)

at oracle.jdbc.driver.OracleStatement.getWarnings(OracleStatement.java:3097)

at oracle.jdbc.driver.OracleStatementWrapper.getWarnings(OracleStatementWrapper.java:348)

at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.getWarnings(NewProxyPreparedStatement.java:1031)

at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.handleAndClearWarnings(SqlExceptionHelper.java:320)

at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.logAndClearWarnings(SqlExceptionHelper.java:273)

at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.close(JdbcCoordinatorImpl.java:529)

at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.cleanup(JdbcCoordinatorImpl.java:509)

at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.close(JdbcCoordinatorImpl.java:204)

at org.hibernate.engine.transaction.internal.TransactionCoordinatorImpl.close(TransactionCoordinatorImpl.java:297)

at org.hibernate.internal.SessionImpl.close(SessionImpl.java:369)

at org.keedio.flume.source.HibernateHelper.resetConnection(HibernateHelper.java:200)

at org.keedio.flume.source.HibernateHelper.executeQuery(HibernateHelper.java:125)

at org.keedio.flume.source.DBSQLSource.process(DBSQLSource.java:102)

at org.apache.flume.source.PollableSourceRunner$PollingRunner.run(PollableSourceRunner.java:129)

at java.lang.Thread.run(Thread.java:748)

2019/07/30 10:28:57,097

[c3p0] A PooledConnection that has already signalled a Connection error is still in use!

2019/07/30 10:28:57,097

[c3p0] Another error has occurred [ java.sql.SQLRecoverableException: Closed Connection ] which will not be reported to listeners!

java.sql.SQLRecoverableException: Closed Connection

at oracle.jdbc.driver.OracleStatement.ensureOpen(OracleStatement.java:4061)

at oracle.jdbc.driver.OracleStatement.clearWarnings(OracleStatement.java:3106)

at oracle.jdbc.driver.OracleStatementWrapper.clearWarnings(OracleStatementWrapper.java:222)

at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.clearWarnings(NewProxyPreparedStatement.java:1057)

at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.handleAndClearWarnings(SqlExceptionHelper.java:329)