天池数据挖掘比赛-心跳信号分类04-建模调参建模与调参

122 阅读 0 评论 81 点赞

我是靠谱客的博主健康御姐，这篇文章主要介绍天池数据挖掘比赛-心跳信号分类04-建模调参建模与调参，现在分享给大家，希望可以做个参考。

建模与调参

1、模型：逻辑回归模型、树模型、集成模型

2、模型对比与性能评估

3、模型调参：贪心调参、网格调参、贝叶斯调参

集成模型包括：

基于bagging思想的集成模型：随机森林模型

基于boosting思想的集成模型：XGBoost模型、LightGBM模型、CatBoost模型

二、模型对比与性能评估

逻辑回归：

优点：训练速度较快，分类的时候，计算量仅仅只和特征的数目相关；简单易理解，模型的可解释性非常好，从特征的权重可以看到不同的特征对最后结果的影响；适合二分类问题，不需要缩放输入特征；内存资源占用小，只需要存储各个维度的特征值；

缺点：逻辑回归需要预先处理缺失值和异常值；不能用Logistic回归去解决非线性问题，因为Logistic的决策面是线性的；对多重共线性数据较为敏感，且很难处理数据不平衡的问题；准确率并不是很高，因为形式非常简单，很难去拟合数据的真实分布；

决策树模型：

优点：简单直观，生成的决策树可以可视化展示；数据不需要预处理，不需要归一化，不需要处理缺失数据；既可以处理离散值，也可以处理连续值

缺点：决策树算法非常容易过拟合，导致泛化能力不强（可进行适当的剪枝）；采用的是贪心算法，容易得到局部最优解

集成模型集成方法：

集成模型通过组合多个学习器来完成学习任务，将多个弱学习器组合成一个强分类器，其泛化能力一般比单一分类器要好

集成方法主要包括Bagging和Boosting，区别总结如下：

样本选择上： Bagging方法的训练集是从原始集中有放回的选取，所以从原始集中选出的各轮训练集之间是独立的；而Boosting方法需要每一轮的训练集不变，只是训练集中每个样本在分类器中的权重发生变化。而权值是根据上一轮的分类结果进行调整
样例权重上： Bagging方法使用均匀取样，所以每个样本的权重相等；而Boosting方法根据错误率不断调整样本的权值，错误率越大则权重越大
预测函数上： Bagging方法中所有预测函数的权重相等；而Boosting方法中每个弱分类器都有相应的权重，对于分类误差小的分类器会有更大的权重
并行计算上： Bagging方法中各个预测函数可以并行生成；而Boosting方法各个预测函数只能顺序生成，因为后一个模型参数需要前一轮模型的结果

模型评估方法：

数据集的划分，满足以下两个条件：

训练集和测试集的分布要与样本真实分布一致；
训练集和测试集要互斥

数据集的划分有三种方法：

留出法、交叉验证法、自助法

对于数据量充足的时候，通常采用留出法或者k折交叉验证法来进行训练/测试集的划分；
对于数据集小且难以有效划分训练/测试集时使用自助法；
对于数据集小且可有效划分的时候最好使用留一法来进行划分，因为这种方法最为准确；

三、代码示例

导入库函数

复制代码

1
2
3
4
5
6
7
8
9
import warnings
warnings.filterwarnings('ignore')
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
import os
from sklearn.metrics import f1_score  # f1-score的模型评价标准

读取数据

复制代码

def reduce_mem_usage(df):
    start_mem = df.memory_usage().sum() / 1024**2 
    print('Memory usage of dataframe is {:.2f} MB'.format(start_mem))
    
    for col in df.columns:
        col_type = df[col].dtype
        
        if col_type != object:
            c_min = df[col].min()
            c_max = df[col].max()
            if str(col_type)[:3] == 'int':
                if c_min > np.iinfo(np.int8).min and c_max < np.iinfo(np.int8).max:
                    df[col] = df[col].astype(np.int8)
                elif c_min > np.iinfo(np.int16).min and c_max < np.iinfo(np.int16).max:
                    df[col] = df[col].astype(np.int16)
                elif c_min > np.iinfo(np.int32).min and c_max < np.iinfo(np.int32).max:
                    df[col] = df[col].astype(np.int32)
                elif c_min > np.iinfo(np.int64).min and c_max < np.iinfo(np.int64).max:
                    df[col] = df[col].astype(np.int64)  
            else:
                if c_min > np.finfo(np.float16).min and c_max < np.finfo(np.float16).max:
                    df[col] = df[col].astype(np.float16)
                elif c_min > np.finfo(np.float32).min and c_max < np.finfo(np.float32).max:
                    df[col] = df[col].astype(np.float32)
                else:
                    df[col] = df[col].astype(np.float64)
        else:
            df[col] = df[col].astype('category')

end_mem = df.memory_usage().sum() / 1024**2 
    print('Memory usage after optimization is: {:.2f} MB'.format(end_mem))
    print('Decreased by {:.1f}%'.format(100 * (start_mem - end_mem) / start_mem))
    
    return df

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
def reduce_mem_usage(df):
    start_mem = df.memory_usage().sum() / 1024**2 
    print('Memory usage of dataframe is {:.2f} MB'.format(start_mem))
    
    for col in df.columns:
        col_type = df[col].dtype
        
        if col_type != object:
            c_min = df[col].min()
            c_max = df[col].max()
            if str(col_type)[:3] == 'int':
                if c_min > np.iinfo(np.int8).min and c_max < np.iinfo(np.int8).max:
                    df[col] = df[col].astype(np.int8)
                elif c_min > np.iinfo(np.int16).min and c_max < np.iinfo(np.int16).max:
                    df[col] = df[col].astype(np.int16)
                elif c_min > np.iinfo(np.int32).min and c_max < np.iinfo(np.int32).max:
                    df[col] = df[col].astype(np.int32)
                elif c_min > np.iinfo(np.int64).min and c_max < np.iinfo(np.int64).max:
                    df[col] = df[col].astype(np.int64)  
            else:
                if c_min > np.finfo(np.float16).min and c_max < np.finfo(np.float16).max:
                    df[col] = df[col].astype(np.float16)
                elif c_min > np.finfo(np.float32).min and c_max < np.finfo(np.float32).max:
                    df[col] = df[col].astype(np.float32)
                else:
                    df[col] = df[col].astype(np.float64)
        else:
            df[col] = df[col].astype('category')

    end_mem = df.memory_usage().sum() / 1024**2 
    print('Memory usage after optimization is: {:.2f} MB'.format(end_mem))
    print('Decreased by {:.1f}%'.format(100 * (start_mem - end_mem) / start_mem))
    
    return df

复制代码

1
2
3
4
5
6
7
8
9
10
11
12
# 读取数据
data = pd.read_csv('train.csv')
# 简单预处理
data_list = []
for items in data.values:
    data_list.append([items[0]] + [float(i) for i in items[1].split(',')] + [items[2]])

data = pd.DataFrame(np.array(data_list))
data.columns = ['id'] + ['s_'+str(i) for i in range(len(data_list[0])-2)] + ['label']

data = reduce_mem_usage(data)

复制代码

1
2
3
4
Memory usage of dataframe is 157.93 MB
Memory usage after optimization is: 39.67 MB
Decreased by 74.9%

简单建模

复制代码

1
2
3
4
5
6
7
8
9
10
11
# 建模之前的预操作
from sklearn.model_selection import KFold
# 分离数据集，方便进行交叉验证
X_train = data.drop(['id','label'], axis=1)
y_train = data['label']

# 5折交叉验证
folds = 5
seed = 2021
kf = KFold(n_splits=folds, shuffle=True, random_state=seed)

复制代码

1
2
3
4
5
6
7
# 因为树模型中没有f1-score评价指标，所以需要自定义评价指标，在模型迭代中返回验证集f1-score变化情况。
def f1_score_vali(preds, data_vali):
    labels = data_vali.get_label()
    preds = np.argmax(preds.reshape(4, -1), axis=0)
    score_vali = f1_score(y_true=labels, y_pred=preds, average='macro')
    return 'f1_score', score_vali, True

复制代码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
# 使用Lightgbm进行建模
"""对训练集数据进行划分，分成训练集和验证集，并进行相应的操作"""
from sklearn.model_selection import train_test_split
import lightgbm as lgb
# 数据集划分
X_train_split, X_val, y_train_split, y_val = train_test_split(X_train, y_train, test_size=0.2)
train_matrix = lgb.Dataset(X_train_split, label=y_train_split)
valid_matrix = lgb.Dataset(X_val, label=y_val)

params = {
    "learning_rate": 0.1,
    "boosting": 'gbdt',  
    "lambda_l2": 0.1,
    "max_depth": -1,
    "num_leaves": 128,
    "bagging_fraction": 0.8,
    "feature_fraction": 0.8,
    "metric": None,
    "objective": "multiclass",
    "num_class": 4,
    "nthread": 10,
    "verbose": -1,
}

"""使用训练集数据进行模型训练"""
model = lgb.train(params, 
                  train_set=train_matrix, 
                  valid_sets=valid_matrix, 
                  num_boost_round=2000, 
                  verbose_eval=50, 
                  early_stopping_rounds=200,
                  feval=f1_score_vali)

复制代码

1
2
3
4
5
6
7
8
9
Training until validation scores don't improve for 200 rounds
[50]	valid_0's multi_logloss: 0.0503058	valid_0's f1_score: 0.959819
[100]	valid_0's multi_logloss: 0.0451365	valid_0's f1_score: 0.967628
[150]	valid_0's multi_logloss: 0.0472827	valid_0's f1_score: 0.968307
[200]	valid_0's multi_logloss: 0.049286	valid_0's f1_score: 0.969802
[250]	valid_0's multi_logloss: 0.0508972	valid_0's f1_score: 0.970252
Early stopping, best iteration is:
[85]	valid_0's multi_logloss: 0.0448914	valid_0's f1_score: 0.966127

复制代码

1
2
3
4
5
6
# 对验证集进行预测
val_pre_lgb = model.predict(X_val, num_iteration=model.best_iteration)
preds = np.argmax(val_pre_lgb, axis=1)
score = f1_score(y_true=y_val, y_pred=preds, average='macro')
print('未调参前lightgbm单模型在验证集上的f1：{}'.format(score))

复制代码

1
2
未调参前lightgbm单模型在验证集上的f1：0.9661274849389851

复制代码

"""使用lightgbm 5折交叉验证进行建模预测"""
cv_scores = []
for i, (train_index, valid_index) in enumerate(kf.split(X_train, y_train)):
    print('************************************ {} ************************************'.format(str(i+1)))
    X_train_split, y_train_split, X_val, y_val = X_train.iloc[train_index], y_train[train_index], X_train.iloc[valid_index], y_train[valid_index]
    
    train_matrix = lgb.Dataset(X_train_split, label=y_train_split)
    valid_matrix = lgb.Dataset(X_val, label=y_val)

params = {
                "learning_rate": 0.1,
                "boosting": 'gbdt',  
                "lambda_l2": 0.1,
                "max_depth": -1,
                "num_leaves": 128,
                "bagging_fraction": 0.8,
                "feature_fraction": 0.8,
                "metric": None,
                "objective": "multiclass",
                "num_class": 4,
                "nthread": 10,
                "verbose": -1,
            }
    
    model = lgb.train(params, 
                      train_set=train_matrix, 
                      valid_sets=valid_matrix, 
                      num_boost_round=2000, 
                      verbose_eval=100, 
                      early_stopping_rounds=200,
                      feval=f1_score_vali)
    
    val_pred = model.predict(X_val, num_iteration=model.best_iteration)
    
    val_pred = np.argmax(val_pred, axis=1)
    cv_scores.append(f1_score(y_true=y_val, y_pred=val_pred, average='macro'))
    print(cv_scores)

print("lgb_scotrainre_list:{}".format(cv_scores))
print("lgb_score_mean:{}".format(np.mean(cv_scores)))
print("lgb_score_std:{}".format(np.std(cv_scores)))

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
"""使用lightgbm 5折交叉验证进行建模预测"""
cv_scores = []
for i, (train_index, valid_index) in enumerate(kf.split(X_train, y_train)):
    print('************************************ {} ************************************'.format(str(i+1)))
    X_train_split, y_train_split, X_val, y_val = X_train.iloc[train_index], y_train[train_index], X_train.iloc[valid_index], y_train[valid_index]
    
    train_matrix = lgb.Dataset(X_train_split, label=y_train_split)
    valid_matrix = lgb.Dataset(X_val, label=y_val)

    params = {
                "learning_rate": 0.1,
                "boosting": 'gbdt',  
                "lambda_l2": 0.1,
                "max_depth": -1,
                "num_leaves": 128,
                "bagging_fraction": 0.8,
                "feature_fraction": 0.8,
                "metric": None,
                "objective": "multiclass",
                "num_class": 4,
                "nthread": 10,
                "verbose": -1,
            }
    
    model = lgb.train(params, 
                      train_set=train_matrix, 
                      valid_sets=valid_matrix, 
                      num_boost_round=2000, 
                      verbose_eval=100, 
                      early_stopping_rounds=200,
                      feval=f1_score_vali)
    
    val_pred = model.predict(X_val, num_iteration=model.best_iteration)
    
    val_pred = np.argmax(val_pred, axis=1)
    cv_scores.append(f1_score(y_true=y_val, y_pred=val_pred, average='macro'))
    print(cv_scores)

print("lgb_scotrainre_list:{}".format(cv_scores))
print("lgb_score_mean:{}".format(np.mean(cv_scores)))
print("lgb_score_std:{}".format(np.std(cv_scores)))

复制代码

************************************ 1 ************************************
Training until validation scores don't improve for 200 rounds
[100]	valid_0's multi_logloss: 0.0408155	valid_0's f1_score: 0.966797
[200]	valid_0's multi_logloss: 0.0437957	valid_0's f1_score: 0.971239
Early stopping, best iteration is:
[96]	valid_0's multi_logloss: 0.0406453	valid_0's f1_score: 0.967452
[0.9674515729721614]
************************************ 2 ************************************
Training until validation scores don't improve for 200 rounds
[100]	valid_0's multi_logloss: 0.0472933	valid_0's f1_score: 0.965828
[200]	valid_0's multi_logloss: 0.0514952	valid_0's f1_score: 0.968138
Early stopping, best iteration is:
[87]	valid_0's multi_logloss: 0.0467472	valid_0's f1_score: 0.96567
[0.9674515729721614, 0.9656700872844327]
************************************ 3 ************************************
Training until validation scores don't improve for 200 rounds
[100]	valid_0's multi_logloss: 0.0378154	valid_0's f1_score: 0.971004
[200]	valid_0's multi_logloss: 0.0405053	valid_0's f1_score: 0.973736
Early stopping, best iteration is:
[93]	valid_0's multi_logloss: 0.037734	valid_0's f1_score: 0.970004
[0.9674515729721614, 0.9656700872844327, 0.9700043639844769]
************************************ 4 ************************************
Training until validation scores don't improve for 200 rounds
[100]	valid_0's multi_logloss: 0.0495142	valid_0's f1_score: 0.967106
[200]	valid_0's multi_logloss: 0.0542324	valid_0's f1_score: 0.969746
Early stopping, best iteration is:
[84]	valid_0's multi_logloss: 0.0490886	valid_0's f1_score: 0.965566
[0.9674515729721614, 0.9656700872844327, 0.9700043639844769, 0.9655663272378014]
************************************ 5 ************************************
Training until validation scores don't improve for 200 rounds
[100]	valid_0's multi_logloss: 0.0412544	valid_0's f1_score: 0.964054
[200]	valid_0's multi_logloss: 0.0443025	valid_0's f1_score: 0.965507
Early stopping, best iteration is:
[96]	valid_0's multi_logloss: 0.0411855	valid_0's f1_score: 0.963114
[0.9674515729721614, 0.9656700872844327, 0.9700043639844769, 0.9655663272378014, 0.9631137190307674]
lgb_scotrainre_list:[0.9674515729721614, 0.9656700872844327, 0.9700043639844769, 0.9655663272378014, 0.9631137190307674]
lgb_score_mean:0.9663612141019279
lgb_score_std:0.0022854824074775683

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
************************************ 1 ************************************
Training until validation scores don't improve for 200 rounds
[100]	valid_0's multi_logloss: 0.0408155	valid_0's f1_score: 0.966797
[200]	valid_0's multi_logloss: 0.0437957	valid_0's f1_score: 0.971239
Early stopping, best iteration is:
[96]	valid_0's multi_logloss: 0.0406453	valid_0's f1_score: 0.967452
[0.9674515729721614]
************************************ 2 ************************************
Training until validation scores don't improve for 200 rounds
[100]	valid_0's multi_logloss: 0.0472933	valid_0's f1_score: 0.965828
[200]	valid_0's multi_logloss: 0.0514952	valid_0's f1_score: 0.968138
Early stopping, best iteration is:
[87]	valid_0's multi_logloss: 0.0467472	valid_0's f1_score: 0.96567
[0.9674515729721614, 0.9656700872844327]
************************************ 3 ************************************
Training until validation scores don't improve for 200 rounds
[100]	valid_0's multi_logloss: 0.0378154	valid_0's f1_score: 0.971004
[200]	valid_0's multi_logloss: 0.0405053	valid_0's f1_score: 0.973736
Early stopping, best iteration is:
[93]	valid_0's multi_logloss: 0.037734	valid_0's f1_score: 0.970004
[0.9674515729721614, 0.9656700872844327, 0.9700043639844769]
************************************ 4 ************************************
Training until validation scores don't improve for 200 rounds
[100]	valid_0's multi_logloss: 0.0495142	valid_0's f1_score: 0.967106
[200]	valid_0's multi_logloss: 0.0542324	valid_0's f1_score: 0.969746
Early stopping, best iteration is:
[84]	valid_0's multi_logloss: 0.0490886	valid_0's f1_score: 0.965566
[0.9674515729721614, 0.9656700872844327, 0.9700043639844769, 0.9655663272378014]
************************************ 5 ************************************
Training until validation scores don't improve for 200 rounds
[100]	valid_0's multi_logloss: 0.0412544	valid_0's f1_score: 0.964054
[200]	valid_0's multi_logloss: 0.0443025	valid_0's f1_score: 0.965507
Early stopping, best iteration is:
[96]	valid_0's multi_logloss: 0.0411855	valid_0's f1_score: 0.963114
[0.9674515729721614, 0.9656700872844327, 0.9700043639844769, 0.9655663272378014, 0.9631137190307674]
lgb_scotrainre_list:[0.9674515729721614, 0.9656700872844327, 0.9700043639844769, 0.9655663272378014, 0.9631137190307674]
lgb_score_mean:0.9663612141019279
lgb_score_std:0.0022854824074775683

四、模型调参

1、贪心调参

先使用当前对模型影响最大的参数进行调优，达到当前参数下的模型最优化，再使用对模型影响次之的参数进行调优，如此下去，直到所有的参数调整完毕。
**缺点：**可能会调到局部最优而非全局最优

日常调参过程中的常用参数和调参顺序：
①：max_depth、num_leaves
②：min_data_in_leaf、min_child_weight
③：bagging_fraction、 feature_fraction、bagging_freq
④：reg_lambda、reg_alpha
⑤：min_split_gain

复制代码

from sklearn.model_selection import cross_val_score
# 调objective
best_obj = dict()
for obj in objective:
    model = LGBMRegressor(objective=obj)
    """预测并计算roc的相关指标"""
    score = cross_val_score(model, X_train, y_train, cv=5, scoring='f1').mean()
    best_obj[obj] = score

# num_leaves
best_leaves = dict()
for leaves in num_leaves:
    model = LGBMRegressor(objective=min(best_obj.items(), key=lambda x:x[1])[0], num_leaves=leaves)
    """预测并计算roc的相关指标"""
    score = cross_val_score(model, X_train, y_train, cv=5, scoring='f1').mean()
    best_leaves[leaves] = score

# max_depth
best_depth = dict()
for depth in max_depth:
    model = LGBMRegressor(objective=min(best_obj.items(), key=lambda x:x[1])[0],
                          num_leaves=min(best_leaves.items(), key=lambda x:x[1])[0],
                          max_depth=depth)
    """预测并计算roc的相关指标"""
    score = cross_val_score(model, X_train, y_train, cv=5, scoring='f1').mean()
    best_depth[depth] = score

"""
可依次将模型的参数通过上面的方式进行调整优化，并且通过可视化观察在每一个最优参数下模型的得分情况
"""

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
from sklearn.model_selection import cross_val_score
# 调objective
best_obj = dict()
for obj in objective:
    model = LGBMRegressor(objective=obj)
    """预测并计算roc的相关指标"""
    score = cross_val_score(model, X_train, y_train, cv=5, scoring='f1').mean()
    best_obj[obj] = score

# num_leaves
best_leaves = dict()
for leaves in num_leaves:
    model = LGBMRegressor(objective=min(best_obj.items(), key=lambda x:x[1])[0], num_leaves=leaves)
    """预测并计算roc的相关指标"""
    score = cross_val_score(model, X_train, y_train, cv=5, scoring='f1').mean()
    best_leaves[leaves] = score

# max_depth
best_depth = dict()
for depth in max_depth:
    model = LGBMRegressor(objective=min(best_obj.items(), key=lambda x:x[1])[0],
                          num_leaves=min(best_leaves.items(), key=lambda x:x[1])[0],
                          max_depth=depth)
    """预测并计算roc的相关指标"""
    score = cross_val_score(model, X_train, y_train, cv=5, scoring='f1').mean()
    best_depth[depth] = score

"""
可依次将模型的参数通过上面的方式进行调整优化，并且通过可视化观察在每一个最优参数下模型的得分情况
"""

2、网格搜索

sklearn 提供GridSearchCV用于进行网格搜索，只需要把模型的参数输进去，就能给出最优化的结果和参数。相比起贪心调参，网格搜索的结果会更优，但是网格搜索只适合于小数据集，一旦数据的量级上去了，很难得出结果。

复制代码

"""通过网格搜索确定最优参数"""
from sklearn.model_selection import GridSearchCV

def get_best_cv_params(learning_rate=0.1, n_estimators=581, num_leaves=31, max_depth=-1, bagging_fraction=1.0, 
                       feature_fraction=1.0, bagging_freq=0, min_data_in_leaf=20, min_child_weight=0.001, 
                       min_split_gain=0, reg_lambda=0, reg_alpha=0, param_grid=None):
    # 设置5折交叉验证
    cv_fold = KFold(n_splits=5, shuffle=True, random_state=2021)

model_lgb = lgb.LGBMClassifier(learning_rate=learning_rate,
                                   n_estimators=n_estimators,
                                   num_leaves=num_leaves,
                                   max_depth=max_depth,
                                   bagging_fraction=bagging_fraction,
                                   feature_fraction=feature_fraction,
                                   bagging_freq=bagging_freq,
                                   min_data_in_leaf=min_data_in_leaf,
                                   min_child_weight=min_child_weight,
                                   min_split_gain=min_split_gain,
                                   reg_lambda=reg_lambda,
                                   reg_alpha=reg_alpha,
                                   n_jobs= 8
                                  )

f1 = make_scorer(f1_score, average='micro')
    grid_search = GridSearchCV(estimator=model_lgb, 
                               cv=cv_fold,
                               param_grid=param_grid,
                               scoring=f1

)
    grid_search.fit(X_train, y_train)

print('模型当前最优参数为:{}'.format(grid_search.best_params_))
    print('模型当前最优得分为:{}'.format(grid_search.best_score_))

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
"""通过网格搜索确定最优参数"""
from sklearn.model_selection import GridSearchCV

def get_best_cv_params(learning_rate=0.1, n_estimators=581, num_leaves=31, max_depth=-1, bagging_fraction=1.0, 
                       feature_fraction=1.0, bagging_freq=0, min_data_in_leaf=20, min_child_weight=0.001, 
                       min_split_gain=0, reg_lambda=0, reg_alpha=0, param_grid=None):
    # 设置5折交叉验证
    cv_fold = KFold(n_splits=5, shuffle=True, random_state=2021)

    model_lgb = lgb.LGBMClassifier(learning_rate=learning_rate,
                                   n_estimators=n_estimators,
                                   num_leaves=num_leaves,
                                   max_depth=max_depth,
                                   bagging_fraction=bagging_fraction,
                                   feature_fraction=feature_fraction,
                                   bagging_freq=bagging_freq,
                                   min_data_in_leaf=min_data_in_leaf,
                                   min_child_weight=min_child_weight,
                                   min_split_gain=min_split_gain,
                                   reg_lambda=reg_lambda,
                                   reg_alpha=reg_alpha,
                                   n_jobs= 8
                                  )

    f1 = make_scorer(f1_score, average='micro')
    grid_search = GridSearchCV(estimator=model_lgb, 
                               cv=cv_fold,
                               param_grid=param_grid,
                               scoring=f1

                              )
    grid_search.fit(X_train, y_train)

    print('模型当前最优参数为:{}'.format(grid_search.best_params_))
    print('模型当前最优得分为:{}'.format(grid_search.best_score_))

复制代码

"""以下代码未运行，耗时较长，请谨慎运行，且每一步的最优参数需要在下一步进行手动更新，请注意"""

"""
需要注意一下的是，除了获取上面的获取num_boost_round时候用的是原生的lightgbm（因为要用自带的cv）
下面配合GridSearchCV时必须使用sklearn接口的lightgbm。
"""
"""设置n_estimators 为581，调整num_leaves和max_depth，这里选择先粗调再细调"""
lgb_params = {'num_leaves': range(10, 80, 5), 'max_depth': range(3,10,2)}
get_best_cv_params(learning_rate=0.1, n_estimators=581, num_leaves=None, max_depth=None, min_data_in_leaf=20, 
                   min_child_weight=0.001,bagging_fraction=1.0, feature_fraction=1.0, bagging_freq=0, 
                   min_split_gain=0, reg_lambda=0, reg_alpha=0, param_grid=lgb_params)

"""num_leaves为30，max_depth为7，进一步细调num_leaves和max_depth"""
lgb_params = {'num_leaves': range(25, 35, 1), 'max_depth': range(5,9,1)}
get_best_cv_params(learning_rate=0.1, n_estimators=85, num_leaves=None, max_depth=None, min_data_in_leaf=20, 
                   min_child_weight=0.001,bagging_fraction=1.0, feature_fraction=1.0, bagging_freq=0, 
                   min_split_gain=0, reg_lambda=0, reg_alpha=0, param_grid=lgb_params)

"""
确定min_data_in_leaf为45，min_child_weight为0.001 ，下面进行bagging_fraction、feature_fraction和bagging_freq的调参
"""
lgb_params = {'bagging_fraction': [i/10 for i in range(5,10,1)], 
              'feature_fraction': [i/10 for i in range(5,10,1)],
              'bagging_freq': range(0,81,10)
             }
get_best_cv_params(learning_rate=0.1, n_estimators=85, num_leaves=29, max_depth=7, min_data_in_leaf=45, 
                   min_child_weight=0.001,bagging_fraction=None, feature_fraction=None, bagging_freq=None, 
                   min_split_gain=0, reg_lambda=0, reg_alpha=0, param_grid=lgb_params)

"""
确定bagging_fraction为0.4、feature_fraction为0.6、bagging_freq为 ，下面进行reg_lambda、reg_alpha的调参
"""
lgb_params = {'reg_lambda': [0,0.001,0.01,0.03,0.08,0.3,0.5], 'reg_alpha': [0,0.001,0.01,0.03,0.08,0.3,0.5]}
get_best_cv_params(learning_rate=0.1, n_estimators=85, num_leaves=29, max_depth=7, min_data_in_leaf=45, 
                   min_child_weight=0.001,bagging_fraction=0.9, feature_fraction=0.9, bagging_freq=40, 
                   min_split_gain=0, reg_lambda=None, reg_alpha=None, param_grid=lgb_params)

"""
确定reg_lambda、reg_alpha都为0，下面进行min_split_gain的调参
"""
lgb_params = {'min_split_gain': [i/10 for i in range(0,11,1)]}
get_best_cv_params(learning_rate=0.1, n_estimators=85, num_leaves=29, max_depth=7, min_data_in_leaf=45, 
                   min_child_weight=0.001,bagging_fraction=0.9, feature_fraction=0.9, bagging_freq=40, 
                   min_split_gain=None, reg_lambda=0, reg_alpha=0, param_grid=lgb_params)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
"""以下代码未运行，耗时较长，请谨慎运行，且每一步的最优参数需要在下一步进行手动更新，请注意"""

"""
需要注意一下的是，除了获取上面的获取num_boost_round时候用的是原生的lightgbm（因为要用自带的cv）
下面配合GridSearchCV时必须使用sklearn接口的lightgbm。
"""
"""设置n_estimators 为581，调整num_leaves和max_depth，这里选择先粗调再细调"""
lgb_params = {'num_leaves': range(10, 80, 5), 'max_depth': range(3,10,2)}
get_best_cv_params(learning_rate=0.1, n_estimators=581, num_leaves=None, max_depth=None, min_data_in_leaf=20, 
                   min_child_weight=0.001,bagging_fraction=1.0, feature_fraction=1.0, bagging_freq=0, 
                   min_split_gain=0, reg_lambda=0, reg_alpha=0, param_grid=lgb_params)

"""num_leaves为30，max_depth为7，进一步细调num_leaves和max_depth"""
lgb_params = {'num_leaves': range(25, 35, 1), 'max_depth': range(5,9,1)}
get_best_cv_params(learning_rate=0.1, n_estimators=85, num_leaves=None, max_depth=None, min_data_in_leaf=20, 
                   min_child_weight=0.001,bagging_fraction=1.0, feature_fraction=1.0, bagging_freq=0, 
                   min_split_gain=0, reg_lambda=0, reg_alpha=0, param_grid=lgb_params)

"""
确定min_data_in_leaf为45，min_child_weight为0.001 ，下面进行bagging_fraction、feature_fraction和bagging_freq的调参
"""
lgb_params = {'bagging_fraction': [i/10 for i in range(5,10,1)], 
              'feature_fraction': [i/10 for i in range(5,10,1)],
              'bagging_freq': range(0,81,10)
             }
get_best_cv_params(learning_rate=0.1, n_estimators=85, num_leaves=29, max_depth=7, min_data_in_leaf=45, 
                   min_child_weight=0.001,bagging_fraction=None, feature_fraction=None, bagging_freq=None, 
                   min_split_gain=0, reg_lambda=0, reg_alpha=0, param_grid=lgb_params)

"""
确定bagging_fraction为0.4、feature_fraction为0.6、bagging_freq为 ，下面进行reg_lambda、reg_alpha的调参
"""
lgb_params = {'reg_lambda': [0,0.001,0.01,0.03,0.08,0.3,0.5], 'reg_alpha': [0,0.001,0.01,0.03,0.08,0.3,0.5]}
get_best_cv_params(learning_rate=0.1, n_estimators=85, num_leaves=29, max_depth=7, min_data_in_leaf=45, 
                   min_child_weight=0.001,bagging_fraction=0.9, feature_fraction=0.9, bagging_freq=40, 
                   min_split_gain=0, reg_lambda=None, reg_alpha=None, param_grid=lgb_params)

"""
确定reg_lambda、reg_alpha都为0，下面进行min_split_gain的调参
"""
lgb_params = {'min_split_gain': [i/10 for i in range(0,11,1)]}
get_best_cv_params(learning_rate=0.1, n_estimators=85, num_leaves=29, max_depth=7, min_data_in_leaf=45, 
                   min_child_weight=0.001,bagging_fraction=0.9, feature_fraction=0.9, bagging_freq=40, 
                   min_split_gain=None, reg_lambda=0, reg_alpha=0, param_grid=lgb_params)

复制代码

"""
参数确定好了以后，我们设置一个比较小的learning_rate 0.005，来确定最终的num_boost_round
"""
# 设置5折交叉验证
# cv_fold = StratifiedKFold(n_splits=5, random_state=0, shuffle=True, )
final_params = {
                'boosting_type': 'gbdt',
                'learning_rate': 0.01,
                'num_leaves': 29,
                'max_depth': 7,
                'objective': 'multiclass',
                'num_class': 4,
                'min_data_in_leaf':45,
                'min_child_weight':0.001,
                'bagging_fraction': 0.9,
                'feature_fraction': 0.9,
                'bagging_freq': 40,
                'min_split_gain': 0,
                'reg_lambda':0,
                'reg_alpha':0,
                'nthread': 6
               }

cv_result = lgb.cv(train_set=lgb_train,
                   early_stopping_rounds=20,
                   num_boost_round=5000,
                   nfold=5,
                   stratified=True,
                   shuffle=True,
                   params=final_params,
                   feval=f1_score_vali,
                   seed=0,
                  )

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
"""
参数确定好了以后，我们设置一个比较小的learning_rate 0.005，来确定最终的num_boost_round
"""
# 设置5折交叉验证
# cv_fold = StratifiedKFold(n_splits=5, random_state=0, shuffle=True, )
final_params = {
                'boosting_type': 'gbdt',
                'learning_rate': 0.01,
                'num_leaves': 29,
                'max_depth': 7,
                'objective': 'multiclass',
                'num_class': 4,
                'min_data_in_leaf':45,
                'min_child_weight':0.001,
                'bagging_fraction': 0.9,
                'feature_fraction': 0.9,
                'bagging_freq': 40,
                'min_split_gain': 0,
                'reg_lambda':0,
                'reg_alpha':0,
                'nthread': 6
               }

cv_result = lgb.cv(train_set=lgb_train,
                   early_stopping_rounds=20,
                   num_boost_round=5000,
                   nfold=5,
                   stratified=True,
                   shuffle=True,
                   params=final_params,
                   feval=f1_score_vali,
                   seed=0,
                  )

在实际调整过程中，可先设置一个较大的学习率（上面的例子中0.1），通过Lgb原生的cv函数进行树个数的确定，之后再通过上面的实例代码进行参数的调整优化。

最后针对最优的参数设置一个较小的学习率（例如0.05），同样通过cv函数确定树的个数，确定最终的参数。

需要注意的是，针对大数据集，上面每一层参数的调整都需要耗费较长时间。

3、贝叶斯调参

给定优化的目标函数(广义的函数，只需指定输入和输出即可，无需知道内部结构以及数学性质)，通过不断地添加样本点来更新目标函数的后验分布(高斯过程,直到后验分布基本贴合于真实分布）。简单的说，就是考虑了上一次参数的信息，从而更好的调整当前的参数。
调参步骤：

定义优化函数(rf_cv）
建立模型
定义待优化的参数
得到优化结果，并返回要优化的分数指标

复制代码

from sklearn.model_selection import cross_val_score
from sklearn.metrics import make_scorer

"""定义优化函数"""
def rf_cv_lgb(num_leaves, max_depth, bagging_fraction, feature_fraction, bagging_freq, min_data_in_leaf, 
              min_child_weight, min_split_gain, reg_lambda, reg_alpha):
    # 建立模型
    model_lgb = lgb.LGBMClassifier(boosting_type='gbdt', objective='multiclass', num_class=4,
                                   learning_rate=0.1, n_estimators=5000,
                                   num_leaves=int(num_leaves), max_depth=int(max_depth), 
                                   bagging_fraction=round(bagging_fraction, 2), feature_fraction=round(feature_fraction, 2),
                                   bagging_freq=int(bagging_freq), min_data_in_leaf=int(min_data_in_leaf),
                                   min_child_weight=min_child_weight, min_split_gain=min_split_gain,
                                   reg_lambda=reg_lambda, reg_alpha=reg_alpha,
                                   n_jobs= 8
                                  )
    f1 = make_scorer(f1_score, average='micro')
    val = cross_val_score(model_lgb, X_train_split, y_train_split, cv=5, scoring=f1).mean()

return val

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
from sklearn.model_selection import cross_val_score
from sklearn.metrics import make_scorer

"""定义优化函数"""
def rf_cv_lgb(num_leaves, max_depth, bagging_fraction, feature_fraction, bagging_freq, min_data_in_leaf, 
              min_child_weight, min_split_gain, reg_lambda, reg_alpha):
    # 建立模型
    model_lgb = lgb.LGBMClassifier(boosting_type='gbdt', objective='multiclass', num_class=4,
                                   learning_rate=0.1, n_estimators=5000,
                                   num_leaves=int(num_leaves), max_depth=int(max_depth), 
                                   bagging_fraction=round(bagging_fraction, 2), feature_fraction=round(feature_fraction, 2),
                                   bagging_freq=int(bagging_freq), min_data_in_leaf=int(min_data_in_leaf),
                                   min_child_weight=min_child_weight, min_split_gain=min_split_gain,
                                   reg_lambda=reg_lambda, reg_alpha=reg_alpha,
                                   n_jobs= 8
                                  )
    f1 = make_scorer(f1_score, average='micro')
    val = cross_val_score(model_lgb, X_train_split, y_train_split, cv=5, scoring=f1).mean()

    return val

复制代码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
from bayes_opt import BayesianOptimization
"""定义优化参数"""
bayes_lgb = BayesianOptimization(
    rf_cv_lgb, 
    {
        'num_leaves':(10, 200),
        'max_depth':(3, 20),
        'bagging_fraction':(0.5, 1.0),
        'feature_fraction':(0.5, 1.0),
        'bagging_freq':(0, 100),
        'min_data_in_leaf':(10,100),
        'min_child_weight':(0, 10),
        'min_split_gain':(0.0, 1.0),
        'reg_alpha':(0.0, 10),
        'reg_lambda':(0.0, 10),
    }
)

"""开始优化"""
bayes_lgb.maximize(n_iter=10)

复制代码

|   iter    |  target   | baggin... | baggin... | featur... | max_depth | min_ch... | min_da... | min_sp... | num_le... | reg_alpha | reg_la... |
-------------------------------------------------------------------------------------------------------------------------------------------------
[LightGBM] [Warning] feature_fraction is set=0.52, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.52
[LightGBM] [Warning] min_data_in_leaf is set=77, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=77
[LightGBM] [Warning] bagging_fraction is set=0.85, subsample=1.0 will be ignored. Current value: bagging_fraction=0.85
[LightGBM] [Warning] bagging_freq is set=97, subsample_freq=0 will be ignored. Current value: bagging_freq=97
[LightGBM] [Warning] feature_fraction is set=0.52, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.52
[LightGBM] [Warning] min_data_in_leaf is set=77, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=77
[LightGBM] [Warning] bagging_fraction is set=0.85, subsample=1.0 will be ignored. Current value: bagging_fraction=0.85
[LightGBM] [Warning] bagging_freq is set=97, subsample_freq=0 will be ignored. Current value: bagging_freq=97
[LightGBM] [Warning] feature_fraction is set=0.52, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.52
[LightGBM] [Warning] min_data_in_leaf is set=77, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=77
[LightGBM] [Warning] bagging_fraction is set=0.85, subsample=1.0 will be ignored. Current value: bagging_fraction=0.85
[LightGBM] [Warning] bagging_freq is set=97, subsample_freq=0 will be ignored. Current value: bagging_freq=97
[LightGBM] [Warning] feature_fraction is set=0.52, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.52
[LightGBM] [Warning] min_data_in_leaf is set=77, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=77
[LightGBM] [Warning] bagging_fraction is set=0.85, subsample=1.0 will be ignored. Current value: bagging_fraction=0.85
[LightGBM] [Warning] bagging_freq is set=97, subsample_freq=0 will be ignored. Current value: bagging_freq=97
[LightGBM] [Warning] feature_fraction is set=0.52, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.52
[LightGBM] [Warning] min_data_in_leaf is set=77, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=77
[LightGBM] [Warning] bagging_fraction is set=0.85, subsample=1.0 will be ignored. Current value: bagging_fraction=0.85
[LightGBM] [Warning] bagging_freq is set=97, subsample_freq=0 will be ignored. Current value: bagging_freq=97
| [0m 1       [0m | [0m 0.9739  [0m | [0m 0.8546  [0m | [0m 97.23   [0m | [0m 0.5183  [0m | [0m 15.71   [0m | [0m 5.494   [0m | [0m 77.91   [0m | [0m 0.7621  [0m | [0m 23.36   [0m | [0m 6.242   [0m | [0m 4.806   [0m |
[LightGBM] [Warning] feature_fraction is set=0.64, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.64
[LightGBM] [Warning] min_data_in_leaf is set=82, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=82
[LightGBM] [Warning] bagging_fraction is set=0.82, subsample=1.0 will be ignored. Current value: bagging_fraction=0.82
[LightGBM] [Warning] bagging_freq is set=59, subsample_freq=0 will be ignored. Current value: bagging_freq=59
[LightGBM] [Warning] feature_fraction is set=0.64, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.64
[LightGBM] [Warning] min_data_in_leaf is set=82, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=82
[LightGBM] [Warning] bagging_fraction is set=0.82, subsample=1.0 will be ignored. Current value: bagging_fraction=0.82
[LightGBM] [Warning] bagging_freq is set=59, subsample_freq=0 will be ignored. Current value: bagging_freq=59
[LightGBM] [Warning] feature_fraction is set=0.64, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.64
[LightGBM] [Warning] min_data_in_leaf is set=82, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=82
[LightGBM] [Warning] bagging_fraction is set=0.82, subsample=1.0 will be ignored. Current value: bagging_fraction=0.82
[LightGBM] [Warning] bagging_freq is set=59, subsample_freq=0 will be ignored. Current value: bagging_freq=59
[LightGBM] [Warning] feature_fraction is set=0.64, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.64
[LightGBM] [Warning] min_data_in_leaf is set=82, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=82
[LightGBM] [Warning] bagging_fraction is set=0.82, subsample=1.0 will be ignored. Current value: bagging_fraction=0.82
[LightGBM] [Warning] bagging_freq is set=59, subsample_freq=0 will be ignored. Current value: bagging_freq=59
[LightGBM] [Warning] feature_fraction is set=0.64, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.64
[LightGBM] [Warning] min_data_in_leaf is set=82, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=82
[LightGBM] [Warning] bagging_fraction is set=0.82, subsample=1.0 will be ignored. Current value: bagging_fraction=0.82
[LightGBM] [Warning] bagging_freq is set=59, subsample_freq=0 will be ignored. Current value: bagging_freq=59
| [95m 2       [0m | [95m 0.9751  [0m | [95m 0.8192  [0m | [95m 59.49   [0m | [95m 0.6409  [0m | [95m 6.577   [0m | [95m 1.987   [0m | [95m 82.2    [0m | [95m 0.667   [0m | [95m 193.8   [0m | [95m 4.968   [0m | [95m 7.509   [0m |
[LightGBM] [Warning] feature_fraction is set=0.83, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.83
[LightGBM] [Warning] min_data_in_leaf is set=16, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=16
[LightGBM] [Warning] bagging_fraction is set=0.51, subsample=1.0 will be ignored. Current value: bagging_fraction=0.51
[LightGBM] [Warning] bagging_freq is set=16, subsample_freq=0 will be ignored. Current value: bagging_freq=16
[LightGBM] [Warning] feature_fraction is set=0.83, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.83
[LightGBM] [Warning] min_data_in_leaf is set=16, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=16
[LightGBM] [Warning] bagging_fraction is set=0.51, subsample=1.0 will be ignored. Current value: bagging_fraction=0.51
[LightGBM] [Warning] bagging_freq is set=16, subsample_freq=0 will be ignored. Current value: bagging_freq=16
[LightGBM] [Warning] feature_fraction is set=0.83, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.83
[LightGBM] [Warning] min_data_in_leaf is set=16, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=16
[LightGBM] [Warning] bagging_fraction is set=0.51, subsample=1.0 will be ignored. Current value: bagging_fraction=0.51
[LightGBM] [Warning] bagging_freq is set=16, subsample_freq=0 will be ignored. Current value: bagging_freq=16
[LightGBM] [Warning] feature_fraction is set=0.83, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.83
[LightGBM] [Warning] min_data_in_leaf is set=16, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=16
[LightGBM] [Warning] bagging_fraction is set=0.51, subsample=1.0 will be ignored. Current value: bagging_fraction=0.51
[LightGBM] [Warning] bagging_freq is set=16, subsample_freq=0 will be ignored. Current value: bagging_freq=16
[LightGBM] [Warning] feature_fraction is set=0.83, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.83
[LightGBM] [Warning] min_data_in_leaf is set=16, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=16
[LightGBM] [Warning] bagging_fraction is set=0.51, subsample=1.0 will be ignored. Current value: bagging_fraction=0.51
[LightGBM] [Warning] bagging_freq is set=16, subsample_freq=0 will be ignored. Current value: bagging_freq=16
| [95m 3       [0m | [95m 0.9853  [0m | [95m 0.5143  [0m | [95m 16.62   [0m | [95m 0.8255  [0m | [95m 18.26   [0m | [95m 1.743   [0m | [95m 16.21   [0m | [95m 0.0749  [0m | [95m 76.97   [0m | [95m 1.132   [0m | [95m 8.145   [0m |
[LightGBM] [Warning] feature_fraction is set=0.58, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.58
[LightGBM] [Warning] min_data_in_leaf is set=73, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=73
[LightGBM] [Warning] bagging_fraction is set=0.88, subsample=1.0 will be ignored. Current value: bagging_fraction=0.88
[LightGBM] [Warning] bagging_freq is set=77, subsample_freq=0 will be ignored. Current value: bagging_freq=77
[LightGBM] [Warning] feature_fraction is set=0.58, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.58
[LightGBM] [Warning] min_data_in_leaf is set=73, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=73
[LightGBM] [Warning] bagging_fraction is set=0.88, subsample=1.0 will be ignored. Current value: bagging_fraction=0.88
[LightGBM] [Warning] bagging_freq is set=77, subsample_freq=0 will be ignored. Current value: bagging_freq=77
[LightGBM] [Warning] feature_fraction is set=0.58, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.58
[LightGBM] [Warning] min_data_in_leaf is set=73, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=73
[LightGBM] [Warning] bagging_fraction is set=0.88, subsample=1.0 will be ignored. Current value: bagging_fraction=0.88
[LightGBM] [Warning] bagging_freq is set=77, subsample_freq=0 will be ignored. Current value: bagging_freq=77
[LightGBM] [Warning] feature_fraction is set=0.58, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.58
[LightGBM] [Warning] min_data_in_leaf is set=73, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=73
[LightGBM] [Warning] bagging_fraction is set=0.88, subsample=1.0 will be ignored. Current value: bagging_fraction=0.88
[LightGBM] [Warning] bagging_freq is set=77, subsample_freq=0 will be ignored. Current value: bagging_freq=77
[LightGBM] [Warning] feature_fraction is set=0.58, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.58
[LightGBM] [Warning] min_data_in_leaf is set=73, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=73
[LightGBM] [Warning] bagging_fraction is set=0.88, subsample=1.0 will be ignored. Current value: bagging_fraction=0.88
[LightGBM] [Warning] bagging_freq is set=77, subsample_freq=0 will be ignored. Current value: bagging_freq=77
| [0m 4       [0m | [0m 0.9806  [0m | [0m 0.8751  [0m | [0m 77.71   [0m | [0m 0.5783  [0m | [0m 3.271   [0m | [0m 6.331   [0m | [0m 73.03   [0m | [0m 0.03976 [0m | [0m 33.28   [0m | [0m 5.119   [0m | [0m 9.69    [0m |
[LightGBM] [Warning] feature_fraction is set=0.59, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.59
[LightGBM] [Warning] min_data_in_leaf is set=75, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=75
[LightGBM] [Warning] bagging_fraction is set=0.83, subsample=1.0 will be ignored. Current value: bagging_fraction=0.83
[LightGBM] [Warning] bagging_freq is set=80, subsample_freq=0 will be ignored. Current value: bagging_freq=80
[LightGBM] [Warning] feature_fraction is set=0.59, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.59
[LightGBM] [Warning] min_data_in_leaf is set=75, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=75
[LightGBM] [Warning] bagging_fraction is set=0.83, subsample=1.0 will be ignored. Current value: bagging_fraction=0.83
[LightGBM] [Warning] bagging_freq is set=80, subsample_freq=0 will be ignored. Current value: bagging_freq=80
[LightGBM] [Warning] feature_fraction is set=0.59, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.59
[LightGBM] [Warning] min_data_in_leaf is set=75, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=75
[LightGBM] [Warning] bagging_fraction is set=0.83, subsample=1.0 will be ignored. Current value: bagging_fraction=0.83
[LightGBM] [Warning] bagging_freq is set=80, subsample_freq=0 will be ignored. Current value: bagging_freq=80
[LightGBM] [Warning] feature_fraction is set=0.59, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.59
[LightGBM] [Warning] min_data_in_leaf is set=75, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=75
[LightGBM] [Warning] bagging_fraction is set=0.83, subsample=1.0 will be ignored. Current value: bagging_fraction=0.83
[LightGBM] [Warning] bagging_freq is set=80, subsample_freq=0 will be ignored. Current value: bagging_freq=80
[LightGBM] [Warning] feature_fraction is set=0.59, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.59
[LightGBM] [Warning] min_data_in_leaf is set=75, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=75
[LightGBM] [Warning] bagging_fraction is set=0.83, subsample=1.0 will be ignored. Current value: bagging_fraction=0.83
[LightGBM] [Warning] bagging_freq is set=80, subsample_freq=0 will be ignored. Current value: bagging_freq=80
| [0m 5       [0m | [0m 0.9776  [0m | [0m 0.8291  [0m | [0m 80.11   [0m | [0m 0.5907  [0m | [0m 6.362   [0m | [0m 5.753   [0m | [0m 75.3    [0m | [0m 0.781   [0m | [0m 139.7   [0m | [0m 1.716   [0m | [0m 6.868   [0m |
[LightGBM] [Warning] feature_fraction is set=0.7, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.7
[LightGBM] [Warning] min_data_in_leaf is set=71, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=71
[LightGBM] [Warning] bagging_fraction is set=0.73, subsample=1.0 will be ignored. Current value: bagging_fraction=0.73
[LightGBM] [Warning] bagging_freq is set=75, subsample_freq=0 will be ignored. Current value: bagging_freq=75
[LightGBM] [Warning] feature_fraction is set=0.7, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.7
[LightGBM] [Warning] min_data_in_leaf is set=71, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=71
[LightGBM] [Warning] bagging_fraction is set=0.73, subsample=1.0 will be ignored. Current value: bagging_fraction=0.73
[LightGBM] [Warning] bagging_freq is set=75, subsample_freq=0 will be ignored. Current value: bagging_freq=75
[LightGBM] [Warning] feature_fraction is set=0.7, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.7
[LightGBM] [Warning] min_data_in_leaf is set=71, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=71
[LightGBM] [Warning] bagging_fraction is set=0.73, subsample=1.0 will be ignored. Current value: bagging_fraction=0.73
[LightGBM] [Warning] bagging_freq is set=75, subsample_freq=0 will be ignored. Current value: bagging_freq=75
[LightGBM] [Warning] feature_fraction is set=0.7, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.7
[LightGBM] [Warning] min_data_in_leaf is set=71, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=71
[LightGBM] [Warning] bagging_fraction is set=0.73, subsample=1.0 will be ignored. Current value: bagging_fraction=0.73
[LightGBM] [Warning] bagging_freq is set=75, subsample_freq=0 will be ignored. Current value: bagging_freq=75
[LightGBM] [Warning] feature_fraction is set=0.7, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.7
[LightGBM] [Warning] min_data_in_leaf is set=71, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=71
[LightGBM] [Warning] bagging_fraction is set=0.73, subsample=1.0 will be ignored. Current value: bagging_fraction=0.73
[LightGBM] [Warning] bagging_freq is set=75, subsample_freq=0 will be ignored. Current value: bagging_freq=75
| [0m 6       [0m | [0m 0.9757  [0m | [0m 0.7278  [0m | [0m 75.73   [0m | [0m 0.6989  [0m | [0m 7.171   [0m | [0m 5.236   [0m | [0m 71.18   [0m | [0m 0.3932  [0m | [0m 34.32   [0m | [0m 5.842   [0m | [0m 8.412   [0m |
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=52, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=52
[LightGBM] [Warning] bagging_fraction is set=0.82, subsample=1.0 will be ignored. Current value: bagging_fraction=0.82
[LightGBM] [Warning] bagging_freq is set=82, subsample_freq=0 will be ignored. Current value: bagging_freq=82
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=52, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=52
[LightGBM] [Warning] bagging_fraction is set=0.82, subsample=1.0 will be ignored. Current value: bagging_fraction=0.82
[LightGBM] [Warning] bagging_freq is set=82, subsample_freq=0 will be ignored. Current value: bagging_freq=82
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=52, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=52
[LightGBM] [Warning] bagging_fraction is set=0.82, subsample=1.0 will be ignored. Current value: bagging_fraction=0.82
[LightGBM] [Warning] bagging_freq is set=82, subsample_freq=0 will be ignored. Current value: bagging_freq=82
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=52, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=52
[LightGBM] [Warning] bagging_fraction is set=0.82, subsample=1.0 will be ignored. Current value: bagging_fraction=0.82
[LightGBM] [Warning] bagging_freq is set=82, subsample_freq=0 will be ignored. Current value: bagging_freq=82
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=52, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=52
[LightGBM] [Warning] bagging_fraction is set=0.82, subsample=1.0 will be ignored. Current value: bagging_fraction=0.82
[LightGBM] [Warning] bagging_freq is set=82, subsample_freq=0 will be ignored. Current value: bagging_freq=82
| [0m 7       [0m | [0m 0.9688  [0m | [0m 0.823   [0m | [0m 82.94   [0m | [0m 0.8162  [0m | [0m 4.985   [0m | [0m 5.682   [0m | [0m 52.74   [0m | [0m 0.8187  [0m | [0m 137.4   [0m | [0m 9.497   [0m | [0m 7.829   [0m |
[LightGBM] [Warning] feature_fraction is set=0.94, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.94
[LightGBM] [Warning] min_data_in_leaf is set=90, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=90
[LightGBM] [Warning] bagging_fraction is set=0.91, subsample=1.0 will be ignored. Current value: bagging_fraction=0.91
[LightGBM] [Warning] bagging_freq is set=66, subsample_freq=0 will be ignored. Current value: bagging_freq=66
[LightGBM] [Warning] feature_fraction is set=0.94, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.94
[LightGBM] [Warning] min_data_in_leaf is set=90, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=90
[LightGBM] [Warning] bagging_fraction is set=0.91, subsample=1.0 will be ignored. Current value: bagging_fraction=0.91
[LightGBM] [Warning] bagging_freq is set=66, subsample_freq=0 will be ignored. Current value: bagging_freq=66
[LightGBM] [Warning] feature_fraction is set=0.94, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.94
[LightGBM] [Warning] min_data_in_leaf is set=90, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=90
[LightGBM] [Warning] bagging_fraction is set=0.91, subsample=1.0 will be ignored. Current value: bagging_fraction=0.91
[LightGBM] [Warning] bagging_freq is set=66, subsample_freq=0 will be ignored. Current value: bagging_freq=66
[LightGBM] [Warning] feature_fraction is set=0.94, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.94
[LightGBM] [Warning] min_data_in_leaf is set=90, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=90
[LightGBM] [Warning] bagging_fraction is set=0.91, subsample=1.0 will be ignored. Current value: bagging_fraction=0.91
[LightGBM] [Warning] bagging_freq is set=66, subsample_freq=0 will be ignored. Current value: bagging_freq=66
[LightGBM] [Warning] feature_fraction is set=0.94, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.94
[LightGBM] [Warning] min_data_in_leaf is set=90, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=90
[LightGBM] [Warning] bagging_fraction is set=0.91, subsample=1.0 will be ignored. Current value: bagging_fraction=0.91
[LightGBM] [Warning] bagging_freq is set=66, subsample_freq=0 will be ignored. Current value: bagging_freq=66
| [0m 8       [0m | [0m 0.9757  [0m | [0m 0.9121  [0m | [0m 66.45   [0m | [0m 0.9389  [0m | [0m 3.0     [0m | [0m 0.8741  [0m | [0m 90.29   [0m | [0m 0.06163 [0m | [0m 31.66   [0m | [0m 9.637   [0m | [0m 0.05176 [0m |
[LightGBM] [Warning] feature_fraction is set=0.7, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.7
[LightGBM] [Warning] min_data_in_leaf is set=42, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=42
[LightGBM] [Warning] bagging_fraction is set=0.91, subsample=1.0 will be ignored. Current value: bagging_fraction=0.91
[LightGBM] [Warning] bagging_freq is set=36, subsample_freq=0 will be ignored. Current value: bagging_freq=36
[LightGBM] [Warning] feature_fraction is set=0.7, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.7
[LightGBM] [Warning] min_data_in_leaf is set=42, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=42
[LightGBM] [Warning] bagging_fraction is set=0.91, subsample=1.0 will be ignored. Current value: bagging_fraction=0.91
[LightGBM] [Warning] bagging_freq is set=36, subsample_freq=0 will be ignored. Current value: bagging_freq=36
[LightGBM] [Warning] feature_fraction is set=0.7, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.7
[LightGBM] [Warning] min_data_in_leaf is set=42, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=42
[LightGBM] [Warning] bagging_fraction is set=0.91, subsample=1.0 will be ignored. Current value: bagging_fraction=0.91
[LightGBM] [Warning] bagging_freq is set=36, subsample_freq=0 will be ignored. Current value: bagging_freq=36
[LightGBM] [Warning] feature_fraction is set=0.7, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.7
[LightGBM] [Warning] min_data_in_leaf is set=42, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=42
[LightGBM] [Warning] bagging_fraction is set=0.91, subsample=1.0 will be ignored. Current value: bagging_fraction=0.91
[LightGBM] [Warning] bagging_freq is set=36, subsample_freq=0 will be ignored. Current value: bagging_freq=36
[LightGBM] [Warning] feature_fraction is set=0.7, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.7
[LightGBM] [Warning] min_data_in_leaf is set=42, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=42
[LightGBM] [Warning] bagging_fraction is set=0.91, subsample=1.0 will be ignored. Current value: bagging_fraction=0.91
[LightGBM] [Warning] bagging_freq is set=36, subsample_freq=0 will be ignored. Current value: bagging_freq=36
| [0m 9       [0m | [0m 0.9785  [0m | [0m 0.9148  [0m | [0m 36.21   [0m | [0m 0.7003  [0m | [0m 8.766   [0m | [0m 8.293   [0m | [0m 42.82   [0m | [0m 0.3236  [0m | [0m 100.6   [0m | [0m 6.452   [0m | [0m 8.626   [0m |
[LightGBM] [Warning] feature_fraction is set=0.84, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.84
[LightGBM] [Warning] min_data_in_leaf is set=14, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=14
[LightGBM] [Warning] bagging_fraction is set=0.71, subsample=1.0 will be ignored. Current value: bagging_fraction=0.71
[LightGBM] [Warning] bagging_freq is set=99, subsample_freq=0 will be ignored. Current value: bagging_freq=99
[LightGBM] [Warning] feature_fraction is set=0.84, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.84
[LightGBM] [Warning] min_data_in_leaf is set=14, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=14
[LightGBM] [Warning] bagging_fraction is set=0.71, subsample=1.0 will be ignored. Current value: bagging_fraction=0.71
[LightGBM] [Warning] bagging_freq is set=99, subsample_freq=0 will be ignored. Current value: bagging_freq=99
[LightGBM] [Warning] feature_fraction is set=0.84, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.84
[LightGBM] [Warning] min_data_in_leaf is set=14, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=14
[LightGBM] [Warning] bagging_fraction is set=0.71, subsample=1.0 will be ignored. Current value: bagging_fraction=0.71
[LightGBM] [Warning] bagging_freq is set=99, subsample_freq=0 will be ignored. Current value: bagging_freq=99
[LightGBM] [Warning] feature_fraction is set=0.84, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.84
[LightGBM] [Warning] min_data_in_leaf is set=14, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=14
[LightGBM] [Warning] bagging_fraction is set=0.71, subsample=1.0 will be ignored. Current value: bagging_fraction=0.71
[LightGBM] [Warning] bagging_freq is set=99, subsample_freq=0 will be ignored. Current value: bagging_freq=99
[LightGBM] [Warning] feature_fraction is set=0.84, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.84
[LightGBM] [Warning] min_data_in_leaf is set=14, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=14
[LightGBM] [Warning] bagging_fraction is set=0.71, subsample=1.0 will be ignored. Current value: bagging_fraction=0.71
[LightGBM] [Warning] bagging_freq is set=99, subsample_freq=0 will be ignored. Current value: bagging_freq=99
| [0m 10      [0m | [0m 0.9751  [0m | [0m 0.7051  [0m | [0m 99.73   [0m | [0m 0.8385  [0m | [0m 6.302   [0m | [0m 0.2434  [0m | [0m 14.85   [0m | [0m 0.6174  [0m | [0m 129.1   [0m | [0m 4.334   [0m | [0m 8.77    [0m |
[LightGBM] [Warning] feature_fraction is set=0.72, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.72
[LightGBM] [Warning] min_data_in_leaf is set=60, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=60
[LightGBM] [Warning] bagging_fraction is set=0.74, subsample=1.0 will be ignored. Current value: bagging_fraction=0.74
[LightGBM] [Warning] bagging_freq is set=86, subsample_freq=0 will be ignored. Current value: bagging_freq=86
[LightGBM] [Warning] feature_fraction is set=0.72, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.72
[LightGBM] [Warning] min_data_in_leaf is set=60, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=60
[LightGBM] [Warning] bagging_fraction is set=0.74, subsample=1.0 will be ignored. Current value: bagging_fraction=0.74
[LightGBM] [Warning] bagging_freq is set=86, subsample_freq=0 will be ignored. Current value: bagging_freq=86
[LightGBM] [Warning] feature_fraction is set=0.72, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.72
[LightGBM] [Warning] min_data_in_leaf is set=60, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=60
[LightGBM] [Warning] bagging_fraction is set=0.74, subsample=1.0 will be ignored. Current value: bagging_fraction=0.74
[LightGBM] [Warning] bagging_freq is set=86, subsample_freq=0 will be ignored. Current value: bagging_freq=86
[LightGBM] [Warning] feature_fraction is set=0.72, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.72
[LightGBM] [Warning] min_data_in_leaf is set=60, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=60
[LightGBM] [Warning] bagging_fraction is set=0.74, subsample=1.0 will be ignored. Current value: bagging_fraction=0.74
[LightGBM] [Warning] bagging_freq is set=86, subsample_freq=0 will be ignored. Current value: bagging_freq=86
[LightGBM] [Warning] feature_fraction is set=0.72, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.72
[LightGBM] [Warning] min_data_in_leaf is set=60, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=60
[LightGBM] [Warning] bagging_fraction is set=0.74, subsample=1.0 will be ignored. Current value: bagging_fraction=0.74
[LightGBM] [Warning] bagging_freq is set=86, subsample_freq=0 will be ignored. Current value: bagging_freq=86
| [0m 11      [0m | [0m 0.9744  [0m | [0m 0.7444  [0m | [0m 86.48   [0m | [0m 0.7192  [0m | [0m 17.36   [0m | [0m 8.871   [0m | [0m 60.56   [0m | [0m 0.6246  [0m | [0m 153.4   [0m | [0m 8.237   [0m | [0m 0.305   [0m |
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=74, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=74
[LightGBM] [Warning] bagging_fraction is set=0.85, subsample=1.0 will be ignored. Current value: bagging_fraction=0.85
[LightGBM] [Warning] bagging_freq is set=75, subsample_freq=0 will be ignored. Current value: bagging_freq=75
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=74, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=74
[LightGBM] [Warning] bagging_fraction is set=0.85, subsample=1.0 will be ignored. Current value: bagging_fraction=0.85
[LightGBM] [Warning] bagging_freq is set=75, subsample_freq=0 will be ignored. Current value: bagging_freq=75
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=74, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=74
[LightGBM] [Warning] bagging_fraction is set=0.85, subsample=1.0 will be ignored. Current value: bagging_fraction=0.85
[LightGBM] [Warning] bagging_freq is set=75, subsample_freq=0 will be ignored. Current value: bagging_freq=75
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=74, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=74
[LightGBM] [Warning] bagging_fraction is set=0.85, subsample=1.0 will be ignored. Current value: bagging_fraction=0.85
[LightGBM] [Warning] bagging_freq is set=75, subsample_freq=0 will be ignored. Current value: bagging_freq=75
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=74, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=74
[LightGBM] [Warning] bagging_fraction is set=0.85, subsample=1.0 will be ignored. Current value: bagging_fraction=0.85
[LightGBM] [Warning] bagging_freq is set=75, subsample_freq=0 will be ignored. Current value: bagging_freq=75
| [0m 12      [0m | [0m 0.9781  [0m | [0m 0.854   [0m | [0m 75.34   [0m | [0m 0.8216  [0m | [0m 19.38   [0m | [0m 9.529   [0m | [0m 74.56   [0m | [0m 0.7511  [0m | [0m 26.53   [0m | [0m 1.126   [0m | [0m 8.934   [0m |
[LightGBM] [Warning] feature_fraction is set=0.81, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.81
[LightGBM] [Warning] min_data_in_leaf is set=42, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=42
[LightGBM] [Warning] bagging_fraction is set=0.7, subsample=1.0 will be ignored. Current value: bagging_fraction=0.7
[LightGBM] [Warning] bagging_freq is set=47, subsample_freq=0 will be ignored. Current value: bagging_freq=47
[LightGBM] [Warning] feature_fraction is set=0.81, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.81
[LightGBM] [Warning] min_data_in_leaf is set=42, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=42
[LightGBM] [Warning] bagging_fraction is set=0.7, subsample=1.0 will be ignored. Current value: bagging_fraction=0.7
[LightGBM] [Warning] bagging_freq is set=47, subsample_freq=0 will be ignored. Current value: bagging_freq=47
[LightGBM] [Warning] feature_fraction is set=0.81, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.81
[LightGBM] [Warning] min_data_in_leaf is set=42, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=42
[LightGBM] [Warning] bagging_fraction is set=0.7, subsample=1.0 will be ignored. Current value: bagging_fraction=0.7
[LightGBM] [Warning] bagging_freq is set=47, subsample_freq=0 will be ignored. Current value: bagging_freq=47
[LightGBM] [Warning] feature_fraction is set=0.81, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.81
[LightGBM] [Warning] min_data_in_leaf is set=42, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=42
[LightGBM] [Warning] bagging_fraction is set=0.7, subsample=1.0 will be ignored. Current value: bagging_fraction=0.7
[LightGBM] [Warning] bagging_freq is set=47, subsample_freq=0 will be ignored. Current value: bagging_freq=47
[LightGBM] [Warning] feature_fraction is set=0.81, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.81
[LightGBM] [Warning] min_data_in_leaf is set=42, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=42
[LightGBM] [Warning] bagging_fraction is set=0.7, subsample=1.0 will be ignored. Current value: bagging_fraction=0.7
[LightGBM] [Warning] bagging_freq is set=47, subsample_freq=0 will be ignored. Current value: bagging_freq=47
| [0m 13      [0m | [0m 0.9814  [0m | [0m 0.6981  [0m | [0m 47.89   [0m | [0m 0.8106  [0m | [0m 17.46   [0m | [0m 1.529   [0m | [0m 42.19   [0m | [0m 0.1615  [0m | [0m 102.6   [0m | [0m 3.602   [0m | [0m 6.993   [0m |
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=17, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=17
[LightGBM] [Warning] bagging_fraction is set=0.62, subsample=1.0 will be ignored. Current value: bagging_fraction=0.62
[LightGBM] [Warning] bagging_freq is set=81, subsample_freq=0 will be ignored. Current value: bagging_freq=81
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=17, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=17
[LightGBM] [Warning] bagging_fraction is set=0.62, subsample=1.0 will be ignored. Current value: bagging_fraction=0.62
[LightGBM] [Warning] bagging_freq is set=81, subsample_freq=0 will be ignored. Current value: bagging_freq=81
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=17, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=17
[LightGBM] [Warning] bagging_fraction is set=0.62, subsample=1.0 will be ignored. Current value: bagging_fraction=0.62
[LightGBM] [Warning] bagging_freq is set=81, subsample_freq=0 will be ignored. Current value: bagging_freq=81
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=17, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=17
[LightGBM] [Warning] bagging_fraction is set=0.62, subsample=1.0 will be ignored. Current value: bagging_fraction=0.62
[LightGBM] [Warning] bagging_freq is set=81, subsample_freq=0 will be ignored. Current value: bagging_freq=81
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=17, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=17
[LightGBM] [Warning] bagging_fraction is set=0.62, subsample=1.0 will be ignored. Current value: bagging_fraction=0.62
[LightGBM] [Warning] bagging_freq is set=81, subsample_freq=0 will be ignored. Current value: bagging_freq=81
| [0m 14      [0m | [0m 0.982   [0m | [0m 0.6212  [0m | [0m 81.38   [0m | [0m 0.8243  [0m | [0m 17.16   [0m | [0m 4.533   [0m | [0m 17.17   [0m | [0m 0.1799  [0m | [0m 171.7   [0m | [0m 1.985   [0m | [0m 8.425   [0m |
[LightGBM] [Warning] feature_fraction is set=0.98, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.98
[LightGBM] [Warning] min_data_in_leaf is set=47, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=47
[LightGBM] [Warning] bagging_fraction is set=0.75, subsample=1.0 will be ignored. Current value: bagging_fraction=0.75
[LightGBM] [Warning] bagging_freq is set=17, subsample_freq=0 will be ignored. Current value: bagging_freq=17
[LightGBM] [Warning] feature_fraction is set=0.98, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.98
[LightGBM] [Warning] min_data_in_leaf is set=47, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=47
[LightGBM] [Warning] bagging_fraction is set=0.75, subsample=1.0 will be ignored. Current value: bagging_fraction=0.75
[LightGBM] [Warning] bagging_freq is set=17, subsample_freq=0 will be ignored. Current value: bagging_freq=17
[LightGBM] [Warning] feature_fraction is set=0.98, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.98
[LightGBM] [Warning] min_data_in_leaf is set=47, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=47
[LightGBM] [Warning] bagging_fraction is set=0.75, subsample=1.0 will be ignored. Current value: bagging_fraction=0.75
[LightGBM] [Warning] bagging_freq is set=17, subsample_freq=0 will be ignored. Current value: bagging_freq=17
[LightGBM] [Warning] feature_fraction is set=0.98, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.98
[LightGBM] [Warning] min_data_in_leaf is set=47, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=47
[LightGBM] [Warning] bagging_fraction is set=0.75, subsample=1.0 will be ignored. Current value: bagging_fraction=0.75
[LightGBM] [Warning] bagging_freq is set=17, subsample_freq=0 will be ignored. Current value: bagging_freq=17
[LightGBM] [Warning] feature_fraction is set=0.98, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.98
[LightGBM] [Warning] min_data_in_leaf is set=47, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=47
[LightGBM] [Warning] bagging_fraction is set=0.75, subsample=1.0 will be ignored. Current value: bagging_fraction=0.75
[LightGBM] [Warning] bagging_freq is set=17, subsample_freq=0 will be ignored. Current value: bagging_freq=17
| [0m 15      [0m | [0m 0.98    [0m | [0m 0.7488  [0m | [0m 17.31   [0m | [0m 0.9775  [0m | [0m 15.46   [0m | [0m 9.21    [0m | [0m 47.1    [0m | [0m 0.4995  [0m | [0m 138.3   [0m | [0m 3.361   [0m | [0m 4.646   [0m |
=================================================================================================================================================

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
|   iter    |  target   | baggin... | baggin... | featur... | max_depth | min_ch... | min_da... | min_sp... | num_le... | reg_alpha | reg_la... |
-------------------------------------------------------------------------------------------------------------------------------------------------
[LightGBM] [Warning] feature_fraction is set=0.52, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.52
[LightGBM] [Warning] min_data_in_leaf is set=77, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=77
[LightGBM] [Warning] bagging_fraction is set=0.85, subsample=1.0 will be ignored. Current value: bagging_fraction=0.85
[LightGBM] [Warning] bagging_freq is set=97, subsample_freq=0 will be ignored. Current value: bagging_freq=97
[LightGBM] [Warning] feature_fraction is set=0.52, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.52
[LightGBM] [Warning] min_data_in_leaf is set=77, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=77
[LightGBM] [Warning] bagging_fraction is set=0.85, subsample=1.0 will be ignored. Current value: bagging_fraction=0.85
[LightGBM] [Warning] bagging_freq is set=97, subsample_freq=0 will be ignored. Current value: bagging_freq=97
[LightGBM] [Warning] feature_fraction is set=0.52, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.52
[LightGBM] [Warning] min_data_in_leaf is set=77, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=77
[LightGBM] [Warning] bagging_fraction is set=0.85, subsample=1.0 will be ignored. Current value: bagging_fraction=0.85
[LightGBM] [Warning] bagging_freq is set=97, subsample_freq=0 will be ignored. Current value: bagging_freq=97
[LightGBM] [Warning] feature_fraction is set=0.52, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.52
[LightGBM] [Warning] min_data_in_leaf is set=77, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=77
[LightGBM] [Warning] bagging_fraction is set=0.85, subsample=1.0 will be ignored. Current value: bagging_fraction=0.85
[LightGBM] [Warning] bagging_freq is set=97, subsample_freq=0 will be ignored. Current value: bagging_freq=97
[LightGBM] [Warning] feature_fraction is set=0.52, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.52
[LightGBM] [Warning] min_data_in_leaf is set=77, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=77
[LightGBM] [Warning] bagging_fraction is set=0.85, subsample=1.0 will be ignored. Current value: bagging_fraction=0.85
[LightGBM] [Warning] bagging_freq is set=97, subsample_freq=0 will be ignored. Current value: bagging_freq=97
| [0m 1       [0m | [0m 0.9739  [0m | [0m 0.8546  [0m | [0m 97.23   [0m | [0m 0.5183  [0m | [0m 15.71   [0m | [0m 5.494   [0m | [0m 77.91   [0m | [0m 0.7621  [0m | [0m 23.36   [0m | [0m 6.242   [0m | [0m 4.806   [0m |
[LightGBM] [Warning] feature_fraction is set=0.64, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.64
[LightGBM] [Warning] min_data_in_leaf is set=82, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=82
[LightGBM] [Warning] bagging_fraction is set=0.82, subsample=1.0 will be ignored. Current value: bagging_fraction=0.82
[LightGBM] [Warning] bagging_freq is set=59, subsample_freq=0 will be ignored. Current value: bagging_freq=59
[LightGBM] [Warning] feature_fraction is set=0.64, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.64
[LightGBM] [Warning] min_data_in_leaf is set=82, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=82
[LightGBM] [Warning] bagging_fraction is set=0.82, subsample=1.0 will be ignored. Current value: bagging_fraction=0.82
[LightGBM] [Warning] bagging_freq is set=59, subsample_freq=0 will be ignored. Current value: bagging_freq=59
[LightGBM] [Warning] feature_fraction is set=0.64, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.64
[LightGBM] [Warning] min_data_in_leaf is set=82, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=82
[LightGBM] [Warning] bagging_fraction is set=0.82, subsample=1.0 will be ignored. Current value: bagging_fraction=0.82
[LightGBM] [Warning] bagging_freq is set=59, subsample_freq=0 will be ignored. Current value: bagging_freq=59
[LightGBM] [Warning] feature_fraction is set=0.64, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.64
[LightGBM] [Warning] min_data_in_leaf is set=82, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=82
[LightGBM] [Warning] bagging_fraction is set=0.82, subsample=1.0 will be ignored. Current value: bagging_fraction=0.82
[LightGBM] [Warning] bagging_freq is set=59, subsample_freq=0 will be ignored. Current value: bagging_freq=59
[LightGBM] [Warning] feature_fraction is set=0.64, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.64
[LightGBM] [Warning] min_data_in_leaf is set=82, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=82
[LightGBM] [Warning] bagging_fraction is set=0.82, subsample=1.0 will be ignored. Current value: bagging_fraction=0.82
[LightGBM] [Warning] bagging_freq is set=59, subsample_freq=0 will be ignored. Current value: bagging_freq=59
| [95m 2       [0m | [95m 0.9751  [0m | [95m 0.8192  [0m | [95m 59.49   [0m | [95m 0.6409  [0m | [95m 6.577   [0m | [95m 1.987   [0m | [95m 82.2    [0m | [95m 0.667   [0m | [95m 193.8   [0m | [95m 4.968   [0m | [95m 7.509   [0m |
[LightGBM] [Warning] feature_fraction is set=0.83, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.83
[LightGBM] [Warning] min_data_in_leaf is set=16, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=16
[LightGBM] [Warning] bagging_fraction is set=0.51, subsample=1.0 will be ignored. Current value: bagging_fraction=0.51
[LightGBM] [Warning] bagging_freq is set=16, subsample_freq=0 will be ignored. Current value: bagging_freq=16
[LightGBM] [Warning] feature_fraction is set=0.83, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.83
[LightGBM] [Warning] min_data_in_leaf is set=16, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=16
[LightGBM] [Warning] bagging_fraction is set=0.51, subsample=1.0 will be ignored. Current value: bagging_fraction=0.51
[LightGBM] [Warning] bagging_freq is set=16, subsample_freq=0 will be ignored. Current value: bagging_freq=16
[LightGBM] [Warning] feature_fraction is set=0.83, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.83
[LightGBM] [Warning] min_data_in_leaf is set=16, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=16
[LightGBM] [Warning] bagging_fraction is set=0.51, subsample=1.0 will be ignored. Current value: bagging_fraction=0.51
[LightGBM] [Warning] bagging_freq is set=16, subsample_freq=0 will be ignored. Current value: bagging_freq=16
[LightGBM] [Warning] feature_fraction is set=0.83, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.83
[LightGBM] [Warning] min_data_in_leaf is set=16, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=16
[LightGBM] [Warning] bagging_fraction is set=0.51, subsample=1.0 will be ignored. Current value: bagging_fraction=0.51
[LightGBM] [Warning] bagging_freq is set=16, subsample_freq=0 will be ignored. Current value: bagging_freq=16
[LightGBM] [Warning] feature_fraction is set=0.83, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.83
[LightGBM] [Warning] min_data_in_leaf is set=16, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=16
[LightGBM] [Warning] bagging_fraction is set=0.51, subsample=1.0 will be ignored. Current value: bagging_fraction=0.51
[LightGBM] [Warning] bagging_freq is set=16, subsample_freq=0 will be ignored. Current value: bagging_freq=16
| [95m 3       [0m | [95m 0.9853  [0m | [95m 0.5143  [0m | [95m 16.62   [0m | [95m 0.8255  [0m | [95m 18.26   [0m | [95m 1.743   [0m | [95m 16.21   [0m | [95m 0.0749  [0m | [95m 76.97   [0m | [95m 1.132   [0m | [95m 8.145   [0m |
[LightGBM] [Warning] feature_fraction is set=0.58, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.58
[LightGBM] [Warning] min_data_in_leaf is set=73, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=73
[LightGBM] [Warning] bagging_fraction is set=0.88, subsample=1.0 will be ignored. Current value: bagging_fraction=0.88
[LightGBM] [Warning] bagging_freq is set=77, subsample_freq=0 will be ignored. Current value: bagging_freq=77
[LightGBM] [Warning] feature_fraction is set=0.58, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.58
[LightGBM] [Warning] min_data_in_leaf is set=73, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=73
[LightGBM] [Warning] bagging_fraction is set=0.88, subsample=1.0 will be ignored. Current value: bagging_fraction=0.88
[LightGBM] [Warning] bagging_freq is set=77, subsample_freq=0 will be ignored. Current value: bagging_freq=77
[LightGBM] [Warning] feature_fraction is set=0.58, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.58
[LightGBM] [Warning] min_data_in_leaf is set=73, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=73
[LightGBM] [Warning] bagging_fraction is set=0.88, subsample=1.0 will be ignored. Current value: bagging_fraction=0.88
[LightGBM] [Warning] bagging_freq is set=77, subsample_freq=0 will be ignored. Current value: bagging_freq=77
[LightGBM] [Warning] feature_fraction is set=0.58, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.58
[LightGBM] [Warning] min_data_in_leaf is set=73, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=73
[LightGBM] [Warning] bagging_fraction is set=0.88, subsample=1.0 will be ignored. Current value: bagging_fraction=0.88
[LightGBM] [Warning] bagging_freq is set=77, subsample_freq=0 will be ignored. Current value: bagging_freq=77
[LightGBM] [Warning] feature_fraction is set=0.58, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.58
[LightGBM] [Warning] min_data_in_leaf is set=73, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=73
[LightGBM] [Warning] bagging_fraction is set=0.88, subsample=1.0 will be ignored. Current value: bagging_fraction=0.88
[LightGBM] [Warning] bagging_freq is set=77, subsample_freq=0 will be ignored. Current value: bagging_freq=77
| [0m 4       [0m | [0m 0.9806  [0m | [0m 0.8751  [0m | [0m 77.71   [0m | [0m 0.5783  [0m | [0m 3.271   [0m | [0m 6.331   [0m | [0m 73.03   [0m | [0m 0.03976 [0m | [0m 33.28   [0m | [0m 5.119   [0m | [0m 9.69    [0m |
[LightGBM] [Warning] feature_fraction is set=0.59, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.59
[LightGBM] [Warning] min_data_in_leaf is set=75, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=75
[LightGBM] [Warning] bagging_fraction is set=0.83, subsample=1.0 will be ignored. Current value: bagging_fraction=0.83
[LightGBM] [Warning] bagging_freq is set=80, subsample_freq=0 will be ignored. Current value: bagging_freq=80
[LightGBM] [Warning] feature_fraction is set=0.59, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.59
[LightGBM] [Warning] min_data_in_leaf is set=75, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=75
[LightGBM] [Warning] bagging_fraction is set=0.83, subsample=1.0 will be ignored. Current value: bagging_fraction=0.83
[LightGBM] [Warning] bagging_freq is set=80, subsample_freq=0 will be ignored. Current value: bagging_freq=80
[LightGBM] [Warning] feature_fraction is set=0.59, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.59
[LightGBM] [Warning] min_data_in_leaf is set=75, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=75
[LightGBM] [Warning] bagging_fraction is set=0.83, subsample=1.0 will be ignored. Current value: bagging_fraction=0.83
[LightGBM] [Warning] bagging_freq is set=80, subsample_freq=0 will be ignored. Current value: bagging_freq=80
[LightGBM] [Warning] feature_fraction is set=0.59, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.59
[LightGBM] [Warning] min_data_in_leaf is set=75, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=75
[LightGBM] [Warning] bagging_fraction is set=0.83, subsample=1.0 will be ignored. Current value: bagging_fraction=0.83
[LightGBM] [Warning] bagging_freq is set=80, subsample_freq=0 will be ignored. Current value: bagging_freq=80
[LightGBM] [Warning] feature_fraction is set=0.59, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.59
[LightGBM] [Warning] min_data_in_leaf is set=75, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=75
[LightGBM] [Warning] bagging_fraction is set=0.83, subsample=1.0 will be ignored. Current value: bagging_fraction=0.83
[LightGBM] [Warning] bagging_freq is set=80, subsample_freq=0 will be ignored. Current value: bagging_freq=80
| [0m 5       [0m | [0m 0.9776  [0m | [0m 0.8291  [0m | [0m 80.11   [0m | [0m 0.5907  [0m | [0m 6.362   [0m | [0m 5.753   [0m | [0m 75.3    [0m | [0m 0.781   [0m | [0m 139.7   [0m | [0m 1.716   [0m | [0m 6.868   [0m |
[LightGBM] [Warning] feature_fraction is set=0.7, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.7
[LightGBM] [Warning] min_data_in_leaf is set=71, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=71
[LightGBM] [Warning] bagging_fraction is set=0.73, subsample=1.0 will be ignored. Current value: bagging_fraction=0.73
[LightGBM] [Warning] bagging_freq is set=75, subsample_freq=0 will be ignored. Current value: bagging_freq=75
[LightGBM] [Warning] feature_fraction is set=0.7, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.7
[LightGBM] [Warning] min_data_in_leaf is set=71, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=71
[LightGBM] [Warning] bagging_fraction is set=0.73, subsample=1.0 will be ignored. Current value: bagging_fraction=0.73
[LightGBM] [Warning] bagging_freq is set=75, subsample_freq=0 will be ignored. Current value: bagging_freq=75
[LightGBM] [Warning] feature_fraction is set=0.7, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.7
[LightGBM] [Warning] min_data_in_leaf is set=71, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=71
[LightGBM] [Warning] bagging_fraction is set=0.73, subsample=1.0 will be ignored. Current value: bagging_fraction=0.73
[LightGBM] [Warning] bagging_freq is set=75, subsample_freq=0 will be ignored. Current value: bagging_freq=75
[LightGBM] [Warning] feature_fraction is set=0.7, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.7
[LightGBM] [Warning] min_data_in_leaf is set=71, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=71
[LightGBM] [Warning] bagging_fraction is set=0.73, subsample=1.0 will be ignored. Current value: bagging_fraction=0.73
[LightGBM] [Warning] bagging_freq is set=75, subsample_freq=0 will be ignored. Current value: bagging_freq=75
[LightGBM] [Warning] feature_fraction is set=0.7, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.7
[LightGBM] [Warning] min_data_in_leaf is set=71, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=71
[LightGBM] [Warning] bagging_fraction is set=0.73, subsample=1.0 will be ignored. Current value: bagging_fraction=0.73
[LightGBM] [Warning] bagging_freq is set=75, subsample_freq=0 will be ignored. Current value: bagging_freq=75
| [0m 6       [0m | [0m 0.9757  [0m | [0m 0.7278  [0m | [0m 75.73   [0m | [0m 0.6989  [0m | [0m 7.171   [0m | [0m 5.236   [0m | [0m 71.18   [0m | [0m 0.3932  [0m | [0m 34.32   [0m | [0m 5.842   [0m | [0m 8.412   [0m |
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=52, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=52
[LightGBM] [Warning] bagging_fraction is set=0.82, subsample=1.0 will be ignored. Current value: bagging_fraction=0.82
[LightGBM] [Warning] bagging_freq is set=82, subsample_freq=0 will be ignored. Current value: bagging_freq=82
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=52, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=52
[LightGBM] [Warning] bagging_fraction is set=0.82, subsample=1.0 will be ignored. Current value: bagging_fraction=0.82
[LightGBM] [Warning] bagging_freq is set=82, subsample_freq=0 will be ignored. Current value: bagging_freq=82
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=52, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=52
[LightGBM] [Warning] bagging_fraction is set=0.82, subsample=1.0 will be ignored. Current value: bagging_fraction=0.82
[LightGBM] [Warning] bagging_freq is set=82, subsample_freq=0 will be ignored. Current value: bagging_freq=82
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=52, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=52
[LightGBM] [Warning] bagging_fraction is set=0.82, subsample=1.0 will be ignored. Current value: bagging_fraction=0.82
[LightGBM] [Warning] bagging_freq is set=82, subsample_freq=0 will be ignored. Current value: bagging_freq=82
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=52, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=52
[LightGBM] [Warning] bagging_fraction is set=0.82, subsample=1.0 will be ignored. Current value: bagging_fraction=0.82
[LightGBM] [Warning] bagging_freq is set=82, subsample_freq=0 will be ignored. Current value: bagging_freq=82
| [0m 7       [0m | [0m 0.9688  [0m | [0m 0.823   [0m | [0m 82.94   [0m | [0m 0.8162  [0m | [0m 4.985   [0m | [0m 5.682   [0m | [0m 52.74   [0m | [0m 0.8187  [0m | [0m 137.4   [0m | [0m 9.497   [0m | [0m 7.829   [0m |
[LightGBM] [Warning] feature_fraction is set=0.94, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.94
[LightGBM] [Warning] min_data_in_leaf is set=90, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=90
[LightGBM] [Warning] bagging_fraction is set=0.91, subsample=1.0 will be ignored. Current value: bagging_fraction=0.91
[LightGBM] [Warning] bagging_freq is set=66, subsample_freq=0 will be ignored. Current value: bagging_freq=66
[LightGBM] [Warning] feature_fraction is set=0.94, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.94
[LightGBM] [Warning] min_data_in_leaf is set=90, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=90
[LightGBM] [Warning] bagging_fraction is set=0.91, subsample=1.0 will be ignored. Current value: bagging_fraction=0.91
[LightGBM] [Warning] bagging_freq is set=66, subsample_freq=0 will be ignored. Current value: bagging_freq=66
[LightGBM] [Warning] feature_fraction is set=0.94, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.94
[LightGBM] [Warning] min_data_in_leaf is set=90, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=90
[LightGBM] [Warning] bagging_fraction is set=0.91, subsample=1.0 will be ignored. Current value: bagging_fraction=0.91
[LightGBM] [Warning] bagging_freq is set=66, subsample_freq=0 will be ignored. Current value: bagging_freq=66
[LightGBM] [Warning] feature_fraction is set=0.94, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.94
[LightGBM] [Warning] min_data_in_leaf is set=90, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=90
[LightGBM] [Warning] bagging_fraction is set=0.91, subsample=1.0 will be ignored. Current value: bagging_fraction=0.91
[LightGBM] [Warning] bagging_freq is set=66, subsample_freq=0 will be ignored. Current value: bagging_freq=66
[LightGBM] [Warning] feature_fraction is set=0.94, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.94
[LightGBM] [Warning] min_data_in_leaf is set=90, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=90
[LightGBM] [Warning] bagging_fraction is set=0.91, subsample=1.0 will be ignored. Current value: bagging_fraction=0.91
[LightGBM] [Warning] bagging_freq is set=66, subsample_freq=0 will be ignored. Current value: bagging_freq=66
| [0m 8       [0m | [0m 0.9757  [0m | [0m 0.9121  [0m | [0m 66.45   [0m | [0m 0.9389  [0m | [0m 3.0     [0m | [0m 0.8741  [0m | [0m 90.29   [0m | [0m 0.06163 [0m | [0m 31.66   [0m | [0m 9.637   [0m | [0m 0.05176 [0m |
[LightGBM] [Warning] feature_fraction is set=0.7, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.7
[LightGBM] [Warning] min_data_in_leaf is set=42, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=42
[LightGBM] [Warning] bagging_fraction is set=0.91, subsample=1.0 will be ignored. Current value: bagging_fraction=0.91
[LightGBM] [Warning] bagging_freq is set=36, subsample_freq=0 will be ignored. Current value: bagging_freq=36
[LightGBM] [Warning] feature_fraction is set=0.7, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.7
[LightGBM] [Warning] min_data_in_leaf is set=42, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=42
[LightGBM] [Warning] bagging_fraction is set=0.91, subsample=1.0 will be ignored. Current value: bagging_fraction=0.91
[LightGBM] [Warning] bagging_freq is set=36, subsample_freq=0 will be ignored. Current value: bagging_freq=36
[LightGBM] [Warning] feature_fraction is set=0.7, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.7
[LightGBM] [Warning] min_data_in_leaf is set=42, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=42
[LightGBM] [Warning] bagging_fraction is set=0.91, subsample=1.0 will be ignored. Current value: bagging_fraction=0.91
[LightGBM] [Warning] bagging_freq is set=36, subsample_freq=0 will be ignored. Current value: bagging_freq=36
[LightGBM] [Warning] feature_fraction is set=0.7, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.7
[LightGBM] [Warning] min_data_in_leaf is set=42, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=42
[LightGBM] [Warning] bagging_fraction is set=0.91, subsample=1.0 will be ignored. Current value: bagging_fraction=0.91
[LightGBM] [Warning] bagging_freq is set=36, subsample_freq=0 will be ignored. Current value: bagging_freq=36
[LightGBM] [Warning] feature_fraction is set=0.7, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.7
[LightGBM] [Warning] min_data_in_leaf is set=42, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=42
[LightGBM] [Warning] bagging_fraction is set=0.91, subsample=1.0 will be ignored. Current value: bagging_fraction=0.91
[LightGBM] [Warning] bagging_freq is set=36, subsample_freq=0 will be ignored. Current value: bagging_freq=36
| [0m 9       [0m | [0m 0.9785  [0m | [0m 0.9148  [0m | [0m 36.21   [0m | [0m 0.7003  [0m | [0m 8.766   [0m | [0m 8.293   [0m | [0m 42.82   [0m | [0m 0.3236  [0m | [0m 100.6   [0m | [0m 6.452   [0m | [0m 8.626   [0m |
[LightGBM] [Warning] feature_fraction is set=0.84, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.84
[LightGBM] [Warning] min_data_in_leaf is set=14, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=14
[LightGBM] [Warning] bagging_fraction is set=0.71, subsample=1.0 will be ignored. Current value: bagging_fraction=0.71
[LightGBM] [Warning] bagging_freq is set=99, subsample_freq=0 will be ignored. Current value: bagging_freq=99
[LightGBM] [Warning] feature_fraction is set=0.84, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.84
[LightGBM] [Warning] min_data_in_leaf is set=14, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=14
[LightGBM] [Warning] bagging_fraction is set=0.71, subsample=1.0 will be ignored. Current value: bagging_fraction=0.71
[LightGBM] [Warning] bagging_freq is set=99, subsample_freq=0 will be ignored. Current value: bagging_freq=99
[LightGBM] [Warning] feature_fraction is set=0.84, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.84
[LightGBM] [Warning] min_data_in_leaf is set=14, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=14
[LightGBM] [Warning] bagging_fraction is set=0.71, subsample=1.0 will be ignored. Current value: bagging_fraction=0.71
[LightGBM] [Warning] bagging_freq is set=99, subsample_freq=0 will be ignored. Current value: bagging_freq=99
[LightGBM] [Warning] feature_fraction is set=0.84, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.84
[LightGBM] [Warning] min_data_in_leaf is set=14, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=14
[LightGBM] [Warning] bagging_fraction is set=0.71, subsample=1.0 will be ignored. Current value: bagging_fraction=0.71
[LightGBM] [Warning] bagging_freq is set=99, subsample_freq=0 will be ignored. Current value: bagging_freq=99
[LightGBM] [Warning] feature_fraction is set=0.84, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.84
[LightGBM] [Warning] min_data_in_leaf is set=14, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=14
[LightGBM] [Warning] bagging_fraction is set=0.71, subsample=1.0 will be ignored. Current value: bagging_fraction=0.71
[LightGBM] [Warning] bagging_freq is set=99, subsample_freq=0 will be ignored. Current value: bagging_freq=99
| [0m 10      [0m | [0m 0.9751  [0m | [0m 0.7051  [0m | [0m 99.73   [0m | [0m 0.8385  [0m | [0m 6.302   [0m | [0m 0.2434  [0m | [0m 14.85   [0m | [0m 0.6174  [0m | [0m 129.1   [0m | [0m 4.334   [0m | [0m 8.77    [0m |
[LightGBM] [Warning] feature_fraction is set=0.72, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.72
[LightGBM] [Warning] min_data_in_leaf is set=60, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=60
[LightGBM] [Warning] bagging_fraction is set=0.74, subsample=1.0 will be ignored. Current value: bagging_fraction=0.74
[LightGBM] [Warning] bagging_freq is set=86, subsample_freq=0 will be ignored. Current value: bagging_freq=86
[LightGBM] [Warning] feature_fraction is set=0.72, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.72
[LightGBM] [Warning] min_data_in_leaf is set=60, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=60
[LightGBM] [Warning] bagging_fraction is set=0.74, subsample=1.0 will be ignored. Current value: bagging_fraction=0.74
[LightGBM] [Warning] bagging_freq is set=86, subsample_freq=0 will be ignored. Current value: bagging_freq=86
[LightGBM] [Warning] feature_fraction is set=0.72, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.72
[LightGBM] [Warning] min_data_in_leaf is set=60, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=60
[LightGBM] [Warning] bagging_fraction is set=0.74, subsample=1.0 will be ignored. Current value: bagging_fraction=0.74
[LightGBM] [Warning] bagging_freq is set=86, subsample_freq=0 will be ignored. Current value: bagging_freq=86
[LightGBM] [Warning] feature_fraction is set=0.72, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.72
[LightGBM] [Warning] min_data_in_leaf is set=60, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=60
[LightGBM] [Warning] bagging_fraction is set=0.74, subsample=1.0 will be ignored. Current value: bagging_fraction=0.74
[LightGBM] [Warning] bagging_freq is set=86, subsample_freq=0 will be ignored. Current value: bagging_freq=86
[LightGBM] [Warning] feature_fraction is set=0.72, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.72
[LightGBM] [Warning] min_data_in_leaf is set=60, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=60
[LightGBM] [Warning] bagging_fraction is set=0.74, subsample=1.0 will be ignored. Current value: bagging_fraction=0.74
[LightGBM] [Warning] bagging_freq is set=86, subsample_freq=0 will be ignored. Current value: bagging_freq=86
| [0m 11      [0m | [0m 0.9744  [0m | [0m 0.7444  [0m | [0m 86.48   [0m | [0m 0.7192  [0m | [0m 17.36   [0m | [0m 8.871   [0m | [0m 60.56   [0m | [0m 0.6246  [0m | [0m 153.4   [0m | [0m 8.237   [0m | [0m 0.305   [0m |
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=74, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=74
[LightGBM] [Warning] bagging_fraction is set=0.85, subsample=1.0 will be ignored. Current value: bagging_fraction=0.85
[LightGBM] [Warning] bagging_freq is set=75, subsample_freq=0 will be ignored. Current value: bagging_freq=75
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=74, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=74
[LightGBM] [Warning] bagging_fraction is set=0.85, subsample=1.0 will be ignored. Current value: bagging_fraction=0.85
[LightGBM] [Warning] bagging_freq is set=75, subsample_freq=0 will be ignored. Current value: bagging_freq=75
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=74, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=74
[LightGBM] [Warning] bagging_fraction is set=0.85, subsample=1.0 will be ignored. Current value: bagging_fraction=0.85
[LightGBM] [Warning] bagging_freq is set=75, subsample_freq=0 will be ignored. Current value: bagging_freq=75
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=74, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=74
[LightGBM] [Warning] bagging_fraction is set=0.85, subsample=1.0 will be ignored. Current value: bagging_fraction=0.85
[LightGBM] [Warning] bagging_freq is set=75, subsample_freq=0 will be ignored. Current value: bagging_freq=75
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=74, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=74
[LightGBM] [Warning] bagging_fraction is set=0.85, subsample=1.0 will be ignored. Current value: bagging_fraction=0.85
[LightGBM] [Warning] bagging_freq is set=75, subsample_freq=0 will be ignored. Current value: bagging_freq=75
| [0m 12      [0m | [0m 0.9781  [0m | [0m 0.854   [0m | [0m 75.34   [0m | [0m 0.8216  [0m | [0m 19.38   [0m | [0m 9.529   [0m | [0m 74.56   [0m | [0m 0.7511  [0m | [0m 26.53   [0m | [0m 1.126   [0m | [0m 8.934   [0m |
[LightGBM] [Warning] feature_fraction is set=0.81, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.81
[LightGBM] [Warning] min_data_in_leaf is set=42, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=42
[LightGBM] [Warning] bagging_fraction is set=0.7, subsample=1.0 will be ignored. Current value: bagging_fraction=0.7
[LightGBM] [Warning] bagging_freq is set=47, subsample_freq=0 will be ignored. Current value: bagging_freq=47
[LightGBM] [Warning] feature_fraction is set=0.81, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.81
[LightGBM] [Warning] min_data_in_leaf is set=42, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=42
[LightGBM] [Warning] bagging_fraction is set=0.7, subsample=1.0 will be ignored. Current value: bagging_fraction=0.7
[LightGBM] [Warning] bagging_freq is set=47, subsample_freq=0 will be ignored. Current value: bagging_freq=47
[LightGBM] [Warning] feature_fraction is set=0.81, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.81
[LightGBM] [Warning] min_data_in_leaf is set=42, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=42
[LightGBM] [Warning] bagging_fraction is set=0.7, subsample=1.0 will be ignored. Current value: bagging_fraction=0.7
[LightGBM] [Warning] bagging_freq is set=47, subsample_freq=0 will be ignored. Current value: bagging_freq=47
[LightGBM] [Warning] feature_fraction is set=0.81, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.81
[LightGBM] [Warning] min_data_in_leaf is set=42, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=42
[LightGBM] [Warning] bagging_fraction is set=0.7, subsample=1.0 will be ignored. Current value: bagging_fraction=0.7
[LightGBM] [Warning] bagging_freq is set=47, subsample_freq=0 will be ignored. Current value: bagging_freq=47
[LightGBM] [Warning] feature_fraction is set=0.81, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.81
[LightGBM] [Warning] min_data_in_leaf is set=42, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=42
[LightGBM] [Warning] bagging_fraction is set=0.7, subsample=1.0 will be ignored. Current value: bagging_fraction=0.7
[LightGBM] [Warning] bagging_freq is set=47, subsample_freq=0 will be ignored. Current value: bagging_freq=47
| [0m 13      [0m | [0m 0.9814  [0m | [0m 0.6981  [0m | [0m 47.89   [0m | [0m 0.8106  [0m | [0m 17.46   [0m | [0m 1.529   [0m | [0m 42.19   [0m | [0m 0.1615  [0m | [0m 102.6   [0m | [0m 3.602   [0m | [0m 6.993   [0m |
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=17, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=17
[LightGBM] [Warning] bagging_fraction is set=0.62, subsample=1.0 will be ignored. Current value: bagging_fraction=0.62
[LightGBM] [Warning] bagging_freq is set=81, subsample_freq=0 will be ignored. Current value: bagging_freq=81
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=17, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=17
[LightGBM] [Warning] bagging_fraction is set=0.62, subsample=1.0 will be ignored. Current value: bagging_fraction=0.62
[LightGBM] [Warning] bagging_freq is set=81, subsample_freq=0 will be ignored. Current value: bagging_freq=81
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=17, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=17
[LightGBM] [Warning] bagging_fraction is set=0.62, subsample=1.0 will be ignored. Current value: bagging_fraction=0.62
[LightGBM] [Warning] bagging_freq is set=81, subsample_freq=0 will be ignored. Current value: bagging_freq=81
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=17, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=17
[LightGBM] [Warning] bagging_fraction is set=0.62, subsample=1.0 will be ignored. Current value: bagging_fraction=0.62
[LightGBM] [Warning] bagging_freq is set=81, subsample_freq=0 will be ignored. Current value: bagging_freq=81
[LightGBM] [Warning] feature_fraction is set=0.82, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.82
[LightGBM] [Warning] min_data_in_leaf is set=17, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=17
[LightGBM] [Warning] bagging_fraction is set=0.62, subsample=1.0 will be ignored. Current value: bagging_fraction=0.62
[LightGBM] [Warning] bagging_freq is set=81, subsample_freq=0 will be ignored. Current value: bagging_freq=81
| [0m 14      [0m | [0m 0.982   [0m | [0m 0.6212  [0m | [0m 81.38   [0m | [0m 0.8243  [0m | [0m 17.16   [0m | [0m 4.533   [0m | [0m 17.17   [0m | [0m 0.1799  [0m | [0m 171.7   [0m | [0m 1.985   [0m | [0m 8.425   [0m |
[LightGBM] [Warning] feature_fraction is set=0.98, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.98
[LightGBM] [Warning] min_data_in_leaf is set=47, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=47
[LightGBM] [Warning] bagging_fraction is set=0.75, subsample=1.0 will be ignored. Current value: bagging_fraction=0.75
[LightGBM] [Warning] bagging_freq is set=17, subsample_freq=0 will be ignored. Current value: bagging_freq=17
[LightGBM] [Warning] feature_fraction is set=0.98, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.98
[LightGBM] [Warning] min_data_in_leaf is set=47, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=47
[LightGBM] [Warning] bagging_fraction is set=0.75, subsample=1.0 will be ignored. Current value: bagging_fraction=0.75
[LightGBM] [Warning] bagging_freq is set=17, subsample_freq=0 will be ignored. Current value: bagging_freq=17
[LightGBM] [Warning] feature_fraction is set=0.98, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.98
[LightGBM] [Warning] min_data_in_leaf is set=47, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=47
[LightGBM] [Warning] bagging_fraction is set=0.75, subsample=1.0 will be ignored. Current value: bagging_fraction=0.75
[LightGBM] [Warning] bagging_freq is set=17, subsample_freq=0 will be ignored. Current value: bagging_freq=17
[LightGBM] [Warning] feature_fraction is set=0.98, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.98
[LightGBM] [Warning] min_data_in_leaf is set=47, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=47
[LightGBM] [Warning] bagging_fraction is set=0.75, subsample=1.0 will be ignored. Current value: bagging_fraction=0.75
[LightGBM] [Warning] bagging_freq is set=17, subsample_freq=0 will be ignored. Current value: bagging_freq=17
[LightGBM] [Warning] feature_fraction is set=0.98, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.98
[LightGBM] [Warning] min_data_in_leaf is set=47, min_child_samples=20 will be ignored. Current value: min_data_in_leaf=47
[LightGBM] [Warning] bagging_fraction is set=0.75, subsample=1.0 will be ignored. Current value: bagging_fraction=0.75
[LightGBM] [Warning] bagging_freq is set=17, subsample_freq=0 will be ignored. Current value: bagging_freq=17
| [0m 15      [0m | [0m 0.98    [0m | [0m 0.7488  [0m | [0m 17.31   [0m | [0m 0.9775  [0m | [0m 15.46   [0m | [0m 9.21    [0m | [0m 47.1    [0m | [0m 0.4995  [0m | [0m 138.3   [0m | [0m 3.361   [0m | [0m 4.646   [0m |
=================================================================================================================================================

复制代码

1
2
3
"""显示优化结果"""
bayes_lgb.max

复制代码

1
2
3
4
5
6
7
8
9
10
11
12
{'target': 0.9852874999999999,
 'params': {'bagging_fraction': 0.5142972966264403,
  'bagging_freq': 16.622164310094046,
  'feature_fraction': 0.8255184232001205,
  'max_depth': 18.26425100653768,
  'min_child_weight': 1.743138428108859,
  'min_data_in_leaf': 16.209077338448033,
  'min_split_gain': 0.07490156409730242,
  'num_leaves': 76.97233519507536,
  'reg_alpha': 1.1323161637099144,
  'reg_lambda': 8.144859038214168}}

复制代码

"""调整一个较小的学习率，并通过cv函数确定当前最优的迭代次数"""
train_matrix = lgb.Dataset(X_train_split, label=y_train_split)
base_params_lgb = {
                    'boosting_type': 'gbdt',
                    'objective': 'multiclass',
                    'num_class': 4,
                    'learning_rate': 0.01,
                    'num_leaves': 77,
                    'max_depth': 18,
                    'min_data_in_leaf': 16,
                    'feature_pre_filter': False,
                    'min_child_weight':1.7,
                    'bagging_fraction': 0.64,
                    'feature_fraction': 0.83,
                    'bagging_freq': 17,
                    'reg_lambda': 8,
                    'reg_alpha': 1.13,
                    'min_split_gain': 0.075,
                    'nthread': 10,
                    'verbose': -1,
}

cv_result_lgb = lgb.cv(
    train_set=train_matrix,
    early_stopping_rounds=1000, 
    num_boost_round=20000,
    nfold=5,
    stratified=True,
    shuffle=True,
    params=base_params_lgb,
    feval=f1_score_vali,
    seed=0,
)
print('迭代次数{}'.format(len(cv_result_lgb['f1_score-mean'])))
print('最终模型的f1为{}'.format(max(cv_result_lgb['f1_score-mean'])))

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
"""调整一个较小的学习率，并通过cv函数确定当前最优的迭代次数"""
train_matrix = lgb.Dataset(X_train_split, label=y_train_split)
base_params_lgb = {
                    'boosting_type': 'gbdt',
                    'objective': 'multiclass',
                    'num_class': 4,
                    'learning_rate': 0.01,
                    'num_leaves': 77,
                    'max_depth': 18,
                    'min_data_in_leaf': 16,
                    'feature_pre_filter': False,
                    'min_child_weight':1.7,
                    'bagging_fraction': 0.64,
                    'feature_fraction': 0.83,
                    'bagging_freq': 17,
                    'reg_lambda': 8,
                    'reg_alpha': 1.13,
                    'min_split_gain': 0.075,
                    'nthread': 10,
                    'verbose': -1,
}

cv_result_lgb = lgb.cv(
    train_set=train_matrix,
    early_stopping_rounds=1000, 
    num_boost_round=20000,
    nfold=5,
    stratified=True,
    shuffle=True,
    params=base_params_lgb,
    feval=f1_score_vali,
    seed=0,
)
print('迭代次数{}'.format(len(cv_result_lgb['f1_score-mean'])))
print('最终模型的f1为{}'.format(max(cv_result_lgb['f1_score-mean'])))

复制代码

1
2
3
迭代次数3660
最终模型的f1为0.9628270908713847

复制代码

# 模型参数已经确定，建立最终模型并对验证集进行验证
import lightgbm as lgb
"""使用lightgbm 5折交叉验证进行建模预测"""
cv_scores = []
for i, (train_index, valid_index) in enumerate(kf.split(X_train, y_train)):
    print('************************************ {} ************************************'.format(str(i+1)))
    X_train_split, y_train_split, X_val, y_val = X_train.iloc[train_index], y_train[train_index], X_train.iloc[valid_index], y_train[valid_index]

train_matrix = lgb.Dataset(X_train_split, label=y_train_split)
    valid_matrix = lgb.Dataset(X_val, label=y_val)

params = {
                'boosting_type': 'gbdt',
                'objective': 'multiclass',
                'num_class': 4,
                'learning_rate': 0.01,
                'num_leaves': 138,
                'max_depth': 11,
                'min_data_in_leaf': 43,
                'min_child_weight':6.5,
                'bagging_fraction': 0.64,
                'feature_fraction': 0.93,
                'bagging_freq': 49,
                'reg_lambda': 7,
                'reg_alpha': 0.21,
                'min_split_gain': 0.288,
                'nthread': 10,
                'verbose': -1,
    }

model = lgb.train(params, train_set=train_matrix, num_boost_round=4833, valid_sets=valid_matrix, 
                      verbose_eval=1000, early_stopping_rounds=200, feval=f1_score_vali)
    val_pred = model.predict(X_val, num_iteration=model.best_iteration)
    val_pred = np.argmax(val_pred, axis=1)
    cv_scores.append(f1_score(y_true=y_val, y_pred=val_pred, average='macro'))
    print(cv_scores)

print("lgb_scotrainre_list:{}".format(cv_scores))
print("lgb_score_mean:{}".format(np.mean(cv_scores)))
print("lgb_score_std:{}".format(np.std(cv_scores)))

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
# 模型参数已经确定，建立最终模型并对验证集进行验证
import lightgbm as lgb
"""使用lightgbm 5折交叉验证进行建模预测"""
cv_scores = []
for i, (train_index, valid_index) in enumerate(kf.split(X_train, y_train)):
    print('************************************ {} ************************************'.format(str(i+1)))
    X_train_split, y_train_split, X_val, y_val = X_train.iloc[train_index], y_train[train_index], X_train.iloc[valid_index], y_train[valid_index]

    train_matrix = lgb.Dataset(X_train_split, label=y_train_split)
    valid_matrix = lgb.Dataset(X_val, label=y_val)

    params = {
                'boosting_type': 'gbdt',
                'objective': 'multiclass',
                'num_class': 4,
                'learning_rate': 0.01,
                'num_leaves': 138,
                'max_depth': 11,
                'min_data_in_leaf': 43,
                'min_child_weight':6.5,
                'bagging_fraction': 0.64,
                'feature_fraction': 0.93,
                'bagging_freq': 49,
                'reg_lambda': 7,
                'reg_alpha': 0.21,
                'min_split_gain': 0.288,
                'nthread': 10,
                'verbose': -1,
    }

    model = lgb.train(params, train_set=train_matrix, num_boost_round=4833, valid_sets=valid_matrix, 
                      verbose_eval=1000, early_stopping_rounds=200, feval=f1_score_vali)
    val_pred = model.predict(X_val, num_iteration=model.best_iteration)
    val_pred = np.argmax(val_pred, axis=1)
    cv_scores.append(f1_score(y_true=y_val, y_pred=val_pred, average='macro'))
    print(cv_scores)

print("lgb_scotrainre_list:{}".format(cv_scores))
print("lgb_score_mean:{}".format(np.mean(cv_scores)))
print("lgb_score_std:{}".format(np.std(cv_scores)))

复制代码

************************************ 1 ************************************
Training until validation scores don't improve for 200 rounds
[1000]	valid_0's multi_logloss: 0.050037	valid_0's f1_score: 0.958168
Early stopping, best iteration is:
[1639]	valid_0's multi_logloss: 0.0439137	valid_0's f1_score: 0.961506
[0.9615056903324599]
************************************ 2 ************************************
Training until validation scores don't improve for 200 rounds
[1000]	valid_0's multi_logloss: 0.0562826	valid_0's f1_score: 0.953819
[2000]	valid_0's multi_logloss: 0.0484745	valid_0's f1_score: 0.959567
Early stopping, best iteration is:
[1869]	valid_0's multi_logloss: 0.0488369	valid_0's f1_score: 0.959783
[0.9615056903324599, 0.9597829114711733]
************************************ 3 ************************************
Training until validation scores don't improve for 200 rounds
[1000]	valid_0's multi_logloss: 0.0491551	valid_0's f1_score: 0.958783
[2000]	valid_0's multi_logloss: 0.0417199	valid_0's f1_score: 0.963393
Early stopping, best iteration is:
[2405]	valid_0's multi_logloss: 0.0409952	valid_0's f1_score: 0.964476
[0.9615056903324599, 0.9597829114711733, 0.9644760387635415]
************************************ 4 ************************************
Training until validation scores don't improve for 200 rounds
[1000]	valid_0's multi_logloss: 0.0553984	valid_0's f1_score: 0.957148
[2000]	valid_0's multi_logloss: 0.0486412	valid_0's f1_score: 0.961739
Early stopping, best iteration is:
[2254]	valid_0's multi_logloss: 0.0482131	valid_0's f1_score: 0.962201
[0.9615056903324599, 0.9597829114711733, 0.9644760387635415, 0.9622009947666585]
************************************ 5 ************************************
Training until validation scores don't improve for 200 rounds
[1000]	valid_0's multi_logloss: 0.0492426	valid_0's f1_score: 0.957039
Early stopping, best iteration is:
[1433]	valid_0's multi_logloss: 0.0445974	valid_0's f1_score: 0.960794
[0.9615056903324599, 0.9597829114711733, 0.9644760387635415, 0.9622009947666585, 0.9607941521618003]
lgb_scotrainre_list:[0.9615056903324599, 0.9597829114711733, 0.9644760387635415, 0.9622009947666585, 0.9607941521618003]
lgb_score_mean:0.9617519574991267
lgb_score_std:0.0015797109890455313

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
************************************ 1 ************************************
Training until validation scores don't improve for 200 rounds
[1000]	valid_0's multi_logloss: 0.050037	valid_0's f1_score: 0.958168
Early stopping, best iteration is:
[1639]	valid_0's multi_logloss: 0.0439137	valid_0's f1_score: 0.961506
[0.9615056903324599]
************************************ 2 ************************************
Training until validation scores don't improve for 200 rounds
[1000]	valid_0's multi_logloss: 0.0562826	valid_0's f1_score: 0.953819
[2000]	valid_0's multi_logloss: 0.0484745	valid_0's f1_score: 0.959567
Early stopping, best iteration is:
[1869]	valid_0's multi_logloss: 0.0488369	valid_0's f1_score: 0.959783
[0.9615056903324599, 0.9597829114711733]
************************************ 3 ************************************
Training until validation scores don't improve for 200 rounds
[1000]	valid_0's multi_logloss: 0.0491551	valid_0's f1_score: 0.958783
[2000]	valid_0's multi_logloss: 0.0417199	valid_0's f1_score: 0.963393
Early stopping, best iteration is:
[2405]	valid_0's multi_logloss: 0.0409952	valid_0's f1_score: 0.964476
[0.9615056903324599, 0.9597829114711733, 0.9644760387635415]
************************************ 4 ************************************
Training until validation scores don't improve for 200 rounds
[1000]	valid_0's multi_logloss: 0.0553984	valid_0's f1_score: 0.957148
[2000]	valid_0's multi_logloss: 0.0486412	valid_0's f1_score: 0.961739
Early stopping, best iteration is:
[2254]	valid_0's multi_logloss: 0.0482131	valid_0's f1_score: 0.962201
[0.9615056903324599, 0.9597829114711733, 0.9644760387635415, 0.9622009947666585]
************************************ 5 ************************************
Training until validation scores don't improve for 200 rounds
[1000]	valid_0's multi_logloss: 0.0492426	valid_0's f1_score: 0.957039
Early stopping, best iteration is:
[1433]	valid_0's multi_logloss: 0.0445974	valid_0's f1_score: 0.960794
[0.9615056903324599, 0.9597829114711733, 0.9644760387635415, 0.9622009947666585, 0.9607941521618003]
lgb_scotrainre_list:[0.9615056903324599, 0.9597829114711733, 0.9644760387635415, 0.9622009947666585, 0.9607941521618003]
lgb_score_mean:0.9617519574991267
lgb_score_std:0.0015797109890455313