elasticsearch mapping问题解决

1.报错信息如下：

复制代码[2018-07-16T00:10:17,743][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"filebeat-api-2018.07.15", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x38a5460c>], :response=>{"index"=>{"_index"=>"filebeat-api-2018.07.15", "_type"=>"doc", "_id"=>"erO0nmQB3VHsYd1y-IbI", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [params.end_day]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: """}}}}}

1
[2018-07-16T00:10:17,743][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"filebeat-api-2018.07.15", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x38a5460c>], :response=>{"index"=>{"_index"=>"filebeat-api-2018.07.15", "_type"=>"doc", "_id"=>"erO0nmQB3VHsYd1y-IbI", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [params.end_day]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: """}}}}}

日期格式不对报错，解决办法如下，建立一个map,关掉日期格式检测，对需要日期格式的字段加上date类型字段。这里也实现了field限制，将默认的1000加到了3000.

报错：

复制代码"reason"=>"object mapping for [params.commoditys] tried to parse field [commoditys] as object, but found a concrete value"}}}}

1
"reason"=>"object mapping for [params.commoditys] tried to parse field [commoditys] as object, but found a concrete value"}}}}

这个ignore_malformed设置为true解决

复制代码curl -XPUT "10.66.178.20:9200/_template/template_api" -H 'Content-Type: application/json' -d'{ "index_patterns":["filebeat-api-*"], "order":0, "settings":{ "number_of_replicas":0, "index":{ "mapping":{ "ignore_malformed": true, "total_fields": { "limit": "3000" } } } }, "mappings": { "doc": { "date_detection": false, "properties": { "@timestamp": { "type": "date" } } } } }'

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
curl -XPUT "10.66.178.20:9200/_template/template_api" -H 'Content-Type: application/json' -d'{
"index_patterns":["filebeat-api-*"],
"order":0,
"settings":{
"number_of_replicas":0,
"index":{
"mapping":{
"ignore_malformed": true,
"total_fields": {
"limit": "3000"
}
}
}
},
"mappings": {
"doc": {
"date_detection": false,
"properties": {
"@timestamp": {
"type": "date"
}
}
}
}
}'