ELK 显示地图

131 阅读 0 评论 87 点赞

我是靠谱客的博主微笑野狼，最近开发中收集的这篇文章主要介绍ELK 显示地图，觉得挺不错的，现在分享给大家，希望可以做个参考。

概述

ELK：5.1.1.1版本

---------------

架构：

nginx->logstash->redis->logstash->elasticsearch->kibana

-------------------

机器：

nginx logstash1 redis ：123.56.222.120

logstash2 elasticsearch kibana：10.130.24.111

---------------

nginx 配置：

-----------------nginx
-----------------
log_format main '$remote_addr [$time_local] '
'"$host" "$request" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent"';
access_log logs/access.log main;

配置
/usr/local/nginx/conf/nginx.conf

nginx stop:
pkill nginx
/usr/local/nginx/sbin -s stop

nginx log:
/usr/local/nginx/logs
tail -f /usr/local/nginx/logs/access.log
echo "">/usr/local/nginx/logs/access.log

-----redis
配置
/usr/local/etc/redis.conf
requirepass 1234567
注释bind
deamon yes
protected-mode no

启动脚本
/usr/local/bin/redis-cli -h 123.56.222.120 -p 6378
/usr/local/bin/redis-server /usr/local/etc/redis.conf &

开放防火强
/sbin/iptables -I INPUT -p tcp --dport 6378 -j ACCEPT

------收集器部分logstash
wget https://artifacts.elastic.co/downloads/logstash/logstash-5.0.0.tar.gz
vi /home/songlk/logstash-5.1.1/config/nginx.conf

input {
   file {
       path => ["/usr/local/nginx/logs/access.log"]
       type => "nginx_access"
       }
   }
output {
   if [type] == "nginx_access"{
       redis{
               host => ["123.56.222.120:6378"]
               data_type =>"list"
               password => "1234567"
               key => "nginx"

       }
   }
}

-------接收器转换器部分-------

logstash:接受redis发送到elasticsearch
logstash_nginx.conf
一定要注意格式空格以及utf8编码

input {
   redis {
       host => "123.56.222.120"
       port => 6378
       password => "1234567"
       data_type => "list"
       key => "nginx"
   }
}
filter {
   grok {
       match => {
           "message" => "%{IPORHOST:clientip} [%{HTTPDATE:timestamp}] %{NOTSPACE:http_name} "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?:%{NUMBER:bytes:float}|-) %{QS:referrer} %{QS:agent}"
       }
   }
   date {
       match => [ "timestamp" , "dd/MMM/YYYY:HH:mm:ss Z" ]
   }
   geoip {
       source => "clientip"
       target => "geoip"
       database => "/home/elasticresearch/soft/logstash-5.1.1/GeoLite2-City.mmdb"
       add_field => ["[geoip][coordinates]", "%{[geoip][longitude]}"]
       add_field => ["[geoip][coordinates]", "%{[geoip][latitude]}"]
   }
   mutate {
       convert => [ "[geoip][coordinates]", "float"]
   }
}
output {
   elasticsearch {
       action => "index"
       hosts =>"10.130.24.111:9200"
       index => "logstash-nginx-%{+yyyy.MM.dd}"
   }
}

/home/elasticresearch/soft/logstash-5.1.1/bin/logstash -f /home/elasticresearch/soft/logstash-5.1.1/logstash_nginx.conf &

-------------------------

elasticsearch

--------------------------
/home/elasticresearch/soft/elasticsearch-5.1.1/bin/elasticsearch -d

-----------------
kibana

/home/elasticresearch/soft/kibana-5.1.1-linux-x86_64/bin/kibana &

kibana.yml
tilemap.url: 'http://webrd02.is.autonavi.com/appmaptile?lang=zh_cn&size=1&scale=1&style=7&x={x}&y={y}&z={z}'
tilemap.options.minZoom: "1"
tilemap.options.maxZoom: "10"

测试：

kibana

http://10.130.24.111:5601