ES集群搭建

一、安装软件准备

系统：Centos7-2003

软件版本：JDK15.x、ES7.11.X。

官方下载地址：

https://www.elastic.co/cn/downloads/

社区镜像下载

http://dl.elasticsearch.cn/

华为镜像下载

https://mirrors.huaweicloud.com/

Docker容器下载

可选择ES官方镜像或者国内阿里云/网易等镜像

二、操作系统设置

生产环境必须部署在Linux，并需要一些系统配置才可生效。个人环境可以选择Windows，几乎无需配置。无论是单实例还是集群，所有的节点都要以下配置。注意以下配置时，请用root权限账号操作。

2.1 关闭防火墙状态

#关闭命令
service firewalld stop
chkconfig firewalld off

2.2 关闭SELINUX

配置文件：/etc/selinux/config

#SELINUX=enforcing
SELINUX=disabled

2.3 设置hostname

• 若是虚拟机，建议配置独立的主机名，便于区分
• 配置文件：/etc/sysconfig/network

方式一：直接hostnamectl命令

hostnamectl set-hostname elkserver

方式二：配置文件修改

NETWORKING=yes
HOSTNAME=elkserver

2.4 关闭swap

• 配置文件 /etc/sysctl.conf

[root@elkserver /]# vim /etc/sysctl.conf
# 禁用内存与硬盘交换
vm.swappiness=1
# 设置虚拟内存大小
vm.max_map_count=262144

• 执行/sbin/sysctl -p 立即生效

[root@elkserver /]# /sbin/sysctl -p

2.5 文件句柄设置

配置文件： /etc/security/limits.conf

[root@elkserver /]# vim /etc/security/limits.conf
# 进程线程数
* 		soft nproc 131072
* 		hard nproc 131072
# 文件句柄数
*	    soft nofile 131072
* 		hard nofile 131072
# jvm内存锁定交换
*       soft memlock unlimited
*       hard memlock unlimited	

2.6 ES专用账号并授权

• 创建ES专用账号

useradd es
#授权ES程序目录es账号权限
#假设 ES程序目录、数据目录、日志目录都在/es目录下
chown -R es:es /es/*

2.7 JDK配置

将jdk-15.0.2_linux-x64_bin.tar.gz放到路径/usr/local/src/下并解压

[root@elkserver src]# tar -zxvf jdk-15.0.2_linux-x64_bin.tar.gz

• JDK配置（可选）：/etc/profile

ES最新版本自带jdk版本，默认可以不需要配置，建议配置，便于安装其它java程序辅助

本内容中要求安装JDK15.x

[root@elkserver src]# vim /etc/profile
export JAVA_HOME=/usr/local/src/jdk-15.0.2
export JRE_HOME=$JAVA_HOME/jre
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib:$JRE_HOME/lib

• 保存退出后，使配置生效：

[root@elkserver src]# source /etc/profile

[root@elkserver src]# java -version
java version "15.0.2" 2021-01-19
Java(TM) SE Runtime Environment (build 15.0.2+7-27)
Java HotSpot(TM) 64-Bit Server VM (build 15.0.2+7-27, mixed mode, sharing)

2.8 JVM临时目录

如在es目录下创建/es/jvm-tmpdir

[root@elkserver es]# mkdir jvm_tmpdir
[root@elkserver elk]# chown -R es:es /es/*		

• 方式一：修改环境变量

[root@elkserver src]# vim /etc/profile
export ES_TMPDIR=/es/jvm_tmpdir
[root@elkserver src]# source /etc/profile

• 方式二：在jvm.options中设置（原来就有）

-Djava.io.tmpdir=${ES_TMPDIR}

2.9 JNA临时目录

创建ES临时文件目录：jna_tmpdir，仅限制Linux操作系统

如在es目录下创建/es/jna-tmpdir

在启动JVM中增加此参数

在jvm.options中修改

-Djna.tmpdir=/es/jna_tmpdir

[root@elkserver es]# mkdir jna_tmpdir
[root@elkserver es]# ls
elasticsearch-7.11.1-linux-x86_64.tar.gz  es-9200  jna_tmpdir  jvm_tmpdir  kibana-7.11.1-linux-x86_64.tar.gz

三、脚本命令

启动ES脚本

Windows系统：elasticsearch.bat

Linux系统：

​ 当前窗口启动： ./elasticsearch

​ 后台进程启动： ./elasticsearch -d

辅助工具

​ 安全、插件、环境、迁移。。。

四、ES配置文件

elasticsearch.yml 核心配置文件、节点实例属性参数

jvm.options 配置堆栈参数等

log4j2.properties 日志常规配置，默认就好

其他

账户安全/动态生成/keystore安全文件

配置elasticsearch.yml

• 集群名称

  默认无需设置，建议设置

  如：cluster.name: elk01

• 节点名称

  单实例不用设置，集群下必须区分设置，建议设置

  如：${HOSTNAME}-9200

• IP地址

  必须设置，默认仅限本地访问

  如：network.host: 192.168.237.127

• 访问端口

  http端口，默认无需设置

  tcp端口，默认需求设置

  如：http.port: 9200
  transport.port: 9300

• 集群发现

  discovery.xxx 单节点无需配置

  cluster.xx.xx 单节点默认可无需配置

  单节点如：discovery.seed_hosts: [“192.168.237.127:9300”]

  cluster.initial_master_nodes: [“192.168.237.127:9300”]

  多节点集群如：

  discovery.seed_hosts: [“192.168.237.127:9301”,“192.168.237.127:9302”,“192.168.237.127:9303”]

  cluster.initial_master_nodes: [“192.168.237.127:9301”,“192.168.237.127:9302”,“192.168.237.127:9303”]

• 数据目录与日志目录

  path.data: /es/es-9200/elk/data

  path.logs: /es/es-9200/elk/logs

• 内存交换锁定，此处需要操作系统设置才生效

  bootstrap.memory_lock: true

• 防止批量删除索引
  action.destructive_requires_name: true

• 设置处理器数量4，默认无需设置，单机器多实例需要设置
  node.processors: 4

• 设置跨域操作

  http.cors.enabled 是否支持跨域，默认为false http.cors.allow-origin 当设置允许跨域，默认为*,表示支持所有域名，如果我们只是允许某些网站能访问，那么可以使用正则表达式。比如只允许本地地址。

  如：http.cors.enabled: true
  http.cors.allow-origin: “*”

jvm.options

• JDK 选择

  自主配置或者自带

• GC选择

  7.8.1已经采用G1、原则JDK14以上采用G1，以下采用CMS、7.11.x

• 堆栈大小

  默认4GB

  不超过1/2系统内存

  空余1/2闲置内存

  内存上限不超过32G，且不能等于32G

• GC日志

• gc.log路径设置

五、Kibana安装配置

Kibana作为ES的开发者工具、管理运维工具（不存储数据）。

• bin

  启动脚本、常用工具脚本

• config

  文件配置

• 其它

  默认，无需关系

kibana.yml

• ES配置指向 必须配置，默认仅本地

  如：elasticsearch.hosts: [“http://192.168.237.127:9201”]

• IP+Port设置 IP地址必须配置，默认仅限本机访问

  如：server.port: 6601

  ​ server.host: “192.168.237.127”

• 元数据索引名称

  默认无需配置，建议配置修改

  如：kibana.index: “.kibana-elk03”

• 其它设置

  安全、语言、其它

六、单节点实战部署

虚拟机：内存8G及以上，处理器4核，硬盘30G

命令 free -m查看系统资源

之前准备工作创建es文件资源目录 mkdir es，进入目录后pwd命令如下

[root@elkserver /]# cd es
[root@elkserver es]# pwd
/es
[root@elkserver es]# hostname
elkserver

在es目录下放入es安装包并解压，

[root@elkserver es]# tar -zxvf elasticsearch-7.11.1-linux-x86_64.tar.gz

解压后将解压出的文件夹改名为es-9200

[root@elkserver es]# mv elasticsearch-7.11.1 es-9200

[root@elkserver es-9200]# cd elk
[root@elkserver elk]# mkdir data
[root@elkserver elk]# mkdir logs
[root@elkserver elk]# ls
data logs

同样解压Kibana，并重命名文件夹

[root@elkserver es]# tar -zxvf kibana-7.11.1-linux-x86_64.tar.gz

[root@elkserver es]# mv kibana-7.11.1-linux-x86_64 kibana-5601

改变目录权限（上面已提到过）

[root@elkserver es]# chown -R es:es /es/*

进入es-9200文件夹，查看es自带jdk版本

[root@elkserver es-9200]# cd jdk/bin
[root@elkserver bin]# ./java -version
openjdk version "15.0.1" 2020-10-20
OpenJDK Runtime Environment AdoptOpenJDK (build 15.0.1+9)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 15.0.1+9, mixed mode, sharing)

进入到es-9200/config/文件夹下进行配置文件的配置

[root@elkserver config]# vim jvm.options

增加如下：

-Xms1g
-Xmx1g

其他不建议修改，如修改比如log目录等。保存后退出 :wq

[root@elkserver config]# vim elasticsearch.yml

详细配置见上面 第四章的内容，附录单节点9200的elasticsearch.yml文件内容如下：（注意将临时文件及日志文件路径事先创建好并授权给es用户）

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: elk01
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: ${HOSTNAME}-9200
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /es/es-9200/elk/data
#
# Path to log files:
#
path.logs: /es/es-9200/elk/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 192.168.237.127
#
# Set a custom port for HTTP:
#
http.port: 9200
transport.port: 9300
# Lock the memory on startup:
# #内存交换锁定，此处需要操作系统设置才生效
bootstrap.memory_lock: true
# #
# #防止批量删除索引
action.destructive_requires_name: true
# discovery.type : single-node
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.seed_hosts: ["192.168.237.127:9300"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
cluster.initial_master_nodes: ["192.168.237.127:9300"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
http.cors.enabled: true
http.cors.allow-origin: "*"

进入kibana配置路径并修改配置

[root@elkserver es-9200]# cd ..
[root@elkserver es]# ls
elasticsearch-7.11.1-linux-x86_64.tar.gz  es-9200  es-9201  es-9202  es-9203  jna-tmpdir  kibana-5601  kibana-6601  kibana-7.11.1-linux-x86_64.tar.gz
[root@elkserver es]# cd kibana-5601
[root@elkserver kibana-5601]# ls
bin  config  data  LICENSE.txt  node  node_modules  NOTICE.txt  package.json  plugins  README.txt  src  x-pack
[root@elkserver kibana-5601]# cd config
[root@elkserver config]

[root@elkserver config]# vim kibana.yml

详细配置见上面 第四章的内容，附录单节点5601的kibana.yml文件内容如下：

# Kibana is served by a back end server. This setting specifies the port to use.
server.port: 5601

# Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
server.host: "192.168.237.127"

# Enables you to specify a path to mount Kibana at if you are running behind a proxy.
# Use the `server.rewriteBasePath` setting to tell Kibana if it should remove the basePath
# from requests it receives, and to prevent a deprecation warning at startup.
# This setting cannot end in a slash.
#server.basePath: ""

# Specifies whether Kibana should rewrite requests that are prefixed with
# `server.basePath` or require that they are rewritten by your reverse proxy.
# This setting was effectively always `false` before Kibana 6.3 and will
# default to `true` starting in Kibana 7.0.
#server.rewriteBasePath: false

# Specifies the public URL at which Kibana is available for end users. If
# `server.basePath` is configured this URL should end with the same basePath.
#server.publicBaseUrl: ""

# The maximum payload size in bytes for incoming server requests.
#server.maxPayloadBytes: 1048576

# The Kibana server's name.  This is used for display purposes.
#server.name: "your-hostname"

# The URLs of the Elasticsearch instances to use for all your queries.
elasticsearch.hosts: ["http://192.168.237.127:9200"]

# Kibana uses an index in Elasticsearch to store saved searches, visualizations and
# dashboards. Kibana creates a new index if the index doesn't already exist.
kibana.index: ".kibana-elk01"

# The default application to load.
#kibana.defaultAppId: "home"

# If your Elasticsearch is protected with basic authentication, these settings provide
# the username and password that the Kibana server uses to perform maintenance on the Kibana
# index at startup. Your Kibana users still need to authenticate with Elasticsearch, which
# is proxied through the Kibana server.
#elasticsearch.username: "kibana_system"
#elasticsearch.password: "pass"

# Enables SSL and paths to the PEM-format SSL certificate and SSL key files, respectively.
# These settings enable SSL for outgoing requests from the Kibana server to the browser.
#server.ssl.enabled: false
#server.ssl.certificate: /path/to/your/server.crt
#server.ssl.key: /path/to/your/server.key

# Optional settings that provide the paths to the PEM-format SSL certificate and key files.
# These files are used to verify the identity of Kibana to Elasticsearch and are required when
# xpack.security.http.ssl.client_authentication in Elasticsearch is set to required.
#elasticsearch.ssl.certificate: /path/to/your/client.crt
#elasticsearch.ssl.key: /path/to/your/client.key

# Optional setting that enables you to specify a path to the PEM file for the certificate
# authority for your Elasticsearch instance.
#elasticsearch.ssl.certificateAuthorities: [ "/path/to/your/CA.pem" ]

# To disregard the validity of SSL certificates, change this setting's value to 'none'.
#elasticsearch.ssl.verificationMode: full

# Time in milliseconds to wait for Elasticsearch to respond to pings. Defaults to the value of
# the elasticsearch.requestTimeout setting.
#elasticsearch.pingTimeout: 1500

# Time in milliseconds to wait for responses from the back end or Elasticsearch. This value
# must be a positive integer.
#elasticsearch.requestTimeout: 30000

# List of Kibana client-side headers to send to Elasticsearch. To send *no* client-side
# headers, set this value to [] (an empty list).
#elasticsearch.requestHeadersWhitelist: [ authorization ]

# Header names and values that are sent to Elasticsearch. Any custom headers cannot be overwritten
# by client-side headers, regardless of the elasticsearch.requestHeadersWhitelist configuration.
#elasticsearch.customHeaders: {}

# Time in milliseconds for Elasticsearch to wait for responses from shards. Set to 0 to disable.
#elasticsearch.shardTimeout: 30000

# Logs queries sent to Elasticsearch. Requires logging.verbose set to true.
#elasticsearch.logQueries: false

# Specifies the path where Kibana creates the process ID file.
#pid.file: /run/kibana/kibana.pid

# Enables you to specify a file where Kibana stores log output.
#logging.dest: stdout

# Set the value of this setting to true to suppress all logging output.
#logging.silent: false

# Set the value of this setting to true to suppress all logging output other than error messages.
#logging.quiet: false

# Set the value of this setting to true to log all events, including system usage information
# and all requests.
#logging.verbose: false

# Set the interval in milliseconds to sample system and process performance
# metrics. Minimum is 100ms. Defaults to 5000.
#ops.interval: 5000

# Specifies locale to be used for all localizable strings, dates and number formats.
# Supported languages are the following: English - en , by default , Chinese - zh-CN .
#i18n.locale: "en"

启动ES

切换到es用户，并回到es-9200目录下进行命令启动

[root@elkserver es-9200]# su es
[es@elkserver es-9200]# ./bin/elasticsearch

• 启动成功后，windows主机浏览器输入es的ip及端口如：http://192.168.237.127:9200/

  显示如下，即ES启动成功

{
  "name" : "elkserver-9200",
  "cluster_name" : "elk01",
  "cluster_uuid" : "_o7bbB61RHSVd9kLbTKCiQ",
  "version" : {
    "number" : "7.11.1",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "ff17057114c2199c9c1bbecc727003a907c0db7a",
    "build_date" : "2021-02-15T13:44:09.394032Z",
    "build_snapshot" : false,
    "lucene_version" : "8.7.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

• 检查节点的健康状态

  http://192.168.237.127:9200/_cat/health

  1621091301 15:08:21 elk01 green 1 1 6 6 0 0 0 0 - 100.0%
  

• 查看节点状态

  http://192.168.237.127:9200/_cat/nodes

  192.168.237.127 28 88 4 0.04 0.30 0.27 cdhilmrstw * elkserver-9200
  

• xshell中关闭 ctrl +c

启动Kibana

进入到es/kibana-5601文件夹下，执行如下命令启动kibana

[es@elkserver kibana-5601]$ ./bin/kibana

• 启动成功后，windows主机浏览器访问kibana的ip及端口如： http://192.168.237.127:5601

  • 点击ADD data

  • 点击Sample data 依次添加样例数据（样例数据很重要，后续会使用）

  • 进入左上角菜单 Management——>Stack Monitoring

  • 点击 Or, set up with self monitoring 使用内部手机设置

  • 打开 Turn on monitoring

• 如看到es节点变黄，则在kibana中的Dev tools中执行如下即可

  PUT /_settings {  "number_of_replicas": 0 }
  

  xshell中关闭 ctrl +c

七、集群环境搭建

ES集群配置

Elastic集群模式必须至少2个实例以上，一般建议3个节点以上，可以保障其中一个节点失效，集群仍然可以服务。集群模式与单实例模式大部分配置上是一样的，仅需修改集群通信差异部分。

集群环境说明

在这里在一台机器但起3个节点服务的集群。原则上与三台机器各起一个节点服务是一样的。

服务器IP：192.168.237.127

服务器hostname：elkserver

ES文件夹目录：es/es-9201、es/es-9202、es/es-9203

三个服务的es端口号：9201、9202、9203

Kibana启动一个节点即可：es kibana-6601 端口号：6601

安装说明

注意需重新解压一个es并命名为es-9201

[root@elkserver es]# tar -zxvf elasticsearch-7.11.1-linux-x86_64.tar.gz
[root@elkserver es]# mv elasticsearch-7.11.1 es-9201

注意需重新解压一个kibana并命名为 kibana-6601

[root@elkserver es]# tar -zxvf kibana-7.11.1-linux-x86_64.tar.gz
[root@elkserver es]# mv kibana-7.11.1-linux-x86_64 kibana-6601

ES的9201配置文件内容如下：

jvm.options

################################################################
##
## JVM configuration
##
################################################################
##
## WARNING: DO NOT EDIT THIS FILE. If you want to override the
## JVM options in this file, or set any additional options, you
## should create one or more files in the jvm.options.d
## directory containing your adjustments.
##
## See https://www.elastic.co/guide/en/elasticsearch/reference/current/jvm-options.html
## for more information.
##
################################################################



################################################################
## IMPORTANT: JVM heap size
################################################################
##
## The heap size is automatically configured by Elasticsearch
## based on the available memory in your system and the roles
## each node is configured to fulfill. If specifying heap is
## required, it should be done through a file in jvm.options.d,
## and the min and max should be set to the same value. For
## example, to set the heap to 4 GB, create a new file in the
## jvm.options.d directory containing these lines:
##
## -Xms4g
## -Xmx4g
##
## See https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html
## for more information
##
################################################################

-Xms1g
-Xmx1g


################################################################
## Expert settings
################################################################
##
## All settings below here are considered expert settings. Do
## not adjust them unless you understand what you are doing. Do
## not edit them in this file; instead, create a new file in the
## jvm.options.d directory containing your adjustments.
##
################################################################

## GC configuration
8-13:-XX:+UseConcMarkSweepGC
8-13:-XX:CMSInitiatingOccupancyFraction=75
8-13:-XX:+UseCMSInitiatingOccupancyOnly

## G1GC Configuration
# NOTE: G1 GC is only supported on JDK version 10 or later
# to use G1GC, uncomment the next two lines and update the version on the
# following three lines to your version of the JDK
# 10-13:-XX:-UseConcMarkSweepGC
# 10-13:-XX:-UseCMSInitiatingOccupancyOnly
14-:-XX:+UseG1GC

## JVM temporary directory
-Djava.io.tmpdir=${ES_TMPDIR}
## heap dumps

# generate a heap dump when an allocation from the Java heap fails; heap dumps
# are created in the working directory of the JVM unless an alternative path is
# specified
-XX:+HeapDumpOnOutOfMemoryError

# specify an alternative path for heap dumps; ensure the directory exists and
# has sufficient space
-XX:HeapDumpPath=data

# specify an alternative path for JVM fatal error logs
-XX:ErrorFile=logs/hs_err_pid%p.log

## JDK 8 GC logging
8:-XX:+PrintGCDetails
8:-XX:+PrintGCDateStamps
8:-XX:+PrintTenuringDistribution
8:-XX:+PrintGCApplicationStoppedTime
8:-Xloggc:logs/gc.log
8:-XX:+UseGCLogFileRotation
8:-XX:NumberOfGCLogFiles=32
8:-XX:GCLogFileSize=64m

# JDK 9+ GC logging
9-:-Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m

附es-9201节点的elasticsearch.yml

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: elk03
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: ${HOSTNAME}-9201
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /es/es-9201/elk/data
#
# Path to log files:
#
path.logs: /es/es-9201/elk/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
# Lock the memory on startup:
#内存交换锁定，此处需要操作系统设置才生效
bootstrap.memory_lock: true
#
#防止批量删除索引
action.destructive_requires_name: true
#
#设置处理器数量4，默认无需设置，单机器多实例需要设置
node.processors: 4
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 192.168.237.127
#
# Set a custom port for HTTP:
#
http.port: 9201
transport.port: 9301
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.seed_hosts: ["192.168.237.127:9301","192.168.237.127:9302","192.168.237.127:9303"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
cluster.initial_master_nodes: ["192.168.237.127:9301","192.168.237.127:9302","192.168.237.127:9303"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
http.cors.enabled: true
http.cors.allow-origin: "*"

• 创建日志路径及数据路径

[root@elkserver es-9201]# mkdir elk
[root@elkserver es-9201]# ls
bin  config  elk  jdk  lib  LICENSE.txt  logs  modules  NOTICE.txt  plugins  README.asciidoc
[root@elkserver es-9201]# cd elk
[root@elkserver elk]# mkdir data
[root@elkserver elk]# mkdir logs
[root@elkserver elk]# ls
data  logs
[root@elkserver elk]# chown -R es:es /es/*

es-9202、es-9203文件夹的生成方法

• 其余两个节点，可用第一个节点文件夹复制的形式，再修改配置。

[root@elkserver es]# cp -r es-9201/ es-9202
[root@elkserver es]# cp -r es-9201/ es-9203

• 只需修改两个节点的elasticsearch.yml 以及临时文件和日志文件的名称即可。

附es-9202节点的elasticsearch.yml

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: elk03
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: ${HOSTNAME}-9202
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /es/es-9202/elk/data
#
# Path to log files:
#
path.logs: /es/es-9202/elk/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
# Lock the memory on startup:
#内存交换锁定，此处需要操作系统设置才生效
bootstrap.memory_lock: true
#
#防止批量删除索引
action.destructive_requires_name: true
#
#设置处理器数量4，默认无需设置，单机器多实例需要设置
node.processors: 4
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 192.168.237.127
#
# Set a custom port for HTTP:
#
http.port: 9202
transport.port: 9302
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.seed_hosts: ["192.168.237.127:9301","192.168.237.127:9302","192.168.237.127:9303"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
cluster.initial_master_nodes: ["192.168.237.127:9301","192.168.237.127:9302","192.168.237.127:9303"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true

附es-9203节点的elasticsearch.yml

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: elk03
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: ${HOSTNAME}-9203
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /es/es-9203/elk/data
#
# Path to log files:
#
path.logs: /es/es-9203/elk/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
# Lock the memory on startup:
#内存交换锁定，此处需要操作系统设置才生效
bootstrap.memory_lock: true
#
#防止批量删除索引
action.destructive_requires_name: true
#
#设置处理器数量4，默认无需设置，单机器多实例需要设置
node.processors: 4
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 192.168.237.127
#
# Set a custom port for HTTP:
#
http.port: 9203
transport.port: 9303
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.seed_hosts: ["192.168.237.127:9301","192.168.237.127:9302","192.168.237.127:9303"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
cluster.initial_master_nodes: ["192.168.237.127:9301","192.168.237.127:9302","192.168.237.127:9303"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true

• 注意：新生成文件夹后需要进行重新授权。chown -R es:es /es/*

启动ES集群环境

• 切换到es账户下 su - es，分别依次启动三个节点的es服务

[es@elkserver es-9201]$ ./bin/elasticsearch
[es@elkserver es-9202]$ ./bin/elasticsearch
[es@elkserver es-9203]$ ./bin/elasticsearch

检查三个节点启动是否成功

• 依次在windows主机的浏览器中输入三个ES节点的地址信息进行查看

http://192.168.237.127:9201/

http://192.168.237.127:9202/

http://192.168.237.127:9203/

检查三个节点的健康状态

http://192.168.237.127:9201/_cat/health

结果：1621074718 10:31:58 elk03 green 3 3 53 25 0 0 0 0 - 100.0%

http://192.168.237.127:9202/_cat/health

结果：1621074689 10:31:29 elk03 green 3 3 53 25 0 0 0 0 - 100.0%

http://192.168.237.127:9203/_cat/health

结果：1621074658 10:30:58 elk03 green 3 3 53 25 0 0 0 0 - 100.0%

检查集群是否成功启动

-在任意节点看即可

http://192.168.237.127:9202/_cat/nodes

结果：

192.168.237.127 39 97 7 0.28 1.19 0.88 cdhilmrstw * elkserver-9201
192.168.237.127 26 97 7 0.28 1.19 0.88 cdhilmrstw - elkserver-9203
192.168.237.127 43 97 6 0.28 1.19 0.88 cdhilmrstw - elkserver-9202

配置Kibana

附6601节点Kibana的kibana.yml配置

• 注elasticsearch.hosts只需配置集群中一个节点即可

# Kibana is served by a back end server. This setting specifies the port to use.
server.port: 6601

# Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
server.host: "192.168.237.127"

# Enables you to specify a path to mount Kibana at if you are running behind a proxy.
# Use the `server.rewriteBasePath` setting to tell Kibana if it should remove the basePath
# from requests it receives, and to prevent a deprecation warning at startup.
# This setting cannot end in a slash.
#server.basePath: ""

# Specifies whether Kibana should rewrite requests that are prefixed with
# `server.basePath` or require that they are rewritten by your reverse proxy.
# This setting was effectively always `false` before Kibana 6.3 and will
# default to `true` starting in Kibana 7.0.
#server.rewriteBasePath: false

# Specifies the public URL at which Kibana is available for end users. If
# `server.basePath` is configured this URL should end with the same basePath.
#server.publicBaseUrl: ""

# The maximum payload size in bytes for incoming server requests.
#server.maxPayloadBytes: 1048576

# The Kibana server's name.  This is used for display purposes.
#server.name: "your-hostname"

# The URLs of the Elasticsearch instances to use for all your queries.
elasticsearch.hosts: ["http://192.168.237.127:9201"]

# Kibana uses an index in Elasticsearch to store saved searches, visualizations and
# dashboards. Kibana creates a new index if the index doesn't already exist.
kibana.index: ".kibana-elk03"

# The default application to load.
#kibana.defaultAppId: "home"

# If your Elasticsearch is protected with basic authentication, these settings provide
# the username and password that the Kibana server uses to perform maintenance on the Kibana
# index at startup. Your Kibana users still need to authenticate with Elasticsearch, which
# is proxied through the Kibana server.
#elasticsearch.username: "kibana_system"
#elasticsearch.password: "pass"

# Enables SSL and paths to the PEM-format SSL certificate and SSL key files, respectively.
# These settings enable SSL for outgoing requests from the Kibana server to the browser.
#server.ssl.enabled: false
#server.ssl.certificate: /path/to/your/server.crt
#server.ssl.key: /path/to/your/server.key

# Optional settings that provide the paths to the PEM-format SSL certificate and key files.
# These files are used to verify the identity of Kibana to Elasticsearch and are required when
# xpack.security.http.ssl.client_authentication in Elasticsearch is set to required.
#elasticsearch.ssl.certificate: /path/to/your/client.crt
#elasticsearch.ssl.key: /path/to/your/client.key

# Optional setting that enables you to specify a path to the PEM file for the certificate
# authority for your Elasticsearch instance.
#elasticsearch.ssl.certificateAuthorities: [ "/path/to/your/CA.pem" ]

# To disregard the validity of SSL certificates, change this setting's value to 'none'.
#elasticsearch.ssl.verificationMode: full

# Time in milliseconds to wait for Elasticsearch to respond to pings. Defaults to the value of
# the elasticsearch.requestTimeout setting.
#elasticsearch.pingTimeout: 1500

# Time in milliseconds to wait for responses from the back end or Elasticsearch. This value
# must be a positive integer.
#elasticsearch.requestTimeout: 30000

# List of Kibana client-side headers to send to Elasticsearch. To send *no* client-side
# headers, set this value to [] (an empty list).
#elasticsearch.requestHeadersWhitelist: [ authorization ]

# Header names and values that are sent to Elasticsearch. Any custom headers cannot be overwritten
# by client-side headers, regardless of the elasticsearch.requestHeadersWhitelist configuration.
#elasticsearch.customHeaders: {}

# Time in milliseconds for Elasticsearch to wait for responses from shards. Set to 0 to disable.
#elasticsearch.shardTimeout: 30000

# Logs queries sent to Elasticsearch. Requires logging.verbose set to true.
#elasticsearch.logQueries: false

# Specifies the path where Kibana creates the process ID file.
#pid.file: /run/kibana/kibana.pid

# Enables you to specify a file where Kibana stores log output.
#logging.dest: stdout

# Set the value of this setting to true to suppress all logging output.
#logging.silent: false

# Set the value of this setting to true to suppress all logging output other than error messages.
#logging.quiet: false

# Set the value of this setting to true to log all events, including system usage information
# and all requests.
#logging.verbose: false

# Set the interval in milliseconds to sample system and process performance
# metrics. Minimum is 100ms. Defaults to 5000.
#ops.interval: 5000

# Specifies locale to be used for all localizable strings, dates and number formats.
# Supported languages are the following: English - en , by default , Chinese - zh-CN .
#i18n.locale: "en"

启动Kibana环境

[es@elkserver kibana-6601]$ ./bin/kibana

检查Kibana环境启动是否成功

• windows主机浏览器访问：

http://192.168.237.127:6601/

• 启动成功后，windows主机浏览器访问kibana的ip及端口如： http://192.168.237.127:6601

  • 点击ADD data

  • 点击Sample data 依次添加样例数据（样例数据很重要，后续会使用）

  • 进入左上角菜单 Management——>Stack Monitoring

  • 点击 Or, set up with self monitoring 使用内部手机设置

  • 打开 Turn on monitoring

• 如看到es节点变黄，则在kibana中的Dev tools中执行如下即可

  PUT /_settings {  "number_of_replicas": 0 }
  

  xshell中关闭 ctrl +c

• 安装部署成功！

最后

以上就是认真薯片为你收集整理的ES集群搭建ES集群搭建的全部内容，希望文章能够帮你解决ES集群搭建ES集群搭建所遇到的程序开发问题。

如果觉得靠谱客网站的内容还不错，欢迎将靠谱客网站推荐给程序员好友。