Hive搭建（基于Linux）

• Hive组件部署.

1、基础环境准备.

① 传输Hive安装包.（SFTP界面）

注：hive安装包可去apache官方下载，版本为2.0.0

注：基于使用etc连接虚拟机

注：Ctrl + p 可打开SFTP界面,注意传输路径.

② 安装Hive.（root界面）

指令：tar -zxvf /opt/software/apache-hive-2.0.0-bin.tar.gz -C /opt/module/

③ 将解压后的 apache-hive-2.0.0-bin 文件夹更名为 hive.

指令：mv /opt/module/apache-hive-2.0.0-bin opt/module/hive

④ 修改权限.

指令：chown -R hadoop:hadoop /opt/module/hive

2、安装MySQL数据库

① 卸载 MariaDB 数据库.

查询指令：rpm -qa | grep mariadb

删除指令：rpm -e --nodeps mariadb-libs-5.5.56-2.el7.x86_64

② 安装MySQL数据库.

1）创建mysql目录.并拉入MySQL安装包.

注：在/opt/software下安装mysql.

指令：mkdir /opt/software/mysql

2）按照下面顺序安装MySQL.

进入目录：cd /opt/software/mysql

安装：rpm -ivh mysql-community-common-5.7.18-1.el7.x86_64.rpm

     rpm -ivhmysql-community-libs-5.7.18-1.el7.x86_64.rpm

     rpm -ivh mysql-community-client-5.7.18-1.el7.x86_64.rpm

注：安装依赖包再安装server.

安装依赖包：yum install –y net-tools

   yum install –y perl

安装：rpm -ivh mysql-community-server-5.7.18-1.el7.x86_64.rpm

③ 配置信息添加到/etc/my.cnf 文件 symbolic-links=0 下.

指令：vi /etc/my.cnf

配置内容：

default-storage-engine=innodb

innodb_file_per_table

collation-server=utf8_general_ci

init-connect='SET NAMES utf8'

character-set-server=utf8

④ 启动 MySQL 数据库. 

指令：systemctl start mysqld

⑤ 查看MySQL 数据库状态.

指令：systemctl status mysqld

注：mysqld 进程状态为 active (running)，则表示 MySQL 数据库正常运行.

mysqld 进程状态为 failed，则表示 MySQL 数据库启动异常；此时需要排查

/etc/my.cnf 文件.

⑥ 查询数据库默认密码.

指令：cat /var/log/mysqld.log | grep password

注：localhost：后即为密码.

⑦ MySQL 数据库初始化会出现以下交互确认信息.

指令：mysql_secure_installation

注：绿色为信息，蓝色为输入内容，#后面是注释.

交互信息：

Securing the MySQL server deployment.

Enter password for user root:

# 输入/var/log/mysqld.log 文件中查询到的默认 root 用户登录密码.

The 'validate_password' plugin is installed on the server.

The subsequent steps will run with the existing configurationof the plugin.

Using existing password for root.

Estimated strength of the password: 100

Change the password for root ? ((Press y|Y for Yes, any other key for No) : y

New password: Password123$

Re-enter new password: Password123$

Estimated strength of the password: 100

Do you wish to continue with the password provided?(Press y|Y for Yes, any other key for No) : y

By default, a MySQL installation has an anonymous user,

allowing anyone to log into MySQL without having to have

a user account created for them. This is intended only for

testing, and to make the installation go a bit smoother.

You should remove them before moving into a production

environment.

Remove anonymous users? (Press y|Y for Yes, any other key for No) : y

Success.

Normally, root should only be allowed to connect from

'localhost'. This ensures that someone cannot guess at

the root password from the network.

Disallow root login remotely? (Press y|Y for Yes, any other key for No) : n

 ... skipping.

By default, MySQL comes with a database named 'test' that

anyone can access. This is also intended only for testing,

and should be removed before moving into a production

environment.

Remove test database and access to it? (Press y|Y for Yes, any other key for No) : y

- Dropping test database...

Success.

- Removing privileges on test database...

Success.

Reloading the privilege tables will ensure that all changes

made so far will take effect immediately.

Reload privilege tables now? (Press y|Y for Yes, any other key for No) : y

Success.

All done!

⑧ 添加 root 用户.

指令：mysql -u root -p

注：绿色为信息，蓝色为输入内容，#后面是注释.

将会出现确认信息：

Enter password: Password123$ # 输入新设定的密码 Password123$

Welcome to the MySQL monitor.  Commands end with ; or g.

Your MySQL connection id is 14

Server version: 5.7.18 MySQL Community Server (GPL)

Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its

affiliates. Other names may be trademarks of their respective

owners.

Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.

mysql> grant all privileges on *.* to root@'localhost'identified by 'Password123$';

# 添加 root 用户本地访问授权.

Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql> grant all privileges on *.* to root@'%' identified by'Password123$';

# 添加 root 用户远程访问授权.

Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql> flush privileges; # 刷新授权.

Query OK, 0 rows affected (0.00 sec)

mysql> select user,host from mysql.user where user='root';

# 查询 root 用户授权情况.

+------+-----------+

| user | host      |

+------+-----------+

| root | %         |

| root | localhost |

+------+-----------+

2 rows in set (0.00 sec)

mysql> exit;  # 退出数据库.

Bye

3、配置Hive

① 配置Hive环境变量并生效.

指令：vi /etc/profile

配置内容：

export HIVE_HOME=/opt/module/hive

export PATH=$PATH:$HIVE_HOME/bin

生效：source /etc/profile

② 重命名Hive 配置文件.

1）登录Hadoop.

指令：su - hadoop

2）重命名hive-default.xml.template为hive-site.xml.

注：在/opt/module/hive/conf目录下操作.

指令：cp hive-default.xml.template hive-site.xml

③ 修改hive-site.xml 文件.

指令：vi /opt/module/hive/conf/hive-site.xml

注：将原内容修改.（搜索标题<name>xxx</name>之间的内容.然后修改<value>’蓝色’</value>）

（搜索：vi文件之后，按esc然后 / +搜索的内容，回车.）

1）设置MySQL数据库连接.（蓝色为修改内容）

原：<name>javax.jdo.option.ConnectionURL</name>

     <value>jdbc:derby:;databaseName=metastore_db;create=true</value>

<description>JDBC connect string for a JDBC metastore</description>

改：<name>javax.jdo.option.ConnectionURL</name>

<value>jdbc:mysql://master:3306/hive?createDatabaseIfNotExist=true&a

p;useSSL=false</value>

<description>JDBC connect string for a JDBC metastore</description>

2）配置MySQL数据库root的密码.

原：<name>javax.jdo.option.ConnectionPassword</name>

     <value>mine</value>

<description>password to use against metastore database</description>

改：<name>javax.jdo.option.ConnectionPassword</name>

     <value>Password123$</value>

<description>password to use against metastore database</description>

3）验证元数据存储版本一致性.若默认false，则不用修改.

<name>hive.metastore.schema.verification</name>

     <value>false</value>

<description>

4）配置数据库驱动.

原：<name>javax.jdo.option.ConnectionDriverName</name>

     <value>org.apache.derby.jdbc.EmbeddedDriver</value>

<description>Driver class name for a JDBC metastore</description>

改：<name>javax.jdo.option.ConnectionDriverName</name>

<value>com.mysql.jdbc.Driver</value>

<description>Driver class name for a JDBC metastore</description>

5）配置数据库用户名javax.jdo.option.ConnectionUserName为root.

原：<name>javax.jdo.option.ConnectionUserName</name>

     <value>root</value>

<description>Username to use against metastore database</description>

6）将下4个文件的${system:java.io.tmpdir}/${system:user.name}替换为 “/opt/module/hive/tmp”目录及其子目录.

原：① <name>hive.querylog.location</name>

     <value>${system:java.io.tmpdir}/${system:user.name}</value>

     <description>Location of Hive run time structured log file</description>

② <name>hive.exec.local.scratchdir</name>

     <value>${system:java.io.tmpdir}/${system:user.name}</value>

     <description>Local scratch space for Hive jobs</description>

③ <name>hive.downloaded.resources.dir</name>

     <value>${system:java.io.tmpdir}/${hive.session.id}_resources</value>

注：这修改成/opt/module/hive/tmp/resources

<description>Temporary local directory for added resources in the remote file system.</description>

④ <name>hive.server2.logging.operation.log.location</name>

     <value>${system:java.io.tmpdir}/${system:user.name}/operation_logs</value>

7）保存退出,创建时文件夹tmp

指令：mkdir /opt/module/hive/tmp

④ 初始Hive元数据.

1）导入MySQL数据库驱动.

注：在SFTP界面将mysql-connector-java-5.1.46.jar拉到Hive安装目录的lib.

2）修改权限.（root界面）

指令：chown -R hadoop：hadoop /opt/module/hive

3）删除会导致Hive元数据初始化失败的文件.

指令：rm -rf /opt/module/hadoop/share/hadoop/yarn/lib/jline-0.9.94.jar

4）重启Hadoop.

停止：stop-all.sh（未启动不需停）

重启：start-all.sh

⑤ 初始化数据库.

指令：schematool -initSchema -dbType mysql 

注：出现*** schemaTool completed *** 即成功.

出现*** schemaTool failed *** 报错，可能是主键重复，需删除MySQL中的Hive数据库，重新初 .指令：mysql -u root -p # 进入数据库删除Hive数据库.退出MySQL再重新初始化.

⑥ 启动hive.

指令：hive

最后

以上就是虚幻哈密瓜为你收集整理的Hive搭建（基于Linux）的全部内容，希望文章能够帮你解决Hive搭建（基于Linux）所遇到的程序开发问题。

如果觉得靠谱客网站的内容还不错，欢迎将靠谱客网站推荐给程序员好友。