我是靠谱客的博主 机智煎蛋,最近开发中收集的这篇文章主要介绍hbase 单机连接hadoop_单机环境下Hive安装使用及与HBase整合,觉得挺不错的,现在分享给大家,希望可以做个参考。

概述

auth  修改为:auth

hive-site.xml 文件:

首先修改配置项

hive.metastore.schema.verification

false(这个值改成false)

文件最后加上几行:

hive.aux.jars.path

file:///home/hadoop/hive-0.12.0/lib/hive-hbase-handler-0.12.0.jar,file:///home/hadoop/hive-0.12.0/lib/hbase-0.94.18.jar,file:///home/hadoop/hive-0.12.0/lib/zookeeper-3.4.5.jar

注意写法,目录形式是file:///,一定要一字不差,然后将对应的jar包从hbase目录下拷贝到相应位置,如果hive下已有对应jar包,建议删除后使用hbase目录下的。

二、启动并测试hive(以下内容来自http://blog.csdn.net/luckymelina/article/details/16707801)

进入hive目录下,敲命令:

bin/hive -hiveconf hbase.master=master:60000

a)       创建hbase识别的表

CREATE TABLE hbase_table_1(key int, value string) STORED

BY'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH

SERDEPROPERTIES("hbase.columns.mapping" = ":key,cf1:val")TBLPROPERTIES ("hbase.table.name" = "xyz");

b)       新建hive的数据表

create table pokes(foo int,bar string)row format delimited fields terminated by ',';

c)       批量导入数据

load data local inpath '/home/hadoop/1.txt' overwrite into table pokes;

1.txt文件的内容为

1,hello

2,pear

3,world

d)      使用sql导入hbase_table_1

insert overwrite table hbase_table_1 select * from pokes;

这一步如果正常说明hbase与hive已做到了正常连接,输出日志:

Total MapReduce jobs = 1

Launching Job 1 out of 1

Number of reduce tasks is set to 0 since there's no reduce operator

Starting Job = job_201404191501_0011, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201404191501_0011

Kill Command = /home/hadoop/hadoop/libexec/../bin/hadoop job  -kill job_201404191501_0011

Hadoop job information for Stage-0: number of mappers: 1; number of reducers: 0

2014-04-19 21:14:21,307 Stage-0 map = 0%,  reduce = 0%

2014-04-19 21:14:35,384 Stage-0 map = 100%,  reduce = 0%, Cumulative CPU 2.39 sec

2014-04-19 21:14:36,391 Stage-0 map = 100%,  reduce = 0%, Cumulative CPU 2.39 sec

2014-04-19 21:14:37,398 Stage-0 map = 100%,  reduce = 0%, Cumulative CPU 2.39 sec

2014-04-19 21:14:38,406 Stage-0 map = 100%,  reduce = 0%, Cumulative CPU 2.39 sec

2014-04-19 21:14:39,416 Stage-0 map = 100%,  reduce = 100%, Cumulative CPU 2.39 sec

MapReduce Total cumulative CPU time: 2 seconds 390 msec

Ended Job = job_201404191501_0011

MapReduce Jobs Launched:

Job 0: Map: 1   Cumulative CPU: 2.39 sec   HDFS Read: 232 HDFS Write: 0 SUCCESS

Total MapReduce CPU Time Spent: 2 seconds 390 msec

OK

Time taken: 57.471 seconds

e)       查看数据

hive>select * from hbase_table_1;

OK

1 hello

2 pear

3 world

说明相应数据已成功的从Hive导入到HBase中

最后

以上就是机智煎蛋为你收集整理的hbase 单机连接hadoop_单机环境下Hive安装使用及与HBase整合的全部内容,希望文章能够帮你解决hbase 单机连接hadoop_单机环境下Hive安装使用及与HBase整合所遇到的程序开发问题。

如果觉得靠谱客网站的内容还不错,欢迎将靠谱客网站推荐给程序员好友。

本图文内容来源于网友提供,作为学习参考使用,或来自网络收集整理,版权属于原作者所有。
点赞(44)

评论列表共有 0 条评论

立即
投稿
返回
顶部