Sqoop使用和简介

序号

命令/command

类

说明

1

impor

ImportTool

从关系型数据库中导入数据(来自表或者查询语句)到HDFS中

2

export

ExportTool

将HDFS中的数据导入到关系型数据库中

3

codegen

CodeGenTool

获取数据库中某张表数据生成Java并打成jar包

4

create-hive-table

CreateHiveTableTool

创建Hive表

5

eval

EvalSqlTool

查看SQL执行结果

6

import-all-tables

ImportAllTablesTool

导入某个数据库下所有表到HDFS中

7

job

JobTool

8

list-databases

ListDatabasesTool

列出所有数据库名

9

list-tables

ListTablesTool

列出某个数据库下所有表

10

merge

MergeTool

11

metastore

MetastoreTool

12

help

HelpTool

查看帮助

13

version

VersionTool

查看版本

序号

参数

说明

样例

1

connect

连接关系型数据库的URL

jdbc:mysql://localhost/sqoop_datas

2

connection-manager

连接管理类,一般不用

3

driver

连接驱动

4

hadoop-home 

hadoop目录

/home/hadoop

5

help

查看帮助信息

6

password

连接关系型数据库的密码

7

username

链接关系型数据库的用户名

8

verbose

查看更多的信息,其实是将日志级别调低

该参数后面不接值

--append

Append data to an existing dataset in HDFS

--as-avrodatafile

Imports data to Avro Data Files

--as-sequencefile

Imports data to SequenceFiles

--as-textfile

Imports data as plain text (default)

--boundary-query <statement>

Boundary query to use for creating splits

--columns <col,col,col…>

Columns to import from table

--direct

Use direct import fast path

--direct-split-size <n>

Split the input stream every n bytes when importing in direct mode

--inline-lob-limit <n>

Set the maximum size for an inline LOB

-m,--num-mappers <n>

Use n map tasks to import in parallel

-e,--query <statement>

Import the results of statement.

--split-by <column-name>

Column of the table used to split work units

--table <table-name>

Table to read

--target-dir <dir>

HDFS destination dir

--warehouse-dir <dir>

HDFS parent for table destination

--where <where clause>

WHERE clause to use during import

-z,--compress

Enable compression

--compression-codec <c>

Use Hadoop codec (default gzip)

--null-string <null-string>

The string to be written for a null value for string columns

--null-non-string <null-string>

The string to be written for a null value for non-string columns

--check-column (col)

Specifies the column to be examined when determining which rows to import.

--incremental (mode)

Specifies how Sqoop determines which rows are new. Legal values for mode include append and lastmodified.

--last-value (value)

Specifies the maximum value of the check column from the previous import.

--enclosed-by <char>

Sets a required field enclosing character

--escaped-by <char>

Sets the escape character

--fields-terminated-by <char>

Sets the field separator character

--lines-terminated-by <char>

Sets the end-of-line character

--mysql-delimiters

Uses MySQL’s default delimiter set: fields: , lines: n escaped-by:  optionally-enclosed-by: '

--optionally-enclosed-by <char>

Sets a field enclosing character

--hive-home <dir>

Override $HIVE_HOME

--hive-import

Import tables into Hive (Uses Hive’s default delimiters if none are set.)

--hive-overwrite

Overwrite existing data in the Hive table.

--create-hive-table

If set, then the job will fail if the target hive

table exits. By default this property is false.

--hive-table <table-name>

Sets the table name to use when importing to Hive.

--hive-drop-import-delims

Drops n, r, and 1 from string fields when importing to Hive.

--hive-delims-replacement

Replace n, r, and 1 from string fields with user defined string when importing to Hive.

--hive-partition-key

Name of a hive field to partition are sharded on

--hive-partition-value <v>

String-value that serves as partition key for this imported into hive in this job.

--map-column-hive <map>

Override default mapping from SQL type to Hive type for configured columns.

--column-family <family>

Sets the target column family for the import

--hbase-create-table

If specified, create missing HBase tables

--hbase-row-key <col>

Specifies which input column to use as the row key

--hbase-table <table-name>

Specifies an HBase table to use as the target instead of HDFS

--bindir <dir>

Output directory for compiled objects

--class-name <name>

Sets the generated class name. This overrides --package-name. When combined with --jar-file, sets the input class.

--jar-file <file>

Disable code generation; use specified jar

--outdir <dir>

Output directory for generated code

--package-name <name>

Put auto-generated classes in this package

--map-column-java <m>

Override default mapping from SQL type to Java type for configured columns.