SCENARIO:
We need to move a table named "daily_input" from cluster whose master server is "hadoop-1-test-master" (source) to "hadoop-2-test-master" (master of destination cluster).
PROCEDURE:
STEP 1 :EXPORT THE TABLE TO HDFS OF CURRENT CLUSTER
Run the following command in hive shell of hadoop-1-test-master:
---------
EXPORT TABLE daily_input TO '/tmp/backup/table-source/';
---------
The above command will export the table "daily_input" to HDFS of source cluster inside the folder named "/tmp/backup1/table-source/".
STEP 2: COPY FROM SOURCE HDFS TO DESTINATION HDFS:
Run the below command in destination cluster:
------------------
hadoop distcp -skipcrccheck -update hftp://hadoop-1-test-master.maninmanoj.com:50070/tmp/backup/table-source hdfs://hadoop-2-test-master.maninmanoj.com:8020/tmp/backup/table-destination
--------------------
The above command will copy the exported table from source "hadoop-1-test-master" to destination "hadoop-2-test-master" HDFS folder "/tmp/backup/table-destination"
STEP 3: IMPORT THE TABLES:
Run the following command in the destination hive shell:
------------
IMPORT TABLE daily_input FROM '/tmp/backup/table-destination';
-----------
The above command will import the table "daily_input" to destination hive table.
IMPORTANT: No need to manually create the table in the destination, it will be automatically created during import.
Reference : http://www.ericlin.me/hive-export-import-command-transfering-data-between-hive-instances
We need to move a table named "daily_input" from cluster whose master server is "hadoop-1-test-master" (source) to "hadoop-2-test-master" (master of destination cluster).
PROCEDURE:
STEP 1 :EXPORT THE TABLE TO HDFS OF CURRENT CLUSTER
Run the following command in hive shell of hadoop-1-test-master:
---------
EXPORT TABLE daily_input TO '/tmp/backup/table-source/';
---------
The above command will export the table "daily_input" to HDFS of source cluster inside the folder named "/tmp/backup1/table-source/".
STEP 2: COPY FROM SOURCE HDFS TO DESTINATION HDFS:
Run the below command in destination cluster:
------------------
hadoop distcp -skipcrccheck -update hftp://hadoop-1-test-master.maninmanoj.com:50070/tmp/backup/table-source hdfs://hadoop-2-test-master.maninmanoj.com:8020/tmp/backup/table-destination
--------------------
The above command will copy the exported table from source "hadoop-1-test-master" to destination "hadoop-2-test-master" HDFS folder "/tmp/backup/table-destination"
STEP 3: IMPORT THE TABLES:
Run the following command in the destination hive shell:
------------
IMPORT TABLE daily_input FROM '/tmp/backup/table-destination';
-----------
The above command will import the table "daily_input" to destination hive table.
IMPORTANT: No need to manually create the table in the destination, it will be automatically created during import.
Reference : http://www.ericlin.me/hive-export-import-command-transfering-data-between-hive-instances
No comments:
Post a Comment
Note: only a member of this blog may post a comment.