Hadoop Sqoop Interview Questions

hadoop jira, hadoop command, hadoop icon, hadoop with python, ubuntu install hadoop, hadoop in the cloud, hadoop fs, hadoop hdfs commands, hadoop data, hadoop capacity scheduler

1. Where & why is Sqoop is used? Explain with syntax
Ans: Sqoop is used for importing data from any rdbms using jdbc connector to hdfs or export an HDFS directory to a database table
Can I directly import data to Hive tables using Sqoop?
Ans: Yes

2. How do you perform a incremental data load?
Ans: By importing data using sqoop command or loading data to hive without overwrite.

--check-column (col) Specifies the column to be examined when determining which rows to import.
--incremental
(mode) Specifies how Sqoop determines which rows are new. Legal values for mode include append and lastmodified.
--last-value
(value) Specifies the maximum value of the check column from the previous import.

Sqoop supports two types of incremental imports: append and lastmodified. You can use the --incremental argument to specify the type of incremental import to perform.
You should specify append mode when importing a table where new rows are continually being added with increasing row id values. You specify the column containing the row’s id with --check-column. Sqoop imports rows where the check column has a value greater than the one specified with --last-value.


3. Can you do incremental import using sqoop? – didn’t answer the exact syntax but, told the underlying concept . answered above

4. You have a data in HDFS sytem, if you want to put some more data to into the same table, will it append the data or overwrite?

Ans:No it can’t overwrite, one way to do is copy the new file in HDFS.
Then concatenate files to a new file as shown below.

bin/hadoop fs -cat /user/username/folder/csv1.csv /user/username/folder/csv2.csv | hadoop fs -put - /user/username/folder/output.csv

hadoop fs -cat myfolder/* | hadoop fs -put - myfolder/output.txt

for multiple folders use:
for i in `hadoop fs -ls my-job-folder | cut -d' ' -f19` ;do `hadoop fs -cat $i/* |hadoop fs -put - $i/output.csv`; done


A table has 1000 records, but only 100 records are needed. How to achieve that?
Ans: Assign sequence number to each row and select sequence id < 101.
Else for random 100 records select top function or sample 100