This tutorial is all about on how to establish connectivity between HDFS and Teradata using Sqoop teradata connector
Installing Cloudera Connector for Teradata:
To import/export data form Hdfs to teradata using Sqoop(Assuming Sqoop is installed)
1)Download sqoop-td-connector-1.0.5.jar and put it in /user/lib/sqoop/lib
2)To ensure that Cloudera connector for Teradata can be used, copy the Teradata JDBC drivers (terajdbc4.jar and tdgssconfig.jar) to the lib directory of Sqoop installation. Without these drivers, the connector will not function correctly.
3)Create a text file called connectors in a directory named managers.d within the configuration directory of Sqoop. Create the directory managers.d if it does not exist.
4)The connectors file must have the connector class name followed by the complete path to the directory where the connector jar is located.
for Example
com.cloudera.sqoop.manager.TeradataManagerFactory=/usr/lib/sqoop/lib/sqoop-td-connector-1.0.5.jar
The TeradataManagerFactory acts as a single point of delegation for invoking the connector bundled with this distribution. An alternate way to specify TeradataManagerFactory is to add the following inside a sqoop-site.xml file, which must be inside a classpath directory:
<configuration>
<property>
<name>sqoop.connection.factories</name>
<value>com.cloudera.sqoop.manager.TeradataManagerFactory</value>
</property>
</configuration>
Using Cloudera Connector for Teradata:
Now you can use this connector by invoking the Sqoop tools with the appropriate connection string.
The connection string must be of the form jdbc:teradata://<td-host>/DATABASE=<td-
instance> where
<td-host> is the host name of the machine where Teradata server runs.
<td-instance> is the Teradata database instance name.
The connector automatically uses FastExport/FastLoad for better performance.
For example, the following command invokes the Sqoop import tool with three mappers and uses the
Cloudera connector for Teradata:
$ sqoop import --connect jdbc:teradata://localhost/DATABASE=sqooptest --username sqooptest --password xxxxx --table MY_TABLE --num-mappers 3 --escaped-by '\\' --fields-terminated-by ',' --lines-terminated-by '\n'
The following command invokes the Sqoop export tool with three mappers and uses the Cloudera
connector for Teradata:
$ sqoop export --connect jdbc:teradata://localhost/DATABASE=sqooptest --username sqooptest --password xxxxx --export-dir /user/john/MY_TABLE --table MY_TABLE_TARGET --num-mappers 3 --input-escaped-by '\\' --input-fields-terminated-by ','
Installing Cloudera Connector for Teradata:
To import/export data form Hdfs to teradata using Sqoop(Assuming Sqoop is installed)
1)Download sqoop-td-connector-1.0.5.jar and put it in /user/lib/sqoop/lib
2)To ensure that Cloudera connector for Teradata can be used, copy the Teradata JDBC drivers (terajdbc4.jar and tdgssconfig.jar) to the lib directory of Sqoop installation. Without these drivers, the connector will not function correctly.
3)Create a text file called connectors in a directory named managers.d within the configuration directory of Sqoop. Create the directory managers.d if it does not exist.
4)The connectors file must have the connector class name followed by the complete path to the directory where the connector jar is located.
for Example
com.cloudera.sqoop.manager.TeradataManagerFactory=/usr/lib/sqoop/lib/sqoop-td-connector-1.0.5.jar
The TeradataManagerFactory acts as a single point of delegation for invoking the connector bundled with this distribution. An alternate way to specify TeradataManagerFactory is to add the following inside a sqoop-site.xml file, which must be inside a classpath directory:
<configuration>
<property>
<name>sqoop.connection.factories</name>
<value>com.cloudera.sqoop.manager.TeradataManagerFactory</value>
</property>
</configuration>
Using Cloudera Connector for Teradata:
Now you can use this connector by invoking the Sqoop tools with the appropriate connection string.
The connection string must be of the form jdbc:teradata://<td-host>/DATABASE=<td-
instance> where
<td-host> is the host name of the machine where Teradata server runs.
<td-instance> is the Teradata database instance name.
The connector automatically uses FastExport/FastLoad for better performance.
For example, the following command invokes the Sqoop import tool with three mappers and uses the
Cloudera connector for Teradata:
$ sqoop import --connect jdbc:teradata://localhost/DATABASE=sqooptest --username sqooptest --password xxxxx --table MY_TABLE --num-mappers 3 --escaped-by '\\' --fields-terminated-by ',' --lines-terminated-by '\n'
The following command invokes the Sqoop export tool with three mappers and uses the Cloudera
connector for Teradata:
$ sqoop export --connect jdbc:teradata://localhost/DATABASE=sqooptest --username sqooptest --password xxxxx --export-dir /user/john/MY_TABLE --table MY_TABLE_TARGET --num-mappers 3 --input-escaped-by '\\' --input-fields-terminated-by ','