Using sftp in Informatica PowerCenter Applies to: Informatica PowerCenter Summary This article briefs about how to push/pull files using SFTP program in Informatica PowerCenter. Author Bio Author(s): Sukumar Balasubramanian Company: CIBC Created on: February 02, 2010 Sukumar Balasubramanian is an experienced Informatica ETL Consultant working with CIBC, Canada. He has good exposure to Data Integration/Data Warehousing Projects. He is also a key contributor in informatica-l group of ITtoolbox. 2009 Informatica Corporation. All Rights Reserved. 1
Table of Contents INTRODUCTION... 3 WHAT IS SFTP, AND HOW DO I USE IT TO TRANSFER FILES?... 3 PREREQUISITES... 3 SFTP for UNIX... 3 SFTP for Windows... 3 HOW SFTP Works... 3 INFORMATICA SFTP CONNECTOR... 4 SHELL SCRIPT IN INFORMATICA... 5 SAMPLE SHELL SCRIPT... 6 FILE TRANSFER SPEED, SCP VS SFTP... 6 Disclaimer and Liability notice... 7 2009 Informatica Corporation. All Rights Reserved. 2
INTRODUCTION This article briefs about how to push/pull files using SFTP program in Informatica PowerCenter WHAT IS SFTP, AND HOW DO I USE IT TO TRANSFER FILES? SFTP, or secure FTP, is a program that uses SSH to transfer files. Unlike standard FTP, it encrypts both commands and data, preventing passwords and sensitive information from being transmitted in the clear over the network. It is functionally similar to FTP, but because it uses a different protocol, you can't use a standard FTP client to talk to an SFTP server, nor can you connect to an FTP server with a client that supports only SFTP. PREREQUISITES SFTP for UNIX 1. By default all the UNIX environment have SSH pre installed. SFTP for Windows 1. You need to configure Windows to use SSH protocol. 2. There are many free wares like WINSCP, OpenSSH and Putty. You need to install this on your windows machine. 3. Most of the free wares have GUI but you must choose the one that has the command line execution. HOW SFTP Works 2009 Informatica Corporation. All Rights Reserved. 3
You need to create a ssh keys on the informatica server using ssh keygen command in Unix Share the public key with the team that maintains your SFTP server and ask them to register this public key on the SFTP server. Execute the sftp <command> very first time it will create a finger key print between the Informatica Server and the SFTP server. Subsequently you can just issue SFTP or SCP file1 file2. You need not specify any password. INFORMATICA SFTP CONNECTOR An easy approach would be creating a SFTP connector in Informatica. Before you can configure a session to use FTP or SFTP, you must create and configure the FTP connection properties in the Workflow Manager. The Integration Service uses the FTP connection properties to create an SFTP or FTP connection To create an FTP connection: 1. In the Workflow Manager, connect to a repository. 2. Click Connections > FTP. The FTP Connection Browser appears. 3. Click New. 4. Enter the properties for the FTP connection 5. Click OK. To connect to an SFTP server, create an FTP connection and enable SFTP. SFTP uses the SSH2 authentication protocol. Configure the authentication properties to use the SFTP connection. You can configure publickey or password authentication. The Integration Service connects to the SFTP server with the authentication properties you configure. If the authentication does not succeed, the session fails The following table describes the key properties that you configure for an SFTP: Host Name If you enable SFTP, specify a host name or port number for an SFTP server. If you do not specify a port number, the Integration Service uses 22 by default. Default Remote Directory Default directory on the FTP host used by the Integration Service. Do not enclose the directory in quotation marks. You can enter a parameter or variable for the directory. Use any parameter or variable type that you can define in the parameter file. Depending on the FTP server you use, you may have limited options to enter FTP directories. For details, see the FTP server documentation. In the session, when you enter a file name without a directory, the Integration Service appends the file name to this directory. This path must contain the appropriate trailing delimiter. For example, if you enter c:\staging\ 2009 Informatica Corporation. All Rights Reserved. 4
and specify data.out in the session, the Integration Service reads the path and file name as c:\staging\data.out. Public Key File Name Public key file path and file name. Required if the SFTP server uses publickey authentication. Enabled for SFTP. Private Key File Name Private key file path and file name. Required if the SFTP server uses publickey authentication. Enabled for SFTP. Private Key File Password Private key file password used to decrypt the private key file. Required if the SFTP server uses public key authentication and the private key is encrypted. Enabled for SFTP. After the Informatica SFTP connector is configured we can use this connector in the session properties and just provide the name of the file that we like to push/pull from the SFTP server. Your session will directly read or write from/to the SFTP server. SHELL SCRIPT IN INFORMATICA Informatica does not support all the SSH software s so you may have difficulties in configuring an SFTP connector in Informatica. Shell scripts can be used to SFTP / SCP a file from informatica server. First you need to create the ssh keys and share the keys You can execute the following command to verify your setup. sftp username@ host Once you issue the above command you must be taken to the sftp prompt. You can execute any of the sftp commands and see whether it works. You can develop your own shell scripts and call using the pre/post session command Another approach would be having a wrapper shell script that push/pull files from SFTP and then execute a pmcmd command to execute your workflow. 2009 Informatica Corporation. All Rights Reserved. 5
SAMPLE SHELL SCRIPT sftp username@host if [ $? = 0 ] then echo "Connected" else echo "ERROR in connecting to the SFTP server " fi FILE TRANSFER SPEED, SCP VS SFTP Although both SCP and SFTP utilize the same SSH encryption during file transfer with the same general level of overhead, SCP is usually much faster than SFTP at transferring files especially on high latency networks. This happens because SCP implements a more efficient transfer algorithm, one which does not require waiting for packet confirmations. This leads to faster speed but comes at the expense of not being able to interrupt a transfer, so unlike SFTP, SCP transfer cannot be canceled without terminating the session. Example: scp username@ host:sourcefile Destinationfile if [ $? = 0 ] then echo "Connected and copied the file else echo "ERROR in copying the file using SCP " fi 2009 Informatica Corporation. All Rights Reserved. 6
Disclaimer and Liability notice Informatica offers no guarantees and assumes no responsibility or liability of any type with respect to the content of this software asset, including any liability resulting from incompatibility between the content within this asset and the materials and services offered by Informatica. You agree that you will not hold, or seek to hold, Informatica responsible or liable with respect to the content of this software asset. 2009 Informatica Corporation. All Rights Reserved. 7