SFTP SHELL SCRIPT USER GUIDE

Transcription

1 SFTP SHELL SCRIPT USER GUIDE FCA US INFORMATION & COMMUNICATION TECHNOLOGY MANAGEMENT

2 Overview The EBMX SFTP shell scripts provide a parameter driven workflow to place les on the EBMX servers and queue the les for processing. SFTP is the protocol used to transmit and queue the data to EBMX. SFTP requires that each environment (yours and EBMX) exchange Public Keys before any transmissions can occur. In order to expedite the setup, please submit a Trouble Ticket requesting SFTP setup to the TTTS group EBMX. [Please include server name/ip, application name, AMS#, and technical contact(s)]. The EBMX group will work with you during the setup and testing if you are unfamiliar with SFTP. The shell scripts (Korn & Bourne shell) were developed using Ubuntu Linux and tested on AIX and Solaris. The EBMX Group provides these scripts as-is, if you nd there is an issue or have a question about functionality, please submit a Trouble Ticket and route to EBMX. The EBMX SFTP shell script is one of two options for applications to access the functions of EBMX. EBMX also provides an interface based on IBM WebSphere MQ Series. These options should be discussed with an EBMX analyst to determine the best choice when selecting an interface. Features Transfer le using SFTP Accept parameters as arguments Check for expected response Full activity logging Return code setting Korn shell version (ksh) Bourne shell version (sh) Operating Environment The EBMX SFTP shell script is expected to run on UNIX variants such as Solaris, AIX, and Linux. The operating system must be able to support either the Korn shell or the Bourne shell and have a standard SFTP client installed. 04/29/ FCA US

3 Operation The EBMX SFTP shell script has two versions: Description Korn Shell Version Bourne Shell Version Shell Script Name sftp_put_rename.ksh sftp_put_rename.sh The EBMX SFTP shell script requires eight arguments: Name Description Value User ID The EBMX member ID of the sender of the data. Supplied by the EBMX administrative team. SFTP Host Name The IP address or DNS name of the EBMX system. EBMX Production = ebmxprod.appl.chrysler.com EBMX Development = ebmxdev.appl.chrysler.com SFTP Port SFTP port number Local File Name Name of the le to send including path, if necessary. Supplied by the user of the script. Sender ID The EBMX member ID of the sender of the data. Supplied by the EBMX administrative team. Receiver ID The EBMX member ID of the receiver of the data. Supplied by the EBMX administrative team. Document Type The document type of the data Supplied by the EBMX Log le name being sent. The log le name for activity logging including path, if necessary. administrative team. Supplied by the user of the script. All arguments are mandatory. The shell script exits with at return code of zero if the process was successful. The shell script exits with at return code of 1 if the process fails. It is the responsibility of the user of the shell script to check the return code for success or failure. If the process fails, interrogate the log le to determine the reason for the failure. The EBMX administrative team can assist in the interpretation of the log le. If the log le does not exist, the shell script will create a new one. If the log le does exist, the shell script will append the results to the end of the existing log. If the put and rename is successful, the log le will contain the character string Success!. If an error occurs at any time during the workflow, the character string Failure! is placed in the log le. Therefore, the EBMX team recommends that log les are kept for a minimum of seven days. 04/29/ FCA US

4 Examples Bourne Shell./sftp_put_rename.sh testdoc clmsd058b.cldc.chrysler.com test_le_1 TESTDOC LOOPTEST LOOPBACK mylog.log Korn shell./sftp_put_rename.ksh testdoc clmsd058b.cldc.chrysler.com test_le_1 TESTDOC LOOPTEST LOOPBACK mylog.log Note: The above examples should be entered as one continuous command and not broken into individual lines. 04/29/ FCA US

5 Example Log File Each run of the EBMX SFTP shell script will produce the following log: Success Example: Running on: :14:52:21 number of arguments received: 8 ARG[1] remoteuser: testdoc ARG[2] remotehost: clmsd058b.cldc.chrysler.com ARG[3] remoteport: ARG[4] remotele: test_le_1 ARG[5] ecxsender: TESTDOC ARG[6] ecxreceiver: LOOPTEST ARG[7] ecxdoctype: LOOPBACK ARG[8] remotelog: mylog.log localle for put is: test_le_ Connecting to clmsd058b.cldc.chrysler.com... You are accessing FCA (Fiat Chrysler Automobiles) systems. Connected to EBMX SFTP server. Connection accepted. Access to FCA's computer systems is controlled. FCA authorizes use for its business purposes only. Unauthorized access is prohibited due to risk of irreparable harm to FCA. FCA management may monitor use to ensure compliance with its policies. FCA may terminate access privileges, take disciplinary action and/or institute civil or criminal proceedings to enforce this policy. If any part of this policy is unacceptable to you, please disconnect now!!! Thank you. YOUR ACCESS IS BEING LOGGED & MONITORED!!!! sftp> dir ERRORS RESULTS TESTDOC TMP sftp> cd /TMP sftp> put test_le_1 test_le_ Uploading test_le_1 to /TMP/test_le_ sftp> rename test_le_ /TESTDOC/LOOPTEST/LOOPBACK/test_le_ sftp> quit End of SFTP PUT and RENAME About to check exit status... Success! 04/29/ FCA US

6 Failure Example: (Invalid le) Running on: :14:52:41 number of arguments received: 8 ARG[1] remoteuser: testdoc ARG[2] remotehost: clmsd058b.cldc.chrysler.com ARG[3] remoteport: ARG[4] remotele: xtest_le_1 ARG[5] ecxsender: TESTDOC ARG[6] ecxreceiver: LOOPTEST ARG[7] ecxdoctype: LOOPBACK ARG[8] remotelog: mylog.log localle for put is: xtest_le_ Connecting to clmsd058b.cldc.chrysler.com... You are accessing FCA (Fiat Chrysler Automobiles) systems. Connected to EBMX SFTP server. Connection accepted. Access to FCA's computer systems is controlled. FCA authorizes use for its business purposes only. Unauthorized access is prohibited due to risk of irreparable harm to FCA. FCA management may monitor use to ensure compliance with its policies. FCA may terminate access privileges, take disciplinary action and/or institute civil or criminal proceedings to enforce this policy. If any part of this policy is unacceptable to you, please disconnect now!!! Thank you. YOUR ACCESS IS BEING LOGGED & MONITORED!!!! sftp> dir ERRORS RESULTS TESTDOC TMP sftp> cd /TMP sftp> put xtest_le_1 xtest_le_ stat xtest_le_1: No such le or directory sftp> rename xtest_le_ /TESTDOC/LOOPTEST/LOOPBACK/xtest_le_ Couldn't rename le "/TMP/xtest_le_ " to "/TESTDOC/LOOPTEST/LOOPBACK/xtest_le_ ": No such le or directory sftp> quit End of SFTP PUT and RENAME About to check exit status... Failure! 04/29/ FCA US

7 Failure Example: (Invalid Public Key) Running on: :14:53:16 number of arguments received: 8 ARG[1] remoteuser: testdoc ARG[2] remotehost: clmsd058b.cldc.chrysler.com ARG[3] remoteport: ARG[4] remotele: xtest_le_1 ARG[5] ecxsender: TESTDOC ARG[6] ecxreceiver: LOOPTEST ARG[7] ecxdoctype: LOOPBACK ARG[8] remotelog: mylog.log localle for put is: xtest_le_ Connecting to clmsd058b.cldc.chrysler.com... You are accessing FCA (Fiat Chrysler Automobiles) systems. Connected to EBMX SFTP server. Connection accepted. Access to FCA's computer systems is controlled. FCA authorizes use for its business purposes only. Unauthorized access is prohibited due to risk of irreparable harm to FCA. FCA management may monitor use to ensure compliance with its policies. FCA may terminate access privileges, take disciplinary action and/or institute civil or criminal proceedings to enforce this policy. If any part of this policy is unacceptable to you, please disconnect now!!! Thank you. YOUR ACCESS IS BEING LOGGED & MONITORED!!!! Permission denied (publickey). Couldn't read packet: Connection reset by peer End of SFTP PUT and RENAME About to check exit status... Failure! 04/29/ FCA US

8 Appendix A. Shell Scripts (Korn) Copy and save to a le with the extenstion.ksh #!/bin/ksh # # This script will accept parameters/arguments and connect to the EBMX app # server via the OpenSourceSSH SFTP server and PUT a le. The reply # is examined for any errors - this determines success/failure of this # script. This script is intended for use with ksh. # # store the date for display/le naming purposes datetime=`date '+%Y%m%d%H%M%S'` displaydatetime=`date '+%Y-%m-%d:%H:%M:%S'` sftp_response_log=" " # store the number of arguments received and the arguments themselves numargsreceived=$# argumentsreceived=$* remoteuser=$1 remotehost=$2 remoteport=$3 remotele=$4 ecxsender=$5 ecxreceiver=$6 ecxdoctype=$7 remotelog=$8 # construct the name of the local le to be PUT localle="$remotele.$datetime.$random" # this routine echos the date, arguments and localle echo_args () { echo "Running on: $displaydatetime"; echo "number of arguments received: $numargsreceived"; echo; # check to see if the number of arguments is correct if [ $numargsreceived -eq 8 ]; then echo "ARG[1] remoteuser: $remoteuser"; echo "ARG[2] remotehost: $remotehost"; echo "ARG[3] remoteport: $remoteport"; echo "ARG[4] remotele: $remotele"; echo "ARG[5] ecxsender: $ecxsender"; echo "ARG[6] ecxreceiver: $ecxreceiver"; echo "ARG[7] ecxdoctype: $ecxdoctype"; echo "ARG[8] remotelog: $remotelog"; echo; elif [ $numargsreceived -gt 8 ]; then echo "Too many arguments received!!!"; echo "Received $numargsreceived arguments"; echo "Arguments received: $argumentsreceived"; echo "Arguments expected: <remoteuser> <remotehost> <remoteport> <remotele> <ecxsender> <ecxreceiver> <ecxdoctype> [<remotelog>]"; exit 1; elif [ $numargsreceived -lt 8 ]; then echo "Too few arguments received!!!"; echo "Received $numargsreceived arguments"; echo "Arguments received: $argumentsreceived"; echo "Arguments expected: <remoteuser> <remotehost> <remoteport> <remotele> <ecxsender> <ecxreceiver> <ecxdoctype> [<remotelog>]"; exit 1; echo "localle for put is: $localle"; echo; } 04/29/ FCA US

9 Appendix A. Shell Scripts (Korn) (continued) # this routine is responsible for the PUT of the le and RENAME # we also do a DIR and PWD for completeness sftp_put_rename () { # open sftp as a co-process (ksh only) so that we may write to/read from the # process. The reads are stored in a shell variable sftp -oport=$remoteport $remoteuser@$remotehost 2>&1 & print -p dir print -p cd /TMP print -p put $remotele $localle print p chmod 666 $localle print -p rename $localle /$ecxsender/$ecxreceiver/$ecxdoctype/$localle print -p quit print -p : # read all the responses at the end of the process while read -p sftp_response do # add a new line so it looks right - otherwise, it wraps on one line without # newlines xx="$sftp_response"; echo "$xx" >> $remotelog; sftp_response_log="$sftp_response_log$xx"; done } : # this runs the echo_args routine - the eval is necessary for lename # expansion prior to redirection eval echo_args 1>>$remotelog 2>&1 # run the sftp_put_rename routine - since we use a coprocess to store output # in a shell variable, we don't need to worry about redirection sftp_put_rename # grep the SFTP session log looking for Couldn denied - if we nd one, it failed echo "End of SFTP PUT and RENAME" >>$remotelog 2>&1 echo "About to check exit status..." >>$remotelog 2>&1 if [ `echo $sftp_response_log egrep -c '(Couldn denied)'` -eq 0 ] then echo "Success!" >>$remotelog 2>&1; exit 0; else echo "Failure!" >>$remotelog 2>&1; exit 1; 04/29/ FCA US

10 Appendix B. Shell Scripts (Bourne Shell) Copy and save to a le with the extenstion.sh #!/bin/sh # # This script will accept parameters/arguments and connect to the EBMX app # server via the OpenSourceSSH SFTP server and PUT a le. The reply # is examined for any errors - this determines success/failure of this # script. This script is intended for use with ksh. # # store the date for display/le naming purposes datetime=`date '+%Y%m%d%H%M%S'` displaydatetime=`date '+%Y-%m-%d:%H:%M:%S'` # store the number of arguments received and the arguments themselves numargsreceived=$# argumentsreceived=$* remoteuser=$1 remotehost=$2 remoteport=$3 remotele=$4 ecxsender=$5 ecxreceiver=$6 ecxdoctype=$7 remotelog=$8 #echo $datetime; # dene temporary log... tempremotelog="./sftp_log.$datetime"; # construct the name of the local le to be PUT localle="$remotele.$datetime" if [ $# -gt 0 ] then numargsreceived=$#; else numargsreceived=0; argsreceived=$*; # this routine echos the date, arguments and localle echo_args () { echo "Running on: $displaydatetime"; echo "number of arguments received: $numargsreceived"; # check to see if the number of arguments is correct if [ $numargsreceived -eq 8 ]; then echo "Start of FTP PUT and SUBMIT"; exiterror=0; elif [ $numargsreceived -eq 7 ]; # generate a remote log le when only 7 arguments are passed then remotelog=$tempremotelog; exiterror=0; elif [ $numargsreceived -gt 8 ]; then echo "Too many arguments received!!!"; echo "Received $numargsreceived arguments"; echo "Arguments received: $argsreceived"; echo "Arguments expected: <remoteuser> <remotehost> <remoteport> <remotele> <ecxsender> <ecxreceiver> <ecxdoctype> [<remotelog>]"; exiterror=1; elif [ $numargsreceived -lt 7 ]; then echo "Too few arguments received!!!"; echo "Received $numargsreceived arguments"; echo "Arguments received: $argsreceived"; echo "Arguments expected: <remoteuser> <remotehost> <remoteport> <remotele> <ecxsender> <ecxreceiver> <ecxdoctype> [<remotelog>]"; exiterror=1; 04/29/ FCA US

11 Appendix B. Shell Scripts (Bourne Shell) (continued) echo; echo "ARG[1] remoteuser: $remoteuser"; echo "ARG[2] remotehost: $remotehost"; echo "ARG[3] remoteport: $remoteport"; echo "ARG[4] remotele: $remotele"; echo "ARG[5] ecxsender: $ecxsender"; echo "ARG[6] ecxreceiver: $ecxreceiver"; echo "ARG[7] ecxdoctype: $ecxdoctype"; echo "ARG[8] remotelog: $remotelog"; echo "Local File: $localle"; echo "Temp Log: $tempremotelog"; echo; } # this routine is responsible for the PUT of the le and ensuing SUBMIT # we also do a DIR and PWD for completeness sftp_put_rename () { sftp -oport=$remoteport $remoteuser@$remotehost <<!EOF cd /TMP put $remotele $localle chmod 666 $localle rename $localle /$ecxsender/$ecxreceiver/$ecxdoctype/$localle quit!eof echo End of SFTP PUT and RENAME; : } # this runs the echo_args routine - the eval is necessary for lename # expansion prior to redirection eval echo_args 1>>$tempremotelog 2>&1 # this runs the sftp_put_rename routine - the eval is necessary for lename # expansion prior to redirection if [ $exiterror -ne 1 ]; then eval sftp_put_rename 1>>$tempremotelog 2>&1; else echo "Incorrect number of Arguments, please see $tempremotelog"; exit; echo About to check exit status... >>$tempremotelog # grep the remote log looking for a tracking ID - if we nd one, it # worked if [ $exiterror -ne 1 ]; then it_worked=`egrep -c '(Couldn denied)' $tempremotelog`; else it_worked=0; # if we nd one tracking ID, it worked. If we nd more than one, the log # contains multiple sessions, and we can't tell for sure if the last one # worked if [ $it_worked -eq 0 ]; then echo "Success!" >>$tempremotelog; exitcode=0; else echo "Failure!" >>$tempremotelog; exitcode=1; if [ "$tempremotelog"!= "$remotelog" ]; then cat $tempremotelog >>$remotelog; rm $tempremotelog; exit $exitcode; 04/29/ FCA US