Oracle Cloud E

Transcription

1 Oracle Cloud Using Oracle Managed File Transfer Cloud Service E June 2016 Oracle Managed File Transfer (MFT) is a standards-based, endto-end managed file gateway. Security is maintained with a number of security policies such as OWSM. This guide describes how to use MFT in the cloud.

2 Oracle Cloud Using Oracle Managed File Transfer Cloud Service, E Copyright 2016, Oracle and/or its affiliates. All rights reserved. Primary Authors: Larry Hoffman, Mary Kennedy This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing. If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, the following notice is applicable: U.S. GOVERNMENT END USERS: Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agencyspecific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs. No other rights are granted to the U.S. Government. This software or hardware is developed for general use in a variety of information management applications. It is not developed or intended for use in any inherently dangerous applications, including applications that may create a risk of personal injury. If you use this software or hardware in dangerous applications, then you shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure its safe use. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerous applications. Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners. Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group. This software or hardware and documentation may provide access to or information about content, products, and services from third parties. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services unless otherwise set forth in an applicable agreement between you and Oracle. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services, except as set forth in an applicable agreement between you and Oracle.

3 Contents Preface... Audience... Related Resources... Conventions... v v v v 1 Getting Started with Oracle Managed File Transfer Cloud Service About Oracle Managed File Transfer Cloud Service Differences Between the Cloud and On-Premises Environments Provisioning Oracle Managed File Transfer Cloud Service User and Administrator Tasks Review and Configure Retention Policies Post-Provisioning Tasks Configuring Compute and OTD for the MFT Embedded Server Configuring the Compute Environment Configuring Oracle Traffic Director (OTD) Configuring DBFS for Shared Storage iii

4 iv

5 Preface Using Oracle Managed File Transfer Cloud Service describes how to use Oracle Managed File Transfer Cloud Service in the cloud. Topics: Audience Related Resources Conventions Audience Using Oracle Managed File Transfer Cloud Service is intended for users who want to transfer files with Oracle Managed File Transfer Cloud Service in the cloud. Related Resources Conventions For more information, see these Oracle resources: Oracle Public Cloud Getting Started with Oracle Cloud Using Oracle SOA Cloud Service The following text conventions are used in this document: Convention boldface italic monospace Meaning Boldface type indicates graphical user interface elements associated with an action, or terms defined in text or the glossary. Italic type indicates book titles, emphasis, or placeholder variables for which you supply particular values. Monospace type indicates commands within a paragraph, URLs, code in examples, text that appears on the screen, or text that you enter. v

6

7 1 Getting Started with Oracle Managed File Transfer Cloud Service The following topics describe Oracle Managed File Transfer Cloud Service concepts, provisioning requirements and post-provisioning tasks. Topics About Oracle Managed File Transfer Cloud Service Differences Between the Cloud and On-Premises Environments Provisioning Oracle Managed File Transfer Cloud Service Post-Provisioning Tasks About Oracle Managed File Transfer Cloud Service Oracle Managed File Transfer (MFT) is a standards-based, end-to-end managed file gateway. Security is maintained with a number of security policies such as OWSM. To learn more about Oracle Managed File Transfer Cloud Service, see Using Oracle Managed File Transfer. Differences Between the Cloud and On-Premises Environments Some features of Oracle Managed File Transfer behave differently in the cloud than in an on-premises environment. Oracle Managed File Transfer Cloud Service You provision Oracle Managed File Transfer Cloud Service from the SOA Cloud Service console. Run the Oracle SOA Cloud Service provisioning wizard and select Oracle Managed File Transfer Cloud Service. See Provisioning Oracle Managed File Transfer Cloud Service for information about the Oracle SOA Cloud Service provisioning wizard, including prerequisites for running the wizard. The first release of Oracle Managed File Transfer Cloud Service is This differs from the latest release of on-premises Oracle Managed File Transfer Cloud Service. Oracle Managed File Transfer On Premises You must manually install Oracle Managed File Transfer on your own hardware. The latest release of on-premises Oracle Managed File Transfer is Getting Started with Oracle Managed File Transfer Cloud Service 1-1

8 Provisioning Oracle Managed File Transfer Cloud Service Oracle Managed File Transfer Cloud Service Oracle Managed File Transfer On Premises File source and targets are available, but not typically used in the cloud. The FTP sever is not typically used in secure environments. It is recommended that you use sftp exclusively in the cloud. For a complete list of differences between Oracle SOA in the cloud and on-premises environments, see Differences Between the Cloud and On-Premises Environments of Using Oracle SOA Cloud Service. Provisioning Oracle Managed File Transfer Cloud Service Select Oracle Managed File Transfer Cloud Service when you run the Oracle SOA Cloud Service provisioning wizard. From the Oracle SOA Cloud Service console, click the Create Instance button to start the provisioning wizard. On the Service Type page, select MFT Cluster, then complete the provisioning process. Preprovisioning and Provisioning Tasks The following table provides an overview of the pre-provisioning and provisioning tasks to complete to subscribe to Oracle Managed File Transfer Cloud Service. Task For More Information, See... Before you begin running the Oracle SOA Cloud Service provisioning wizard, you must satisfy the following prerequisites: Subscribe to Oracle Database Cloud Service Subscribe to Oracle Storage Cloud Service Obtain a secure shell (SSH) public/ private key pair When you run the Oracle SOA Cloud Service provisioning wizard, you are prompted to enter details about these services and components. Understand how the Oracle SOA Cloud Service provisioning wizard works. Understand how you access the Oracle SOA Cloud Service Console to run the provisioning wizard. Run the provisioning wizard and select MFT Cluster as the service type. Before You Begin with Oracle SOA Cloud Service of Using Oracle SOA Cloud Service. and Prerequisites to Provisioning Oracle SOA Cloud Service of Using Oracle SOA Cloud Service. Subscribing to Oracle SOA Cloud Service of Using Oracle SOA Cloud Service. Accessing Oracle SOA Cloud Service Console of Using Oracle SOA Cloud Service. Using the Provisioning Wizard of Using Oracle SOA Cloud Service. 1-2 Using Oracle Managed File Transfer Cloud Service

9 User and Administrator Tasks Note: After you complete the provisioning process, you must complete the postprovisioning tasks described in Post-Provisioning Tasks. User and Administrator Tasks The following table provides an overview of the user and administrator tasks you can perform after you have provisioned Oracle Managed File Transfer Cloud Service and completed the post-provisioning tasks. User and Administrator Tasks Task For More Information, See... See what you can do with Oracle Managed File Transfer. Learn about Oracle Managed File Transfer use case patterns. Understand Oracle Managed File Transfer architecture. Understand Oracle Managed File Transfer components. Understand Oracle Managed File Transfer sources, targets, and transfers. Understand Oracle Managed File Transfer user roles. Understand Oracle Managed File Transfer navigation. Review and Configure Retention Policies What You Can Do with Oracle Managed File Transfer Oracle Managed File Transfer Functional Use Case Patterns Oracle Managed File Transfer Architecture Components of Oracle Managed File Transfer Artifacts: Sources, Targets, and Transfers Oracle Managed File Transfer User Roles Screen Navigation in Oracle Managed File Transfer Review and Configure Retention Policies Review and Configure Retention Policies File storage in the cloud might be constrained so it s important to proactively manage limited file storage services by purging old instances and payloads. In order to keep your database and storage resources from filling up, periodically purge artifacts from previous transfers. By default, Oracle Managed File Transfer Cloud Service checks daily at 8:00 AM for files that have been around longer than seven days and purges any that it finds. You can change these default values to suit your needs. How often you purge depends on your usage as does the length of time you retain the artifacts. The tasks in the following topics describe how to change the retention value and the schedule. Getting Started with Oracle Managed File Transfer Cloud Service 1-3

10 User and Administrator Tasks Topics Changing the Purge Retention Policy Changing the Purge Schedule Changing the Purge Retention Policy If you want to change the length of time you retain transfer artifacts such as old instances and payloads, edit the purgeinstancedata.py file located on each MFT managed server. The following steps describe how to change the retention policy. 1. ssh to the mft-purge directory located under the domain home directory on each managed server VM. 2. On each MFT managed server, modify the retention days value by editing the following line in purgeinstancedata.py. retentiondays = 7 # Duration for which MFT Data has to be retained during purge operation For example: retentiondays = 7 purges all data older than 1 week (default) retentiondays = 14 purges all data older than 2 weeks retentiondays = 1 purges all data older than 1 day Note: This file can be edited at any time and the change takes effect for all future scheduled purges. Changing the Purge Schedule By default, Oracle Managed File Transfer Cloud Service checks daily at 8:00 AM for files that have been around longer than the retention value and purges any that it finds. You can change the schedule to suit your needs. How often you purge depends on your usage. The following steps describe how to change the schedule by editing thepurge_schedule.py script, stopping and deleting the original scheduler job and then running the startschedulepurge.sh script to start the new schedule. 1. ssh to the mft-purge directory located under the domain home directory on each managed server VM. 2. On each MFT managed server, modify the purge schedule by editing the values for frequency, interval, and begin_time in the following line in purge_schedule.py. manageschedulerschedule('create', 'MFTCustomHostingApp',schName = schedulename, frequency = 'DAY',interval = 1,begin_time = '08:00:00:01:01:2015') frequency: The frequency of recurrence. Valid values are: [SECOND, MINUTE,HOUR,DAY, WEEK,MONTH, YEAR] 1-4 Using Oracle Managed File Transfer Cloud Service

11 User and Administrator Tasks interval: The integer value specifying the repeat interval for frequency. For example, an interval value of 2 combined with a frequency of WEEK, runs once every 2 weeks. begin_time: The start time for the recurring schedule. The format is: HH:MM:SS:DD:MM:YYYY Examples: To run once per day at 8:00 AM: manageschedulerschedule('create', 'MFTCustomHostingApp',schName = schedulename, frequency = 'DAY',interval = 1, begin_time = '08:00:00:01:01:2016') To run every two days at 8:00 AM: manageschedulerschedule('create', 'MFTCustomHostingApp',schName = schedulename, frequency = 'DAY',interval = 2, begin_time = '08:00:00:01:01:2016') To run once per week at midnight: manageschedulerschedule('create', 'MFTCustomHostingApp',schName = schedulename, frequency = 'WEEK',interval = 1, begin_time = '00:00:00:01:01:2016') 3. On any one of the running managed servers, follow these steps to cancel all existing jobs associated with the schedule. a. Log in to the Oracle Enterprise Manager (EM) console. b. Find Scheduling Services in the EM user interface. c. Click ESSAPP(mft_managed_server). For example, ESSAP(mft_server1). d. Click Show All Running Jobs. e. A search menu should appear. Find the name of the job you want to delete. For more information, see Searching for a Job Request Using Simple Search in Administering Oracle Enterprise Scheduler. f. Click the job request that corresponds to the job you want to delete. g. Click Actions and then select Cancel to stop the job. 4. Connect to WLST. a. ssh to any one of the MFT managed server virtual machines and log in. b. cd to the bin directory under the MFT installation directory: cd mft_installation_directory/oracle_common/common/bin c. Run the./wlst.sh command. d. Use the following command to connect to the server: connect( username, password, t3://url_to_server:port_number ); 5. Use the following commands to delete the schedule that you want to update: Getting Started with Oracle Managed File Transfer Cloud Service 1-5

12 User and Administrator Tasks manageschedulerschedule('delete','mftcustomhostingapp',schname='mft/ purgemftinstances') manageschedulerjobdefn('delete','mftcustomhostingapp',jobname='mft/mft_purge') 6. Use the following commands to verify that the schedule and job have been deleted from the hosting application. a. manageschedulerjobdefn('show','mftcustomhostingapp') This command lists jobs running on the hosting application. Verify that the job definition you deleted is not listed. b. manageschedulerschedule('show','mftcustomhostingapp') This command lists schedules running on the hosting application. Verify that the schedule you deleted is not listed. 7. On any one of the Oracle Managed File Transfer managed server nodes, run the startschedulepurge.sh script located in the mft-purge directory located under the domain home directory on each managed server VM. For more information about running Oracle Enterprise Scheduler WLST commands, see Oracle Enterprise Scheduler Custom WLST Commands. For information about using Oracle Enterprise Manager to monitor Oracle Enterprise Scheduler jobs, see Managing Oracle Enterprise Scheduler Requests in Administering Oracle Enterprise Scheduler. 1-6 Using Oracle Managed File Transfer Cloud Service

13 2 Post-Provisioning Tasks For the current release of Oracle Managed File Transfer Cloud Service, there are a few additional post-provisioning tasks you have to complete for the service to work correctly. The following topics describe these tasks. Topics Configuring DBFS for Shared Storage Configuring Compute and OTD for the MFT Embedded Server Configuring Compute and OTD for the MFT Embedded Server For this release of Oracle Managed File Transfer (MFT), you must open the required port(s) in the pod compute configuration to allow SFTP traffic to MFT. The MFT embedded SFTP server allows SFTP clients to connect to MFT to upload and download files. The connection is initiated by the client running elsewhere on the network, therefore the connection is inbound from the perspective of the SOACS MFT pod. In the default configuration of MFT/SOACS pods, only the following two types of inbound connections are allowed: ssh to the administration server or OTD http or https to OTD and the managed servers By default, all other inbound connections are blocked. Note that while SFTP is an extension of the SSH protocol, by defalut the MFT embedded SFTP server uses port 7522 rather than the SSH port 22. Therefore, you must open the required port(s) in the pod compute configuration to allow SFTP traffic to MFT. In addition, a clustered environment running OTD as the load-balancer also requires changes in OTD in order for the inbound SFTP connections to be proxied to the MFT managed servers. Note: The compute network configuration changes only have to be done once per port that is opened, but the OTD server pool configuration has to be synchronized with the actual cluster configuration following a scale-in or scale-out operation. Topics Configuring the Compute Environment Configuring Oracle Traffic Director (OTD) Post-Provisioning Tasks 2-1

14 Configuring Compute and OTD for the MFT Embedded Server Configuring the Compute Environment Use the Oracle Compute Cloud Service console to configure the compute environment. In order to open a port, use the Oracle Compute Cloud Service console to make the following three changes: Add a security application that defines the port to be opened. Define a security rule that allows anyone coming from the public internet to connect to OTD with the above defined security application. Define another security rule that allows OTD to connect to all the managed servers with the same security application. Follow the step-by-step instructions in the tutorial Permitting Public TCP Traffic to Oracle Compute Cloud Service Instances. Note the following alterations to the tutorial required for MFT: 1. Define the security application as described in the tutorial. Note that the default MFT SFTP port is The tutorial only creates one security rule. Create the following two security rules instead: The first rule should allow the security IP list source public-internet to connect to the destination ora-otd. The second rule should allow the security list source ora-otd to connect to destination ora_ms. Configuring Oracle Traffic Director (OTD) In order for OTD to load-balance inbound SFTP traffic, a TCP proxy has to be set up with a listener configured on the required port, and a server pool has to be configured to which OTD forwards the incoming requests. 1. Create a TCP proxy. Follow the procedure described in Creating a TCP Proxy in the Oracle Traffic Director Administrator's Guide. Note that the wizard allows you to create the TCP listener and the server pool at the same time. Enter the port that you opened in Configuring the Compute Environment in the New TCP Proxy Wizard as the TCP listener port. 2. Create the server pool. Follow the procedure described in Creating an Origin-Server Pool in the Oracle Traffic Director Administrator's Guide. Add all the managed server hosts in the pool with the previously opened TCP port. You ll need to find out the names of the VMs beforehand. One way to do that is to look at the existing HTTP server pool. Note that after a scale-in or scale-out, you must manually adjust the server pool by deleting removed servers and adding new servers to the pool. 2-2 Using Oracle Managed File Transfer Cloud Service

15 Configuring DBFS for Shared Storage 3. Deploy the changes. Follow the procedure described in Deploying a Configuration in the Oracle Traffic Director Administrator's Guide. Don t forget to deploy the changes after the proxy, the listener and the server pool are created or altered. Configuring DBFS for Shared Storage The Database File System (DBFS) uses database features to store files and manage relational data to implement a standard file system interface for files stored in the database. With this interface, storing files in the database is not limited to programs written to use BLOB and CLOB programmatic interfaces. Files in the database are transparently accessed using any operating system program that acts on files. In a clustered environment with more than one server the embedded sftp server stores its files in local storage. DBFS is used to provide shared mount points in all servers for access to the files. For example, if you run a cluster, you must provide a shared file system. This script allows you to use the database service for shared storage of all files. Execute the following steps to configure DBFS: 1. Stop the administration server and all managed servers. See Shutting Down and Starting the WebLogic Server Managed Servers and Administration Server Processes on VMs in Using Oracle Java Cloud Service for information. 2. Run the dbfswa.sh command on all servers in the cluster. The following commands have to be executed once on each managed server after initial provisioning, or on newly added servers after scale-out. The commands must be executed as the opc user. In order to connect to the second (or later) server as the opc user, you must copy (for example, with scp) the ssh_private_key to the administration server first, and then connect to the second server from the administration server. See Creating an SSH Key Pair for information about creating an SSH key pair. ssh -i ssh_private_key opc@ip.xxx.yyy.zzz /tmp/dbfswa.sh If the dbfswa.sh script completes successfully it displays the following message: DBFS configuration was successful If the configuration fails, the script reports the failure and exits with an error code of 1. Note: When you scale out a new managed server (see Scaling An Oracle SOA Cloud Service Instance), the dbfswa.sh command does not exist on the new server. You must manually copy the /u01/data/domains/[domain]/ dbfs directory from the administration server to the newly scaled out server, and run the /tmp/dbfswa.sh command on the newly scaled out server. 3. Start the administration server and all managed servers. Stop all managed servers. See Shutting Down and Starting the WebLogic Server Managed Servers and Administration Server Processes on VMs in Using Oracle Java Cloud Service for information. Post-Provisioning Tasks 2-3

16 Configuring DBFS for Shared Storage 2-4 Using Oracle Managed File Transfer Cloud Service