Administration Guide for SAP HANA Smart Data Integration and SAP HANA Smart Data Quality

PUBLIC SAP HANA Platform SPS 12 Document Version: 1.0 2016-05-11 Administration Guide for SAP HANA Smart Data Integration and SAP HANA Smart Data

Content 1 Getting Started....8 1.1 Open a Support Connection....8 2 Overview of the Architecture....10 2.1 Architecture....10 2.2 Administration Tools.... 12 3 Configure Smart Data Integration....13 3.1 Enable the Data Provisioning Server.... 13 3.2 Download and Deploy the Data Provisioning Delivery Unit....14 Download the Data Provisioning Delivery Unit....15 Deploy the Delivery Unit from SAP HANA Studio....15 Deploy the Delivery Unit from SAP HANA Application Lifecycle Management....16 3.3 Install and Configure the Data Provisioning Agent...17 Planning and Preparation....17 Install the Data Provisioning Agent....23 Configuring the Data Provisioning Agent....26 Register Adapters with SAP HANA....36 3.4 Set up Data Provisioning monitoring....39 Grant Roles to Users... 39 Assign Catalog to Data Provisioning Monitoring Role....40 Add Data Provisioning Tiles to SAP HANA Cockpit....41 3.5 Enable Enterprise Semantic Services.... 42 Setting Up the SAP HANA Instance for Enterprise Semantic Services....43 Download Enterprise Semantic Services Delivery Unit....46 Import the Enterprise Semantic Services Delivery Unit....47 Install or Upgrade Enterprise Semantic Services (install.html)...48 Grant Enterprise Semantic Services Roles and Privileges to Users...49 Uninstall Enterprise Semantic Services.... 50 4 Configure Smart Data....51 4.1 Enable the Script Server....51 4.2 Download and Deploy Directory Files....52 4.3 Update Directories....53 5 Update the Data Provisioning Agent....54 5.1 Update the SAP ASE Adapter Preferences....56 6 Monitor and Operate....57 2 2016 SAP SE or an SAP affiliate company. All rights reserved. Content

6.1 Monitoring Data Provisioning....57 SAP HANA Data Provisioning Monitoring.... 58 SAP HANA Tile Catalog for Data Provisioning....58 Access the Data Provisioning Monitors....58 Configure User Settings Profiles.... 59 6.2 Monitoring Data Provisioning Agents....60 Information Available on the Data Provisioning Agent Monitor.... 60 Manage Agents from the Data Provisioning Agent Monitor....61 Manage Adapters from the Data Provisioning Agent Monitor.... 63 6.3 Monitoring Remote Subscriptions....64 Remote Subscription States for Data Provisioning....64 Information Available on the Data Provisioning Remote Subscription Monitor.... 65 Suspend and Resume Remote Sources.... 66 Manage Remote Subscriptions.... 68 Processing Remote Source or Remote Subscription Exceptions....69 6.4 Monitoring Design Time Objects....69 Information Available on the Data Provisioning Design Time Object Monitor.... 69 Execute Flowgraphs and Replication Tasks....71 Schedule Flowgraphs and Replication Tasks....72 Stop Non-Realtime Flowgraph Executions....74 6.5 Monitoring Data Provisioning Tasks....74 Information Available on the Data Provisioning Task Monitor.... 75 Start and Stop Data Provisioning Tasks.... 78 Schedule Data Provisioning Tasks....79 Change Retention Period for Data Provisioning Task Monitor....80 Troubleshooting Data Provisioning Tasks....81 6.6 Create Status Notifications.... 89 6.7 Create Monitoring Alerts....91 6.8 Monitoring and Troubleshooting Enterprise Semantic Services...91 Privileges for Administering Enterprise Semantic Services....92 Enterprise Semantic Services Publications and Entity Grid....93 Publishing and Profiling Artifacts....93 Monitor the Status of Publishing Requests....96 Monitor the Enterprise Semantic Services Entity Grid.... 100 Data Profiling....103 Set Configuration Parameters....104 Troubleshoot Installation Issues....105 Troubleshoot Publishing Issues....106 Troubleshoot Data Profiling Issues....107 Troubleshoot Search Issues....107 API Error Messages....109 Content 2016 SAP SE or an SAP affiliate company. All rights reserved. 3

Troubleshooting Tips....112 7 Administer....115 7.1 Assign Roles and Privileges....115 7.2 Create Remote Sources.... 119 Add a Remote Source in the Web-based Development Workbench....119 Adapter Remote Source User Credentials....119 7.3 Activate and Execute a Replication Task....120 7.4 Activate and Execute a Flowgraph....122 7.5 Administering Agent Groups....123 Create or Remove an Agent Group....124 Configure Agents in an Agent Group....125 Configure Remote Sources in an Agent Group....125 7.6 Uninstall the Data Provisioning Agent....126 Uninstall from the Command Line....126 7.7 Troubleshooting the Data Provisioning Agent....127 Data Provisioning Agent Log Files and Scripts....127 Agent Was Started by a Root User.... 128 Agent JVM Out of Memory....129 Adapter Prefetch Times Out....129 Agent Reports Errors when Stopping or Starting....130 7.8 Configuring the Operation Cache....130 7.9 Ensure Workload Management and Resource Consumption....132 8....133 8.1 Setting up....135 8.2 Custom Adapters....135 8.3 Apache Camel Facebook.... 136 Set up the Camel Facebook Adapter....136 Camel Facebook Adapter Remote Source Configuration....137 Facebook Tables....137 8.4 Apache Camel Informix.... 140 Set Up the Camel Informix Adapter....141 Camel Informix Remote Source Configuration....142 Camel Informix Remote Source SQL Configuration.... 142 Camel Informix to SAP HANA Data Type Conversion.... 143 SQL Conversion....144 8.5 IBM DB2 for z/os.... 145 Setting DB2 UDB Environment Variables....146 Bind the DB2 SYSHL Package.... 146 Preparing JDBC Files....147 IBM DB2 z/os Remote Source Configuration.... 147 4 2016 SAP SE or an SAP affiliate company. All rights reserved. Content

IBM DB2 z/os to SAP HANA Data Type Mapping....148 8.6 File....149 Configure the File Adapter.... 151 File Format Configuration Files....152 Generate the Configuration File.... 153 File Adapter Remote Source Configuration....154 Remote Source Tables.... 156 HDFS Target Files....156 8.7 Google+.... 157 Google+ Remote Source Configuration....157 Google+ Remote Source SQL Configuration.... 158 Google+ Tables....158 8.8 Hive....164 Hive Remote Source Configuration....165 Hive to SAP HANA Data Type Mapping....167 8.9 IBM DB2 Log Reader.... 167 Remote Database Setup for DB2 Realtime.... 169 DB2LogReaderAdapter Preferences....177 DB2 Log Reader Remote Source Configuration.... 178 DB2 to SAP HANA Data Type Mapping....180 DB2 Database Cleanup.... 181 Log Reader Adapter Log Files....181 8.10 Microsoft SQL Server Log Reader....182 Remote Database Setup for SQL Server Realtime.... 183 MssqlLogReaderAdapter Preferences....190 MS SQL Server Log Reader Remote Source Configuration.... 193 MS SQL Server to SAP HANA Data Type Mapping....198 SQL Server Database Cleanup....199 Log Reader Adapter Log Files....199 8.11 Oracle Log Reader....199 Remote Database Setup for Oracle Realtime....201 Oracle Log Reader Remote Source Configuration.... 205 OracleLogReaderAdapter Preferences.... 211 Oracle to SAP HANA Data Type Mapping....214 Database Cleanup....215 Log Reader Adapter Log Files....216 8.12 Microsoft Excel.... 216 MS Excel Adapter Preferences....217 MS Excel Remote Source Configuration.... 217 MS Excel to SAP HANA Data Type Mapping....219 8.13 OData....220 Content 2016 SAP SE or an SAP affiliate company. All rights reserved. 5

OData Adapter....220 Installation and Deployment....221 OData to SAP HANA Data Type Mapping....223 Deploy HTTPS OData Services on S/4HANA or HANA Cloud Portal....224 8.14 Microsoft Outlook....224 Outlook Adapter Preferences....224 Outlook Remote Source Configuration....225 Outlook Remote Source SQL Configuration....225 PST File Tables....225 8.15 SAP ASE....227 Set Up Your SAP ASE Database....228 SAP ASE Remote Source Configuration Parameters.... 229 SAP ASE Adapter Preferences....230 SAP ASE to SAP HANA Data Type Mapping.... 230 8.16 SAP ECC....232 Terminology....234 Installation and Setup....234 Permissions for ECC Dictionary Tables....235 Create an ECC Remote Source....235 Loading Metadata for Cluster and Pooled Tables....236 Data Type Mapping....236 Adapter Preferences....238 8.17 SAP ABAP....245 ABAP Adapter Remote Source Configuration....246 Data Type Mapping....247 Change Data Capture....249 Initial Load....249 Delta Modes Support.... 250 8.18 SAP HANA....250 SAP HANA Remote Source Configuration....252 SAP HANA DDL Propagation Scan Interval....254 SAP HANA Adapter Preferences.... 255 SAP HANA (Remote) to SAP HANA (Target) Data Type Mapping.... 255 8.19 SOAP....256 SOAP Adapter Remote Source Configuration.... 257 SOAP Operations as Virtual Function....257 Process the Response....258 Setting up the SOAP Adapter: SQL Example....259 8.20 Teradata....260 Teradata Remote Source Configuration....262 Teradata Create Remote Source SQL Syntax.... 265 6 2016 SAP SE or an SAP affiliate company. All rights reserved. Content

Teradata DDL Propagation Scan Interval.... 266 Teradata Adapter Preferences....266 Teradata to SAP HANA Data Type Mapping....267 Data Types and Writing to Teradata Virtual Tables....269 8.21 Twitter....270 Twitter Terminology....272 Installation and Deployment.... 273 Batch Twitter Queries....276 Real Time Twitter Queries...282 Restrictions and Limitations.... 285 9 Security.... 286 9.1 Authentication.... 286 9.2 Authorizations....287 Activating and Executing Task Flowgraphs and Replication Tasks....288 9.3 Communication Channel Security....289 9.4 Auditing Activity on SAP HANA Smart Data Integration Objects....290 Content 2016 SAP SE or an SAP affiliate company. All rights reserved. 7

1 Getting Started This guide describes the main tasks and concepts necessary for the ongoing operation and administration of SAP HANA smart data integration and SAP HANA smart data quality. The following areas are covered: Architecture Administrator tasks to enable functionality Data Provisioning Agent installation and configuration Security Data Provisioning adapters Monitoring For information about administration of the overall SAP HANA system, refer to the SAP HANA Administration Guide. Related Information SAP HANA Administration Guide (PDF) SAP HANA Administration Guide (HTML) 1.1 Open a Support Connection In some support situations, it may be necessary to allow an SAP support engineer to log into your system to analyze the situation. Procedure 1. To enable a support user to log on to your system, complete the following tasks: a. Install the SAProuter as described on SAP Support Portal. b. Set up a support connection as described in SAP Note 1634848 (SAP HANA database service connections). c. Configure a Telnet connection as described in SAP Note 37001 (Telnet link to customer systems) d. Configure an SAP HANA database connection as described in SAP Note 1592925 (SAP HANA studio service connection). e. Configure a TREX/BIA/HANA service connection as described in SAP Note 1058533 (TREX/BIA/HANA service connection to customer systems). 8 2016 SAP SE or an SAP affiliate company. All rights reserved. Getting Started

2. Create a database user and grant the MONITORING role. The MONITORING role allows a database user to open the SAP HANA Administration Console perspective with read-only access to the system, system views, statistics views, trace files, and so on. However, this role does not provide any privileges for accessing application data. With the MONITORING role, it is also not possible to change the configuration of or start and stop a system. You can grant the MONITORING role to a support engineer if SAP support needs to connect to the system. Depending on the issue to be analyzed, further privileges may be needed to allow sufficient analysis (for example, to access application data or data models). Related Information SAP Note 1634848 SAP Note 37001 SAP Note 1592925 SAP Note 1058533 SAProuter Getting Started 2016 SAP SE or an SAP affiliate company. All rights reserved. 9

2 Overview of the Architecture 2.1 Architecture These diagrams represent common deployment architectures for using smart data integration and smart data quality with SAP HANA. In all deployments, the basic components are the same. However, the connections between the components may differ depending on whether SAP HANA is deployed on premise, in the cloud, or behind a firewall. Figure 1: SAP HANA deployed on premise 10 2016 SAP SE or an SAP affiliate company. All rights reserved. Overview of the Architecture

Figure 2: SAP HANA deployed in the cloud or behind a firewall The following tables explain the diagram and the network connections in more detail. Outbound Connections Client Protocol and additional information Default Port Data Provisioning Agent When SAP HANA is deployed on premise, the Data Provisioning Server within SAP HANA connects to the agent using the TCP/IP protocol. To manage the listening port used by the agent, edit the adapter framework preferences with the Data Provisioning Agent Configuration tool. 5050 Sources Examples: The connections to external data sources depend on the type of adapter used to access the source. C++ adapters running in the Data Provisioning Server connect to the source using a source-defined protocol. Java adapters deployed on the Data Provisioning Agent connect to the source using a source-defined protocol. Varies by source Overview of the Architecture 2016 SAP SE or an SAP affiliate company. All rights reserved. 11

Inbound Connections Client Protocol and additional information Default Port Data Provisioning Agent When SAP HANA is deployed in the cloud or behind a firewall, the Data Provisioning Agent connects to the SAP HANA XS engine using the HTTP/S protocol. For information about configuring the port used by the SAP HANA XS engine, see the SAP HANA Administration Guide. 80xx 43xx Related Information Configuring the Data Provisioning Agent [page 26] SAP HANA Administration Guide (HTML) SAP HANA Administration Guide (PDF) 2.2 Administration Tools Several tools are available for the administration of SAP HANA smart data integration and SAP HANA smart data quality. Tool SAP HANA studio Data Provisioning Agent Configuration tool SAP HANA cockpit Description The SAP HANA Administration Console perspective of the SAP HANA studio is the main tool for general system administration and monitoring tasks. This tool manages Data Provisioning Agents and adapters, and connections to SAP HANA The SAP HANA cockpit is an SAP Fiori Launchpad site that provides you with a single point-of-access to a range of Web-based applications for the administration of SAP HANA. You access the SAP HANA cockpit through a Web browser. Through the SAP HANA cockpit, you can monitor Data Provisioning Agents, tasks, and remote subscriptions. SAP HANA Enterprise Semantic Services Administration tool The SAP HANA Enterprise Semantic Services Administration user interface is a browser-based application that lets you manage artifacts for semantic services. To launch the SAP HANA Enterprise Semantic Services Administration tool, enter the following URL in a web browser:http:// <your_hana_instance:port>/sap/hana/im/ess/ui 12 2016 SAP SE or an SAP affiliate company. All rights reserved. Overview of the Architecture

3 Configure Smart Data Integration Below is a list of high level tasks needed to set up SAP HANA smart data integration. 1. Enable the Data Provisioning Server [page 13] Enabling the Data Provisioning Server gives you the ability to use SAP HANA smart data integration. 2. Download and Deploy the Data Provisioning Delivery Unit [page 14] Download and import the Data Provisioning delivery unit using SAP HANA studio or SAP HANA Application Lifecycle Management. 3. Install and Configure the Data Provisioning Agent [page 17] The Data Provisioning Agent provides secure connectivity between the SAP HANA database and your onpremise, adapter-based sources. 4. Set up Data Provisioning monitoring [page 39] After you install the SAP HANA smart data integration option for SAP HANA, you must take several actions to enable and access the monitoring user interfaces for Data Provisioning agents, remote subscriptions, and tasks. 5. Enable Enterprise Semantic Services [page 42] Enterprise Semantic Services provides an API to enable searching for publication artifacts or run-time objects based on their metadata and contents. 3.1 Enable the Data Provisioning Server When SAP HANA is first installed, the Data Provisioning Server is disabled. Enabling the Data Provisioning Server gives you the ability to use SAP HANA smart data integration. Procedure 1. In the Administration editor, choose the Configuration tab. 2. Expand the daemon.ini configuration file. 3. Select and expand dpserver, select Instances, right click, and choose Change. 4. Set Instances to 1. Results The Data Provisioning Server is enabled. Configure Smart Data Integration 2016 SAP SE or an SAP affiliate company. All rights reserved. 13

Next Steps To enable the Data Provisioning Server on tenants in a multi-database container environment, use the ALTER DATABASE SQL command. For example, ALTER DATABASE <(><<)>TENANT> ADD 'dpserver'. Task overview: Configure Smart Data Integration [page 13] Next: Download and Deploy the Data Provisioning Delivery Unit [page 14] Related Information SAP HANA Administration Guide: Change a System Property (HTML) SAP HANA Administration Guide: Change a System Property (PDF) 3.2 Download and Deploy the Data Provisioning Delivery Unit Download and import the Data Provisioning delivery unit using SAP HANA studio or SAP HANA Application Lifecycle Management. You will need to download the Data Provisioning delivery unit. Then, using SAP HANA studio or SAP HANA Application Lifecycle Management tools, deploy the delivery unit to obtain the following functionality: Functionality Monitoring Proxy Admin Description The monitoring application provides a browser based interface to monitor agents, tasks and remote subscriptions created in the SAP HANA system. The monitor application needs to be configured in the SAP HANA Cockpit. The Proxy application provides a way for the Data Provisioning Agent to communicate with the Data Provisioning Server. It is required when SAP HANA is running in the cloud or when the remote sources are behind a firewall. In this case the Data Provisioning Agent stays behind the firewall (that is, close to the remote source) and communicates with SAP HANA (specifically the dpserver) via the Proxy application running in the XS engine. The Admin application provides a way for the Data Provisioning Agent Configuration tool to issue SQL commands necessary to register the agent and the adapters in the SAP HANA system. This application is used when SAP HANA is in the cloud and the Data Provisioning Agent is behind a firewall. Parent topic: Configure Smart Data Integration [page 13] Previous task: Enable the Data Provisioning Server [page 13] Next: Install and Configure the Data Provisioning Agent [page 17] 14 2016 SAP SE or an SAP affiliate company. All rights reserved. Configure Smart Data Integration

3.2.1 Download the Data Provisioning Delivery Unit Download the data provisioning delivery unit from the SAP Software Download Center. Context The data provisioning delivery unit is available in the same download area as the data provisioning agent. Procedure 1. Go to the SAP Software Download Center, and navigate to the following location: SAP Software Download Center Software Downloads Installations & Upgrades By Alphabetical Index (A-Z) H SAP HANA SDI SAP HANA SDI 1.0 2. In the Downloads tab, choose Installation. 3. Select the ZIP file to download, and choose Download from the action menu. 4. In the downloaded ZIP file, navigate to DATA_UNIT/HANA_DP_10, and extract the HANAIMDP<##>_0.ZIP file. 5. In the HANAIMDP<##>_0.ZIP file, find and extract the HANAIMDP.tgz file. This is the delivery unit file that needs to be imported into SAP HANA. Related Information SAP Software Download Center 3.2.2 Deploy the Delivery Unit from SAP HANA Studio You can import the Data Provisioning delivery unit from SAP HANA studio. Note When SAP HANA is deployed in a multitenant database container configuration, you must import the delivery unit into the tenant database. Prerequisites Ensure that you have been granted the SYSTEM privilege REPO.IMPORT to be able to import the DU. Configure Smart Data Integration 2016 SAP SE or an SAP affiliate company. All rights reserved. 15

Procedure 1. Log in to SAP HANA studio as user SYSTEM. 2. In the upper left corner, click File Import. 3. On the Import dialog, type delivery into the search box for Select an import source. 4. Click Delivery Unit on the resulting navigation tree and click Next. 5. Select <your SAP HANA Server name>, and click Next. 6. On the Import Through Delivery Unit dialog, select either the Client or Server radio button, depending on whether the delivery unit is on the client or server machine. a. If you select Client, click Browse and navigate to the location where you downloaded the delivery unit, select HANAIMDP.tgz, and click Open. b. If you select Server, then select the DU you want to import from the dropdown list. 7. Click Finish. 3.2.3 Deploy the Delivery Unit from SAP HANA Application Lifecycle Management You can import the Data Provisioning delivery unit through SAP HANA Application Lifecycle Management. Note When SAP HANA is deployed in a multitenant database container configuration, you must import the delivery unit into the tenant database. Procedure 1. If not already granted, grant the role sap.hana.xs.lm.roles::administrator to the user name you will use to login to ALM. a. In SAP HANA studio Systems view, expand the name of your SAP HANA server and choose Security Users System. b. On the Granted Roles tab, click the green + icon in the upper left corner. c. On the Select Roles dialog, type lm in the search string box. d. Select role sap.hana.xs.lm.roles::administrator and click OK. 2. Access ALM by typing the following URL in a web browser: <host name>:80<2-digit instance number>/sap/hana/xs/lm 3. Log in to ALM as the user name you authorized in step 1. The first time you log in, a pop-up window appears to enter a name for this server. 4. On the ALM Home tab, click the Delivery Units tile. 5. Click the Import tab. 16 2016 SAP SE or an SAP affiliate company. All rights reserved. Configure Smart Data Integration

6. Click Browse and navigate to the location where you downloaded the delivery unit, select HANAIMDP.tgz and click Open. 7. Click Import. After successful import, the name HANA_IM_DP (sap.com) appears in the list of delivery units on the left. 3.3 Install and Configure the Data Provisioning Agent The Data Provisioning Agent provides secure connectivity between the SAP HANA database and your on-premise, adapter-based sources. 1. Planning and Preparation [page 17] Before you install the Data Provisioning Agent, plan your installation to ensure that it meets your system landscape's needs. 2. Install the Data Provisioning Agent [page 23] You can install the Data Provisioning Agent as a standalone installation on a Windows or Linux machine. 3. Configuring the Data Provisioning Agent [page 26] Use the SAP Data Provisioning Agent Configuration Tool to connect to the SAP HANA server and configure the agent and adapters. 4. Register Adapters with SAP HANA [page 36] Before you can connect to remote sources using adapters, you must register them with SAP HANA. Parent topic: Configure Smart Data Integration [page 13] Previous: Download and Deploy the Data Provisioning Delivery Unit [page 14] Next: Set up Data Provisioning monitoring [page 39] 3.3.1 Planning and Preparation Before you install the Data Provisioning Agent, plan your installation to ensure that it meets your system landscape's needs. When planning your installation, consider the following questions. Where should the Data Provisioning Agent be installed? You can install the agent on any host system that has access to the sources you want to access, meets the minimum system requirements, and has any middleware required for source access installed. In most cases, the agent is installed on the source system. Restriction You cannot install the agent on the SAP HANA system. Configure Smart Data Integration 2016 SAP SE or an SAP affiliate company. All rights reserved. 17

Note The Data Provisioning Agent must be installed on the same operating system as your source database, but not necessarily on the same machine. For information about operating system and DBMS compatibility, refer to the Product Availability Matrix. How many Data Provisioning Agents should be installed? You can install one or many agents depending on your landscape requirements. Note On Linux, you can install multiple agents on a single host machine. On Windows, only a single Data Provisioning Agent installation is supported per host. Which network protocols are required in your system landscape? Depending on whether SAP HANA is installed on premise, in the cloud, or behind a firewall, the connection between the agent and SAP HANA can use TCP/IP or HTTP. For security purposes, be sure to correctly enable SSL for Framework listener port using the Data Provisioning Configuration Tool for On-Premise application. For SAP HANA on Cloud, use HTTPS to communicate with SAP HANA and configure the agent to communicate using the HTTPS protocol using the Data Provisioning Configuration Tool. Can the host system support the load from the Data Provisioning Agent? Generally speaking, the agent generates minimal additional load on the host system. The agent translates the source's format and commands to and from the SAP HANA format and commands. Additionally, the system utilization will vary depending on the type and number of adapters deployed. Parent topic: Install and Configure the Data Provisioning Agent [page 17] Next task: Install the Data Provisioning Agent [page 23] Related Information Supported Platforms and System Requirements [page 18] Product Availability Matrix 3.3.1.1 Supported Platforms and System Requirements Install the Data Provisioning Agent on a supported platform that meets the minimum system requirements. Note You can find a complete list of all SAP HANA components and the respective SAP HANA hardware and software requirements in the Product Availability Matrix (PAM) on the Support Portal and in the SAP Community Network. 18 2016 SAP SE or an SAP affiliate company. All rights reserved. Configure Smart Data Integration

Operating System for the Data Provisioning Agent For the Data Provisioning Agent host system, the following 64-bit platforms are supported: Microsoft Windows Server SUSE Linux Enterprise Server (SLES) Red Hat Enterprise Linux (RHEL) Software Requirements The host system must have the 64-bit Java JDK installed. For more information about supported Java versions, see the Product Availability Matrix (PAM). On Linux platforms, the following additional requirements apply: In order to install or uninstall the agent, you must use a user that has root privileges. The system must have gcc 4.7 to run the Data Provisioning Agent service. For more information see SAP Note 2001528. Related Information Product Availability Matrix SAP Note 2001528 - Linux: SAP HANA Database SPS 08 revision 80 (or higher) on RHEL 6 or SLES 11 3.3.1.2 Software Download The Data Provisioning Agent installation package is available as optional component SAP HANA SDI 1.0 on the SAP Software Download Center. Note Installation of the Data Provisioning Agent requires the correct version of SAP HANA. Subsequent support packages or revisions of SAP HANA may require an equivalent update to the Data Provisioning Agent. For details, see the Product Availability Matrix. On the SAP Software Download Center, you can find the installation packages in the following locations: Installation media for an SAP HANA SPS: SAP Software Download Center Software Downloads Installations & Upgrades By Alphabetical Index (A-Z) H SAP HANA SDI SAP HANA SDI 1.0 In the Downloads tab, click Installation. Select the package to download, and choose Download from the action menu. Configure Smart Data Integration 2016 SAP SE or an SAP affiliate company. All rights reserved. 19

Support Packages and Patches for SAP HANA: SAP Software Download Center Software Downloads Support Packages & Patches By Alphabetical Index (A-Z) H SAP HANA SDI SAP HANA SDI 1.0 Comprised Software Component Versions HANA DP AGENT 1.0 In the Downloads tab, choose your platform from the dropdown list. Select the package to download, and choose Download from the action menu. To verify the agent version, see the Product Availability Matrix. Related Information SAP Software Download Center Product Availability Matrix 3.3.1.3 Prepare the Amazon Web Services (AWS) Environment Before you can install the Data Provisioning Agent on AWS, you must prepare the environment. Procedure 1. Launch the Amazon Web Services (AWS) instance. 2. Copy the Java Development Kit and the Data Provisioning Agent packages to /download on the AWS host: For example: jdk-8u60-linux-x64.rpm.rpm IMDB_DPAGENT100_00_2-70000174.SAR Note You may need to create the /download directory if it does not exist. 3. Log in to the AWS host as ec2-user and start a sudo bash command line. sudo bash 4. Install the Java Development Kit. zypper install jdk-8u60-linux-x64.rpm.rpm 5. Change to the ec2-user user and extract the Data Provisioning Agent installation program. su ec2-user./sapcar xvf IMDB_DPAGENT100_00_2-70000174.SAR 20 2016 SAP SE or an SAP affiliate company. All rights reserved. Configure Smart Data Integration

Results The Java Development Kit is installed and the Data Provisioning Agent installation program is available on the AWS host. You can continue to install the Data Provisioning Agent from the command line. Related Information Install from the Command Line [page 25] 3.3.1.4 Prepare SSL when SAP HANA is on premise When SAP HANA is installed on premise, you must obtain a certificate for the agent and import certificates on both the agent host machine and the SAP HANA system. Prerequisites Before configuring the agent, ensure that the SAP HANA system is already configured for SSL. For more information, see the SAP HANA Security Guide. Note You need the password for the keytool Java program to generate a keystore and import a HANA certificate. You can find the password, commands and the instructions in the keytool.txt file at <DPAgent_root>\ssl \keytool.txt. Change the default password of the keystore to safeguard your certificates. Procedure 1. Generate a keystore on the agent host machine. keytool -genkeypair -alias DPAgent -keyalg RSA -keysize 2048 -dname "CN=<agent_hostname>,OU=<organizational_unit>,O=<organization>,L=<city>,S=<state>,C=<country>" -keypass <key_password> -storepass <store_password> -keystore cacerts <agent_hostname> must be the fully qualified hostname of the machine where the agent is installed. 2. Obtain a signed certificate for the agent and import it into the keystore. a. Generate the certificate request. keytool -certreq -file <request_filename>.cer alias DPAgent keystore cacerts Configure Smart Data Integration 2016 SAP SE or an SAP affiliate company. All rights reserved. 21

b. Send the certificate request to a Certificate Authority (CA) to be signed. c. After receiving the signed certificate from the CA, import it into the agent keystore. keytool -importcert -keystore cacerts -storepass <store_password> -file <certificate_filename>.der -noprompt -alias DPAgent 3. Import the SAP HANA server root certificate into the agent keystore. keytool -importcert -keystore cacerts -storepass <store_password> -file <certificate_filename>.der -noprompt You can obtain the certificate by exporting it with the SAP Web Dispatcher. For more information, see SAP Note 2009483. 4. On SAP HANA, add the signed agent certificate to the sapcli Personal Security Environment (PSE). You can add the certificate with the SAP Web Dispatcher. For more information, see SAP Note 2009483. 5. Configure the data provisioning agent for SSL. Related Information SAP HANA Security Guide (PDF) SAP HANA Security Guide (HTML) Start and Connect the Configuration Tool [page 27] Register the Data Provisioning Agent with SAP HANA [page 35] SAP Note 2009483 - PSE Management in Web Administration Interface of SAP Web Dispatcher Connect to SAP HANA on-premise with SSL [page 28] 3.3.1.5 Download the SAP HANA Server Certificate When SAP HANA is configured for HTTPS, you need a copy of the server certificate to configure the SAP HANA Data Provisioning Agent. Tip To verify whether the SAP HANA server is configured for HTTPS, examine the port number being used. If the port number is 80<xx>, the server is using standard HTTP. If the port number is 43<xx>, the server is using HTTPS. When SAP HANA is located in the cloud, it is always configured for HTTPS communication. Context The SAP HANA server certificate can be downloaded from a web browser. 22 2016 SAP SE or an SAP affiliate company. All rights reserved. Configure Smart Data Integration

Tip If the agent keystore does not have the server certificate required for HTTPS communication, the Data Provisioning Agent Configuration tool allows you to directly import the server certificate into the agent keystore. This procedure is required only if you do not want the configuration tool to directly import the certificate, and want to manually import it separately. Procedure 1. Navigate to the SAP HANA server in a web browser. For example, https://<hostname>.us1.hana.ondemand.com. 2. Open the certificate information for the page. The exact steps to open the certificate information depend on your browser. For Internet Explorer, click on the lock icon in the address bar, and click View Certificates. For Chrome, click on the lock icon in the address bar, and click Connection Certificate Information. 3. In the Details tab, click Copy to file. The Certificate Export Wizard is displayed. Click Next. 4. Select DER encoded binary X.509 (.CER) and click Next. 5. Specify a name for the certificate file and save it locally. For example, HANA_Certificate_06Oct2015.cer. 6. Transfer the certificate file to a location accessible by the Data Provisioning Agent. 3.3.2 Install the Data Provisioning Agent You can install the Data Provisioning Agent as a standalone installation on a Windows or Linux machine. The default installation manager is a graphical installation tool. Prerequisites When you install the Data Provisioning Agent on Linux, there are additional prerequisites: GTK 2 is installed so that you can run the graphical installation tool. You are logged on as the root user. This user is authorized to open a display. The Java runtime is accessible either via the PATH or JAVA_HOME variables or by specifying the --vm option. Configure Smart Data Integration 2016 SAP SE or an SAP affiliate company. All rights reserved. 23

Tip To set the -vm option, open the dpagent.ini configuration file located by default at /usr/sap/ dataprovagent, and add -vm <path_to_java_installation>/<java_executable>. For example, -vm /usr/java/jdk1.8.0_60/bin/javaw A local X11 graphical environment and an X server must be available on the machine where you perform the installation. Context Caution When you install the Data Provisioning Agent, the agent will use, by default, a non-secure channel when communicating with the SAP HANA server. To enable secure communication, you must configure SSL with the Data Provisioning Agent Configuration tool after installation. For more information, see Connect to SAP HANA on-premise with SSL and Connect to SAP HANA in the cloud. Procedure 1. Download and extract the software to an empty directory. 2. Navigate to the directory where you unpacked the software. 3. Call the installation manager. On Windows, run hdbsetup.exe. On Linux, run./hdbsetup with root privileges. For example, sudo./hdbsetup. 4. Choose Install new SAP HANA Data Provisioning Agent and specify the installation path. 5. Specify the user credentials to use for the agent service. On Windows, the username (<domain>\<username>) and password of the user that should run the Agent service. Also, a user that will run the agent service must have write access to the installation directory so that configuration files can be updated. On Linux, the user name (user ID) of the installation owner. This user will be granted all permissions to the installed directory. Caution Do not specify root as the installation owner. The installation owner must be a user other than the root user. 6. If you want to use agent clustering, specify the shared directory for the agent group. 24 2016 SAP SE or an SAP affiliate company. All rights reserved. Configure Smart Data Integration

Results The Data Provisioning Agent is installed or updated. Task overview: Install and Configure the Data Provisioning Agent [page 17] Previous: Planning and Preparation [page 17] Next: Configuring the Data Provisioning Agent [page 26] Related Information Connect to SAP HANA on-premise with SSL [page 28] Connect to SAP HANA on Cloud [page 29] 3.3.2.1 Install from the Command Line If you cannot use or do not want to use the graphical installation manager, you can install the Data Provisioning Agent using the command line tool. Procedure 1. Download and extract the software to an empty directory. 2. Navigate to the directory where you unpacked the software. 3. On Windows, create the password XML file one directory level above hdbinst.exe: <?xml version="1.0" encoding="utf-8"?> <Passwords> <service_password><password></service_password> </Passwords> 4. Call the installation program. On Windows, run more..\passwords.xml hdbinst.exe --silent --batch -- path="<dpagent_root>" --service_user=<domain>\<username> -- read_password_from_stdin=xml On Linux, run./hdbinst --silent --batch --path="<dpagent_root>" --user_id=<user ID> Configure Smart Data Integration 2016 SAP SE or an SAP affiliate company. All rights reserved. 25

Results The Data Provisioning Agent is installed without displaying the graphical installation manager. 3.3.2.2 Installation Logs The Data Provisioning Agent installation is logged by the system. There are two files written during installation. *.log: can be read using a text editor *.msg: XML format for display in the installation tool with the graphical user interface The log files are stored in the following locations: On Windows, %TEMP%\hdb_dataprovagent_<timestamp> On Linux, /var/tmp/hdb_dataprovagent_<timestamp> 3.3.2.3 Default Installation Paths The default installation paths are specific to the operating system on which the Data Provisioning Agent is installed. Platform Package Version Default Installation Path Microsoft Windows x86, 64-bit 64-bit C:\usr\sap\dataprovagent Linux x86, 64-bit 64-bit /usr/sap/dataprovagent In this documentation, these root installation paths are represented by the variable <DPAgent_root>. 3.3.3 Configuring the Data Provisioning Agent Use the SAP Data Provisioning Agent Configuration Tool to connect to the SAP HANA server and configure the agent and adapters. The configuration tool allows you to perform standard administrative tasks: Connect to the SAP HANA server Manage the agent service Register the agent with SAP HANA Register adapters with SAP HANA Configure settings for adapters and the adapter framework Parent topic: Install and Configure the Data Provisioning Agent [page 17] Previous task: Install the Data Provisioning Agent [page 23] 26 2016 SAP SE or an SAP affiliate company. All rights reserved. Configure Smart Data Integration

Next task: Register Adapters with SAP HANA [page 36] 3.3.3.1 Start and Connect the Configuration Tool Before you can use the configuration tool to register the agent or deploy and register adapters, you must connect to the SAP HANA server. The steps required to connect the Data Provisioning Agent to the SAP HANA server vary depending on whether the SAP HANA server is installed on-premise or in the cloud, and whether it is configured for secure SSL connections. Connect to SAP HANA on-premise [page 27] Specify connection information and administrator credentials when the SAP HANA system is located onpremise and does not require a secure SSL connection. Connect to SAP HANA on-premise with SSL [page 28] Specify connection information, administrator credentials, and SSL configuration information when the SAP HANA system is located on-premise and requires a secure SSL connection. Connect to SAP HANA on Cloud [page 29] Configure SAP HANA users and specify connection information, user credentials, and SSL configuration information when the SAP HANA system is located in the cloud. 3.3.3.1.1 Connect to SAP HANA on-premise Specify connection information and administrator credentials when the SAP HANA system is located on-premise and does not require a secure SSL connection. Procedure 1. Navigate to the <DPAgent_root>/configTool directory. 2. Start the configuration tool. On Windows, run dpagentconfigtool.exe. On Linux, run./dpagentconfigtool. 3. Connect to the SAP HANA server. a. Click Connect to HANA. b. Specify the hostname, port, and HANA administrator credentials for the SAP HANA server. Note The administrator user that you use to connect to the SAP HANA system must have been granted the AGENT ADMIN and ADAPTER ADMIN system privileges. If the user that you want to use does not already have these privileges, you must grant them before you can connect to the SAP HANA system. Configure Smart Data Integration 2016 SAP SE or an SAP affiliate company. All rights reserved. 27

Tip To determine the correct port number when SAP HANA is deployed in a multi-database configuration, execute the following SQL statement: SELECT DATABASE_NAME,SERVICE_NAME,PORT,SQL_PORT,(PORT + 2) HTTP_PORT FROM SYS_DATABASES.M_SERVICES WHERE DATABASE_NAME='<DBNAME>' and ((SERVICE_NAME='indexserver' and COORDINATOR_TYPE= 'MASTER') or (SERVICE_NAME='xsengine')) Related Information Default Installation Paths [page 26] Assign Roles and Privileges [page 115] 3.3.3.1.2 Connect to SAP HANA on-premise with SSL Specify connection information, administrator credentials, and SSL configuration information when the SAP HANA system is located on-premise and requires a secure SSL connection. Prerequisites Before you can configure the Data Provisioning Agent to use SSL with SAP HANA on premise, you must obtain the SSL certificates and import them to both the agent host machine and the SAP HANA system. For more information, see Prepare SSL when SAP HANA is on premise. Procedure 1. Navigate to the <DPAgent_root>/configTool directory. 2. Start the configuration tool. On Windows, run dpagentconfigtool.exe. On Linux, run./dpagentconfigtool. 3. In the configuration tool, choose Configure SSL, enter the SSL configuration information, and select Enable SSL for Agent to HANA communication on TCP. 4. Connect to the SAP HANA server. a. Return to the configuration tool, or restart it if needed. b. Click Connect to HANA. 28 2016 SAP SE or an SAP affiliate company. All rights reserved. Configure Smart Data Integration

c. Specify the hostname, port, and HANA administrator credentials for the SAP HANA server. Note The administrator user that you use to connect to the SAP HANA system must have been granted the AGENT ADMIN and ADAPTER ADMIN system privileges. If the user that you want to use does not already have these privileges, you must grant them before you can connect to the SAP HANA system. Tip To determine the correct port number when SAP HANA is deployed in a multi-database configuration, execute the following SQL statement: SELECT DATABASE_NAME,SERVICE_NAME,PORT,SQL_PORT,(PORT + 2) HTTP_PORT FROM SYS_DATABASES.M_SERVICES WHERE DATABASE_NAME='<DBNAME>' and ((SERVICE_NAME='indexserver' and COORDINATOR_TYPE= 'MASTER') or (SERVICE_NAME='xsengine')) Related Information Default Installation Paths [page 26] Assign Roles and Privileges [page 115] Prepare SSL when SAP HANA is on premise [page 21] 3.3.3.1.3 Connect to SAP HANA on Cloud Configure SAP HANA users and specify connection information, user credentials, and SSL configuration information when the SAP HANA system is located in the cloud. Context When SAP HANA is in the cloud, all communication is initiated by the agent. The agent polls the server to see if there are any messages for the agent to act upon. Procedure 1. Import the Data Provisioning Delivery Unit. For complete information, see Download and Deploy the Data Provisioning Delivery Unit. 2. Create or grant privileges to the HANA administration and HANA XS Agent users. Configure Smart Data Integration 2016 SAP SE or an SAP affiliate company. All rights reserved. 29

a. Configure the HANA administration user. This user connects to the SAP HANA system via the configuration tool to perform administrative tasks such as registering agents, registering adapters, and so on. Create a new user or grant an existing user the following privileges: Application privilege sap.hana.im.dp.admin::administrator System privilege ADENT ADMIN System privilege ADAPTER ADMIN b. Configure the HANA XS Agent user. The HANA XS Agent user is used only for messaging between the Data Provisioning Agent and SAP HANA on Cloud. The credentials for this user are saved in the Data Provisioning Agent's secure store for use at runtime. Caution It is strongly recommended that this user has only the minimally required application privilege, and no additional administrative privileges. Create a new user or grant an existing user the following privileges: Application privilege sap.hana.im.dp.proxy::agentmessaging Tip The Data Provisioning Agent Configuration tool can create the HANA XS Agent user during the agent configuration process. If you want the configuration tool to create the user, the administration user that you use to connect to the SAP HANA system must have the USER ADMIN system privilege, as well as the EXECUTE object privilege on the procedure GRANT_APPLICATION_PRIVILEGE. For complete information about creating new users and granting permissions, see the SAP HANA Administration Guide. 3. Connect to the SAP HANA server. a. Click Connect to HANA. b. Select HANA On Cloud. c. Select Use HTTPS. When you attempt to connect to HANA on Cloud with HTTPS for the first time, the configuration tool allows you to automatically download and import the SAP HANA server certificates into the Data Provisioning Agent keystore. Note If you prefer not to import the server certificates by this method, you must manually download and import the certificates. For more information, see Manually Configure SSL for HANA on Cloud. d. Specify the hostname, HTTP(s) port, and HANA administrator user credentials for the SAP HANA server. The hostname should include the instance name. For example, <your_instance_name>.hana.ondemand.com. e. If there is a firewall between the SAP HANA server and the agent, specify any required proxy information. 30 2016 SAP SE or an SAP affiliate company. All rights reserved. Configure Smart Data Integration

f. Specify the HANA XS Agent user credentials if the user already exists. The HANA XS Agent user is used only for messaging between the Data Provisioning Agent and the SAP HANA server, and must be different from the HANA administration user that you used to connect to the SAP HANA server. Choose Create User if you want the configuration tool to create a new user. Tip To create a new user from the configuration tool, the administration user that you use to connect to the SAP HANA system must be granted the USER ADMIN system privilege, as well as the EXECUTE object privilege on the procedure GRANT_APPLICATION_PRIVILEGE. Choose Update User Credentials if you already specified a HANA XS Agent user and want to change the user's credentials. 4. Register the Data Provisioning Agent with SAP HANA. Specify the agent name and click Register. The agent service is stopped and restarted. Related Information Default Installation Paths [page 26] Assign Roles and Privileges [page 115] Download and Deploy the Data Provisioning Delivery Unit [page 14] (Optional) Manually Configure SSL for HANA on Cloud If you do not want to automatically download the SAP HANA server certificates the first time you attempt to connect to HANA on Cloud, you must manually download and import the certificates. Procedure 1. Obtain and import the SAP HANA server root certificate. a. Download the SAP HANA server certificate to a location on the Data Provisioning Agent host machine. For complete information, see Download the SAP HANA Server Certificate. b. Import the SAP HANA server root certificate into the agent keystore. keytool -importcert -keystore cacerts -storepass <store_password> -file <certificate_filename>.der -noprompt Configure Smart Data Integration 2016 SAP SE or an SAP affiliate company. All rights reserved. 31

Note You need the password for the Java keytool program to generate a keystore and import the SAP HANA server certificate. For the password, commands, and additional information, see the keytool.txt file located at <DPAgent_root>\ssl\keytool.txt. Tip Change the default password for the keystore to safeguard your certificates. 2. Configure the Data Provisioning Agent for SSL. a. Navigate to the <DPAgent_root>/configTool directory. b. Start the configuration tool. On Windows, run dpagentconfigtool.exe. On Linux, run./dpagentconfigtool. c. Click Configure SSL. d. Enter the relative path (ssl/cacerts) and password for the keystore that contains the SAP HANA server certificate. For the password, if you explicitly changed the keystore password, specify the new password here. Otherwise, leave the default password as-is.. e. Select Use SSL to communicate with HANA on Cloud and click Save. Related Information Download the SAP HANA Server Certificate [page 22] 3.3.3.2 Start and Connect from the Command Line If you cannot use or do not want to use the graphical configuration tool, you can start the Data Provisioning Agent and connect to SAP HANA using command line tools. Procedure 1. Navigate to <DPAgent_root> and open dpagentconfig.ini in a text editor. 2. Configure the agent name, cloud deployment options, and SAP HANA connection information. For example, for a deployment on Amazon Web Services (AWS): agent.name=cloud_agent cloud.deployment=aws_ cloud.usessl=true hana.port=443 32 2016 SAP SE or an SAP affiliate company. All rights reserved. Configure Smart Data Integration

hana.server=<server_name> Note The cloud.deployment should always be left blank unless SAP HANA is deployed on AWS. 3. If there is a firewall between the agent host and the SAP HANA server, specify additional proxy configuration parameters. cloud.useproxy=true proxyhost=<hostname> proxyport=<port_number> 4. Start the Data Provisioning Agent../dpagent 5. Navigate to <DPAgent_root>/agentutils and configure the SAP HANA XS username and password. a. Set the SAP HANA XS username. java -jar DPAgentInstallerUtils-1.0.8.jar -setpassword hana.xs.username <user_name> 5051 Note The version of DPAgentInstallerUtils-<version>.jar depends on the version of the Data Provisioning Agent. Ensure that you specify the filename for the version included with your agent installation. The success message SUCCESS for key hana.xs.username is displayed. b. Set the SAP HANA XS password. java -jar DPAgentInstallerUtils-1.0.8.jar -setpassword hana.xs.password <password> 5051 The success message SUCCESS for key hana.xs.password is displayed. 6. If the SAP HANA server is configured to use HTTPS, import the server certificate into the agent keystore. a. Download the SAP HANA server certificate and transfer it to the agent host machine. b. Use the keytool command to import the certificate. keytool -importcert -keystore <DPAgent_root>/ssl/cacerts -storepass <password> -file <path_to/certificate_file> -alias <alias> -noprompt c. Navigate to <DPAgent_root>/agentutils and set the framework keystore password. java -jar DPAgentInstallerUtils-1.0.8.jar -setpassword framerwork.keystore.password <password> 5051 d. Stop the Data Provisioning Agent. e. Edit <DPAgent_root>/dpagentconfig.ini in a text editor and set the hanaoncloud and hana.usessl parameters. hanaoncloud=true hana.usessl=true Configure Smart Data Integration 2016 SAP SE or an SAP affiliate company. All rights reserved. 33

f. Restart the Data Provisioning Agent. Related Information Download the SAP HANA Server Certificate [page 22] Manage the Agent Service [page 34] 3.3.3.3 Manage the Agent Service Use the configuration tool to stop or start the agent service. Procedure 1. Start the agent configuration tool. 2. Click Start Agent to start the agent service or Stop Agent to stop the agent service. Results The configuration tool indicates whether the agent service is running and the listening port in use by the agent. Next Steps On Windows, you can also manage the agent service from the standard Windows Services tool. On Linux, you can also manage the agent with a shell script. The shell script is located at <DPAgent_root>/bin/ dpagent_service.sh and supports the following commands:./dpagent_service.sh start./dpagent_service.sh stop./dpagent_service.sh restart./dpagent_service.sh ping Tip If the shell scripts are unable to correctly stop the agent service, you can use shutdown.sh to kill the agent process. 34 2016 SAP SE or an SAP affiliate company. All rights reserved. Configure Smart Data Integration

Related Information Start and Connect the Configuration Tool [page 27] 3.3.3.4 Register the Data Provisioning Agent with SAP HANA Before you can use adapters deployed on the Data Provisioning Agent, you must register the agent with SAP HANA. Prerequisites For HANA on Cloud, the HANA XS Agent user in this procedure has to be different than the one used to connect to HANA in the Data Provisioning Agent Configuration tool. The HANA XS Agent user must be granted the AGENT ADMIN system privilege as well as the sap.hana.im.dp.proxy::agentmessaging application privilege. Procedure 1. Start the agent configuration tool and connect to the SAP HANA server. 2. Click Register Agent. 3. Specify the agent connection information. If SAP HANA is not in the cloud, specify the agent name and hostname. Ensure that the SAP HANA server can communicate with the agent host. Depending on the network configuration, you may need to fully qualify the agent hostname. Ensure that your firewall settings allow the connection from the SAP HANA server to the agent host on the listener port. By default, port 5050. If SAP HANA is in the cloud, specify the agent name. When SAP HANA is in the cloud, the agent service will be restarted to complete the registration process. 4. Click Register. Results The agent is registered with SAP HANA. If SAP HANA is in the cloud, the agent service is automatically restarted. Next Steps To unregister the agent, click Unregister Agent. Configure Smart Data Integration 2016 SAP SE or an SAP affiliate company. All rights reserved. 35

Caution Unregistering the agent from the SAP HANA server performs a cascade drop of the agent. As a result, any remote subscriptions that use the agent will also be deleted, even if they are active. Related Information Start and Connect the Configuration Tool [page 27] 3.3.4 Register Adapters with SAP HANA Before you can connect to remote sources using adapters, you must register them with SAP HANA. Prerequisites Before you register your adapters, be sure that you have downloaded and installed the required JDBC drivers (if necessary). Procedure 1. Start the Data Provisioning Agent Configuration tool and connect to SAP HANA. 2. For custom adapters, click Deploy Adapter and point to the adapter JAR files. Note SAP-delivered data provisioning adapters are automatically deployed on the agent during agent installation. 3. Select the adapters to register and click Register Adapter. Results The selected adapters are registered with SAP HANA and become available for use with remote sources. 36 2016 SAP SE or an SAP affiliate company. All rights reserved. Configure Smart Data Integration

Next Steps Note For SAP HANA in the cloud, you must restart the agent service to complete the registration of adapters. If the registration succeeds and the restart of the service fails, or the registration of all adapters fails, then the registration is rolled back. Task overview: Install and Configure the Data Provisioning Agent [page 17] Previous: Configuring the Data Provisioning Agent [page 26] Related Information Start and Connect the Configuration Tool [page 27] Adapter Preferences [page 238] 3.3.4.1 Adapter Framework Preferences The adapter framework preferences provide advanced configuration options for the data provisioning agent. To modify the adapter framework preferences, choose Config Preferences in the Data Provisioning Agent Configuration tool, and then select Adapter Framework. By default, the agent is configured to start in TCP mode and monitor port 5050 for requests from SAP HANA. Key Description Default Value Framework listener port The port the agent monitors for requests from the SAP HANA server. Note The Framework listener port should be SSL enabled for security. 5050 Configure Smart Data Integration 2016 SAP SE or an SAP affiliate company. All rights reserved. 37

Key Description Default Value Admin port The local port used for internal communication between the agent and the agent configuration tool. Note The admin port should not be enabled within a firewall (that is, it should be blocked from outside access), in order to prevent unauthorized changes on the agent. 5051. Worker thread pool The number of worker threads. 10 Polling timeout The length of time to perform a blocking wait on queue. 10 Polling timeout unit The unit used by the polling timeout. SECONDS Max data size in bytes The maximum amount of data to fetch. 100000000 Row fetch size (max) The maximum number of items (browse nodes or rows) to fetch from an adapter. 1000 Row fetch size (min) The minimum number of rows to fetch from an adapter. 10 Max number of retries The maximum number of times the agent tries to connect after a registration or ping failure. 10 Time to wait before retry The amount of time to wait before retrying. 30 Shared Directory for Agent Group Logging level The shared directory for the agent group to which this agent instance belongs, if any. The type of logging to perform for the agent: None TRACE TRACE DEBUG ERROR ALL Log max backup The number of log files to keep. 10 Log file max file size Trace message max size Trace ping message Trace all data Max HTTP Connection per route The maximum file size, in MB or KB, that the log file should use. Truncates after printing the specified number of characters when tracing is enabled. Enables printing the ping message. Valid only when tracing is enabled. Enables printing the content of the data (rows) being sent to server. The number of maximum connections to be created by internal HTTP client. 10MB 1024 false false 20 Max available HTTP connection The maximum number of connections. 20 Proxy type The type of proxy being used. http 38 2016 SAP SE or an SAP affiliate company. All rights reserved. Configure Smart Data Integration

3.4 Set up Data Provisioning monitoring After you install the SAP HANA smart data integration option for SAP HANA, you must take several actions to enable and access the monitoring user interfaces for Data Provisioning agents, remote subscriptions, and tasks. These actions will allow you to access the Data Provisioning monitors by either typing the URL directly in your browser or through tiles in SAP HANA Cockpit. Parent topic: Configure Smart Data Integration [page 13] Previous: Install and Configure the Data Provisioning Agent [page 17] Next: Enable Enterprise Semantic Services [page 42] Related Information Download and Deploy the Data Provisioning Delivery Unit [page 14] Grant Roles to Users [page 39] Assign Catalog to Data Provisioning Monitoring Role [page 40] Add Data Provisioning Tiles to SAP HANA Cockpit [page 41] Monitor and Operate [page 57] 3.4.1 Grant Roles to Users You must grant the appropriate roles to users who will perform the various tasks to set up Data Provisioning monitoring in SAP HANA Cockpit. Prerequisites Ensure that you have been granted the SYSTEM privilege USER ADMIN to be able to create, alter, or delete users. Procedure 1. Log in to SAP HANA studio with a user name that has been granted the USER ADMIN system privilege. 2. Grant the role sap.hana.uis.db::site_designer to the user who will do the RBA (Role-Based Authorization) UI configuration. Configure Smart Data Integration 2016 SAP SE or an SAP affiliate company. All rights reserved. 39

3. Grant the application privilege sap.hana.im.dp.monitor.roles::monitoring to the user who will add the Data Provisioning tiles to SAP HANA Cockpit and will also perform monitoring tasks. a. In the Systems view, expand your SAP HANA server name and expand Security. b. Double click the user name. c. On the Granted Roles tab, click the + icon in the upper left corner. d. On the Select Roles dialog, type dp in the search string box. e. Select role sap.hana.im.dp.roles::monitoring and click OK. This role sap.hana.im.dp.roles::monitoring will allow the user to access the SAP HANA Data Provisioning monitoring user interfaces. Next Steps To use the Data Provisioning Monitoring user interface through the SAP HANA Cockpit you must also assign the appropriate catalog to this monitoring role (Assign Catalog to Data Provisioning Monitoring Role [page 40]). However, users can also view the monitors directly by entering the following URLs in a web browser: <host name>:80<2 digit instance number>/sap/hana/im/dp/monitor/?view=dpagentmonitor <host name>:80<2 digit instance number>/sap/hana/im/dp/monitor/? view=dpsubscriptionmonitor <host name>:80<2 digit instance number>/sap/hana/im/dp/monitor/?view=imtaskmonitor 3.4.2 Assign Catalog to Data Provisioning Monitoring Role This procedure uses the Role-Based Authorization (RBA) user interface to assign the SAP HANA Data Provisioning Monitoring catalog to the role sap.hana.im.dp.roles::monitoring). Prerequisites You have the role sap.hana.uis.db::site_designer. For information about how to grant this role, see Grant Roles to Users [page 39]. 40 2016 SAP SE or an SAP affiliate company. All rights reserved. Configure Smart Data Integration

Context To configure the RBA for the SAP HANA Data Provisioning catalog in SAP HANA Cockpit: Procedure 1. To launch the RBA user interface, type the following URL address into a web browser: http://<host>:80<2-digit instance>/sap/hana/uis/clients/role-editor/ RoleEditor.html?scenario=onPremise 2. Log in with a user name that has the role sap.hana.uis.db::site_designer. 3. In the Roles pane on the left, go to the role sap.hana.im.dp.roles::monitoring and select it. You can search for this role using the search text box in the Roles pane. 4. To assign the catalogs to this role, click on the + button next to Assigned Catalogs. A list of catalogs appears. 5. Select HANA Data Provisioning Monitoring from the list of catalogs. You may select other catalogs if you wish to use their tiles in the SAP HANA Cockpit. 6. When you are done selecting catalogs, click OK. 3.4.3 Add Data Provisioning Tiles to SAP HANA Cockpit Create a Data Provisioning group in SAP HANA Cockpit and add the Data Provisioning tiles to the group to enable monitoring of Data Provisioning agents, remote subscriptions, and tasks. Prerequisites You have the role sap.hana.im.dp.roles::monitoring. The role sap.hana.im.dp.roles::monitoring has been authorized to catalog HANA Data Provisioning Monitoring. Procedure 1. Launch SAP HANA Cockpit by taking one of the following actions: In the Systems view of SAP HANA studio, right-click the name of your SAP HANA server and choose Configuration and Monitoring Open SAP HANA Cockpit. In a web browser, type the following URL address: http://<<host name>:80<2 digit instance number>/sap/hana/admin/cockpit 2. Log in to SAP HANA Cockpit with a user name that has the role sap.hana.im.dp.roles::monitoring. Configure Smart Data Integration 2016 SAP SE or an SAP affiliate company. All rights reserved. 41

Note If you cannot launch SAP HANA Cockpit, then ask your administrator to grant you either the application privilege sap.hana.admin::monitoring or the role sap.hana.admin.roles::monitoring. 3. Click on the Show/hide group panel icon in the upper left corner to display options New Group and Tile Catalog. 4. Click New Group and enter a name. 5. Click Tile Catalog to display the tiles in the catalog HANA Data Provisioning Monitoring. 6. To add each tile (Agent Monitor, Remote Subscription Monitor, Task Monitor, and Smart Data Integration for Documentation): a. Click the + icon at the bottom of the tile. b. On the Add Tile To Groups window, select the name of your group and click OK. As you add each tile, the + icon turns into a checkmark. 7. Click Home to access your group and tiles. 3.5 Enable Enterprise Semantic Services Enterprise Semantic Services provides an API to enable searching for publication artifacts or run-time objects based on their metadata and contents. Note Enterprise Semantic Services (ESS) is required for SAP HANA Agile Data Preparation. However, ESS is optional for smart data integration (SDI); therefore, you are not required to download and install ESS for SDI. Parent topic: Configure Smart Data Integration [page 13] Previous: Set up Data Provisioning monitoring [page 39] Related Information SAP HANA Enterprise Semantic Services JavaScript API Reference SAP HANA Enterprise Semantic Services REST API Reference 42 2016 SAP SE or an SAP affiliate company. All rights reserved. Configure Smart Data Integration

3.5.1 Setting Up the SAP HANA Instance for Enterprise Semantic Services The Enterprise Semantic Services component supports both on-premise (single database and multitenant), and SAP HANA cloud platform deployments. For details on supported versions, see the Product Availability Matrix at https://apps.support.sap.com/ sap(bd1lbizjptawmq==)/support/pam/pam.html?smpsrv=https%3a%2f%2fwebsmp202.sapag.de&pvnr=73554900100900000125&pt=g%7cd#pvnr=73554900100900000125&ts=0&pt=g%7cd 3.5.1.1 On Premise Single Database Deployment For a single database deployment, Enterprise Semantic Services requires the SAP HANA job scheduler and script server. Context Perform the following procedures to set up the SAP HANA instance for Enterprise Semantic Services in an on premise single database deployment. Procedure 1. Enable the XS Job Scheduler. 2. Enable the Script Server. Related Information Enable the Script Server [page 51] Configure Smart Data Integration 2016 SAP SE or an SAP affiliate company. All rights reserved. 43

3.5.1.1.1 Enable the XS Job Scheduler Enable the XS Job Scheduler for Enterprise Semantic Services. Context Procedure 1. Log in to SAP HANA studio as the SYSTEM user or user name with the equivalent SYSTEM privileges. 2. Navigate to the appropriate.ini element and set the parameters as follows. a. Double-click your system in the Systems view. b. Choose the Configuration tab. c. Expand the xsengine.ini element, and go to the scheduler section. d. Set the Enabled parameter to true. e. Set the sessiontimeout parameter to 36000. Note If the scheduler section does not exist, create it and set its parameters with the following steps: 1. Right-click xsengine.ini and select Add Section. 2. Type scheduler for Section Name and click Next. 3. Click Next on the Scope Selection dialog. Type Enabled for Key and type true for Value. 4. Click Add New Pair. 5. Type sessiontimeout for Key and type 36000 for Value. Set the sessiontimeout parameter to 36000. 6. Click Finish. 3. Alternatively, you can open the SQL console of the SAP HANA studio and execute the following statements: ALTER SYSTEM ALTER CONFIGURATION ('xsengine.ini','system') SET ('scheduler','enabled')='true' with reconfigure; ALTER SYSTEM ALTER CONFIGURATION ('xsengine.ini', 'SYSTEM') SET ('scheduler', 'sessiontimeout')='36000'; 44 2016 SAP SE or an SAP affiliate company. All rights reserved. Configure Smart Data Integration

3.5.1.1.2 Enable the Script Server Enable the Script Server if you will be using SAP HANA smart data quality. Context If smart data quality is already installed, you do not need to enable the Script Server. Procedure 1. In the Administration editor, choose the Configuration tab. 2. Expand the daemon.ini configuration file. 3. Select and expand scriptserver, select Instances, right click, and choose Change. 4. Set Instances to 1. Related Information SAP HANA Administration Guide: Change a System Property (HTML) SAP HANA Administration Guide: Change a System Property (PDF) 3.5.1.2 On Premise Multitenant Deployment For a multitenant deployment, Enterprise Semantic Services requires the SAP HANA script server and access to multitenant database containers. Prerequisites A HANA instance has been installed with multiple containers At least one tenant database container (for example, DB0) has been created Procedure 1. Add the scriptserver service to the tenant database. Configure Smart Data Integration 2016 SAP SE or an SAP affiliate company. All rights reserved. 45

For example, in the Web-based Development Workbench or SAP HANA studio, enter the following SQL statement: ALTER DATABASE DB0 ADD 'scriptserver'; Note If the Smart Data component is already installed, then the scriptserver service is already added. 2. Configure HTTP access to Multitenant Database Containers. Use the information in the topic "Configure HTTP Access to Multitenant Database Containers" in SAP HANA Administration Guide. Note To determine the port of the XS engine service used by a particular tenant database, use the following SQL query: SELECT DATABASE_NAME, SERVICE_NAME, PORT, SQL_PORT, (PORT + 2) HTTP_PORT FROM SYS_DATABASES.M_SERVICES WHERE DATABASE_NAME='<DBNAME>' and ((SERVICE_NAME='indexserver' and COORDINATOR_TYPE= 'MASTER') or (SERVICE_NAME='xsengine')) 3. Add the tenant database alias names that you configured in step 2 using the parameter SRCVHOST to your Domain Name System (DNS). 3.5.2 Download Enterprise Semantic Services Delivery Unit Download the Enterprise Semantic Services delivery unit and deploy it to enable semantic searches of data sources. Context The HANA_IM_ESS delivery unit is included with the smart data integration (SDI) download package. 46 2016 SAP SE or an SAP affiliate company. All rights reserved. Configure Smart Data Integration

Procedure 1. Go to the SAP Software Download Center. 2. Select Software Downloads. 3. Select the Support Packages & Patches tab, expand By Alphabetical Index (A-Z), and select H. 4. Scroll down and select SAP HANA SDI SAP HANA SDI 1.0. 5. Select Comprised Software Component Versions HANA ESS 1.0 # OS independent SAP HANA database. 6. From the list of downloads, select the desired version. 7. In the downloaded ZIP file, locate the HANAIMESS.tgz file and extract. Related Information Product Availability Matrix 3.5.3 Import the Enterprise Semantic Services Delivery Unit To install Enterprise Semantic Services (ESS), after downloading the ESS delivery unit, you then import it. Procedure 1. Log in to SAP HANA studio as user SYSTEM. 2. In the upper left corner, click File Import. 3. On the Import dialog, type delivery unit into the search box for Select an import source. 4. Click Next. 5. Select <your SAP HANA Server name> for the target system, and click Next. 6. Select the HANAIMESS.tgz file that you downloaded. 7. Click Finish. 8. In "Job Log" view, the status should be "Completed successfully". Configure Smart Data Integration 2016 SAP SE or an SAP affiliate company. All rights reserved. 47

3.5.4 Install or Upgrade Enterprise Semantic Services (install.html) After downloading and importing the Enterprise Semantic Services (ESS) delivery unit, install this component to enable semantic searches of data sources. Prerequisites If you are upgrading ESS: Upgrade your SAP HANA instance if you need to upgrade to a new SPS revision. If installed, uninstall the DEMO delivery unit. Uninstall Enterprise Semantic Services. Context If you have ESS version SPS01 Patch 1 (also known as 1.0 SP00 Rev1) or earlier, then follow the procedure that requires the installation script install.xsjs. If you have ESS version SPS01 Patch 2 (also known as 1.0 SP01 Rev2) or later, then follow this procedure, which requires the installation script install.html. Procedure 1. Access the following URL: http://<your_hana_instance:port>/sap/hana/im/ess/setup/ install.html 2. Log in with the SYSTEM user. If the SYSTEM user is not available, then use any other administrative user and assign it the sap.hana.im.ess.roles.setup::se_installer role for the time of the installation. 3. On the Welcome page, click Run ESS Setup. You can monitor the details of the installation status. 4. You are prompted set the password for ESS technical users. Enter a password and click Submit. 5. Ensure the _HANA_IM_ESS, _HANA_IM_ESS_CTID, and _HANA_IM_ESS_PROFILING technical users have been created by the installation script in SAP HANA. Results Successful installation is indicated with the message Setup completed including a Status table that lists each setting. 48 2016 SAP SE or an SAP affiliate company. All rights reserved. Configure Smart Data Integration

At any point you can monitor the installation status by accessing the install.html URL. Any errors will display with messages for corrective actions. Related Information Uninstall Enterprise Semantic Services [page 50] 3.5.5 Grant Enterprise Semantic Services Roles and Privileges to Users After Enterprise Semantic Services is installed, grant the necessary roles to the SAP HANA users who will interact directly or indirectly with Enterprise Semantic Services. Prerequisites Ensure that you have been granted the EXECUTE privilege on the GRANT_ACTIVATED_ROLE procedure. Procedure 1. Log in to SAP HANA studio. 2. In the Systems view, expand your SAP HANA server name and expand Security. 3. Grant the following roles: sap.hana.im.ess.roles::administrator to the user who will access the Enterprise Semantic Services Administration user interface sap.hana.im.ess.roles::publisher to the user who will access the Enterprise Semantic Services publication API to define content to be published in the Entity Grid sap.hana.im.ess.roles::user to the user who will access the Enterprise Semantic Services consumption (read-only) APIs such as Search, Autocomplete, and content-type identification (CTID). For each user who will use Enterprise Semantic Services, do the following steps to grant a role: a. Double-click the user name. b. On the Granted Roles tab, click the + icon in the upper left corner. c. On the Select Roles dialog, type ess in the search string box. d. Select the appropriate role for this user and click OK. Configure Smart Data Integration 2016 SAP SE or an SAP affiliate company. All rights reserved. 49

3.5.6 Uninstall Enterprise Semantic Services These instructions describe the process to permanently uninstall Enterprise Semantic Services, for example in the case of an upgrade. Prerequisites If SAP Agile Data Preparation is installed, uninstall it first. Add the role sap.hana.im.ess.role::administrator to the SYSTEM user or the user who will uninstall. Procedure 1. Remove the delivery unit a. Enter the following URL in a web browser: http://<<your_hana_instance:port>>/sap/hana/xs/lm b. Choose Products Delivery Units. c. Select HANA_IM_ESS. d. Click Delete. e. Click the checkbox including objects and packages. f. Confirm deletion. 2. Remove users. In the Web-based Development Workbench or SAP HANA studio, drop the Enterprise Semantic Services users. For example, in SAP HANA studio, enter the following SQL statements: DROP USER _HANA_IM_ESS CASCADE; DROP USER _HANA_IM_ESS_PROFILING CASCADE; DROP USER _HANA_IM_ESS_CTID CASCADE 3. Remove the schema. In the Web-based Development Workbench or SAP HANA studio, drop the HANA_IM_ESS schema. For example, in SAP HANA studio, enter the following SQL statement: DROP SCHEMA SAP_HANA_IM_ESS CASCADE; 50 2016 SAP SE or an SAP affiliate company. All rights reserved. Configure Smart Data Integration

4 Configure Smart Data To take advantage of smart data quality functionality, you will need to perform a few tasks. You must do the following to use smart data quality: Enable the Script Server Download and deploy the smart data quality directories (Optional) Set alerts to inform you when the directories will expire Related Information Enable the Script Server [page 51] Download and Deploy Directory Files [page 52] Create Monitoring Alerts [page 91] 4.1 Enable the Script Server Enable the Script Server if you will be using SAP HANA smart data quality. Context If smart data quality is already installed, you do not need to enable the Script Server. Procedure 1. In the Administration editor, choose the Configuration tab. 2. Expand the daemon.ini configuration file. 3. Select and expand scriptserver, select Instances, right click, and choose Change. 4. Set Instances to 1. Configure Smart Data 2016 SAP SE or an SAP affiliate company. All rights reserved. 51

Related Information SAP HANA Administration Guide: Change a System Property (HTML) SAP HANA Administration Guide: Change a System Property (PDF) 4.2 Download and Deploy Directory Files Download and deploy optional directories to take advantage of smart data quality functionality. Context The Cleanse and Geocode nodes rely on reference data found in directories you download and deploy to the SAP HANA server. If reference data is not provided, the Cleanse node will perform parsing, but it will not do assignment. Additionally, you will be able to create and activate flowgraphs that include the Geocode node, but their execution will fail. You may need to download multiple directories, depending on your license agreement. You need to follow steps 4-8 for each of the directories that you need to download. Note We recommend that, before you install your directories, you stop the Script server, and then restart it once the installation is complete. We also recommend restarting the SAP HANA database after installing and making sure that you do not have any running flowgraph tasks. Procedure 1. Access the SAP Support Portal. 2. Click Software Downloads. 3. Click the Installation and Upgrades tab, expand By Alphabetical Index (A-Z), and click A ADDRESS DIRECTORIES & REFERENCE DATA. 4. Click <directory_name>. (For example, SAP ADDR DIR ALL WORLD 4.X) 5. Click <directory_name> again to display the list of download files. 6. Click the check box to the left of the item or items that you want to install, and choose Download from the action menu. 7. To download directories, click the Address Directories & Reference Data link above the Downloads section. 8. Follow steps 4-8 for each directory as needed. 9. Run the Download Manager to start the download process. 52 2016 SAP SE or an SAP affiliate company. All rights reserved. Configure Smart Data

10. Once they have been downloaded, extract the directory files, and then copy and paste your directories (.dir) into the following SAP HANA Server installation location: /usr/sap/<sid>/sys/global/hdb/im/reference_data Next Steps Note If you need to change the directory location, you can do so by running the following: ALTER SYSTEM ALTER CONFIGURATION ('scriptserver.ini', 'SYSTEM') SET ('adapter_framework', 'dq_reference_data_path')='<your Path>' WITH RECONFIGURE; Related Information Address Directories & Reference Data 4.3 Update Directories Be aware of the following when updating your directories. To update, remove, or add reference data to the existing directories, create a new directory location. Copy and paste any unchanged data to the new directory location, and add any new or updated directories. Then make sure to update the directory location of the dq_reference_data in your SAP HANA applications to reference the new location. Changing the reference data path will drop all cached operations (if cache is enabled), and recreate the cache. We recommend that you update the dq_reference_data path when flowgraph task plans are not running, or when the server is not busy. You can also set alerts to inform you when the directories will expire. Related Information Create Monitoring Alerts [page 91] Configure Smart Data 2016 SAP SE or an SAP affiliate company. All rights reserved. 53

5 Update the Data Provisioning Agent You can update the Data Provisioning Agent by running the installation program in update mode. Prerequisites Before you update the Data Provisioning Agent, ensure that your SAP HANA server has already been updated to the same revision. If your agent has remote subscriptions for real-time data capture, you must also suspend capture before upgrading the agent. To suspend active remote source subscriptions, use the SQL console in the SAP HANA Studio or Web-based Development Workbench: ALTER REMOTE SOURCE <remote_source_name> SUSPEND CAPTURE Note To verify the success of the upgrade for log reader adapters, you should also set the adapter framework logging level to INFO. To change the adapter framework logging level, choose Preferences Adapter Framework Logging Level in the SAP HANA Data Provisioning Agent Configuration tool. Procedure 1. Download and extract the software to an empty directory. 2. Stop the existing SAP HANA Data Provisioning Agent service. On Windows, use the Services manager in Control Panel. On Linux, run./dpagent_service.sh stop. 3. Navigate to the directory where you unpacked the software. 4. Call the installation manager. On Windows, run hdbsetup.exe. On Linux, run./hdbsetup with root privileges. For example, sudo./hdbsetup. Tip To upgrade the agent in command-line mode, use hdbinst.exe on Windows or./hdbinst on Linux. 5. Choose Update SAP HANA Data Provisioning Agent and select the path of the existing agent that you want to update. In command-line mode, enter the number of the existing agent as listed by the installation program. 54 2016 SAP SE or an SAP affiliate company. All rights reserved. Update the Data Provisioning Agent

6. On Linux, update the adapter configurations to the latest version and restart the Agent service. a. Stop any running dpagent processes. b. Navigate to the <DPAgent_root>/bin directory. c. Run./dpagent_upgrade.sh. 7. On Linux, restart the Agent service. a. Navigate to the <DPAgent_root>/bin directory. b. Run./dpagent_service.sh start. 8. Apply new adapter capabilities. To allow SAP HANA to detect any new adapter capabilities, use the SQL console in the SAP HANA Studio or Web-based Development Workbench. a. Retrieve a list of the adapters configured in your environment. select * from "SYS"."ADAPTERS" where "IS_SYSTEM_ADAPTER" like 'FALSE' b. Refresh each adapter listed by the previous command. ALTER ADAPTER "<adapter_name>" REFRESH AT LOCATION "<agent_name>" Note If you have multiple agents in your SAP HANA environment, you only need to refresh each adapter a single time with an upgraded agent. Refreshing the adapters with each agent is not necessary. Results The Data Provisioning Agent is updated to the new version. Next Steps If you suspended capture for remote source subscriptions on the agent, you can now resume capture. Caution Before resuming capture, you must first upgrade all agents in your SAP HANA environment. If you have not upgraded all agents, do that first and then return to this section. After all agents have been upgraded, use the SQL console in the SAP HANA Studio or Web-based Development Workbench to resume capture: ALTER REMOTE SOURCE <remote_source_name> RESUME CAPTURE Repeat the command for each remote source subscription in your environment. Update the Data Provisioning Agent 2016 SAP SE or an SAP affiliate company. All rights reserved. 55

Tip After you resume a remote source subscription, additional automatic upgrade steps take approximately 10 minutes to complete. To verify that the process has completed successfully, view the Data Provisioning Agent framework log: When the adapter upgrade has been completed, you should see the message <instance_name> has been upgraded successfully. When the real-time replication has been resumed, you should see the message <instance_name> is resumed successfully. 5.1 Update the SAP ASE Adapter Preferences After updating the Data Provisioning Agent, you must update the SAP ASE Adapter Preferences before you can use it in SAP HANA smart data integration processes. Procedure 1. Add the following line to <DPAgent_root>/adapters/cppframework.props: ASEAdapter.ld_library_path=<installpath>/Sybase/ci/64bit,<installpath>/Sybase/ OCS-16_0/lib,<installpath>/Sybase/OCS-16_0/lib3p64 Replace <installpath> with the actual path to the Data Provisioning Agent installation. 2. Add the following lines to <DPAgent_root>/bin/dpagent_env.sh: SYBASE=<installpath>/Sybase export SYBASE SYBASE_OCS=OCS-16_0 export SYBASE_OCS Replace <installpath> with the actual path to the Data Provisioning Agent installation. 3. Add the following lines to <DPAgent_root>/Sybase/interfaces: ASEAdapterOCSServer master tcp ether <hostname> <adapteragentport > query tcp ether <hostname> <adapteragentport> Replace <hostname> with the host name or IP address of the Data Provisioning Agent host machine, and <adapteragentport> with the port number configured for the Data Provisioning Agent. 56 2016 SAP SE or an SAP affiliate company. All rights reserved. Update the Data Provisioning Agent

6 Monitor and Operate This section explains Data Provisioning monitoring. Monitoring Data Provisioning [page 57] Monitor the Data Provisioning Server availability and key performance indicators regularly. Monitoring Data Provisioning Agents [page 60] You can monitor basic system information of an agent (such as CPU, memory, timestamps) and the time it last connected with the Data Provisioning Server. Monitoring Remote Subscriptions [page 64] The Data Provisioning Remote Subscription Monitor tile in the SAP HANA Cockpit provides you with information about how well data is being replicated to the Data Provisioning Server. Monitoring Design Time Objects [page 69] The Data Provisioning Design Time Object Monitor tile in the SAP HANA cockpit provides you with information about your design time objects, including flowgraphs and replication tasks. Monitoring Data Provisioning Tasks [page 74] The Data Provisioning Task Monitor tile in the SAP HANA cockpit provides you with information about your replication tasks and transformation tasks. Create Status Notifications [page 89] Create email notifications for various task, remote subscription, and design time object statuses. Create Monitoring Alerts [page 91] Create monitoring alerts for various functions. Monitoring and Troubleshooting Enterprise Semantic Services [page 91] Monitor Enterprise Semantic Services publications and the Entity Grid to ensure that the semantic search for publication artifacts or run-time objects works smoothly. Troubleshooting actions are available on the Enterprise Semantic Services monitors. 6.1 Monitoring Data Provisioning Monitor the Data Provisioning Server availability and key performance indicators regularly. The following monitoring tools are available: The Data Provisioning Monitor provides a browser based interface to monitor agents, tasks, design time objects (flowgraphs and replication tasks), and remote subscriptions created in the SAP HANA system. After you take steps to set up this Data Provisioning Monitor, you can view the monitors in the following ways: Enter the URL address of each monitor directly into a web browser. Access Data Provisioning tiles in SAP HANA Cockpit. The SAP HANA Administration and Monitoring group of tiles in SAP HANA Cockpit provides a Database Status application in which you monitor the Data Provisioning Server. For more information, see the SAP HANA Administration Guide, section Analyze Status and Resource Usage of Services. Monitor and Operate 2016 SAP SE or an SAP affiliate company. All rights reserved. 57

6.1.1 SAP HANA Data Provisioning Monitoring The Data Provisioning monitor user interface is delivered through the Data Provisioning delivery unit. A catalog is also delivered with this delivery unit for integration of these monitors into the SAP HANA Cockpit. Related Information Set up Data Provisioning monitoring [page 39] 6.1.2 SAP HANA Tile Catalog for Data Provisioning The tile catalog defines the set of all tiles available in the SAP HANA cockpit. Within the main tile catalog, tiles are grouped into sub-catalogs according to functional area. The tile group SAP HANA Data Provisioning contains the tiles for monitoring agents, remote subscriptions, tasks, and design time objects, as well as a tile to reference the documentation on the SAP Help Portal. The tiles are dynamic and update automatically with information from each monitor. 6.1.3 Access the Data Provisioning Monitors You can access the Data Provisioning Monitors through either SAP HANA Cockpit or by typing the URLs directly into your web browsers. Prerequisites You have the role sap.hana.im.dp.roles::monitoring which allows you to access the SAP HANA Data Provisioning monitoring user interfaces. For information about how to grant this role, see Grant Roles to Users [page 39]. You have the role sap.hana.im.dp.roles::operations, which allows you to perform all actions, such a START TASK, STOP TASK, SCHEDULE TASK, and PROCESS REMOTE EXCEPTIONS. Your Web browser supports the SAPUI5 library sap.m (for example, Internet Explorer 9). For more information about SAPUI5 browser support, see SAP Note 1716423 and the Product Availability Matrix (PAM) for SAPUI5. 58 2016 SAP SE or an SAP affiliate company. All rights reserved. Monitor and Operate

Procedure 1. To open the Data Provisioning monitors through SAP HANA Cockpit: a. Enter the SAP HANA cockpit URL in your browser. http://<host>:<port>/sap/hana/admin/cockpit b. If required, enter your database user name and password. c. Go to the group of tiles that contain the Data Provisioning monitors. 2. To open the Data Provisioning monitors directly, enter the following URLs in your browser. Option Data Provisioning Agent Monitor Data Provisioning Subscription Monitor Data Provisioning Task Monitor Data Provisioning Design Time Object Monitor Description <host name>:80<2 digit instance number>/sap/hana/im/dp/ monitor/index.html?view=dpagentmonitor <host name>:80<2 digit instance number>/sap/hana/im/dp/ monitor/index.html?view=dpsubscriptionmonitor <host name>:80<2 digit instance number>/sap/hana/im/dp/ monitor/index.html?view=imtaskmonitor <host name>:80<2 digit instance number>/sap/hana/im/dp/ monitor/index.html?view=imdesigntimeobjectmonitor 6.1.4 Configure User Settings Profiles By creating user settings profiles, you can quickly switch between different monitor layouts.settings profiles contain information about visible columns, column order, column width, column filters, table visibility, and slider positions. Context You can create, modify, or remove settings profiles in each Data Provisioning Monitor by clicking the Settings button. Procedure To add a new settings profile, click Add. a. Specify a name for the profile and whether to make it the default profile. b. Click Add. A new profile is created using the current layout and column display settings. To switch to an existing settings profile, select the profile and click Load. The current layout and column display settings are updated from the settings saved in the profile. Monitor and Operate 2016 SAP SE or an SAP affiliate company. All rights reserved. 59

To modify an existing settings profile, select the profile and click Update. a. If you want to make the profile the default profile, select Default. b. Click Update. The selected profile is updated with the current layout and column display settings. To remove an existing settings profile, select the profile and click Delete. The selected profile is removed from the Profiles table. 6.2 Monitoring Data Provisioning Agents You can monitor basic system information of an agent (such as CPU, memory, timestamps) and the time it last connected with the Data Provisioning Server. When you first open the Data Provisioning Agent Monitor, it displays information for all agents and all adapters. You can select an agent to show information only for that agent. Information Available on the Data Provisioning Agent Monitor The Data Provisioning Agent Monitor displays information in two tables Agent Monitor and Adapter Agent Mapping. If you select an agent in the first table, the second table shows only the adapters registered to that agent. Errors, warnings, and action execution information are displayed in the Status Console. Table 1: Information Available in Agent Monitor Table Column Agent Name Agent Host Agent Port State Description Name of the Data Provisioning Agent. Name of the host on which the agent is running Port that the agent uses to communicate with the Data Provisioning Server. Indicates the state of the agent. The following states are possible: CONNECTING DISCONNECTED CONNECTED Last Connect Last Connect Time Adapters Elapsed time since the last connection from the Data Provisioning Server to the Data Provisioning Agent. The last connect time from the Data Provisioning Server to the Data Provisioning Agent. Number of adapters defined for this Data Provisioning Agent. 60 2016 SAP SE or an SAP affiliate company. All rights reserved. Monitor and Operate

Column Protocol Description Type of network protocol used between the Data Provisioning Agent and the Data Provisioning Server. The following protocols are possible: TCP HTTP Used Memory Used Swap Space Free Memory Free Swap Space Amount of memory currently used by the agent. Amount of swap space currently used by the agent. Amount of free memory on the agent host. Amount of free swap space on the agent host. Table 2: Information Available in Adapter Agent Mapping Table Column Adapter Name Agent Name Description Name of the adapter registered to the Data Provisioning agent. Name of the Data Provisioning Agent. 6.2.1 Manage Agents from the Data Provisioning Agent Monitor You can use the Data Provisioning Agent Monitor to perform basic administration tasks such as registering, altering, or dropping Data Provisioning Agents. Prerequisites The user must have the following roles or privileges to manage agents. Table 3: Roles and Privileges Action Role or Privilege Add Data Provisioning Agent Role: sap.hana.im.dp.monitor.roles::operations Application privilege: sap.hana.im.dp.monitor::createagent System privilege: AGENT ADMIN Alter Data Provisioning Agent Role: sap.hana.im.dp.monitor.roles::operations Application privilege: sap.hana.im.dp.monitor::alteragent System privilege: AGENT ADMIN Remove Data Provisioning Agent Role: sap.hana.im.dp.monitor.roles::operations Application privilege: sap.hana.im.dp.monitor::dropagent System privilege: AGENT ADMIN Monitor and Operate 2016 SAP SE or an SAP affiliate company. All rights reserved. 61

Context Use the buttons in the Agent Monitor table to perform the action. Procedure 1. Click Create Agent to register a new agent with the SAP HANA system. a. Specify the name of the agent and relevant connection information. b. If the agent uses a secure SSL connection, check Enable SSL. c. Click Create Agent. The new agent appears in the Agent Monitor table. 2. Click Alter Agent to make connection configuration changes on an agent already registered in the SAP HANA system. a. Specify the new connection information for the agent. You cannot change the name or connection protocol for an existing agent. b. If the agent uses a secure SSL connection, check Enable SSL. c. Click Alter Agent. The updated agent information appears in the Agent Monitor table. 3. Click Drop Agent to remove an agent from the SAP HANA system. a. To automatically drop any dependent objects such as registered adapters, choose CASCADE option. You cannot remove an agent while it has dependent objects such as registered adapters. First manually remove the adapters from the agent, or check CASCADE option. b. Click Drop Agent. The agent is removed from the Agent Monitor table. Related Information Assign Roles and Privileges [page 115] 62 2016 SAP SE or an SAP affiliate company. All rights reserved. Monitor and Operate

6.2.2 Manage Adapters from the Data Provisioning Agent Monitor You can use the Data Provisioning Agent Monitor to perform basic administration tasks such as adding adapters to or removing adapters from a Data Provisioning Agent instance. Prerequisites The user must have the following roles or privileges to manage adapters. Table 4: Roles and Privileges Action Role or Privilege Add adapter Role: sap.hana.im.dp.monitor.roles::operations Application privilege: sap.hana.im.dp.monitor::addlocationtoadapter System privilege: ADAPTER ADMIN Remove adapter Role: sap.hana.im.dp.monitor.roles::operations Application privilege: sap.hana.im.dp.monitor::removelocationfromadapter System privilege: ADAPTER ADMIN Context Use the buttons in the Agent Monitor and Agent Adapter Mapping tables to perform the action. Procedure 1. To add adapters to an agent instance, select the agent and click Add Adapters in the Agent Monitor table. a. Select the desired adapters from the list of adapters deployed on the agent instance. b. Click Add Adapters. The selected adapters appear in the Adapter Agent Mapping table. 2. To remove an adapter from an agent instance, select the adapter and click Remove Location in the Adapter Agent Mapping table. a. If the adapter is registered on only one agent instance, you can choose to remove with the CASCADE option. b. Click Remove Location. The adapter is removed from the Adapter Agent Mapping table. Monitor and Operate 2016 SAP SE or an SAP affiliate company. All rights reserved. 63

Related Information Assign Roles and Privileges [page 115] 6.3 Monitoring Remote Subscriptions The Data Provisioning Remote Subscription Monitor tile in the SAP HANA Cockpit provides you with information about how well data is being replicated to the Data Provisioning Server. When you first open the Data Provisioning Remote Subscription Monitor, it displays information for all remote sources, all remote subscriptions, and all remote subscription statistics. You can select a remote source to view its corresponding remote subscriptions and remote subscription statistics. 6.3.1 Remote Subscription States for Data Provisioning You create remote subscriptions when you want to listen for real-time changes to data that you replicated into the Data Provisioning Server. A remote subscription can have seven different states. The following diagram shows where the states occur during the initial load and subsequent updates. The state names and their descriptions are in the table below. Table 5: Remote Subscription States State Name CREATED MAT_START_BEG_MARKER MAT_START_END_MARKER MAT_COMP_BEG_MARKER MAT_COMP_END_MARKER AUTO_CORRECT_CHANGE_DATA Description Remote subscription is created by the replication task. The receiver is waiting for the begin marker that indicates the first changed data to queue while the initial load is running. The receiver queues the rows and is waiting for the end marker that indicates the last row of the initial load. The receiver is waiting for the begin marker that indicates the first row to queue after the initial load has completed. The receiver queues the changed rows and is waiting for the end marker that indicates the last row of the initial load. The initial load has completed and the end marker is sent to the adapter. If the state does not change to AUTO_COR RECT_CHANGE_DATA, the adapter or source system is slow in capturing the changes. When the end marker is received for the initial load, the applier loads the changed data captured (and queued during the initial load) to the target. If a lot of changes occurred after the initial load started, this state might take a long time to change to APPLY_CHANGE_DATA. APPLY_CHANGE_DATA All of the changes captured while the initial load was running have completed and are now loaded to the target. 64 2016 SAP SE or an SAP affiliate company. All rights reserved. Monitor and Operate

6.3.2 Information Available on the Data Provisioning Remote Subscription Monitor The Data Provisioning Remote Subscription Monitor provides information in tables. The Data Provisioning Remote Subscription Monitor displays information in three tables Remote Source Monitor, Remote Subscription Monitor (for all sources or one source), and Remote Subscription Statistics (for all sources or one source). If you select a remote source in the first table, the second and third tables show only information for that remote source. Errors, warnings, and action execution information are displayed in the Status Console. Table 6: Information Available in Remote Source Monitor Table Column Remote Source Name Adapter Name Location Agent Name CDC Status Description Name of the remote source. Name of the adapter. Port that the agent uses to communicate with the Data Provisioning Server. Name of the agent for this remote source. The status of changed-data capture on this remote source. The following status values are possible: OK ERROR SUSPENDED Subscriptions Number of subscriptions defined for this remote source. Table 7: Information Available in Remote Subscription Monitor Table Column Subscription Name Schema Name Remote Source Name Design Time Name Description Name of the remote subscription. Name of the schema (user name) in the remote source. Name of the remote source for which this subscription is defined. Name of corresponding Design Time Object. Tip You can click the name of a design time object to open that object in the Data Provisioning Design Time Object Monitor. Design Time Type Valid State Last Processed Type of corresponding Design Time Object. Values would be hdbflowgraph, hdbreptask, or empty. Whether or not the remote subscription is valid. Name of the state of the remote subscription. For more information, see Remote Subscription States for Data Provisioning [page 64]. Elapsed time since the last changed data was processed. Monitor and Operate 2016 SAP SE or an SAP affiliate company. All rights reserved. 65

Column Last Processed Transaction Time Subscription Type Description Time the last changed data was processed. Type of subscription. The following values are possible: TABLE VIRTUAL TABLE Target Type Type of target. The following values are possible: TABLE VIRTUAL TABLE Table 8: Information Available in Remote Subscription Statistics Table Column Subscription Name Schema Name Received Count Applied Count Received Size Applied Size Rejected Count Receiver Latency Applier Latency Last Message Received Last Message Applied Description Name of the remote subscription. Name of the schema (user name) in the remote source. Total number of messages received by the Data Provisioning server. Total number of messages applied. Total size of messages received by Data Provisioning server. Total size of messages applied. Total number of messages rejected. Duration Duration Time the last message was received. Time the last message was applied. 6.3.3 Suspend and Resume Remote Sources You can suspend and resume capture and distribution for remote sources within the Data Provisioning Remote Subscription Monitor. Prerequisites The user must have the following roles or privileges to suspend and resume capture and distribution: 66 2016 SAP SE or an SAP affiliate company. All rights reserved. Monitor and Operate

Table 9: Roles and Privileges Action Role or Privilege Suspend capture or distribution Role: sap.hana.im.dp.monitor.roles::operations Application privilege: sap.hana.im.dp.monitor::alterremotesource Object privilege: ALTER on the remote source Resume capture or distribution Role: sap.hana.im.dp.monitor.roles::operations Application privilege: sap.hana.im.dp.monitor::alterremotesource Object privilege: ALTER on the remote source Context Use the Alter Remote Source button in the monitor to perform the action. Procedure 1. Select the remote source in the Remote Source Monitor table and click Alter Remote Source. 2. Click Suspend or Resume for CAPTURE or DISTRIBUTION. Confirmation of the action is displayed in the status console. 3. Close the Alter Remote Source dialog. Results Capture or distribution for the selected remote source is suspended or resumed. Related Information Assign Roles and Privileges [page 115] Monitor and Operate 2016 SAP SE or an SAP affiliate company. All rights reserved. 67

6.3.4 Manage Remote Subscriptions You can drop, queue, distribute, and reset remote subscriptions within the Data Provisioning Remote Subscription Monitor. Prerequisites The user must have the following roles or privileges to manage remote subscriptions: Table 10: Roles and Privileges Action Role or Privilege Reset, queue, or distribute remote subscription Role: sap.hana.im.dp.monitor.roles::operations Application privilege: sap.hana.im.dp.monitor::alterremotesubscription Object privilege: ALTER on the remote subscription Drop remote subscription Role: sap.hana.im.dp.monitor.roles::operations Application privilege: sap.hana.im.dp.monitor::dropremotesubscription Object privilege: DROP on the remote subscription Context Use the buttons in the Remote Subscription Monitor table to perform the action. Procedure 1. Select the remote subscription in the Remote Subscription Monitor table. 2. Click Queue, Distribute, Reset, or Drop. Note A warning appears if you attempt to drop a remote subscription that is used by any flowgraphs or replication tasks. Click Drop if you want to continue and drop the remote subscription anyway. Results The remote subscription is queued, distributed, or reset. If you drop a remote subscription, the subscription is removed from the Remote Subscription Monitor table. 68 2016 SAP SE or an SAP affiliate company. All rights reserved. Monitor and Operate

Related Information Assign Roles and Privileges [page 115] 6.3.5 Processing Remote Source or Remote Subscription Exceptions Remote subscription exceptions need to be evaluated and resolved. In the case of an error or if the row count on the target table does not match, for example, you will need to look at the Exceptions table and process the entries. Use the following syntax: PROCESS REMOTE SUBSCRIPTION EXCEPTION <exception_oid> IGNORE RETRY; To process a remote source or remote subscription exception using the monitoring UI: 1. Click the status of the remote source or remote subscription. 2. Select the error. 3. Click either Retry Operations or Ignore Error. 6.4 Monitoring Design Time Objects The Data Provisioning Design Time Object Monitor tile in the SAP HANA cockpit provides you with information about your design time objects, including flowgraphs and replication tasks. For example, you can see the duration of a task execution for a flowgraph and how many records have been processed. When you first open the Data Provisioning Design Time Object Monitor, it displays information for all design time objects. You can select a flowgraph or replication task to show information for only that object. You can also sort and hide individual columns by right-clicking a row and selecting your display preferences. Information Available on the Data Provisioning Design Time Object Monitor The Data Provisioning Task Monitor displays information in three tables: Design Time Objects, Task Monitor, and Remote Subscription Monitor. If you select a design time object in the first table, the other tables show information for only that object. Errors, warnings, and action execution information are displayed in the Status Console. Monitor and Operate 2016 SAP SE or an SAP affiliate company. All rights reserved. 69

Table 11: Information Available in Design Time Objects Table Column Package Object Name Object Type Schema Name Realtime Has Table Type Input Description Name of the package to which the design time object belongs. Name of the design time object. Type of design time object, either hdbreptask or hdbflowgraph. Name of the schema in which the object was created. 'TRUE' if the object is a realtime object, else 'FALSE'. 'TRUE' if the object is modeled with a table type as input. This means data would need to be passed (pushed) at execution time. Table 12: Information Available in Task Monitor Table Column Task Name Description Name of the data provisioning task. Tip You can click the name of a task to open that task in the Data Provisioning Task Monitor. Last Start Time Last Duration Day, date, and time when the last execution of the task started. Total elapsed time from start to end for COMPLETED or FAILED tasks. Current elapsed time for RUNNING tasks. Last Status Current status of the last execution of the task. The following values are possible: STARTING RUNNING COMPLETED FAILED CANCELLED (or, CANCELLING) Tip You can click the status to display additional information about the task execution. Last Processed Records Number of records that the last execution of the task has processed. 70 2016 SAP SE or an SAP affiliate company. All rights reserved. Monitor and Operate

Table 13: Information Available in Remote Subscription Monitor Table Column Subscription Name Description Name of the remote subscription. Tip You can click the name of a remote subscription to open that subscription in the Data Provisioning Remote Subscription Monitor. State Status Name of the state of the remote subscription. For more information, see Remote Subscription States for Data Provisioning [page 64]. Either 'ERROR' or 'SUCCESSFUL'. Tip You can click on the status of a remote subscription to process any exceptions for the subscription. For more information, see Processing Remote Source or Remote Subscription Exceptions [page 69]. Received Count Rejected Count Last Message Received Time Total number of messages received by the Data Provisioning server. Total number of messages rejected. Time the last message was received. 6.4.1 Execute Flowgraphs and Replication Tasks You can execute design time objects including flowgraphs and replication tasks from the Data Provisioning Design Time Object Monitor. Restriction Realtime flowgraphs and replication tasks cannot be executed from the Data Provisioning Design Time Object Monitor. Prerequisites The user must have the following roles or privileges to schedule flowgraphs and replication tasks. Monitor and Operate 2016 SAP SE or an SAP affiliate company. All rights reserved. 71

Table 14: Roles and Privileges Action Role or Privilege Execute flowgraph or replication task Role: sap.hana.im.dp.monitor.roles::operations Application privilege: sap.hana.im.dp.monitor::executedesigntimeobject Object privilege: EXECUTE on the object schema Object privilege: Any additional object privileges needed within the task (for example, ALTER, CREATE ANY, DELETE, DROP, EXECUTE, INDEX, INSERT, and so on.) Procedure 1. Select the flowgraph or replication task in the Design Time Objects table. 2. Click Execute. a. If the object uses table type parameters, select the tables to use when executing the object. b. If the object uses variable parameters, specify the values to use when executing the object. c. Click Execute. Results The object execution begins and the task appears in the Task Monitor table. Related Information Assign Roles and Privileges [page 115] 6.4.2 Schedule Flowgraphs and Replication Tasks You can schedule design time objects including flowgraphs and replication tasks within the Data Provisioning Design Time Object Monitor. Restriction Realtime flowgraphs and replication tasks cannot be scheduled from the Data Provisioning Design Time Object Monitor. 72 2016 SAP SE or an SAP affiliate company. All rights reserved. Monitor and Operate

Prerequisites The user must have the following roles or privileges to schedule flowgraphs and replication tasks. Table 15: Roles and Privileges Action Enable users to schedule design time objects Role or Privilege Role: sap.hana.xs.admin.roles::jobscheduleradministrator Schedule flowgraph or replication task Role: sap.hana.im.dp.monitor.roles::operations Application privilege: sap.hana.im.dp.monitor::scheduledesigntimeobject To activate task scheduling, the following must occur: Scheduling needs to be enabled via XS Job Admin Dashboard /sap/hana/xs/admin/jobs/ (The user that enables other users to schedule needs the role sap.hana.xs.admin.roles::jobscheduleradministrator). To schedule design time objects, the Job sap.hana.im.dp.monitor.jobs::scheduletask needs to be enabled in the XS Job Details page: /sap/hana/xs/admin/jobs/#/package/sap.hana.im.dp.monitor.jobs/job/ scheduletask. Procedure 1. Select the flowgraph or replication task in the Design Time Objects table. 2. Click the Schedules button. The Schedules dialog appears. 3. To create a new schedule for the task, click Add. a. Select the frequency (once or recurring), interval if recurring (year, month, week, day, hour, minute, second), and the time (local, not server time or UTC) for the object execution. b. If the object uses table type parameters, select the tables to use when executing the object. c. If the object uses variable parameters, specify the values to use when executing the object. d. Click Schedule. The new schedule is added to the list of schedules for the object. 4. To remove an existing schedule, select the schedule and click Delete. The schedule is removed from the list of schedules for the object. 5. Close the Schedules dialog. Results Your object executes as scheduled and you can monitor the results of each execution of the object. Monitor and Operate 2016 SAP SE or an SAP affiliate company. All rights reserved. 73

Related Information Assign Roles and Privileges [page 115] 6.4.3 Stop Non-Realtime Flowgraph Executions You can stop the execution of non-realtime flowgraphs within the Data Provisioning Design Time Object Monitor. Prerequisites The user must have the following roles or privileges to stop flowgraph execution. Table 16: Roles and Privileges Action Role or Privilege Stop flowgraph execution Role: sap.hana.im.dp.monitor.roles::operations Application privilege: sap.hana.im.dp.monitor::stoptask Procedure 1. Select the task for the flowgraph in the Task Monitor table. 2. Click Stop. Results The selected flowgraph execution instance is stopped. 6.5 Monitoring Data Provisioning Tasks The Data Provisioning Task Monitor tile in the SAP HANA cockpit provides you with information about your replication tasks and transformation tasks. For example, you can see the duration of a task and how many records have been processed. When you first open the Data Provisioning Task Monitor, it displays information for all tasks. You can select a task to show information for only that task. You can also sort and hide individual columns by right-clicking a row and selecting your display preferences. 74 2016 SAP SE or an SAP affiliate company. All rights reserved. Monitor and Operate

Information Available on the Data Provisioning Task Monitor The Data Provisioning Task Monitor displays information in three tables: Task Overview, Task Execution Monitor and Task Operation Execution Monitor. If you select a task in the first table, the other tables show information for only that task. Note By default, the Task Execution Monitor and Task Operation Execution Monitor tables are limited to 500 entries, and the limit for each table can be controlled individually. After selecting a different task, you may need to refresh the tables to see all corresponding information. Errors, warnings, and action execution information are displayed in the Status Console. Table 17: Information Available in Task Overview Table Column Task Type Task Name Task ID SQL Security Schema Name Procedure Schema Procedure Name Plan Version Owner Name Output Parameter Count Memory Size Is Valid Is Realtime Task Is Read-Only Input Parameter Count Has Table Type Input Has SDQ Description Type of task. Derived from task plan Name of the data provisioning task Unique identifier for a task Security model for the task, either 'DEFINER' or 'INVOKER' Name of the schema in which the task was created. If the task was created with a procedure instead of a plan, this attribute will contain the schema name of the stored procedure If the task was created with a procedure instead of a plan, this attribute will contain the name of the stored procedure Version of the task plan Owner of the task Number of output (tabletype) parameters Memory size of loaded task 'TRUE' if the task is in a valid state, 'FALSE' if it has been invalidated by a dependency 'TRUE' if the task is a realtime task, else 'FALSE' 'TRUE' if the task is read only (has only table type outputs), 'FALSE' if it writes to non-tabletype outputs Number of input (tabletype) parameters 'TRUE' if the task is modeled with a table type as input. This means data would need to be passed (pushed) at execution time 'TRUE' if the task contains SDQ (Smart Data ) functionality Monitor and Operate 2016 SAP SE or an SAP affiliate company. All rights reserved. 75

Column Design Time Type Design Time Name Description Type of corresponding Design Time Object. Values would be hdbflowgraph, hdbreptask, or empty. Name of corresponding Design Time Object. Tip You can click the name of a design time oject to open that object in the Data Provisioning Design Time Object Monitor. Create Time Comments The time that the task was created. Description of the task, from the task plan Table 18: Information Available in Task Execution Monitor Table Column Task Name Schema Name Host Port Task Execution ID Start Time End Time Duration Description Name of the Data Provisioning task. Name of the schema in which the task was created. Name of the host on which the task is running. Port number that the task uses to communicate with the Data Provisioning Server. Unique identifier for the task. Day, date, and time when the task started. Day, date, and time when the task ended. Total elapsed time from start to end for COMPLETED or FAILED tasks. Current elapsed time for RUNNING tasks. Status Current status of the task. The following values are possible: STARTING RUNNING COMPLETED FAILED CANCELLED (or, CANCELLING) Tip You can click the status to display additional information about the task execution. Total Progress Processed Records Percentage completed. Number of records that the task has processed. 76 2016 SAP SE or an SAP affiliate company. All rights reserved. Monitor and Operate

Column Async Description Indicates whether or not the task is running as a background task. Possible values are: TRUE FALSE Parameters Hana User Application Parameters passed to the task. Name of the user that started the execution of this task. Application user set in the session context. Tip Use the Number of rows menu to limit the number of task executions displayed at one time in the Task Execution Monitor table. Table 19: Information Available in Task Operation Execution Monitor Table Column Operation Description Name of the operation that the task is currently performing. For a transformation task, the name of the operation is the name of the node that appears in the flowgraph. Schema Name Task Name Task Execution ID Host Port Operation Type Start Time End Time Duration Status Name of the schema (user name) in the remote source. Name of the task. Identification number of the task. Name of the host on which the task is running. Port number that the task uses to communicate with the Data Provisioning Server. Current type of operation. For example, the operation type can be Table Writer, Adapter, Projection, and so forth. Day, date, and time when the task started. Day, date, and time when the task ended. Total elapsed time from start to end. Current status of the task. The following values are possible: STARTING RUNNING COMPLETED FAILED Progress Processed Records Percentage completed. Number of records that the task has processed. Monitor and Operate 2016 SAP SE or an SAP affiliate company. All rights reserved. 77

Column Side Effects Description Indicates whether or not this operation generates side effect statistics. Possible values are: TRUE FALSE Tip Use the Number of rows menu to limit the number of operations displayed at one time in the Task Execution Operations Monitor table. 6.5.1 Start and Stop Data Provisioning Tasks You can start and stop tasks within the Data Provisioning Task Monitor. Prerequisites The user must have the following privileges to start or stop tasks: Table 20: Privileges Action Start task Stop task Privilege sap.hana.im.dp.monitor::starttask sap.hana.im.dp.monitor::stoptask Procedure To start a task, select a task in the Task Overview table. a. Click Start. b. If the object uses table type parameters, select the tables to use when executing the object. c. If the object uses variable parameters, specify the values to use when executing the object. Note Tasks that belong to realtime design time objects cannot be started or scheduled from the Data Provisioning Task Monitor. To stop a task, select the running task in the Task Execution Monitor table and click Stop. 78 2016 SAP SE or an SAP affiliate company. All rights reserved. Monitor and Operate

Related Information Assign Roles and Privileges [page 115] 6.5.2 Schedule Data Provisioning Tasks You can schedule tasks within the Data Provisioning Task Monitor. Prerequisites The user must have the following roles or privileges to schedule tasks. Table 21: Roles and Privileges Action Enable users to schedule task Role or Privilege Role: sap.hana.xs.admin.roles::jobscheduleradministrator Schedule task Role: sap.hana.im.dp.monitor.roles::operations Application privilege: sap.hana.im.dp.monitor::scheduletask To activate task scheduling, the following must occur: Scheduling needs to be enabled via XS Job Admin Dashboard /sap/hana/xs/admin/jobs/ (The user that enables other users to schedule needs the role sap.hana.xs.admin.roles::jobscheduleradministrator). To schedule tasks, the Job sap.hana.im.dp.monitor.jobs::scheduletask needs to be enabled in the XS Job Details page: /sap/hana/xs/admin/jobs/#/package/sap.hana.im.dp.monitor.jobs/job/scheduletask. Procedure 1. Select the task in the Task Monitor. 2. Click the Schedules button. The Schedules dialog appears. 3. To create a new schedule for the task, click Add. a. Select the frequency (once or recurring), interval if recurring (year, month, week, day, hour, minute), and the time (local, not server time or UTC) for the task execution. b. If the task uses table type parameters, select the tables to use when executing the task. c. If the task uses variable parameters, specify the values to use when executing the task. d. Click Schedule. The new schedule is added to the list of schedules for the task. 4. To remove an existing schedule, select the schedule and click Delete. The schedule is removed from the list of schedules for the task. Monitor and Operate 2016 SAP SE or an SAP affiliate company. All rights reserved. 79

5. Close the Schedules dialog. Results Your task executes as scheduled and you can monitor the results of each execution of the task. Related Information Assign Roles and Privileges [page 115] 6.5.3 Change Retention Period for Data Provisioning Task Monitor Change the retention period for Data Provisioning statistics tables if you want to retain the data in the Task Execution Monitor and Task Operation Execution Monitor for longer than five minutes. Context The following parameters specify how long to keep the statistics data and when to delete them: The task_data_retention_period parameter specifies the period of time the data is kept in the statistics tables. This period is calculated from the time the task reached the COMPLETED, FAILED, or CANCELLED status. The default value is -1. A value of -1, means never delete the data. The task_data_retention_period_check_interval parameter specifies how often the data is actually deleted by the garbage collection thread. The default value is 300 seconds (five minutes). To change the default values of these parameters, you must add a new section named task_framework to each of the indexserver.ini, scriptserver.ini, and the xsengin.ini files. Note If you need to change these options you can do so by running the following: ALTER SYSTEM ALTER CONFIGURATION ('<server type>.ini', 'SYSTEM') SET ('task_framework', ' task_data_retention_period_check_interval )='<in secs>' WITH RECONFIGURE; ALTER SYSTEM ALTER CONFIGURATION ('<server type>.ini', 'SYSTEM') SET ('task_framework', ' task_data_retention_period )='<in secs>' WITH RECONFIGURE; 80 2016 SAP SE or an SAP affiliate company. All rights reserved. Monitor and Operate

Procedure 1. Login to SAP HANA studio as SYSTEM user. 2. In the Systems view, right-click the name of your SAP HANA server and choose Configuration and Monitoring Open Administration. 3. Click the Configuration tab. 4. Select indexserver.ini, right-click and choose Add Section. 5. On the Section Name screen of the Add Section Wizard, enter task_framework for the section name, and click Next. 6. On the Scope Selection screen, select System from the drop-down list of Assign Values to, and click Next. 7. On the Key Value Pairs screen, enter task_data_retention_period in the Key field, and enter the number of seconds you want the statistics data to be kept in the statistics tables. Note If you set the retention period to 0 (zero) or less than zero, statistics data will not be deleted. If you set the retention period to a negative number, the default value (-1) will be used. 8. Click Finish. 9. Select indexserver.ini, right-click and choose Add Parameter. 10. On the Add New Parameter screen, enter task_data_retention_period_check_interval in the Key field, and enter the time interval for the Task Data Cleanup Process to run. Note If you set the task_data_retention_period_check_interval to less than 60 seconds, the default value (five minutes) will be used. 11. Select scriptserver.ini and do the following substeps. a. Right-click and choose Add Section. b. Repeat steps 5 through 10 for scriptserver.ini. 12. If your tasks will be called by the XS application, select xsengine.ini and do the following substeps. a. Right-click and choose Add Section. b. Repeat steps 5 through 10 for xsengine.ini. 6.5.4 Troubleshooting Data Provisioning Tasks This section describes error situations related to replication tasks and transformation tasks and their solutions. Monitor and Operate 2016 SAP SE or an SAP affiliate company. All rights reserved. 81

6.5.4.1 Activate Additional Trace Logging for the Data Provisioning Server The trace logs contain detailed information about actions in the Data Provisioning server.when you are troubleshooting issues, you can gain additional insight about the root cause by increasing the level of detail in the logs. Prerequisites To configure traces, you must have the system privilege TRACE ADMIN. Procedure 1. Log on to SAP HANA studio with a user that has system privilege TRACE ADMIN. 2. In the Administration editor, choose the Trace Configuration tab. 3. Choose Edit Configuration under Database Trace. 4. Filter the component list for dp*. 5. Set the System Trace Level to DEBUG or for any relevant components under the DPSERVER and INDEXSERVER nodes. Scenario Traces Debugging Initial Load DPSERVER dpframework Trace information for message handling between the index server, Data Provisioning Server, and Data Provisioning Agent. dpframeworkprefetch Prefetch-specific trace information. dpframeworkmessagebody Trace information useful for diagnosing incorrect numbers of rows or suspected data corruption. Note This option can generate a large amount of data in the trace log. INDEXSERVER dpadaptermanager Trace information for initial smart data access requests. XSENGINE dpadaptermanager Trace information for initial smart data access requests from the SAP HANA Web-based Development Workbench. Debugging Real-time Replication DPSERVER 82 2016 SAP SE or an SAP affiliate company. All rights reserved. Monitor and Operate

Scenario Traces dpframework Trace information for message handling between the index server, Data Provisioning Server, and Data Provisioning Agent. dpreceiver Trace information for message receiver, including the hand off of row data received by the framework. dpdistributor Trace information for the message distributor. dpapplier Trace information for the message applier. dpserver Trace information for communication between the Data Provisioning Server and the index server. dpremotesubscriptionmanager Trace information for remote subsciption runtime details on the Data Provisioning Server side. dpframeworkmessagebody Trace information useful for diagnosing incorrect numbers of rows or suspected data corruption. Note This option can generate a large amount of data in the trace log. INDEXSERVER dpdistributor Trace information for the message distributor. dpapplier Trace information for the message applier. dpremotesubscriptionmanager Trace information for remote subscription runtime details on the index server side. Tip To ensure that applicable trace information is captured at the time of the error and not overwritten by log wrapping, you can increase the maximum log size while capturing the traces. In the GLOBAL node, increase the value of the maxfilesize parameter. 6. Click Finish. Results Additional debug information for the modified components is added to the trace log. Tip You can also modify the trace log levels by issuing commands from a SQL console and manually specifying the component to adjust. Monitor and Operate 2016 SAP SE or an SAP affiliate company. All rights reserved. 83

For example, to set the Data Provisioning Server dpframework trace to DEBUG: ALTER SYSTEM ALTER CONFIGURATION ('dpserver.ini','system') SET ('trace','dpframework') = 'DEBUG' WITH RECONFIGURE; 6.5.4.2 Replication Has Stopped Replication may stop due to a variety of problems. Generally, you can start to resolve the issue and resume replication by resetting the remote subscription. Reset the Remote Subscription [page 84] When log reader-based real-time replication has stopped, you can try resetting the remote subscription. Clear Remote Subscription Exceptions [page 86] Replication may be stopped due to remote subscription exceptions. For example, a remote subscription exception may be generated when a primary key violation, constraint violation, or null primary key occurs. Check Log Reader Errors [page 86] Replication may stop if there are log reader errors. Source Database Cannot be Reached [page 86] Replication may stop when the source database cannot be reached due to a network or hardware issue. Oracle Replication is Stopped or Delayed [page 87] If replication from an Oracle source system is stopped or delayed, or you notice poor performance, check the instance log for reports of unsupported transactions. 6.5.4.2.1 Reset the Remote Subscription When log reader-based real-time replication has stopped, you can try resetting the remote subscription. Procedure 1. From a SQL console, reset the remote subscription. ALTER REMOTE SUBSCRIPTION <subscription_name> RESET; Tip If you have the required privileges, you can also reset remote subscriptions from the Data Provisioning Remote Subscription Monitor. For more information, see Manage Remote Subscriptions. 2. Check the Data Provisioning Agent for the adapter instance. In <DPAgent_root>/LogReader, look for a directory with the same name as your remote subscription instance. Back up and delete the folder, if it exists. 84 2016 SAP SE or an SAP affiliate company. All rights reserved. Monitor and Operate

3. Execute the cleanup script for your database type. The cleanup scripts are located in <DPAgent_root>/LogReader/scripts. Note Execute the script using the same user that is configured for replication. 4. Stop the SAP HANA Data Provisioning Agent service. On Windows, use the Services manager in Control Panel. On Linux, run./dpagent_service.sh stop. 5. (Optional) Enable additional logging on the Data Provisioning Agent. a. Open the Agent configuration file in a text editor. The configuration file is located at <DPAgent_root>/dpagentconfig.ini. b. Change the parameter framework.log.level from INFO to ALL. Increasing the log level generates additional information useful for further debugging, if necessary. You can safely revert the log level after resolving the issue. 6. Restart the SAP HANA Data Provisioning Agent service. On Windows, use the Services manager in Control Panel. On Linux, run./dpagent_service.sh start. 7. From a SQL console, queue and distribute the remote subscription. ALTER REMOTE SUBSCRIPTION <subscription_name> QUEUE ALTER REMOTE SUBSCRIPTION <subscription_name> DISTRIBUTE Tip If you have the required privileges, you can also queue and distribute remote subscriptions from the Data Provisioning Remote Subscription Monitor. For more information, see Manage Remote Subscriptions. Next Steps If you are unable to reset the remote subscription, you may need to clear any outstanding remote subscription exceptions first. Related Information Clear Remote Subscription Exceptions [page 86] Manage Remote Subscriptions [page 68] Monitor and Operate 2016 SAP SE or an SAP affiliate company. All rights reserved. 85

6.5.4.2.2 Clear Remote Subscription Exceptions Replication may be stopped due to remote subscription exceptions. For example, a remote subscription exception may be generated when a primary key violation, constraint violation, or null primary key occurs. Remote subscription exceptions are reported in multiple ways. Remote subscription exceptions appear in the task execution log. Email alerts may be sent to the administrator. Exceptions may be displayed in monitoring, or if you query the remote subscription exception public view. After correcting the root cause of the remote subscription exception, you can clear the exceptions with a SQL statement: PROCESS REMOTE SUBSCRIPTION EXCEPTION <subscription_id> IGNORE 6.5.4.2.3 Check Log Reader Errors Replication may stop if there are log reader errors. For more information about log reader errors, check the trace file located in the log folder of your Data Provisioning Agent instance. You may also choose to increase the trace log levels for the Data Provisioning Server. Related Information Activate Additional Trace Logging for the Data Provisioning Server [page 82] 6.5.4.2.4 Source Database Cannot be Reached Replication may stop when the source database cannot be reached due to a network or hardware issue. An unreachable source database can be detected by multiple error types in the logs. For example: Could not connect to <jdbc:oracle:thin:@host:1675:ddcstd>: The Network Adapter could not establish the connection java.lang.exception: Log scanner <LogMinerScanner_1> stopped because of error: No more data to read from socket When the source database cannot be reached, you should suspend capture for any affected remote source subscriptions. After the issue has been resolved, resume capture for the remote source subscriptions. 86 2016 SAP SE or an SAP affiliate company. All rights reserved. Monitor and Operate

6.5.4.2.5 Oracle Replication is Stopped or Delayed If replication from an Oracle source system is stopped or delayed, or you notice poor performance, check the instance log for reports of unsupported transactions. Context When the Oracle LogMiner starts scanning from the middle of a transaction and fails to translate the raw record, it reports an unsupported operation. This may occur most often on UPDATE operations involving wide tables. The Oracle Log Reader adapter can manage these records by using a standby scanner, but frequent occurrence of unsupported operations can slow scan performance. You have several options to reduce the number of unsupported operations. Procedure Upgrade to a newer version of the Data Provisioning Agent, if possible. If parallel scanning is enabled, increase the value of the Parallel scan SCN range parameter. Disable parallel scanning. Related Information Data Provisioning Agent Log Files and Scripts [page 127] 6.5.4.3 Replication Agent Cannot be Initialized The replication agent may fail to be initialized for multiple reasons. The steps required to correct the issue depend on the nature of the error message reported, as well as the type of the source database involved. Monitor and Operate 2016 SAP SE or an SAP affiliate company. All rights reserved. 87

6.5.4.3.1 DB2 Native Connection Failed When the DB2 native connection fails, the replication agent may fail to be initialized. When the native connection fails, you may see the following error in the log: 2016-01-13 18:44:44,147 [ERROR] com.sap.hana.dp.db2logreaderadapter.db2repagentwrapper.initialize[858] - Failed to initialize RepAgent. Could not find Resource Bundle containing index: Could not get the log end locator because: Native database connection failed with code <-1>. Verify that the connection details for your DB2 database are configured correctly: Host Port Database Name Database Source Name 6.5.4.3.2 Microsoft SQL Server Log is Full The replication agent may fail to initialize when the Microsoft SQL Server log cannot be accessed. When the Microsoft SQL Server log file cannot be accessed, the following error is reported: com.microsoft.sqlserver.jdbc.sqlserverexception: The transaction log for database 'BUDGET_LCL_DTM_HANA' is full due to 'ACTIVE_TRANSACTION'. Verify that the auto-extend option for the Microsoft SQL Server log file is enabled, and that the disk where the log file is located has available free space. 6.5.4.3.3 Microsoft SQL Server Version is Unsupported The replication agent cannot be initialized with unsupported versions of Microsoft SQL Server. When the replication agent attempts to initialize on an unsupported version, the following error is reported in the framework log: Work is being rolled back because of problems executing script: This edition of SQL Server does not support publications. To correct the error, use a supported Enterprise Edition of Microsoft SQL Server. Other versions such as Microsoft SQL Server Express do not support transaction replication. 88 2016 SAP SE or an SAP affiliate company. All rights reserved. Monitor and Operate

6.6 Create Status Notifications Create email notifications for various task, remote subscription, and design time object statuses. Prerequisites The user must have the following roles or privileges to create status notifications: Table 22: Roles and Privileges Action Role or Privilege Enable users to schedule task Role: sap.hana.xs.admin.roles::jobscheduleradministrator Configure SMTP Role: sap.hana.xs.admin.roles::smtpdestadministrator Create status notifications Role: sap.hana.im.dp.monitor.roles::operations Application privilege: sap.hana.im.dp.monitor::notificationadministration To activate status notifications, the following must occur: Scheduling must be enabled in the XS Job Admin Dashboard at http://<host>:<port>/sap/hana/xs/ admin/jobs. The user that enables other users to schedule needs the role sap.hana.xs.admin.roles::jobscheduleradministrator. To create notifications, the Job sap.hana.im.dp.monitor.jobs/checknotifications must be enabled in the XS Job Details page: http://<host>:<port>/sap/hana/xs/admin/jobs/#/package/ sap.hana.im.dp.monitor.jobs/job/checknotifications. A sender address must be configured in the monitor. If no sender address is configured, notification emails will not be sent. To configure the sender address, click the Set sender email address button in the monitor. The SAP HANA SMTP mail client must be correctly configured in the SMTP Configurations page at http:// <host>:<port>/sap/hana/xs/admin/index.html#smtp. The user that configures the SMTP client needs the role sap.hana.xs.admin.roles::smtpdestadministrator Context You can create notifications for the following statuses: Monitor and Operate 2016 SAP SE or an SAP affiliate company. All rights reserved. 89

Table 23: Supported Statuses Object Type Supported Statuses Task execution COMPLETED FAILED CANCELLED Remote subscription ERROR WARNING Design time object COMPLETED FAILED CANCELLED ERROR WARNING Procedure 1. In the Task Overview, Remote Subscription Monitor, or Design Time Objects table, select the object for which you want to create a notification. 2. Click the Notifications button. The list of notifications for the object is displayed. 3. Click the Add button to create a new notification. 4. Specify a name, status conditions, and recipient email addresses for the notification. 5. If you want to enable the notification immediately, select Is active. 6. Click Create Notification. The new notification is added to the list of notifications for the object. When the conditions for the notification are met, users in the recipient list are sent an email containing details about the event that triggered the notification. Related Information Assign Roles and Privileges [page 115] 90 2016 SAP SE or an SAP affiliate company. All rights reserved. Monitor and Operate

6.7 Create Monitoring Alerts Create monitoring alerts for various functions. Context You can receive alerts for the following functions: Agent availability Agent memory usage Remote subscription exception Data reference data Long-running tasks The process for creating alerts is the same for each of these alerts in HANA studio. Note You can also configure alerts in the SAP HANA Cockpit by adding and clicking the Configure Alerts tile. Procedure 1. In HANA studio, click the Alerts tab, and click the Configure button. 2. Configure the sender and recipient information. 3. Select the Alerts for which you want to receive email notifications. 4. Click OK. 6.8 Monitoring and Troubleshooting Enterprise Semantic Services Monitor Enterprise Semantic Services publications and the Entity Grid to ensure that the semantic search for publication artifacts or run-time objects works smoothly. Troubleshooting actions are available on the Enterprise Semantic Services monitors. The SAP HANA Enterprise Semantic Services Administration user interface is a browser-based application that lets you manage artifacts for semantic services. To launch the SAP HANA Enterprise Semantic Services Administration tool, enter the following URL in a web browser: http://<<your_hana_instance:port>>/sap/hana/im/ess/ui Monitor and Operate 2016 SAP SE or an SAP affiliate company. All rights reserved. 91

The interface includes the following components (tiles): Publish and Scheduling Monitor Publish and unpublish artifacts. Schedule publishing and data profiling requests. Request Monitor View the details and status of all requests. Entity Grid Monitor View and remove artifacts from the Entity Grid. Data Profiling Blacklist Prevent data profiling for selected artifacts. Once you have drilled into a component, you can click the navigation menu in the upper-left corner to open other components or return to the Home page. 6.8.1 Privileges for Administering Enterprise Semantic Services Verify application access, roles, and privileges for Enterprise Semantic Services (ESS) before contacting customer support. Ensure you have access to the SAP HANA XS Administration console and the SAP HANA studio. The following privileges are also necessary: Description To use the SAP HANA ESS Administration tool To grant missing ESS roles To use SAP HANA lifecycle management To administer ESS jobs Required privileges sap.hana.im.ess.roles::administrator sap.hana.im.ess:: Administrator Execute on object privilege: GRANT_ACTIVATED_ROLE sap.hana.xs.lm.roles::administrator. sap.hana.xs.admin.role::jobadministrator sap.hana.xs.admin.role::jobscheduleradministrator To enable the job scheduler (change.ini file) To grant object privileges on catalog objects INFILE ADMIN DATABASE ADMIN (multi-tenant) ROLE ADMIN 92 2016 SAP SE or an SAP affiliate company. All rights reserved. Monitor and Operate

Description Installation troubleshooting Required privileges System privilege: USER ADMIN Object privilege EXECUTE on: _SYS_REPO. GRANT_SCHEMA_PRIVILEGE_ON_ACTIVATED_CONTENT _SYS_REPO. REVOKE_SCHEMA_PRIVILEGE_ON_ACTIVATED_CONTENT _SYS_REPO. GRANT_ACTIVATED_ROLE _SYS_REPO. REVOKE_ACTIVATED_ROLE 6.8.2 Enterprise Semantic Services Publications and Entity Grid An Enterprise Semantic Services publication request extracts information from a publication artifact (which is either a catalog object or a remote object) and stores it in an Entity Grid that is later searched when a user tries to find catalog or remote objects based on their metadata and contents. Related Information Monitor the Status of Publishing Requests [page 96] 6.8.3 Publishing and Profiling Artifacts The SAP Enterprise Semantic Services (ESS) Administration tool lets you publish (or unpublish) artifacts. You can publish or unpublish an artifact programmatically using the on-demand Enterprise Semantic Services API, which requires a loose coupling of an application with Enterprise Semantic Services. This mode is well suited for applications that manage the life cycle of their artifacts, that is, applications that create, delete, and update SAP HANA artifacts. This way, the application determines which artifacts must be published, republished, or unpublished to Enterprise Semantic Services. A typical example is the SAP Agile Data Preparation application for artifacts that are created by business users. Administrators can also use the SAP HANA ESS Administration tool to publish or unpublish artifacts. This is well suited for applications that manage the life cycle of SAP HANA artifacts but do not want (or cannot easily) couple their management of artifacts with Enterprise Semantic Services. A typical example is the SAP ERP application. In those cases, it is easier to delegate to an administrator the task of determining which artifacts should be published to Enterprise Semantic Services depending on the needs of applications that require access to semantic services like search. Monitor and Operate 2016 SAP SE or an SAP affiliate company. All rights reserved. 93

A best practice is to separate the artifacts published by applications using the on-demand ESS API from those published by an administrator using the SAP ESS Administration tool. These artifacts thereby belong to different publisher groups, as depicted in the SAP ESS Administration tool Entity Grid Monitor. Related Information Monitor the Enterprise Semantic Services Entity Grid [page 100] 6.8.3.1 Publish Artifacts How to use the SAP HANA ESS Administration tool to publish artifacts in the Entity Grid. Procedure 1. Select the Publish and Scheduling Monitor tile. 2. In the Publishable Artifact Browser, expand the nodes and select an artifact. 3. To include the artifact, select Include for publication. Children of the artifact inherit the INCLUDED configuration of the parent unless specifically excluded. 4. Configure the publication schedule as follows. a. For Next Scheduled Date, click in the box to select a date and time to next publish the artifact. If you do not enter a date-time, it is set to the current day and time. b. Enter a frequency Period. 5. Configure Profiling Options. a. Select Discover content type to include content types in the publication; however, this can impact performance and is suggested for production scenarios. b. Select Extract searchable values to extract values; however, this can impact performance and is suggested for production scenarios. 6. Select Active to enable the schedule. To deactivate a schedule, clear the check box and click Save. 7. Select Save. The schedule displays in the Schedules table. 8. To stop (cancel) the publication of an artifact: open the Request Monitor, refresh the view, select the Stop check box for the artifact(s), and select the Stop icon in the upper right corner of the window. 9. Optionally, from the navigation menu, open the Request Monitor to confirm the status of the request (for example, REQUEST_PENDING or DONE). 10. To exclude objects, select the object, select Exclude for publication, and select Save. 94 2016 SAP SE or an SAP affiliate company. All rights reserved. Monitor and Operate

Results The objects display in the browser tree marked with a solid green plus symbol (for included objects) or a solid red minus symbol (for excluded objects). Inherited objects display an outlined green or red symbol. These markers indicate that a request action has been initiated but is independent of the actual status of the request. To view the status of the request, open the Request Monitor. To view the results of requests, view the Entity Grid Monitor. 6.8.3.2 Information Available on the Schedules Monitor The Publish and Scheduling Monitor displays a table that describes all of the scheduled publication and data profiling requests. Column Name Description Filterable Publication Artifact Fully qualified name of the published artifact Yes Publication The mode selected for the schedule (INCLUDE, EXCLUDE, INHERIT) Yes Next Scheduled Date The timestamp for when the schedule will next execute Yes Period The selected frequency of the schedule Yes Active Whether or not the schedule has been activated Yes Discover Whether or not the option for Discover content type was selected Yes Extracted Searchable Values Whether or not the option for Extract searchable values was selected Yes Warning A warning indicates that a scheduled artifact has been deleted. The Delete option will be enabled on the schedule. Yes Delete To delete a schedule for an artifact that has been deleted, select the check box and click Delete. No 6.8.3.3 Remove a Schedule for a Deleted Artifact Remove a publication schedule for an artifact that was deleted. Context When an artifact is configured as included for publication, a publication schedule displays in the table of schedules. If the artifact gets deleted, the publication schedule will remain until the crawler cannot detect it. A warning will then display for that schedule. Monitor and Operate 2016 SAP SE or an SAP affiliate company. All rights reserved. 95

Procedure 1. Select the Publish and Scheduling Monitor tile. 2. For the artifact that displays a warning, the Delete option will be enabled on that schedule. Select the check box and click Delete above the column. 6.8.4 Monitor the Status of Publishing Requests Use the Request Monitor to view and monitor publishing and profiling requests. Context A user can only search for catalog or remote objects that are described in the Entity Grid as a result of successful publication requests. However, if the name of an artifact unexpectedly does not appear in the search results, the publication of the corresponding artifact might have failed. To monitor the status of all requests, Select the Request Monitor tile. Related Information Information Available on the Request Monitor [page 97] 96 2016 SAP SE or an SAP affiliate company. All rights reserved. Monitor and Operate

6.8.4.1 Information Available on the Request Monitor The Enterprise Semantic Services Request Monitor displays the status and any error messages for each request. Column Name Description Filterable Detail Click the magnifying glass icon to drill into the request for more details. The Detail page displays the following information and statistics: No Request type (see list below) Submission timestamp Request status (see list below) Publisher (user) Publisher Group Number of basic artifacts to publish Number of basic artifacts to unpublish Number of basic artifacts to profile For the latter three statistics, you can see the associated number of requests that are Successful, Failed, In progress, or Not started. The table displays each Publication Artifact and its Publication Status, Data Profiling Status, Error Code, and Error Message if any. Requests with a FAILED status include a Retry check box. To retry the request, select the check box and click Retry at the top of the window. To retry all FAILED requests, select the Retry check box in the column heading. ID A number that helps identify a request in the list of requests. Note that this number might not be unique in some cases. Yes Request Date Timestamp of when the request was executed Yes Publication Artifact Fully qualified name of catalog object or repository object that was published Yes Artifact Type The type of artifact to publish: Yes SAP HANA views in the repository: attributeview, analyticview, calculationview SAP HANA catalog objects: table, view, columnview, virtualtable Publisher Group Indicates whether a publication was scheduled using the SAP HANA ESS Administration tool or using the REST API. In the former case, the predefined publisher group is sap.hana.im.ess.adminpublishergroup. In the latter case, a call to the publish() API must specify a publishergroup parameter that defines the ownership of the specified publication in the Entity Grid. Yes Publisher Name of the SAP HANA user who submitted the request Yes Monitor and Operate 2016 SAP SE or an SAP affiliate company. All rights reserved. 97

Column Name Description Filterable Request Type Request types on the Request Monitor home page include: ON_DEMAND_PUBLISH ON_DEMAND_UNPUBLISH SCHEDULED_PUBLISH MONITORING_UNPUBLISH RETRY_ON_DEMAND_PUBLISH RETRY_ON_DEMAND_UNPUBLISH RETRY_SCHEDULED_PUBLISH Request types on the artifact Detail page include following: PUBLISH_NOT_STARTED UNPUBLISH_NOT_STARTED UNPUBLISH_IN_PROGRESS PUBLISH_DONE PUBLISH_FAILED UNPUBLISH_FAILED Yes 98 2016 SAP SE or an SAP affiliate company. All rights reserved. Monitor and Operate

Column Name Description Filterable Status Status values on the Request Monitor home page include: REQUEST_PENDING IN_PROGRESS DONE DONE_WITH_ERRORS NOTHING_DONE STOPPING STOPPED Status values on the artifact Detail page include: PROFILING_NOT_STARTED PROFILING_IN_PROGRESS PROFILING_DONE PROFILING_FAILED INACTIVATED NOT PROFILABLE BLACKLISTED OBSOLETE PUBLICATION_FAILED STOPPED Yes Error Code Error code when the status is FAILED. Each range of numbers indicates a specific area as follows: Yes 100-199: SAP HANA adapter errors 200-399: Prepare, extract, load, and deploy jobs 700-799: Miscellaneous Error Message Description of the error No Retry To retry one or more requests, select the Retry check box for each, or select the Retry check box in the column heading to select all failed requests, and select the Retry button. No Monitor and Operate 2016 SAP SE or an SAP affiliate company. All rights reserved. 99

Column Name Description Filterable Stop To display requests that are currently in progress, select the Refresh icon to update the Status column. To move the latest publication requests to the top of the list, for Request Date select Sort Descending. To stop (cancel) one or more in-progress requests, select the Stop check box for each, or select the Stop check box in the column heading to select all requests, and select the Stop button. No 6.8.5 Monitor the Enterprise Semantic Services Entity Grid Use the Entity Grid to view the artifacts that have been published to the grid and to remove (unpublish) artifacts from the grid. Context The Entity Grid is a knowledge graph that describes the semantics of published artifacts (datasets). Metadata crawlers and data profiling requests let you publish artifacts to the Entity Grid. Thereby, applications such as SAP Agile Data Preparation or SAP HANA smart data integration can search for and locate these objects and their metadata. There are two ways to publish artifacts to the Entity Grid: The HTTP REST API publish() method used by applications such as SAP HANA ADP, and the SAP HANA Enterprise Semantic Services Administration tool. If the same artifact gets published by both mechanisms, the artifact is identified in the Entity Grid Monitor as belonging to a corresponding publisher group. Therefore, publisher groups define ownership of specific publications in the Entity Grid. When an artifact is published with a specific publisher group, it can only be unpublished by that group. If the same artifact has been published with multiple publisher groups, it can only unpublished when all corresponding publisher groups unpublish it. This control helps avoid conflicts between applications and an administrator using the Administration tool. Otherwise, an application could publish an artifact and another application or administrator could unpublish it. In the case of the HTTP publish() API, the publisher group name is specific to the application; for example for SAP HANA ADP, it could be com.sap.hana.im.adp. For the SAP HANA ESS Administration tool, the predefined publisher group name is sap.hana.im.ess.adminpublishergroup. To limit the size of both extracted metadata elements and extracted searchable attribute values in Entity Grid, you can also select artifacts to unpublish. Procedure 1. Select the Entity Grid Monitor tile. 100 2016 SAP SE or an SAP affiliate company. All rights reserved. Monitor and Operate

2. Expand the nodes on the Entity Grid artifact browser to find the object to monitor and select it. 3. Select the Publisher Group as necessary. 4. The table displays all of the published artifacts, when they were last refreshed, and the number of metadata elements in each. 5. To remove an artifact and its data profiling information, select its Unpublish check box and click Save. To unpublish all displayed artifacts, select the Unpublish check box in the column heading and click Save. View the Request Monitor to confirm that the object was removed. For example, the Request Type would indicate MONITORING_UNPUBLISH. Related Information Information Available on the Request Monitor [page 97] 6.8.5.1 Information Available on the Entity Grid Monitor The Enterprise Semantic Services Entity Grid Monitor displays artifacts that have been published and also lets you remove (unpublish) artifacts from the Entity Grid. The Entity Grid Artifact browser displays all the published objects available in the Catalog, Content, and Remote Source folders. The Entity Grid size of an artifact is measured as the total number of searchable metadata elements and searchable attribute values extracted from that artifact. Expand the nodes, select the object to view, and select the Application Scope. To view all published artifacts, select the scope <All_SCOPE>. Name Description Filterable Publisher Group Indicates whether a publication was scheduled using the SAP HANA ESS Administration tool or using the REST HTTP publish() API. When an artifact is published with a specific publisher group, the artifact can only be unpublished by the same group. If a same artifact has been published with different publisher groups, the artifact will be unpublished when all associated groups have unpublished it. For the SAP HANA ESS Administration tool, the predefined publisher group name is sap.hana.im.ess.adminpublishergroup. For the REST HTTP publish() API, a call to the publish() API must specify a publishergroup parameter that determines the name of the publisher group. Publication Artifact name Name of the selected artifact. Monitor and Operate 2016 SAP SE or an SAP affiliate company. All rights reserved. 101

Name Description Filterable Number of published artifacts Number of basic artifacts recursively contained in the selected artifact when the selected artifact is a container. If the selected artifact is a basic artifact, the number of published artifacts is equal to 1. Number of metadata elements Total number of extracted metadata elements in the selected artifact. Number of extracted values Total number of attribute values extracted in the selected artifact. Publication Artifact Qualified name of an artifact that was published or contains published artifacts. The fully qualified name is described by three attributes: Yes Wildcards Origin: Catalog or Content Container: Schema name, package path, or virtual container name Artifact: Name of the artifact (basic or container) in its parent container Oldest Refresh Oldest date of updated basic artifacts in the corresponding container. This date is NULL in the case of a basic artifact. Yes Wildcards Last Refresh Most recent date of updated basic artifacts in the corresponding container. This date is the last update in the case of a basic artifact. Yes Basic Artifacts Number of published basic artifacts recursively contained in the corresponding container. This value is 1 in the case of a basic artifact. Yes Metadata Elements Number of non-shared metadata elements. It indicates the number of searchable metadata elements extracted from the corresponding published artifacts that are not shared with other published artifacts. This number gives an indication of how many metadata elements would be removed if you unpublished the artifact. Yes Extracted Values Number of searchable attribute values extracted for the catalog object represented by the published artifact. It indicates the number of metadata elements that are not shared with other published artifacts. This number gives an indication of how many profiled values would be removed in the case of unpublishing. Yes Unpublish To unpublish the artifact, select the Unpublish check box and click Save. To unpublish all displayed artifacts, select the Unpublish check box in the column heading and click Save. No 102 2016 SAP SE or an SAP affiliate company. All rights reserved. Monitor and Operate

6.8.6 Data Profiling Enterprise Semantic Services can profile the contents of artifacts that have been published to the Entity Grid. Data profiling is a process that analyzes the values contained in specific columns of a dataset (the columns to analyze are specified internally using ESS logic). Analysis of the data in a column discovers business types, and searchable values can then be extracted and indexed using SAP HANA full text index. Enterprise Semantic Services can profile the contents of the following artifacts: SQL tables Column views issued from graphical Calculation views, Attribute, and Analytic views Virtual tables created from remote objects of a remote source with the PASSWORD credential type (see the topic CREATE REMOTE SOURCE in the SAP HANA SQL and System Views Reference). Note When requesting profiling of a catalog object that does not result from an activation, you must assign the role SELECT with grant option to the technical user _HANA_IM_ESS. (For activated objects, there is nothing to do.) Current limitations include the following: Calculation views with dynamic privileges will not be profiled Views with required parameters will not be profiled Calculation views with dependencies to other views will only be profiled if they are referenced by exactly the same set of analytic privileges as their dependent views. It is advised in this version to only create a single analytic privilege that references all views. Future versions will handle dependencies with different privileges. 6.8.6.1 Limiting Objects to Profile You can prevent artifacts (limited to catalog objects) from being profiled. Context Limiting the artifacts to profile lets you control the volume of searchable attribute values or avoid extracting searchable values from datasets that hold sensitive or personal data. In order to prevent an artifact from being profiled, an administrator can blacklist artifacts. When a catalog object that was previously profiled is blacklisted, all its extracted searchable attribute values are immediately removed from Entity Grid. The catalog object will never be profiled again, even if there is still a data profiling schedule associated with the object(s). To blacklist an artifact, follow these steps: Monitor and Operate 2016 SAP SE or an SAP affiliate company. All rights reserved. 103

Procedure 1. Select the Data Profiling Blacklist tile. 2. In the Catalog Object Browser, expand the nodes and select an artifact to blacklist. You can also select a schema to list all its children then select objects to blacklist from within it. 3. To blacklist the artifact, select the Blacklisted check box and click Save. To blacklist all displayed artifacts, select the Blacklisted check box in the column heading and click Save. To re-enable data profiling for an artifact, clear the check box and click Save. 6.8.6.2 Information Available on the Data Profiling Blacklist Monitor The Enterprise Semantic Services Data Profiling Blacklist Monitor lets you view and choose which artifacts to blacklist (remove data profiling values). Name Description Filterable Catalog Object Blacklisted catalog objects Extracted values Fully qualified name of the selected object Number of blacklisted objects in the selected artifact The total number of extracted values for the selected object Schema Name Name of the schema to which the artifact belongs Yes Wildcards Catalog Object Catalog object name in the schema Yes Extracted Values Number of extracted searchable values for the object Yes Blacklisted Date Timestamp of when the object was blacklisted Yes Blacklisted Select the check box to blacklist the object and click Save. Clear the check box to enable data profiling for the object and click Save. No 6.8.7 Set Configuration Parameters As an Enterprise Semantic Services administrator, you can set configuration parameter values such as maximum sizes of persistent queues, rolling policy of persistent queues, and so on. To set configuration parameters, in the SAP HANA studio Administration Console, a system administrator sets values in the reserved table sap.hana.im.ess.eg.configuration::configuration. To do so, specific database procedures and user privileges are required. 104 2016 SAP SE or an SAP affiliate company. All rights reserved. Monitor and Operate

Example To set the parameters for "'MAX_ESS_PROFILING_JOBS_SCHEDULE_TIME": 1. Re-enable _HANA_IM_ESS technical user: ALTER USER _HANA_IM_ESS ENABLE PASSWORD LIFETIME; 2. Connect with the _HANA_IM_ESS user with its password (the one used during installation). 3. Execute the procedure to increase configuration parameter MAX_ESS_PROFILING_JOBS_SCHEDULE_TIME: CALL "SAP_HANA_IM_ESS"."sap.hana.im.ess.eg.configuration::SET_CONFIGURATION_VALUE"(' MAX_ESS_PROFILING_JOBS_SCHEDULE_TIME', value) where VALUE can be up to 1150000 (the default is 500000 ms). 4. When finished, disable the password: ALTER USER _HANA_IM_ESS DISABLE PASSWORD LIFETIME; 6.8.8 Troubleshoot Installation Issues Troubleshoot Enterprise Semantic Services (ESS) installation issues. Symptom Solution When importing the HANA_IM_ESS delivery unit (DU), an activation error occurs and displays in the SAP HANA studio job log view. Check the job log details in SAP HANA studio. If the error message is not meaningful, then: Access the diagnosis file for the index server. From the bottom of the trace log, look for the first Check results message, which should indicate the root cause of the activation failure and suggest how to solve it. Symptom Cause Solution The ESS DU has been uninstalled using the uninstallation procedure. When you reimport the ESS DU, activation errors occur, showing that dependent objects are not found. Activated objects may have dependent objects that do not yet exist and therefore cause an error. Check the following: Verify that all ESS DUs (including the DEMO DU) have been properly uninstalled through the SAP HANA Application Lifecycle Management console. Verify that all related packages have been deleted (those with a naming of sap.hana.im.ess...); otherwise remove them as follows: Create a workspace in the Repositories tab of SAP HANA studio. Remove the packages from there. Monitor and Operate 2016 SAP SE or an SAP affiliate company. All rights reserved. 105

6.8.9 Troubleshoot Publishing Issues Troubleshoot Enterprise Semantic Services (ESS) publishing issues. Symptom Cause Solution The Request Monitor displays a message that includes the phrase If the failure repeats, contact SAP support. Transaction serialization failures in concurrency scenarios. If the transaction that failed was a publish request, on the Request Monitor for the artifact in question, select the Retry check box and the Retry button. Symptom Cause Publishing request cannot be submitted to ESS. An internal server error message is returned to the application. Investigate the error as follows: In the SAP HANA Administration Console, set the trace level for xsa:sap.hana.im.ess to INFO. See Activate Error Trace for Enterprise Semantic Services [page 113]. Inspect latest diagnosis file xsengine_alert_xxx.trc. Check for the following error message: Error: import: package access failed due to missing authorization ( ) This means that the user who is publishing has not been granted the activated ESS role sap.hana.im.ess.roles::publisher or the application privilege sap.hana.im.ess::publisher. Solution Grant the Publisher role to the user who is publishing. Activated roles are granted through the execution of the procedures GRANT_ACTIVATED_ROLE and GRANT_ APPLICATION_PRIVILEGE. Therefore, to be able to grant a role, a user must have the SQL object privilege EXECUTE on these procedures. Symptom Solution Publishing requests appear as not processed in the SAP HANA ESS Administration tool's Publish and Scheduling Monitor view. Request Status remains REQUEST PENDING or REQUESTED. Rerun the installation script install.html to verify the installation.. Symptom Cause 1 Solution 1 Cause 2 Solution 2 Cause 3 Solution 3 A publishing request failed (the Request Status is FAILED in the SAP HANA ESS Administration tool Publish and Scheduling Monitor view). SAP HANA view definition format is not supported. The user can upgrade the format of the view by editing (make a small change such as adding a space) and saving it. The SAP HANA view is not supported. No user action. API error Invalid arguments have been passed to the API. Related Information API Error Messages [page 109] 106 2016 SAP SE or an SAP affiliate company. All rights reserved. Monitor and Operate

6.8.10 Troubleshoot Data Profiling Issues Troubleshoot Enterprise Semantic Services (ESS) data profiling issues. Symptom Cause Publishing requests have been processed and all ESS background jobs are active, but data profiling requests appear as not processed in the SAP HANA ESS Administration tool Data Profiling Monitor view. Profiling Status remains as REQUEST PENDING. Investigate the error as follows: Enable the trace level for xsa:sap.hana.im.ess to ERROR in the SAP HANA Administration Console Inspect the latest diagnosis file xsengine_alert_xxx.trc. Check for the following error message: ESS815=Profiling Internal Error: Script server must be enabled to profile data Verify whether the script server was created during installation. To do so, in the SAP HANA Administration Console, view the Configuration tab, and in daemon.ini, expand scriptserver. Solution See Enable the Script Server [page 51]. Symptom Cause Solution A run-time object has the Request Status of FAILED in SAP HANA ESS Administration tool Data Profiling Monitor view. An error with message code ESS805 Insufficient privilege - user xxx must have SELECT with GRANT option on xxx is returned. If the run-time object is not an activated object, then check that the SELECT right on the run-time object has been granted WITH GRANT OPTION to the technical user _HANA_IM_ESS. Run the following SQL command in SAP HANA studio: GRANT SELECT ON <catalog object> TO _HANA_IM_ESS WITH GRANT OPTION Symptom Cause 1 Solution 1 Cause 2 Solution 2 A data profiling request failed (the Request Status is FAILED in the SAP HANA ESS Administration tool Data Profiling Monitor view). Insufficient privileges to _HANA_IM_ESS. Rerun the install.html script. API error Invalid arguments have been passed to the API. See API Error Messages [page 109]. 6.8.11 Troubleshoot Search Issues Troubleshoot Enterprise Semantic Services (ESS) search issues. Symptom Cause The SAP HANA user 'User' cannot perform a search using the ESS API. An internal server error message is returned to the application. Investigate the error as follows: Monitor and Operate 2016 SAP SE or an SAP affiliate company. All rights reserved. 107

In the SAP HANA Administration Console, set the trace level for xsa:sap.hana.im.ess to INFO. See Activate Error Trace for Enterprise Semantic Services [page 113]. Inspect latest diagnosis file xsengine_alert_xxx.trc. Check for the following error message: Error: import: package access failed due to missing authorization ( ) This means that the 'User' who is publishing has not been granted the privilege Role: sap.hana.im.ess.roles::user. Solution Grant the following privilege using SQL commands: CALL "_SYS_REPO"."GRANT_ACTIVATED_ROLE"('sap.hana.im.ess.roles::User','user'); Symptom A search query does not return an expected catalog object that exists in the SAP HANA instance. OR Suggestions do not show an expected term, although that term is associated with a database object in the SAP HANA instance. Cause Solution Insufficient privileges. Verify the user who is posing the search query has sufficient privileges to access the searchable elements of the expected catalog object as in the following table. Database object type Searchable elements Privileges needed table, SQL view, virtual table, column view metadata Owner, object privilege, READ CATALOG, DATA ADMIN table, SQL view, virtual table Profiled data Owner, SELECT object privilege column view Profiled data Owner, SELECT object privilege, analytic privilege Symptom Cause Solution A search query does not return an expected database object that exists in the SAP HANA instance. Assuming that the user has sufficient required authorizations, check the syntax of the search query. See the following examples of common errors. Search query Unmatched searchable element Correction Customer name Does not match NAME1 Name* Or follow suggested words Sales ATT Does not match value AT&T AT T will match AT&T, AT-T, AT/T Search query Unexpected match Correction Sales_2006 Sales Sales_2007 sales_2006 Unit sales Product unit unit sales Open insurance contract Closed insurance contract +open insurance contract 108 2016 SAP SE or an SAP affiliate company. All rights reserved. Monitor and Operate

Search query Unexpected match Correction open insurance contract Symptom Cause Solution Acronyms or abbreviations are not matched by a search query, which, as a result, does not return an expected database object that exists in the SAP HANA instance. A configuration is missing in the term-mapping table: SAP_HANA_IM_ESS. sap.hana.im.ess.services.search::mapping To modify the entries in the term mapping table, see "Search Term Mapping" in the Configuration Guide for SAP HANA Smart Data Integration and SAP HANA Smart Data. Symptom Cause Solution A search query does not return an expected database object that exists in the SAP HANA instance Assuming that the user has sufficient authorizations and the syntax of the search query is correct, then use the SAP HANA ESS Administration tool Publish and Unpublish Monitor view to verify whether the database object has been successfully published to ESS, that is, its Request Status is SUCCESSFUL. If the Request Status is FAILED, take the appropriate actions according to the error code and message. 6.8.12 API Error Messages Troubleshoot Enterprise Semantic Services (ESS) API errors. Error API Action ESS100=Error occurred when extracting metadata from a publication artifact. Publish Not necessarily an API error. First check trace log file for details on the error. ESS153=Metadata Extraction Internal Error: No view definition. Publish Check API artifact argument or SAP HANA view was concurrently deleted. ESS154=Metadata Extraction Internal Error: No package name. Publish Check API artifact argument or package was concurrently deleted. ESS158=Package name {0} does not exist. Publish Check API artifact argument or artifact was concurrently deleted. ESS159=HANA view {0}/ {1}.{2} does not exist. Publish Check API artifact argument or artifact was concurrently deleted. ESS160=Publication artifact of type {0} is not supported. Publish Check API arguments and list of supported types of publication artifacts. Monitor and Operate 2016 SAP SE or an SAP affiliate company. All rights reserved. 109

Error API Action ESS161=Schema name {0} does not exist. Publish Check API artifact argument or schema was concurrently deleted. ESS162=Catalog object XXX does not exist or is not supported. Publish Check API artifact argument or catalog object was concurrently deleted. ESS163=Invalid publication artifact qualified name {0}. Publish Not necessarily an API error. Verify the artifact exists. ESS164=Invalid container path {0}. Errors 180 186 Publish Check API artifact argument or updates happened concurrently. Verify the path still exists. ESS180=Expecting character {0} but encountered character {1} in the publication artifact. Publish Parsing error in artifact name. Check API artifact argument. ESS181=Close quote without a matching open quote in string {0} of the publication artifact. Publish Parsing error in artifact name. Check API artifact argument. ESS182=Invalid catalog publication artifact [ {0} ]. Publish Parsing error in artifact name. Check API artifact argument. ESS183=Invalid publication artifact {0} [ {1} ]. Publish Parsing error in artifact name. Check API artifact argument. ESS184=First identification element of the publication artifact is not a catalog or content {0}. Publish Parsing error in artifact name. Check API artifact argument. ESS185=Unmatched quote for identification element {0} in publication artifact. Publish Parsing error in artifact name. Check API artifact argument. ESS186=Identification element {0} should not be empty. Publish Parsing error in artifact name. Check API artifact argument. Error containing string: Search query syntax error Errors 500 526 Search Search syntax error. First check explanation given in error message. Then check search query in trace log with trace level DEBUG. Error containing string: Request error message Errors 600 645 Search Request message error. First check explanation given in error message. Then check request message in trace log with trace level DEBUG. 110 2016 SAP SE or an SAP affiliate company. All rights reserved. Monitor and Operate

Error API Action ESS705=Invalid scope name [{0}]. Publish Only roman letters (both lowercase and uppercase), digits, and underscore characters are valid. ESS715=Invalid type filter [{0}] for container [{1}]. Please use one of [table,virtualtable,view, columnview] Publish Invalid argument for typefilter in API. Check explanation given in error message in trace log. ESS720=External source [{0}] not supported. Publish Check API parameter source. Can only be LOCAL. ESS725=Mandatory argument [{0}] is missing in function [{1}]. Publish CT on-demand Search Mandatory API argument is missing. First check explanation given in error message. Then check search query in trace log with trace level DEBUG. ESS730=Invalid value [{0}] for argument [{1}] in function [{2}], expecting: [{3}]. Publish CT on-demand Search Invalid argument in API. First check explanation given in error message. Then check search query in trace log with trace level DEBUG. ESS735=Invalid value [{0}] at position [{1}] for argument [{2}] in function [{3}], expecting: [{4}]. Publish CT on-demand Search Invalid argument in API. First check explanation given in error message. Then check search query in trace log with trace level DEBUG. ESS808=Profiling Error: Invalid Runtime Object Type {0}. Only the following input values are supported: {1} CT on-demand Request to profile an unknown type of run-time object. First check explanation given in error message. Then check search query in trace log with trace level DEBUG. ESS809=Profiling Error: Column View {0}. {1} cannot be profiled CT on-demand Non-profilable column view due to current restrictions. ESS810=Profiling Error: {0} {1}. {2} does not exist. Runtime object does not exist CT on-demand Check API argument or artifact was concurrently deleted. ESS811=Profiling Error: At least one attribute to profile must be given CT on-demand Check API argument. ESS812=Profiling Error: {0} is not attribute of {1} {2}. {3} CT on-demand Check API argument or artifact has been updated concurrently. Monitor and Operate 2016 SAP SE or an SAP affiliate company. All rights reserved. 111

6.8.13 Troubleshooting Tips Tips for troubleshooting and preparing disgnostic information for SAP support. 6.8.13.1 Troubleshoot Repeated Errors Procedure for error messages in HANA ESS Administration Monitor view that include If the failure repeats. Prerequisites The error trace for Enterprise Semantic Services must be activated before you retry the operation. Context To retry the failed operation: Procedure 1. Go to the Enterprise Semantic Services Administration Monitor by entering the following URL in a web browser: http://<<your_hana_instance:port>>/sap/hana/im/ess/ui 2. If the transaction that failed was a publish request, click the Publish and Unpublish Monitor tile, find your runtime object in the Published Artifact column, and click the Retry icon. Next Steps After you retry the operation and it still fails, collect diagnostic information. 112 2016 SAP SE or an SAP affiliate company. All rights reserved. Monitor and Operate

6.8.13.2 Activate Error Trace for Enterprise Semantic Services The error trace obtains detailed information about actions in Enterprise Semantic Services. Prerequisites To configure traces, you must have the system privilege TRACE ADMIN. Context To activate the error trace for Enterprise Semantic Services, follow these steps: Procedure 1. Log on to SAP HANA studio with a user that has system privilege TRACE ADMIN. 2. In the Administration editor, choose the Trace Configuration tab. 3. Choose the Edit Configuration button for the trace that you want to configure. 4. Expand the XS ENGINE node. Note Ensure you select the checkbox Show All Components. 5. Locate xsa:sap.hana.im.ess and check that system trace level is INFO, ERROR, or DEBUG, because it is usually set to DEFAULT. 6. Click Finish. 6.8.13.3 Open the Error Log After the Enterprise Semantic Services error trace is activated, find the error message. Procedure 1. Log on to SAP HANA studio with a user name that has system privilege CATALOG READ. 2. In the Administration editor, choose the Trace Configuration tab and go to Diagnosis Files tab. Monitor and Operate 2016 SAP SE or an SAP affiliate company. All rights reserved. 113

3. Look for one of the two most recent files xsengine_alert_xxx.trc or indexserver_alert_xxx.trc.. 4. Go to the end of the file to see the error. 6.8.13.4 Collect Diagnostic Information Procedure to obtain diagnostic information to send to SAP support. Procedure 1. Log on to SAP HANA studio with a user name that has the system privilege CATALOG READ. 2. In the Administration editor, choose the Trace Configuration tab and go to Diagnosis Files tab. 3. Choose Diagnosis Information Collect. 4. Send the following information to SAP support: Error message and error code Collected diagnosis files If the error appeared in the Publish and Scheduling Monitor, send all other fields of the displayed row with the error message. 114 2016 SAP SE or an SAP affiliate company. All rights reserved. Monitor and Operate

7 Administer Administrators can manage roles and privileges, manage agent groups, create remote sources, and so on. Assign Roles and Privileges [page 115] Add roles and privileges for users to perform various tasks. Create Remote Sources [page 119] Once everything has been installed, registered, configured, and so on, you can begin creating remote sources. Activate and Execute a Replication Task [page 120] Activation generates the run time objects necessary for data movement from one or many source tables to one or more target tables. Activate and Execute a Flowgraph [page 122] After your flowgraph is created and configured, activate it to create the run-time objects. Administering Agent Groups [page 123] Agent grouping provides fail-over and load balancing capabilities by combining individual Data Provisioning Agents installed on separate host systems. Uninstall the Data Provisioning Agent [page 126] You can uninstall the Data Provisioning Agent in graphical mode on Windows and Linux. Troubleshooting the Data Provisioning Agent [page 127] This section describes error situations related to the Data Provisioning Agent and their solutions. Configuring the Operation Cache [page 130] You can improve performance by using an operation cache in some script servers. Ensure Workload Management and Resource Consumption [page 132] Ensure that in circumstances of limited memory or CPU resources that processes and system resources remain responsive. 7.1 Assign Roles and Privileges Add roles and privileges for users to perform various tasks. The following is a list of common tasks and roles or privileges that an administrator needs to assign to complete those tasks. Note Permissions may also be required for accessing a particular database through a data provisioning adapter. See the section for more information. Administer 2016 SAP SE or an SAP affiliate company. All rights reserved. 115

Task Role Privileges Description Register a DP Agent Register an adapter Configure DP Agent to use HTTP (cloud) protocol Create an Agent or adapter when SAP HANA is in the cloud System privilege: AGENT ADMIN System privilege: ADAPTER ADMIN Application privilege: sap.hana.im.dp.proxy::agentmessaging sap.hana.im.dp.admin::administrator Whoever sets the DP Agent to use HTTP (cloud) in the DP Agent Configuration tool needs to be assigned this role. Needed when administrator want to create adapters/agent from agent config tool when SAP HANA is on the cloud (or Agent uses HTTP protocol). sap.hana.xs.lm.roles: :Administrator This role is necessary if you are using SAP HANA Application Lifecycle Management to import the data provisioning delivery unit. Import a delivery unit using SAP HANA studio System privilege: REPO.IMPORT Configure role based access in SAP HANA Cockpit sap.hana.uis.db::sit E_DESIGNER Monitoring sap.hana.im.dp.moni tor.roles::monitoring Application privilege: Import a delivery unit using SAP HANA Application Lifecycle Management sap.hana.im.dp.monitor::monitoring sap.hana.im.dp.moni tor.roles::operations The Operations role includes the following application privileges: START TASK STOP TASK SCHEDULE TASK PROCESS REMOTE EXCEPTION (This includes both remote source and remote subscription exceptions) Enable users to schedule a task sap.hana.xs.ad Schedule a task sap.hana.im.dp.moni tor.roles::operations min.roles::jobsche- duleradministrator Application privilege: sap.hana.im.dp.monitor::scheduletask 116 2016 SAP SE or an SAP affiliate company. All rights reserved. Administer

Task Role Privileges Description Start a task Application privilege: Stop a task Application privilege: sap.hana.im.dp.monitor::starttask sap.hana.im.dp.monitor::stoptask Create a remote source System privilege: CREATE REMOTE SOURCE If a user can create a remote source (has CREATE REMOTE SOURCE system privilege), that user automatically has CREATE VIRTUAL TABLE, DROP, CREATE REMOTE SUBSCRIP TIONS and PROCESS REMOTE SUBSCRIP TION EXCEPTION privileges; they don t need to be assigned to the user. However, this only applies to remote sources that the user creates themselves. If a remote source is created by someone else, then those privileges must be assigned, for each remote source, in order to perform those tasks. Drop a remote source Object privilege: DROP Must be explicitly granted for a remote source created by another user. Search for an object in a remote source Object privilege: ALTER on the remote source to be searched. In order to search for remote objects (e.g. tables) in a remote source, the user must have the ALTER object privilege on the remote source, so that a dictionary can be created. Add a virtual table Object privilege CREATE VIRTUAL TABLE Must be explicitly granted for a remote source created by another user. Create a remote subscription Object privilege: CREATE REMOTE SUB SCRIPTION Must be explicitly granted for a remote source created by another user. Create a flowgraph sap.hana.xs.ide.roles ::EditorDeveloper Object privilege: EXECUTE on "_SYS_REPO"."TEXT_AC CESSOR" and "_SYS_REPO"."MULTI_TEXT _ACCESSOR" Allows creation of.hdbflowgraph. Create a flowgraph of type Task Object privilege: SELECT (for input/output schema) Create a replication task sap.hana.xs.ide.roles ::EditorDeveloper Allows creation of.hdbreptask. Administer 2016 SAP SE or an SAP affiliate company. All rights reserved. 117

Task Role Privileges Description Activate replication task (.hdbreptask) Object privileges:. SELECT on the source schema Must be granted to _SYS_REPO. Activate flowgraph (.hdbflowgrap h) Execute a stored procedure Execute a task Process remote subscription exceptions Use the JIT (justin-time) Data Preview option Object privileges: SELECT on the source table INSERT, UPDATE, and DE LETE on the target table SELECT on the target schema (only when using a Template Table as a target) Object privilege: EXECUTE Object privilege: EXECUTE INSERT UPDATE SELECT DELETE Object privilege: PROCESS REMOTE SUB SCRIPTION EXCEPTION Object privilege: SELECT and EXECUTE with GRANT OPTION Must be granted to _SYS_REPO. Needed on the schema where the stored procedure is located. Needed on the schema where the task is located. Must be explicitly granted for a remote source created by another user. Must be granted to _SYS_REPO user. Needed on the schema where the task or stored procedure is located. Related Information SAP HANA Security Guide: Developer Authorization for the Repository (HTML) SAP HANA Security Guide: Developer Authorization for the Repository (PDF) SAP HANA Developer Guide: SAP HANA Web-based Development Workbench: Editor (HTML) SAP HANA Developer Guide: SAP HANA Web-based Development Workbench: Editor (PDF) Grant Roles to Users [page 39] 118 2016 SAP SE or an SAP affiliate company. All rights reserved. Administer

7.2 Create Remote Sources Once everything has been installed, registered, configured, and so on, you can begin creating remote sources. For more information about how to create remote sources, see the SAP HANA Administration Guide. Also, see the section in this guide for information about creating remote sources for these adapters. Related Information [page 133] Adding Remote Data Sources SAP HANA Administration Guide (PDF) 7.2.1 Add a Remote Source in the Web-based Development Workbench How to add a remote source in the Web-based Development Workbench. Procedure 1. In the Catalog editor, right-click the Provisioning Remote Sources folder, and choose New Remote Source. 2. Enter the required information including the adapter and agent names. 3. Click Save. 7.2.2 Adapter Remote Source User Credentials Administrators must provide user credentials for those who need to access a remote source. They must also be aware of the adapter-specific limitations involving user types and remote sources, as well as other functionality. The following must be taken into consideration; A remote source created with a secondary user can only be used for querying virtual tables. If the remote source will be used for designing a.hdbreptask or.hdbflowgraph enabled for real time, use technical user If you create remote subscription using the CREATE REMOTE SUBSCRIPTION SQL statement, use technical user. Administer 2016 SAP SE or an SAP affiliate company. All rights reserved. 119

Create Credentials for a Secondary User The syntax for creating secondary user credentials for SDI data provisioning adapters is different than the syntax for SAP HANA system adapters. create credential for user <user_name> component 'SAPHANAFEDERATION' purpose <remote_source_name> type 'PASSWORD' using <CredentialEntry name="credential"> <user><user_name></user> <password><password></password> </CredentialEntry> 7.3 Activate and Execute a Replication Task Activation generates the run time objects necessary for data movement from one or many source tables to one or more target tables. Context The replication task creates the following run time objects. Virtual table(s): Generated in the specified virtual table schema. You can display the contents of the virtual table in SAP HANA studio. Remote subscription(s): Generated in the schema selected for the virtual table. This is only generated when the Initial load only option is not selected. Task(s): Generated in the same schema as the target table. View(s): Generated in the same schema as the virtual table. Target table(s): Populated with the content after execution. Procedure: Generated in the schema of the target table, the procedure performs three functions. 1. Sets the remote subscription to the Queue status. Note The remote subscription is only created when Initial load only is unselected. 2. Calls Start Task to perform the initial load of the data. 3. Sets the remote subscription to the Distribute status. Any changes, additions or deletions made to the source data during the initial load are updated in the target system. Any changes to the source data thereafter are updated real time to the target. Note The remote subscription is only created when Initial load only is unselected. 120 2016 SAP SE or an SAP affiliate company. All rights reserved. Administer

Procedure 1. After the replication task is configured, click Save to activate. 2. Go to the Catalog view and navigate to the stored procedure you just created. Note You can access the Catalog view on the SAP HANA XS Web server at the following URL http:// <WebServerHost>:80<SAPHanaInstance>/sap/hana/xs/ide/catalog. Choose one of the following options to activate the replication task. Right-click the stored procedure, and then select Invoke Procedure. To call the stored procedure, use the following SQL script: CALL "<schema_name>"."<package_name>::<target_table_name>".start_replication. The replication begins. You can right-click and select Open Contents to view the data in the target table in the Catalog view. Note If the replication task takes longer than 300 seconds to process, you might receive an error about the XMLHttpRequest failing. You can correct this issue by increasing the maximum run time option in the xsengin.ini file. Follow these steps: 1. Login to SAP HANA studio as a SYSTEM user. 2. In the Systems view, right-click the name of your SAP HANA server, and then choose Configuration and Monitoring Open Administration. 3. Click the Configuration tab. 4. Select xsengine.ini. 5. Expand httpserver. 6. Click Add parameter. 7. In the Assign Values to option, select System, and then Next. 8. In the Key option, enter max_request_runtime and then enter a value. For example, you might want to enter 1200. The value is in seconds. 9. Click Finish and then close the Configuration tab and execute the replication task again. Results You can use SAP HANA Cockpit to monitor the results. Administer 2016 SAP SE or an SAP affiliate company. All rights reserved. 121

Related Information Monitor and Operate [page 57] SAP HANA SQL and System Views Reference (PDF) SAP HANA SQL and System Views Reference (HTML) 7.4 Activate and Execute a Flowgraph After your flowgraph is created and configured, activate it to create the run-time objects. Context Activation creates the run-time objects based on the options set in the flowgraph. Procedure 1. From the Project Explorer, right-click on the.hdbflowgraph that you created. 2. Choose Team Activate. The run time objects are created. 3. Choose one of the following: If you configured the flowgraph for initial load only, use the following SQL to run the generated task: START TASK "<schema_name>"."<package_name>::<flowgraph_name>" Note You can also specify a variable when running Start Task. For example, if you have a Filter node set to output records for a specific country, you can enter it in a similar way to the following. START TASK "<schema_name>"."<package_name>::<flowgraph_name>" (country => '''Spain'''); If you configured the flowgraph for real time, use the following SQL script to execute the generated initialization procedure: CALL "<package_name>::<flowgraph_name>_sp" If you configured the flowgraph for real time and want to pass a variable value, use the following script to execute the generated initialization procedure: CALL "<package_name>::<flowgraph_name>_sp"(""'spain'"") 122 2016 SAP SE or an SAP affiliate company. All rights reserved. Administer

For more information about Start Task and calling a table type, see the Start Task topic. Related Information SAP HANA SQL and System Views Reference (PDF) SAP HANA SQL and System Views Reference (HTML) 7.5 Administering Agent Groups Agent grouping provides fail-over and load balancing capabilities by combining individual Data Provisioning Agents installed on separate host systems. About fail-over When an agent that is part of a group is inaccessible for a time longer than the configured heart beat time limit, the Data Provisioning Server chooses a new active agent within the group and resumes replication for any remote subscriptions active on the original agent. Any remote subscriptions between the Queue and Distribute states at the time the agent became unavailable are logged as exceptions, and must be manually reset with new Queue and Distribute commands. About load balancing Data Provisioning Agent groups provide load balancing by using a round robin policy. For example, with multiple agents in the group and a remote source configured on the group, each smart data access request to the remote source is load balanced by rotating through the agents in the group. For change data capture and real-time operations, an agent in the group is selected when a QUEUE command is executed on a remote subscription. Any following CDC requests are directed to the agent selected during the QUEUE command. Planning considerations Before configuring agent group, review the following considerations and limitations: Each agent in a group must be installed on a different host system. All agents in a group must have identical adapter configurations. Administer 2016 SAP SE or an SAP affiliate company. All rights reserved. 123

All agents in a group must use the same communication protocol. You cannot mix on-premise agents (TCP) and cloud-based agents (HTTP) in a single group. For log reader adapters that do not support disaster recovery, you must create a shared work directory that is accessible by all agents in the group. Copy the LogReader instance directory to the shared location, and specify the shared instance name in the remote source configuration. Note Disaster recovery is supported only in the Oracle log reader adapter. 7.5.1 Create or Remove an Agent Group You can create a new agent group or remove an existing group in the SAP HANA studio or Web-based Development Workbench. Prerequisites Note You must have the AGENT_ADMIN privilege to create or remove agent groups. Procedure 1. Open the SQL console in the SAP HANA studio or Web-based Development Workbench. 2. Execute the CREATE or DROP statement in the SQL console. To create a new agent group: CREATE AGENT GROUP <group_name> To remove an existing agent group: DROP AGENT GROUP <group_name> 124 2016 SAP SE or an SAP affiliate company. All rights reserved. Administer

7.5.2 Configure Agents in an Agent Group You can create add agents to an agent group or remove agents from a group in the SAP HANA studio or Webbased Development Workbench. Prerequisites Note You must have the AGENT_ADMIN privilege to add or remove agents in an agent group. Procedure 1. Open the SQL console in the SAP HANA studio or Web-based Development Workbench. 2. Execute the CREATE or ALTER AGENT statement in the SQL console. To create a new agent in the group: CREATE AGENT <agent_name> PROTOCOL <protocol> HOST <hostname> AGENT GROUP <group_name> To add an existing agent to the group: ALTER AGENT <agent_name> SET AGENT GROUP <group_name> To remove an agent from the group: ALTER AGENT <agent_name> UNSET AGENT GROUP <group_name> 7.5.3 Configure Remote Sources in an Agent Group To receive the benefits of fail over and load balancing from an agent group, you must configure your remote sources on the agent group. Procedure 1. Open the SQL console in the SAP HANA studio or Web-based Development Workbench. 2. Execute the CREATE or ALTER REMOTE SOURCE statement in the SQL console. Administer 2016 SAP SE or an SAP affiliate company. All rights reserved. 125

To create a new remote source in the group: CREATE REMOTE SOURCE <source_name> ADAPTER <adapter_name> AT LOCATION AGENT GROUP <group_name> CONFIGURATION ' 'WITH CREDENTIAL TYPE 'PASSWORD' USING ' ' To add an existing remote source to the group: ALTER REMOTE SOURCE <source_name> ADAPTER <adapter_name> AT LOCATION AGENT GROUP <group_name> 7.6 Uninstall the Data Provisioning Agent You can uninstall the Data Provisioning Agent in graphical mode on Windows and Linux. Procedure On Windows, call the uninstallation program from Control Panel. Programs and Features SAP HANA Data Provisioning Agent Uninstall On Linux, call the uninstallation program from the command line. a. Navigate to the <DPAgent_root>/install directory. For example, /usr/sap/dataprovagent/install. b. Run./hdbuninst --main SDB::Install::App::Gui::Uninstallation::main --path "<DPAgent_root>". Results The Data Provisioning Agent is uninstalled from the system. 7.6.1 Uninstall from the Command Line If you cannot use or do not want to use the graphical uninstallation mode, you can uninstall the Data Provisioning Agent using the command line. Procedure 1. Navigate to the <DPAgent_root>/install directory. 126 2016 SAP SE or an SAP affiliate company. All rights reserved. Administer

For example, C:\usr\sap\dataprovagent\install or /usr/sap/dataprovagent/install. 2. Call the uninstallation program. On Windows, run hdbuninst.exe --path "<DPAgent_root>". On Linux, run./hdbuninst --path "<DPAgent_root>". Results The Data Provisioning Agent is uninstalled from the system without displaying the graphical uninstallation manager. 7.7 Troubleshooting the Data Provisioning Agent This section describes error situations related to the Data Provisioning Agent and their solutions. 7.7.1 Data Provisioning Agent Log Files and Scripts Various log files are available to monitor and troubleshoot the Data Provisioning Agent. The following log files are available: Log File Name and Location <DPAgent_root>/log/ dpagent_service_eventlog_<date>.log <DPAgent_root>/log/framework_alert.trc <DPAgent_root>/log/framework.trc <DPAgent_root>/log/configtool.log <DPAgent_root>/LogReader/<instance>/log/ <instance>.log <DPAgent_root>/LogReader/admin_logs/ admin<timestamp>.log Description Data Provisioning Agent event log Data Provisioning Agent adapter framework log. Use this file to monitor data provisioning agent statistics. Data Provisioning Agent adapter framework trace log. Use this file to trace and debug data provisioning agent issues Data Provisioning Agent Configuration Tool log Instance log for log reader-based adapters Administrative log for log reader-based adapters Additionally, scripts for performing source database initialization and cleanup operations can be found at <DPAgent_root>/LogReader/scripts. Administer 2016 SAP SE or an SAP affiliate company. All rights reserved. 127

7.7.2 Agent Was Started by a Root User If the Data Provisioning Agent is accidentally started by a root user, you should clean up the Agent installation. Context In a normal configuration, the Data Provisioning Agent should be run only by the non-root installation owner defined during the Agent installation. If the Agent is accidentally started by a root user, additional files are created in the installation that should be removed. Procedure 1. Navigate to the configuration directory in the Agent installation location. For example, /usr/sap/dataprovagent/configuration 2. Remove the following files: com.sap.hana.dp.adapterframework org.eclipse.core.runtime org.eclipse.osgi Caution Do not remove the config.ini or org.eclipse.equinox.simpleconfigurator files. 3. Remove the log and secure_storage directories. For example, /usr/sap/dataprovagent/log and /usr/sap/dataprovagent/secure_storage. Note Depending on the permissions, you may require sudo access to remove the secure_storage directory. For example, sudo rm -rf secure_storage. 128 2016 SAP SE or an SAP affiliate company. All rights reserved. Administer

7.7.3 Agent JVM Out of Memory If the Java Virtual Machine on the Data Provisioning Agent is running out of memory, you may need to adjust configuration parameters on the Agent or your adapters. Oracle log reader adapters For Oracle log reader adapters, there are multiple possible solutions. Check the Queue size of parallel scan tasks parameter. The default value is 0, which allows for an unlimited queue size. To find the maximum queue size that results in acceptable memory use, set the queue size to 10,000 and increase in increments of 50,000 or 100,000. Check the Parallel scan SCN range parameter. Decrease the value as necessary. Check the Number of parallel scanners parameter. Decrease the value from 4 to 2. General virtual memory issues For general memory issues, you can increase the JVM memory usage. In dpconfig.ini, change the memory allocation parameter, and restart the Agent. For example, to increase the memory to 8GB, -Xmx8096m. 7.7.4 Adapter Prefetch Times Out When task execution fails due to an adapter timeout, you may need to adjust the adapter prefetch timeout parameter. In the Index Server trace log, you may notice an error similar to the following: DPAdapterAccess.cpp(01825) : DPAdapterAccess::Fetch: failed with error: Prefetch timed out. When this type of error appears, you can adjust the value of the adapter prefetch timeout parameter. From a SQL console: ALTER SYSTEM ALTER CONFIGURRATION ('dpserver.ini','system') SET ('framework','prefetchtimeout')='<seconds>' WITH RECONFIGURE; For example, you might change the value to 600 to set the timeout to 10 minutes. Administer 2016 SAP SE or an SAP affiliate company. All rights reserved. 129

7.7.5 Agent Reports Errors when Stopping or Starting If the Data Provisioning Agent reports error code 5 when stopping or starting on Windows, you may need to run the configuration tool as an administrator. To run the configuration tool once as an administrator, right-click on dpagentconfigtool.exe and choose Run as administrator. Alternatively, you can set the configuration tool to always run as an administrator. Right-click on dpagentconfigtool.exe and choose Properties. From the Compatibility tab, choose Run this program as an administrator and click OK. 7.8 Configuring the Operation Cache You can improve performance by using an operation cache in some script servers. The operation cache holds operation instances for Global Address Cleanse, Universal Data Cleanse, Geocode, and Type Identifier (TID), which are initialized and ready for use during task plan execution. This improves performance by avoiding the process of task plan operation initialization/create and deletion, and allows the reuse of the cached instances both inside a single plan and across plans. Having more instances cached improves performance, but those additional cached instances consume more memory. Operation cache instances are type specific and are set in the file scriptserver.ini. You can use the following monitoring views to verify the cached operations, usage count, and so on. Select * from sys.m_caches; Select * from sys.m_cache_entries; The operation cache can be configured in SAP HANA studio by editing the file scriptserver.ini. Enabling operation cache To start operation cache instances, you must set the enable_adapter_operation_cache parameter to yes. The following sample SQL statement sets the parameter to yes. ALTER SYSTEM ALTER CONFIGURATION ('scriptserver.ini','system') SET ('adapter_operation_cache', 'enable_adapter_operation_cache') = 'yes' WITH RECONFIGURE; 130 2016 SAP SE or an SAP affiliate company. All rights reserved. Administer

Changing the operation cache default settings You can turn off the cache for each node, or change the default number of instances. The following sample SQL statement changes the number of instances. ALTER SYSTEM ALTER CONFIGURATION ('scriptserver.ini', 'SYSTEM') SET ('adapter_framework_cache', 'gac')='30'; By default, the operation cache is enabled for all the supported types of operations (which is recommended) and has the following number of instances. Table 24: Operation cache default settings Option Number of instances default setting Geocode 10 Global Address Cleanse (in the Cleanse node) 20 Universal Data Cleanse (in the Cleanse node) 60 Type Identification 20 Ensuring that real time jobs have priority The operation cache is used for both batch jobs and real time jobs. Batch jobs can exhaust the operation cache, leaving insufficient resources to optimize the running of real time jobs. If you run real time jobs, use these settings to ensure that a dedicated number of operation cache instances are available for real time tasks. By default, the number of instances made available to real time tasks is half the total number of instances for each option. Table 25: Real time default settings Option Number of instances default setting Geocode 5 Global Address Cleanse (in the Cleanse node) 10 Universal Data Cleanse (in the Cleanse node) 30 Type Identification 0 Caching performance with the Global Address Cleanse Within the Cleanse node, you can configure two options that are relevant to Global Address Cleanse. When caching is enabled, we recommend for better performance that these options are set as follows: Table 26: Best performance Global Address Cleanse options Option Country Identification Mode Recommended setting assign Administer 2016 SAP SE or an SAP affiliate company. All rights reserved. 131

Option Default Country Recommended setting NONE 7.9 Ensure Workload Management and Resource Consumption Ensure that in circumstances of limited memory or CPU resources that processes and system resources remain responsive. Information on the specifics and procedures of workload management is found in the SAP HANA Administration Guide. SAP HANA smart data integration takes advantage of this framework and allows you to better handle circumstances of limited resources. Workload management in SAP HANA allows you to optimize for your system. This framework also works within the limited memory or CPU resources, as you define them in the workload class and mapping. For example, if the workload class sets STATEMENT THREAD LIMIT = 5, then SAP HANA creates up to five instances per node or operation in parallel during the task plan execution. If the workload class sets STATEMENT MEMORY LIMIT = 2GB, but any of the nodes or operation in the task plan require more than 2GB of memory, then the task would fail with an error [MEMORY_LIMIT_VIOLATION] Information about current memory composite-limit violation. You must consider these options and constraints to create the best possible performance. Related Information SAP HANA Administration Guide 132 2016 SAP SE or an SAP affiliate company. All rights reserved. Administer

8 Use data provisioning adapters to connect to, and in some cases, write back to a data source. The following adapters are delivered with the Data Provisioning Agent. A brief description of their functionality is provided. For information about specific functionality for each adapter, see the individual adapter documentation in this guide, where you can find out whether any of the following is supported: Real-time change data capture Virtual table as a target Replication monitoring and statistics Parallel readers Loading options for target tables DDL propagation Searching for tables See the SAP HANA SAP HANA Smart Data Integration and SAP HANA Smart Data Configuration Guide for information about those features. Adapter Name DB2LogReaderAdapter OracleLogReaderAdapter MssqlLogReaderAdapter DB2ECCAdapter OracleECCAdapter MssqlECCAdapter AseECCAdapter CamelFacebookAdapter Description This adapter retrieves data from DB2. It can also receive changes that occur to tables in real time. You can also write back to a virtual table. This adapter retrieves data from Oracle. It can also receive changes that occur to tables in real time. You can also write back to a virtual table. This adapter retrieves data from SQL Server. It can also receive changes that occur to tables in real time. You can also write back to a virtual table. This adapter retrieves data from an SAP ERP system running on DB2. It can also receive changes that occur to tables in real time. The only difference between this adapter and the DB2LogReaderAdapter is that this adapter uses the data dictionary in the SAP ERP system when browsing metadata. This adapter retrieves data from an SAP ERP system running on Oracle. It can also receive changes that occur to tables in real time. The only difference between this adapter and the OracleLogReaderAdapter is that this adapter uses the data dictionary in the SAP ERP system when browsing metadata. This adapter retrieves data from an SAP ERP system running on SQL Server. It can also receive changes that occur to tables in real time. The only difference between this adapter and the MssqlLogReaderAdapter is that this adapter uses the data dictionary in the SAP ERP system when browsing metadata. This adapter retrieves data from an SAP ERP system running on SAP ASE. The Camel Facebook adapter is a pre-delivered component that is based upon Camel adapter. Use the Camel Facebook component to connect to and retrieve data from Facebook. In addition to Facebook, many other components are available from the Apache Software Foundation website. 2016 SAP SE or an SAP affiliate company. All rights reserved. 133

Adapter Name CamelInformixAdapter DB2MainframeAdapter OutlookAdapter ExcelAdapter FileAdapter GoogleAdapter HanaAdapter HiveAdapter ODataAdapter ASEAdapter ABAPAdapter SoapAdapter TwitterAdapter TeradataAdapter Description The Camel Informix adapter is a pre-delivered component that is based upon Camel adapter. This adapter retrieves data from an Informix source. It can also write back to an Informix virtual table. This adapter retrieves data from a DB2 for z/os system. You can also write back to a virtual table. This adapter retrieves data from Microsoft Outlook. This adapter retrieves data from Microsoft Excel. This adapter retrieves data from formatted text files. You can also write back to a virtual table. This adapter retrieves data from Google+. This adapter provides real-time change data capture capability in order to replicate data from a remote SAP HANA database to a target SAP HANA database. You can also write back to a virtual table. This adapter retrieves data from HADOOP. This adapter retrieves data from an OData service. This adapter retrieves data from SAP ASE. It can also receive changes that occur to tables in real time. You can also write back to a virtual table. This adapter retrieves data from virtual tables through RFC for ABAP tables and ODP extractors. It also provides real-time change data capture for ODP extractors. This adapter is a SOAP web services client that can talk to a web service using the HTTP protocol to download the data. The SOAP adapter uses virtual functions instead of virtual tables to expose server side operations as it closely relates to how the operation is invoked. This adapter retrieves data from Twitter. It can also receive new data from Twitter in real time. This adapter retrieves data from Teradata. It can also receive changes that occur to tables in real time. You can also write back to a virtual table. If the source you are using is not on the list, you can use the Adapter SDK to create custom adapters to suit your needs. See the SAP HANA Smart Data Integration Adapter SDK Guide for more information. See the SAP HANA Smart Data Integration Product Availability Matrix for information about supported versions. Related Information SAP HANA Smart Data Integration Adapter SDK Guide (PDF) SAP HANA Smart Data Integration Adapter SDK Guide (HTML) SAP HANA Smart Data Integration Product Availability Matrix. 134 2016 SAP SE or an SAP affiliate company. All rights reserved.

8.1 Setting up Follow these steps to configure and deploy your adapters. Setting up your adapters consists of the following steps. These are the basic steps for configuring your adapters for batch loads. If you plan to use realtime, your adapters (if realtime is supported) and source systems may require additional configuration: 1. Install, configure, and register the Data Provisioning Agent. 2. Download and install the necessary JDBC drivers (if required). 3. Deploy (if necessary) and register adapters. SAP-delivered adapters are deployed when the Data Provisioning Agent is installed. 4. Create a remote source using any of the registered adapters. Depending on the adapter you are setting up, there may be other setup requirements. Note The OData adapter is not part of the Data Provisioning Agent installation. The OData adapter is installed with SAP HANA Server, and requires configuration that cannot be done using the Data Provisioning Agent Configuration tool. Related Information Register Adapters with SAP HANA [page 36] OData [page 220] 8.2 Custom Adapters Use the Adapter SDK to create your own custom adapters. The adapters that are described in this document and that come installed with the Data Provisioning Agent were created using the Adapter SDK. You can use the Adapter SDK to create your own custom adapters and to connect to virtually any source you require. Administrators will need to set up the remote sources to these custom adapters on the appropriate agent. Related Information SAP HANA SAP HANA Smart Data Integration Adapter SDK Guide (PDF) Register Adapters with SAP HANA [page 36] 2016 SAP SE or an SAP affiliate company. All rights reserved. 135

8.3 Apache Camel Facebook The Apache Camel Facebook adapter is created based on Camel Adapter. It makes use of the Facebook Component (http://camel.apache.org/facebook.html) to access Facebook APIs. Facebook data of the configured Facebook user, like friends, families, moviews, etc., are exposed to SAP HANA server via virtual tables by Camel Facebook Adapter. 8.3.1 Set up the Camel Facebook Adapter Set up the Camel Facebook adapter, prior to using it. Context By default, Camel Facebook Adapter is not available in Data Provisioning Agent. To use it, you must do the following: Procedure 1. Open <DPAgent_root>/camel/adapters.xml and uncomment the configuration of Camel Facebook Adapter. <Adapter type="camelfacebookadapter" displayname="camel Facebook Adapter"> <RemoteSourceDescription> <PropertyGroup name="configuration" displayname="configuration"> <PropertyEntry name="httpproxyhost" displayname="http Proxy Host" description="http Proxy Host" isrequired="false"/> <PropertyEntry name="httpproxyport" displayname="http Proxy Port" description="http Proxy Port" isrequired="false"/> </PropertyGroup> <CredentialEntry name="app_credential" displayname="app Credential" userdisplayname="app ID" passworddisplayname="app Secret"/> <CredentialEntry name="user_credential" displayname="user Credential" userdisplayname="user ID" passworddisplayname="user Access Token"/> </RemoteSourceDescription> <Capabilities>CAP_SELECT</Capabilities> <RouteTemplate>facebook.xml</RouteTemplate> </Adapter> 2. Download Facebook Component 2.15.3 jar file camel-facebook-2.15.3.jar from http://camel.apache.org/ download.html, and put it in the <DPAgent_root>/camel/lib directory. 3. Download Facebook4j 1.1.12 http://facebook4j.org/download/facebook4j-core-1.1.12.jar, and put it in the <DPAgent_root>/camel/lib directory. 136 2016 SAP SE or an SAP affiliate company. All rights reserved.

8.3.2 Camel Facebook Adapter Remote Source Configuration The Camel Facebook adapter has the following remote source configuration parameters. All these parameters are used to configure Facebook component options (See <DPAgent_root>/camel/facebook.xml). If you need to specify non-default values for Facebook component options (See http://camel.apache.org/facebook.html for a complete list of these options), you can add more remote source parameters in the adapter configuration in adapters.xml, and update the Facebook component bean in <DPAgent_root>/camel/facebook.xml accordingly. Facebook requires the use of OAuth for all client application authentication. In order to use the Camel Facebook adapter with your account, you'll need to create a new application within Facebook at https:// developers.facebook.com/apps and grant the application access to your account. The Facebook application's ID and secret will allow access to Facebook APIs which do not require a current user. A user access token is required for APIs that require a logged in user. More information about obtaining a user access token can be found at https://developers.facebook.com/docs/facebook-login/access-tokens/. Category Option Description Configuration HTTP Proxy Host HTTP proxy host HTTP Port HTTP port App Credential App ID Facebook application ID App Secret Facebook application Secret User Credential User ID Facebook user ID User Access Token The user access token. More information on obtaining a user access token can be found at https://developers.facebook.com/docs/facebook-login/ access-tokens/ 8.3.3 Facebook Tables Facebook data for the configured Facebook user, such as friends, families, movies, news, and so on, are exposed in SAP HANA server via virtual tables by the Camel Facebook adapter. The exposed table names and their definitions are below. Table 27: ACCOUNT Column Name SQL Data Type id name 2016 SAP SE or an SAP affiliate company. All rights reserved. 137

Column Name category accesstoken SQL Data Type Table 28: BOOK Column Name id name category createdtime SQL Data Type TIMESTAMP Table 29: FAMILY Column Name id name category SQL Data Type Table 30: FRIEND Column Name id name gender email firstname middlename lastname locale link username thirdpartyid SQL Data Type 138 2016 SAP SE or an SAP affiliate company. All rights reserved.

Column Name timezone updatedtime verified bio birthday political quotes relationnshipstatus religion website SQL Data Type DOUBLE TIMESTAMP TINYINT Table 31: ME Column Name id name firstname middlename lastname gender locale link username thirdpartyid timezone updatedtime bio birthday SQL Data Type DOUBLE TIMESTAMP 2016 SAP SE or an SAP affiliate company. All rights reserved. 139

Column Name cover education email SQL Data Type Table 32: MOVIE Column Name id name category createdtime SQL Data Type TIMESTAMP Table 33: MUSIC Column Name id name category createdtime SQL Data Type TIMESTAMP Table 34: TELEVISION Column Name id name category createdtime SQL Data Type TIMESTAMP 8.4 Apache Camel Informix Use the Camel Informix adapter to connect to an IBM Informix remote source. You can connect to an Informix source using the Camel Informix adapter. 140 2016 SAP SE or an SAP affiliate company. All rights reserved.

Note Before registering your adapter with the SAP HANA system, be sure that you have downloaded and installed the correct JDBC libraries. This adapter supports the following functionality: Virtual table as a source Virtual table as a target using a Data Sink node in a flowgraph 8.4.1 Set Up the Camel Informix Adapter Set up the Camel Informix adapter, prior to using it. Context By default, Camel Informix Adapter is not available in Data Provisioning Agent. To use it, you must do the following: Procedure 1. Open <DPAgent_root>/camel/adapters.xml and uncomment the configuration of Camel Informix Adapter. <Adapter type="cameldatabaseadapter" displayname="camel database Adapter"> <RemoteSourceDescription> <PropertyGroup name="configuration" displayname="configuration"> <PropertyEntry name="host" displayname="host" description="host name" isrequired="true"/> <PropertyEntry name="port" displayname="port" description="port number" isrequired="true"/> <PropertyEntry name="dbname" displayname="database" description="database Name" isrequired="true"/> <PropertyEntry name="servername" displayname="server Name" description="server Name" isrequired="false"/> <PropertyEntry name="delimident" displayname="delimident" description="delimident" isrequired="false"/> </PropertyGroup> <CredentialEntry name="db_credential" displayname="credential" userdisplayname="user" passworddisplayname="password"/> </RemoteSourceDescription> <Capabilities> <Informix List of Capabilities> </Capabilities> <RouteTemplate>jdbc.xml</RouteTemplate> </Adapter> 2. Copy <DPAgent_root>/camel/samples/sample-jdbc.xml to <DPAgent_root>/camel/ and rename to jdbc.xml. 2016 SAP SE or an SAP affiliate company. All rights reserved. 141

3. Download camel-jdbc.jar from http://camel.apache.org/download.html, and copy it to the <DPAgent_root>/camel/lib directory. 4. Download the Informix jdbc driver ifxjdbc.jar, and put it in the <DPAgent_root>/camel/lib directory. 8.4.2 Camel Informix Remote Source Configuration Configure the following options in smart data access to configure your connection to an Informix remote source. Category Option Description Configuration Host Host Port Port Database Name Database name Server Name Server name Delimident If set to True, the Informix database object name will be enclosed with double quotation marks. Credentials Credentials Mode technical user or secondary user Select one of the choices, depending on the purpose of the remote source you want to create. User Credential User The Informix user name Password The Informix user name's password 8.4.3 Camel Informix Remote Source SQL Configuration Configure your Informix remote source using SQL. The following is an example of SQL code used to configure an Informix remote source. CREATE REMOTE SOURCE "CamelInformixAdapter" ADAPTER "CamelInformixAdapter" AT LOCATION AGENT "mexcel-agentlocal" CONFIGURATION '<?xml version="1.0" encoding="utf-8" standalone="yes"?> <ConnectionProperties name="configurations"> <PropertyGroup name="file System" displayname="file System"> <PropertyEntry name="folder" displayname="folder" isrequired="true">demo</ PropertyEntry> </PropertyGroup> <PropertyGroup name="table" displayname="table"> <PropertyEntry name="host" displayname="host"><host_name></propertyentry> <PropertyEntry name="port" displayname="port"><port_number></propertyentry> <PropertyEntry name="dbname" displayname="db name"><database_name></ PropertyEntry> <PropertyEntry name="servername" displayname="server name"><server_name></ PropertyEntry> 142 2016 SAP SE or an SAP affiliate company. All rights reserved.

<PropertyEntry name="delimident" displayname="delimident">true</ PropertyEntry> </PropertyGroup> </ConnectionProperties>' WITH CREDENTIAL TYPE 'PASSWORD' USING '<CredentialEntry name="db_credential"> <user><informix_user></user> <password><password></password> </CredentialEntry>'; 8.4.4 Camel Informix to SAP HANA Data Type Conversion Data type conversion between Informix and SAP HANA. Camel Informix BIGINT BIGSERIAL BLOBcan be up to 4 terabytes BOOLEAN BYTE The BYTE data type has no maximum size. A BYTE column has a theoretical limit of 231 CHAR(n) 1 n 32,767 CHARACTER(n) CHARACTER VARYING(m,r) CLOB 4 terabytes DATE DATETIME DEC DECIMAL DOUBLE DOUBLE FLOAT(n) IDSSECURITYLABEL INT INT8 SAP HANA BIGINT BIGINT BLOB(2G) TINYINT BLOB(2G) (1-5000)/CLOB(2GB) (1-5000)/CLOB(2GB) (1-5000)/CLOB(2GB) CLOB(2GB) DATE TIMESTAMP DECIMAL DECIMAL PRECISION REAL/DOUBLE (128) INTEGER BIGINT 2016 SAP SE or an SAP affiliate company. All rights reserved. 143

Camel Informix INTEGER L(m) <32,739 bytes NCHAR(n) NUMERIC(p,s) N(m,r) REAL SERIAL(n) SERIAL8(n) SMALLFLOAT SMALLINT TEXT (m,r) <255 bytes SAP HANA INTEGER (1-5000)/CLOB(2GB) (1-5000)/CLOB(2GB) DECIMAL(p,s) (1-5000)/CLOB(2GB) REAL INTEGER BIGINT REAL SMALLINT TEXT (1-5000)/CLOB(2GB) 8.4.5 SQL Conversion Convert Informix SQL to SAP HANA SQL In some cases, Informix SQL and SAP HANA SQL is not compatible. The following table lists the conversions that may take place: Informix $1::DECIMAL $1::DECIMAL($2) $1::REAL $1::INT $1::BIGINT $1::SMALLINT $1::INTEGER SAP HANA TO_DECIMAL($1) TO_DECIMAL($1,$2) TO_REAL($1) TO_INT($1) TO_BIGINT($1) TO_SMALLINT($1) TO_INTEGER($1) 144 2016 SAP SE or an SAP affiliate company. All rights reserved.

8.5 IBM DB2 for z/os The IBM DB2 for z/os adapter is a data provisioning adapter that provides DB2 client access to the database deployed on IBM DB2 for z/os system with its service provider and data/metadata. The z/os DB2 service provider is created as a data provisioning remote source. DB2 database resources are exposed as remote objects of the remote source. These remote objects can be added as data provisioning virtual tables. The collection of DB2 data entries are represented as rows of the virtual table. Adapter Functionality This adapter supports the following functionality: Virtual table as a source Virtual table as a target using a Data Sink in a flowgraph (Only INSERT is supported) In addition, this adapter supports the following capabilities: Table 35: Global Settings Functionality SELECT from a virtual table INSERT into a virtual table Execute UPDATE statements on a virtual table Execute DELETE statements on a virtual table Different capabilities per table Different capabilities per table column Realtime Supported? Yes Yes No No No No No Table 36: Select Options Functionality Select individual columns Add a WHERE clause JOIN multiple tables Aggregate data via a GROUP BY clause Support a DISTINCT clause in the select Supported? Yes Yes Yes Yes Yes 2016 SAP SE or an SAP affiliate company. All rights reserved. 145

Functionality Supported? Support a TOP or LIMIT clause in the select Yes ORDER BY Yes GROUP BY Yes 8.5.1 Setting DB2 UDB Environment Variables The method to set DB2 UDB environment variables depends on the operating system. For Linux, the DB2 UDB installation provides two scripts for setting up the DB2 UDB environment variables: db2cshrc for C shell and db2profile for Bourne or Korn shell. These scripts set the library path environment variable based on the bit size of the installed server or client. For Windows, the installation sets all necessary environment variables. For Linux platforms, the 32-bit and 64-bit versions of the driver and API libraries are located in <$HOME/sqllib/ lib32> and <$HOME/sqllib/lib64>, respectively, where <$HOME> is the home directory of the DB2 UDB instance owner. Note If the Data Provisioning Agent is installed on Linux, the library path environment variable must point to the 64- bit libraries. For Windows, the library path environment variable must point to the 32-bit libraries. Note We recommend that you add a line to the <DPAgent_root>/bin/dpagent_env.sh file to source the db2profile. This will ensure that when you use dpagent_service.sh to start and stop the DPAgent service, the DB2 UDB environment variables will be sourced automatically. For example, you could add a line such as source /home/db2inst1/sqllib/db2profile 8.5.2 Bind the DB2 SYSHL Package You may need to bind the DB2 SYSHL package. Context If you receive the following error from the adapter, follow these steps to bind the DB2 SYSHL package: DB2 SQL Error: SQLCODE=-805, SQLSTATE=51002, SQLERRMC=DB1SLOC.NULLID.SYSLH 146 2016 SAP SE or an SAP affiliate company. All rights reserved.

Procedure 1. Install and open the IBM DB2 Configuration Assistant. 2. In the source window, connect to the DB2 database deployed on the z/os mainframe system. 3. Test the connection 4. Bind the DB2 SYSHL package through the Configuration Assistant 5. Select ddcsmvs.lst as the bind file location, enter the connection username and password, and then click Bind button. 8.5.3 Preparing JDBC Files Prepare the IBM DB2 JDBC JAR files to use the IBM DB2 for z/os adapter. In order to use the IBM z/os DB2 adapter, you are required to copy the following IBM DB2 JDBC JAR files to the /lib folder of the Data Provisioning Agent installation directory (<DPAgent_root>\lib). db2jcc4.jar (Required) You can download this file here: http://www-01.ibm.com/support/docview.wss?uid=swg21363866 sure to download the JDBC JAR file according to your DB2 database version. db2jcc_license_cisuz.jar (Required) You can find information about this file here: http://www-01.ibm.com/support/docview.wss? uid=swg21191319. Be You can find these JAR files in the following ways: These JAR files will be available in the installation directory after you have installed the IBM DB2 client. For example, on a Windows System, the JAR files will be located in C:\Program Files\IBM\SQLLIB\java Download them from the IBM Support and Download Center. Note If the source z/os DB2 system contains a non-english CCSID table space, you are required to update the JVM to an internationalized version. At a minimum, the charsets.jar file within the current JVM should contain the required CharToByteCP<XXX>.class, where <XXX> corresponds to the source system s language locale. 8.5.4 IBM DB2 z/os Remote Source Configuration Options for connecting to the remote z/os data server. Category Option Description Database Host Host name or IP address on which the remote DB2 data server is running. Port Number The DB2 data server port number. 2016 SAP SE or an SAP affiliate company. All rights reserved. 147

Category Option Description z/os DB2 Additional Info Database Name Database Source Name Bind Packages The DB2 database name. The DB2 instance name. When this option is set to Yes, the DB2 z/os adapter automatically checks and binds all of the required missing JAR files. We recommend that you set this option to Yes the first time you attempt to establish a connection, then set this option to No for subsequent attempts. Note If any necessary packages are missing, an error occurs. Credential Properties Credentials Mode User Name Password Setting this option for subsequent attempts will help improve performance by eliminating the need for redundant checks for the necessary JDBC packages. technical user or secondary user Select one of the choices, depending on the purpose of the remote source you want to create. The DB2 user that has access to the tables that will be added as virtual tables in HANA. DB2 database user password. 8.5.5 IBM DB2 z/os to SAP HANA Data Type Mapping DB2 z/os Data Type BIGINT BINARY BLOB CHAR CLOB DATE DBCLOB DECIMAL SAP HANA Data Type BIGINT VARBINARY BLOB CLOB DATE CLOB DECIMAL 148 2016 SAP SE or an SAP affiliate company. All rights reserved.

DB2 z/os Data Type DECFLOAT SAP HANA Data Type DOUBLE DOUBLE DOUBLE GRAPHIC INTEGER LONGVARBINARY N INTEGER BLOB LONG CLOB LONGVARGRAPHIC NCLOB REAL ROWID NCLOB NCLOB REAL INTEGER SMALLINT TIME TIMESTAMP VARBINARY SMALLINT TIME TIMESTAMP VARBINARY VARGRAPHIC N 8.6 File Use the File adapter to read formatted and free form text files. The File adapter enables SAP HANA users to read formatted and free form text files which are accessible on the computer running the adapter with the operating user permissions the adapter/agent had been started under. In order to specify the file format (for example, delimited character), a configuration file (.cfg) has to be created containing this information. Then each file can be read and parsed through this format, returning the data in columns of a virtual table. For free form text (unstructured) files, you do not need to designate a file format definition, and you can use the FILECONTENT virtual table to view the data. Note Data files must be located in the same directory where the data provisioning agent was installed. (<DPAgent_root>/ (dataprovagent) directory, or a sub-directory) 2016 SAP SE or an SAP affiliate company. All rights reserved. 149

Adapter Functionality This adapter supports the following functionality: Virtual table as a source Virtual table as a target using a Data Sink in a flowgraph (Only INSERT is supported) HDFS target file support Parallel readers (Partitioning) In addition, this adapter supports the following capabilities: Table 37: Global Settings Functionality SELECT from a virtual table INSERT into a virtual table Execute UPDATE statements on a virtual table Execute DELETE statements on a virtual table Different capabilities per table Different capabilities per table column Realtime Supported? Yes Yes No No Yes Yes No Table 38: Select Options Functionality Select individual columns Add a WHERE clause JOIN multiple tables Aggregate data via a GROUP BY clause Support a DISTINCT clause in the select Support a TOP or LIMIT clause in the select ORDER BY GROUP BY Supported? Yes Yes No No No Yes No No Related Information Remote Source Tables [page 156] 150 2016 SAP SE or an SAP affiliate company. All rights reserved.

HDFS Target Files [page 156] 8.6.1 Configure the File Adapter Follow these steps to configure the File adapter. Context The File adapter is already deployed with the Data Provisioning Agent that you installed. However, you will need to configure and register the adapter. Procedure 1. Launch dpagentconfigtool.exe under <DPAgent_root>/configTool. 2. Go to Configure Preferences, and click FileAdapter. 3. Enter configuration information for your adapter. 4. Click Apply to save the changes. Next Steps Now, you can register your adapter with the Data Provisioning Agent. Related Information FileAdapter Preferences [page 152] 2016 SAP SE or an SAP affiliate company. All rights reserved. 151

8.6.1.1 FileAdapter Preferences Configuration parameters for the File adapter. Parameter Root directory File format root directory Access Token Description The root directory for your data files. This is used for security. No remote source can reach beyond this directory for data files. The root directory for your File Format definitions. This is used for security. No remote source can reach beyond this directory for data files. A password. An access token protects the files from access from different agents. Use this same password when creating a remote source. 8.6.2 File Format Configuration Files Create a file format configuration file to work with your data file. Each configuration file is a text file and must match the following format: The first line must contain a comment or a description. This line will be ignored for processing. Then a set of key-value-pairs to specify the various parsing options A set of COLUMN=<name>;<Hana datatype>;<optional description> Example Fileformat to read US census data, see https://www.census.gov/econ/cbp/download/ FORMAT=CSV FORCE_FILENAME_PATTERN=us_county_census_%.txt FORCE_DIRECTORY_PATTERN= CODEPAGE= LOCALE= COLUMN_DELIMITER=, ROW_DELIMITER=\r\n ESCAPE_CHAR= TEXT_QUOTES=" TEXT_QUOTES_ESCAPE_CHAR= SKIP_HEADER_LINES=1 QUOTED_TEXT_CONTAIN_ROW_DELIMITER=false DATEFORMAT=yyyy.MM.dd HH:mm:ss COLUMN=FIPSTATE;(2);FIPS State Code COLUMN=EMP;INTEGER;Total Mid-March Employees with Noise COLUMN=QP1_NF;(1);Total First Quarter Payroll Noise Flag COLUMN=CENSTATE;(2);Census State Code COLUMN=CENCTY;(3);Census County Code COLUMN=PARTITION COLUMNSTARTENDPOSITION= Fixed Format files Fixed file formats are also supported. (FORMAT=fixed). The formatting can be specified using the COLUMSTARTENDPOSITION, or the ROW_DELIMITER and ROWLENGTH parameters. 152 2016 SAP SE or an SAP affiliate company. All rights reserved.

Example Fixed Fileformat FORMAT=FIXED FORCE_FILENAME_PATTERN=fixed%.txt CODEPAGE=UTF-8 LOCALE= ROW_DELIMITER=\r\n SKIP_HEADER_LINES=1 COLUMNSSTARTENDPOSITION=0-1;2-7;8-15 ROWLENGTH=16 COLUMN=COL1;(2) COLUMN=COL2;(6) COLUMN=COL3;(8) 8.6.3 Generate the Configuration File Use a Data Provisioning utility to create the CFG file. Procedure 1. Navigate to <DPAgent_root>\agentutils 2. Run the following from command line: For Windows: createfileformat.bat -file <PATH TO DATA FILE> -cfgdir <DIRECTORY TO STORE GENERATED CFG FILES> <OPTION> <VALUE>... For UNIX: createfileformat.sh -file <PATH TO DATA FILE> -cfgdir <DIRECTORY TO STORE GENERATED CFG FILES> <OPTION> <VALUE>... Only the -file and -cfgdir parameters are required. The value for the -file parameter is the path to the directory containing one or more data files or path to a single file name for which the configuration files need to be generated. The value for the -cfgdir is the path to the output directory where the generated configuration files will be stored. A number of options and value pairs can be provided as additional parameters. The following are supported: Parameter -coldelimiter -rowdelimiter -txtdelimiter -escchar Description The column delimiter The row delimiter Text quotes Escape character 2016 SAP SE or an SAP affiliate company. All rights reserved. 153

Parameter -txtescchar -dateformat -timeformat -seconddateformat Description Quote escape character Default date format Default time format Default second date format If no options are provided as parameters, the default delimiters are: Delimiter Character Column delimiter, Row delimiter \n (UNIX) \r\n (Windows) Escape character \ Quote escape character " Default text quotes " Note Only one format of each type (date, time, second date) is allowed per file. So, if you have two columns containing different formatted dates in it, only the first one will be recognized. The second will be Varchar. Example Run this tool to generate a configuration file named call_center.dat that has ';' as a column delimiter and '\n' as a row delimiter: createfileformat.sh -file C:\usr\sap\dataprovagent\sasi\FileServer \call_center.dat -cfgdir C:\usr\sap\dataprovagent\sasi\FileDefinitions -coldelimiter ; - rowdelimiter \n 8.6.4 File Adapter Remote Source Configuration File adapter remote source configuration parameters. The active parameters may change, depending on which options you choose. Category Option Description ConnectionInfo Root Directory The root directory for your data files. This is used for security. No remote source can reach beyond this directory for data files. 154 2016 SAP SE or an SAP affiliate company. All rights reserved.

Category Option Description Directory of the file format definitions Target Options Location where you store your file format definition files. Local File System: Specifies that the target file is on a local system. Remote HDFS: Specifies that the target file is on a remote HDFS system Host Name The remote URL to connect to the remote HDFS, usually defined in the core-site.xml.. Port Number Port Number to connect to remote HDFS, usually defined in the coresite.xml. Target File Location The location where to store copied target files in a remote HDFS. For example, you might want to copy a target file from a local to a remote HDFS. Hadoop User Name The user of Hadoop system when loging in. It can also be the owner of the file location you want to put the copied target files. Make sure you have the proper permissions to access this location. Credentials Credentials Mode technical user or secondary user Select one of the choices, depending on the purpose of the remote source you want to create. AccessTokenEntry AccessToken A password. An access token protects the files from access from different agents. Use this same password when creating a remote source. Related Information HDFS Target Files [page 156] 2016 SAP SE or an SAP affiliate company. All rights reserved. 155

8.6.5 Remote Source Tables The remote source provides certain tables. After a remote source is created, you can browse the remote source. Each of the configured CFG files is shown as a remote table under the remote source and can be imported as a virtual table. The following tables are always included: Table CODEPAGES FILECONTENT FILECONTENTROWS FILECONTENTTEXT FILEDIRECTORY LOCALES STATISTICS_CHAR Description Use this table to retrieve all supported codepages of the Java installation and optionally specify one in the various file format configuration files. The codepage controls the used character encodings of the source files. This virtual table has one row per file and the entire file content is inside a BLOB column. Use this table for unstructured data files. Similar to FILECONTENT, this table returns the data as is, without any conversion, but splits the file at every <newline> character into a new row. This virtual table has one row per file, and the entire file content is inside a NCLOB column. Use this table for unstructured data files. In case the file has no ByteOrderMark (BOM) header to identify the codepage, or the operating system default codepage is not the proper one, the reader option CODEPAGE can be supplied. Returns the list of all files in the remote source configured root directory and its sub directories. This table returns all supported Java locales, and the values can be used to control the locale of the file read, which impacts the decimal format, the month names of a date format, and so on. Calculates the number of occurrences of each character in the files. Characters that occur often will be column delimiters, optional text quotes, and row delimiter characters. 8.6.6 HDFS Target Files You can write to an HDFS target file using the File adapter. The File adapter supports target files copied from a local machine to a remote HDFS when processing commit changes after finishing an insert operation. You must configure your HDFS target file using the File adapter remote source configuration parameters. An HDFS target accepts all file formats that are already supported in the File adapter. Related Information File Adapter Remote Source Configuration [page 154] 156 2016 SAP SE or an SAP affiliate company. All rights reserved.

8.7 Google+ Access Google+ data with the GoogleAdapter. Use the Google+ adapter to access social information from the Google+ website. This adapter supports the following functionality: Virtual table as a source Set up the Google+ Adapter Before you can create a Google+ remote source, you will need to acquire a Google+ user ID and a verification code to use when creating a Google+ remote source. 1. Create a Google account to obtain a user ID (a Google e-mail address).. 2. Copy and paste the following URL into your browser: https://accounts.google.com/o/oauth2/auth? access_type=offline&approval_prompt=force&client_id=272692714371- lk8ugf7fbourfojrhg68sno2tle3c82q.apps.googleusercontent.com&redirect_uri=urn:ietf:wg:oauth: 2.0:oob&response_type=code&scope=https://www.googleapis.com/auth/plus.login 3. Sign in to your Google account. 4. Click the Allow button. (You will not be able to retrieve your verification code if you click Deny) 5. Copy and paste your verification code into the Google+ Verification Code parameter when creating your remote source. Related Information Google+ Remote Source Configuration [page 157] 8.7.1 Google+ Remote Source Configuration Configure a remote source to access Google+ data. Configure the following options in smart data access to configure your connection to your Google+ source. Option Description HTTP Proxy Host Enter the HTTP proxy host name. HTTP Proxy Port Enter the HTTP proxy port number. 2016 SAP SE or an SAP affiliate company. All rights reserved. 157

Option Google+ User ID Google+ Verification Code Description The user ID for the Google+ account being used. This is your Google e-mail address. The verification code you are given when accessing the following URL (Existing Google account neccessary for access.) https://accounts.google.com/o/oauth2/auth?access_type=offline&approval_prompt=force&client_id=272692714371-lk8ugf7fbourfojrhg68sno2tle3c82q.apps.googleusercontent.com&redirect_uri=urn:ietf:wg:oauth: 2.0:oob&response_type=code&scope=https://www.googleapis.com/auth/plus.login Note You may need to copy and paste the URL into a browser. 8.7.2 Google+ Remote Source SQL Configuration Example SQL code for creating a Google+ remote source. The following is an example of the SQL code to use for creating a Google+ remote source. CREATE REMOTE SOURCE "GP_REMOTE_SOURCE" ADAPTER "GooglePlusAdapter" AT LOCATION AGENT "GooglePlusAgent" CONFIGURATION '<?xml version="1.0" encoding="utf-8" standalone="yes"?> <ConnectionProperties name="configurations" displayname="configurations"> <PropertyGroup name="proxy" displayname="proxy Setting" isrequired="false"> <PropertyEntry name="httpproxyhost" displayname="http Proxy Host"><Host Name></PropertyEntry> <PropertyEntry name="httpproxyport" displayname="http Proxy Port"><Port></PropertyEntry> </PropertyGroup> </ConnectionProperties>' WITH CREDENTIAL TYPE 'PASSWORD' USING '<CredentialEntry name="credential"> <user><user ID></user> <password displayname="google plus Verification Code"><Password></password> </CredentialEntry>'; 8.7.3 Google+ Tables Access Google+ tables to retrieve data from a Google+ source People Table The following tables show you the kind of data that you can retrieve from a Google+ source. 158 2016 SAP SE or an SAP affiliate company. All rights reserved.

Table 39: People Table Column Name kind nickname occupation skills birthday gender objecttype id displayname name.formatted name.familyname name.givenname name.middlename name.honorificprefix name.honorificsuffix tagline braggingrights aboutme relationshipstatus url image.url isplususer language agerange.min agerange.max plusonecount Data Type N N N N N N N N N N N TINYINT N INTEGER INTEGER INTEGER 2016 SAP SE or an SAP affiliate company. All rights reserved. 159

Column Name circledbycount verified domain Data Type INTEGER TINYINT N Table 40: Emails Table Column Name value type Data Type Table 41: URLs Table for People Column Name value type label Data Type Table 42: Organizations Table Column Name name department title type startdate enddate location description primary Data Type N N N N N TINYINT 160 2016 SAP SE or an SAP affiliate company. All rights reserved.

Activities Table Column Name kind title published updated id url actor.id actor.displayname actor.name.familyname actor.name.givenname actor.url actor.image.url verb object.objecttype object.id object.actor.id object.actor.displayname object.actor.url object.actor.image.url object.content object.originalcontent object.url object.replies.totalitems object.replies.selflink object.plusoners.totalitems Data Type N TIMESTAMP TIMESTAMP N N N N N N INTEGER INTEGER 2016 SAP SE or an SAP affiliate company. All rights reserved. 161

Column Name object.plusoners.selflink object.resharers.totalitems object.resharers.selflink annotation crosspostsource provider.title access.kind access.description geocode address radius placeid placename location.kind location.id location.displayname location.position.latitude location.position.longitude location.address.formatted Data Type INTEGER N N N N N N N Attachments Table Column Name objecttype displayname id Data Type N 162 2016 SAP SE or an SAP affiliate company. All rights reserved.

Column Name content url image.url image.type image.height image.width fullimage.url fullimage.type fullimage.height fullimage.width embed.url embed.type Data Type N INTEGER INTEGER INTEGER INTEGER Comments Table Column Name kind verb id published updated actor.id actor.displayname actor.url actor.image.url object.objecttype Data Type TIMESTAMP TIMESTAMP N 2016 SAP SE or an SAP affiliate company. All rights reserved. 163

Column Name Data Type object.content N object.originalcontent N selflink plusoners.totalitems INTEGER 8.8 Hive The Hive adapter supports Hadoop via Hive. Hive is the data warehouse that sits on top of Hadoop and includes a SQL interface. While Hive SQL does not fully support all SQL operations, the majority of SELECT features are available. Because Hadoop stores its data in distributed files using one of a number of file formats, INSERT, UPDATE and DELETE functions are more complicated than with standard SQL, and therefore they are not supported in this first version of the Hive Adapter. Transactions are currently not supported by Hive. The Hive adapter service provider is created as a remote source, and requires the support of artifacts like virtual table and remote subscription for each source table to perform replication. Note Before registering your adapter with the SAP HANA system, be sure that you have downloaded and installed the correct JDBC libraries. For information about the proper JDBC library for your source, see the Product Availability Matrix. The Product Availability Matrix also provides information about supported Hive versions. Place your files in <DPAgent_root>/lib, and you will need to manually create the /lib folder. Adapter Functionality This adapter supports the following functionality: Virtual table as a source In addition, this adapter supports the following capabilities: Table 43: Global Settings Functionality Supported? SELECT from a virtual table Yes INSERT into a virtual table No 164 2016 SAP SE or an SAP affiliate company. All rights reserved.

Functionality Execute UPDATE statements on a virtual table Execute DELETE statements on a virtual table Different capabilities per table Different capabilities per table column Realtime Supported? No No No No No Table 44: Select Options Functionality Select individual columns Add a WHERE clause JOIN multiple tables Aggregate data via a GROUP BY clause Support a DISTINCT clause in the select Support a TOP or LIMIT clause in the select ORDER BY GROUP BY Supported? Yes No Yes Yes No No Yes Yes 8.8.1 Hive Remote Source Configuration Configure the following options in smart data access to configure your connection to a Hive remote source. Category Option Description Connection Hive Version The version of Hive. For Hive 0.12.0, set its value to 0.12.0; For Hive 0.13.0 and 0.13.1, set its value to 0.13.1. Host Port Database Name The host of Hive The port number of Hive The Hive database name Security Use SSL Specifies whether to use SSL. The default value is false. 2016 SAP SE or an SAP affiliate company. All rights reserved. 165

Category Option Description Additional Additional JDBC settings that are added directly to the JDBC URL. The parameters must be specified in the following format: key=value,key=value,... Logon Mechanism Controls the authentication (+authorization) method used, and which username and password values to enter in the credentials field below. Default Kerberos: If set to Kerberos, the Kerberos Realm, Kerberos KDC, and Kerberos Principal settings are used when making the Hive connection. Kerberos Realm (Optional when using Kerberos) Authenticate using a principal from this realm (instead of the systems default realm). The Realm option must be used together with KDC. Kerberos KDC (Optional when using Kerberos) The address of the KDC (Key Distribution Center) to be used with the specified Realm (has to be used together with Realm) Credentials Kerberos Principal Credentials Mode The Kerberos principal. Speeds up retrieving large amounts of data from Hive (disabled when using Kerberos authentication) technical user or secondary user Select one of the choices, depending on the purpose of the remote source you want to create. JDBC Credentials Username The Hive user name Password The Hive user password 166 2016 SAP SE or an SAP affiliate company. All rights reserved.

8.8.2 Hive to SAP HANA Data Type Mapping The following table shows data type mappings from Hive to SAP HANA. Hive Data Type SAP HANA Data Type Notes TINYINT TINYINT 1-byte signed integer, from -128 to 127 SMALLINT SMALLINT 2-byte signed integer, from -32,768 to 32,767 INT INT 4-byte signed integer, from -2,147,4STRING83,648 to 2,147,483,647 BIGINT BIGINT 8-byte signed integer, from -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807 FLOAT DOUBLE 4-byte single precision floating point number DOUBLE DOUBLE 8-byte double precision floating point number range (approximately -10 308 to 10 308 ) or very close to zero (-10-308 to 10-308 ) DECIMAL DECIMAL Introduced in Hive 0.11.0 with a precision of 38 digits Hive 0.13.0 introduced user definable precision and scale TIMESTAMP TIMESTAMP Timestamp format "YYYY-MM-DD HH:MM:SS.fffffffff" (9 decimal place precision) DATE DATE DATE values describe a particular year/ month/day, in the form {{YYYY- MM- DD}} Only available starting with Hive 0.12.0 STRING Only available starting with Hive 0.12.0 CHAR CHAR Only available starting with Hive 0.13.0 BOOLEAN BINARY VRCHAR VARBINARY 8.9 IBM DB2 Log Reader The DB2 Log Reader adapter provides real-time changed-data capture capability to replicate changed data from a database to SAP HANA in real time. You can also use it for batch loading. 2016 SAP SE or an SAP affiliate company. All rights reserved. 167

The Log Reader service provider is created as a remote source, and it requires the support of artifacts like virtual tables and remote subscriptions for each source table to perform replication. Note Before registering your adapter with the SAP HANA system, be sure that you have downloaded and installed the correct JDBC libraries. For information about the proper JDBC library for your source, see the Product Availability Matrix. Place your files in <DPAgent_root>/lib, and you will need to manually create the /lib folder. Adapter Functionality This adapter supports the following functionality: Multiple instances of the Log Reader adapter are supported for each Data Provisioning Agent. Virtual table as a source Real-time change data capture Note Log Reader adapters do not support the truncate table operation. Virtual table as a target using a Data Sink node in a flowgraph Parallel reader (flowgraph and replication) Loading options for target tables DDL propagation. The supported schema changes are: ADD COLUMN DROP COLUMN Replication monitoring and statistics Search for tables In addition, this adapter supports the following capabilities: Table 45: Global Settings Functionality Supported? SELECT from a virtual table Yes INSERT into a virtual table Yes Execute UPDATE statements on a virtual table Yes Execute DELETE statements on a virtual table Yes Different capabilities per table No Different capabilities per table column No 168 2016 SAP SE or an SAP affiliate company. All rights reserved.

Functionality Supported? Realtime Yes Table 46: Select Options Functionality Supported? Select individual columns Yes Add a WHERE clause Yes JOIN multiple tables Yes Aggregate data via a GROUP BY clause Yes Support a DISTINCT clause in the select Yes Support a TOP or LIMIT clause in the select Yes ORDER BY Yes GROUP BY Yes 8.9.1 Remote Database Setup for DB2 Realtime The remote database must be set up properly for Log Reader adapters to function correctly when using realtime replication. This setup process is necessary only when using realtime. Remember the following when setting up for replication: If you have a UDB client instance and a UDB server instance on different machines, the client and server must be of the same UDB version. On a Windows system, the DB2 connectivity autocommit parameter must be turned on (automcommit=1). The autocommit parameter is specified in the DB2 call level interface (CLI) configuration file for the primary database. If the autocommit parameter is not turned on, a deadlock problem can occur. The path to the CLI configuration file is: %DB2DIR% \sqllib\db2cli.ini where %DB2DIR% is the path to the DB2 client installation. Alternatively, to turn on autocommit, open the DB2 administrative command line console and run: db2set DB2OPTIONS=+c To initialize Replication Agent without error, the database must have a tablespace created with these characteristics: The tablespace should be a user temporary tablespace. By default, user temporary tablespaces are not created when a database is created. The tablespace must be a system-managed space (SMS). The PAGESIZE parameter must be set to 8192 (8 kilobytes) or greater. 2016 SAP SE or an SAP affiliate company. All rights reserved. 169

8.9.1.1 Setting DB2 UDB Environment Variables The method to set DB2 UDB environment variables depends on the operating system. For Linux, the DB2 UDB installation provides two scripts for setting up the DB2 UDB environment variables: db2cshrc for C shell and db2profile for Bourne or Korn shell. These scripts set the library path environment variable based on the bit size of the installed server or client. For Windows, the installation sets all necessary environment variables. For Linux platforms, the 32-bit and 64-bit versions of the driver and API libraries are located in <$HOME/sqllib/ lib32> and <$HOME/sqllib/lib64>, respectively, where <$HOME> is the home directory of the DB2 UDB instance owner. Note If the Data Provisioning Agent is installed on Linux, the library path environment variable must point to the 64- bit libraries. For Windows, the library path environment variable must point to the 32-bit libraries. Note We recommend that you add a line to the <DPAgent_root>/bin/dpagent_env.sh file to source the db2profile. This will ensure that when you use dpagent_service.sh to start and stop the DPAgent service, the DB2 UDB environment variables will be sourced automatically. For example, you could add a line such as source /home/db2inst1/sqllib/db2profile 8.9.1.2 Installing Data Provisioning Agent and DB2 Server on Different Servers Additional steps are necessary when installing the Data Provisioning Agent and DB2 Server on different servers. 8.9.1.2.1 DB2 Connectivity The method for configuring DB2 connectivity varies by operating system. On a Windows system, you must configure a DB2 Universal Database JDBC data source in the DB2 Administration Client, then use the database name and database alias specified for that DB2 Universal Database JDBC data source when you configure DB2 LogReader Adapter connectivity. On a Linux system, catalog the node and the primary database in DB2. Set the DB2 LogReader Adapter pds_datasource_name parameter to the database alias. Also set the pds_host_name and pds_host_number. 170 2016 SAP SE or an SAP affiliate company. All rights reserved.

8.9.1.2.2 Catalog the Remote TCP/IP Node from the DB2 Client Follow these steps to catalog the remote TCP/IP node from the DB2 client. Procedure 1. Log in as the DB2 instance owner. (For Linux only) Logging in sets up your DB2 environment variables by executing the environment scripts. You can also execute these scripts manually as follows. In Korn shell, source the db2profile file: <$HOME>/sqllib/db2profile In C shell, source the db2cshrc file: source <$HOME>/sqllib/db2cshrc (where $HOME is the home directory of the DB2 instance owner.) 2. Start the DB2 command-line processor by typing the following DB2 command: %>db2 db2 => 3. Catalog the remote TCP/IP node using this command at the DB2 prompt: db2 => catalog TCPIP node MYNODE remote MYHOST server XXXX (where MYNODE is the node name, MYHOST is the host name or IP address of the data server, and XXXX is the data server port number.) 4. Verify the catalog entry: db2 => list node directory DB2 should return something similar to: Node 1 entry: Node name = MYNODE Comment = Directory entry type = LOCAL Protocol = TCPIP Hostname = MYHOST Service name = XXXX 8.9.1.2.3 Catalog the Primary Database from the DB2 Client Follow these steps to catalog the primary database from the DB2 client. Procedure 1. Catalog the primary database using this command at the DB2 prompt: db2 => catalog database MYDB as MYDB_ALIAS at node MYNODE Parameter MYDB MYDB_ALIAS MYNODE Description Database name Alias for the database The node name used in the catalog TCPIP node command 2. Verify the catalog entry: db2 => list database directory 2016 SAP SE or an SAP affiliate company. All rights reserved. 171

DB2 should return something similar to: System Database Directory Number of entries in the directory Database 1 entry 1 Database alias = MYDB_ALIAS Database name = MYDB Node name = MYNODE Database release level = b.00 Comment = Directory entry type = Remote 8.9.1.3 Primary DB2 UDB Database Configuration for Replication Configure your DB2 UDB database to work with the Log Reader adapter and replication. For successful replication, remember the following: If you have a UDB client instance and a UDB server instance on different machines, the client and server must be of the same UDB version. On a Windows system, the DB2 connectivity autocommit parameter must be turned on (autocommit=1). The autocommit parameter is specified in the DB2 call level interface (CLI) configuration file for the primary database. If the autocommit parameter is not turned on, a deadlock problem can occur. The path to the CLI configuration file is: %DB2DIR% \sqllib\db2cli.ini where %DB2DIR% is the path to the DB2 client installation. Alternatively, to turn on autocommit, open the DB2 administrative command line console and run: db2set DB2OPTIONS=+c To initialize Replication Agent without error, the database must have a tablespace created with these characteristics: The tablespace should be a user temporary tablespace. By default, user temporary tablespaces are not created when a database is created. The tablespace must be a system-managed space (SMS). The PAGESIZE parameter must be set to 8192 (8 kilobytes) or greater. 172 2016 SAP SE or an SAP affiliate company. All rights reserved.

8.9.1.3.1 Add a Temporary Tablespace to the Primary Database These steps show how to add a temporary tablespace to the primary database. Procedure 1. Start the DB2 UDB command-line processor: %>bash %>source /db2home/db2inst1/sqllib/db2profile %>db2 2. Connect to the primary DB2 UDB database: db2 => connect to pdb user db2_admin_user using db2_admin_password (where db2_admin_user and db2_admin_password are the administrative user ID and password for the primary database.) 3. Create a buffer pool: db2 => create bufferpool pdb_buffer_name size 1000 pagesize same_pagesize_as_pdb (where pdb_buffer_name is the buffer name, and same_pagesize_as_pdb is the page size at the primary database.) 4. Create a temporary tablespace: db2=>create user temporary tablespace pdb_temp_space_name pagesize same_pagesize_as_pdb managed by automatic storage bufferpool pdb_buffer_name (where pdb_temp_space_name is the tablespace name, same_pagesize_as_pdb is the page size at the primary database, and pdb_buffer_name is the buffer name.) Note Determine the DB2 UDB page size using the LIST TABLESPACES SHOW DETAIL command. For example, to create a temporary tablespace named deep13 with a 16KB page size and buffer pool named tom_servo, enter: db2 => create user temporary tablespace deep13 pagesize 16K managed by automatic storage bufferpool tom_serv 2016 SAP SE or an SAP affiliate company. All rights reserved. 173

8.9.1.3.2 Verify the Current Archive Setting of the Transaction Log Set the primary DB2 UDB database transaction logging to archive logging, instead of circular logging for DB2 LogReader Adapter. Procedure 1. Determine the LOGARCHMETH1 setting. a. Connect to the database by entering this command at the CLP prompt. db2 => CONNECT TO dbalias USER db2_user USING db2_user_ps (where dbalias is the cataloged alias of the primary database, db2_user is the primary database user, and db2_user_ps is the password.) b. Determine the LOGARCHMETH1 setting: db2 => GET DB CFG FOR dbalias 2. If the results do not show that LOGARCHMETH1 is set to LOGRETAIN or to the path name of the directory to which logs are archived, set it: To use the default archive location: db2 => UPDATE DATABASE CONFIGURATION USING logarchmeth1 LOGRETAIN To use a specific archive location: db2 => UPDATE DATABASE CONFIGURATION USING logarchmeth1 DISK:path (where path is the full path name of the directory where the archive logs are to be stored. If you change the setting of the DB2 UDB logarchmeth1 parameter, DB2 UDB requires you to back up the database. Use your normal backup procedure, or see the IBM documentation for information on the BACKUP DATABASE command.) 3. Reactive and backup DB2 UDB database to make the configuration change take effect: a. DEACTIVE the database db2 => DEACTIVE DATABASE dbalias USER db2_user USING db2_user_ps (where dbalias is the cataloged alias of the primary database, db2_user is the primary database user, and db2_user_ps is the password.) b. BACKUP database LOGARCHMETH1=LOGRETAIN db2 => BACKUP DATABASE dbalias USER db2_user USING db2_user_ps LOGARCHMETH1=DISK: path db2 => BACKUP DATABASE dbalias TO path USER db2_user USING db2_user_ps (where dbalias is the cataloged alias of the primary database, path is the log archive path you specified, db2_user is the primary database user, and db2_user_ps is the password.) 174 2016 SAP SE or an SAP affiliate company. All rights reserved.

c. ACTIVE the database again db2 => ACTIVE DATABASE dbalias USER db2_user USING db2_user_ps (where dbalias is the cataloged alias of the primary database, db2_user is the primary database user, and db2_user_ps is the password.) 4. Verify the configuration change db2=> CONNECT TO dbalias USER db2_user USING db2_user_ps (where dbalias is the cataloged alias of the primary database, db2_user is the primary database user, and db2_user_ps is the password.) db2=> CALL SYSPROC.GET_DB_CONFIG() db2=> SELECT DBCONFIG_TYPE, LOGARCHMETH1 FROM SESSION.DB_CONFIG The last SELECT statement returns two rows, one for on-disk (DBCONFIG_TYPE=0) value and another for inmemory (DBCONFIG_TYPE=1) value. Make sure that both of the values are changed to LOGRETAIN or DISK. 8.9.1.3.3 Create a DB2 UDB User and Grant Permissions These steps show how to create a DB2 UDB user and grant permissions. Context DB2 LogReader Adapter requires a DB2 UDB login that has permission to access data and create new objects in the primary database. The DB2 UDB login must have SYSADM or DBADM authority to access the primary database transaction log. Procedure 1. Create a new operating system user named ra_user using commands appropriate for your operating system. For example, to create a user named ra_user on a Linux operating system, use: %>useradd -gusers - Gmgmt -s/bin/shell -psybase -d/home/ra_user -m ra_user (where sybase is the password corresponding to the ra_user user name.) 2. Start the DB2 UDB command-line processor: %>bash %>source /db2home/db2inst1/sqllib/db2profile %>db2 3. Connect to the primary DB2 UDB database: db2=>connect to pdb user db2_admin_user using db2_admin_password (where db2_admin_user and db2_admin_password are the administrative user ID and password for the primary database.) 4. Grant all necessary authorities to ra_user: GRANT DBADM ON DATABASE TO USER <user> 2016 SAP SE or an SAP affiliate company. All rights reserved. 175

GRANT CREATETAB ON DATABASE TO USER <user> GRANT BINDADD ON DATABASE TO USER <user> GRANT CONNECT ON DATABASE TO USER <user> GRANT CREATE_NOT_FENCED_ROUTINE ON DATABASE TO USER <user> GRANT IMPLICIT_SCHEMA ON DATABASE TO USER <user> GRANT LOAD ON DATABASE TO USER <user> GRANT CREATE_EXTERNAL_ROUTINE ON DATABASE TO USER <user> GRANT QUIESCE_CONNECT ON DATABASE TO USER <user> 8.9.1.4 DB2 Log Reader Adapter Required Libraries The following table contains a list of the required libraries for setting up the DB2 Log Reader Adapter. Library DB2 UDB JDBC driver Notes Include the DB2 JDBC driver library in the Data Provisioning Agent CLASSPATH environment variable. Please use the corresponding version of JDBC driver listed in the IBM documentation. This JAR file (db2jcc4.jar) needs to be copied to the following directory: <DPAgent_root>/lib Log Reader native interface The DB2 Log Reader Adapter calls a C-based native interface to access the DB2 Log Reader API to read its log record. Include the native interface library into the DPAgent PATH (for Windows) or LD_LIBRARY_PATH (for Linux) environment variable or JVM -Djava.libary.path variable if you startup the Data Provisioning Agent from Eclipse. Platform Windows (64-bit) Notes sybrauni_x64.dll (to support DB2 v9.5) sybrauni97_x64.dll (to support DB2 v9.7 sybrauni98_x64.dll(to support DB2 v10.1, v10.5) Linux (64-bit) libsybrauni.so (to support DB2 v9.5) libsybrauni97.so (to support DB2 v9.7) libsybrauni98.so (to support DB2 v9.8, v10.1, v10.5) Note The native interface libraries will be packaged into Data Provisioning Agent installer. 176 2016 SAP SE or an SAP affiliate company. All rights reserved.

Library Notes DB2 Log Reader Adapter internal libraries sybrautrunc.jar sybraudb2flsn.jar These libraries will be installed into DB2 database during replication initialization for specific procedure calls. Please include them in the Data Provisioning Agent CLASSPATH environment variable. These libraries will be packaged into Data Provisioning Agent installer. Related Information IBM driver documentation 8.9.2 DB2LogReaderAdapter Preferences Configuration parameters for the DB2 Log Reader adapter. Note Log Reader adapter preferences (except for Number of wrapped log files, Enable verbose trace, and Maximum log file size) are no longer set in the Data Provisioning Agent Configuration Tool. They are now moved to the remote source configuration options in SAP HANA. If you have upgraded from a previous version, then the settings you find in the Agent Configuration Tool are your previous settings, displayed for your reference. You can adjust DB2 Log Reader adapter settings in the Data Provisioning Agent Configuration Tool. (<DPAgent_root>/configTool/dpagentconfigtool.exe) Parameter Description Default value Maximum operation queue size Maximum scan queue size Ignore log record processing errors Replicate LOB columns Database connection pool size The maximum number of operations permitted in the log reader operation queue during replication. The maximum number of log records permitted in the log reader log scan queue during replication. Determines whether to ignore log record processing errors. Determines whether the agent applies each individual LOB change. Maximum number of connections allowed in the connection pool on a secondary node. 1000 1000 false true 15 2016 SAP SE or an SAP affiliate company. All rights reserved. 177

Parameter Description Default value Number of times to retry to connect if a connections fails Timeout in seconds to retry connecting Number of wrapped log files Enable verbose trace Maximum log file size Instructs the client library (DBLIB, ODBC, ADO, and so on) to keep retrying the connection attempt, as long as the server is not found, for the specified number of times. The number of seconds the agent waits between retry attempts to connect to the primary database. The maximum size, in 1K blocks, of the Replication Agent system log file before wrapping. Enables or disables additional diagnostic information in the agent system log files. Limit the size of the message log to conserve disk space. 5 10 3 false 1000 8.9.3 DB2 Log Reader Remote Source Configuration Configure the following options in smart data access to configure your connection to a DB2 remote source. Note When setting up a remote source and you use an instance name longer than 30 characters, the generated log reader folder name under <DPAgent_root>/LogReader/ will be converted to "AGENT<xxxx>". The log file is located at <DPAgent_root>/log/Framework.trc, and will read: The instance name <original_name> exceeds 30 characters and it is converted to <converted_name>. Category Sub-category Option Description Generic Instance Name The name of the Log Reader instance to be created. You can specify the same name as the remote source name. This option must be configured only if you are using realtime. Instance Administration Port The administrator port of the Log Reader instance. If not specified, the default 13456 is used. The remote source also uses the port that is one greater 178 2016 SAP SE or an SAP affiliate company. All rights reserved.

Category Sub-category Option Description than the instance administration port. For example, if the instance administration port is specified with 13456, both 13456 and 13457 are used. This option needs to be configured only if you are using realtime. Database Host The host name or IP address on which the DB2 Server is running. Port Number Database Name Database Source Name The port number of the DB2 Server The name of the database Set the Source Name parameter to MYDB_ALIAS when creating a remote source (where MYDB_ALIAS is the database alias that was used when cataloging the primary database) Schema Alias Replacements Schema Alias Schema name to be replaced with the schema given in Schema Alias Replacement. If given, accessing tables under it is considered to be accessing tables under the schema given in Schema Alias Replacement. Schema Alias Replacement Schema name to be used to replace the schema given in Schema Alias. Security Use SSL Specifies whether you are using SSL. The default value is false. Credentials Credentials Mode technical user or secondary user Select one of the choices, depending on the purpose of the remote source you want to create. 2016 SAP SE or an SAP affiliate company. All rights reserved. 179

Category Sub-category Option Description Credential User Name The name of the DB2 database user which DB2 LogReader Adapter uses to access the DB2 database Password The password of the DB2 database user which DB2 LogReader Adapter uses to access the DB2 database Related Information SAP HANA Security Guide: SAP HANA Smart Data Access (HTML) SAP HANA Security Guide: SAP HANA Smart Data Access (PDF) SAP HANA Smart Data Integration and SAP HANA Smart Data Configuration Guide: Create Remote Subscription (PDF) Adapter Remote Source User Credentials [page 119] 8.9.4 DB2 to SAP HANA Data Type Mapping The following table shows the conversion between DB2 data types and SAP HANA data types. DB2 data type BIGINT DECFLOAT(16) DECFLOAT(32) DECIMAL DOUBLE INTEGER REAL SMALLINT GRAPHIC VARGRAPHIC LONGVARGRAPHIC CHAR LONG CHAR FOR BIT DATA SAP HANA data type BIGINT DOUBLE DECIMAL DECIMAL DOUBLE INTEGER REAL SMALINT N N NCLOB CLOB VARBINARY 180 2016 SAP SE or an SAP affiliate company. All rights reserved.

DB2 data type LONG FOR BIT DATA DATE TIME TIMESTAMP BLOB CLOB DBCLOB ROWID XML User-defined data types SAP HANA data type BLOB DATE TIME TIMESTAMP BLOB CLOB NCLOB Unsupported Unsupported Unsupported 8.9.5 DB2 Database Cleanup Run SQL scripts from to manually clean objects from the DB2 source database. Cleanup scripts are used to drop database-level objects. Usually, you do not need to execute a cleanup script after an adapter is dropped, because database-level objects are dropped automatically by the adapter. However, in some cases, if any errors occur during or before automatically dropping these objects, the objects may not be dropped. At that point, you may need to execute the cleanup script to drop the objects. You can find the DB2 cleanup script file at <DPAgent_root>\LogReader\scripts \db2_logreader_cleanup.sql. 8.9.6 Log Reader Adapter Log Files You can review processing information in the Log Reader log files. The following files are available: Log file name and location <DPAgent_root>/LogReader/admin_logs/ admin<instance>.log <DPAgent_root>/log/repagent.log Description Log Reader administration log Log Reader instance log 2016 SAP SE or an SAP affiliate company. All rights reserved. 181

8.10 Microsoft SQL Server Log Reader The MS SQL Server Log Reader adapter provides real-time changed-data capture capability to replicate changed data from a database to SAP HANA in real time. You can also use it for batch loading. Note Before registering your adapter with the SAP HANA system, be sure that you have downloaded and installed the correct JDBC libraries. For information about the proper JDBC library for your source, see the Product Availability Matrix. Place your files in <DPAgent_root>/lib, and you will need to manually create the /lib folder. The Log Reader service provider is created as a remote source, and it requires the support of artifacts like virtual tables and remote subscriptions for each source table to perform replication. Note Smart data integration relies on database logs to perform data movement. This means that logs must be available until the data is successfully read and replicated to the target SAP HANA database. You should configure MS SQL Server in Full Recovery Mode to ensure that the data will be replicated to SAP HANA. Note The user configured during the installation of the Data Provisioning agent must have read access to the transaction log (the.ldf file). Adapter Functionality This adapter supports the following functionality: Multiple instances of the Log Reader adapter are supported for each Data Provisioning Agent. Virtual table as a source Real-time change data capture Note Log Reader adapters do not support the truncate table operation. The MS SQL Server Log Reader adapter does not support WRITETEXT and UPDATETEXT. Virtual table as a target using a Data Sink node in a flowgraph Parallel reader (flowgraph and replication) Loading options for target tables DDL propagation. The supported schema changes are: ADD COLUMN DROP COLUMN 182 2016 SAP SE or an SAP affiliate company. All rights reserved.

RENAME TABLE RENAME COLUMN ALTER COLUMN Replication monitoring and statistics Search for tables In addition, this adapter supports the following capabilities: Table 47: Global Settings Functionality SELECT from a virtual table INSERT into a virtual table Execute UPDATE statements on a virtual table Execute DELETE statements on a virtual table Different capabilities per table Different capabilities per table column Realtime Supported? Yes Yes Yes Yes No No Yes Table 48: Select Options Functionality Select individual columns Add a WHERE clause JOIN multiple tables Aggregate data via a GROUP BY clause Support a DISTINCT clause in the select Support a TOP or LIMIT clause in the select ORDER BY GROUP BY Supported? Yes Yes Yes Yes Yes Yes Yes Yes 8.10.1 Remote Database Setup for SQL Server Realtime The remote database must be set up properly for Log Reader adapters to function correctly when using realtime replication. This setup process is necessary only when using realtime. 2016 SAP SE or an SAP affiliate company. All rights reserved. 183

8.10.1.1 Create Users and Grant Privileges Follow these steps to create users and grant privileges. Procedure Create an SQL Server user (for example, DP_USER) for the remote source. Grant the required privileges as follows: use master go create login DP_USER with password = MyPW go use <primary database> go create user DP_USER for login DP_USER go EXEC sp_addsrvrolemember DP_USER, sysadmin go 8.10.1.2 Enable Remote DAC Follow these steps to enable DAC to allow remote connections. Procedure 1. Log on to Microsoft SQL Server using the newly created user, and change the Microsoft SQL Server Remote Admin Connections Configuration option to enable DAC to allow remote connections. sp_configure remote admin connections, 1 go 2. Update the remote admin connection setting. reconfigure go 3. Verify the remote admin connection string. sp_configure remote admin connections go 184 2016 SAP SE or an SAP affiliate company. All rights reserved.

8.10.1.2.1 Make Log Files Readable Install and set up the sybfilter driver so that the Log Reader can read the primary transaction log files. Procedure 1. In Windows Explorer, navigate to the sybfilter driver installation directory. This directory is located at <DPAgent_root>\LogReader\sybfilter\system\<platform>, where <DPAgent_root> is the root directory or Data Provisioning Agent installation, and <platform> is winx86 or winx64. winx86 is for 32-bit Windows Server 2008, Windows Server 2008 R2, and Windows 7 winx64 is for 64-bit Windows Server 2008, Windows Server 2008 R2, and Windows 7 2. Right-click the sybfilter.inf to install the sybfilter driver. 3. Under any directory, create a configuration file to save all log file paths for primary databases. The configuration file must have a.cfg suffix. For example, under <DPAgent_root>\LogReader\sybfilter \system\<platform>, create a file named LogPath.cfg. 4. Add a system environment variable named <RACFGFilePath>, and set its value to the path and the file name of the configuration file. 5. In Windows Explorer, navigate to <DPAgent_root>\LogReader\sybfilter\bin\, and right-click the sybfiltermgr.exe file and then select Run as administrator to start the sybfilter driver management console. 6. To start the sybfilter driver, enter start at the management console. 7. Add the log file path to the sybfilter driver with the user manager or by modifying the configuration file: User manager use the add command in the management console. The syntax for this command is as follows. add servername dbname logfilepath. For example, to add log file named pdb1_log.ldf for the database pdb1 on the data server PVGD50857069A\MSSQLSERVER, use this: add PVGD50857069A\MSSQLSERVER pdb1 C:\Mssql2012\MSSQL11.MSSQLSERVER\MSSQL\DATA \pdb1_log.ldf Or, add the following into LogPath.cfg file directly. [PVGD50857069A\MSSQLSERVER, pdb1] log_file_path=c:\mssql2012\mssql11.mssqlserver\mssql\data\pdb1_log.ldf 8. If you added a log file for your primary database before adding the log file path to the sybfilter driver, restart Microsoft SQL Server to make the log file readable. 9. At the management console, verify that log files are readable. If some log files are unreadable, make sure the files have been created and that Microsoft SQL Server has been restarted, if necessary. Execute check command in sybfilter console screen. Related Information MS SQL Server Log Reader Remote Source Configuration [page 193] 2016 SAP SE or an SAP affiliate company. All rights reserved. 185

8.10.1.3 Enable TCP/IP Follow these steps to enable TCP/IP. Procedure 1. Go to SQL Server Configuration Tools, and choose SQL Server Configuration Manager SQL Server Network Configuration Protocols for <SQLInstanceName>, where <SQLInstanceName> is your SQL Server instance. 2. Right-click TCP/IP, and choose Enable. 8.10.1.4 Configure the Primary Data Server for the First Time Before you can begin, you must configure the primary data server. Context Note When you create a remote source, if the Database Data Capture Mode option is set to MSSQL CDC Mode, this step is not necessary. Procedure 1. Stop the Microsoft SQL Server service. a. In Control Panel Administrative Tools Services, find the service named MicrosoftSQLServer (<SERVER>), where <SERVER> is the name of your Microsoft SQL Server data server. For example Microsoft SQL Server (TEAMSTER) b. Stop the service. 2. Restart Microsoft SQL Server in single-user mode. a. Click Start Control Panel Administrative Tools Services b. Right-click your Microsoft SQL Server instance, and choose Properties c. In the General tab, click Stop d. Under Start parameters, enter m e. Click Start 186 2016 SAP SE or an SAP affiliate company. All rights reserved.

Tip You can also restart Microsoft SQL Server in single-user mode from the command line. For example, Run > C:\Program Files\Microsoft SQL Server\MSSQL12.MSSQLSERVER\MSSQL \Binn>>.\sqlservr.exe -smssqlserver -m 3. Connect to Microsoft SQL Server using dedicated administrator connection (DAC) a. In SQL Server Management Studio, with no other DACs open, on the toolbar, click Database Engine Query. b. In the Connect to Database Engine dialog box, in the Server name box, type ADMIN:, followed by the name of the server instance. For example, to connect to a server instance named ACCT\PAYABLE, type ADMIN:ACCT\PAYABLE. c. Complete the Authentication section, providing credentials for a member of the sysadmin group, and then click Connect. d. The connection is made. (If the DAC is already in use, the connection will fail with an error indicating it cannot connect.) 4. Execute script <DPAgent_root>\LogReader\scripts\mssql_server_init.sql to initialize the server. Script <DPAgent_root>\LogReader\scripts\mssql_server_deinit.sql can be used to de-initialize the server if necessary. 5. Stop and restart Microsoft SQL Server service back to normal mode. Related Information MS SQL Server Log Reader Remote Source Configuration [page 193] 8.10.1.5 Installing Data Provisioning Agent and MS SQL Server on Different Servers Additional steps are necessary when installing the Data Provisioning Agent and MS SQL server on different machines. Context The following procedure is an example. The example shows you how to set up an environment with an MS SQL Server database named mypdb on machine A and a DP Agent installed on a different machine ( B ) 2016 SAP SE or an SAP affiliate company. All rights reserved. 187

Procedure 1. Install and configure Sybfilter on machine A (<host_name>). Sybfilter can be copied from the DP Agent install directory on machine B (for example, C:\usr\sap \dataprovagent\logreader\sybfilter). 2. Share the folders which contain mypbd database log files on machine A. Note Grant READ permissions for the database log files to the DPAGENT user on machine B. Log files: C:\Program Files\Microsoft SQL Server\MSSQL11.MSSQL2K12SP1\MSSQL\DATA \mypdb_log.ldf C:\Program Files\Microsoft SQL Server\MSSQL11.MSSQL2K12SP1\MSSQL\DATA \mypdb_log_2.ldf C:\MSSQL_LOG\mypdb\mypdb_log_3.ldf Share folders: C:\Program Files\Microsoft SQL Server\MSSQL11.MSSQL2K12SP1\MSSQL\DATA\ \ \<host_name>\mssql_data C:\MSSQL_LOG\ \\<host_name>\mssql_log 3. Edit and include the mapping relationship into this file as shown in the example below. Also, provide the database name, as shown in the example below. Because the mapping is based on a parent directory and not on the log file itself, only one entry is sufficient for both mypdb_log.ldf and mypdb_log_2.ldf. ## Put the original path on the left side of the equal symbol and the UNC pathname of the share folder on the right side. C:\Program Files\Microsoft SQL Server\MSSQL11.MSSQL2K12SP1\MSSQL\DATA=\ \<host_name>\mssql_data C:\MSSQL_LOG\mypdb=\\<host_name>\mssql_log\mypdb You can also include multiple databases in the same file. 4. If you are using remote databases, you should edit the mssql_log_path_mapping.props file to differentiate database names and instances. For example, suppose you have the following scenario: User1 creates remote source RS1, connecting to DB1, and provides an instance name of ISNT1 User2 creates remote source RS2, connecting to DB2, and provides an instance name of INST2 In this case, you would add the following to the mssql_log_path_mapping.props file: If DB1 and DB2 have different names: [DB1] D:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\DATA=\ \machine1\mssql_data [DB2] D:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\DATA=\ \machine2\mssql_data 188 2016 SAP SE or an SAP affiliate company. All rights reserved.

If DB1 and DB2 have the same name, add an instance name to differentiate: [INST1:DB1] D:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\DATA=\ \machine1\mssql_data [INST2:DB1] D:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\DATA=\ \machine2\mssql_data 5. When you create the remote source, set the value of property Use Remote Database to true. [mypdb] Related Information MS SQL Server Log Reader Remote Source Configuration [page 193] 8.10.1.6 Run a Microsoft SQL Server Log Reader Adapter on Linux Install and configure your systems to run a Microsoft SQL Server Log Reader adapter on Linux. Context You will need to install the Data Provisioning agent, configure Sybfilter, and other operations to use Linux with your SQL Server adapter: Procedure 1. Install the Data Provisioning agent on a Linux machine 2. Copy Sybfilter from the Data Provisioning agent install root (for example, /usr/sap/dataprovagent to the SQL Server Windows machine (For example, /usr/sap/dataprovagent/logreader/sybfilter), and configure SQL Server and Sybfilter on Windows, as required. 3. Share the SQL Server transaction log directory and mount to the Linux machine. For example: sudo mount -v -t cifs -o username=srjones,password=********,domain=sybase,ro,cache=none,directio // 10.172.162.145/WORK /replinuxb27-1/srjones/repwinb12 4. Configure mssql_log_path_mapping.props file. 5. Create a remote source, making sure that the Use Remote Database option is set to true. 2016 SAP SE or an SAP affiliate company. All rights reserved. 189

Related Information Install the Data Provisioning Agent [page 23] MS SQL Server Log Reader Remote Source Configuration [page 193] 8.10.2 MssqlLogReaderAdapter Preferences Configuration parameters for the MS SQL Log Reader adapter. Note Log Reader adapter preferences (except for Number of wrapped log files, Enable verbose trace, and Maximum log file size) are no longer set in the Data Provisioning Agent Configuration Tool. They are now moved to the remote source configuration options in SAP HANA. If you have upgraded from a previous version, then the settings you find in the Agent Configuration Tool are your previous settings, displayed for your reference. You can adjust SQL Server Log Reader adapter settings in the Data Provisioning Agent Configuration Tool. (<DPAgent_root>/configTool/dpagentconfigtool.exe) Parameter Description Default value Maximum operation queue size Maximum scan queue size The maximum number of operations permitted in the log reader operation queue during replication. The maximum number of log records permitted in the log reader log scan queue during replication. 1000 1000 190 2016 SAP SE or an SAP affiliate company. All rights reserved.

Parameter Description Default value Maximum wait interval between log scans The maximum wait interval between Log Reader transaction log scans. 2 Note The value of the parameter is the maximum number of seconds that can elapse before the Log Reader component scans the transaction log for a transaction to be replicated, after a previous scan yields no such transaction. For reduced replication latency in an infrequently updated database, we recommends lower number settings for the parameter. If the primary database is continuously updated, the value of the parameter is not significant to performance. 2016 SAP SE or an SAP affiliate company. All rights reserved. 191

Parameter Description Default value Seconds to add to each log scan wait interval The number of seconds to add to each wait interval before scanning the transaction log, after a previous scan yields no transaction to be replicated. 0 Note The value of the parameter is the number of seconds added to each wait interval before the LogReader component scans the log for a transaction to be replicated, after a previous scan yields no such transaction. The number of seconds specified by the parameter is added to each wait interval, until the wait interval reaches the value specified by the Maximum wait interval between log scans parameter. For optimum performance, the value of the parameter should be balanced with the average number of operations in the primary database over a period of time. In general, better performance results from reading more operations from the transaction log during each LogReader scan. With a primary database that is less frequently updated, increasing the value of the parameter may improve overall performance. If the database is continuously updated, the value of the parameter may not be significant to performance. Replicate LOB columns Database connection pool size Determines whether the agent applies each individual LOB change. Maximum number of connections allowed in the connection pool on a secondary node. true 15 192 2016 SAP SE or an SAP affiliate company. All rights reserved.

Parameter Description Default value Number of times to retry to connect if a connections fails Timeout in seconds to retry connecting Number of wrapped log files Enable verbose trace Maximum log file size Instructs the client library (DBLIB, ODBC, ADO, and so on) to keep retrying the connection attempt, as long as the server is not found, for the specified number of times. The number of seconds the agent waits between retry attempts to connect to the primary database. The maximum size, in 1K blocks, of the agent system log file before wrapping. Enables or disables additional diagnostic information in the agent system log files. Limit the size of the message log to conserve disk space. 5 10 3 false 1000 8.10.3 MS SQL Server Log Reader Remote Source Configuration Configure the following options in smart data access to configure your connection to a SQL Server remote source. Note When setting up a remote source and you use an instance name longer than 30 characters, the generated log reader folder name under <DPAgent_root>/LogReader/ will be converted to "AGENT<xxxx>". The log file is located at <DPAgent_root>/log/Framework.trc, and will read: The instance name <original_name> exceeds 30 characters and it is converted to <converted_name>. Category Sub-category Option Description Generic Instance Name The name of the Log Reader instance to be created. You can specify the same name as the remote source name. This option needs to be configured only if you are using realtime. Instance Administration Port The administrator port of the Log Reader instance. If not specified, the default 13456 is used. The remote source also uses the port that is one greater 2016 SAP SE or an SAP affiliate company. All rights reserved. 193

Category Sub-category Option Description than the instance administration port. For example, if the instance administration port is specified with 13456, both 13456 and 13457 are used. This option needs to be configured only if you are using realtime. Database Data Server The MS SQL Data Server name Port Number DAC Port Number The MS SQL Data Server port number The MS SQL Data Server DAC port number. The default DAC port number is 1434. Tip You can find the DAC port for your server by searching for Dedicated admin in the ERRORLOG file located in the log directory of your Microsoft SQL Server. Database Name The MS SQL database name Database Data Capture Mode Native Mode (default) MSSQL CDC Mode: The adapter uses the MSSQL CDC API to mark a table for replication, so that the first time you use it, MSSQL server-level initialization (needs to switch MSSQL to single user mode) is not required. When the Database Data Capture Mode option is set to MSSQL CDC Mode, the adapter does not support replicating the 194 2016 SAP SE or an SAP affiliate company. All rights reserved.

Category Sub-category Option Description TRUNCATE TABLE operation. Note SQL Server CDC does not need to be enabled. Use Remote Database Set to True if you are running the DP Agent on a different machine than the MS SQL Server database. The default value is false Schema Alias Replacements Schema Alias Schema name to be replaced with the schema given in Schema Alias Replacement. If given, accessing tables under it is considered to be accessing tables under the schema given in Schema Alias Replacement. Schema Alias Replacement Schema name to be used to replace the schema given in Schema Alias. Security Use SSL Specifies whether you are using SSL The default value is false Host Name in Certificate Enter the host name found in the SSL certificate. CDC Properties LogReader Ignore Log Record Processing Errors Specifies whether the Log Reader ignores the errors that occur during log record processing. If set to True, the replication will not stop if log record processing errors occur. The default value is False. Maximum operation queue size The maximum number of operations permitted in the log reader operation queue during replication. 2016 SAP SE or an SAP affiliate company. All rights reserved. 195

Category Sub-category Option Description The default value is 1000 Maximum scan queue size The maximum number of log records permitted in the log reader log scan queue during replication. The default value is 1000 Maximum wait interval between log scans The default value is 2 Seconds to add to each log scan wait interval Replicate LOB columns The default value is 0 Determines whether the agent applies each individual LOB change. The default value is true Database connection pool size Maximum number of connections allowed in the connection pool on a secondary node. The default value is 15 Number of times to retry to connect if a connection fails Instructs the client library (DBLIB, ODBC, ADO, and so on) to keep retrying the connection attempt, as long as the server is not found, for the specified number of times. The default value is 5 Timeout in seconds to retry connecting The number of seconds the agent waits between retry attempts to connect to the primary database. The default value is 10 Credentials Credentials Mode technical user or secondary user Select one of the choices, depending on the purpose of 196 2016 SAP SE or an SAP affiliate company. All rights reserved.

Category Sub-category Option Description the remote source you want to create. User Name Password Support Multiple LogReader Adapter Instances In The Same Agent MS SQL user name MS SQL user password Specifies whether to allow the DP Agent to run multiple MS SQL LogReader instances (each instance for one MSSQL database instance) at the same time. The default value is True. Note Each remote source (LogReader instance) occupies two port numbers. For example, if you set the Administrator port to 12345 when you create a remote source, both port 12345 and 12346 will be used. Therefore, the next remote source Administrator port should be started from 12347. You are not allowed to set up multiple remote sources on one MSSQL database. Each remote source should have different Instance Name. Related Information SAP HANA Security Guide: SAP HANA Smart Data Access (HTML) SAP HANA Security Guide: SAP HANA Smart Data Access (PDF) 2016 SAP SE or an SAP affiliate company. All rights reserved. 197

SAP HANA Smart Data Integration and SAP HANA Smart Data Configuration Guide: Create Remote Subscription (PDF) Adapter Remote Source User Credentials [page 119] SAP HANA Smart Data Integration and SAP HANA Smart Data Configuration Guide (HTML) SAP HANA Smart Data Integration and SAP HANA Smart Data Configuration Guide (PDF) 8.10.4 MS SQL Server to SAP HANA Data Type Mapping The following table shows the conversion between SQL Server data types and SAP HANA data types. MS SQL Server data type BIT TINYINT SMALLINT INT FLOAT REAL BIGINT MONEY SMALMONEY DECIMAL NUMERIC CHAR NCHAR UNIQUEIDENTIFIER DATETIMEOFFSET SMALL DATETIME DATETIME DATETIME2 DATE TIME BINARY TIMESTAMP IMAGE VARBINARY TEXT NTEXT SAP HANA data type TINYINT TINYINT SMALLINT INTEGER DOUBLE REAL DECIMAL DECIMAL DECIMAL DECIMAL DECIMAL N (36) (34) SECONDDATE TIMESTAMP TIMESTAMP DATE TIMESTAMP VARBINARY VARBINARY(8) BLOB VARBINARY CLOB CLOB 198 2016 SAP SE or an SAP affiliate company. All rights reserved.

MS SQL Server data type N XML SPATIAL HIERARCHYID GEOGRAPHY GEOMETRY TABLE SQL_VARIANT SAP HANA data type N <NOT SUPPORTED> <NOT SUPPORTED> <NOT SUPPORTED> <NOT SUPPORTED> <NOT SUPPORTED> <NOT SUPPORTED> <NOT SUPPORTED> 8.10.5 SQL Server Database Cleanup Run SQL scripts from to manually clean objects from the SQL Server source database. Cleanup scripts are used to drop database-level objects. Usually, you do not need to execute a cleanup script after an adapter is dropped, because database-level objects are dropped automatically by the adapter. However, in some cases, if any errors occur during or before automatically dropping these objects, the objects may not be dropped. At that point, you may need to execute the cleanup script to drop the objects. You can find the SQL Server cleanup script file at <DPAgent_root>\LogReader\scripts \mssql_logreader_cleanup.sql. 8.10.6 Log Reader Adapter Log Files You can review processing information in the Log Reader log files. The following files are available: Log file name and location <DPAgent_root>/LogReader/admin_logs/ admin<instance>.log <DPAgent_root>/log/repagent.log Description Log Reader administration log Log Reader instance log 8.11 Oracle Log Reader The Oracle Log Reader adapter provides real-time changed-data capture capability to replicate changed data from a database to SAP HANA in real time. You can also use it for batch loading. 2016 SAP SE or an SAP affiliate company. All rights reserved. 199

The Log Reader service provider is created as a remote source, and it requires the support of artifacts like virtual tables and remote subscriptions for each source table to perform replication. Multiple instances of the Log Reader adapter are supported for each Data Provisioning Agent. Note There are three important tasks that you must perform to set up your Oracle database: Before registering your adapter with the SAP HANA system, be sure that you have downloaded and installed the correct JDBC libraries. For information about the proper JDBC library for your source, see the Product Availability Matrix. Place your files in <DPAgent_root>/lib, and you will need to manually create the /lib folder. Set up database permissions Decide if you want table- (default) or database-level supplemental logging for realtime replication and set it up. Adapter Functionality This adapter supports the following functionality: Oracle 12c multitenant database support Virtual table as a source Real-time change data capture Note Log Reader adapters do not support the truncate table operation. Virtual table as a target using a Data Sink node in a flowgraph Parallel reader (flowgraph and replication) Loading options for target tables DDL propagation. The supported schema changes are: ADD COLUMN DROP COLUMN Replication monitoring and statistics Search for tables In addition, this adapter supports the following capabilities: Table 49: Global Settings Functionality Supported? SELECT from a virtual table Yes INSERT into a virtual table Yes Execute UPDATE statements on a virtual table Yes 200 2016 SAP SE or an SAP affiliate company. All rights reserved.

Functionality Execute DELETE statements on a virtual table Different capabilities per table Different capabilities per table column Realtime Supported? Yes No No Yes Table 50: Select Options Functionality Select individual columns Add a WHERE clause JOIN multiple tables Aggregate data via a GROUP BY clause Support a DISTINCT clause in the select Support a TOP or LIMIT clause in the select ORDER BY GROUP BY Supported? Yes Yes Yes Yes Yes Yes Yes Yes 8.11.1 Remote Database Setup for Oracle Realtime The remote database must be set up properly for Log Reader adapters to function correctly when using realtime replication. This setup process is necessary only when using realtime. Oracle 12c Multitenant support Multitenant databases are supported for Oracle 12c. Be aware that some of the setup procedures are different for multitenant, for example in Remote source configuration, permissions, and cleanup. 2016 SAP SE or an SAP affiliate company. All rights reserved. 201

8.11.1.1 Oracle Database Permissions Database users need to have certain permissions granted to them. The following scripts can be found in the oracle_init_example.sql file, which is located in the Scripts folder of the Data Provisioning Agent installation (<DPAgent_root>\LogReader\Scripts). Note Be aware that the oracle_init_example.sql file is a template script. You may need to alter the following: Change LR_USER to the configured database user name in the remote source options, if it is not LR_USER. Change <_replace_with_password> in line 85 to the password of the database user. Also, be sure that you grant select access to the log reader user by issuing the following statement. In these examples, taken from the oracle_init_example.sql file, the user is named LR_USER. Change this user name to whatever you need. GRANT SELECT ON <table_to_be_replicated> TO LR_USER; For convenience, you can also grant SELECT ANY TABLE. Note We have removed the following permissions, as of SPS 11: CREATE PUBLIC SYNONYM DROP PUBLIC SYNONYM ALTER SYSTEM EXECUTE on DBMS_FLASHBACK If you are using Oracle 12c, the following permission is required: LOGMINING GRANT CREATE SESSION TO LR_USER; GRANT CREATE TRIGGER TO LR_USER; GRANT CREATE SEQUENCE TO LR_USER; GRANT CREATE TABLE TO LR_USER; GRANT CREATE ANY PROCEDURE TO LR_USER; GRANT ALTER ANY TABLE TO LR_USER; GRANT SELECT ON SYS.TABPART$ TO LR_USER; GRANT SELECT ON SYS.INDCOMPART$ TO LR_USER; GRANT SELECT ON SYS.LOBCOMPPART$ TO LR_USER; GRANT SELECT ON SYS.ATTRIBUTE$ TO LR_USER; GRANT SELECT ON SYS.TABCOMPART$ TO LR_USER; GRANT SELECT ON SYS.ARGUMENT$ TO LR_USER; GRANT SELECT ON SYS.CDEF$ TO LR_USER; GRANT SELECT ON SYS.NTAB$ TO LR_USER; GRANT SELECT ON SYS.LOBFRAG$ TO LR_USER; GRANT SELECT ON SYS.MLOG$ TO LR_USER; GRANT SELECT ON SYS.OBJ$ TO LR_USER; GRANT SELECT ON SYS.COLTYPE$ TO LR_USER; GRANT SELECT ON SYS.COLLECTION$ TO LR_USER; GRANT SELECT ON SYS.IND$ TO LR_USER; GRANT SELECT ON SYS.TS$ TO LR_USER; GRANT SELECT ON SYS.CON$ TO LR_USER; GRANT SELECT ON SYS.SEQ$ TO LR_USER; GRANT SELECT ON SYS.PARTOBJ$ TO LR_USER; GRANT SELECT ON SYS.RECYCLEBIN$ TO LR_USER; GRANT SELECT ON SYS.REDEF_OBJECT$ TO LR_USER; 202 2016 SAP SE or an SAP affiliate company. All rights reserved.

GRANT SELECT ON SYS.TABSUBPART$ TO LR_USER; GRANT SELECT ON SYS.OPQTYPE$ TO LR_USER; GRANT SELECT ON SYS.USER$ TO LR_USER; GRANT SELECT ON SYS.SEG$ TO LR_USER; GRANT SELECT ON SYS.TYPE$ TO LR_USER; GRANT SELECT ON SYS.LOB$ TO LR_USER; GRANT SELECT ON V_$LOGMNR_CONTENTS TO LR_USER; GRANT SELECT ON SYS.DEFERRED_STG$ TO LR_USER; GRANT SELECT ON SYS.ICOL$ TO LR_USER; GRANT SELECT ON SYS.INDSUBPART$ TO LR_USER; GRANT SELECT ON SYS.SNAP$ TO LR_USER; GRANT SELECT ON V_$DATABASE TO LR_USER; GRANT SELECT ON SYS.INDPART$ TO LR_USER; GRANT SELECT ON V_$LOGMNR_LOGS TO LR_USER; GRANT SELECT ON SYS.TAB$ TO LR_USER; GRANT SELECT ON SYS.COL$ TO LR_USER; GRANT SELECT ON SYS.PROCEDUREINFO$ TO LR_USER; GRANT SELECT ON SYS.CCOL$ TO LR_USER; GRANT ADMINISTER DATABASE TRIGGER TO LR_USER; GRANT EXECUTE_CATALOG_ROLE TO LR_USER; GRANT SELECT_CATALOG_ROLE TO LR_USER; GRANT SELECT ANY TRANSACTION TO LR_USER; Multitenant Database Permissions To set permissions for a multitenant database, run the scripts in the following files. They are also located in <DPAgent_root>\LogReader\Scripts. The same rules apply to this script as above, concerning <LR_USER>, and so on. Note The <C##LR_USER> in the container database must be the common user and the <LR_USER> user in the pluggable database is the local user oracle_multitenant_init_example_for_container_database.sql oracle_multitenant_init_example_for_pluggable_database.sql Related Information Oracle Log Reader Remote Source Configuration [page 205] 8.11.1.2 Oracle Supplemental Logging Decide which logging level is best for you and set it up. Set your logging level in the Adapter Preferences window of the Data Provisioning Agent configuration tool for the Oracle Log Reader adapter. Then, run the necessary scripts found in the oracle_init_example.sql file, located in <DPAgent_root>\LogReader\Scripts. 2016 SAP SE or an SAP affiliate company. All rights reserved. 203

Note Be aware that the oracle_init_example.sql file is a template script. You may need to alter the following: Uncomment line 38 and comment line 42-86 if you want database-level supplemental logging. Table-level Logging We recommend table-level logging, which turns on supplemental logging for subscribed tables and some required system tables. Run the scripts located in the oracle_init_example.sql file on your Oracle client to set this up. Database-level Logging Database level turns on supplemental logging for all tables, including system tables. Execute the following DDL statements, which are located in the oracle_init_example.sql file. Then, set the Oracle supplemental logging level Adapter Preferences option to Database. ALTER DATABASE ADD SUPPLEMENTAL LOG DATA; ALTER DATABASE ADD SUPPLEMENTAL LOG DATA (PRIMARY KEY, UNIQUE) COLUMNS; Related Information OracleLogReaderAdapter Preferences [page 211] 8.11.1.3 Installing Data Provisioning Agent and Oracle Server on Different Servers Additional steps are necessary when installing the Data Provisioning Agent and Oracle server on different machines. Context When the DP Agent is installed on a different server than the Oracle instance, the DP Agent needs access to the Oracle timezone file. 204 2016 SAP SE or an SAP affiliate company. All rights reserved.

Procedure 1. Copy the timezone_<version_number>.dat file from your Oracle instance, and paste it anywhere on the server where the Data Provisioning agent is installed. 2. When creating a remote source, configure the Oracle Timezone File with the path AND the name of the file to where you copied the timezone file. ) Related Information OracleLogReaderAdapter Preferences [page 211] 8.11.2 Oracle Log Reader Remote Source Configuration Configure the following options in smart data access to configure your connection to an Oracle remote source. Note When setting up a remote source and you use an instance name longer than 30 characters, the generated log reader folder name under <DPAgent_root>/LogReader/ will be converted to "AGENT<xxxx>". The log file is located at <DPAgent_root>/log/Framework.trc, and will read: The instance name <original_name> exceeds 30 characters and it is converted to <converted_name>. Category Sub-category Option Description Generic Instance Name Instance Administration Port The name of the Log Reader instance to be created. You can specify the same name as the remote source name. This option needs to be configured only if you are using realtime. The administrator port of the Log Reader instance. If not specified, the default 13456 is used. The remote source also uses the port that is one greater than the instance administration port. For example, if the instance 2016 SAP SE or an SAP affiliate company. All rights reserved. 205

Category Sub-category Option Description administration port is specified with 13456, both 13456 and 13457 are used. This option needs to be configured only if you are using realtime. Database Multitenant Database Specifies whether to enable multitenant support. True: Replicate multitenant database False: Replicate normal database Use Oracle TNSNAMES File Specifies whether to use the connection information from the Oracle tnsnames.ora file to connect to the Oracle database. If set it to False, you must configure the Host, Port Number, and Database Name options. If set to True, configure the Oracle TNSNAMES File and Oracle TNSNAMES Connection options. The default value is False. Host Port Number Database Name Container Database Service Name The host name or IP address on which the remote Oracle database is running The Oracle Data Server port number The Oracle database name The service name of the container database. Pluggable Database Service Name The service name of the pluggable database. Oracle TNSNAMES File The full path of the Oracle tnsnames.ora file. For example, C: \oracle11g2_4_x64\pr oduct 206 2016 SAP SE or an SAP affiliate company. All rights reserved.

Category Sub-category Option Description \11.2.0\dbhome_1\NET WORK\ADMIN \tnsnames.ora Oracle TNSNAMES connection The Oracle connection name that identifies the primary database connection in the Oracle tnsnames.ora file. Container Database TNSNAMES Connection The Oracle connection name that identifies the container database connection in the Oracle tnsnames.ora file. Pluggable Database TNSNAMES Connection The Oracle connection name that identifies the pluggable database connection in the Oracle tnsnames.ora file. Database Byte Order Specifies the byte order of the source Oracle database. If you run the DP Agent and Oracle database on different machines, and those two machines belong to different OS platforms and have different byte orders, this property must be set correctly. Values: BIG_ENDIAN (for Unix) LITTLE_ENDIAN (for Windows and Linux) Schema Alias Replacements Schema Alias Schema name to be replaced with the schema given in Schema Alias Replacement. If given, accessing tables under it is considered to be accessing tables under the schema given in Schema Alias Replacement. Schema Alias Replacement Schema name to be used to replace the schema given in Schema Alias. 2016 SAP SE or an SAP affiliate company. All rights reserved. 207

Category Sub-category Option Description Security Use SSL Specifies whether you are using SSL. The default value is false. Distinguished Name (DN) in Certificate The distinguished name (DN) of the primary data server certificate. Note This parameter is only valid if Use SSL is set to true. If this parameter is set, the DN field in the server certificate is verified to match this parameter. If it does not match, the connection to the primary data server fails. Require Client Authentication CDC Properties Database Configuration Oracle Timezone File Specifies whether client authentication is required. If client authentication is required, the client sends its own certificate to the server, and the server verifies that the client's certificate was signed by a trusted CA. This parameter needs to be set only when the Data Provisioning Agent and the remote Oracle database are on different machines. If this is the case, enter the full path AND the name of the Oracle time zone file. Oracle Archive Log Path The parameter identifies the directory path where we expect to find Oracle archived redo log files. When archived redo log files are stored in the file system, the configuration parameter is set to a file system path. 208 2016 SAP SE or an SAP affiliate company. All rights reserved.

Category Sub-category Option Description If LogReader cannot find an expected log record in the Oracle online redo logs, it will search this directory for the archived log file containing the required record. Oracle Supplemental logging level Specifies the level of supplemental logging. Table: Table level turns on supplemental logging for subscribed tables and some required system tables. Database: Database level turns on supplemental logging for all tables, including system tables. Parallel Scan Enable parallel scanning Table is the default value. Specifies whether to turn on parallel scanning. To achieve better performance for high-volume log throughput, set the parameter to true to enable parallel scanning. The default value is False. Number of parallel scanners Specifies the number of parallel scanners. Queue size of parallel scan tasks Specifies the number of tasks in the queue. LogReader SCN range of parallel scan tasks Ignore Log Record Processing Errors The maximum number of system change numbers (SCN) processed by each Oracle LogMiner scanner, when parallel scan is enabled, that is, when lr_parallel_scan is true. Specifies whether the Log Reader should ignore the errors that occur during log record processing. If set to 2016 SAP SE or an SAP affiliate company. All rights reserved. 209

Category Sub-category Option Description True, the replication will not stop if log record processing errors occur. The default value is False. Maximum operation queue size Maximum scan queue size Maximum session cache size Number of log record rows fetched by the scanner at a time Replicate LOB columns Ignore data of unsupported types stored in ANYDATA Database connection pool size Number of times to retry to connect if the connection fails The maximum number of operations permitted in the log reader operation queue during replication. The maximum number of log records permitted in the log reader log scan queue during replication. The maximum number of Oracle sessions to be cached in memory during replication. Specifies the number of log record rows fetched by the scanner. Oracle logs all LOB data (except for BFILE datatypes) in the Oracle redo log. This allows the agent to apply each individual LOB change. However, for BFILE data, the same technique is used. Specifies whether you want to ignore data with unsupported types housed in ANYDATA wrapper. Maximum number of connections allowed in the connection pool on a secondary node. Instructs the client library (DBLIB, ODBC, ADO, and so on) to keep retrying the connection attempt, as long as the server is not found, for the specified number of times. 210 2016 SAP SE or an SAP affiliate company. All rights reserved.

Category Sub-category Option Description Timeout in seconds to retry connecting The number of seconds the agent waits between retry attempts to connect to the primary database. Credentials Credentials Mode technical user or secondary user Select one of the choices, depending on the purpose of the remote source you want to create. Oracle Connection Credential User Name Oracle user name Oracle Multitenant Credential Password Common User Name Oracle user password The common user name in container database Common User Password The common user password in container database Related Information SAP HANA Security Guide: SAP HANA Smart Data Access (HTML) SAP HANA Security Guide: SAP HANA Smart Data Access (PDF) Configuration Guide for SAP HANA Smart Data Integration and SAP HANA Smart Data : Create Remote Subscription (PDF) Adapter Remote Source User Credentials [page 119] 8.11.3 OracleLogReaderAdapter Preferences Configuration parameters for the Oracle Log Reader adapter. Note Log Reader adapter preferences (except for Number of wrapped log files, Enable verbose trace, and Maximum log file size) are no longer set in the Data Provisioning Agent Configuration Tool. They are now moved to the remote source configuration options in SAP HANA. If you have upgraded from a previous version, then the settings you find in the Agent Configuration Tool are your previous settings, displayed for your reference. You can adjust Oracle Log Reader adapter settings in the Data Provisioning Agent Configuration Tool. (<DPAgent_root>/configTool/dpagentconfigtool.exe) 2016 SAP SE or an SAP affiliate company. All rights reserved. 211

Parameter Description Default value Timezone file path This parameter needs to be set only when the Data Provisioning Agent and the remote Oracle database are on different machines. If this is the case, enter the full path AND the name of the Oracle time zone file. Directory containing Oracle archive redo log files Specifies the location of the archived redo log files. If the LogReader cannot find an expected log record in the Oracle online redo logs, it will search this directory for the archived log file containing the required record. Distinguished Name (DN) in Certificate The distinguished name (DN) of the primary data server certificate. This parameter is only valid if Use SSL is set to true. If this parameter is set, the DN field in the server certificate is verified to match this parameter. If it does not match, the connection to the primary data server fails. Oracle supplemental logging level Specifies the level of supplemental logging. table Table: Table level turns on supplemental logging for subscribed tables and some required system tables. Database: Database level turns on supplemental logging for all tables, including system tables. Maximum operation queue size Maximum scan queue size Maximum session cache size The maximum number of operations permitted in the log reader operation queue during replication. The maximum number of log records permitted in the log reader log scan queue during replication. The maximum number of Oracle sessions to be cached in memory during replication. 1000 1000 1000 212 2016 SAP SE or an SAP affiliate company. All rights reserved.

Parameter Description Default value Enable parallel scanning Queue size of parallel scan tasks Parallel scan SCN range Number of parallel scanners Number of log record rows fetched by the scanner at a time Ignore log record processing errors Replicate LOB columns Ignore data of unsupported types stored in ANYDATA Database connection pool size Number of times to retry to connect if a connections fails Timeout in seconds to retry connecting Number of wrapped log files Enable verbose trace Maximum log file size Specifies whether to turn on parallel scanning. To achieve better performance for highvolume log throughput, set the parameter to true to enable parallel scanning. Specifies the number of tasks in the queue. The maximum number of system change numbers (SCN) processed by each Oracle LogMiner scanner, when parallel scan is enabled, that is, when lr_parallel_scan is true. Specifies the number of parallel scanners. Specifies the number of log record rows fetched by the scanner. Determines whether to ignore log record processing errors. Oracle logs all LOB data (except for BFILE datatypes) in the Oracle redo log. This allows the agent to apply each individual LOB change. However, for BFILE data, the same technique is used. Specifies whether you want to ignore data with unsupported types housed in ANYDATA wrapper. Maximum number of connections allowed in the connection pool on a secondary node. Instructs the client library (DBLIB, ODBC, ADO, and so on) to keep retrying the connection attempt, as long as the server is not found, for the specified number of times. The number of seconds the agent waits between retry attempts to connect to the primary database. The maximum size, in 1K blocks, of the agent system log file before wrapping. Enables or disables additional diagnostic information in the agent system log files. Limit the size of the message log to conserve disk space. false 0 1024 4 1 false true false 15 5 10 3 false 1000 2016 SAP SE or an SAP affiliate company. All rights reserved. 213

Related Information Oracle Database Permissions [page 202] Oracle Supplemental Logging [page 203] 8.11.4 Oracle to SAP HANA Data Type Mapping The following table shows the conversion between Oracle data types and SAP HANA data types. Oracle data type INTEGER NUMBER NUMBER(19)-NUMBER(38) NUMBER(10)-NUMBER(18) NUMBER(5)-NUMBER(9) NUMBER(2)-NUMBER(4) NUMBER(1) SAP HANA data type DECIMAL DECIMAL DECIMAL BIGINT INTEGER SMALLINT TINYINT NUMBER(p,s) DOUBLE (if s > p) DECIMAL (if 0 < s <= P) SMALLINT (if s < 0 and p-s <= 4) INTEGER (if s < 0 and 4 < p-s <= 9) BIGINT (if s < 0 and 9 < p-s <= 18) DECIMAL (if s< 0 and p-s > 18) FLOAT FLOAT(1)-FLOAT(24) FLOAT(25)-FLOAT(126) BINARY_FLOAT BINARY_DOUBLE DATE TIMESTAMP(n) CHAR NCHAR 2 N N2 BLOB DOUBLE REAL DOUBLE REAL DOUBLE TIMESTAMP TIMESTAMP N N N BLOB 214 2016 SAP SE or an SAP affiliate company. All rights reserved.

Oracle data type BFILE RAW LONG LONG RAW CLOB NCLOB INTERVAL TIMESTAMP WITH TIME ZONE TIMESTAMP WITH LOCAL TIME ZONE ROWID UROWID ANYDATA VARRAY NESTEDTAB OBJECT REF XMLANY SAP HANA data type BLOB VARBINARY CLOB BLOB CLOB/NCLOB NCLOB Not Supported Not Supported Not Supported Not Supported Not Supported Not Supported Not Supported Not Supported 8.11.5 Database Cleanup Run SQL scripts to manually clean objects from the source database. Cleanup scripts are used to drop database-level objects. Usually, you do not need to execute a cleanup script after an adapter is dropped, because database-level objects are dropped automatically by the adapter. However, in some cases, if any errors occur during or before automatically dropping these objects, the objects may not be dropped. At that point, you may need to execute the cleanup script to drop the objects. The following files are available from the <DPAgent_root>\LogReader\scripts\ directory : Database IBM DB2 Oracle MS SQL Server File name db2_logreader_cleanup.sql oracle_logreader_cleanup.sql mssql_logreader_cleanup.sql 2016 SAP SE or an SAP affiliate company. All rights reserved. 215

8.11.6 Log Reader Adapter Log Files You can review processing information in the Log Reader log files. The following files are available: Log file name and location <DPAgent_root>/LogReader/admin_logs/ admin<instance>.log <DPAgent_root>/log/repagent.log Description Log Reader administration log Log Reader instance log 8.12 Microsoft Excel Connect to MS Excel remote sources using the Excel Adapter. The Microsoft Excel adapter enables SAP HANA users to read Excel files that are accessible on the computer running the Adapter with the operating user permissions the adapter/agent had been started under. Note Excel files must be located in the same directory where the data provisioning agent was installed. (<DPAgent_root>/dataprovagent/excel directory, or a sub-directory). Adapter Functionality This adapter supports the following functionality: Virtual table as a source Search for tables In addition, this adapter supports the following capabilities: Table 51: Global Settings Functionality SELECT from a virtual table INSERT into a virtual table Execute UPDATE statements on a virtual table Execute DELETE statements on a virtual table Different capabilities per table Supported? Yes No No No No 216 2016 SAP SE or an SAP affiliate company. All rights reserved.

Functionality Different capabilities per table column Realtime Supported? No No 8.12.1 MS Excel Adapter Preferences Options for controlling the MS Excel adapter. Option Access Token Description A password. An access token protects the Excel files from access from a different remote source. Use this same password when creating a remote source. 8.12.2 MS Excel Remote Source Configuration Options for connecting to the remote MS Excel data. Category Parameter Description File System Folder The directory of Excel files. It must point to a sub-directory of the data provisioning install root, and it will be treated as the parent directory of this remote source. All Excel files that you want to load must be under this parent directory. If the subdirectory does not currently exist, the Excel Adapter will create it automatically. Table First Row as Header Determines whether the first row of the sheet is considered the header. If set to True, each column's content will be used as the column name of the virtual table in HANA. Values: True (Default): The columns of first row will be used as column name of virtual table in HANA. False: The column names of virtual table in HANA will be defined as COL1, COL2, COL3,.. 2016 SAP SE or an SAP affiliate company. All rights reserved. 217

Category Parameter Description Start Row of Data Determines which row of the sheet is the first data row the Excel adapter loads into virtual table. The value must be no greater than the max row number of the sheet. If "First Row as Header" is set to TRUE, its default value is 2. If "First Row as Header" is set to FALSE, its default value is 1. End Row of Data Determines which row of the sheet is the last data row the adapter loads into virtual table. The value must be no greater than the max row number of the sheet. By default, all rows of the sheet will be loaded. Show Hidden Column and Rows Determines whether to process the columns which are hidden from the sheet. Values: True: The hidden columns will be processed as normal columns. False (Default): The hidden columns will be ignored. Column Filter The list of columns which will be processed. Any column that does not exist in the list will be ignored. Enter a list of column names separated by semicolon. Note If the "Column Filter" option is empty, all columns will be processed; If the "Column Filter" option is not empty, only the listed columns will be processed. For example, if you set the option to COL1;COL2;COL3, any other columns than COL1, COL2 and COL3 will be ignored. Default: All columns will be processed Credentials Credentials mode technical user or secondary user Select one of the choices, depending on the purpose of the remote source you want to create. 218 2016 SAP SE or an SAP affiliate company. All rights reserved.

Category Parameter Description User Token for Excel Folder Access The same password as the adapter "Access Token" preference. If this is left blank or is different from the Access Token, the remote source is not allowed to read the Excel files. 8.12.3 MS Excel to SAP HANA Data Type Mapping The following table shows the data type conversion between a remote MS Excel source and an SAP HANA target. Because MS Excel does not allow data type definition, like a database, SAP HANA data types are assigned based on the cell format. Excel Column SAP HANA Data Type Examples NUMERIC (values) INTEGER Integer number (>= -2,147,483,648 and <= 2,147,483,647) BIGINT Integer number >= -9,223,372,036,854,775,808 and < -2,147,483,648 > 2,147,483,647 and <= 9,223,372,036,854,775,807 DECIMAL (34, 15) DATE All non-integer numbers are mapped to DECIMAL (34, 15). Integer numbers of < -9,223,372,036,854,775,808 or > 9,223,372,036,854,775,807 are mapped to DECIMAL (34, 15). The column style format meets either of below conditions: Format contains both d and m Format contains both m and y TIME The column style format meets either of below conditions: Format contains both h and m Format contains both m and s TIMESTAMP The column style format meets the conditions of both TIME and DATE BOOLEAN (values) INTEGER OTHERS N(n) n is the max length of all rows' content of the same column Note If different rows of the same column have different column style format: If those different column formats are incompatible, the column will be mapped to N in SAP HANA. (For example, one row has numeric value and another row has a text value.) 2016 SAP SE or an SAP affiliate company. All rights reserved. 219

If those different column formats are compatible, the column will be mapped to the data type which can cover both of them. For example, if one row has an integer value and another row has a decimal value, then it will be mapped to DECIMAL. 8.13 OData The Open Data Protocol (OData) is a standardized protocol for exposing and accessing information from various sources based on core protocols including HTTP, AtomPub (Atom Publishing Protocol), XML, and JSON (Java Script Object Notation). OData provides a standard API on service data and metadata presentation and data operations. The OData adapter supports the following functionality: Virtual table as a source Virtual table as a target using a Data Sink in a flowgraph 8.13.1 OData Adapter Get access to the OData service provider and its data and metadata. The SAP OData Adapter is a data provisioning adapter that provides OData client access to the OData service provider and its data/metadata. The OData service provider is created as a remote source. OData resources are exposed as metadata tables of the remote source. These metadata tables can be added as virtual tables. OData data can then be accessed via SAP HANA SQL query. Collections of OData data entries are represented as rows of the virtual table. The data of the main navigation entities can be accessed via SQL, with the following restrictions: Without a Join, selected projection columns will appear in the OData system query $select. With a Join, columns of the joined table (associated OData entity) can occur in the projection. Selected projection columns will appear in the OData system query $select. All joined tables will appear in the OData system query $expand. Due to a restriction of the OData system queries $select and $orderby, no expressions can occur in the Projection and the Order By clause. The Where clause supports logical, arithmetic, and ISNULL operators, string functions, and date functions. The expression is translated into the OData system query $filter. Refer to OData documentation for the OData URI conventions. Related Information URI Conventions 220 2016 SAP SE or an SAP affiliate company. All rights reserved.

8.13.2 Installation and Deployment You must configure the SAP HANA server and provide the appropriate settings when you create a remote source to connect to the service provider. Unlike other adapters, the OData adapter is not installed with the Data Provisioning Agent. 8.13.2.1 Set up the SAP HANA Server Follow these steps to set up the SAP HANA server. Procedure 1. Ensure that the Data Provisioning server is enabled. 2. If the OData service provider site only supports HTTPS connection 3. Use SAP's cryptography tool SAGENPSE delivered with SAP HANA to create a personal security environment (PSE) file and to import the public certificate provided by the server. Refer to Creating a PSE for the Server Using SAPGENPSE for command options. Make sure you put the PSE file into the SECUDIR (<HDB_INSTANCE>/<HOST_NAME>/sec) folder. 4. If the OData service provider requires HTTP basic authentication, set up a user and password. Related Information Enable the Data Provisioning Server [page 13] Creating a PSE for the Server Using SAPGENPSE 8.13.2.2 Create an OData Remote Source Follow these steps to create an OData remote source. Procedure 1. Make sure the "SYS"."ADAPTERS_" table exists with the ODataAdapter Adapter Type. To add the OData adapter to the ADAPTERS table, run the following SQL: CREATE ADAPTER "ODataAdapter" PROPERTIES 'display_name=odata Adapter;description=OData Adapter' AT LOCATION DPSERVER; 2016 SAP SE or an SAP affiliate company. All rights reserved. 221

2. Create a remote source of ODataAdapter type, and provide the following information: Option URL Proxy Server Proxy Port Trust Store Support Format Query Require CSRF Header CSRF Header Name CSRF Header Fetch Value Credentials Mode Description OData service provider URL Proxy host if URL is outside the firewall Proxy port if URL is outside the firewall Trust store file in SECUDIR that contains the OData client s public certificate If set to true, $format=json is appended to get OData entities in JSON format. If set to false, no format query is appended. The default value is true. Enter true if OData Service requires CSRF Header. The default value is true. Enter the name used for CSRF Header. The default value is X-CSRF-Token Enter the value used for CSRF Header Fetch. The default value is Fetch technical user or secondary user Select one of the choices, depending on the purpose of the remote source you want to create. User Name Password User name for HTTP Basic Authentication in username or username/companyid format Password for HTTP Basic Authentication 3. Expand the new OData adapter remote source to browse for table metadata information, and add tables from that remote source as virtual tables. 4. Expand the Catalog/<SCHEMA>/Tables to find the added virtual tables to view the table definition or to preview table data. Related Information Adapter Remote Source User Credentials [page 119] 222 2016 SAP SE or an SAP affiliate company. All rights reserved.

8.13.3 OData to SAP HANA Data Type Mapping The following table shows the mapping of data types between OData and SPA HANA. OData Data Type Edm.String Edm.Boolean Edm.Guid Edm.Binary Edm.Byte Edm.SByte Edm.Int16 Edm.Int32 Edm.Int64 Edm.Decimal Edm.Single Edm.Double Edm.DataTimeOffset Edm.Date Edm.DateTime Edm.Time Edm.Stream Edm.ComplexType Edm.EnumType Edm.Geography Edm.Geometry Edm.DateTimeOffset SAP HANA Data Type nvarchar varchar varchar blob tinyint tinyint smallint integer bigint decimal real double timestamp date timestamp time blob nvarchar (JSON format) nvarchar nvarchar (JSON format) nvarchar (JSON format) timestamp 2016 SAP SE or an SAP affiliate company. All rights reserved. 223

8.13.4 Deploy HTTPS OData Services on S/4HANA or HANA Cloud Portal Deploy HTTPS OData services on S/4HANA or HCP. Context You will need to use the Trust Manager to successfully deploy OData services on S/4HANA or HCP. Procedure 1. Use the XS Admin Trust Manager SAML Trust Store and import the S/4HANA or HCP certificate to the SAML trust store. This will import the certificate to the sapsrv.pse file in SECUDIR. See the SAP HANA Administration Guide for more information about the Trust Manager and trust relationships. 2. Create the S/4HANA or HCP Remote Source with the PSE file pointing to the sapsrv.pse file. 3. Use SAP Hana studio or Web Workbench to browse the remote source and create virtual tables. Related Information Managing Trust Relationships 8.14 Microsoft Outlook Access Microsoft Outlook data by using the Outlook adapter. You can access Microsoft Outlook data (PST file) using the Outlook adapter. This adapter supports the following functionality: Virtual table as a source 8.14.1 Outlook Adapter Preferences Configuration parameters for the Outlook adapter. You can adjust Outlook adapter settings in the Data Provisioning Agent Configuration Tool. (<DPAgent_root>/ configtool/dpagentconfigtool.exe) 224 2016 SAP SE or an SAP affiliate company. All rights reserved.

Parameter Description Default value Access Token A password to access Outlook PST files. This exact value must be used when setting up an Outlook remote source. 8.14.2 Outlook Remote Source Configuration Configuration settings for accessing a Microsoft Outlook source. Configure the following options in smart data access to configure your connection to an Outlook PST file. Option PST File Location Credentials Mode PST File Access Token Description Specifies the path and file name to the PST file the adapter will read from. The user of the DP agent must have permission to access this PST file. Technical user or secondary user. Select one of the choices, depending on the purpose of the remote source you want to create. Specifies the access token. This value must be the same as the Access Token value in the Outlook adapter preferences set in the Data Provisioning agent configuration tool. 8.14.3 Outlook Remote Source SQL Configuration CREATE REMOTE SOURCE "PST_REMOTE_SOURCE" ADAPTER "OutlookAdapter" AT LOCATION AGENT "OutlookAgent" CONFIGURATION '<?xml version="1.0" encoding="utf-8" standalone="yes"?> <ConnectionProperties name="configurations" displayname="configurations"> <PropertyGroup name="pstfileinformation" displayname="pst File Information"> <PropertyEntry name="pstlocation" displayname="pst File Location" description="indication where the PST file locate" isrequired="true"><pst File Location></PropertyEntry> </PropertyGroup> </ConnectionProperties>' WITH CREDENTIAL TYPE 'PASSWORD' USING '<CredentialEntry name="pstaccesstoken"> <password><pst Access Token></password> </CredentialEntry>'; 8.14.4 PST File Tables You can access the mail message and mail attachment tables. The only tables that the adapter can access are the mail message table and the mail attachment table. 2016 SAP SE or an SAP affiliate company. All rights reserved. 225

Mail Message Table This table will store all the mail messages in this folder. Mails in sub-folders will not be replicated. Column Name MSG_ID SUBJECT SENDER_NAME CREATION_TIME LAST_MDF_TIME COMMENT DESC_NODE_ID SENDER_MAIL_ADDR RECIPIENTS DISPLAYTO DISPLAYCC DISPLAYBCC IMPORTANCE PRIORITY ISFLAGGED MESSAGEBODY Data Type (256) N(4096) N(4096) TIMESTAMP TIMESTAMP NCLOB (1024) (256) CLOB CLOB CLOB CLOB (100) (100) TINYINT NCLOB Mail Attachment Table This table stores attachments attached to the mails in a folder and no sub-folder will be expand to search attachments. Column Name MSG_ID LONG_FILENAME Data Type (1024) N(4096) 226 2016 SAP SE or an SAP affiliate company. All rights reserved.

Column Name FILENAME DISPLAY_NAME PATH_NAME CREATION_TIME MODIFICATION_TIME SIZE COMMENT CONTENT Data Type N(4096) N(1024) N(1024) TIMESTAMP TIMESTAMP INTEGER NCLOB BLOB 8.15 SAP ASE The SAP ASE adapter provides realtime replication and change data capture functionality to SAP HANA or back to a virtual table. The SAP ASE Adapter is a data provisioning adapter that receives the data stream from SAP ASE database, reformats the data and then sends the data change to the downstream Data Provisioning Server component to provide real-time change data capture capability in order to replicate change data from SAP ASE to SAP HANA in real-time manner. The SAP ASE adapter service provider is created as a remote source, and requires the support of artifacts like virtual tables and remote subscription for each source table to perform replication. Note Be sure that after you have upgraded to the SPS 10 Data Provisioning agent you finish completing the setup of the SAP ASE adapter by following the instructions found in the topic Update the SAP ASE Adapter Preferences. The SAP ASE adapter supports the following functionality: Virtual table as a source Realtime change data capture Virtual table as a target using a Data Sink in a flowgraph (including CDC) Parallel reader (flowgraph) Loading options for target tables Search for a table Replication monitoring and statistics 2016 SAP SE or an SAP affiliate company. All rights reserved. 227

Related Information Update the SAP ASE Adapter Preferences [page 56] 8.15.1 Set Up Your SAP ASE Database The remote database must be set up properly when using the SAP ASE adapter. Procedure 1. Connect to an SAP ASE data server using ISQL or other utility and create a database you re going to replicate (if it doesn t already exist). 2. Create the primary user and grant permissions. Sample Code SQL> use master SQL> go SQL> create login <login_name> with password <password> default database <database_name> SQL> go SQL> use <database_name> SQL> go SQL> sp_adduser <login_name> SQL> go SQL> sp_role grant, replication_role, <login_name> SQL> go 3. Create the maintenance user. Sample Code SQL> use master SQL> go SQL> create login <login_name> with password <password> default database <database_name> SQL> go SQL> use <database_name> SQL> go SQL> sp_adduser <login_name> SQL> go 4. Add a new entry for the SAP ASE adapter in the interface file of the SAP ASE data server. For example: Sample Code <entry name> master tcp ether <host name or IP> <port> query tcp ether <host name or IP> <port> 228 2016 SAP SE or an SAP affiliate company. All rights reserved.

Note The entry name must be the same as the Adapter Instance Name specified when creating the remote source. The host name or IP must be the name IP of the computer where the SAP ASE adapter will be running. The port must be the same as the SAP ASE Adapter Server port that you set up in the SAP ASE adapter interface file located in <DPAgent_root>/Sybase/interfaces. 5. Reboot the SAP ASE data server. Related Information SAP ASE Remote Source Configuration Parameters [page 229] SAP ASE Adapter Preferences [page 230] 8.15.2 SAP ASE Remote Source Configuration Parameters Options for connecting to the remote SAP ASE data server. Category Option Description Data Server Information Data Server Name The SAP ASE data server name. Data Server Host Host name or IP address on which the remote SAP ASE data server is running. Data Server Port Number The SAP ASE data server port number. Security Properties Database Name Enable SSL Encryption The SAP ASE database name. Specifies whether you will be using SSL encryption between the source SAP ASE data server and SAP ASE adapter. Adapter Properties Common Name in Server Certificate Adapter Instance Name The common name in SAP ASE Adapter certificate file. The SAP ASE adapter instance name, which needs to be specified when creating remote source and will be shown in HANA studio after the remote source is created. The adapter instance name can be determined by you, and it should be unique in the same HANA server. 2016 SAP SE or an SAP affiliate company. All rights reserved. 229

Category Option Description Credential Properties Data Transfer Protocol Maintenance User Credentials Mode The protocol the SAP ASE data server and SAP ASE adapter will be using to transfer data. Only CI or LTL is allowed. The default value is CI. The maintenance user that will be used by the ASE Replication Agent Thread to filter transactions applied by this user. technical user or secondary user Select one of the choices, depending on the purpose of the remote source you want to create. User Name The SAP ASE database user which ASE adapter needs to log on to SAP ASE database to configure the SAP ASE Replication Agent, query data for initial load, and write back data into SAP ASE when updating a virtual table in HANA. Password SAP ASE database user password. 8.15.3 SAP ASE Adapter Preferences Options to control the SAP ASE adapter. Parameter Adapter Server Name Adapter Server Port Enable SSL for Adapter Server SSL Certificate File Path SSL Certificate File Password Description The name of the SAP ASE adapter server that receives data changes from the SAP ASE data server. The port number for the SAP ASE adapter server. Specifies whether to use SSL for the adapter server. Location of the SSL certificate file. The password for accessing the SSL certificate file. 8.15.4 SAP ASE to SAP HANA Data Type Mapping The following table shows data type mappings from SAP ASE to SAP HANA. The SAP ASE adapter supports compressed LOB. 230 2016 SAP SE or an SAP affiliate company. All rights reserved.

SAP ASE Data Type BIGDATATIME BIGINT BIGTIME BINARY BIT CHAR DATE DATETIME DECIMAL DOUBLE PRECISION FLOAT IDENTITY INT IMAGE SAP HANA Data Type TIMESTAMP BIGINT TIMESTAMP VARBINARY TINYINT DATE TIMESTAMP DECIMAL DOUBLE DOUBLE DOUBLE INTEGER BLOB LONGSYSNAME (255) MONEY DECIMAL (19,4) NCHAR NUMERIC N REAL SMALLDATETIME SMALLINT N DECIMAL N REAL TIMESTAMP SMALLINT SMALLMONEY DECIMAL (10,4) SYSNAME (3) TEXT CLOB TIMESTAMP VARBINARY (8) 2016 SAP SE or an SAP affiliate company. All rights reserved. 231

SAP ASE Data Type SAP HANA Data Type TINYINT TINYINT TIME TIME UNICHAR UVCHAR UNI N UNSIGNED bigint DECIMAL UNSIGNED int BIGINT UNSIGNED smallint INTEGER UNITEXT NCLOB VARBINARY VARBINARY 8.16 SAP ECC SAP ECC adapters are a set of data provisioning adapters to provide access to and interaction with SAP ECC data and metadata. All adapters designed to work with SAP ECC are built on top of Data Provisioning Log Reader adapters for the same database. Currently supported are the following: IBM DB2 Oracle MS SQL Server SAP ASE These adapters provide additional ECC-specific functionality: ECC metadata browsing and support for cluster tables and pooled tables in SAP ECC. Please see the description of Log Reader adapters for the common functionality. Note Before registering your adapter with the SAP HANA system, be sure that you have downloaded and installed the correct JDBC libraries. For information about the proper JDBC library for your source, see the Product Availability Matrix. You will also find information about supported ECC versions. Place your files in <DPAgent_root>/lib, and you will need to manually create the /lib folder. 232 2016 SAP SE or an SAP affiliate company. All rights reserved.

Adapter Functionality The ECC adapters support the following functionality: Realtime change data capture Parallel reader (flowgraph and replication) Loading options for target tables DDL propagation (transparent tables only) Search for tables In addition, this adapter supports the following capabilities: Table 52: Global Settings Functionality SELECT from a virtual table INSERT into a virtual table Execute UPDATE statements on a virtual table Execute DELETE statements on a virtual table Different capabilities per table Different capabilities per table column Realtime Supported? Yes No No No Yes Yes Yes Table 53: Select Options Functionality Select individual columns Add a WHERE clause JOIN multiple tables Aggregate data via a GROUP BY clause Support a DISTINCT clause in the select Support a TOP or LIMIT clause in the select ORDER BY GROUP BY Supported? Yes Yes Yes Yes Yes Yes Yes Yes 2016 SAP SE or an SAP affiliate company. All rights reserved. 233

Related Information SAP HANA Smart Data IntegrationProduct Availability Matrix SAP ASE [page 227] IBM DB2 Log Reader [page 167] Microsoft SQL Server Log Reader [page 182] Oracle Log Reader [page 199] 8.16.1 Terminology Setting up ECC adapters requires an understanding of certain SAP ERP and ECC concepts. Here are some key terms and concepts that will help understand how to set up your ECC adapters. Term SAP ERP SAP ECC (SAP ERP Central Component) Cluster table Pooled table Description Enterprise Resource Planning software that allows you to leverage role-based access to critical data, applications, and analytical tools and streamline your processes across procurement, manufacturing, service, sales, finance, and HR. The central technical component of SAP ERP system. A logical table type where the data of several such tables are stored together as a table cluster in the database. The intersection of the key fields of the cluster tables forms the primary key of the table cluster. Therefore, a cluster table is known in the ABAP Dictionary, but not in the database. A logical table type where the data of several such tables are stored together as a table pool in the database. Therefore, a pooled table is known in the ABAP Dictionary, but not in the database. Related Information Loading Metadata for Cluster and Pooled Tables [page 236] 8.16.2 Installation and Setup Refer to Log Reader and SAP ASE adapters for installation and setup information. Because the SAP ECC adapters are built on top of existing Data Provisioning adapters, you must use the procedures of those adapters to build your SAP ECC adapters. 234 2016 SAP SE or an SAP affiliate company. All rights reserved.

Related Information SAP ASE [page 227] 8.16.3 Permissions for ECC Dictionary Tables To replicate SAP ECC dictionary tables, you'll need specific permissions, depending on the database you are using. Table 54: Database Permissions or instructions DB2 GRANT DBADM, CREATETAB, BINDADD, CONNECT,CREATE_NOT_FENCED_ROUTINE, IM PLICIT_SCHEMA, LOAD, CREATE_EXTERNAL_ROUTINE, QUIESCE_CONNECT ON DATABASE TO USER DPADM Oracle Permissions are granted when setting up your adapter by running the script found in the oracle_init_example.sql file, which is located in the Scripts folder of the Data Provisioning Agent installation (<DPAgent_root>\LogReader\Scripts). 8.16.4 Create an ECC Remote Source Create a remote source for your ECC adapter. Context The following is an example of creating an ECC Adapter remote source in SAP HANA studio. Procedure 1. In the Systems view, open Provisioning Remote Sources 2. Right click Remote Sources, and select New Remote Source. 3. Enter the required information for your particular database and ECC information. The following are the ECC-specific parameters for creating a remote source. You can find information about database-specific parameter information in the remote source parameter topics for Log Reader adapters. 2016 SAP SE or an SAP affiliate company. All rights reserved. 235

Option Description ERP Additional Info Owner/Schema The owner of the ECC tables or the schema where the ECC tables are located. ERP Additional Info Dictionary Schema If you want to use pool or cluster tables, you have to replicate a set of DD* tables into SAP HANA. The Dictionary Schema must be the schema name where those tables are replicated. 4. Click the Save the Editor icon in the upper right-hand corner of the screen. Related Information MS SQL Server Log Reader Remote Source Configuration [page 193] DB2 Log Reader Remote Source Configuration [page 178] Oracle Log Reader Remote Source Configuration [page 205] Adapter Remote Source User Credentials [page 119] 8.16.5 Loading Metadata for Cluster and Pooled Tables Load cluster and pooled table metadata. Before working with cluster or pooled tables, their metadata must be loaded into SAP HANA into a schema specified by the dictionaryschema attribute. To do this, execute the replication_dictionary.sql script, and execute the stored procedures that it creates. 1. In SAP HANA studio, choose File Open and navigate to <DPAgent_root>/LogReader/scripts. 2. Click Execute. This creates two stored procedures. 3. Execute both stored procedures: call <schema_name>.create_table_subscription( <schema_name>, <remote_source_name> ) call <schema_name>.operate_subscription( <schema_name>, <remote_source_name> ) 8.16.6 Data Type Mapping Data type mapping is determined by the kind of table used. For transparent tables, the mapping is taken from the underlying database. For cluster and pooled tables, the following table shows the ABAP data types and the corresponding SAP HANA data types. ABAP Data Type Data Type in ABAP Dictionary SAP HANA Data Type CHAR N 236 2016 SAP SE or an SAP affiliate company. All rights reserved.

ABAP Data Type Data Type in ABAP Dictionary SAP HANA Data Type CLNT CUKY LANG SSTR STRG UNIT VARC ACCP NUMC DATS TIMS LCHR D16D D34D CURR QUAN DEC PREC D16R D16S D34R D34S RAW FLTP INT1 INT2 N N N N N N N NCLOB DECIMAL DECIMAL DECIMAL DECIMAL DECIMAL DECIMAL VARBINARY VARBINARY VARBINARY VARBINARY VARBINARY DOUBLE SMALLINT SMALLINT 2016 SAP SE or an SAP affiliate company. All rights reserved. 237

ABAP Data Type Data Type in ABAP Dictionary SAP HANA Data Type INT4 INTEGER LRAW BLOB RSTR BLOB F P D T N C DOUBLE DECIMAL N If a data type is not defined in the table above, it is imported as VARBINARY. The order in the table determines the order of mapping: for example, an LCHR field will be imported as VARBINARY even though it has ABAP data type of C. 8.16.7 Adapter Preferences Access adapter settings specific to your source. You can adjust adapter settings specific to your source in the Data Provisioning Agent Configuration Tool. (<DPAgent_root>/configTool/dpagentconfigtool.exe) Use these settings to tune performance, enhance security, and so on. 8.16.7.1 DB2ECCAdapter Preferences Configuration parameters for the DB2 ECC adapter. Note Log Reader and ECC adapter preferences (except for Number of wrapped log files, Enable verbose trace, and Maximum log file size) are no longer set in the Data Provisioning Agent Configuration Tool. They are now moved to the remote source configuration options in SAP HANA. If you have upgraded from a previous version, then the settings you find in the Agent Configuration Tool are your previous settings, displayed for your reference. Parameter Description Default value Maximum operation queue size The maximum number of operations permitted in the log reader operation queue during replication. 1000 238 2016 SAP SE or an SAP affiliate company. All rights reserved.

Parameter Description Default value Maximum scan queue size Ignore log record processing errors Replicate LOB columns Database connection pool size Number of times to retry to connect if a connections fails Timeout in seconds to retry connecting Number of wrapped log files Enable verbose trace Maximum log file size The maximum number of log records permitted in the log reader log scan queue during replication. Determines whether to ignore log record processing errors. Determines whether the agent applies each individual LOB change. Maximum number of connections allowed in the connection pool on a secondary node. Instructs the client library (DBLIB, ODBC, ADO, and so on) to keep retrying the connection attempt, as long as the server is not found, for the specified number of times. The number of seconds the agent waits between retry attempts to connect to the primary database. The maximum size, in 1K blocks, of the Replication Agent system log file before wrapping. Enables or disables additional diagnostic information in the agent system log files. Limit the size of the message log to conserve disk space. 1000 false true 15 5 10 3 false 1000 8.16.7.2 MssqlECCAdapter Preferences Configuration parameters for the MS SQL ECC adapter. Note Log Reader and ECC adapter preferences (except for Number of wrapped log files, Enable verbose trace, and Maximum log file size) are no longer set in the Data Provisioning Agent Configuration Tool. They are now moved to the remote source configuration options in SAP HANA. If you have upgraded from a previous version, then the settings you find in the Agent Configuration Tool are your previous settings, displayed for your reference. Parameter Description Default value Maximum operation queue size The maximum number of operations permitted in the log reader operation queue during replication. 1000 2016 SAP SE or an SAP affiliate company. All rights reserved. 239

Parameter Description Default value Maximum scan queue size Maximum wait interval between log scans The maximum number of log records permitted in the log reader log scan queue during replication. The maximum wait interval between Log Reader transaction log scans. 1000 60 Note The value of the parameter is the maximum number of seconds that can elapse before the Log Reader component scans the transaction log for a transaction to be replicated, after a previous scan yields no such transaction. For reduced replication latency in an infrequently updated database, we recommend lower number settings for the parameter. If the primary database is continuously updated, the value of the parameter is not significant to performance. 240 2016 SAP SE or an SAP affiliate company. All rights reserved.

Parameter Description Default value Seconds to add to each log scan wait interval The number of seconds to add to each wait interval before scanning the transaction log, after a previous scan yields no transaction to be replicated. 5 Note The value of the parameter is the number of seconds added to each wait interval before the LogReader component scans the log for a transaction to be replicated, after a previous scan yields no such transaction. The number of seconds specified by the parameter is added to each wait interval, until the wait interval reaches the value specified by the Maximum wait interval between log scans parameter. For optimum performance, the value of the parameter should be balanced with the average number of operations in the primary database over a period of time. In general, better performance results from reading more operations from the transaction log during each LogReader scan. With a primary database that is less frequently updated, increasing the value of the parameter may improve overall performance. If the database is continuously updated, the value of the parameter may not be significant to performance. Replicate LOB columns Database connection pool size Determines whether the agent applies each individual LOB change. Maximum number of connections allowed in the connection pool on a secondary node. true 15 2016 SAP SE or an SAP affiliate company. All rights reserved. 241

Parameter Description Default value Number of times to retry to connect if a connections fails Timeout in seconds to retry connecting Number of wrapped log files Enable verbose trace Maximum log file size Instructs the client library (DBLIB, ODBC, ADO, and so on) to keep retrying the connection attempt, as long as the server is not found, for the specified number of times. The number of seconds the agent waits between retry attempts to connect to the primary database. The maximum size, in 1K blocks, of the agent system log file before wrapping. Enables or disables additional diagnostic information in the agent system log files. Limit the size of the message log to conserve disk space. 5 10 3 false 1000 8.16.7.3 OracleECCAdapter Preferences Configuration parameters for the Oracle ECC adapter. Note Log Reader and ECC adapter preferences (except for Number of wrapped log files, Enable verbose trace, and Maximum log file size) are no longer set in the Data Provisioning Agent Configuration Tool. They are now moved to the remote source configuration options in SAP HANA. If you have upgraded from a previous version, then the settings you find in the Agent Configuration Tool are your previous settings, displayed for your reference. Parameter Description Default value Timezone file path This parameter needs to be set only when the Data Provisioning Agent and the remote Oracle database are on different machines. If this is the case, enter the full path to the Oracle timezone file AND the timezone file name. Directory containing Oracle archive redo log files Specifies the location of the archived redo log files. If the LogReader cannot find an expected log record in the Oracle online redo logs, it will search this directory for the archived log file containing the required record. 242 2016 SAP SE or an SAP affiliate company. All rights reserved.

Parameter Description Default value Distinguished Name (DN) in Certificate The distinguished name (DN) of the primary data server certificate. his parameter is only valid if Use SSL is set to true. If this parameter is set, the DN field in the server certificate is verified to match this parameter. If it does not match, the connection to the primary data server fails. Oracle supplemental logging level Specifies the level of supplemental logging. table Table: Table level turns on supplemental logging for subscribed tables and some required system tables. Database: Database level turns on supplemental logging for all tables, including system tables. Maximum operation queue size Maximum scan queue size Maximum session cache size Enable parallel scanning Queue size of parallel scan tasks Parallel scan SCN range Number of parallel scanners Number of log record rows fetched by the scanner at a time Specifies the maximum number of operations permitted in the log reader operation queue during replication. The maximum number of log records permitted in the log reader log scan queue during replication. The maximum number of Oracle sessions to be cached in memory during replication Specifies whether to turn on parallel scanning. To achieve better performance for highvolume log throughput, set the parameter to true to enable parallel scanning. Specifies the number of tasks in the queue. The maximum number of system change numbers (SCN) processed by each Oracle LogMiner scanner, when parallel scan is enabled, that is, when lr_parallel_scan is true. Specifies the number of parallel scanners. Specifies the number of log record rows fetched by the scanner. 1000 1000 1000 false 0 1024 4 1 2016 SAP SE or an SAP affiliate company. All rights reserved. 243

Parameter Description Default value Ignore log record processing errors Replicate LOB columns Ignore data of unsupported types stored in ANYDATA Database connection pool size Number of times to retry to connect if a connections fails Timeout in seconds to retry connecting Number of wrapped log files Enable verbose trace Maximum log file size Determines whether to ignore log record processing errors. Oracle logs all LOB data (except for BFILE datatypes) in the Oracle redo log. This allows the agent to apply each individual LOB change. However, for BFILE data, the same technique is used. Specifies whether you want to ignore data with unsupported types housed in ANYDATA wrapper. Maximum number of connections allowed in the connection pool on a secondary node. Instructs the client library (DBLIB, ODBC, ADO, and so on) to keep retrying the connection attempt, as long as the server is not found, for the specified number of times. The number of seconds the agent waits between retry attempts to connect to the primary database. The maximum size, in 1K blocks, of the agent system log file before wrapping. Enables or disables additional diagnostic information in the agent system log files. Limit the size of the message log to conserve disk space. false true false 15 5 10 3 false 1000 Related Information Oracle Database Permissions [page 202] Oracle Supplemental Logging [page 203] 8.16.7.4 SAP ASE ECC Adapter Preferences Options to control the SAP ASE ECC adapter. Parameter Adapter Server Name Description The name of the SAP ASE adapter server that receives data changes from the SAP ASE data server. 244 2016 SAP SE or an SAP affiliate company. All rights reserved.

Parameter Adapter Server Port Enable SSL for Adapter Server SSL Certificate File Path SSL Certificate File Password Description The port number for the SAP ASE adapter server. Specifies whether to use SSL for the adapter server. Location of the SSL certificate file. The password for accessing the SSL certificate file. 8.17 SAP ABAP Use the ABAP adapter to retrieve various types of SAP data. The ABAP adapter retrieves data from virtual tables through RFC for ABAP tables and ODP extractors. Adapter Functionality This adapter supports the following functionality: Virtual table as a source realtime change data capture for ODP extractors. In addition, this adapter supports the following capabilities: Table 55: Global Settings Functionality SELECT from a virtual table INSERT into a virtual table Execute UPDATE statements on a virtual table Execute DELETE statements on a virtual table Different capabilities per table Different capabilities per table column Realtime Supported? Yes No No No Yes No No 2016 SAP SE or an SAP affiliate company. All rights reserved. 245

Table 56: Select Options Functionality Select individual columns Add a WHERE clause JOIN multiple tables Aggregate data via a GROUP BY clause Support a DISTINCT clause in the select Support a TOP or LIMIT clause in the select ORDER BY GROUP BY Supported? Yes Yes No No No Yes No No 8.17.1 ABAP Adapter Remote Source Configuration Remote source configuration options for the ABAP adapter. Option Host Client System ID Username Password Description The IP address or DNS name of the host running the ECC to connect to. The ECC client number The ECC instance number. The user name that is used to connect to the ECC system. The user password. After you have created the remote source, the directory structure will look similar to this, depending on the structure of the source system. 246 2016 SAP SE or an SAP affiliate company. All rights reserved.

8.17.2 Data Type Mapping Data type mapping is determined by the kind of table used. For transparent tables, the mapping is taken from the underlying database. For cluster and pooled tables, the following table shows the ABAP data types and the corresponding SAP HANA data types. ABAP Data Type Data Type in ABAP Dictionary SAP HANA Data Type CHAR CLNT CUKY LANG SSTR STRG UNIT VARC N N N N N N N N 2016 SAP SE or an SAP affiliate company. All rights reserved. 247

ABAP Data Type Data Type in ABAP Dictionary SAP HANA Data Type ACCP NUMC DATS TIMS LCHR D16D D34D CURR QUAN DEC PREC D16R D16S D34R D34S RAW FLTP INT1 INT2 INT4 LRAW RSTR NCLOB DECIMAL DECIMAL DECIMAL DECIMAL DECIMAL DECIMAL VARBINARY VARBINARY VARBINARY VARBINARY VARBINARY DOUBLE SMALLINT SMALLINT INTEGER BLOB BLOB F P D T N C DOUBLE DECIMAL N 248 2016 SAP SE or an SAP affiliate company. All rights reserved.

If a data type is not defined in the table above, it is imported as VARBINARY. The order in the table determines the order of mapping: for example, an LCHR field will be imported as VARBINARY even though it has ABAP data type of C. 8.17.3 Change Data Capture The ABAP adapter relies on a periodic pull when implementing change data capture. The ABAP adapter uses a periodic pull to retrieve changed data from the source. The default interval value is 3600 seconds. An extractor can specify the interval itself in the DELTA_PERIOD_MINIMUM field of the ET_ADDITIONAL_INFO return table field of the details request. You can also override the interval value either in virtual table creation or in remote subscription creation. See the examples below. Example Setting Extraction Period in the Virtual Table create virtual table dp1_z_rodps_repl_test_aied at "DP1"."<NULL>"."<NULL>"."SAPI.Z_RODPS_REPL_TEST_AIED" remote property 'dataprovisioning_parameters'= '<Parameter name="extractionperiod">3</parameter>'; Example Setting Extraction Period in the Remote Subscription create remote subscription rs_z_rodps_repl_test_aied as (select "T1"."MANDT", "T1"."KEYFIELD", "T1"."LNR", "T1"."DATAFIELD", "T1"."CHARFIELD" from dp1_z_rodps_repl_test_aied T1 with dataprovisioning parameters ('<PropertyGroup name="ababadapter"> <PropertyEntry name="extractionperiod">5</propertyentry> </PropertyGroup>')) target table z_rodps_repl_test_aied_s; 8.17.4 Initial Load To correctly initialize the delta extraction process, you need to set the extractionname parameter to the remote subscription value of the subscription, which can be found in M_REMOTE_SSUBSCRIPTIONS view: select remote_subscription from sys.m_remote_subscriptions where subscription_name = 'RS_Z_RODPS_REPL_TEST_AIED'; insert into z_rodps_repl_test_aied (select * from dp1_z_rodps_repl_test_aied ) with dataprovisioning parameters ('<PropertyGroup> 2016 SAP SE or an SAP affiliate company. All rights reserved. 249

<PropertyGroup name="dp1_z_rodps_repl_test_aied"> <PropertyEntry name="extractionname">id185060536</propertyentry> </PropertyGroup></PropertyGroup>') 8.17.5 Delta Modes Support The RODPS supports the following delta modes: Complete Delta with Deletion Flag After-Images of Updated Records After-Images with Deletion Flag Only New Records (Inserts) 8.18 SAP HANA The SAP HANA adapter provides real-time change data capture capability in order to replicate data from a remote SAP HANA database to a target SAP HANA database. Unlike LogReader adapters, which read a remote database's log to get change data, the SAP HANA adapter is trigger-based: change data is captured by triggers, and the adapter needs to continuously query the target database to get the change data. When a table is subscribed to replicate, the adapter creates three triggers on the table for capturing INSERT, UPDATE, and DELETE, respectively. The adapter also creates a shadow table for the subscribed table. Except for a few extra columns for supporting replication, the shadow table has the same columns as its replicated table. Triggers record change data in shadow tables. For each adapter instance (remote source), the adapter creates a Trigger Queue table to mimic a queue. Each row in shadow tables has a corresponding element (or placeholder) in the queue. The adapter continuously scans the queue elements and corresponding shadow table rows to get change data and replicate them to target HANA database. Adapter Functionality This adapter supports the following functionality: Virtual table as a source Virtual table as a target using a Data Sink in a flowgraph Search for tables in a remote source DDL propagation. This adapter supports the DROP TABLE, ADD COLUMN, and DROP COLUMN operations. In addition, this adapter supports the following capabilities: 250 2016 SAP SE or an SAP affiliate company. All rights reserved.

Table 57: Global Settings Functionality SELECT from a virtual table INSERT into a virtual table Execute UPDATE statements on a virtual table Execute DELETE statements on a virtual table Different capabilities per table Different capabilities per table column Realtime Supported? Yes Yes Yes Yes No No Yes Table 58: Select Options Functionality Select individual columns Add a WHERE clause JOIN multiple tables Aggregate data via a GROUP BY clause Support a DISTINCT clause in the select Support a TOP or LIMIT clause in the select ORDER BY GROUP BY Supported? Yes Yes Yes Yes Yes Yes Yes Yes Related Information SAP HANA DDL Propagation Scan Interval [page 254] 2016 SAP SE or an SAP affiliate company. All rights reserved. 251

8.18.1 SAP HANA Remote Source Configuration Use the SAP HANA adapter to move data from one SAP HANA instance to another. Privileges The following schema privileges on the schemas, under which there are tables to be accessed, must be granted to the configured user on the remote SAP HANA database: For realtime change data capture: CREATE ANY, and DROP SELECT For SAP HANA virtual tables used as a target (Data Sink) in an.hdbflowgraph: INSERT, UPDATE, and DELETE. Usually, a user inherently has the above schema privileges if they are the schema owner (that is, the schema name and user name are the same). But if it's not, the above schema privileges must be granted to the user using the following access control statement: GRANT CREATE ANY, DROP, SELECT, INSERT, UPDATE, DELETE ON SCHEMA <schema_name> TO <user_name>; Remote Source Parameters Category Option Description Database Host The host name or IP address on which the remote SAP HANA server is running. Port Number The port number of the remote SAP HANA server. If the remote SAP HANA server is a single-container system, the port number is 3<instance>15, and the Database Name parameter must be empty. If the remote SAP HANA server is a multiple-container system, the port number is 3<instance>13, and the tenant database name must be specified in the Database Name parameter. Execute the following query to obtain the system database s SQL port. SELECT DATABASE_NAME, SQL_PORT FROM SYS_DATABASES.M_SERVICES WHERE DATABASE_NAME='SYSTEMDB' and SERVICE_NAME='nameserver' and COORDINATOR_TYPE= 'MASTER'; 252 2016 SAP SE or an SAP affiliate company. All rights reserved.

Category Option Description Database Name The database name, if the remote database is a tenant database. Leave this parameter empty if the remote database is not a tenant database. Schema Note This option is no longer required. It is visible solely for backward compatibility purposes. It was used in previous versions to restrict the viewing of tables to those under the given schema. Now, you can view all tables, regardless of the schema they are located under. For those remote sources that were created in a previous version, this option value must keep unchanged. Schema Alias Replacements Schema Alias Schema name to be replaced with the schema given in Schema Alias Replacement. If given, accessing tables under it is considered to be accessing tables under the schema given in Schema Alias Replacement. Schema Alias Replacement Schema name to be used to replace the schema given in Schema Alias. CDC Properties System Object Prefix The prefix of the names of the SAP HANA adapter system objects created in the source SAP HANA database by the adapter. We recommend keeping the default value of HADP_. Connection Pool Size Minimum Scan Interval in Seconds Maximum Scan Interval in Seconds DDL Scan Interval in Minutes Maximum number of connections allowed in the connection pool. The default value is 4. The minimum interval in seconds that the adapter scans the Trigger Queue table to get change data. The default value is 0 (seconds), which means there is no waiting time before the next scan. The maximum interval in seconds that the adapter scans the Trigger Queue table to get change data. The default value is 10 (seconds). If the adapter scans the queue and finds that the queue is empty, it will gradually increase the scan interval from the minimum scan interval to the maximum scan interval. The interval for detecting DDL changes in the source. A zero or negative integer disables this parameter. The default value is 10 (minutes). Maximum Batch Size Batch Queue Size The maximum number of consecutive change data on the same table that are batched to process and send to Data Provisioning Server together. The default value is 128. The internal batch queue size. The batch queue size determines the maximum number of batches of change data that are queued in memory. The default value is 64. 2016 SAP SE or an SAP affiliate company. All rights reserved. 253

Category Option Description Connection Security Enable SSL Encryption Validate Server Certificate Host Name in Server Certificate Specifies whether to enable SSL encryption on connections to a remote SAP HANA database. The default value is false. Specifies whether to validate the certificate of a remote SAP HANA server. This configuration is only meaningful if Enable SSL encryption is set to true. The default value is false Controls the verification of the host name field of the server certificate: If not set, the host name used for the connection is used for verification. Note that SSL is name-based; connecting to an IP address, or to localhost is unlikely to work. If set to a string, If the string is "*", any name matches. If the string starts with CN=, it is treated as a common name, and the textual representation of the common name entry in the certificate must be exactly the same. Otherwise, the host name in the server certificate must match this string (case insensitive). Credentials Credentials Mode User Password technical user or secondary user Select one of the choices, depending on the purpose of the remote source you want to create. Database user name The appropriate corresponding password Related Information SAP HANA Remote Source Configuration [page 252] SAP HANA DDL Propagation Scan Interval [page 254] 8.18.2 SAP HANA DDL Propagation Scan Interval The DDL Scan Interval in Minutes remote source configuration option is important to review when setting up DDL propagation. Enabling DDL propagation can impact the performance of the source SAP HANA database. Setting an appropriate value for the remote source option DDL Scan Interval in Minutes matters. From the time the DDL changes occurs on the source database to the time the DDL changes are propagated to the target SAP HANA database, no DML changes on the tables are allowed. At configured intervals (DDL Scan Interval in Minutes. By default, 10 minutes), the HANA adapter queries the metadata of all subscribed tables from the source HANA database, and it determines if changes to the DDL have occurred. If changes are detected, it will propagate the DDL changes to the target database through the Data Provisioning Server. 254 2016 SAP SE or an SAP affiliate company. All rights reserved.

Because the HANA adapter detects DDL changes by querying source HANA system tables, the source database might be burdened if you configure a small value for the DDL Scan Interval in Minutes option. However, configuring a large value would increase the latency of DDL propagation. Therefore, you need to figure out what value works best for you. If changes to the DDL are rare, you might even want to disable DDL propagation by setting the value of the DDL Scan Interval in Minutes option to zero. This will prevent the HANA adapter from querying metadata from the source database periodically. Related Information SAP HANA Remote Source Configuration [page 252] 8.18.3 SAP HANA Adapter Preferences Parameter Thread Pool Size Description The size of the SAPHANA adapter global thread pool. The thread pool is shared by the SAP HANA adapter remote sources. The thread pool is used to execute jobs of querying shadow tables to get change data. We recommend that you configure the thread pool size to the number of available processors in the system, if possible. 8.18.4 SAP HANA (Remote) to SAP HANA (Target) Data Type Mapping The following table shows the data type conversion between a remote SAP HANA source and an SAP HANA target. Remote SAP HANA Data Type ALPHANUM BIGINT BINARY BINTEXT BLOB BOOLEAN Target SAP HANA Data Type ALPHANUM BIGINT VARBINARY NCLOB BLOB TINYINT 2016 SAP SE or an SAP affiliate company. All rights reserved. 255

Remote SAP HANA Data Type CHAR CLOB DECIMAL Target SAP HANA Data Type CLOB DECIMAL Note An exception of the DECIMAL type mapping is that DECI MAL(34, 0) is always mapped to floating-point DECIMAL data type DOUBLE DATE INTEGER NCHAR NCLOB N REAL SECONDDATE SHORTTEXT SMALLDECIMAL SMALLINT TEXT TIME TIMESTAMP TINYINT VARBINARY DOUBLE DATE INTEGER N NCLOB N REAL SECONDDATE N DECIMAL SMALLINT NCLOB TIME TIMESTAMP TINYINT VARBINARY 8.19 SOAP The SOAP adapter provides access to SOAP Web Services via HANA SQL. The SOAP adapter is a SOAP web services client that can talk to a web service using the HTTP protocol to download the data. The SOAP adapter uses virtual functions instead of virtual tables to expose server side operations as it closely relates to how the operation is invoked. 256 2016 SAP SE or an SAP affiliate company. All rights reserved.

The SOAP adapter supports the following functionality: Virtual function as a source Related Information CREATE VIRTUAL FUNCTION 8.19.1 SOAP Adapter Remote Source Configuration Configuration settings for creating a SOAP adapter remote source. Option WSDL File Use System Proxy Treat WebServiceError (SOAP Fault) as failure Description The location of the WSDL file. Enter a URL or a path to the local file. If set to yes, the adapter will use the proxy information saved in the dpagentconfig.ini or that is set up in the DP Agent Configuration Tool. If set to True, fail web services call in case of failure. The default value is No. By default, the SOAP Adapter will write the fault to one of the output columns if the call fails. In certain scenarios, you may want to fail the call itself. For example, suppose the proxy was incorrectly configured. Sample Code Select * from getweatherbyzip(.94403..) If set to No, the select call is a success and the actual error is populated in SOAP_FAULT column. If set to Yes, the select call will fail itself and write the error to a caller. 8.19.2 SOAP Operations as Virtual Function Create a virtual function to retrieve data. SOAP web service browsing will display all of the operations as a leaf node. Because SOAP requests and responses are both XML messages, you will use the Hierarchical node in a flowgraph, with the virtual function as your source. We will use Virtual Function Configuration to keep the root schemas and referenced schemas from the WSDL file. Use the GET_REMOTE_SOURCE_FUNCTION_DEFINITION procedure, and as part of this call, the SOAP adapter will provide the following information to HANA. 2016 SAP SE or an SAP affiliate company. All rights reserved. 257

Note You can only create virtual function using SQL or the Web-based Development Workbench. We provide sample SQL code here. For more information about creating a virtual function using the Web-based Development Workbench, see the Configuration Guide for SAP HANA Smart Data Integration and SAP HANA Smart Data. Related Information Setting up the SOAP Adapter: SQL Example [page 259] 8.19.3 Process the Response Process the SOAP response using the Hierarchical node. Context You have created a virtual function, and now you can process the response using the Hierarchical node in the Web-based Development Workbench. For more information and details about the Hierarchical node, see Configure a Hierarchical Node in the SAP HANA SAP HANA Smart Data Integration and SAP HANA Smart Data Configuration Guide. Procedure 1. Open the Catalog editor and import the virtual function you want to execute. 2. Configure the remote source to use the imported virtual function. 3. Add a Hierarchical node to the flowgraph, connect it to the data source, and configure the node. 4. Add a target table to the flowgraph, connect it upstream, and name it. 5. Save the flowgraph, and execute it. 6. Click the Data Preview button on the target table to confirm that the data is saved correctly. 258 2016 SAP SE or an SAP affiliate company. All rights reserved.

8.19.4 Setting up the SOAP Adapter: SQL Example SQL example for setting up the SOAP adapter to use a remote webservice. Here are the SQL to setup the SOAP Adapter to use a remote webservice. http://www.webservicex.net/ stockquote.asmx?wsdl. You can use the Web-based Development Workbench to complete some of these tasks (for example, creating a virtual function). Sample Code drop agent "agent_local" cascade; CREATE AGENT "agent_local" PROTOCOL 'TCP' HOST 'pald00486396a.amer.global.corp.sap' PORT 5050; drop adapter "SOAPAdapter" cascade; CREATE ADAPTER "SOAPAdapter" at location agent "agent_local"; drop remote source "WebServiceRS" cascade; CREATE REMOTE SOURCE "WebServiceRS" ADAPTER "SOAPAdapter" AT LOCATION AGENT "agent_local" CONFIGURATION '<?xml version="1.0" encoding="utf-8"?> <ConnectionProperties name="connection"> <PropertyEntry name="wsdl">http://www.webservicex.net/stockquote.asmx?wsdl</ PropertyEntry> <PropertyEntry name="useproxy">yes</propertyentry> <PropertyEntry name="axisfault">no</propertyentry> </ConnectionProperties>' WITH CREDENTIAL TYPE 'PASSWORD' USING '<CredentialEntry name="webservice_credential"><user></user><password></ password></credentialentry>'; --Get the function definition from adapter CALL "PUBLIC"."GET_REMOTE_SOURCE_FUNCTION_DEFINITION" ('WebServiceRS','{"serviceName":"StockQuote","portName":"StockQuoteSoap", "operationname":"getquote","nodetype":"operation"}',?,?,?); --Using the json configuration we can now create a virtual function. drop FUNCTION getstockquote; CREATE VIRTUAL FUNCTION getstockquote(soap_request Nvarchar(5000)) RETURNS TABLE (SOAP_HEADERS Nvarchar(5000), SOAP_FAULT N(5000), SOAP_BODY NCLOB) CONFIGURATION 'use the json data returned from GET_REMOTE_SOURCE_FUNCTION_DEFINITION' AT "WebServiceRS"; --Invoking the SOAP Request is as below. select * from getstockquote( '<soapenv:envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:web="http://www.webservicex.net/"> <soapenv:header/> <soapenv:body> <web:getquote xmlns:web="http://www.webservicex.net/"> <!--Optional:--> <web:symbol>nasdaq</web:symbol> </web:getquote> </soapenv:body> </soapenv:envelope> '); 2016 SAP SE or an SAP affiliate company. All rights reserved. 259

8.20 Teradata The Teradata adapter can be used to connect to a Teradata remote source, and create a virtual table to read from and write to. Note Before registering your adapter with the SAP HANA system, be sure that you have downloaded and installed the correct JDBC libraries. For information about the proper JDBC library for your source, see the Product Availability Matrix. Place your files in <DPAgent_root>/lib, and you will need to manually create the /lib folder. Adapter Functionality This adapter supports the following functionality: Virtual table as a source Realtime change data capture Virtual table as a target using a Data Sink in a flowgraph (including CDC) Parallel reader (flowgraph and replication) Search for tables Loading options for target tables DDL propagation. In addition, this adapter supports the following capabilities: Table 59: Global Settings Functionality Supported? SELECT from a virtual table Yes INSERT into a virtual table Yes Execute UPDATE statements on a virtual table Yes Execute DELETE statements on a virtual table Yes Different capabilities per table No Different capabilities per table column No Realtime No 260 2016 SAP SE or an SAP affiliate company. All rights reserved.

Table 60: Select Options Functionality Supported? Select individual columns Yes Add a WHERE clause Yes JOIN multiple tables No Aggregate data via a GROUP BY clause Yes Support a DISTINCT clause in the select Yes Support a TOP or LIMIT clause in the select Yes ORDER BY Yes GROUP BY Yes Authentication You can set up authentication by using TD2, LDAP, or Kerberos. If you are using LDAP, you will need to provide a user name and password when setting up your remote source. If you are using Kerberos, the adapter will use the default Kerberos settings of the machine it is deployed on. If the Kerberos configuration file (krb5.conf) is in a non-standard location, the path has to be specified via the java.security.krb5.conf system property (by adding it to dpagent.ini). The Realm and KDC connection parameters in the remote source are now optional. Both have to be specified in order to override the machine's default krb5.conf settings. To use Kerberos, use the Kerberos principal name for the username with the corresponding password. Privileges The following database user privileges are required for accessing databases, tables, and so on, so that the adapter can read metadata. The user needs SELECT access on the following dbc tables: "DBC"."UDTInfo" "DBC"."DBase" "DBC"."AccessRights" "DBC"."TVM" "DBC"."TVFields" The following privileges are needed to support realtime replication (CDC): CREATE TRIGGER DROP TRIGGER 2016 SAP SE or an SAP affiliate company. All rights reserved. 261

CREATE PROCEDURE DROP PROCEDURE 8.20.1 Teradata Remote Source Configuration Options for setting up the connection to the remote Teradata data server. Configuration parameters Category Option Description Connection Host Port Number Encoding Encrypt traffic Logon mechanism Host name or IP address on which the remote Teradata data server is running The Teradata data server port number Session encoding between the adapter and Teradata. Some restrictions apply to UTF-8. For example, character columns with Graphic encoding are not supported. Controls whether the traffic between the adapter and the database is encrypted. If turned off, data exchanged between the adapter and the database is unencrypted and can be read by anyone with access to the network. Login data is not affected by this setting, because this data is always sent encrypted by the Teradata JDBC driver. Controls the authentication (+authorization) method used, and which username and password values to enter in the credentials field below. TD2 (Teradata Method 2): Use the database username and password LDAP: Directory authentication. Use credentials of users defined and configured via LDAP (Database has to be configured to use external LDAP authentication) Kerberos (KRB5): Use Kerberos principal and its username (Database has to be configured to use Kerberos external authentication) Realm (Optional when using Kerberos) authenticate using a principal from this realm (instead of the systems default realm). The Realm option must be used together with KDC. 262 2016 SAP SE or an SAP affiliate company. All rights reserved.

Category Option Description KDC JDBC FastExport Transaction Mode (Optional when using Kerberos) address of KDC (Key Distribution Center) to be used with the specified Realm (has to be used together with Realm) Speeds up retrieving large amounts of data from Teradata (disabled when using Kerberos authentication) Specifies the transaction mode for the connection. ANSI: Uses American National Standards Institute (ANSI) transaction semantics. This mode is recommended. This is the default value. TERA: Uses legacy Teradata transaction semantics. This mode is only recommended for legacy applications that require Teradata transaction semantics. DEFAULT: Uses the default transaction mode configured for the Teradata Database, which may be either ANSI or TERA mode. Additional Teradata JDBC connection options. The parameters must be specified in the following format: key=value,key=value,... CDC Properties Database Additional Connection Parameters http://developer.teradata.com/doc/connectivity/jdbc/reference/current/frameset.html Name of the database. System Object Prefix The prefix of the names of the Teradata adapter system objects created in the source Teradata database by the adapter. We recommend keeping the default value of TADP_. Shadow Table Prefix The prefix of the names of the Teradata adapter shadow tables created in the source Teradata database by the adapter. The default value is SHADOW_ Stored Procedure Suffix The prefix of the names of the Teradata adapter stored procedures created in the source Teradata database by the adapter. The default value is PROC Trigger Suffix The suffix of the names of the Teradata adapter triggers created in the source Teradata database by the adapter. The default value is TRIG Connection Pool Size Maximum number of connections allowed in the connection pool on a secondary node. The default value is 8. 2016 SAP SE or an SAP affiliate company. All rights reserved. 263

Category Option Description Minimum Scan Interval in Seconds The minimum interval in seconds that the adapter scans the Trigger Queue table to get change data. The default minimum scan interval is 3 seconds to avoid putting excessive load on the database with frequent repeat scans. Maximum Scan Interval in Seconds The maximum interval in seconds that the adapter scans the Trigger Queue table to get change data. The default value is 10 (seconds). If the adapter scans the queue and finds that the queue is empty, it will gradually increase the scan interval from the minimum scan interval to the maximum scan interval. Scan Retry Max Attempts When querying the trigger queue table, the scanner may encounter a deadlock exception. Use this option to set the maximum number of retries before failing (if the retries do not succeed). The default value is 0, which means the adapter will not retry any scans when encountering deadlock exceptions. Credentials Scan Retry Wait Time in Seconds Credentials Mode The number of seconds for the scanner to wait before trying again to query the trigger queue table. This applies only when you encounter a deadlock exception. The default value is 30 seconds. technical user or secondary user Select one of the choices, depending on the purpose of the remote source you want to create. Username Database user name (or LDAP username/kerberos principal name when using logon mechanisms other than TD2) Password The appropriate corresponding password Related Information Teradata DDL Propagation Scan Interval [page 266] Using Prefix and Suffix Options to Manage System Object Name Lengths [page 264] 8.20.1.1 Using Prefix and Suffix Options to Manage System Object Name Lengths The Teradata adapter needs to create a number of system objects on the source database, in order for it to manage realtime replication. These objects include shadow tables, triggers and stored procedures. If your Teradata database has a 30-character name limit, the default remote source settings can lead to Teradata 264 2016 SAP SE or an SAP affiliate company. All rights reserved.

adapter system objects with names greater than 30 characters. By default, the Teradata adapter s system object prefixes and suffixes will add up to 12 extra characters, meaning that only tables with names of up to 18 characters are fully supported in this scenario. To maximize the number of table name characters supported, edit the 4 system object prefix and suffix properties to 1 character each (they cannot be empty). This will ensure that the Teradata adapter uses at most 5 additional characters when creating its system objects, meaning that table names of up to 25 characters can be supported when the 30-character database limit is in place. The following options are available to configure: System Object Prefix Shadow Table Prefix Stored Procedure Prefix Trigger Suffix Note When upgrading, if the Teradata adapter tries to read those properties and they re not present (i.e. not part of the previous remote source before the upgrade), then the adapter will use the default values. When the user edits the remote source after the upgrade, they ll see those default values in the remote source description. Related Information Teradata Remote Source Configuration [page 262] 8.20.2 Teradata Create Remote Source SQL Syntax CREATE REMOTE SOURCE "myteradatars" ADAPTER "TeradataAdapter" AT LOCATION AGENT "MyAgent" CONFIGURATION '<?xml version="1.0" encoding="utf-8" standalone="yes"?> <ConnectionProperties name="configuration"> <PropertyGroup name="connectioninfo"> <PropertyEntry name="host">myhost</propertyentry> <PropertyEntry name="port">1025</propertyentry> <PropertyEntry name="encoding">utf8</propertyentry> <PropertyEntry name="encrypt">on</propertyentry> <PropertyEntry name="logmech">td2</propertyentry> <PropertyEntry name="logdata"></propertyentry> <PropertyEntry name="fast_export">off</propertyentry> <PropertyEntry name="additional"></propertyentry> </PropertyGroup> <PropertyGroup name="cdc_properties"> <PropertyEntry name="database">mydatabase</propertyentry> <PropertyEntry name="prefix">tadp_</propertyentry> <PropertyEntry name="shadow_table_prefix">shadow_</propertyentry> <PropertyEntry name="proc_suffix">proc</propertyentry> <PropertyEntry name="trig_suffix">trig</propertyentry> <PropertyEntry name="conn_pool_size">10</propertyentry> <PropertyEntry name="min_scan_interval">3</propertyentry> <PropertyEntry name="max_scan_interval">10</propertyentry> <PropertyEntry name="ddl_scan_interval">2</propertyentry> <PropertyEntry name="scan_retry_max_attempts">3</propertyentry> <PropertyEntry name="scan_retry_wait_time">10</propertyentry> 2016 SAP SE or an SAP affiliate company. All rights reserved. 265

</PropertyGroup> </ConnectionProperties>' WITH CREDENTIAL TYPE 'PASSWORD' USING '<CredentialEntry name="credential"> <user>myuser</user> <password>mypassword</password> </CredentialEntry>'; 8.20.3 Teradata DDL Propagation Scan Interval The DDL Scan Interval in Minutes Adapter Preference option is important to review when setting up DDL propagation. Enabling DDL propagation can impact the performance of the source Teradata database. Setting an appropriate value for the remote source option DDL Scan Interval in Minutes matters. From the time the DDL changes occurs on the source database to the time the DDL changes are propagated to the target Teradata database, no DML changes on the tables are allowed. At configured intervals (DDL Scan Interval in Minutes. By default, 10 minutes), the Teradata adapter queries the metadata of all subscribed tables from the source Teradata database, and it determines if changes to the DDL have occurred. If changes are detected, it will propagate the DDL changes to the target database through the Data Provisioning Server. Because the Teradata adapter detects DDL changes by querying source Teradata system tables, the source database might be burdened if you configure a small value for the DDL Scan Interval in Minutes option. However, configuring a large value would increase the latency of DDL propagation. Therefore, you need to figure out what value works best for you. If changes to the DDL are rare, you might even want to disable DDL propagation by setting the value of the DDL Scan Interval in Minutes option to zero. This will prevent the Teradata adapter from querying metadata from the source database periodically. Related Information Teradata Remote Source Configuration [page 262] 8.20.4 Teradata Adapter Preferences Options for controlling the Teradata adapter. Parameter Thread Pool Size Description The size of the Teradata global thread pool. The thread pool is shared by the Teradata adapter remote sources. The thread pool is used to execute jobs of querying shadow tables to get change data. 266 2016 SAP SE or an SAP affiliate company. All rights reserved.

8.20.5 Teradata to SAP HANA Data Type Mapping The following table shows the conversion between Teradata data types and SAP HANA data types. Teradata Data Type BLOB BYTE VARBYTE BYTEINT SMALLINT INTEGER/INT BIGINT DECIMAL/DEC/NUMERIC(p,s) FLOAT/REAL/DOUBLE NUMBER DATE TIME TIMESTAMP TIME w/timezone TIMESTAMP w/timezone INTERVAL YEAR INTERVAL YEAR TO MONTH INTERVAL DAY INTERVAL DAY TO HOUR INTERVAL DAY TO MINUTE INTERVAL DAY TO SECOND INTERVAL HOUR INTERVAL HOUR TO MINUTE INTERVAL HOUR TO SECOND INTERVAL MINUTE INTERVAL MINUTE TO SECOND INTERVAL SECOND PERIOD(DATE) PERIOD(TIME) PERIOD(TIME w/timezone) PERIOD(TIMESTAMP) PERIOD(TIMESTAMP w/timezone SAP HANA Data Type BLOB VARBINARY VARBINARY SMALLINT SMALLINT INETEGER BIGINT DECIMAL DOUBLE DECIMAL DATE TIMESTAMP TIMESTAMP TIMESTAMP (TIMEZONE is ignored) TIMSTAMP (TIMEZONE is ignored) SMALLINT 2x SMALLINT SMALLINT 2x SMALLINT 3x SMALLINT 3x SMALLINT, DOUBLE SMALLINT 2x SMALLINT 2x SMALLINT, DOUBLE SMALLINT SMALLINT, DOUBLE DOUBLE 2 DATE columns 2 TIME columns 2 TIMESTAMP columns (TIMEZONE is ignored) 2 TIME columns 2 TIMESTAMP columns (TIMEZONE is ignored) 2016 SAP SE or an SAP affiliate company. All rights reserved. 267

Teradata Data Type CHARACTER/CHAR(n) ASCII CHARACTER/CHAR(n) - Unicode SAP HANA Data Type N(n) N(n) (n) ASCII (n)/clob if n > 5000 (n) Unicode N(n)/NCLOB if n> 5000 CLOB ASCII CLOB Unicode JSON XML CLOB NCLOB CLOB/NCLOB NCLOB Interval Types Intervals are split up into multiple columns, one for each filed of an interval, e.g. day, hour, minute etc. However, since value restrictions apply (e.g. only 11 months, 12 months are carried over to 1 year) special value checks will have to be implemented. If a number of months were added to a month column of an interval year to month in HANA, the insert operation or the update operation to Teradata will have to be intercepted (required for loading to Teradata). INTERVAL YEAR (1-4) -'9' or '9999' SMALLINT with value checks INTERVAL YEAR TO MONTH -'999-11' 2x SMALLINT with value checks INTERVAL MONTH '9999' SMALLINT with value checks INTERVAL DAY '9999' SMALLINT with value checks INTERVAL DAY TO HOUR -'9999 23' 2x SMALLINT with value checks INTERVAL DAY TO MINUTE -'9999 23:59' 3x SMALLINT with value checks INTERVAL DAY TO SECOND -'9999 23:59:59' or -'9999 23:59:59.999999' 3x SMALLINT + 1 DOUBLE with value checks INTERVAL HOUR -'9999' SMALLINT with value checks INTERVAL HOUR TO MINUTE -'9999:59' 2x SMALLINT with value checks INTERVAL HOUR TO SECOND -'9999:59:59.999999' 2x SMALLINT + 1 DOUBLE with value checks INTERVAL MINUTE -'9999' SMALLINT with value checks INTERVAL MINUTE TO SECOND -'9999:59.999999' SMALLINT + 1 DOUBLE with value checks 268 2016 SAP SE or an SAP affiliate company. All rights reserved.

INTERVAL SECOND -'9999.999999' DOUBLE with value checks Related Information Data Types and Writing to Teradata Virtual Tables [page 269] 8.20.6 Data Types and Writing to Teradata Virtual Tables When writing to a virtual table, keep in mind that there are special considerations related to data types, depending on the operation you use. INSERT Inserting values into a Teradata backed virtual table comes with some restrictions: Column data type Period Notes Period columns are split into two columns in SAP HANA (begin and ending bound of the time period). Because the adapter will reconstruct a Teradata period under the hood from these values, Teradata restrictions on periods apply for these columns in SAP HANA, as well. Otherwise, the insert will fail. For example: The ending bound value has to be greater than the beginning value. Null values in the ending bound are not allowed for time periods. Interval All columns have to be "mentioned" in the insert statement and assigned values. If the resulting interval should be negative, all values of the interval have to be negative; no mixing of signs is allowed. This mirrors the behavior of selecting values from negative periods; think additive behaviour of the values. If the resulting interval should be null, all values have to be null. Some values have restricted ranges. For example, the minute column (in SAPA HANA) of a minute to second interval (in Teradata) can only have values in the range of -59 <= x <= 59, even though the minute part may be represented by a smallint column in SAP HANA. See the Teradata documentations for allowed values. 2016 SAP SE or an SAP affiliate company. All rights reserved. 269

UPDATE When updating interval columns, all columns and values have to be specified (see below), because Teradata does not allow partial updates of intervals. Sample Code update "SYSTEM"."local_a_target_interval" set "i_day_minute_day"=5, "i_day_minute_hour"=6, "i_day_minute_minute"=8; The same value restrictions as for insert apply: To make the overall interval null, all interval values have to be set to null To make the interval negative, all values have to be negative Related Information Teradata to SAP HANA Data Type Mapping [page 267] 8.21 Twitter The Twitter adapter provides access to Twitter data via the Data Provisioning Agent. Twitter is a social media website which hosts millions of tweets every day. The Twitter platform provides access to this corpus of data. Twitter has exposed all its data via RESTful API so that it can be consumed with any HTTP client. Twitter APIs allow you to consume tweets in different ways, from getting tweets from a specific user, to performing a public search, or subscribing to real-time feeds for specific users or the entire Twitter community. The Twitter adapter supports the following functionality: Adapter Functionality This adapter supports the following functionality: Virtual table or function as a source Realtime change data capture (flowgraph only) In addition, this adapter supports the following capabilities: 270 2016 SAP SE or an SAP affiliate company. All rights reserved.

Table 61: Global Settings Functionality SELECT from a virtual table INSERT into a virtual table Execute UPDATE statements on a virtual table Execute DELETE statements on a virtual table Different capabilities per table Different capabilities per table column Realtime Supported? Yes No No No Yes Yes Yes Table 62: Select Options Functionality Select individual columns Add a WHERE clause JOIN multiple tables Aggregate data via a GROUP BY clause Support a DISTINCT clause in the select Support a TOP or LIMIT clause in the select ORDER BY GROUP BY Supported? No Yes No No No No No No Twitter Adapter The Twitter adapter is a streaming data provisioning adapter written in Java, and utilizes the Adapter SDK to provide access to Twitter data via SAP HANA SQL (with or without Data Provisioning parameters) or via virtual functions. Using the Adapter SDK and the Twitter4j library, the Twitter adapter consumes the tweets from Twitter and converts to AdapterRow objects to send to SAP HANA server. The tweet is exposed to SAP HANA server via virtual tables. Each Status table is basically a map of JSON data returned from Twitter to tabular form. Currently we expose the following columns in all Status tables. Column name SQL Data Type Dimension Id BIGINT 2016 SAP SE or an SAP affiliate company. All rights reserved. 271

Column name SQL Data Type Dimension ScreenName N 256 Tweet N 256 Source N 256 Truncated InReplyToStatusId InReplyToUserId TINYINT BIGINT BIGINT InReplyToScreenName N 256 Favorited Retweeted FavoriteCount Retweet RetweetCount RetweedByMe PossiblySensitive TINYINT TINYINT INTEGER TINYINT INTEGER TINYINT TINYINT isolanguagecode N 256 CreatedAt Latitude Longitude DATE DOUBLE DOUBLE Country N 256 Place_name N 256 Place_type N 256 UserId BIGINT UserName N 256 UserUrl N 256 CurrentUserRetweetId BIGINT 8.21.1 Twitter Terminology Understanding Twitter terminology helps you get the most out of Twitter data. The following is a brief list of relevant Twitter terms: Term Home timeline Definition The home timeline is the tweets that appear in your home page when you log in. It returns a collection of the most recent tweets and retweets posted by the authenticating user 272 2016 SAP SE or an SAP affiliate company. All rights reserved.

Term Definition and the users they follow. It will contain tweets that you follow, or those that you have tweeted, retweeted, replied to, favorited, mentioned, and so on. User timeline User timeline returns a collection of the most recent Tweets posted by the user indicated by the screen_name. The timeline returned is the equivalent of the one seen when you view a user s profile on twitter.com. When you specify a different user (other than authenticating user), user timeline returns the tweets that are posted by some other user. This is similar to that user s profile on Twitter.com. Each user has a user name or in Twitter it is known as ScreenName. This is the same name that appears after @, used to refer/mention a user. Search tweets User stream Public stream Search tweets (formerly knows as Public Timeline) allows you to query for all tweets posted by all the users of Twitter. It returns a collection of relevant tweets matching a specified query. User streams provide a way for a single user to be streamed the equivalent of their home timeline (the tweets authored by the users they follow) and mentions timeline (the tweets authored by users @mentioning that user). This stream returns public statuses (public data flowing through Twitter) that match one or more filter predicates. For more information, see the Twitter documentation. 8.21.2 Installation and Deployment Though the Twitter adapter is installed with the Data Provisioning Agent, you must configure your Agent to communicate with the SAP HANA server. In addition, you must configure your SAP HANA server and create a remote source. 2016 SAP SE or an SAP affiliate company. All rights reserved. 273

8.21.2.1 Set Up a Twitter Account Follow these steps to set up a Twitter account. Procedure 1. Create a Twitter developer account at https://dev.twitter.com/user/login?destination=home 2. Create an application in Twitter. 3. Obtain the following: API Key API Secret Access Token Access Token Secret See https://dev.twitter.com/docs/auth/oauth/faq to get more insight on how to setup authentication. 8.21.2.2 Create a Twitter Remote Source Follow these steps to create a Twitter remote source. Procedure 1. In the Systems view, open Provisioning Remote Sources 2. Right click Remote Sources, and select New Remote Source. 3. Enter the required information, including the following: API Key API Secret Access Token Access Token Secret 4. Click the Save the Editor icon in the upper right-hand corner of the screen. Results The following directory structure is created, allowing you to create virtual tables or virtual functions as needed. 274 2016 SAP SE or an SAP affiliate company. All rights reserved.

Related Information Adapter Remote Source User Credentials [page 119] 8.21.2.3 Twitter Adapter Preferences Configuration parameters for the Twitter adapter. Parameter Description Default value Validate Server Certificate Specifies whether a valid certificate is on file. true 8.21.2.4 Twitter Adapter Capabilities The Twitter adapter supports the following capabilities. NAME CAP_NON_TRANSACTIONAL_CDC CAP_WHERE CAP_LIKE CAP_SIMPLE_EXPR_IN_WHERE CAP_OR 2016 SAP SE or an SAP affiliate company. All rights reserved. 275

NAME CAP_SELECT CAP_BIGINT_BIND CAP_TABLE_CAP CAP_COLUMN_CAP CAP_METADATA_ATTRIBUTE See the description of these capabilities in the Javadoc documentation, which can be found in <DPAgent_root>/doc/javadoc. 8.21.2.5 Creating Twitter Virtual Tables and Functions Depending on your needs, you will need to create either a virtual table or virtual function to access Twitter data. For batch data queries, you'll need to create a virtual function. For realtime data queries, you'll need to create a virtual table. You will then need to connect to the virtual table or function in the Data Source node in a flowgraph. In the following examples, we provide you with SQL commands for creating these virtual tables and functions. You can, however, also create them using the Web-based Development Workbench UI. For information about creating virtual tables and functions using the UI, see the Configuration Guide for SAP HANA Smart Data Integration and SAP HANA Smart Data. 8.21.3 Batch Twitter Queries Use virtual functions to perform batch Twitter queries. After you create a Twitter remote source, you can create a virtual function to retrieve data from the following Twitter components: Home Timeline User Timeline Search: Tweets Rate Limit In addition, the Twitter adapter supports additional input parameters supported on the above components in a Data Source node for a flowgraph. 276 2016 SAP SE or an SAP affiliate company. All rights reserved.

8.21.3.1 Statuses: Home Timeline Use a virtual function to collect tweets from the Home Timeline. Home Timeline returns a collection of the most recent Tweets and retweets posted by the authenticating user and the users they follow. The home timeline is equivalent to what user sees when they go to their home page on Twitter.com. The example below assumes that you have created a remote source called rsrc_twitter. Sample Code Creating a virtual function DROP FUNCTION GetHomeTimeline; CREATE VIRTUAL FUNCTION GetHomeTimeline(count INTEGER, since_id BIGINT, max_id BIGINT) RETURNS TABLE (Id BIGINT, ScreenName N(256), Tweet N(256), Source N(256), Truncated TINYINT, InReplyToStatusId BIGINT, InReplyToUserId BIGINT, InReplyToScreenName N(256), Favorited TINYINT, Retweeted TINYINT, FavoriteCount INTEGER, Retweet TINYINT, RetweetCount INTEGER, RetweetedByMe TINYINT, CurrentUserRetweetId BIGINT, PossiblySensitive TINYINT, isolanguagecode N(256), CreatedAt TIMESTAMP, Latitude DOUBLE, Longitude DOUBLE, Country N(256), Place_name N(256), Place_type N(256), UserId BIGINT, UserName N(256), UserUrl N(256)) CONFIGURATION '{" DP_UNIQUE_NAME ":"Statuses_Home_Timeline"}' AT "rsrc_twitter"; Sample Code Querying Twitter select * from GetHomeTimeline(10,632283530941214721,932283530941214721); select * from GetHomeTimeline(400,null,null); select * from GetHomeTimeline(null,null,null); Supported Data Provisioning Parameters These are additional, Twitter-specific parameters that you can use and define using SQL or the flowgraph editor in the Web-based Development workbench, Data Source node. 2016 SAP SE or an SAP affiliate company. All rights reserved. 277

DP Parameter Twitter Equivalent Description count count Queryingselect * from GetHomeTimeline(10,632283530941214721,93228353 0941214721);Number of tweets to return (up to 800). Default value is 200. This parameter is optional. since_id since_id Returns results with an ID greater than (that is, more recent than) the specified ID. max_id max_id Returns results with an ID less than (that is, older than) or equal to the specified ID. 8.21.3.2 Statuses: User timeline User timeline returns a collection of the most recent Tweets posted by the user indicated by the screen_name. The timeline returned is the equivalent of the one seen when you view a user s profile on Twitter.com. The example below assumes that you have created a remote source called rsrc_twitter. Sample Code Creating a virtual function DROP FUNCTION GetUserTimeline; CREATE VIRTUAL FUNCTION GetUserTimeline(screen_name N(256), count INTEGER, since_id BIGINT, max_id BIGINT) RETURNS TABLE (Id BIGINT, ScreenName N(256), Tweet N(256), Source N(256), Truncated TINYINT, InReplyToStatusId BIGINT, InReplyToUserId BIGINT, InReplyToScreenName N(256), Favorited TINYINT, Retweeted TINYINT, FavoriteCount INTEGER, Retweet TINYINT, RetweetCount INTEGER, RetweetedByMe TINYINT, CurrentUserRetweetId BIGINT, PossiblySensitive TINYINT, isolanguagecode N(256), CreatedAt TIMESTAMP, Latitude DOUBLE, Longitude DOUBLE, Country N(256), Place_name N(256), Place_type N(256), UserId BIGINT, UserName N(256), 278 2016 SAP SE or an SAP affiliate company. All rights reserved.

UserUrl N(256)) CONFIGURATION '{" DP_UNIQUE_NAME ":"Statuses_User_Timeline"}' AT "rsrc_twitter"; Sample Code Querying Twitter select * from GetUserTimeline('SAP',50,null,null); -- screen_name = SAP select * from GetUserTimeline(null,null,null,null); -- defaults to authenticated user Supported Data Provisioning Parameters These are additional, Twitter-specific parameters that you can use and define using SQL or the flowgraph editor in the Web-based Development workbench, Data Source node. DP Parameter Twitter Equivalent Description screen_name screen_name The screen name of the user you want to return results for. Defaults to authenticating user. E.g. SAP. This parameter is optional. count count Number of tweets to return (up to 3200). Default value is 200. This parameter is optional. since_id since_id Returns results with an ID greater than (that is, more recent than) the specified ID. max_id max_id Returns results with an ID less than (that is, older than) or equal to the specified ID. 8.21.3.3 Search: Tweets Search returns a collection of relevant Tweets matching a specified query. The Search API is not meant to be an exhaustive source of Tweets. Not all Tweets will be indexed or made available via the search interface. Also, note that the search index has a 7-day limit i.e. it does not return any results older than one week. The example below assumes that you have created a remote source called rsrc_twitter. 2016 SAP SE or an SAP affiliate company. All rights reserved. 279

Example Sample Code Creating a virtual function. DROP FUNCTION GetSearchTweets; CREATE VIRTUAL FUNCTION GetSearchTweets(query N(512), count INTEGER, since_id BIGINT, max_id BIGINT, geocode N(256), lang ALPHANUM(2), locale ALPHANUM(2), result_type ALPHANUM(64), until DATE) RETURNS TABLE (Id BIGINT, ScreenName N(256), Tweet N(256), Source N(256), Truncated TINYINT, InReplyToStatusId BIGINT, InReplyToUserId BIGINT, InReplyToScreenName N(256), Favorited TINYINT, Retweeted TINYINT, FavoriteCount INTEGER, Retweet TINYINT, RetweetCount INTEGER, RetweetedByMe TINYINT, CurrentUserRetweetId BIGINT, PossiblySensitive TINYINT, isolanguagecode N(256), CreatedAt TIMESTAMP, Latitude DOUBLE, Longitude DOUBLE, Country N(256), Place_name N(256), Place_type N(256), UserId BIGINT, UserName N(256), UserUrl N(256)) CONFIGURATION '{" DP_UNIQUE_NAME ":"Search_Tweets"}' AT "rsrc_twitter" ; Sample Code Querying Twitter select * from GetSearchTweets('SAP HANA', 20,643480345531273216,643484387561066497,'37.781157,-122.398720,1000mi','de','jp ','recent','2015-09-15'); The previous example just shows how to use all parameters and may not return any results. To see results, specify meaningful search parameters. For example: Sample Code select * from GetSearchTweets('SAP',1100,null,null,null,null,null,null,null); select * from GetSearchTweets('@whitehouse', 40,null,null,null,null,null,null,null); select * from GetSearchTweets('#tgif',40,null,null,null,null,null,null,null); 280 2016 SAP SE or an SAP affiliate company. All rights reserved.

select * from GetSearchTweets('movie -scary :)', 40,null,null,null,null,null,null,null); select * from GetSearchTweets('"happy hour"', 40,null,null,null,null,null,null,null); select * from GetSearchTweets('from:SAP',40,null,null,null,null,null,null,null); select * from GetSearchTweets('SAP HANA', 20,643480345531273216,643484387561066497,'37.781157,-122.398720,1000mi',null,nul l,null,null); Supported Data Provisioning Parameters These are additional, Twitter-specific parameters that you can use and define using SQL or the flowgraph editor in the Web-based Development workbench, Data Source node. DP Parameter Twitter Equivalent Description query q A UTF-8, URL-encoded search query of 500 characters maximum, including operators. This parameter is required. count count Number of tweets to return (up to 1500). Default value is 100. This parameter is optional. since_id since_id Returns results with an ID greater than (that is, more recent than) the specified ID. max_id max_id Returns results with an ID less than (that is, older than) or equal to the specified ID. geocode geocode Returns tweets by users located within a given radius of the given latitude/longitude. Specified by latitude,longitude,radius lang lang Restricts tweets to the given language code locale locale Specify the language of the query you are sending (only ja is currently effective) result_type result_type Specifies what type of search results you would prefer to receive (e.g. mixed, recent, popular) until until Returns tweets created before the given date. Date should be formatted as YYYY-MM-DD 2016 SAP SE or an SAP affiliate company. All rights reserved. 281

8.21.3.4 Application: Rate Limit Status The Rate limit status returns the current rate limits for the endpoints supported by the Twitter adapter s batch querying methods. It shows the limit, remaining requests, and the time until the limits will be reset for each method. Note This uses a virtual table to retrieve the rate limit status, not a virtual function like other batch Twitter queries. The example below assumes that you have created a virtual table called vt_rate_limit_status. Sample Code SELECT * FROM "SYSTEM"."vt_Rate_Limit_Status"; 8.21.4 Real Time Twitter Queries Use virtual tables to subscribe to real time changes on Twitter timelines. After you create a Twitter remote source, you can create a virtual table to retrieve realtime data from the following Twitter components: User Stream User Stream (with tracking keywords) Public Stream In addition, the Twitter adapter supports additional streaming parameters supported on Twitter s User and Public streams in a Data Source node for a flowgraph. Also, do not forget to select the Real-time behavior checkbox in the UI to indicate that this is a realtime operation. 8.21.4.1 User Stream User streams provide a way for a single user to be streamed the equivalent of their home timeline (the tweets authored by the users they follow) and mentions timeline (the tweets authored by users mentioning that user). It also offers the ability to be streamed DMs received by the user when permissions are given. In addition, events like other users favoriting or retweeting that user s tweets, users following that user (or that user following other users), and so on can be streamed. Think of it as a me feed everything on Twitter as it relates to me, the account holder. User Streams provide a stream of data and events specific to the authenticated user. No additional parameters are needed when subscribing to an authenticated user stream. 282 2016 SAP SE or an SAP affiliate company. All rights reserved.

Example Create the virtual table: vt_user_stream Create the target table T_TWITTER_USER_STREAM Sample Code Creating the remote subscription create remote subscription "rsubs_user_stream" as (select * from "SYSTEM"."vt_User_Stream") target table "T_TWITTER_USER_STREAM"; 8.21.4.2 User Stream (With Tracking Keywords) User Streams provide a stream of data and events specific to the authenticated user. In addition to providing user stream data, you can specify a comma-separated list of phrases which will be used to determine what Tweets will be delivered on the stream. A phrase may be one or more terms separated by spaces. Also, a phrase will match if all of the terms in the phrase are present in the Tweet, regardless of order and ignoring case. By this model, you can think of commas as logical ORs, while spaces are equivalent to logical ANDs. For example, 'the twitter' is the AND twitter. the,twitter is the OR twitter. Sample Code Creating a remote subscription Create virtual table: vt_user_stream Create target table: T_TWITTER_USER_STREAM create remote subscription "rsubs_user_stream" as (select * from "SYSTEM"."vt_User_Stream" WITH DATAPROVISIONING PARAMETERS ('<PropertyGroup name =" DP_CDC_READER_OPTIONS "> <PropertyGroup name ="User_Stream"> <PropertyEntry name="track">sap</propertyentry> </PropertyGroup> </PropertyGroup>') ) target table "T_TWITTER_USER_STREAM"; Supported Input Parameters for Virtual Function These are additional, Twitter-specific parameters that you can use and define using SQL or the flowgraph editor in the Web-based Development workbench, Data Source node. 2016 SAP SE or an SAP affiliate company. All rights reserved. 283

Input Parameter Twitter Equivalent Description track track Additional keywords to track along with User stream. This parameter is optional. 8.21.4.3 Public Stream Stream public data through Twitter. This stream returns public statuses (public data flowing through Twitter) that match one or more filter predicates. Create a virtual table called vt_public_stream and the target table T_TWITTER_PUBLIC_STREAM, then execute the following (example) to create the remote subscription. Example Sample Code Creating a remote subscription create remote subscription "rsubs_public_stream" as (select * from "SYSTEM"."vt_Public_Stream" WITH DATAPROVISIONING PARAMETERS ('<PropertyGroup name =" DP_CDC_READER_OPTIONS "> <PropertyGroup name ="Public_Stream"> <PropertyEntry name="track">sap</propertyentry> <PropertyEntry name="follow">5988062,3243510104</propertyentry> <PropertyEntry name="language">en,jp</propertyentry> <PropertyEntry name="locations">-122.75,36.8,-121.75,37.8,-74,40,-73,41</propertyentry> </PropertyGroup> </PropertyGroup>') ) target table "T_TWITTER_PUBLIC_STREAM"; Supported Input Parameters for Virtual Function These are additional, Twitter-specific parameters that you can use and define using SQL or the flowgraph editor in the Web-based Development workbench, Data Source node. Note At least one of track, follow or locations is mandatory. The track, follow, and locations fields should be considered to be combined with an OR operator. track=foo&follow=1234 returns tweets matching foo OR created by user 1234. 284 2016 SAP SE or an SAP affiliate company. All rights reserved.

Input Parameter Twitter Equivalent Description track track A comma-separated list of phrases which will be used to determine what Tweets will be delivered on the stream. This parameter is optional. follow follow A comma-separated list of user IDs, indicating the users whose Tweets should be delivered on the stream. This parameter is optional. locations locations A comma-separated list of longitude, latitude pairs specifying a set of bounding boxes to filter Tweets by. This parameter is optional. language language A comma-separated list of language identifiers to return only the tweets written in the specified languages. This parameter is optional. Note Count is not supported as a parameter, because it requires elevated access to use. A typical use of count in streams is to tell how many past tweets (prior to start of current streaming) to include in the stream. 8.21.5 Restrictions and Limitations A brief list of some of the limitations of the Twitter adapter. Twitter API There are many Twitter APIs; some of them are meaningful and can be used by our framework. Each of the Twitter APIs could be converted to a meaningful SQL statement (with or without Data Provisioning parameters) or a virtual function and be consumed. Currently, the adapter covers only few APIs, and it is very limited as to the APIs provided by Twitter. Site streams are not supported by Twitter adapter. Site streams is an expansion of user stream concept for services that want to stream the equivalent of me feed on behalf of many users at one time. Support for flowgraph and reptask Currently, virtual functions are not supported in flowgraphs. Also, you must use flowgraphs for realtime operations. 2016 SAP SE or an SAP affiliate company. All rights reserved. 285

9 Security The SAP HANA smart data integration option adds new database objects and communication channels to the SAP HANA security landscape. Some aspects of the SAP HANA smart data integration option require specific security-related considerations, such as the communication channel between SAP HANA and the Data Provisioning Agent. However, in general, the SAP HANA smart data integration option follows standard SAP HANA security concepts. For complete information, refer to the SAP HANA Security Guide. Related Information SAP HANA Security Guide (HTML) SAP HANA Security Guide (PDF) 9.1 Authentication The identity of database users accessing SAP HANA is verified through a process called authentication. SAP HANA supports several authentication mechanisms, several of which can be used for the integration of SAP HANA into single sign-on environments (SSO). For complete information about authentication and single sign-on within SAP HANA, refer to the SAP HANA Security Guide. Authentication with Remote Source Systems For remote source systems accessed by Data Provisioning adapters, user name and password authentication is supported. That is, users authenticate themselves by entering their user name and password for the remote source. For custom adapters, the developer is free to implement any type of authentication as required. Related Information SAP HANA Security Guide (HTML) SAP HANA Security Guide (PDF) 286 2016 SAP SE or an SAP affiliate company. All rights reserved. Security

9.2 Authorizations The SAP HANA smart data integration option adds new entities that are stored as catalog objects in the SAP HANA database. Catalog objects such as adapters and remote subscriptions follow standard SAP HANA database security concepts. That is, they follow standard processes for metadata management, system views, public views, authorizations, and so on. In addition to the privileges supported by the GRANT statement in the SAP HANA SQL and System Views Reference, the following privileges are relevant to the SAP HANA smart data integration option and its associated catalog objects. System Privileges System Privilege ADAPTER ADMIN AGENT ADMIN Privilege Purpose Controls the execution of the following adapter-related commands: CREATE ADAPTER, DROP ADAPTER and ALTER ADAPTER. Also allows access to ADAPT ERS and ADAPTER_LOCATIONS system views. Controls the execution of the following agent-related commands: CREATE AGENT, DROP AGENT and ALTER AGENT. Also allows access to AGENTS and ADAPTER_LOCATIONS system views. Source Privileges Source Privilege CREATE REMOTE SUBSCRIPTION PROCESS REMOTE SUBSCRIPTION EX CEPTION Privilege Purpose This privilege allows the creation of remote subscriptions executed on this source entry. Remote subscriptions are created in a schema and point to a virtual table or SQL on tables to capture changed data. This privilege allows processing exceptions on this source entry. Exceptions that are relevant for all remote subscriptions are created for a remote source entry. Security 2016 SAP SE or an SAP affiliate company. All rights reserved. 287

Object Privileges Object Privilege Privilege Purpose Command Types AGENT MESSAGING PROCESS REMOTE SUB SCRIPTION EXCEPTION Authorizes the user with which the agent communicates with the data provisioning server using HTTP protocol. For example: GRANT AGENT MESSAGING ON AGENT CloudAgent TO HANA_USER; Authorizes processing exceptions of a remote subscription. DDL DDL Related Information SAP HANA Security Guide (HTML) SAP HANA Security Guide (PDF) SAP HANA SQL and System Views Reference (HTML) SAP HANA SQL and System Views Reference (PDF) 9.2.1 Activating and Executing Task Flowgraphs and Replication Tasks _SYS_REPO requires additional object or schema authorizations to activate and execute objects such as task flowgraphs and replication tasks. To activate and execute these objects, _SYS_REPO requires the following authorizations: For any input sources: SELECT For any output sources: INSERT, UPDATE, and DELETE For any stored procedures or AFL: EXECUTE For any sequences: SELECT For example, the following statement grants all necessary authorizations to _SYS_REPO on a specific schema: GRANT SELECT, INSERT, UPDATE, DELETE, EXECUTE ON SCHEMA "<schema_name>" TO _SYS_REPO WITH GRANT OPTION; 288 2016 SAP SE or an SAP affiliate company. All rights reserved. Security

9.3 Communication Channel Security The security of communication channels between SAP HANA and adapters hosted by the Data Provisioning Agent depends on the SAP HANA deployment. Additional components added to SAP HANA landscape by SAP HANA Smart Data Integration and SAP HANA smart data quality require security considerations in addition to the information described in the SAP HANA Security Guide. SAP HANA on-premise When SAP HANA and the Data Provisioning Agent are both installed on-premise, or locally in relation to each other, communication is performed using TCP/IP and encrypted with SSL. The agent listens on a port and the Data Provisioning Server connects to it. The agent generates a key pair and stores its public certificate in SAP HANA. The Data Provisioning Server then uses this public certificate to perform SSL server authentication when connecting to the agent. Caution Because passwords for remote systems accessed by adapters are sent in plain text over this communication channel, encryption is mandatory. SAP HANA in the cloud When SAP HANA is in the cloud or a firewall exists between SAP HANA and the Data Provisioning Agent, the agent connects to SAP HANA using a proxy XS application. The proxy performs authentication and authorization before passing messages to or from the Data Provisioning Server. The agent can connect using the user name and password scheme supported by SAP HANA XS applications. Related Information SAP HANA Security Guide (HTML) SAP HANA Security Guide (PDF) Overview of the Architecture [page 10] Security 2016 SAP SE or an SAP affiliate company. All rights reserved. 289

9.4 Auditing Activity on SAP HANA Smart Data Integration Objects Auditing provides you with visibility on who did what in the SAP HANA database (or tried to do what) and when. Actions performed on Enterprise Information Management objects can be audited using the standard auditing tools and processes described in the SAP HANA Security Guide. In addition to the audit actions listed in the SAP HANA SQL and System Views Reference, the following audit actions are available: Audit Action Name Group Number Audit Operation CREATE AGENT 17 Registering a Data Provisioning Agent ALTER AGENT 17 Altering a Data Provisioning Agent registration DROP AGENT 17 Dropping a Data Provisioning Agent registration CREATE ADAPTER 17 Registering a Data Provisioning Adapter ALTER ADAPTER 17 Altering the registration of a Data Provisioning Adapter DROP ADAPTER 17 Dropping the registration of a Data Provisioning Adapter CREATE REMOTE SUBSCRIPTION 17 Creating a subscription to a remote source ALTER REMOTE SUBSCRIPTION 17 Altering a subscription to a remote source DROP REMOTE SUBSCRIPTION 17 Dropping a subscription to a remote source PROCESS REMOTE SUBSCRIPTION EX CEPTION 17 Processing exceptions raised by a subscribed remote source Related Information SAP HANA Security Guide (HTML) SAP HANA Security Guide (PDF) SAP HANA SQL and System Views Reference (HTML) SAP HANA SQL and System Views Reference (PDF) 290 2016 SAP SE or an SAP affiliate company. All rights reserved. Security

Important Disclaimers and Legal Information Coding Samples Any software coding and/or code lines / strings ("Code") included in this documentation are only examples and are not intended to be used in a productive system environment. The Code is only intended to better explain and visualize the syntax and phrasing rules of certain coding. SAP does not warrant the correctness and completeness of the Code given herein, and SAP shall not be liable for errors or damages caused by the usage of the Code, unless damages were caused by SAP intentionally or by SAP's gross negligence. Accessibility The information contained in the SAP documentation represents SAP's current view of accessibility criteria as of the date of publication; it is in no way intended to be a binding guideline on how to ensure accessibility of software products. SAP in particular disclaims any liability in relation to this document. This disclaimer, however, does not apply in cases of wilful misconduct or gross negligence of SAP. Furthermore, this document does not result in any direct or indirect contractual obligations of SAP. Gender-Neutral Language As far as possible, SAP documentation is gender neutral. Depending on the context, the reader is addressed directly with "you", or a gender-neutral noun (such as "sales person" or "working days") is used. If when referring to members of both sexes, however, the third-person singular cannot be avoided or a gender-neutral noun does not exist, SAP reserves the right to use the masculine form of the noun and pronoun. This is to ensure that the documentation remains comprehensible. Internet Hyperlinks The SAP documentation may contain hyperlinks to the Internet. These hyperlinks are intended to serve as a hint about where to find related information. SAP does not warrant the availability and correctness of this related information or the ability of this information to serve a particular purpose. SAP shall not be liable for any damages caused by the use of related information unless damages have been caused by SAP's gross negligence or willful misconduct. All links are categorized for transparency (see: http://help.sap.com/disclaimer). Important Disclaimers and Legal Information 2016 SAP SE or an SAP affiliate company. All rights reserved. 291

go.sap.com/registration/ contact.html 2016 SAP SE or an SAP affiliate company. All rights reserved. No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an SAP affiliate company. The information contained herein may be changed without prior notice. Some software products marketed by SAP SE and its distributors contain proprietary software components of other software vendors. National product specifications may vary. These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP or its affiliated companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP or SAP affiliate company products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty. SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate company) in Germany and other countries. All other product and service names mentioned are the trademarks of their respective companies. Please see http://www.sap.com/corporate-en/legal/copyright/ index.epx for additional trademark information and notices.