Best Practices for SAPBO BI 4.0 Adaptive Processing Servers

Transcription

1 Adaptive Processing Servers Applies to: SAP BusinessObjects BI 4.0, Summary: This whitepaper summarizes best practices, tuning, configuration and troubleshooting to help make your SAP BusinessObjects Adaptive Processing Server perform best. There are plenty of additional whitepapers / notes which delve into significant detail this whitepaper is meant to be a handy summary for an administrator familiar with SAP BusinessObjects Enterprise BI4.0 platform. Document Owner: Jan De Vilder Senior Certified BI / MoBI Consultant SAP Belgium Luxembourg SAP Levallois - France [email protected] Company: SAP Last update: November SAP AG 1

2 Contents 1 Document update history Background Architecture overview Adaptive Processing Server Services Best practices for SAP Business Objects BI4.0 server configuration Introduction Adding multiple APS servers How to create a new APS server in the CMC Cloning APS servers Starting point after installing BI CLONE the original Adaptive Processing Server (APS) Add a dedicated APS for DSL BRIDGE Service Add a dedicated APS for SEARCH Index Add a dedicated APS for ANALYSIS (Multi Dimensional Analysis Service - Analysis for OLAP) Add dedicated APS for ALL OTHER NECESSARY Services Other APS servers to consider Separate APS for DSLBridge Service DSL component in Crystal Reports for Enterprise and Dashboards Isolating DSLBRIDGE: Memory options for DSLBRIDGE: Separate APS for Search Service Separate APS for Analysis Separate APS for Life Cycle Manager (LCM) Configure the APS to handle concurrent publications Separate APS for Heavy Visualisations (Charting) Other APS to consider Separate APS for unnecessary Services Keep the original APS General APS recommendations Command Line Parameters Overview of available JVM X parameters Configuring, monitoring and tracing APS Analyzing APS memory consumption by Heap Dump generation Heap Dump generation: Using MAT to Analyze SAP ME Heap Dumps: Stop unnecessary servers Multi-instances deployment Configuring BICS Result Chunk Sizes (BICS) Adapting the parameters for LOV chunk size For Interactive Analysis Desktop (i.e. WebI Rich Client): For WebIntelligenceProcessingServer: For DSL-BICS layer: How to monitor DSLBRIDGE memory usage SAP AG 2

3 7.7 Getting Traces for investigation SAP Business Objects BI4.0 platform settings CMS settings Web application server (Tomcat) Avoid background activities Deployment Examples Deployment example Deployment example Additional Information Abbreviations SAP Notes Pattern Books BI4 Sizing Companion guide BI4 Sizing Estimator Copyright SAP AG 3

4 1 Document update history Version Modification By(*) Date Comment 0.2 Jan De Vilder June 12 th, 2012 DRAFT first version proposed for review. 0.3 Jan De Vilder June 13 th, 2012 DRAFT second version proposed for review. 0.4 Jan De Vilder June 20 th, 2012 DRAFT third version proposed for review. 0.5 Jan De Vilder June 26 th, 2012 DRAFT fourth version proposed for review. 1.0 Jan De Vilder August 7 th, 2012 Pre-Final version 1.1 Jan De Vilder September 6 th, 2012 Final Version to be published 1.2 Jan De Vilder October 11 th, 2012 Additional information on Crystal Reports for Enterprise Dashboards and LCM 1.3 Jan De Vilder October 26 th, 2012 Updated Sizing info (*) Many people, consultants, product specialists, architects, support engineers have contributed to this document while adding, modifying or reviewing content. Without their help, the document would not be in its current shape. Thanks a lot! 2 Background The default installation of SAP BusinessObjects Business Intelligence Platform installs one Adaptive Processing Server (APS) per host system. The Adaptive Processing Server is a generic server that hosts services responsible for processing requests from a variety of sources. This default configuration is not always optimal, as all services run on a single java APS instance, with a maximum java heap size allocation of 1 GB RAM. The best practices described in this document help the BI Administrator to optimally configure the APS in SAP BO BI4.0. This document is based on the server architecture in BI4.0 SP2 patch 2.12 minimum. APS services are updated on regular basis for every new Service Pack, Support Pack, or Fix Pack. The services listed in this document can get outdated, hopefully not causing confusion. This document is particularly focused on isolation of services in a dedicated APS SAP AG 4

5 3 Architecture overview Tiers: SAP BusinessObjects Business Intelligence platform can be thought of as a series of conceptual tiers: Client tier Management tier Storage tier Processing tier Data tier The APS is a part of the Processing tier. By default, the BI4.0 platform contains one Adaptive Processing Server (APS) per host system. The Adaptive Processing Server is a generic server that hosts multiple services responsible for processing various requests SAP AG 5

6 4 Adaptive Processing Server Services The table below lists all services that are part of the APS (in BI4.0). The purpose of this table is to identify what APS-Services are needed for your deployment. Go through the last column Application, where you ll find the necessary applications required in your environment, list the corresponding services. Service Service Category Service Description Application Adaptive Connectivity Service Connectivity Services Provides connectivity services (replaces Connection Server). Web Intelligence, Dashboards (Xcelsius), Crystal Reports, Explorer BEx Web Application Service Analysis Services Provides integration of SAP Business Warehouse (BW) Business Explorer (BEx) web applications with BI launchpad. BEx Client Auditing Proxy Service Core Services Collects auditing events sent from clients and forwards them to the CMS server. Client tools as Interactive Analysis Desktop Custom Data Access Service Connectivity Services Provides dynamic connections to data sources that do not require a Connection Server. This service allows accessing and refreshing reports created using some personal data provider like CSV files. Web Intelligence, Dashboards (Xcelsius) Data Federation Service Data Federation Services Enables multi-source universes by distributing queries across disparate data sources federates data through a single data foundation Web Intelligence, Dashboards (Xcelsius), Crystal Reports for Enterprise, Explorer Document Recovery Service Web Intelligence Services Web Intelligence document autosave and recovery. Web Intelligence DSL Bridge Service Web Intelligence Services Dual Semantic Layer (DSL) session support. Optimizes BICS connectivity for SAP BW. Handles UNX SQL/LOV generation for relational sources. Web Intelligence, BO Explorer Excel Data Access Service Web Intelligence Services Supports Excel files uploaded to SAP BusinessObjects BI platform as data sources. Web Intelligence Web Intelligence Monitoring Service Web Intelligence Services Monitors Web Intelligence Servers. Web Intelligence 2012 SAP AG 6

7 Service Service Category Service Description Application Insight to Action Service Core Services Enables actions to be invoked and provides support for Report to Report Interface (RRI) Crystal reports Dashboards (Xcelsius) Lifecycle Management ClearCase Service LifecycleManagement Services LCM supports either SubVersion or ClearCase to support versioning of objects. This service is to manage the ClearCase support LCM Lifecycle Management Service LifecycleManagement Services Lifecycle Management Core service. LCM Monitoring Service Core Services Provides monitoring functions All Multi-Dimensional Analysis Service Analysis Services Provides access to multidimensional Online Analytical Processing (OLAP) data; converts the raw data into XML, which can be rendered into Excel, PDF, or Analysis (formerly Voyager) crosstabs and charts. Analysis for OLAP, Creation of OLAP connections Platform Search Service Core Services Provides searching functionality for the platform. Search Publishing Post Processing Service Core Services Performs actions on reports after they have completed, such as sending a report to a specific output location. Publications of Web Intelligence or Crystal reportsl Publishing Service Core Services Coordinates with the Publishing Post Processing Service and Destination Job Service to publish reports to a given output location, such as the file system, FTP, , or a user s inbox. Publications of Web Intelligence or Crystal reports Rebean Service Web Intelligence Services SDK used by Web Intelligence and Explorer Web Intelligence Explorer Security Token Service Core Services SAP Single Sign-On support. Offers Single Sign On (SSO) capability to connect to SAP BW data sources. All Translation Service Core Services Translates InfoObjects with input from the Translation Manager client. All 2012 SAP AG 7

8 Service Service Category Service Description Application Visual Difference Service LifecycleManagement Services Determines whether documents are visually identical for doc promotion and Lifecycle Management. Currently users can view the difference between 2 LCMBIAR files or LCM Jobs. LCM Visualization Service Web Intelligence Services Common Visualization Object Model Service, used by Web Intelligence for charts rendering in WebI reports. The CVOM is dedicated to Web Intelligence reporting. It is what creates the charts displayed in a Web Intelligence document. Web Intelligence Crystal for Enterprise Analysis for OLAP 2012 SAP AG 8

9 5 Best practices for SAP Business Objects BI4.0 server configuration 5.1 Introduction SAP Note (How to size the Adaptive Processing Server), explains in examples how each APS container could be configured. This approach will result in improved response times and larger throughput, but at the expense of higher overall memory utilization on the platform due to the increased number of Java Virtual Machines. When summing the minimum Xmx parameters for the various APS servers (MDAS, DSL, LCM ) from above mentioned SAP Note, the total amount of required RAM is 26 GByte. On a SAP Business Objects BI4.0 server with 32 Gbyte of RAM, there would be only 6 Gbyte working memory left. The advantage if the approach described in this SAP Note, is very useful to isolated workflows for easier troubleshooting and the ability to turn-off specific features without requiring downtime. The approach assumes that customers use all features of BI 4.0. If customers are not using Content Search, MDAS, or BW connectivity, then all these unused services could be moved in a single APS, and stop this APS. Therefore, it is important to identify first in the above table what services are required for your deployment. It is not recommended to remove or delete unused services from the system. Services that are not currently in use may later be needed as business processes evolve and new features are required. Since it is important to understand what each sub-service is responsible for and how they interact with one another, stopping unused services is typically a task of SAP consulting. See BI4 Sizing Estimator, to perform sizing estimates. 5.2 Adding multiple APS servers How to create a new APS server in the CMC You can run multiple instances of the same server on the same machine. To add an APS server, follow the steps below: 1. Log in CMC as Administrator 2012 SAP AG 9

10 2. Go to Organize section, choose Servers 3. In this example, create a new Data Access DSL APS container. On the Manage menu, click New > New Server 4. The "Create New Server" dialog box appears. Choose the Service Category 2012 SAP AG 10

11 5. Choose the type of service that you need from the Select Service list, then click Next. To add an additional service to the server, select the service in the Available Additional Services list and click >. 6. After adding the additional services you want, click Next. If your BI platform architecture is composed of multiple nodes, choose the node where you want to add the new server from the Node list. 7. Type a name for the server in the Server Name box. Each server on the system must have a unique name. The default naming convention is <NODE NAME>.<servertype> (a number is appended if there is more than one server of the same type on the same host machine). 8. Click Create. To include a description for the server, type it into the Description box. The new server appears in the list of servers in the Servers area of the CMC, but it is neither started nor enabled. 9. Review the configuration settings of the created server! 10. Use the CMC to start and enable the new server when you want it to begin responding to BI platform requests Cloning APS servers If you want to add a new server instance to your deployment, you can clone an existing server. The cloned server retains the configuration settings of the original server. This can be particularly useful if you are expanding your deployment (e.g. with additional nodes) and want to create new server instances that use all or almost all of the same server configuration settings as an existing server. The use of configuration templates makes sure the configuration of similar servers is kept in sync. There is no particular recommendation or best practice on whether to clone servers or whether to create new servers. When cloning an APS server, it might be necessary to remove unnecessary services afterwards. When creating a new APS, it is necessary to add the required services. The best practice is to keep the original APS for further use and not delete it! If the original APS is no longer needed, you might consider stopping it SAP AG 11

12 5.3 Starting point after installing BI4.0 What might be done immediately after installing BI4.0? After installing BI4.0 with any SP or patch, the steps below should be considered as an initial configuration. These steps are based on cloning. By creating new servers, the same result can be obtained. Make sure all APS-services are always available in an APS, no matter when it s running, disabled or stopped. The core- APS-services are always needed! It is not recommended to remove or delete unused services from the system. Services that are not currently in use may later be needed as business processes evolve and new features are required. Before creating any additional APS servers, it is recommended to analyze what tools will be used in the BI4.0 environment. The final configuration is highly dependent on the individual customer requirements. The steps below will propose a change of Xmx parameter values. There is no need to change the Xms parameters. Keep the default Xms value CLONE the original Adaptive Processing Server (APS) This best practice is added here, after discovering that many customers modified, corrupted or even deleted the original APS, and could not revert back to the original settings. It is better to keep the original APS for further use. Therefore, stop the original APS and disable it. After the BI4.0 installation, for each node, one AdaptiveProcessingServer is found in CMC / Servers / Nodes. CLONE the original AdaptiveProcessingServer. STOP the original APS and put the start-up type to manual. Uncheck the checkbox: Automatically start this server when the SIA starts Add a dedicated APS for DSL BRIDGE Service This APS will optimize BICS connectivity for Webintelligence and will handle UNX SQL/LOV generation for relational sources. This APS will only work optimal when sufficient memory is allocated. This service is not used by Crystal Reports 2011 nor by for Crystal Reports Enterprise. CLONE the original APS Include following Services in this APS: o o o o DSL Bridge Service Web Intelligence Monitoring Service: is a default service of this APS Security Token Service; Trace log Service: default service Remove all other services Change Xmx1g to something between Xmx4g or Xmx8g, depending on your deployment Add a dedicated APS for SEARCH Index CLONE the original APS Include following Services in this APS: o o Platform Search Service Trace log Service: default service Go to the Application area of the CMC, choose the Platform Search Application : o Change Level of indexing to Platform Metadata o Set action for Error Recovery to Rebuild Index o De-select Content type Universes Remove all other services 2012 SAP AG 12

13 If this APS for Search is intensively used, change Xmx1g to Xmx2g or Xmx3g, depending on your deployment To avoid using Search on content in BI Launchpad, stop this APS Add a dedicated APS for ANALYSIS (Multi Dimensional Analysis Service - Analysis for OLAP) If the applications, Analysis for OLAP or BEx Web, are not used, this step can be skipped. This service is also used to create and edit OLAP connections. CLONE the original APS Include following Services in this APS: o o o Multi Dimensional Analysis Service BEx Web Application Service Trace log Service: default service Remove all other services If this MDAS is intensively used, change Xmx1g to something between Xmx4g or Xmx8g, depending on your deployment Add dedicated APS for ALL OTHER NECESSARY Services CLONE the original APS Remove following Services in this APS (since they got a dedicated APS already): o o o o o Multi Dimensional Analysis Service BEx Web Application Service Platform Search Service DSL Bridge Service Remove all other services not necessary for your deployment Change Xmx1g to something between Xmx2g or Xmx4g, depending on your deployment 2012 SAP AG 13

14 6 Other APS servers to consider Before creating any additional APS servers, it is recommended to analyze what tools will be used in the BI4.0 environment. The final configuration is highly dependent on the individual customer requirements. 6.1 Separate APS for DSLBridge Service DSL Bridge Service is required whenever a document based on UNX or BEx query is refreshed. Purpose of this service: SQL Generation through the new UNX for relational sources Retrieve List of Values (LOV) with UNX/BICS BICS Connectivity, fetching data from BW By default, this service is put together inside a common Adaptive Processing Server. This default server is a java based server containing many other services. Its default settings, in particular its java heap size, are usually not well adapted for optimized performance. Therefore, we recommend doing the following actions: DSL component in Crystal Reports for Enterprise and Dashboards SAP Crystal Reports for Enterprise is a newly designed version of the Crystal Reports designer. It provides better support for the SAP BusinessObjects Semantic Layer (UNX), and allows better connectivity to SAP BW. The Crystal Reports for Enterprise (C4E) architecture is very different from Web Intelligence. C4E has an embedded DSL component (developed in Java). Therefore, C4E does not use the DSL Bridge Service in the APS. The C4E engine runs DSL component and the Connection Server locally in the same process as the CrystalReportsProcessingServer. As a consequence, separating the DSL Bridge Service from the APS, and increasing the Java Heap Size (Xmx) will not have any impact or improvement on the use of C4E. For C4E, the Xmx parameter can be increased directly on the CrystalReportsProcessingServer Java Child VM Arguments as explained in SAP Note The behavior for the Dashboards (Xcelsius) Application and the DashboardsProcessingServer is identical as for C4E: an embedded DSL and Connection Server component Xmx parameter to be modified on the DashboardsProcessingServer Java Child VM Arguments 2012 SAP AG 14

15 6.1.2 Isolating DSLBRIDGE: BICS client natively has to handle a large amount of data in through the DSLBRIDGE process. For an extensive usage, it is recommended to have it isolated from the default Adaptive Processing Server: 1. Open CMC, 2. Stop APS, 3. Remove the DSLBridge Service from APS, 4. Create a new Server with DSLBridge service: a. Service Category: Connectivity Services b. Service: Adaptive Connectivity Service c. Selected Service: DSL Bridge Service. 5. Name it (e.g.: Adaptive Processing Server DSLBRIDGE_1 ). The DSL Bridge APS will contain the following services: DSL Bridge Service Web Intelligence Monitoring Service Security Token Service Trace log Service The next step is to set the new DSLBRIDGE with proper parameters to avoid memory starvation (JVM out of memory or excessive GC activity) Memory options for DSLBRIDGE: The minimum recommended heap size is 4GB (-Xmx4g Xms1g). When using complex queries (with many metadata objects) and/or high volume of row set (number of cells requested), it is wise to extend the heap size beyond 4GB. In practice, proceed as below to modify the DSLBRIDGE options: 1. Log into CMC 2. Edit the properties of the DSLBRIDGE server 3. Add the following to the command line field: -Xmx4g The value of heap size (-Xmx4g) should be adapted to your need: between -Xmx4g.and Xmx8g 4. By default, the heapsize (-Xmx and -Xms) are already set in the APS command line field. For convenience, you can copy/paste the whole command line in notepad and make the modification. (cfr. example below). Each APS has its own unique command line. Don t reuse identical command lines for multiple different APS s SAP AG 15

16 6.2 Separate APS for Search Service After creating a dedicated APS for Search, go to the Application area of the CMC; choose the Platform Search Application : 1. Level of indexing change defaults from Full Content to Platform Metadata or Platform and Document MetaData 2. If possible, change the crawling frequency from continuous to scheduled crawling. 3. De-select Content type Universes (cf.: 4. Set action for Error Recovery to Rebuild Index 5. If necessary, set some documents excluded from indexing The APS for SEARCH will contain the following services: Platform Search Service Trace log Service 2012 SAP AG 16

17 6.3 Separate APS for Analysis When using the client tool Analysis for OLAP intensively, add a dedicated APS for MDAS (Multi Dimensional Analysis Service) If this MDAS APS is intensively used, change Xmx1g to something between -Xmx4g and -Xmx8g The APS for ANALYSIS will contain the following services: Multi Dimensional Analysis Service; BEx Web Application Service; 6.4 Separate APS for Life Cycle Manager (LCM) If LCM is intensively used in a multi-platform environment (DEV, TEST, PROD) is recommended to create an Adaptive LCM Processing Server and isolate the following services: LifeCycle Management Service LifeCycle Management Clearcase Service Visual Difference Service Trace log Service This LCM-APS is responsible for handling the overrides functionality in LCM. It creates the overrides database during initial startup and sets the DB_CREATED flag to true in its metadata in CMS System Database. It is possible that sometimes, the LCM overrides database [apache derby] gets corrupted due to issues with using mixed patch levels or connection caching issues or even multiple APS pointing to the same overrides database. In such cases, even if it is not intended to override a connection, LCM may think that the object (Crystal or WebI) connections need to be overridden and result in modified or missing connections which are wrongfully overridden in target system. In these scenarios, it is required to reset the LCM overrides database to clear corruption. As there is no automated method to do this, SAP Support can provide you with the script: ResetLCMDB.zip. The java utility sets the DB_CREATED flag to false. Then, when the LCM-APS is re-started, it will create the DB again. It s recommended stopping the LCM_APS before running the script and starting it again afterwards. When running the utility, make sure the CMS is running. If DB_CREATED flag is set to true then it will not try to create the DB again even if the flag is missing. This can cause a problem if you manually delete the LCMOverride directory. Due to an issue with the overrides database, it is not possible to run multiple LCM-APS servers pointing to the same overrides database at the same time. This is to prevent a Java Null Pointer Exception when using the LCM overrides functionality. Another issue can occur with LCM overrides as described in SAP Note The APS containing the LCM services should have the default name AdaptiveProcessingServer. If that is the case, rename the default APS to AdaptiveProcessingServer_ORIG and rename the LCM-APS to AdaptiveProcessingServer For LCM, minimum use at least BI4.0 SP02 patch 10 or above 2012 SAP AG 17

18 6.5 Configure the APS to handle concurrent publications. Recommendations: The following recommendations can improve the performance of the Adaptive Processing Server dealing with Publications: If several publications are running concurrently, create multiple instances of the Adaptive Processing Server. In general, it is recommended that you have one Adaptive Processing Server instance for every three concurrent publications. Increase the heap size for the Adaptive Processing Server. To do this, click Manage > Properties, and add the following to the command-line parameter: -Xmx1024M. Run the Publishing Service and Publishing Post-Processing Service on different instances of the Adaptive Processing Server. Adaptive Processing Servers for Publications will contain the following services: Publishing Post Processing Service; Publishing Service; Trace log Service Resolving Out of memory errors If the publisher receives the error message java.lang.outofmemoryerror: unable to create new native thread when a high-volume publication is run, it is because the Adaptive Processing Server and Publishing Service have insufficient stack memory to handle the number of Publishing threads that are generated. This error can occur if stack memory is designated as Java heap space. You can set a thread limit for the Adaptive Processing Server. In the "Servers" area of the CMC, select the Adaptive Processing Server and go to Manage > Properties. In the Command Line Parameters field, enter the following command line parameter: -Dcom.businessobjects.publisher.threadpool.size=threadlimitnumber Replace threadlimitnumber with the thread limit number that you want. If the publisher receives the error message java.lang.outofmemoryerror : Java heap space when you run a high-volume publication, it is because the APS has insufficient heap space. In the "Servers" area of the CMC, select the Adaptive Processing Server and go to Manage > Properties. In the Command Line Parameters field, change the number in the command-line parameter - Xmx1024m to a larger number (for example, -Xmx2048m). Note: In some cases, you may have to create more instances of the Adaptive Processing Server to resolve Out of memory errors. 6.6 Separate APS for Heavy Visualisations (Charting) The same out of memory errors can occur when using chart-heavy reports Recommendations: The following recommendations can improve the stability of chart-heavy reports (Webintelligence or Crystal for Enterprise): Create a new instance of the Adaptive Processing Server. Increase the heap size for the Adaptive Processing Server. To do this, click Manage > Properties, and modify the following to the command-line parameter: -Xmx2g. Increase the number of visualization APS if needed. Adaptive Processing Servers for Heavy Charting will contain the following services: Visualization Service 2012 SAP AG 18

19 Trace log Service 6.7 Other APS to consider Dedicated APS for Client Auditing. Useful to ensure Client tools / CMC / BI Launchpad perform properly, while executing Client Auditing (See Auditing section of Administrators Guide: xi4sp4_bip_admin_en.pdf) Dedicated APS for DataFederation Service and Adaptive Connectivity Service. This will also decouple service categories. This way stopping WebI services will not cause other (CR, etc.) services to stop working because connection service is not available. Dedicated APS for Security Token Service. Issues with Single Sign On to SAP BW could be resolved with a dedicated APS for STS. 6.8 Separate APS for unnecessary Services All services, that are not immediately required, can be grouped in a dedicated APS. You might consider stopping this APS. For example when not using BEx Web Application, not Publications, then gather all the services in one single APS and stop this APS. This is to avoid unnecessary memory and CPU consumption. 6.9 Keep the original APS Leave the original APS as it is. Do not remove any services. Stop and disable this APS. Set the start-up parameter to Manual. To do this, click Manage > Properties in the CMC and uncheck the checkbox: Automatically start this server when the SIA starts. Re-use the original APS by cloning General APS recommendations It is better to clone an existing APS instead of creating a brand new one. Cloning keeps the default settings. While after creating a new APS, default settings have to be manually entered. When increasing the heap size, you have to make sure there is enough physical memory on the machine. Otherwise the jvm process could crash or/and memory goes on disk swapping. That would result to application errors and bad performance. Default parameter: -XX:+ExitVMOnOutOfMemoryError will force an automatic restart of the APS when an Out Of Memory error occurs -XX:+UseParallelOldGC use the parallel old generation garbage collector. Certain phases of an old generation collection can be performed in parallel, speeding up an old generation collection. Warning: In other documents / websites / notes, the parameters below might be considered. The use of these options is absolutely NOT recommended by SAP. Avoid the use of these options, as they will cause failure of the APS server (crash): -XX:+UseParNewGC - XX:+UseConcMarkSweepGC - XX:+UseAdaptiveSizePolicy There is no generally applicable "Best Practise for these parameters SAP AG 19

20 6.11 Command Line Parameters When deploying a new APS on a node (either by copying an existing APS using the server clone command or by creating a new one) you must inspect the APS Command Line Parameters: -Djava.awt.headless=true "-Dcom.businessobjects.datafederator.server.home=E:/Program Files (x86)/sap BusinessObjects/SAP BusinessObjects Enterprise XI 4.0/java/pjs/services/DataFederatorService/resources" "-Dbusinessobjects.logs.home=E:/Program Files (x86)/sap BusinessObjects/SAP BusinessObjects Enterprise XI 4.0/logging/" "-XtraceFile=E:/Program Files (x86)/sap BusinessObjects/SAP BusinessObjects Enterprise XI 4.0/logging/[email protected]" -XX:MaxPermSize=256m "-Dbobj.javaserver.home=E:/Program Files (x86)/sap BusinessObjects/SAP BusinessObjects Enterprise XI 4.0/java/pjs/container/" "-Dbobj.enterprise.home=E:/Program Files (x86)/sap BusinessObjects/SAP BusinessObjects Enterprise XI 4.0/" -XX:LogGcMaxFileCount=3 -XX:+PrintGCTimeStamps "-Dbusinessobjects.connectivity.directory=E:/Program Files (x86)/sap BusinessObjects/SAP BusinessObjects Enterprise XI 4.0//dataAccess/connectionServer" -XsapSystem:08 "-XX:ErrorFile=E:/Program Files (x86)/sap BusinessObjects/SAP BusinessObjects Enterprise XI 4.0/logging/AdaptiveProcessingServer_DSL_TOKEN/aps_NTBIT20.AdaptiveProcessingServer_dump_%p.log" -XX:+HeapDumpOnOutOfMemoryError "- XX:HeapDumpPath=E:/Program Files (x86)/sap BusinessObjects/SAP BusinessObjects Enterprise XI 4.0/logging/" -XX:+ExitVMOnOutOfMemoryError "- Djava.io.tmpdir=E:/Program Files (x86)/sap BusinessObjects/SAP BusinessObjects Enterprise XI 4.0/java/pjs/container/temp" "- XX:GCHistoryFilename=E:/Program Files (x86)/sap BusinessObjects/SAP BusinessObjects Enterprise XI 4.0/logging/AdaptiveProcessingServer_DSL_TOKEN/aps_NTBIT20.AdaptiveProcessingServer_gc.prf" "-Dboe.external.dir=E:/Program Files (x86)/sap BusinessObjects/SAP BusinessObjects Enterprise XI 4.0/java/lib//external" Xms32m -XX:+PrintGCDetails -XX:+UseParallelOldGC "- Dboe.bundles.dir=E:/Program Files (x86)/sap BusinessObjects/SAP BusinessObjects Enterprise XI 4.0/java/lib//bundles" -XX:LogGcMaxFileSize=5m -Xjvmx - Xss1M -Djava.net.preferIPv4Stack=false Xmx8g "-Dbo_cluster.config_dir=E:/Program Files (x86)/sap BusinessObjects/SAP BusinessObjects Enterprise XI 4.0/win64_x64/config" "-Xloggc:E:/Program Files (x86)/sap BusinessObjects/SAP BusinessObjects Enterprise XI 4.0/logging/AdaptiveProcessingServer_DSL_TOKEN/aps_NTBIT20.AdaptiveProcessingServer_gc.log" - Djavax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl -server "-Dboe.common.dir=E:/Program Files (x86)/sap BusinessObjects/SAP BusinessObjects Enterprise XI 4.0/java/lib/" "-Djava.library.path=E:/Program Files (x86)/sap BusinessObjects/SAP BusinessObjects Enterprise XI 4.0//win64_x64" -jar "E:/Program Files (x86)/sap BusinessObjects/SAP BusinessObjects Enterprise XI 4.0/java/pjs/container/bin/boeserver.jar" -workdir "E:/Program Files (x86)/sap BusinessObjects/SAP BusinessObjects Enterprise XI 4.0/java/pjs/container/work/NTBIT20.AdaptiveProcessingServer" 1. Make sure that some of the paths are unique (not the same as on another APS on the same server node). Parameters to check on all APS instances: -XtraceFile= " -XX:ErrorFile= " -XX:GCHistoryFilename= " -Xloggc: -workdir For all these parameters, make sure the paths are different. The paths should be different by default from BI4.0 SP02 patch 2.12 forward. 2. The parameter -Dbusinessobjects.connectivity.directory' on the Adaptive Processing Server hosting the DSL Bridge service can be missing. Generally this can happen after applying a patch. (see SAP Note ) This will cause the error message:"the datasource cannot be generated for the specified datasource name"xxx". (WIS 00000) when selecting a universe (UNX) in Webi Resolution: Create a new temporary Adaptive Processing Server (APS) hosting the DSL Bridge Service from CMC. Copy the parameter "-Dbusinessobjects.connectivity.directory" from this temporary APS add it to the original APS command line. e.g.:"-dbusinessobjects.connectivity.directory= E:/Program Files (x86)/sap BusinessObjects/SAP BusinessObjects Enterprise XI 4.0//dataAccess/connectionServer" Restart the APS hosting the DSL Bridge Service Remove the temporary APS 2012 SAP AG 20

21 6.12 Overview of available JVM X parameters -Xmixed -Xint mixed mode execution (default) interpreted mode execution only -Xbootclasspath: <directories and zip/jar files separated by ;> set search path for bootstrap classes and resources -Xbootclasspath/a: <directories and zip/jar files separated by ;> append to end of bootstrap class path -Xbootclasspath/p: <directories and zip/jar files separated by ;> prepend in front of bootstrap class path -Xnoclassgc -Xincgc -Xloggc: disable class garbage collection enable incremental garbage collection <file> log GC status to a file with time stamps -Xbatch -Xms<size> -Xmx<size> -Xss<size> -Xprof -Xfuture -Xrs disable background compilation set initial Java heap size, usually keep the default setting in APS set maximum Java heap size set java thread stack size output cpu profiling data enable strictest checks, anticipating future default reduce use of OS signals by Java/VM -Xcheck:jni -Xshare:off -Xshare:auto -Xshare:on perform additional checks for JNI functions do not attempt to use shared class data use shared class data if possible (default) require using shared class data, otherwise fail. The -X options above are non-standard and subject to change without notice SAP AG 21

22 7 Configuring, monitoring and tracing APS 7.1 Analyzing APS memory consumption by Heap Dump generation In order to find the biggest memory consumer in your APS process, you will need to have a heap dump. A heap dump is a file that contains a snapshot of the memory allocated by the Java Virtual Machine. The steps below are usually executed by or with the assistance of Engineers from SAP Support Heap Dump generation: If the process is reaching OOM (OutOfMemory), it will generate a heap dump file automatically in C:\Program Files (x86)\businessobjects\sap BusinessObjects Enterprise XI 4.0\logging" With the name java_pid[pid].hprof, e.g: java_pid3736.hprof If the process is not reaching OOM, then you can generate manually a heap dump file : Choose in CMC->Servers and find the PID of the culprit server, e.g.: PID Start a command line tool: cd C:\Program Files (x86)\businessobjects\sap BusinessObjects Enterprise XI 4.0\win64_x64\sapjvm\bin jmap -dump:file=c:\dumpdemo.hprof Using MAT to Analyze SAP ME Heap Dumps: Once the heap dump file is generated (hprof extension), it can be analyzed by using the Memory Analysis Tool: HeapDumpgenerationonaspecificWebIntelligenceBICSworkflow 7.2 Stop unnecessary servers Example: when using Interactive Analysis (Webi) over SAP BICS connection, you basically need following BI4.0 processing servers: Central Management Server (CMS) InputFileRepository Server OutFileRepository Server WebIntelligence Processing Server One or more Adaptive Processing Servers with: i. Visualization Service (for charts rendering in WebI reports). ii. DSLBridge Service. iii. Data Federation Service (DF) if Multi-Source Universes are used. iv. MDAS Service to create, edit and modify OLAP connections. By default, The BI4.0 platform starts automatically all servers. In case you don t need all of them, you may switch them off to save memory and CPU resources. For instance, if you don t need Crystal Reports, you can shut down CrystalReport* servers. This is useful in a test environment when you have little system resources to be shared. This is just an example on how customers could potentially save memory and CPU. Please work with local SAP BO BI4.0 consultants for a recommendation on what services can be stopped SAP AG 22

23 7.3 Multi-instances deployment It is possible to set up several WebI Processing Servers and DSLBRIDGE s. There is no constraint on their numbers. Depending on whether the workload is composed of reporting operations, it is recommended to add more or less instances of WebI Processing Server and DSLBRIDGE. If pure reporting actions are dominant (navigation, chart, edition, etc.), create more WebI Processing Servers. Eg: 2 WebI for 1 DSLBRIDGE. If the workflow is largely composed of data refreshing (run query, LOV refresh), create more DSLBRIDGE. Eg: 1 WebI for 2 DSLBRIDGE. In general, when many users are using WebI documents on BEx, it is safer to make many DSLBRIDGE and few WebI. Having multiple DSLBRIDGE s allows better distributing load with heavy queries the response times will be more consistent with less GCs. Moreover, when the administrator observes the existing DSL being permanently occupying CPU time during concurrent Webi report processing, or when the memory pool consumption for that process is repeatedly approaching the -Xmx limit or if failover tolerance is desired on a single (unclustered) host, adding an additional DSL bridge APS is recommended. 7.4 Configuring BICS Result Chunk Sizes (BICS) When DSL Bridge Service fetches data from BW via BICS, it proceeds by iteration of chunks. One can modify the chunk size so that the fetching could work more efficiently. ChunkSize is like the array fetch universe parameter used with SQL relational universes. This will send smaller quick packets, or larger long packets. The DSL Bridge Service can utilize an optimized BICS client interface which reduces the memory footprint of the DSL Bridge Service. This interface has several parameters which can be tuned. Those parameters are: BICSResultChunkSize = X where X = <# of rows of the biggest rowset> / 50 BICSLOVChunkSIZE =Y where Y = <# of items of the biggest LOV> / 20 To configure those parameters: Windows This is the number of row per chunk the DSL Bridge Service fetches for the result set. Having a high value, for instance 30000, will improve the response time when you refresh a report with a large result set. The default value for this parameter is This parameter in the registry: [HKEY_LOCAL_MACHINE\SOFTWARE\SAP BusinessObjects\Suite XI 4.0\default\WebIntelligence\Calculator] and create the following as DWORD (decimal) values: "BICSResultChunkSize"="X" (where X is determined by the above formula) BICSLOVChunkSIZE = Y (where Y is determined by the above formula) 2012 SAP AG 23

24 UNIX You will need to edit the.registry file for BI 4.0 which is located in the following path (the OS in this example is RedHat): INSTALL_BASE_DIR/sap_bobj/data/.bobj/registry/64/software/sapbusinessobjects/ suite xi 4.0/default/webintelligence/calculator Add the two keys to the.registry file with the values as determined by the formulas above: BICSResultChunkSize= X (where X is determined by the above formula) BICSLOVChunkSIZE=Y (where Y is determined by the above formula) Caution: Setting a high value for those parameters might cause out of memory issue. That is especially true when you work in WebI Rich Client (32 bit runtime). 7.5 Adapting the parameters for LOV chunk size The term of LOV (List of Values) concerns: - The values shown in a prompt panel (BEx Variables Characteristics Values), - The values shown in a Member Selector (Hierarchy nodes and leafs). To improve performance with large LOV, three parameters can be modified on in different levels For Interactive Analysis Desktop (i.e. WebI Rich Client): The parameter WebIParamMaxLOVSize is stored in the file \SAP BusinessObjects Enterprise XI 4.0\platform\config\ WebIContainer_ClientDescriptor.xml: This parameter allows WebI Rich Client to limit the number of LOV rows that are retrieved from DSL SAP AG 24

25 7.5.2 For WebIntelligenceProcessingServer: At the BI4.0 server side, the equivalent parameter is Maximum List of Values Size, property of WebIntelligenceProcessingServer in CMC: For DSL-BICS layer: At the BI4.0 server, DSLBRIDGE side, the parameter BICSLOVChunkSize can be changed. It determines the size of packets sent from Java stack to C++ stack in DSL component. This size is the maximum number of members to send in one trip. This parameter in the registry: [HKEY_LOCAL_MACHINE\SOFTWARE\SAP BusinessObjects\Suite XI 4.0\default\WebIntelligence\ Calculator] "BICSLOVChunkSize"="60000" 2012 SAP AG 25

26 Below a picture is shown, describing the architecture. IAD InfoView WebIntelligence IAD WebIntelligence Processing Server Cube DSL C++ Cube DSL C++ Adaptive Processing Server DSL Java DSL Java (DSLBridge service) BICS + JCo BICS + JCo NetWeaver BW NetWeaver BW *IAD = Interactive Analysis Desktop (= new name for Webi Rich Client) It is wise to set those three parameters to small values (<1000) in the following situations: - User is not inclined to browse long list with large LOV. He prefers rather using the search feature (see above section). - Save memory retrieving large list will cause large memory footprint in WebI Rich Client. It is typically true when you run a very large report and WebI Rich Client cannot afford a heapsize over 2GB due to its 32 bits implementation. Reversely, we can increase those three parameters to high values (>50000) in the following situations. That will help DSL and WebI to limit the number of chunk iteration. - For some reason, user needs to load entirely large LOV. - No limit on memory resource (DSLBRIDGE is configured with large heap size) SAP AG 26

27 7.6 How to monitor DSLBRIDGE memory usage A simple way to check if the DSLBRIDGE suffers from memory starvation is to visualize its GC log file. It is enabled by default in C:\Program Files (x86)\sap BusinessObjects\SAP BusinessObjects Enterprise XI 4.0\logging\. The filename is post fixed by _gc.log. (For example: aps_dewdftf1134as.pjs_1_gc.log). For convenience, you can use gcviewer to view this log file: To install GCViewer simply download the latest version from tagtraum.com: To detect heap memory contention, you can observe the frequency of the full GCs. Full GCs Another way is to observe, in the GCViewer statistics view, the throughput. It is the Time percentage the application was NOT busy with GC. A low throughput means your JVM spends hard time in recycling data. If memory contention is detected, you can enlarge the heap size (-Xmx) of your APS containing the DSLBRIDGE Service - see section above, DSLBridge Service (PJS) SAP AG 27

28 7.7 Getting Traces for investigation Traces are a best help for SAP Support to investigate efficiently on application issue. The procedure is below: 1. Log into CMC. 2. Select Servers / Servers List. 3. Open the properties window of APS. 4. In TraceLog section, set the log level to High: 5. Reproduce the issue. 6. Collect the log files in the folder <BI4 install directory>\ SAP BusinessObjects Enterprise XI 4.0\logging\. 7. On Unix machine, you can look for additional RFC trace files and core file: o o Go to <BI4 Dir>/sap_bobj/enterprise_xi40/<System>/. List the last created files: ls -lrt You will see core file whenever a process has crashed. Possible log files are created (*.trc, *.log) SAP AG 28

29 8 SAP Business Objects BI4.0 platform settings 8.1 CMS settings Increasing instances of CMS helps improve responsiveness of BIP actions (login, logout, folder navigation). This is particular true for small queries scenario (documents very fast to refresh). CMS database must be well tuned. With high number of concurrent users, CMS database could be intensively loaded both in read (select) and in update statements (insert, update, delete). The usual best practices are: - Updates database statistics. - Maximize cache effect to minimize IO (buffer hit ratio = 99%). - Speedy disk access (SCSI, raid system ). 8.2 Web application server (Tomcat) The following parameters are to be verified for the Web application Server (Tomcat): - Maximum of possible threads (maxthread parameter in Tomcat). For Tomcat, it can be set to JVM heapsize. For 128 users, a heapsize of 4GB is enough. Recommended options: -Xmx4g. 8.3 Avoid background activities In case your machine has little resources in CPU or memory, you may want to avoid disturbance by background activities which are turned on by default. You can stop Platform Search indexing service (see section on Search): o Open CMC o Go to Applications o Select Platform Search Application o Edit Properties o Stop Indexing o Modify the Crawling frequency: choose Scheduled crawling instead of Continuous crawling. You can also stop BI platform monitoring. o Open CMC o Go to Applications o Uncheck Enable Monitoring Application SAP AG 29

30 9 Deployment Examples Before deploying multiple APS s, it is important to analyze first how the BI platform will be used: which tools, what connectivity, Multi Source universes, publications, Auditing, LCM etc. It is obvious that the more RAM your platform has available, the more flexibility there is in creating dedicated APS s and corresponding Xmx parameters. BI4.0 Hosts should not have less than 32 GByte of RAM for each Node in the Cluster, but again, this is very much dependent on each individual customer situation. A decent sizing exercise executed by qualified people should tell you in more detail the required RAM and CPU. All of the available RAM should not be consumed by the various APS only. Leave sufficient RAM available for other BI4 processes! As many users have different needs and requirements with their deployments of BusinessObjects BI 4.0, some products may not be used and some services may not be needed. The first step: identify the products that you will be using. The table in section 4 Adaptive Processing Server Services can help. The second step: by looking at the following tables determine: which APS servers you will need to create and the max heap size (value for Xmx) for each APS The third step: create the necessary APS s. The fourth step: perform system sizing and tuning. Note that you do not always have to run all APS s on the same machine, but you may choose to run several APS s on one machine, while other on other machines SAP AG 30

31 9.1 Deployment example 1 This is an example of a deployment with the full set of possible APS s. Choose from the table the APS s you might need. The other services can be grouped in for example AdaptiveProcessingServerOTHER, as in Deployment example 2. See BI4 Sizing Estimator, to perform sizing estimates. APS Xmx Included Services AdaptiveProcessingServerDSLBridge Xmx8g "DSL Bridge Service; This APS is imperative in all Webi deployments Web Intelligence Monitoring Service; Security Token Service; AdaptiveProcessingServerPUBLISHING Xmx4g Publishing Post Processing Service; Publishing Service; AdaptiveProcessingServerWEBI Xmx4g "Rebean Service; Web Intelligence Monitoring Service; Document Recovery Service; Custom Data Access Service; Excel Data Access Service; AdaptiveProcessingServerCHARTS Xmx2g "Visualization Service; Trace log Service AdaptiveProcessingServerANALYSIS Xmx8g "Multi Dimensional Analysis Service; BEx Web Application Service; AdaptiveProcessingServerDF Xmx3g Data Federation Service; AdaptiveProcessingServerMONITORING Xmx1g Monitoring Service; AdaptiveProcessingServerCAPS Xmx512m Client Auditing Proxy Service; AdaptiveProcessingServerCONNECTIVITY Xmx1g "Adaptive Connectivity Service; AdaptiveProcessingServerOTHER Xmx1g Security Token Service; Translation Service; Insight to Action Service; AdaptiveProcessingServerSEARCH Xmx3g "Platform Search Service AdaptiveProcessingServerLCM Xmx2g "Lifecycle Management Service; Lifecycle Management ClearCase Service; Visual Difference Service; 2012 SAP AG 31

32 9.2 Deployment example 2 This is an example of a deployment for Web Intelligence documents with BICS connectivity and Analysis Edition for OLAP. Use of UNX Universe is very limited. Focus is on Publications not on Auditing. See BI4 Sizing Estimator, to perform sizing estimates. APS Xmx Included Services AdaptiveProcessingServerDSLBridge Xmx8g "DSL Bridge Service; This APS is imperative in all Webi deployments Web Intelligence Monitoring Service; Security Token Service; AdaptiveProcessingServerPUBLISHING Xmx4g Publishing Post Processing Service; Publishing Service; AdaptiveProcessingServerWEBI Xmx4g "Visualization Service; Rebean Service; Web Intelligence Monitoring Service; Document Recovery Service; AdaptiveProcessingServerANALYSIS Xmx8g "Multi Dimensional Analysis Service; BEx Web Application Service; AdaptiveProcessingServerOTHER Xmx4g "Adaptive Connectivity Service; Custom Data Access Service; Excel Data Access Service; Client Auditing Proxy Service; Monitoring Service; Security Token Service; Translation Service; Insight to Action Service; Data Federation Service; AdaptiveProcessingServerSEARCH Xmx3g "Platform Search Service AdaptiveProcessingServerLCM Xmx2g "Lifecycle Management Service; Lifecycle Management ClearCase Service; Visual Difference Service; 2012 SAP AG 32

33 10 Additional Information 10.1 Abbreviations Sometimes abbreviations are used in this document: RAM: Random Access Memory. Available memory on a server or Host. BI: Business Intelligence BOE: Business Objects Enterprise; usually BOE is referring to the BI-platform, server or host SAP BO BI: SAP Business objects Business Intelligence; new name for BOE BIP:Business Intelligence Platform GB: Giga Byte or 1024 Mega Byte GByte: Giga Byte CMC: Central Management Console LCM: Life Cycle Management CR: Crystal Reports C4E: Crystal Reports for Enterprise 10.2 SAP Notes SAP Note Added memory usage when splitting Adaptive Procesing Server (APS) services SAP Note How to troubleshoot high CPU consumption issues on BI 4.0 SAP Note Adaptive Processing Server not getting started BI4.0 SAP Note Adaptive Procesing Server crashes repeatedly while indexing a system to enable search functionality in BI Launchpad SAP Note BI 4.0 Consulting:- How to size the Adaptive Processing Server SAP Note LCM Override settings are not working in BI4.0 SAP Note Error: The datasource cannot be generated for the specified datasource name"xxx".(wis 00000). when creating Webi report based on a.unx universe from BILaunchpad SAP Note Error while using scan option within in Promotion Management in SAP BI 4.0 SP04 SAP Note How do I set Java settings such as Xmx for the Crystal Reports Processing Server 10.3 Pattern Books Since setup instructions vary for different operating systems, application server vendors, and database vendors, this wiki contains end-to-end instructions for setting up various deployment scenarios in your company's environment. Each deployment scenario is called a pattern. Go to: SAP AG 33

34 10.4 BI4 Sizing Companion guide A new version of the BI4 Sizing Companion Guide is available here: This document provides recommendations and best practices to help with deploying and scaling the various services in SAP BusinessObjects BI 4. Also, included, are information and details about the sizing tests and methodology used to create the various sizing tools and documents in support of BI 4. We recommend consulting this guide if you are: Working on a sizing strategy, especially in conjunction with the BI 4 Sizing Estimator. Be sure to use the most recent version of the Sizing Estimator. Planning to deploy BusinessObjects BI 4 software Optimizing or tuning an existing BusinessObjects BI 4 deployment For all deployments, it is necessary to consult a Sizing Expert in order to validate sizing exercises after initial planning using the Sizing Estimator and this Sizing Guide. There may have been updates made to this document. The most recent version is always available at this location. This link is at the top of the document, so if readers are ever unsure about having the newest version, they can follow the link in the document. The publish date of the document is on the first page BI4 Sizing Estimator The Sizing Estimator has also been updated and is available here: This version of the Sizing Estimator has significantly refined data from the P&R testing team. It was used to predict existing known good deployments and it has shown itself to be quite accurate. This tool must be used in conjunction with the new BI4 Sizing Companion Guide so its estimates can be employed correctly. The Sizing Estimator shows guidance for deployment on a single large dedicated machine. If the target landscape is not such a device, it is very important to follow the guidance of the BI4 Sizing Companion Guide to learn how to scale-out a deployment successfully SAP AG 34

35 11 Copyright Copyright 2012 SAP AG. All rights reserved. No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice. Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors. Microsoft, Windows, Excel, Outlook, and PowerPoint are registered trademarks of Microsoft Corporation. IBM, DB2, DB2 Universal Database, System i, System i5, System p, System p5, System x, System z, System z10, System z9, z10, z9, iseries, pseries, xseries, zseries, eserver, z/vm, z/os, i5/os, S/390, OS/390, OS/400, AS/400, S/390 Parallel Enterprise Server, PowerVM, Power Architecture, POWER6+, POWER6, POWER5+, POWER5, POWER, OpenPower, PowerPC, BatchPipes, BladeCenter, System Storage, GPFS, HACMP, RETAIN, DB2 Connect, RACF, Redbooks, OS/2, Parallel Sysplex, MVS/ESA, AIX, Intelligent Miner, WebSphere, Netfinity, Tivoli and Informix are trademarks or registered trademarks of IBM Corporation. Linux is the registered trademark of Linus Torvalds in the U.S. and other countries. Adobe, the Adobe logo, Acrobat, PostScript, and Reader are either trademarks or registered trademarks of Adobe Systems Incorporated in the United States and/or other countries. Oracle is a registered trademark of Oracle Corporation. UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group. Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and MultiWin are trademarks or registered trademarks of Citrix Systems, Inc. HTML, XML, XHTML and W3C are trademarks or registered trademarks of W3C, World Wide Web Consortium, Massachusetts Institute of Technology. Java is a registered trademark of Sun Microsystems, Inc. JavaScript is a registered trademark of Sun Microsystems, Inc., used under license for technology invented and implemented by Netscape. SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, SAP Business ByDesign, and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and other countries. Business Objects and the Business Objects logo, BusinessObjects, Crystal Reports, Crystal Decisions, Web Intelligence, Xcelsius, and other Business Objects products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Business Objects S.A. in the United States and in other countries. Business Objects is an SAP company. All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary. These materials are subject to change without notice. These materials are provided by SAP AG and its affiliated companies ("SAP Group") for informational purposes only, without representation or warranty of any kind, and SAP Group shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP Group products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty SAP AG 35