Best practices for Web Intelligence report performance

Transcription

1 Best practices for Web Intelligence report performance Matthew Shaw,

2 Best practices for Web Intelligence report performance Performance is key for user adoption Users expect fast response times After a certain time, users give up/cancel the task they were doing A variety of factors influence Web Intelligence performance System configuration Report design: do not build monster reports Semantic layer configuration and query design Workflows: using the cache and scheduling reports

3 Tips for better Web Intelligence performance #1 - A fast running applet #2 - Steer clear of monster reports #3 - Utilize report linking #4 - Report design best practices #5 - Semantic layer best practices #6 - Know the architecture differences #7 - Stick to a schedule #8 - Server sizing for optimal performance

4 #1 Tips for a fast running applet Everyone hates waiting around for an application to load. Some Java updates can wreak havoc on load time of the Web Intelligence applet Three issues that arise a lot are covered in the next few slides: Online Certificate Revocation checks causing delays New JRE security changes that cause issues and delays Applet comprised of over 60 JAR files, causing many security checks

5 Online Certificate Revocation Checks (1/2) Newer JRE versions automatically check for revoked certificates online! This happens for each JAR file (60+ in BI 4.0) Internet connection speed plays a major factor Can add several minutes on to load times

6 Online Certificate Revocation Checks (2/2) Tips Use only CRLs or choose the Do not check option in the Java Control Panel Test On & Off for Timing differences Can use the Java Console to analyze Read this WIKI for more Details See KBA

7 JRE Security Changes Cause Issues Oracle has tightened up security requirements for applets New JRE versions are released much quicker than our patches in some cases List of known issues and resolutions can be found here: Web Intelligence and Oracle JRE Known Issues

8 Reducing the # of JAR files loaded (1/2) BI 4.0 introduced a new architecture for the Web Intelligence applet Previous versions were a single JAR file ThinCadenza.jar BI 4.0 split this into 60+ individual JARs for easier development and updating New JAVA security updates are not friendly to this architecture

9 Reducing the # of JAR files loaded (2/2) Recommendation: Upgrade to BI 4.1 SP03+ This goes back to the older architecture model and contains a single jar for the applet webiapplet.jar is the new name (webiapplet_en.jar is the resource file that accompanies) Advantage of only 1 security check, 1 cache check. Generally improves performance by a lot! NOTE: It is a 44 MB JAR file. Can take a while to load the first time

10 Other JRE Tweaks Visit the WIKI for some more JRE Client Side tweaks: Ensuring your JRE caching is enabled Ensuring the Java next-generation plug-in is used Tests you can run to find out where the bottlenecks are for slower load times Tips for Fine Tuning Performance for the Webi Applet Interface

11 Tip #2 Steer clear of monster reports

12 Large documents can mean wasted time If only 10-20% of a large document is utilized, 80-90% is waste Avoid using a large # of reports within a document 10 reports per document is a reasonable number Do not exceed 20 reports per document Don t try to account for all scenarios but rather focus on specific scenarios that add value to the consumer 50,000 rows per document is a reasonable number Do not exceed 500,000 rows per document

13 Focus on the Business Need Creating smaller documents for specific business needs allows for faster runtime and analysis Start with the individual business need and build a document based on that need

14 Create smaller, reusable documents By creating smaller, reusable documents you: Reduce the time it takes to load the document in the viewer Reduce the refresh time Reduce the system resources needed on both the client and server side Improve the performance while modifying the document

15 Tip #3 Utilize Report Linking

16 Use Report Linking Instead of using HUGE documents, consider using smaller documents and linking them together! Link reports together using OpenDocument linking and a combination of Prompts and Filters Hyperlink wizard available in HTML interface makes this really simple! This removes performance hits for unnecessary parts of the report Higher design-time costs but better performance for the end user

17 Report Linking Use Case Use Case for Report Linking: Report A is a summary report that summarizes the sales for all 100 sites of Company XYZ Report A is scheduled to run each night and take ~20 minutes to complete Users can view the latest instance of Report A which takes only a few seconds to load Users can drill down into Site data for each of the 100 sites which launches a 2 nd report that displays only the site data for the site they drilled on

18 Tip#4 Report Design Best Practices

19 Limit the number of Data Providers used Best Practice from the field is to use no more than 15 Data Providers per document Data providers run serially so run times add up Refresh time and merging of dimensions can cause some big delays on the Processing Server side Using a Data Warehouse to consolidate sources and ETL tools to produce better reporting sources is a better practice

20 Don t accidentally disable your cache! Web Intelligence has great caching mechanisms for documents that have already been viewed Cache improves load time of documents Using functions such as the ones below means that cache will never be used CurrentTime() CurrentDate() UserName() These functions require updates to the cached files so they have to be regenerated

21 Avoid Auto-Fit When Possible! When used, it forces the full document to be calculated during navigation This can make navigating a report much slower Can make things especially slow when jumping to the last page of a large report

22 Avoid Charts with Many Data Points CVOM is the new charting engine we use in BI 4.x This is hosted by the APS server Visualization Service CVOM is better at creating a large # of smaller charts than creating large ones with many data points May be more efficient to use smaller, more specific charts than huge ones

23 Avoid Nested Sections Nested sections can contribute to performance degradation issues This is especially true if conditions are used such as Hide section when the following is empty

24 Test Query Drill for Drill Down Reports Option called Use query drill in the report properties Modifies the underlying query when drilling. Utilizing the database instead of local data Reduces the amount of data stored locally for a drill session. Can improve performance. Best to test both modes to compare

25 Limit use of Scope of Analysis Scope of Analysis is used to retrieve extra data from the database This extra data is stored in the cube for drill down requests This is great feature but has a performance impact Could utilize the Report Linking for similar results and an ondemand data fetch Tip: BI Launchpad preferences for Web Intelligence allow you to be prompted when Drilling needs more data

26 Formula Best Practices for Performance ForEach and ForAll should only be used when really necessary. Use IN when possible instead Where operator can also take longer to process behind the scenes. Using If Then Else may be better Factorizing variables reduces overhead for calculation engine. For example: vs v_sales = [MeasureA] + [MeasureB] v_salesest = [v_sales] + [MeasureC] v_salesest = [MeasureA] + [MeasureB] + [MeasureC]

27 Tip#5 Semantic Layer Best Practices

28 Build lean queries for your documents Start with the document needs and build your queries specific to these Too often we see mega queries/universes that account for every possible need Obviously this is convenient for report designers, but the consumers pay the price Purpose of a semantic layer is to simplify the querying of data TIP: Build queries that only contain objects used in the document(s)

29 Array Fetch Size Optimization (1/2) Array Fetch Size sets the maximum number of rows with each fetch from the database Tuning this size for your environment can greatly impact performance Set at the Universe Connection level INTERNAL TESTING AFS Time Records Throughput / second / second / second Optim / second

30 Array Fetch Size Optimization (2/2) Optimized by default for new Connections DISABLE_ARRAY_FETCH_SIZE_OPTIMIZATION parameter can be used to disable the optimization** If you wish to override the optimized values, you must set the above parameter to Yes in IDT/UDT Information Design Tool Guide (Chapter ) covers this option

31 Query Filters instead of Report Filters Query Filters modify the SQL query to restrict the data fetched and displayed Report Filters only modify the displayed data Example: A report returns a years worth of data. The consumer is only interested in January data Using a Query Filter, the WHERE clause is modified so that only January data is fetched by the query. Using a Report Filter to show only January data simply filters the data that is displayed. The full year s worth of data is still in the underlying cube

32 Ensure Query Stripping is utilized Query Stripping is new to BI 4.x. Strips away unused objects from the query Can now be utilized for relational database as well as BICS based reports (BI4.1 SP3 and up) Has to be turned on at the Report, Query and Universe levels BICS connections use stripping by default

33 More on Query Stripping For Relational DBs, following parameters must be set: Allow Query Stripping option selected in Business Layer Enable Query Stripping option selected in Query Properties of the Webi Document Enable Query Stripping option select in Document Properties of the Webi Document USE_ENHANCED_QUERY_STRIPPING parameter only optimizes the SELECT and GROUP BY clauses but doesn t modify the joins and other clauses

34 Only merge dimensions that are needed Merging dimensions is necessary in many cases This has a performance impact as logic has to be applied to create a merged dataset Unmerge dimensions that are not needed in a document to save some time

35 Tip #6 - Know the Architecture Differences

36 32-bit vs 64-bit Processing Servers (1/2) XI 3.1 and below were fully 32-bit programs BI 4.x introduces many new 64-bit servers Main advantage is increased resource allocation with 64-bit architecture No more 2GB memory limitations for WebI Processing Servers Can utilize 64-bit client drivers Changes some of the sizing recommendations

37 32-bit vs 64-bit Processing Servers (2/2) New 64-bit processes in BI 4.x Web Intelligence Processing Server Adaptive Processing Server Adaptive Job Server Connection Server (64-bit in most cases) Central Management Server (CMS) File Servers Tomcat/Application Servers (64-bit JVM support) A few 32-bit processes Connection Server (32-bit on some Windows) Web Intelligence Rich Client Universe Designer / Information Design Tool

38 Know which servers are involved! Key to identifying a bottleneck is isolating the cause of the slowdown WebI documents can utilize many servers for processing Adaptive Processing Server (DSL Bridge) Adaptive Processing Server (Adaptive Connectivity) Adaptive Processing Server (Visualization Service) Connection Server 32-bit Web Intelligence Processing Server Case and Point BI 4.x Architecture Diagram

39 Learn these Web Intelligence Workflows Interactive Diagrams are available to help you learn about the servers that are involved View a Webi document on-demand link Refresh a doc in two-tier mode link Refresh a doc in three-tier mode link Refresh a doc based on multisource universe link Refresh a doc based on a dimensional universe link Run a scheduled Webi document link

40 Tip #7 Stick to a Schedule

41 Why Use Scheduling? Scheduling allows lower user wait time when implemented correctly Allows you to offset processing to non-peak times Can help distribute load and reduce sizing requirements for concurrent users Reduces impact on database during peak times

42 Tips on Scheduling Web Intelligence Documents Best practice is to schedule any report that takes over 5 minutes to refresh Allow power users to schedule reports On-Demand if needed You can use Report Linking + Scheduled Instances to strike a balance between view time and age of data Use the cache settings to pre-cache XLS/PDF

43 Tip #8 Server sizing for optimal performance

44 Sizing for Performance BI 4.x is not sized for performance out of the box 64-bit processing servers changes the game vs 3.1 Adaptive Processing Server is heavily utilized now Refer to Sizing Companion Guide for details

45 Sizing Things to know for Web Intelligence 64-bit Webi Processing Server means no more memory limitations May no longer need exactly 1 WIPS process per CPU core. Large scale test on 24 core machine had just 5 WIPS. Adaptive Processing Server (APS) is heavily used for some WebI workflows BICS connections utilize DSL Bridge Service UNX Universes use DSL Bridge Service Charting uses Visualization Service MSU uses Data Federation Service Use System Configuration Wizard to help split the APS

46 Sizing Things to know for Web Intelligence Location can make a big difference Ideally place your Processing Tier near your database This is where most of the data transfer will occur Use a fast connection between the WebI server and your database Using local storage for Cache and Temp directories can improve performance Local storage is generally much faster I/O than network storage CPU speed DOES matter! More relevant on older hardware but 1200Mhz processors, even if there are 128 cores, will process documents slower

47 Tips for better Web Intelligence performance #1 - A fast running applet #2 - Steer clear of monster reports #3 - Utilize report linking #4 - Report design best practices #5 - Semantic layer best practices #6 - Know the architecture differences #7 - Stick to a schedule #8 - Server sizing for optimal performance

48 Thank you Contact Matthew Shaw,