Business Intelligence Application Usage Profiling & Management

Transcription

1 WHITE PAPER Business Intelligence Application Usage Profiling & Management Actionable usage metrics, predictive query controls, & automated user guidance to increase BI performance, compliance & cost savings 0

2 Application Usage Management Actionable usage metrics, query controls, & user guidance to increase BI performance, compliance & cost savings Managing the Unique Demands of BI, Data Warehouse, and Reporting Applications Today, large and growing numbers of business people require access to information in order to manage and grow their businesses. This increasing demand for business intelligence (BI) information is creating significant IT challenges: unpredictable and resource-intensive query traffic, more BI and reporting applications and users to support, declining performance, frequent user errors, rapidly escalating database volumes, and continuing pressure to reduce costs. Ensuring that BI, data warehouse, and reporting applications are performing to expectations and delivering value to the business is very challenging, new users come on-line, business questions change, and data rapidly changes. Success in this dynamic and unpredictable environment requires not only continuous BI activity monitoring, but also ongoing BI management. Teleran Application Usage Profiling & Management Teleran, a leader in application usage management software, delivers a patented product suite that provides comprehensive awareness and control of user behavior, application activity, and database usage. Teleran s automated monitoring and user management capabilities enable IT to improve performance and manageability across all BI applications while reducing support and system costs. isight continuously documents who is using what applications to access what data, and how the system is performing. It directs more efficient application/user management, database tuning, query analysis, compliance auditing, system resource management, capacity planning, and conversions. It also guides the application of iguard query policies. iguard protects corporate data while optimizing business use of information with a policybased query management facility. iguard improves IT efficiency by automatically preventing inappropriate, unauthorized or long-running queries before they reach the database and degrade system performance, return incorrect answers, or breach data security. iguard Messaging guides and trains BI application users with real-time messages associated with iguard policies. Messaging works independently of applications, improving user productivity while reducing helpdesk calls and support costs. Teleran s Access Architecture enables the company s software suite to operate transparently and continuously on the network without degrading database or application performance. 1

3 Unlike snapshot system and database performance monitors, Access Architecture requires no resource-consuming in the database agents, traces or monitors. Network-based Transparent & Scalable Information Requesters Application Databases Application usage management involves understanding and managing how users access and interact with applications and data to improve the performance, quality, compliance and TCO of business-critical applications Claudia Imhoff, BI/DW Expert Bridging Gap Between Managing BI Users & Monitoring System Level Resources Teleran s software products fill a gap that system level database and performance monitors do not address: understanding and managing user and application behavior. With Teleran, organizations can now "holistically" manage the entire BI application environment to get the greatest value from their investments in people, information, software and systems. Teleran usage profiling & management fills the gap between managing users & database infrastructure. BI Users Data Level DBMS System Level Servers Storage Tools such as Tivoli & Oracle Enterprise Manager deliver system level performance monitoring. Network 2

4 Essential Application Usage Profiling & Management Functions 1. isight: BI / Data Warehouse Usage Reporting & Analysis Actionable Usage Metrics to Increase Performance, Compliance & Cost Savings isight captures all user and application interactions with the database and logs them to Teleran s InfoUse database, a published set of tables in the relational database of your choice. Usage information collected by isight includes the queries and reports run, data objects accessed, application, IP addresses, database instance, and application, database or OS user IDs. isight also captures statistics relating to the user s experience with the query (time to first row, elapsed time of query, total number of rows, total number of bytes). Identity Persistence Guarantees Visibility of Application Users Tracking and managing usage by actual application user is critical for compliance and security management. Most applications today log into the database with a single generic database user ID. This effectively masks the true identity of the application user to the database and renders database level auditing and access controls ineffective. With Teleran s ID Persistence feature, isight can correlate the BI application or single sign-on user with the complete database transaction. Identity Persistence also enables iguard policies to be applied by application user, user group or role for effective and granular policy enforcement. Logging and Analysis of Usage Data isight automatically creates, configures, and maintains its usage database or Knowledge Base. At installation and then periodically, isight models the system catalogue tables of the database(s) and applications to which it is profiling access. This enables isight (as well as other Teleran products) to maintain a current understanding of users, applications and database objects, and their relationships including tables, columns, views, joins, indexes, stored procedures, and others. To minimize the Knowledge Base size, isight can automatically ignore specified repetitive set-up queries, applications, and maintenance processes that are not useful to capture and can also periodically purge specified data. Web-based Usage Reporting & Analysis Improves IT Productivity isight includes a web-based reporting package, delivering over 80 categorized reports that analyze InfoUse data. In addition, InfoUse is an open and well documented database that is easily accessed by any SQL query tool. isight reports provide out-of-the-box value by answering essential BI management questions: 3

5 Performance What reports and queries are long-running and why? User Management Who are your problem users? Application Tracking What applications are in use? What licenses are not being used? Data Usage What tables, columns or Universes are most used or never used? Service Levels Is the average query time during peak periods meeting the SLA? Database Tuning Are joins, indexes, and aggregates optimized for actual usage? Application Quality Do some queries produce incorrect results? Maintenance When can updates occur with least disruption to users? Compliance Are security/privacy policies being followed? Budgeting Are system costs fairly allocated to groups based on actual usage? "isight is our eyes and ears to understand and optimize how our business users are leveraging our information." - Bob Sickenger, IT project lead, MetLife isight Reporting Sample isight Application Usage Management Dashboard 4

6 Sample Application Usage Tracking Sample User Analysis Sample Query Performance Assessment 5

7 Sample Compliance/Privacy Reporting Sample Resource Utilization by Group Sample Database Model Tuning 6

8 isight Cost-Effectively Delivers Value to Both IT & Business isight s Web-based usage reporting and analysis facility provides essential, easily distributable BI management information for key organizational roles including analysts, application managers, DBAs, systems staff, IT management as well as business unit managers. Here are examples of how isight reports are distributed and used in one of the world s leading financial services firms: BI Application Liaisons / Business Analysts BI application liaisons in each business department receive usage profiling and user error reports to ensure that applications are being used effectively and to identify where user assistance or training may be required. Application / Data Warehouse Managers Application managers receive summarized performance, resource use, and exception reports that guide the allocation and management of system and staff resources as well as assist in budgeting and capacity planning. Compliance Staff Compliance staff use isight to monitor adherence to compliance policies and to address audit and reporting requirements. DBA / IT Staff DBAs and IT system staff use isight reports and analyses to identify and resolve performance and user problems. These reports give DBAs actionable detail to maintain application performance and quality, and to identify when and where to use point-in-time system-level database administration tools. IT Management Managers receive summarized isight service level reports to monitor commitments to the business and compliance audit reports confirming data privacy policies are being followed. Business Unit Managers Business unit managers use isight reports to identify what information their staff values and to confirm that analytical applications are actually being used as intended. Business managers also receive isight chargeback reports that calculate charges based on actual use. 2. iguard: Automated Query & User Controls Prevents Problem Queries Before They Waste Time & Resources No matter how much planing and support goes into BI, data warehouse, and parameterized reporting applications, end-users unknowingly and inevitably issue inappropriate or "runaway" queries that degrade performance, deliver inaccurate results, or breach data privacy policies. 7

9 isight exception reports identify these user/query problems and direct the application of iguard query control policies and Automated Helpdesk messages. iguard operates in conjunction with isight on the Teleran Access Server. It automatically applies selected query control and user management policies via its patented high speed policy engine. iguard identifies queries that violate active policies and prevents those queries before they reach the database. At the same time, Teleran s Automated Helpdesk facility instantaneously sends a customizable message to the user that guides them on ways to improve the efficiency and effectiveness of their query. Minimizes Helpdesk & Support Costs This approach dynamically tunes the user, improving their productivity while minimizing help desk calls and IT support costs. Those queries that do not violate a policy pass through to the database for processing. iguard performs its user management capability independently, but aware of databases and applications, protecting the system and users from a wide range of inappropriate, inaccurate, and resource wasting queries. "As our user community grows, iguard policies have enabled us to guide and support more people at a reduced cost, maximizing the business value of our applications." Joe Pomerantz, First Vice President, Merrill Lynch Improves Query Performance, IT Productivity & System Efficiency iguard protects the database from the common user errors and inefficient query behavior that slow application performance and consume IT staff time with repetitive manual trouble-shooting and problem resolution processes. Using the iguard "point and shoot" Policy Wizard, policies are selected from the iguard policy library and tailored to address specific query issues associated with user groups, applications, and database objects. Here are several iguard policy examples: Do not do a SELECT from the claims table without a REGION code. (System resources are wasted if regional claims analysts query claims history for the entire country.) Do not permit a join between TABLE A and TABLE B without column A. (Queries issued without an indexed column are too resource intensive.) During the day shift, do not permit unqualified access to TABLE A and TABLE B, but permit access during the evening and night shifts. (Maintains peak period performance by deferring long-running queries to off-peak times.) Do not join a fact table to itself. (Stops Cartesian joins or "runaway" queries from wasting system resources and reducing user productivity.) Do not allow a SELECT * from a fact table without a WHERE clause. (Prevents unconstrained queries from consuming significant system resources.) 8

10 Do not allow specified users or user groups access to specified tables, columns and rows. (Enforces data level access rules to adhere to information privacy policies and regulations.) iguard policies can also prevent user errors that generate inaccurate or misleading results. Do not join two fact tables to each other without qualifiers (Prevents incorrect results.) Do not join TABLES A and B (Prevents a "Chasm Trap", a many-to-one-to-many join that results in incorrect answers.) Delivers Control & Flexibility with Test, Advise & Active Policy Modes iguard policies can be placed in Test, Advise or Active modes. In Test or simulation mode policies do not prevent queries, but the system records and reports on what queries would have been acted upon and how these policies can improve performance, accuracy, or security when activated. In Advise mode queries are not prevented, but users receive Automated Helpdesk messages associated with specific policies that will guide them in best query practices or remind them of corporate data privacy policies in place. Active mode allows policies to prevent queries when violated and simultaneously send Automated Helpdesk messages. iguard Messaging - Real-time User Guidance Trains Users, Improves Performance, & Reduces IT Support Costs iguard Messaging addresses common BI and reporting user problems that drive up support demands and reduce productivity. Using real-time customizable text messages, Messaging works in conjunction with iguard and guides users to more effectively and efficiently access and interact with data. Whenever an iguard Policy is triggered by an inappropriate query, an iguard message is instantaneously sent to the user, independent of the application used, suggesting how he or she can adjust or improve their query. Sample iguard Message 9

11 Improves How Users Interact with Applications and Data iguard Messaging has proven to increase the quality of user interaction with information. Users appreciate the instantaneous guidance delivered by Helpdesk messages. It assists them in getting faster more accurate answers, reduces their frustration, and improves their productivity. From IT s perspective, iguard messaging minimizes the time helpdesk and DBA staff spend dealing with common and recurring end-user query problems, directly reducing helpdesk, training, and IT support costs. "Teleran s user communications capability helps business users be more efficient and smarter in using information. - Matt Pohl, Data Warehouse Director, Corporate Express The following are example messages associated with iguard policies. Performance This query will run for more than 3 hours. Please further constrain your query or schedule it to run overnight using the query scheduler. User Guidance Joining these tables will produce a very large result set. Add a where clause and try again. Priority Schedule Only Finance can access P&L tables at quarter-end, 9:00 11:00 a.m. Maintenance Table X is being updated right now. Run your query after 1:00 p.m. Query Accuracy Joining tables A and B will produce incorrect results. Try again using tables A and C. IT Query Alerts A data security policy has been activated two times by the same user. Look at today s security audit report for more details. User Notification The Sales application has new customer information. Call the Help Desk to schedule training. Data Security You are not authorized to access salary data. Teleran System Architecture - Scalable, Flexible Software Solution The Teleran System consists of three server processes: the Teleran Access Proxy Server, the Administrative Server and the Report Server. The System also includes an Administrative Console (GUI) and requires access to a relational database (Teleran Knowledge Base). Teleran s software components are written in Java and operate efficiently on a wide range of distributed (Windows, UNIX, LINUX) and mainframe operating systems. 10

12 Teleran Access Proxy Server The Teleran Access Proxy Server (Tserver) resides on the TCP/IP network logically between the SQL-generating applications and back-end application databases. The Tserver acts as a passthrough access proxy that logs (isight) and controls query and report activity (iguard). As a network-based proxy, the Tserver does not require the installation of any "in-the-database" agents, traces or monitors. It can be deployed on an application or mid-tier server or co-housed on the same physical server as the application database. One Tserver can profile and manage access to multiple database instances on one or more physical servers. The Tserver s isight process asynchronously logs query activity including SQL statements, application name, application or database user ID, IP address, report name, BI metadata objects accessed. It also logs result set metrics including time to first row, elapsed time of the query, database objects accessed, and total number of rows and bytes returned. The Tserver logs this information to the Teleran Knowledge Base, a published set of tables in the relational database of your choice. Teleran Architecture and Processes Teleran Administrative Server The Teleran Administrative Server (Aserver) manages on-going Teleran system administration processes that can be scheduled or launched on a manual or event basis. These periodic processes include: 1) Modeling the system catalog tables of both the application and the application database or data warehouse, 2) Analysis of the logged query and report activity, and The Aserver can be co-housed with the Tserver or installed on another physical server on the network. It requires access to both the Teleran Knowledge Base and the application database. 11

13 1) Modeling At installation and then periodically, the Aserver interrogates the system catalogue tables of both the application and the database or data warehouse. This process maintains a current model in the Teleran Knowledge Base of all database and application objects and their relationships including user IDs, reports, BI metadata, tables, columns, views, joins, indexes, stored procedures, and others. The modeled application and database information is stored in the Teleran Knowledge Base. Modeled data informs the analysis of logged activity for isight as well as the application of site-specific query and user controls via iguard. 2) Analysis The analysis process will continually or periodically parse and associate logged SQL activity with result set metrics, application metadata and database objects accessed. It then populates isight reporting tables housed in the Teleran Knowledge Base. Teleran Report Server The Teleran Report Server (Rserver) is a Java Servlet Engine that manages the creation, scheduling, retrieval and storage of the Web-based isight Reports. It can be installed on any server class hardware available on the network and requires access to the Teleran Knowledge Base. Teleran Knowledge Base The Teleran Knowledge Base can be a stand alone database instance on the network or a schema within an existing database or data warehouse. It stores query/report activity, Teleran system configuration data, application and database system catalogue information, and usage activity reporting tables. Teleran Knowledge Base tables are automatically created in the specified database during the Teleran installation process. Teleran Administrative Console The Teleran Administrative Console resides on one or more Windows desktops and is used to configure, control, and monitor the Tserver process, manage Aserver processes and to apply iguard query control policies and Automated Helpdesk user messages. The Administrative Console requires network access to the Tserver, the Aserver, the Teleran Knowledge Base, and the application database or data warehouse. Architecture Summary Small footprint Java service No database agents or traces that slow performance No application programming required Efficiently operates at the DB protocol packet level 12

14 Supports DB2 Distributed and Mainframe, Oracle, MS SQL Server Teleran Customer Success The following Teleran customer case study describes the benefits and return on investment organizations have gotten from licensing and deploying Teleran software products. Teleran s unique combination of protection, performance improvements and operational cost savings delivers measurable returns and material business benefits. In 2006 GE Money needed to ensure that its consumer credit card data warehouse was in compliance with the Payment Card Industry (PCI) Data Security Standard that required auditing and securing the use of personal financial data. This critical application was also incurring database performance problems, violating customer service level agreements and impairing productivity of 600 credit analysts worldwide. GE licensed Teleran software to address both their compliance and database performance issues. By implementing Teleran isight and iguard, GE met their PCI audit and control requirements and saved over $800,000 in compliance costs versus developing an in-house solution as they had originally planned. To address their performance issues, GE deployed iguard policies to better control resource wasting SAS queries and unpredictable ad hoc analyses. isight auditing was used to tune the database model based on actual application usage patterns. In addition, isight ensured that Universes were synchronized with ongoing changes at the database. With Teleran, GE data warehouse performance improved by 25%, enabling GE to meet service levels required by customers and employees. William Klancko, IT Leader, GE Corporate At GE Money Teleran products enabled the company to satisfy PCI compliance demands and allowed us to increase system capacity and performance by 25%. As a result, GE Money is planning on rolling out Teleran products enterprise-wide. Teleran provided GE Money several advantages over competitive products in the market. These included: Supporting multiple database technologies on multiple server platforms, enabling enterprise deployment Capturing more data about the users, applications, database and users queries Offering active usage policy management Providing a non-intrusive architecture that did not consume database system resources Delivering integrated analytics and reporting, requiring no report development 13

15 Conclusion BI, data warehouse, enterprise reporting, and analytical applications are subject to unpredictable query patterns, changing user demands, and the inevitable user-created query problems that degrade performance, diminish productivity, and reduce service levels. Regardless of how much planning goes into these applications, user requirements change, databases grow, batch windows expand, user errors occur, and query volumes increase. The challenge: how to continuously manage the performance, user satisfaction, and value of this dynamic use while minimizing support staff, maintenance, and system resource costs. Teleran s objective is to help organizations meet these challenges with products that deliver essential BI and data warehouse management disciplines. Teleran s software products provide comprehensive awareness and management of user behavior, application activity, and database usage. They uniquely complement system level monitors and database utilities, bridging the gap between understanding and managing users and managing underlying database and system resources. With Teleran, organizations can now costeffectively manage their entire BI, data warehouse and reporting environments to get the greatest value from their investments in people, information, and infrastructure. Teleran Technologies, Inc. 333A Route 46 West Fairfield, NJ Phone Fax info@teleran.com Teleran Technologies is the leading provider of software for ensuring the performance, compliance, and cost efficiency of business intelligence (BI) data warehouses, CRM, Web-based and other enterprise applications. Through end-to-end knowledge of the application environment users, queries, applications and databases Teleran software aligns IT processes with business needs, improving compliance, reducing costs and enhancing business performance and productivity. Founded in 1996, Teleran pioneered the concept of activity monitoring and management for data warehouses and analytic applications with its patented policy engine and management process. Today the company provides solutions for many of the world s leading companies, including Allstate, Bristol-Myers Squibb, Citibank, CNA, GE, Mayo Clinic, Merrill Lynch, MetLife, Sun Microsystems, US Army, Unisys, Vanguard and Wachovia Teleran Technologies, Inc. All rights reserved. Teleran and the Teleran logo are registered trademarks and isight, iguard, and Automated Helpdesk are trademarks of Teleran Technologies, Inc. All other names are the property of their respective owners. SO