CONTENT STORE SURVIVAL GUIDE

Transcription

1 REVISED EDITION CONTENT STORE SURVIVAL GUIDE THE COMPLETE MANUAL TO SURVIVE AND MANAGE THE IBM COGNOS CONTENT STORE

2 CONTENT STORE SURVIVAL GUIDE 2 of 24 Table of Contents EXECUTIVE SUMMARY THE ROLE OF THE IBM COGNOS STORE CONTENT STORE TABLE STRUCTURE AND ACCESS TOP 10 KEY METRICS FOR CONTENT STORE ADMINISTRATION MANAGING CONTENT STORE GROWTH MANAGING MULTIPLE CONTENT STORES CONSOLIDATING AND MERGING COGNOS BI ENVIRONMENTS BEST PRACTICES FOR IMPLEMENTING IBM COGNOS BI SECURITY INTRODUCTION In the IBM Cognos world nothing is more shrouded in mystery than the Content Store. While every Cognos administrator knows its importance to the Business Intelligence (BI) environment, few completely understand its role in the BI delivery process. The least understood aspect of the Content Store is that it must be managed if the BI environment is going to perform well and remain healthy over time. This survival guide provides working knowledge about the IBM Cognos Content Store and its management.

3 CONTENT STORE SURVIVAL GUIDE 3 of 24 Executive Summary The IBM Cognos Content Store acts as a repository for both metadata and content for the BI environment. Managing it presents challenges particularly as the environment evolves over time and through different stages of growth. Administrators must develop and pay attention to key metrics of each Content Store and successfully manage them. As the Cognos environment grows complexity increases and administration becomes more of a challenge, and while the magnitude of problems does not always scale along with size, it does often enough to cause real pain and sometimes disastrous results. The lack of tools can contribute to administration issues. Offthe-shelf solutions are available to address this and can greatly aid in administering the Content Store. In addition, there are several key metrics that, when tracked appropriately, decrease the administration effort including that of growth control. The consistent use of best practices reduces much of the risk of administering and managing Content Store environments primarily because when problem issues are addressed early their impact can be reduced or eliminated. This survival guide will help you identify what you need to manage and what best practices might be appropriate for your environment. So what are the keys to successfully managing the Content Store over time? The most obvious ones are: Effective user training. The BI user community must be trained on how to effectively utilize the IBM Cognos BI tools and best practices. Administrators need to track and manage key metrics that relate to their user base and the Content Store. The use of effective tools by the administrators to: Instant access to key information Manage change lifecycle management Manage content Identify and address key issues Manage security

4 CONTENT STORE SURVIVAL GUIDE 4 of 24 The Role of the IBM Cognos Content Store At a recent IBM Cognos session focused on the Content Store, over 100 attendees were polled to determine the level of familiarity with the Content Store. Less than 1 in 5 had any working knowledge of how the Content Store functions or the amount of content it contained. This is not a surprise since for most users the Content Store is simply a part of the Cognos environment. The Cognos Content Store has two essential roles: 1. A repository for Cognos BI metadata. 2. A storehouse for user output. It s important to note that it functions less well in its second role as a storehouse for users output. The Content Store is dynamic. New objects are added and objects are deleted and changed on a daily basis. An object like a report has multiple attributes and each of these can have levels of sub-attributes relating to things like jobs, schedules, triggers, etc. All of these attributes are likely to be important at one time or another. Know which reports have triggers and the name of the trigger can be important to know if reports are not executing on a timely basis. Figure 1: Content Store Roles

5 CONTENT STORE SURVIVAL GUIDE 5 of 24 The Content Store performs extremely well for what it is designed for: High speed reporting. However, it does not provide administrators with many tools to effectively manage the Cognos BI environment on a day to day basis. The following information can be difficult to obtain without specialized tools: Size of the Content Store Make-up of object mix Amount of saved output Rate of growth Areas of growth Diagnostics of problem areas Unfortunately, for many Cognos administrators the lack of tools to get basic information like this can go unrecognized until problems occur. are no IBM Cognos tools for managing or monitoring metadata such as dependencies and relationships on a large scale. It is also difficult to find and use information to easily assess the impact of database changes on reports, queries, or packages. In addition, there are many other activities that must be addressed by Cognos administrators, authors, or developers. There are basically two ways to address this issue: 1. Build a solution in-house. 2. Consider an off-the-shelf solution like NetVisn - Option 2 may be the only practical way to provide the necessary tools to administrators if the financial resources are available. Option 1 requires time, resources, and gets messy over time when new versions of Cognos BI are released. While some of this can be retrieved with individual queries into the Content Store, it is an inefficient process to use. There

6 CONTENT STORE SURVIVAL GUIDE 6 of 24 Content Store Table Structure and Access The metadata in the Content Store is organized in a set of database tables. At last count there were approximately 200 of these. The tables are created when the Cognos Configuration Manager is implemented for the first time. As users are added to the environment and more content is created the Content Store begins to grow in size. Without any rules on saved output retention or active archiving the output portion of the Content Store will typically grow much faster than the metadata portion of the content. This often goes unnoticed until a problem occurs. As mentioned in the previous section, there are no simple or convenient processes to retrieve basic metrics for size, item count, speed of growth and others. Direct queries into the Cognos Content Store can retrieve this information but this can be tedious and time consuming. Figure 2: Saved Output Growth Over Time

7 CONTENT STORE SURVIVAL GUIDE 7 of 24 How large is a typical Content Store? It depends on the age of the environment, user population, number of objects stored, and many more variables. For example, a Cognos environment of 800 to 1,000 users that has been in place for a year or two may have 12,000 to 20,000 objects or more. Each of these objects can have multiple (20+) properties that relate to security, schedules, jobs, and other properties, increasing the level of complexity. A basic tool to get data out of the Content Store is adequate but to address issues such as dependencies, relationships, permissions, and changes a more systematic approach is needed. The Content Manager Browser Tool is available for download on the IBM Cognos web site and can be used by administrators to access this information. This tool provides some basic, though limited, capabilities: Summary of the number and size of the various objects in a simple display format. Ability to query folders and types of objects. In order to see meaningful trends, the user would have to perform these steps on a regular basis and then build a basic data repository to monitor the information over time. While this approach will work it provides only limited information and no ability to drill down on the data. Determining why and where the trends are occurring adds another level of complexity. While there are other ways to directly query these tables for the occasional data need, building ad hoc tools for this may have only limited success: IBM Cognos reserves the right to change these tables and their structure at any time. Query routines or applications may be incompatible with new versions. IBM Cognos recommends using their SDK Validation Tool for systematic access to the Content Store. A robust third party solution is a better alternative for broad and deep BI administration needs.

8 CONTENT STORE SURVIVAL GUIDE 8 of 24 Top 10 Key Metrics for Content Store Administration Managing the Cognos Content Store starts with knowing what s in it, how large it is, how fast it s growing, and where the growth is taking place. The metrics discussed here may not be appropriate for all environments. But it is a place to start the process of managing the Content Store in a way that s aligned with the BI strategy and plans. These are the key metrics that administrators should monitor in terms of production of the Cognos Content Store. Many can directly impact overall performance and user satisfaction. 1. SIZE Unimportant as a single metric. Best used in conjunction with other metrics. 2. RATE OF GROWTH Track in terms of number of objects as well as saved output. More critical in environments that are large and/or growing rapidly. Can be a key indicator of real or potential problems in the Content Store. Tracking growth should be done at least monthly.

9 CONTENT STORE SURVIVAL GUIDE 9 of NUMBER OF OBJECTS BY TYPE Useful metric for determining the mix of metadata objects. It s helpful to track the relationship between the number of reports and queries relative to the number of packages or models over time. Best Practice: Create specific packages for individual areas such as sales, logistics, and marketing instead of relying on one or two very large packages for everything. Example: Figure 3 shows an object count summary by type for a sample Production Content Store. Figure 3: Object Count Summary

10 CONTENT STORE SURVIVAL GUIDE 10 of SAVED OUTPUT Amount of saved output in mid size or large environments can dwarf metadata object file size unless retention rules or output archiving is in place. Users may be saving output from very large reports over time. Best Practice - Set retention rules and archive saved output. Example: Figure 4 is showing significant growth in saved output relative to metadata objects over an eight year period this is a typical example. 5. EXCESSIVE REPORT CREATION Indicates users may not understand bursting or prompts. Figure 4: Saved Output Detail

11 CONTENT STORE SURVIVAL GUIDE 11 of VERSION CREEP Keeping versions of models, packages, or other objects far beyond the needed time period can result in poor performance. Models and packages can be huge keeping unneeded versions of them can negatively impact content store performance. Best Practice - Turn off versioning unless you really need it. 7. MISSING OBJECTS Can result in a null response when users attempt to run or receive a scheduled report. Examples: User creates a view of a report but report is somehow deleted. Distribution list mistakenly deleted and the users do not receive reports. 8. UNUSED OBJECTS Track usage of both content and users across the Cognos environment. Content that has not been used for a period of time (2 3 months or more) should be a candidate for removal. Best practice - Purging public folders of unused content makes searches easier and improves Content Store performance. Unused user licenses can be reassigned to new users. 9. OBJECTS WITH BROKEN LINEAGE Lineage breaks occur when a package is modified and published without consideration for reports or queries using it. Lineage breaks also occur when a data item(s) no longer exists. At any given time the percentage of objects with broken lineage in a typical production Content Store is 10% to 15%. Identification and resolution of objects with broken lineage before users are impacted reduces calls to the help desk and the number of frustrated users. 10. DUPLICATE OBJECTS Identifying two or more copies of the same object across the environment may indicate a need for better user training.

12 CONTENT STORE SURVIVAL GUIDE 12 of 24 Managing Content Store Growth In the years that IBM Cognos 8 and 10 have been available there have been a number of trends underway in terms of how it is used. The most interesting one is the dramatic increase in the amount of overall content in the Cognos environment and the accompanying increase in the size of the Content Store. It is not uncommon to see Content Stores today that are well over 500 Gb in size. Causes of Dramatic Growth The evolution to Cognos 10 has resulted in the perfect balance between standardized, functional reporting and ad hoc individual reports that users create for themselves. But some environments have seen an explosion of My Folders content far beyond what could have ever been predicted. In some cases this reflects the fact that the standard reports may have been poorly designed but it can also reflect users creating detail, specific reports to meet their own job needs. User driven content creation can be a blessing and a curse. Some environments have found that user created reports can have the potential for much broader usage than a single user and often turn them into public reports on a regular basis. But there are also users who create multiple reports, often similar to public reports, and then schedule all of them to run on a regular basis and then save the output. This can create real problems in terms of large increases in the size of the Content Store over time especially as it relates to performance. A large Content Store in and of itself is not necessarily cause for alarm. But size can become a problem when it begins to affect performance. For example, a user backs up a very large model each week to a different file location thinking he s doing the right thing. It soon has a noticeable impact on overall performance. If users do not understand the impact of doing similar things, the Content Store is going to grow but in a way that negatively impacts users performance. Growth should be organic as the result of new users and new content focused on real user needs.

13 CONTENT STORE SURVIVAL GUIDE 13 of 24 Controlling Growth Here are specific practices to keep growth manageable: Monitor Content Store growth on a regular basis using the IBM Cognos utility or a product like NetVisn that will provide broad insight into causes of growth. Training Users - with emphasis on best practices. Effectively done, this is the single most important thing you can do. Administrators and modelers need to know how growth issues can occur. Example: inadvertently or intentionally turning on package versioning on large models. Set and enforce retention policies Set rules on saved output retention, both amount/size and length of time. Obtain user input before setting rules to balance needs across the user base. Archive content that must be retained. Delete orphan content In any large environment users are being added and deleted on a daily basis. Ensure that content from deleted users is also deleted

14 CONTENT STORE SURVIVAL GUIDE 14 of 24 Managing Multiple Content Stores Multiple Content Stores also have a role in the typical IBM Cognos Business Intelligence installation. There are similarities and differences between store types and usage depending on the organization. As customers move to take advantage of what s new in Cognos 10, the role of the Content Store is even more critical. It has more capabilities and more users relying on it. Most Cognos installations will typically have three distinct environments: Production: Deliver BI content to end users. Test or QA or Pre-Production: Validate the functional readiness of new or changed content. Development: Develop new content or modify existing content. Some installations may have additional environments in the BI delivery chain; some fewer. A sandbox environment, for example, is often a good place to just test things without impacting user focused environments. And in small to midsize installations it is not uncommon to see only a production and a development environment where development also serves as the test environment. A Cognos installation may be just a single production environment. In this case both development and testing coexist with production in a single environment. It should be noted that this is a high risk practice and not recommended. Content that has a different purpose at a given point in time should be in a different environment. Using a single environment for multiple activities significantly increases the likelihood of problems with security, data sources, and more. Best Practices There are some themes that run through the best practices of all Content Store environments: Clearly defined policies and procedures in place. Defined data sets for each Content Store. Security profiles for each environment are in place. Readiness rules defined and in place for content movement.

15 CONTENT STORE SURVIVAL GUIDE 15 of 24 Deployment history of all content moved from one environment to the next. This needs to be a complete record of what transpired.(who, what, when, etc.) Tools should be available and enable administrators to work both in and across all environments simultaneously. Monitoring of key metrics: May differ from one installation to another Must include: Usage by content and users Growth in content and users Production Best Practices: Organize content for easy user access Limit folder depth and number of clicks required for content access. Clearly label content makes it easier to find & reduces redundant reports. Create a clean structure for easy navigation. Develop criteria for production ready content. Create rules for managing My Folders. Mandate and routinely test backup and recovery procedures. Manage and validate security in real-time. Production Content Store In a production environment there is no room for error. When there are hundreds or thousands of users there must be policies and procedures in place which control working in the production environment. Validate data sources on an on-going basis. Monitor content and user usage on an on-going basis. Remove unused content and excess output. Archive saved output. Have tools in place for administering the environment. Implement schedules, triggers, distribution lists, etc. No development takes place in production.

16 CONTENT STORE SURVIVAL GUIDE 16 of 24 Test or Pre-Production Content Store Best Practices: The Test or Pre-production environment is similar to production but there are differences. Security should represent the production environment. Use active rather than static data sources. Ability to compare to production environment for: Security Metadata Data sources Content counts (deleting old or unneeded test content) items moved to production (including who-whatwhen). See Figure 5. Practice content version control. Make change management a major focus. Mandate and routinely test backup and recovery procedures. Track usage (is the content really being tested as it should be?) Develop test criteria for production readiness. Keep history of completed testing. Tools in place for administering the environment. Maintain a log history of Figure 5: Log File of Content Promoted to Production

17 CONTENT STORE SURVIVAL GUIDE 17 of 24 Development Content Store The development Content Store is where all new content is authored or created. Tools in place to assist authors/developers in diagnosing problems in the development content. See Figure 6. Make change management a major focus. Development Best Practices: Security model should reflect only the development environment. Design policies and procedures specifically for development. Data sources are typically static but representative of production. Criteria in place for moving content to Test or Preproduction. Maintain a log history of items moved to Test or Preproduction. Design folder structure to meet author/developer requirements. Tools in place for administering the environment. Figure 6: Broken Data Lineage

18 CONTENT STORE SURVIVAL GUIDE 18 of 24 Consolidating and Merging Cognos Business Intelligence Environments Merging or consolidating Cognos environments can be a challenge even in a small environment. In a large environment the difficulties are magnified. Creating a plan that reflects the specific needs when merging environments is essential. Using tools that make the task easier and minimize risk can help insure success and reduce the time and effort involved. Managers or administrators are often faced with the task of consolidating or merging Cognos Business Intelligence environments. This may be due to mergers, a need for cost savings, or simply a desire to reduce the number of environments. Invariably the consolidation involves users and content that will be merged into a new or existing environment. This requires careful planning and flawless execution to succeed. Merging environments is a major project. It involves merging one existing environment into another or merging two or more environments into a new one. An example would be merging two Cognos 8 environments into a new Cognos 10 environment. Step 1: Determine Scope of Merger Determine the scope of the environments to be merged. This requires detailed analyses to discover the data needed for planning. Needed information includes: Extent of similarities and dissimilarities between environments. Number of users. Amount of content. Duplication of users and/or content between environments. Differences in security model (if any) between source and target. Identify and eliminate unused content if possible.

19 CONTENT STORE SURVIVAL GUIDE 19 of 24 Identify special issues depending on the specific needs of the environments. If the merger involves more than two environments this information is required for all environments under consideration. In many respects the task will be significantly easier if there is no overlap in users and content between the environments to be merged. Separate and Distinct Environments: Source and target environments that are dissimilar and have no overlap reduce the tasks involved. See adjacent: Similar or Overlapping Environments: When the source and target environments are similar there are more tasks to address before merging the two environments. Here both source and target are similar raising the possibility for multiple issues on common data sources, folders, and models. Identifying and resolving these issues is absolutely necessary for a successful merger. Step 2: Planning and Execution After determining the scope of the merger using the information gleaned from the environments, identify and layout the key tasks in the right sequence. Most problems in a merger occur when key steps are missed or performed in the wrong order. Planning includes: Collect required information in detail. Decide how to handle common issues. Decide how to handle unused content. Modify (if necessary) the security model in the target environment. Map the tasks to the timeline. Review the plan with key constituencies. Testing and validation. Communicate the plan to users and set expectations. Execute the plan.

20 CONTENT STORE SURVIVAL GUIDE 20 of 24 To avoid a messy deployment you can use a product like NetVisn to perform the merger in a more controlled manner. Advantages: Allows content to move from the source to the target with known outcomes. Greatly reduces risk if hundreds or thousands of users must be moved. Infinite level of selectivity around objects: reports, personal folders and content, etc. Validates all objects as they are being moved to insure that all dependent objects are moved together and will work in the new environment. Alternatively you can use a deployment package instead, but this is an all or nothing approach that can instill fear in even the most careful planners. With a deployment package there is no selectivity below the folder level and personal reports cannot be dealt with at all unless you take all of the personal reports that are in the Content Store. But for many this may be the only alternative. Remember, planning is the single most important part of the entire effort in merging or consolidating environments. The time you spend here has a huge payback in terms or risk avoidance and minimizing problems. Minimizes or eliminates any orphan content.

21 CONTENT STORE SURVIVAL GUIDE 21 of 24 Best Practices for Implementing IBM Cognos BI Security Controlling user access in an IBM Cognos environment can be summarized by these two goals: Secure sensitive data from unwarranted access, but allow the necessary data to be available to all business intelligence consumers. Control access to Cognos BI capabilities, both globally and package based, so that content is created and distributed by approved authors, and that Cognos license limits are respected. The best practices described here may not be the best in all environments but will hopefully help those new to Cognos BI or for those about to refactor how Cognos security is set up. Use Existing Groups If your external security is also used in a corporate environment it s likely that the accounts are maintained in an organization of groups. Study this organization to see if it can be used to control access in Cognos, probably to content. Alternatively, you may be using an external security specifically for Cognos, such as Cognos Series 7. Because an account must belong to a group in Series 7 in order to be recognized by Cognos BI, you have a couple of choices: Create groups in Cognos security to organize accounts that will be used to control access in Cognos, either for capability or content, though probably the latter. Add all accounts to just a single group and manage all access using the Cognos namespace groups and roles. Groups or Roles Group and role objects in the Cognos namespace behave almost identically. The difference is that groups can contain only accounts and other groups, while roles can contain accounts, groups and other roles.

22 CONTENT STORE SURVIVAL GUIDE 22 of 24 Organizing multiple groups in a role could get complicated very quickly, so it may make sense if you use the role for broad access control and the groups for limited access. A simpler rule to follow would be to use roles to control access to capabilities, and groups to manage access to content. Following this simple rule can save you a lot of confusion and headaches down the road. Managing Content Access The design of security access to Cognos BI content first requires an analysis of the types of business data available. Generally data will be organized at a high level by business unit or functionality, such as order processing or finance for example. Data may then be classified by employee position. For example, managers would have access to payroll detail reports but clerks may only view high level summary reports. One solution would be to create a group for the business unit (Payroll Unit) and groups for more limited access (Payroll Managers). Managers would belong to both groups. The reports which all payroll unit employees can view would use Payroll Unit for security and limited access reports would use Payroll Managers. groups; for example, Payroll Unit Consumers and Payroll Unit Authors. In this case, both groups will be used on report security but the access permissions would be set according to how read and write permissions are aligned. Managing Capabilities Capabilities are used in Cognos BI to control access to features and functions such as the reporting studios and administration tools. There are a number of default Cognos namespace groups that are created during the Cognos installation that have certain capabilities defined. For example, Authors and Query Users have access to Query Studio, but Authors also have access to Report Studio. It is recommended that new roles be created to manage user capabilities that match the distribution of your Cognos licenses. For example, a role could be created for power users to access all studios and another role for users which only need a PowerPlay license. The advantage of organizing capabilities this way is that it makes it easier to manage your Cognos BI licensing compliance. You will also need to manage read and write permissions to the BI reports. One method would be to create separate

23 CONTENT STORE SURVIVAL GUIDE 23 of 24 SUMMARY By following many of these best practices you can establish some structure to how security is applied that will help keep order in this area as your BI environment grows and changes. Without this, you are more likely to evolve quickly into a situation where your security is complicated and difficult, or impossible, to maintain. The IBM Cognos Content Store plays a vital role in the administration and productivity of the Cognos Business Intelligence Solution. In a well managed installation with a focus on best practices, and the right tools to work with, the Content Store(s) can function flawlessly. It does not require constant attention but there are some things you need to pay attention to on a regular basis. If you do this you can eliminate pain and risk from your installation and focus on giving your BI users what they need with high uptime.

24 CONTENT STORE SURVIVAL GUIDE 24 of 24 Envisn Information Solutions Envisn's IBM Cognos Blog 233 Ayer Road Harvard, MA USA t: f: e: LIN G+ FB TW RSS