Automated Proactive Solution System (APSS) AUTOMATIC AND HEALTH SOLUTIONS

Automated Proactive Solution System (APSS) AUTOMATIC AND HEALTH SOLUTIONS

Table of Contents Table of Contents... 2 Overview... 3 Automatic Solutions... 3 Using Automatic Solutions... 3 Interactive Session... 5 Commcell Health Solutions... 7 Execution Mode Overview... 7 Automatic Mode... 7 Interactive Mode... 7 Manual Mode... 10 Detection Mechanism... 10 Automatic Client Groups... 11 APSS Solution Administration... 11 2

Overview The Commvault Automated Proactive Solution System (APSS) delivers traditional documentation-based, as well as orchestrationbased solutions from the APSS dashboard located in Commvault Maintenance Advantage (www.ma.commvault.com). Traditional solution-types in APSS are referred to as Intelligent Solutions due to the mapping system that intelligently maps detected failures to solutions. APSS introduces a new concept for delivering solutions that surpasses the traditional Knowledge Base (KB) concept. Instead of following manual steps in a KB to resolve a problem, APSS offers an automated approach for deploying solutions. These solution types include: 1. Automatic Solutions a. Using CommVault workflows to solve an issue 2. Health Solutions a. Using CommVault workflows to monitor and fix CommCell health-related issues. Automatic Solutions Automatic Solutions are based on Commvault Workflow technology. Automatic solutions are executed interactively, or according to Commvault Best Practices, to automate solution steps that are traditionally actionable only by following documented tasks. The APSS system orchestrates the resolution process and allows the CommCell Administrator to execute a single command to resolve detected issues persistently or on a case-by-case basis. Using Automatic Solutions Automatic solutions are obtained from the following locations: 1. Initial automation as bundled with the APSS Installer Platform 2. New automations downloadable from the APSS Dashboard when released A default bundle of solutions is included with the APSS installer. The Maintenance Advantage-based APSS dashboard, or notification based alerts, advise the CommCell administrator if a solution needs to be executed and applied view the APSS dashboard. As seen in Figure 1, there is a solution available for detected Error Code 19:757. 3

Figure 1- Error Code Example When clicking on the solution, the context appropriate landing page is displayed. The solution page contains a description of what the workflow does, and how to execute it. A download link is provided in the workflow description. See Figure 2. If the workflow is already installed and deployed, it is not necessary to re-download and deploy the solution. Figure 2- Solution Landing Page When the solution is applied to the CommCell the APSS dashboard reflects a new status. The new status is reflected after the next hourly data collection occurs. See Figure 3. 4

Figure 3 New Status Interactive Session Automatic Solutions typically use workflow Interactive Sessions to provide user input. An interactive session allows GUI interaction with the workflow solution. This is useful for providing an administrator the ability to select options during the workflow execution. For example, the Automatic Solution may request user input for a client name. See Figure 4. The client name is then processed to provide a new option via Interactive Session to select a subclient from a list. See Figure 5. The administrator then selects the client against which to execute the solution. Figure 4 User Input Figure 5 Subclient List 5

By default, interactive Sessions have a two minute timeout period. If the workflow takes longer than two minutes to provide the interactive menu, then the workflow GUI will time out. For this reason, the APSS Installer increases the timeout value of these sessions to ten minutes. If additional time is needed for session processing time the nwfresponsetimeout REG_DWORD located under HKEY_LOCAL_MACHINE\SOFTWARE\CommVault Systems\Galaxy\Instance001\Database can be further increased. This value is displayed in seconds (600 seconds is 10 minutes) as seen in Figure 6. Figure 6 Processing Timeout 6

Commcell Health Solutions Similar to Automatic Solutions, CommCell Health Solutions are powered by CommVault Workflow technology. This solution type acts as a proactive approach to solving problems in the environment by detecting problems as they occur and offering a solution an administrator can systemically apply. Unlike Automatic Solutions, CommCell Health Solutions remain dormant in the CommCell environment until they are triggered by specific detected criteria. CommCell Health Solutions do not need to be manually executed because they are self-executing on schedule or other conditions. CommCell Health Solutions are bundled with the APSS installer and are configured during deployment. Execution Mode Overview CommCell Health Solution workflows typically have three execution modes: 1. Automatic Mode 2. Interactive Mode 3. Manual Mode The execution mode selected determines how the workflow behaves in the CommCell environment. CommCell Health Solution workflows may have less than three execution modes. In some cases there is no manual execution, or the solution cannot run automatically because of the administrative inputs required to complete the workflow. During the APSS installation process the administrator selects the execution mode in which workflows are run. Automatic Mode When a Health Workflow Solution executes in Automatic mode the administrator is not prompted for user input during the execution. These workflows are automated and based on Commvault best practices. For example, if the apss_health_csdb solution is executed in Automatic mode, and a DBMaintenace Full operation is needed, the workflow will execute the DBmaintenance Full operation with no further action required by the administrator. Automatic Mode is intended for administrators that want the majority of their solutions executed automatically and without need for approval. An email is sent to the CommCell CVProactive administrators when the solution completes successfully. ** Important: Not every Health Solution has an Automatic execution mode. In some cases solutions require user input. Therefore that solution type expects to receive a request for user input for some CommCell Health Solutions even if Automatic Mode was selected. Interactive Mode Unlike Automatic mode, Interactive mode requires user interaction before it can execute the solution. For example, if the Health Solution Workflow detects that a DBmaintenance Full operation needs to run, the administrator is informed, and is prompted to approve or deny the request. The workflow cannot execute until the response is received. If no response is received, the input request times out after 12 hours, and the workflow will end with no action taken. 7

User input is submitted by email. Figure 7 is an example of an email for an Interactive Mode solution requesting user input. Responding to this email with the proper response will determine how the solution is executed. Figure 7 Interactive Mode email Request When the solution completes a summary email is provided. Figure 8 Interactive Mode email Summary 8

Configuring Email for APSS Integration Interactive Mode requires an email mailbox designated for approving workflow responses. It is best to designate a mailbox specifically for this task rather than using an active user mailbox. IMAP must also be enabled on the email server to allow mailbox access. The following inputs are used with the APSS installer to configure the Email Input functionality: Email Approval Host Email server name with IMAP enabled. For example: smtp.company.com Email Approval Mailbox email mailbox used for input approvals. For example: proactive@company.com Email Approval Mailbox Password password for the approval mailbox. This is an encrypted entry. Email Approval Folder folder where input responses are kept. For example: Inbox\wf_response or simply the Inbox if the email account is being used only for the purpose of workflow responses. Additionally these options can be changed manually under the CommServe Additional Settings as seen in Figure 9. **Important: The email response from the administrator MUST be found in the approval mailbox folder in order for the workflow engine to process the message. Additionally when changing these options a manual restart of Tomcat services and WorkflowEngine services is required. An example of how this configuration works, refer to Figure 9. When an administrator responds to a solution input email, the email will be sent to proactive@company.com using smtp.company.com email server. The workflow engine will look for the email response in the Inbox/wf_response folder in order to process the request. If preferred the ApprovalFolder can be set to Inbox. The workflow engine will look in the Inbox folder for approval responses. Figure 9 Workflow Option Changes When a health workflow solution executes and requires user input from the administrator, the workflow engine looks at the approval mailbox, under the folder location for the response email. As soon as the administrator responds to the workflow input request, the response is sent to the Email Approval Mailbox, where the workflow engine reads the response to process the solution. Regular exchange or email filters should process the email to the appropriate folder if necessary (especially if Inbox folder is not being used). Figure 10 below, graphically describes this flow. 9

Figure 10 Workflow Process User input is processed in a First in First Serve basis. This means that if multiple administrators respond to the workflow input, the first response will be selected for processing. Manual Mode Manual mode is an execution mode that allows the administrator to run each workflow manually to perform a single implementation of the solution. Manual Mode does NOT exist in all Health Solution workflows. For tracking purposes, it is best to use the Manual Mode of execution when implementing a solution rather than implementing the solution outside of the workflow. For instance, if the Manual Mode allows the ability to execute a DBmaintenance Full operation, it is best to run it from the Health Solution Workflow rather than manually from the command line. Detection Mechanism CommCell Health Solutions are executed using the following mechanisms: 1. Workflow Schedule 2. Alert Detection a. Standard CommCell Alerts b. Custom Alerts c. Log Monitoring Alerts When executed on schedule the workflow runs through diagnostic queries, or commands, to detect a condition to act upon. When an alert is used, a condition is detected in the CommCell environment which executes the workflow. 10

By design, Solutions configure themselves during the first runtime. If a Health Solution must be scheduled, during first execution the workflow will check if the schedule exists and create one if it does not. The same logic applies to required alerts, or Log Monitoring policies. Health Workflow schedules and alerts follow the same general naming convention. These schedules are prefaced with CVProactive, followed with a descriptive phrase as seen in Figure 11. Alerts, policies or schedules with CVProactive prefixed to the name should not be deleted. If they happen to be deleted simply running the Health Workflow will reconfigure these items. Figure 11 Health Workflow Descriptive Phrase Example Automatic Client Groups Automatic client groups are used to group clients that meet programmatic criteria. For example, to group UNIX clients with DB2 agents installed AND also have attached storage, an automatic group can be created to group the clients. CommCell Health Solutions may utilize custom automatic client groups for quick groupings of certain clients. The client groups are not used for security purposes but rather for grouping certain clients for solution purposes. Automatic client groups will have CVProactive prefixed to the name of the group. Health Workflows automatically create automatic client groups during execution if it is required for the solution. Figure 12 is an example of an automatic group used for a Health Solution Workflow. Figure 12 Example of Automatic Group 11

APSS Solution Administration APSS alert notifications and Health Solution email inputs are sent to all users associated with the CVProactive user security group. This group is automatically created by the APSS installer. See Figure 13. Figure 13 Automatic Group Creation The CVProactive security group does not have any capabilities. An administrator can add CommCell users to this group, whenever access to APSS alerts, and email user input requests want to be sent to those users REV 20160216 2016 Commvault Systems, Inc. All rights reserved. Commvault, Commvault and logo, the C hexagon logo, Commvault Systems, Solving Forward, SIM, Singular Information Management, Simpana, Simpana OnePass, Commvault Galaxy, Unified Data Management, QiNetix, Quick Recovery, QR, CommNet, GridStor, Vault Tracker, InnerVault, QuickSnap, QSnap, Recovery Director, CommServe, CommCell, IntelliSnap, ROMS, Commvault Edge, and CommValue, are trademarks or registered trademarks of Commvault Systems, Inc. All other third party brands, products, service names, trademarks, or registered service marks are the property of and used to identify the products or services of their respective owners. PROTECT. ACCESS. COMPLY. SHARE. COMMVAULT.COM 888.746.3849 GET-INFO@COMMVAULT.COM 2016 COMMVAULT SYSTEMS, INC. ALL RIGHTS RESERVED. 12