Change Control and Configuration Management IVT 11 th Annual conference April 2010 DISCLAIMER Amgen is not responsible for the written or verbal bl content t of fthi this presentation tti Gisele Fahmi, B.Eng., M.A.Sc. 1 2 Agenda-Part 1 Definitions and Overview Change Control Configuration Management Change & Configuration Management Relationship IS Change Control/Management Terminology Key Resources Responsibility Type Categories Phases Process Summary Tips Emergency/Urgent Changes Agenda-Part 2 IS Configuration Management Terminology Key Resources Responsibility Phases Process Summary Tool tips Summary Interactive session Useful/Reference Links 3 4 1
Definitions and Overview Change Control Change Control/Management is a formal process used to ensure that: Changes to a product or system are introduced in a controlled and coordinated manner It reduces the possibility that unnecessary changes will be introduced d to a system without t forethought, ht introducing faults into the system or undoing changes made by other users of software The goals of a change control procedure usually include: Minimal disruption to services Reduction in back-out activities Cost-effective utilization of resources involved in implementing change Definitions and Overview Change Control Change Management is A structured approach to transitioning individuals, teams, and organizations from a current state to a desired future state (Source: Wikipedia) The process during which the changes of a system are implemented in a controlled manner by following a pre- defined framework/model with, to some extent, reasonable modifications A systematic approach to proposing, evaluating, approving, implementing, and reviewing changes (Source: ICH Q10) 5 6 Definitions and Overview Configuration Management Configuration Management () is a field of management that focuses on Establishing and maintaining consistency of a system's s performance, its functional/physical attributes with its requirements/design, and operational information throughout its life Definitions and Overview Configuration Management Configuration Management is responsible for: Identifying, Controlling, and Tracking all versions of hardware, software, documentation, processes, and procedures under the control of Change Management These items are referred to as Configurations Items (CIs) and all changes to them are recorded and tracked throughout the component lifecycle 7 8 2
Definitions and Overview Change & Configuration Management Relationship A CI is an aggregation of hardware or software or both that is designated for Configuration Management and treated as a single entity in the Configuration Management process (Source: IEEE) During the assessment of a proposed change, the Configuration Management process identifies the CIs affected by the Change Change Implementation involves update and verification of impacted CIs and documentation IS Change Control/Management Terminology Baselines: Baselines are the core of Software Change Management (S); they provide a stable platform to work from The Configuration Items that are identified determine the baseline (s) associated with the project 9 10 Change Baseline Scheme Diagram Requirements Analysis/Gathering Design Specification Source Code Software/System Design Testing/Data/Results System Deployed in Production Coding Execution System Requirements Specification Release IS Change Control Terminology Baseline Baseline is a secure specification of the software in the current state: Changes to the baseline can only be made by following strict change control procedures. The baseline must be protected df from any unauthorized changes A new baseline is established for each complete set of approved system changes Each baseline must include a cross-reference, reference, or traceability matrix that maps each design element to their corresponding software requirements 11 12 3
IS Change Control Terminology Change: The addition, modification or removal of anything impacting a regulated system. The scope should include all IT Services, Configuration Items, Process, Documentation, etc. Change Record: A Record containing the details of a Change. Change Records should reference the Configuration Items that are affected by the Change The receipt of a change request initiates the lifecycle of a Change Record IS Change Control Terminology Change Control Board (CCB) or Change Advisory Board (CAB): An authoritative and representative group of cross- functional resources people who assist the Change Manager in the assessment; prioritization and scheduling of changes for all high impact Requests for Change (RFCs) They advise Change Management on the priorities of RFCs and propose allocations of resources to implement those Changes A CAB is an integral part of a defined change management process designed to balance the need for change with the need to minimize inherent risks 13 14 IS Change Control Key Resources Responsibility Determining roles and responsibilities The 4 Ws: who what when hn and why of change IS Change Control Key Resources Responsibility Change Manager () Reviews RFCs to ensure adherence to the Change Management Process Provides guidance and training to s throughout the change control process Accepts or rejects RFCs Plans and chairs the CCB/CAB meetings Closes RFCs upon successful completion of all change related activities 15 16 4
IS Change Control Key Resources Responsibility Change Owner () Owns the change throughout its lifecycle Assigns, coordinates, and ensures completion of assessments and approvals and presents proposed changes to the CCB/CAB Creates implementation i and roll back plans Assures that areas affected by the change are notified in advance of the release and verifies successful implementation of the change Assures testing is performed and results are appropriately documented Notifies Change Manager of any unsuccessful changes and takes appropriate measures to remediate/resolve 17 IS Change Control Key Resources Responsibility Change Requester (CR) Collects basic information to initiate the RFC in the change control management system NOTE: Change Requester is the in most cases Business Owner (BO) Accountable for providing a business assessment to support the change Ensures that new system/service requirements are delivered via the change Assures the impact assessment is appropriate and fulfills the business needs 18 IS Change Control Key Resources Responsibility System Owner (SO) Owner of the impacted CI and accountable for providing a technical assessment to support the change Responsible for ensuring a system is designed to meet the business requirements and steady state support Assures the impact assessment and the rollback plan are appropriate within the planned schedule Works with all interdependent System Owners, if applicable, of the RFC CI prior to the change implementation IS Change Control Key Resources Responsibility Quality Approver (QA) Serves as a CCB/CAB member Reviews and approves or rejects RFCs Ensures change content is compliant with the appropriate regulations and procedures Validation Assessor (VA) Serves as a CCB/CAB member Defines validation activities and strategy required Ensures compliance during the change execution Identifies compliance issues, if applicable, that must be resolved before the RFC moves to the next change phase 19 20 5
IS Change Control Type Categories Critical Change Significant impact on the functionality, including qualified/validated status of GxP systems Highest risk factor/level and impact to critical business processes and service level This change category requires extensive planning, scheduling since it implies cross system impact The change requires planned outage outside of schedule maintenance window defined for the system Examples: Full version upgrade of a GxP application, involves release of a new GxP application 21 IS Change Control Type Categories Medium Requires substantial resources to plan, build and Change implement Medium risk factor/level and minimal impact to business processes The change does not result in significant functional modifications Changes do not pose significant impact to the validated/qualified status of GxP systems 22 IS Change Control Type Categories Minor Change Minor risk factor/level Planning, scheduling, and activity coordination takes place within one single functional area Only impacts a single system Do not involve a full version upgrade of a GxP application Do not involve a new release of a GxP application IS Change Control Type Categories Standard This change category is governed by a specific Change procedure documented in details for auditing purposes Changes classified under this change type are part of routine/maintenance activities Change does not affect the functionality or validated/qualified status of GxP systems 23 24 6
IS Change Control Type Categories Risk factors are determined based on: Number of users Number of system impacted Cross functional/global impact Regulatory status Business activities supported by the system Planned downtime required for implementation/deployment in production Resources required to implement NOTE: Critical and Medium Changes are required to be presented at CCB/CAB meeting. Minor changes are at the discretion of the Change Manager IS Change Control Phases Creating a Change Control Process KIM (Keep In Mind) Documenting the change request life cycle Establishing and communicating change control procedures Facilitating change from requirements through maintenance 25 26 IS Change Control Phases Initiation Change requester initiates the RFC by recording at a minimum: A A change owner if different from change requester Description of the change CIs impacted by this change Requested completion date The recommended change type category Success criteria of the change Roll back plan A A unique identifier is assigned to every RFC IS Change Control Phases Review/ Change Manager reviews the submitted RFC then rejects/accepts the change based on the Initial completeness of information Authorization If rejected, the Change Manager records the rejection reason and the RFC status is changed to return to the Change Requester If accepted, the confirms the change type and promotes the RFC to the next level/phase The SO and BO provide their initial authorization to the change record For Standard Changes, the promotes the RFC and the can perform the change record phases and proceed to closure 27 28 7
IS Change Control Phases Assessment/ assigns the corresponding appropriate Development resources to each required assessment and initiates the assessment phase For Standard Changes, no regulatory assessment nor approval by Quality is required Change Assessors complete the following assessments (refer to next slide): IS Change Control Phases Assessment/ Technical Assessment-SO Development Change Impact: describe changes required to the system design and impact of those Cont d changes on other systems Resource Analysis: estimate the resources needed (Network, Developer with specific skills, etc.) Document Update Plan: identify key change deliverables required but not limited to: Code/design review, Design Specification, Admin SOP, etc. 29 30 IS Change Control Phases IS Change Control Phases Assessment/ Development Cont d Technical Assessment-SO-Cont d Proposed solution/design strategy: Specify technical details of the change and how it is being performed Test plan: testing strategy required to test the change. Includes the rationale for the proposed testing approach Rollback plan: back out plan description in case of a system/change failure Release plan: Implementation schedule, plan and audience to be notified of change release Assessment/ Development Cont d Business Assessment-BO Impact of change: describe the changes needed to support the business process and/or business requirement Resource analysis: analyze what resources are needed to represent the affected business area (s) Document update plan: key change deliverables but not limited to: Requirements Specification, Operations SOP, Training documents Training Plan: specify timeline for roll out of training 31 32 8
IS Change Control Phases Assessment/ Validation Assessment-VA Development Validation Impact summary: Impact to the validated state of the system, impact to Validation Cont d docs and SOPs if applicable Document Update Plan Training Plan: Verify if completed via technical or business assessment Test Strategy: Specify environments that need to be tested and types of testing required. Also provide details about what validation documents and SOPs are required. 33 IS Change Control Phases Assessment/ reviews the accuracy of the Development assessments and obtains approvals/assessments for the change before Cont d proceeding (Authorization to proceed requires and/or CAB approval) SO, BO, CAB- -if selected, and Quality complete their assessment authorization The change enters development and is completed per the requirements and design changes Development summary is documented in the change record Note: CAB includes: technical/business/compliance members 34 IS Change Control Phases IS Change Control Phases Test/Approval to Implement Record test results in the test summary of the change record Required documentation needs to be complete prior to proceeding with CAB approval (i.e. Test results summary recorded and documented in the change record) organizes and presents test results at CAB CAB release approval obtained; then BO, SO, Quality provide their release approval Release/ Deployment After change is approved for release, coordinates change deployment in production ensures release package is scheduled and activities required for release are complete Change is released into production 35 36 9
IS Change Control Phases IS Change Control Assessment Matrix Based on Change Type Closure reviews the post-implementation package to ensure the change record is complete closes the change record Change Type Minimum Required Assessments Technical Assessment Business Assessment Regulatory Assessment Regulatory Critical Medium Regulatory Minor Required Required Required N/A Required Required Standard N/A N/A N/A 37 38 Signature Authorization to proceed Change Type (Regulated) Critical Medium Minor Standard Initiation IS Change Control Process Summary Change control Phases (with Authorizations, Assessments, Tasks, Approvals, and Roles) Review& Assessment Validation Assessment Development Test/Approval Release Release & Initial Assessment Authorization to implement Approval deployment Authorization Authorizations Tasks Tasks Authorizations Tasks Tasks Approvals Tasks Tech-SO Bus-BO Tech-SO Bus-BO Tech-SO Tech-SO Tech-SO Bus-BO Tech-SO Bus-BO Tech-SO Initiate RFC Request- Validation- VA Quality QA Quality QA Quality QA Development Summary- Development Summary- Development Summary- Test Summary- Test Summary- Test Summary- Close RFC- CAB- Tech- SO Bus- BO CAB- CAB- Tech- SO Bus- BO CAB- N/A N/A N/A N/A N/A N/A Quality QA Quality QA Quality QA 39 Closure Tech- SO Tech- SO Bus- BO Bus- BO Initiate RFC Request- Validation- VA Release Verification- IS Change Control Tips Release Verification- Close RFC- Tech- SO Tech- SO Initiate RFC Request- Validation- VA CAB- CAB- Release Verification- Close RFC- Initiate RFC Request- Release Verification- Close RFC- Before implementing any change it is recommended: To have an informal review involving target resources to be impacted by the change (SO, BO, etc.) Review the impact of the change requested. It is not always necessary to apply the change (e.g. it may be a nice-to- have low priority request) Perform an analysis and assessment of the technical proposal of how the requested change will be engineered Review the cost and resource estimates in case further steps are required BENEFIT: Perform the above steps early to identify potential cost/time savings by incorporating into a subsequent release. 40 10
IS Change Control - Emergency/Urgent Changes Definition: Emergency Changes relate to immediate resolution of a known production incident where an outage of a system, application or other service component has occurred Emergency Changes must be associated with a high priority incident 41 IS Change Control - Emergency/Urgent Changes Emergency Process Description: 1. The Change Requestor initiates the RFC: Unless a delay in action would create a major business impact (e.g., potential loss of product, unrestricted spread of a computer virus) If an RFC cannot be created, obtain an emergency change approval from the SO, BO or CAB member before proceeding. This approval can be obtained verbally, followed by an email within 24 hours which will be provided to the Change Manager and attached to the to-be change record Initiate an RFC within 1 business day of performing the emergency remediation 42 IS Change Control - Emergency/Urgent Changes Emergency Process Description-Cont d: 2. The RFC is reviewed by the Change Manager to ensure the required information has been recorded. dd The RFCi is either ih accepted or rejected based on the outcome of the review 3. A change record is created 4. The Change Manager obtains approval/rejection from the SO, BO or CAB member. The Change Manager updates the change record per the decision. This approval can be obtained verbally. 43 IS Change Control - Emergency/Urgent Changes Emergency Process Description-Cont d: 5. The Change is deployed. Ensure that documented evidence of the change is maintained and attached to the change record 6. The Change Owner will notify impacted parties (including Quality for Regulated CIs) within 1 business day of the emergency 7. The remaining tasks in the change record must be executed within 1 month after the release of the change 44 11
IS Configuration Management Terminology Configuration Management Database: A DB is a repository of information related to all the components of an information system A DB helps an organization understand the relationships between the system components by recording configuration items (CI) and details about the important attributes and relationships between CIs IS Configuration Management Terminology Cont d Configuration Management System (S): A set of tools and databases used to manage configuration data. A S is maintained by a configuration management process Configuration Item (CI): An object that is treated as a self-contained unit for the purposes of identification and change control. All configuration items (CIs) are uniquely identified by codes and version numbers 45 46 IS Configuration Management Terminology Cont d Configuration Management oversees the lifecycle of the CIs through a combination of processes and tools The objective of these systems is to avoid the introduction of errors related to lack of testing or incompatibilities with other CIs IS Configuration Management Terminology Cont d From the perspective of the implementer of a change, the configuration item is the "what" of the change. Altering a specific baseline version of a configuration item creates a new baseline version of the same configuration item In examining the effect of a change, first consider: What configuration items are affected? and How have the configuration items been affected? The above considerations are part of the change management impact analysis 47 48 12
IS Configuration Management Terminology Cont d 1. Examples of CI attributes/properties/specifications in scope for the S include (but not limited to): CI Name: Application/System name CI Classification: Business Application: collection of components deployed and assigned a version number Business System: group of interdependent applications and other system resources that interact to accomplish specific business functions Business Service: service delivered to business customers Business Process: a system or procedure that an organization uses to support a business service Infrastructure software and hardware IS Configuration Management Terminology Cont d 2. Software version number 3. Validation, Qualification or N/A 4. SOPs 5. System Owner name 6. Business Owner name Examples of CIs excluded from tracking: laptops/desktops/end user assets (cell phones, memory sticks, etc.) 49 50 IS Configuration Management Key Resources Responsibility CI Owner: = System Owner Configuration Auditor Plans and executes audit of configuration data Identifies unauthorized configuration information Undertakes audits of configuration information Note: configuration auditor will not audit his/her own changes IS Configuration Management Key Resources Responsibility Configuration Librarian: Guardian of master copies of CIs Updates CIs based on Update CIs requests submitted by Configuration Requester Creates new CIs based on Create CIs requests submitted by Configuration Requester Responsible for accuracy of configuration items and associated attributes Generates configuration reports Accepts and records the receipt of new/revised configurations 51 52 13
IS Configuration Management Key Resources Responsibility Configuration Manager (): Responsible for day to day quality and integrity of Configuration Management and process. Represents the site/functional area for the enterprise configuration management process Facilitates resolution of issues with items not complying with the process Supports the process for ensuring accuracy of DB entries throughout the lifecycle of the systems tracked as CIs in the DB Acts as point of contact for system owners Performs reviews at defined interval to ensure the logical depiction in the DB reflects the actual physical environment 53 IS Configuration Management Key Resources Responsibility Configuration Requestor: Submits change requests for updates to CIs discovered during an incident, problem resolution, or change Submits change requests for creation of new CIs 54 IS Config. Management Phases The High level tasks of Configuration Management are: Identification of configuration items to be included in the DB Control of data to ensure that it can only be changed by authorized individuals Status maintenance, which involves ensuring that current status of any CI is consistently recorded and kept updated Verification, through audits and reviews of the data to ensure that it is accurate Submit Config. Request IS Config Management Phases The Configuration Requester submits a request for new or updated configuration items or for a service including: Configuration item (CI) field attributes Configuration g model CI types, attributes, or relationships Reports of status or details on one or more CIs The Configuration Librarian verifies the request. If authorized/appropriate, the Configuration Librarian proceeds. If rejected, communicate the reason of rejection to the Configuration Requester 55 56 14
IS Config. Management Phases Report The Configuration Librarian CIs Retrieves the required information from the Configuration Management System Delivers the report to the Configuration Requester Identify CIs IS Config. Management Phases For new or revised CIs: The Configuration Librarian identifies CI types and assesses impact to S The Configuration Librarian identifies the business reason for the Change and submits a Change Request If the change request is approved, the CI type is created/enabled within the S (including attributes and relationships to be tracked) If the change request is rejected the change requestor is notified and the change is closed 57 58 IS Config. Management Phases Control The Configuration Librarian creates new CIs CIs, updates CI information or makes a CI obsolete based on information provided by the Configuration Requester. IS Config. Management Phases Review The Configuration Manager performs verifications CIs that check the existence of CIs and confirm that CIs are correctly recorded in the S: The Configuration Manager initiates a periodic verification to confirm integrity of the S for their functional area or site The Configuration Manager summarizes CI variances (i.e., what is in the S vs. what was expected to be in it) in the configuration report If Configuration Item errors are discovered, submit a change request to have the errors corrected. The Configuration Manager is responsible for any necessary corrective action 59 60 15
IS Config. Management Phases IS Config. Management Process Summary Review CIs The Configuration Manager submits the configuration verification reports to the Configuration requestor and other appropriate technical, business or compliance areas 61 62 IS Config. Management Tool Tips Some of the existing tools are very configurable Recommendations: Examine the capabilities and limitations of each tool Identify the Configuration Management needs of your organization by: Investigating and discussing process with configuration team Presenting the tools and capabilities and compare against your organization s needs 63 IS Config. Management Tool Tips Determine how your existing configuration management process will change to accommodate the chosen tool Identify and document the impact and ensure all key resources are on board An example of a Change and Configuration Management tool is IBM s Tivoli Change and Configuration Management Database. The low level component of CDB is called Tivoli Application Dependency Discovery Manager (TADDM in short) 64 16
Summary DB acts as a historical record for all changes to the environment. Once the CIs are logged in the DB, they cannot be modified without following the change management process Change and configuration work hand in hand to ensure that an accurate, controlled and complete picture of the system is available Ownership: ensure there is ownership of each part of the configuration management process for DB to be properly maintained Only authorized personnel can modify the DB depending on their role and access privileges Interactive Session Classify the Change Control type Discuss the potential impact Identify the Config Management Process to follow Identify the required resources to complete the Change Control to deployment Identify the steps required for completion and closure of the Change Control 65 66 Useful/Reference Links http://en.wikipedia.org/wiki/change_management http://www.fda.gov/downloads/drugs/guidanceco mplianceregulatoryinformation/guidances/u070 337.pdf http://www.knowledgetransfer.net/dictionary/itil/ en/change_record.htm http://www.knowledgetransfer.net/dictionary/itil/ en/change.htm http://technet.microsoft.com/en- us/library/cc750253.aspx http://www.stsc.hill.af.mil/crosstalk/1995/01/terms. asp 67 Useful/Reference Links Guidelines for Configuration Management: IEEE Std. 1042-19871987 IEEE Guide to Software Configuration Management MIL-HDBK-61A NFIGURATION MANAGEMENT GUIDANCE (7 February 2001) 68 17