How to make auto-discovery work; a dynamic data-driven alternative to the periodic trawl of Discovery and Dependency Mapping (DDM) Look what people are saying, source Linkedin IT Service Management Forum: We ve recently delivered a Configuration Management System (CMS) platform based on the Interlink Software Service Configuration Manager (SCM) to baseline configuration items (CIs) populated by automated feeds from a number of data sources. We maintain service dependency models based on an imported catalogue and have successfully automated reporting and comparisons when configuration changes are detected. We now manage our corporate IT estate CI records via the CMS, these used to be hosted in a standalone Access database. I wanted to validate the approach and mention what s possible Copyright Interlink Software Services Ltd 2013 Page 1 of 10
Introduction Population of our corporate-wide Configuration Management System (CMS) has been proven in numerous large scale environments and is validated by ITIL v3 Auto-discovery tools, Service Desk products and their bundled Configuration Management Databases (CMDBs) present IT organisations with two major challenges: getting accurate data into the CMDB and getting meaningful information out. The Interlink Software Service Configuration Manager (SCM) has enabled many organisations to successfully address both challenges, facilitating the creation of repositories of clean, verified data relating to configuration items (CIs) and their interrelationships. Building on this solid foundation the Interlink Software solution makes it simple to analyse and present that data in ways that make sense to both IT operations staff and business process owners. We really liked Interlink s datadriven approach over competitors. Their ability to populate service models on a data subscription basis. Mark Blackman, British Airways How to make auto-discovery work; a dynamic data-driven alternative to the periodic trawl of Discovery and Dependency Mapping (DDM) Many of our customers have deployed leading DDM tools in an attempt to create and update a single-source-of-truth a.k.a. the CMS or CMDB in their large scale environments with varying degrees of success. Unfortunately, most have found this to be an expensive and difficult task. Key issues that they have encountered include the problem whereby a "trawl" of the network, devices, applications, etc. took a considerable amount of time and that the discovery process often needed to be run in a staggered way by targeting a subnet, domain or location one at a time, once or twice a day. Other customers reported scans taking several days to complete by which time some of the data collected was out of date and therefore many changes to the environment (especially those in virtual and cloud environments) were completely missed by the auto-discovery. SCM does things differently to DDM, using a dynamic data-driven approach. SCM connects to data sources for each domain (application, network, etc.) and creates a baseline of the configuration. It then monitors the environment for updates in near real-time (or as regularly as is practical). SCM can even integrate with change management tools to verify that any changes detected comply with your change policy, saving valuable time and money and eliminating the risk associated with poorly managed change. Most organisations are adverse to auto-discovery scans taking place during business hours, especially for critical services and consequently scans are usually scheduled overnight when there is little or no load on the applications scanned. This often leads to the single-source-of-truth being up to 24 hours out of date. The SCM approach is not nearly as intrusive or risky as it leverages existing data sources and doesn't require intensive network and application scans. In addition SCM takes advantage of any knowledge or configuration data held within applications and systems management tools to enable dependency mapping to occur rather than trying to automatically "learn" from watching data packets on the network or sockets opening and closing. SCM sees how the application is actually configured and what critical components are really being monitored without the challenges of firewalls, encryption, logging onto the application as an automated user, etc. Our 3 Simple Steps to discovering and mapping IT configuration and dependencies: 1. Integration and Collection from existing data sources (Real-time, On-demand or Scheduled). 2. Data Analysis (Cleanse, Normalise, Merge, Enrich, Baseline and Detect Changes) 3. Visualisation and Exception Reporting Copyright Interlink Software Services Ltd 2013 Page 2 of 10
Introducing Service Configuration Manager (SCM) Service Configuration Manager (SCM) delivers a service aware, real-time, change managed Configuration Management System (CMS). Supporting integration with 3rd party products via a combination of real-time integrations and data source federation, SCM is designed to protect our customers existing investment in IT products (e.g. leveraging 3rd party change, inventory, asset, and systems management tools). SCM is shipped with inbuilt service modelling, dependency mapping and risk-of-change analysis tools to help avoid outages to business services and reduce the cost of the introduction of new business services and more of the same changes. SCM is the CMS platform of choice for some of the largest organisations in the world as an enabler for: Compliance: automated audit trail of IT infrastructure changes Compliance: verify disaster recovery systems are in-line with production Visualise infrastructure-to-business dependencies Leverage existing data sources via import and federation (linking) capabilities Model the impact of IT changes on their business services BEFORE the changes are made Identify and re-schedule conflicting changes Deliver a forward schedule of change by business service or technology With regards to audit and compliance, the Interlink Software solution maps to your organisations dynamic network operations or data centre discovering configuration changes automatically and cross references them with your change policy highlighting compliance and security issues. Identify unauthorised changes to their IT infrastructure Automate the management of the lifecycle of IT assets Colin Griffiths, Co-founder, Interlink Software Centralise operational documentation with context sensitive launching Perform configuration comparisons across CIs, Data Centres and Business Services Track infrastructure and business services via an automated Service Catalogue Detect drift and minimise outages related to the provision of virtual or cloud based services Pre-packaged integrations for 3rd party products such as ServiceNow, BMC Atrium, VMware vcenter, etc are available Copyright Interlink Software Services Ltd 2013 Page 3 of 10
Pragmatic, Scalable Deployment Model; Top Down, Service-at-a-Time Interlink Software developed the Service Configuration Manager (SCM) product from the ground up to address the need to maintain up-to-date service dependency mapping linking logical and physical IT elements commonly known as Configuration Items (CIs) to the business functions and overall business services that they support. SCM addresses this requirement by offering a flexible top-down, service-at-a-time deployment methodology in contrast to the boil-the-ocean approach to configuration management that often requires the entire IT estate to be trawled by discovery tools and defined in a monolithic Configuration Management Database (CMDB) before moving on to the next steps in the service management value chain such as Service Level Agreement (SLA) tracking and Business Service Management (BSM). SCM integrates configuration and dependency data from the entire IT stack; hardware, operating systems, virtualization layer, cloud services, messaging layer, middleware, databases, applications, storage, web servers, domain specific auto-discovery tools, systems management tools and agents. Once SCM has been deployed and a configuration baseline has been established it automatically catalogues IT and business services keeping these dependency maps to date by automatically detecting and validating ongoing changes to the environment. SCM's Service Configuration Data Hub component offers data cleansing, merging and reconciliation functionality. Reconciliation presidency rules can be applied to determine which source of data (from the entire IT stack) is the most trusted in each case. Figure 1. Screenshot of dependency mapping and simulation tool. Copyright Interlink Software Services Ltd 2013 Page 4 of 10
Figure 2. SCM showing virtual images to physical hardware mapping. Figure 3. SCM showing numerous different versions of a service configuration accumulated over time. Copyright Interlink Software Services Ltd 2013 Page 5 of 10
How SCM Builds and Maintains Dependency Maps Relationships between CIs can be fixed or based upon subscription rules. Subscription rules allow CIs to be automatically grouped or linked together. A simple example could be that a particular business service is impacted by all Red Hat Enterprise Linux Servers running a specific process across any of the production data centres. By using a subscription rule that selects all eligible Red Hat Enterprise Linux Servers running the specific process across any of the production data centres we avoid hard coding specific CIs in SCM and just instantiate the list automatically keeping it up-todate in near real-time. Subscription rules can be supplemented with impact calculation rules that work on the same principal and automatically apply dependency impact weightings. Figure 4. Pre-packaged integrations for a variety of data sources and 3rd party products such as ServiceNow and VMware vcenter are available. Copyright Interlink Software Services Ltd 2013 Page 6 of 10
How SCM Automatically Detects and Validates Changes The history or delta between versions of service configurations maintained by the SCM product is created automatically. Every change to the underlying configuration data detected by SCM causes an increment to the version of the service configuration. Detected changes can be compared against pre-defined policies or look-ups against a 3rd party change management system can be performed automatically reconciling detected changes with corresponding change records. This approach ensures that only authorised configuration changes are propagated to the business service dependency maps, business service management dashboards and SLA reports. Unauthorised changes can be escalated via visual cues in the Interlink Software dashboards, automated incident creation or notification (SMS, paging, email, etc). Figure 5. The SCM highlight changes facility changed CI attributes are shown in yellow, missing values in red. Copyright Interlink Software Services Ltd 2013 Page 7 of 10
SCM creates date and time stamped audit logs for changes to both data and system configuration. A sample automatically created CI change audit log follows: 30/04/2013 14:30:13 : 1.000003 Initial or first recorded version ---------------------------------------- 30/04/2013 14:41:31 : 1.000004 Merged-Discovery-Hosts > Merged_Discovery > CT_Hosts Property SERIALNO changed from 'NA' to '1526GFH34533' Property LOCATION changed from 'London' to Sheffield: Hall 1: Rack 2 Property OS changed from 'MS Windows' to 'Microsoft Windows 2008 Server SP 2' Property PROCESSOR changed from 'NA' to 'Intel Pentium III 1600 MHz' Property MEMORY changed from '512' to '1024' Property CPUS changed from '1' to '2' Property HOSTTYPE changed from 'Windows Server' to 'Windows Server 2008' UNAUTHORISED CHANGE ---------------------------------------- Copyright Interlink Software Services Ltd 2013 Page 8 of 10
Automated Audit for Change and Compliance According to Gartner, more than 40% of all mission-critical IT service outages are due to people and process errors, with a significant number of those due to a lack of coordination between change, release and configuration management. SCM is a key enabler of the agile data centre providing an automated audit of the configuration of business services and their composite IT highlighting changes that do not have corresponding change records or do not meet change policy. Large financial services organisations use SCM to avoid the need for costly manual effort gathering data to meet financial and regulatory audits. SCM has been demonstrated to automate and enable the visualisation of comparisons between IT configurations across data centres, list software patch levels across the entire company or for a specific business service, identify recent changes for a specific CI or business service, and a host of other permutations. Figure 6. SCM provides the capability to run availability based risk of change simulations, adding or removing components and simulating failures to see the impact on overall service availability before deploying a change in production. Copyright Interlink Software Services Ltd 2013 Page 9 of 10
Summary of Key Features and Benefits Service Configuration Manager (SCM) is the CMS platform of choice for some of the largest organisations in the world, as an enabler for: Compliance: automated audit trail of IT infrastructure changes Compliance: verify disaster recovery systems are in-line with production Visualise infrastructure-to-business dependencies Leverage existing data sources via import and federation (linking) capabilities Model the impact of IT changes on their business services BEFORE the changes are made Identify and re-schedule conflicting changes Deliver a forward schedule of change by business service or technology Identify unauthorised changes to their IT infrastructure Automate the management of the lifecycle of IT assets Centralise operational documentation with context sensitive launching Perform configuration comparisons across CIs, Data Centres and Business Services Track infrastructure and business services via an automated Service Catalogue Detect drift and minimise outages related to the provision of virtual or cloud based services Pre-packaged integrations for 3rd party products such as ServiceNow, BMC Atrium, VMware vcenter, etc are available For further information please see: https://www.interlinksoftware.com/configuration-management/ All names of companies, products and services are trademarks or registered trademarks of their respective owners. Copyright Interlink Software Services Ltd 2013 Page 10 of 10