Change, Configuration and Release Management Techniques for Data Centers Robert Cowham & David Cuthbertson Services Director Managing Director Square Mile Systems 1 Square Mile Systems
Who We Are UK based Cirencester, Glos, UK Develop toolsets and techniques for end to end systems and service mapping of complex infrastructure AssetGen Visio automation utilities Industry bodies & roles BCS-Config Mgmt Specialist Group - Committee BCS-ITIL Specialist Group - Chairman National Outsourcing Assoc. - Director LinkedIn Data Center Engineering / Ops Mgmt BICSI, ITSMF, Microsoft guest speaker Business Processes Departmental, Company Services End user, infrastructure, supplier Applications PC, server, mainframe, SOA Virtual Infrastructure PCs, Network, Servers, Storage, DBMS Hardware Infrastructure PCs, Network, Servers, UPS, Storage, Other Fixed Infrastructure (Cabling, Power, Cabinets, Rooms, Buildings) AssetGen Limited 2
Agenda Data Center Challenges The development of CCRM techniques Managing the complexity problem Projects vs Operations processes Overcoming the chaos of infrastructure documentation How to start well Summary Square Mile Systems 3
Common DC Challenges Technology stress on capacity Power, blade servers, connectivity, storage Increasing complexity Hardware, virtualisation, firewalls, mobile, cloud Increase in control systems Time, cost and risk of implementing changes Multiple supplier / team interfaces Hosting/colo, outsource, platform, service mgmt CCRM techniques can help meet these challenges Square Mile Systems 4
The Numbers Keep Growing Examples 2007 2012 Maximum power rating of a rack (kw) 2 24 Server maximum power rating (kw) 1 13 Physical servers in a rack 8 42 Virtual servers in a rack - 100s Physical network connections in a rack 24 128 Virtual network connections in a rack - 100s Power supplies per server 1-2 6+ Complexity increases at all levels! Square Mile Systems 5
The Development of CCRM Techniques Multiple software teams, often geographically separately working on a common system Where and when to store/compile code Versioning covering requirements through to delivery Dependencies on other systems hardware, software The end result Configuration system(s) for holding documents, code, etc Subversion, Git, Perforce, Rational, DMSs Workflow covering project delivery Interface into operational change management Square Mile Systems 6
And In the Data Center World Multiple platform teams, often geographically separately working on a common IT infrastructure Where and when to change / optimise infrastructure Projects covering requirements through to delivery Dependencies on other systems hardware, software The end result System(s) for holding documents and configurations Excel, Visio, CAD, Access, DCIM, CMDB Workflow covering project delivery Interface into operational change management Square Mile Systems 7
Change, Configuration & Release Software Project Requirement Design Develop Test Software Configuration Management Release Operations Management Change management Release management Data Center Project Requirement Design Hardware Build Data Center Configuration Management Software Configure Service Configuration Management system Square Mile Systems 8
Architectural Complexity Business Processes Departmental, Company Cloud Networks LAN/SAN Applications Servers Mid range Mainframe Data Centre Services End user, infrastructure, supplier Applications PC, server, mainframe, SOA Virtual Infrastructure PCs, Network, Servers, Storage, DBMS Hardware Infrastructure PCs, Network, Servers, UPS, Storage, Other Fixed Infrastructure (Cabling, Power, Cabinets, Rooms, Buildings)
The Service (ITIL) Approach Business Processes Departmental, Company Cloud Networks LAN/SAN Applications Servers Mid range Mainframe Data Centre Service Management CMDB/CMS Services End user, infrastructure, supplier Applications PC, server, mainframe, SOA Virtual Infrastructure PCs, Network, Servers, Storage, DBMS Hardware Infrastructure PCs, Network, Servers, UPS, Storage, Other Fixed Infrastructure (Cabling, Power, Cabinets, Rooms, Buildings)
Infrastructure Knowledge Plan Build Operate Risk Dispose Project and task Speed of communication Ease of distribution Flexible to meet task needs Limited training Examples Project documentation Equipment lists Visio/CAD diagrams Test results Manage and Coordinate Ease of use by many Structured for integration & reporting Support for multiple processes Wide scope the big picture! Examples Asset and Inventory management Business / service dependencies Monitoring of performance, status Risk and Recovery 11
Reducing Plan and Build Times Standard hardware builds, pre-deployed Blade systems, standard platforms Virtualisation Networks, Servers, Storage More efficient usage of resources Use Colo/Hosting Reduce dependence on own physical space Cloud infrastructures Reduce need for physical/software components, IaaS, SaaS There will always be boxes and wires somewhere! Square Mile Systems 12
Scoping & Planning Changes Reliance on local knowledge Site surveys Workshops Excel, Access lists Diagrams Visio, CAD Project documents Operations DR, service catalog Monitoring, DCIM Service desk CMDB Other auto-discovery Workflow Square Mile Systems 13
Real World Complexity
Virtualisation Doesn t Always Make It Easier!
UK_BIRM01_BLADE-05 UK_BIRM01_BLADE-04 UK_BIRM01_BLADE-03 UK_BIRM01_BLADE-02 UK_BIRM01_BLADE-01 BLADE-BIRM01.BLADE-SW2 BLADE-BIRM01.BLADE-SW1 UK_BIRM01_BLADE-12 UK_BIRM01_BLADE-10 UK_BIRM01_BLADE-09 Understanding Server Dependencies Claims Processing Accounts Receivable Accounts Payable Payroll NTU-TEST03 Payment Module Accounts Module Sage Payroll RTR-BHAM-01 Rack Position ACCOUNTS PAYROLL NTU-BHAM-01 Floor Plan PP01-02-01 PWR01-02-B PWR01-02-A BLADE_BIRM01 H/W Build SVRWIN001 Service impact UK_BIRMCC_PDU1 SW-BHAM- CORE1 SW-BHAM- CORE2 9 9 9 9 9 PWR01-03-A PWR01-04-A PWR01-05-A PWR01-06- A PWR01-07-A SW-BHAM-01 SW-BHAM-03 SW-BHAM-06 SW-BHAM-05 SW-BHAM-04 SW-BHAM-02 UK_BIRM_UX01 UK_BIRM_UX02 UK_BIRM_UX09 UK_BIRM_UX10 UK_BIRM_UX03 Power UK_BIRM_UX05 Supply UK_BIRM_UX07 UK_BIRM_UX04 UK_BIRM_UX06 UK_BIRM_UX08 PWR01-03-B PWR01-04- B PWR01-05- B PWR01-06- B PWR01-07- B Network Connections UK_BIRM_UX01 UK_BIRM_UX02 UK_BIRM_UX09 UK_BIRM_UX05 UK_BIRM_UX03 UK_BIRM_UX06 UK_BIRM_UX07 UK_BIRM_UX04 UK_BIRM_UX08 UK_BIRM_UX10 9 9 9 UK_BIRMCC_PDU2 9 9 AssetGen Limited 16
CRITICAL MAJOR MINOR USER COMPACT COMPACT PROLIANT PROLIANT PROLIANT I microsystems 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 PPM POWER SUPPLY PPM PROC PROC UID POWER SUPPLY FANS ONLINE SPARE MIRROR DIMMS 1 2 INTER LOCK OVER TEMP PCI RISER CAGE 440 1 2 3 4 5 6 7 8 Typical Documentation Chaos What should be updated with a rack, server or network change? 1. Asset/inventory list 2. Rack diagrams 3. Network patching records and diagrams 4. Switch port usage and capacity 5. Floor plan rack capacity 6. Power usage spreadsheet(s) 7. Storage / backup system documentation 8. Systems architecture documentation 9. DR lists and documents 10. Maintenance records 11. Billing and charging data 12. Project documentation with the as built details SERVERWIN0099 SERVERWIN0001 SERVERWIN00078 tsr 4554 UK_BIRM_UX01 SVR-BHAM-010301 PP01-03-01 HP ProLiant DL380 G5 www.assetgen.com
CRITICAL MAJOR MINOR USER COMPACT COMPACT PROLIANT PROLIANT PROLIANT I microsystems 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 PPM POWER SUPPLY PPM PROC PROC UID POWER SUPPLY FANS ONLINE SPARE MIRROR DIMMS 1 2 INTER LOCK OVER TEMP PCI RISER CAGE 440 1 2 3 4 5 6 7 8 Typical Documentation Chaos What should be updated with a rack, server or network change? 1. Asset/inventory list 2. Rack diagrams 3. Network patching records and diagrams 4. Switch port usage and capacity 5. Floor plan rack capacity 6. Power usage spreadsheet(s) 7. Storage / backup system documentation 8. Systems architecture documentation 9. DR lists and documents 10. Maintenance records 11. Billing and charging data 12. Project documentation with the as built details The larger the environment the more there is. www.assetgen.com SERVERWIN0099 SERVERWIN0001 SERVERWIN00078 tsr 4554 UK_BIRM_UX01 SVR-BHAM-010301 PP01-03-01 HP ProLiant DL380 G5
Challenges or Opportunities? Understanding dependencies Project delivery Operational coordination Aligning change processes Visibility Naming/cross referencing Improving responsiveness Risk management Team interfaces Getting a baseline Business Processes Departmental, Company Services End user, infrastructure, supplier Applications PC, server, mainframe, SOA Virtual Infrastructure PCs, Network, Servers, Storage, DBMS Hardware Infrastructure PCs, Network, Servers, UPS, Storage, Other Fixed Infrastructure (Cabling, Power, Cabinets, Rooms, Buildings) Square Mile Systems 19
Making CCRM easier in the DC Develop common processes and standards for changes Naming and labelling Easier project / operations handover Reservation of capacity and connectivity Focus on reducing the number of data sets used Spreadsheets, simple databases (e.g. MS Access) Use of specialist infrastructure / DCIM toolsets Diagrams Workflow tools Use a few sources to support multiple views 20
First Steps 1 2 3 4 5 1. Don t look back Buildings & Locations Fixed Infrastructure Racks, cabling, power Inventory Connectivity Application mapping 2. Reduce Data Sources Excel Visio Word Excel Visio Excel Visio Word Word Word Excel Excel Excel Word Visio Visio Excel Excel Trusted sources Excel Visio Word Visio Word Excel AssetGen Limited 21
Evaluate The Use of Specialist Toolsets Physical Changes Location and position Resource reservation for projects Physical connectivity Audits and manual data gathering Impacts and dependencies Adds, moves and changes Trusted sources Discovery systems Monitoring tools Service desk CMDB Spreadsheets risk, contracts Recovery plans Project handover db Excel Visio Word Updates Updates References AssetGen Infrastructure Management Outputs Typical Outputs Capacity management space, power, connectivity Change impact analysis impact, risk, auditing Excel Visio Visio Visio Inventory extracts Custom reporting Physical plans Floor and rack diagrams Topology diagrams Networks, power, storage System & Architecture Maps ITIL services, applications AssetGen Limited 22
Map Dependencies Efficiently Business Processes Departmental, Company Services End user, infrastructure, supplier AssetGen SysMap CMDB AssetGen Connect DCIM Applications PC, server, mainframe, SOA Virtual Infrastructure PCs, Network, Servers, Storage, DBMS Hardware Infrastructure PCs, Network, Servers, UPS, Storage, Other Fixed Infrastructure (Cabling, Power, Cabinets, Rooms, Buildings) SQL Server platform AssetGen Limited 23
Make Configurations More Visible Square Mile Systems 24
Summary Increasing infrastructure complexity Scale and scope local, end to end Understanding risk and dependencies Integrating project and operations change processes How to change Reduce data sets and maintenance effort Start with basics first naming, roles, etc Look at appropriate toolsets for configuration and workflow management Engage specialists to help improve skills and processes
Other Brighttalk webinars Further Information How to document campus infrastructures Making data center migration projects easier to manage Change impact analysis and CMDBs Square mile web site www.squaremilesystems.com Seminar presentations Free Visio add-ins for speeding up diagramming AssetGen website www.assetgen.com Videos, downloads, white papers BCS-Configuration Management Specialist Group www.bcs-cmsg.org.uk Square Mile Systems 26
Questions? Contact details: Robert Cowham, Square Mile Systems robert.cowham@squaremilesystems.com +44 7785 292 719 David Cuthbertson david.cuthbertson@squaremilesystems.com +44 7717 883 177 Square Mile Systems 27