ebook 01 deadly data centre sins. 7how to recognise them. What you will learn from this ebook Choosing a data centre for your most important assets is a risky business. To help you in your quest, we thought we d share with you the criteria that SSE use and the decisions they have to make when bringing a new data centre property to market. Part 1 of 3 As it s all about risk mitigation, we ve focussed on practical things we try to avoid the Sins and the design goals we use as benchmarks. That way, you get real, rather than hypothetical guidance.
THE Seven DEADLY SINS About the authors Commercial data centres are ultimately about two things. Firstly, they must provide an appropriate environment for your IT assets. Secondly, they must appropriately mitigate the many risks (or sins!) associated with using them. A lot of thought goes into choosing a location for a data centre and then a lot more goes into its design, build and operation. Suitability, reliability, security and accessibility are the key aims. 100% reliability and 100% security and 100% accessibility are noble goals but ultimately unaffordable for the majority. Compromise is necessary on your side and by the data centre operator. Compromise introduces risk. Understand the risks inherent in every data centre design then apply those to your own organisation s tolerance for risk and you re on the right track. Manek Dubash is an analyst and journalist with more than 25 years experience. Focused on business technology, he observes and comments on enterprise infrastructure issues for a range of industry-influential websites including The Register, ZDNet UK, Computer Weekly and CloudPro, and produces reports for research firm STL Partners. Study our Seven Deadly Sins and then formulate your position. Knowledge, as they say, is power. Who should read this book? This ebook (the first in a series of three focussed on data centres) provides impartial advice for CIOs, CTOs and IT management professionals who are keen to understand how best to compare and contrast commercial data centre facilities and then to assess how different supplier s approaches and different systems designs might affect any future decision to use them. Scott Gripton joined SSE in 2010 with a wealth of knowledge and experience within the data centre arena. Scott s expertise has been greatly utilised on our new data centre journey, helping us build our data centre estate throughout the UK. Today Scott is the data centre Product Manager ensuring our customers are receiving the most competitive and technically enhanced solutions to suit their needs. For more information call 0845 070 1997 or email enquiries@ssetelecoms.co.uk
Sin no 1: An inappropriate power supply Power interruption ranks among a data centre s greatest risks, so the operator s level of power assurance is key. Power must come from reliable sources, such as major sub-stations rather than smaller, non-diversely connected sites. For higher reliability, there will be two, rather than one, supplying sub-stations, each providing a diversely routed supply cable to the data centre. Within the data centre, risk mitigation demands supply path diversity and equipment redundancy. The Uptime Institute publishes Tier specifications that are widely used within the industry 1. Higher tiers demand increasing levels of fault tolerance at the expense of increasing costs. Tier II may satisfy those with higher tolerance to risk, offering a single, non-redundant path serving the IT systems. For the risk-averse, Tier III specifications insist on supply diversity throughout the data centre, providing much higher levels of availability. Estimates of the cost of data centre downtime range from low thousands to millions of pounds per hour, depending on the business type. Establishing your business s sensitivity to risk is the key to determining an appropriate Tier level. If financial performance is the key metric, simply multiplying the cost of downtime per hour by the total amount of downtime each tier permits may be a helpful guide. The reputational damage of downtime is harder to calculate. Generating lower risk In a Tier III data centre, interruption of power from the local utility is an expected operational condition and the site should be prepared. The generators are the data centre s primary power supply, and must automatically start and assume load when the utility s supply fails 2. 1 Uptime Institute: http://uptimeinstitute.com/publications 2 Ibid.
Tier III specifications at a glance A verified Tier III data centre guarantees a minimum 99.982% availability about 1.6 hours of downtime per year (Tier II guarantees 99.749% availability about 22.0 hours of downtime per year) > Minimum of N+1 concurrently maintainable power infrastructure every component in the power distribution chain: Multiple independent distribution paths Redundant supply components Diverse power to all equipment Multiple power generators N+1 on concurrently maintainable standby with fast start-up times and dual starter motors - UPS batteries to bridge the gap between supply failure and generators on-line > At least 12 hours of on-site generator fuel, with multiple fuel suppliers on contracted standby in case of extended generator usage
Sin no 2: Inadequate Cooling and Energy Efficiency Every single kilowatt of power consumed by IT equipment creates a kilowatt of heat. Modern data centres must be capable of neutralising megawatts of heat. Adequate cooling is critical, yet cooling systems are under more pressure than ever, as rack power densities increase. ASHRAE s 2008 guidelines recommend that equipment be kept between 18 to 27 C with a maximum humidity level of 60% 3. Failure to maintain the correct operating environment for IT equipment increases the risk of equipment failure. An appropriate level of supply and equipment redundancy will reduce downtime due to cooling system failures. At lower levels of redundancy, even cooling equipment maintenance can cause unscheduled downtime. Higher levels of redundancy to equipment and supply path will improve matters. Again, the Uptime Institute s Tier Specifications will offer guidance. Poor energy efficiency (stated as a Power Usage Effectiveness (PUE) ratio - the generally accepted measure of a data centre s energy efficiency 4 ) should also influence decision making, especially if you place a high value on Corporate Social Responsibility (CSR). If CSR is important, energy efficiency assessments should take into account not only the way in which energy is consumed within the data centre but also the re-usable energy generation credentials of the data centre s supplier. Cooling and efficiency at a glance > N+1 redundancy a Tier III facility requires N+1 redundancy to chillers and CRAC units > Multiple independent distribution paths - for both electricity and coolant paths 5 > Power backup for CRAC system and chillers cooling systems backed up by generator sets > CRAC unit location CRAC units located outside of data halls to isolate water supplies from IT areas > Temperature control from better data centre design and information systems: Cold aisle containment increases efficiency and reduces both overheating risks and costs Data from sensors at server inlets (rather than in the computer room generally) reduces risk by warning of local temperatures out of bounds 3 ASHRAE Thermal Guidelines TC 9.9, 2011 4 Green Grid, PUE definitions: http://www.thegreengrid.org/global/content/white-papers/ The-Green-Grid-Data-Center-Power-Efficiency-Metrics-PUE-and-DCiE 5 Uptime Institute: http://uptimeinstitute.com/publications
Sin no 3: Inadequate Communications Communications service availability is critical to data centre operations. If connectivity is compromised, the data centre becomes a liability as a consequence of potential lost revenue and reputational damage. As with power and cooling, equipment and route redundancy are key to communications service uptime. External connectivity > Carrier neutrality connections to multiple carriers at the site increase choice and competition > Diverse site entries mitigate the risk of service disruption > Interconnectedness multiple fibre-optic services to other data centres and tele-hotels increase service choice and improve service availability > Service choice you may want to connect from the data centre to your own sites make sure the carrier choice supports this Internal connectivity > Diverse meet-me rooms can improve availability and diverse fibre building entries should be terminated in separate secure carrier meet-me rooms in the data centre > Diverse network routes 6 availability improves if there are physically separate cables from racks to meet-me rooms > Service provider networking equipment in the data centre should be similarly diverse to improve availability Redundant path advantages Multiple network paths also provide an infrastructure for orchestrated data centres. Connection points between carriers networks inside the data centre meet-me rooms smooth traffic flows, allow for greater flexibility and choice for your customers, and can reduce local loop charges. 6 TIA 942 http://en.wikipedia.org/wiki/tia-942
Sin no 4: In the wrong neighbourhood Risk avoidance starts with data centre location. A data centre located at the end of a runway is arguably at higher risk than one that isn t. So risk increases with proximity to natural occurrences like seismic activity, floods and storms. Man-made risks to be avoided include dangerous facilities such as COMAH sites 7 those likely to house noxious or explosive substances and away from impact from major transit routes. Examples include airports, flight paths, major roads and railways, military installations, power stations and refineries 8. Location and visibility > Natural disaster avoidance sited in an area of low geological activity, away from flood-risk areas 9 > Man-made disaster avoidance at least ten miles from dangerous installations 7 and protected from busy roads and railways > Flight paths commercial or military aircraft flight paths should be no lower than 6,000ft above a data centre > Visibility the building should be unobtrusive and not obviously recognisable as a data centre > Boundaries the site boundaries should not present unauthorised access risks Keeping it low-key A facility s visibility and obviousness of purpose can increase risk. A data centre with a big logo on the wall presents a bigger target than an anonymous warehouse, for example. 7 COMAH sites, Health & Safety Executive: http://www.hse.gov.uk/comah/ 8 Seth Friedman, SANS Institute, 2003: http://www.sans.org/reading_room/whitepapers/physcial/building-ideal-web-hostingfacility-physical-security-perspective_270 9 See: Environment Agency: Flood Risk Map
Sin no 5: AN INSECURE FACILITY Effective risk mitigation must also include inhibiting unauthorised physical access. Visible characteristics of a secure data centre might include ram-proof gates, clamber-proof fencing, elevated landscaping, limited access points, security surveillance and an absence of windows. Site access should be granted to authorised and verified personnel only, with equipment room access requiring the highest level of security clearance, while the individual working on the reception desk might need the lowest. Controlling access > Site controls access controlled by physically preventing unauthorised vehicular and pedestrian access > Surveillance CCTV to all areas, security guards > Access prevention identity checks; secure zones; one entrance apart from a loading bay; no windows 10 ; equipment room walls prevent below-floor or above-ceiling access; ancillary equipment maintenance is performed outside the secured room 11 > Equipment security authorisation to be obtained before equipment is moved, stock areas are secured, all equipment movements are recorded Controlling identity > External access managed by an identity check > Equipment room access controlled by man-traps and two stage identity verification device 12 > Identity check employee backgrounds 10 The role of physical security If a person has physical access to a workstation or server, they control that system and the data stored on it....physical security is... what enables all other security measures to perform effectively and it is an absolute necessity in any comprehensive security plan 13. 10 Manek Dubash, ZDNet UK, 2010: http://www.zdnet.com/data centre-security-a-10-point-checklist-3040088570/ 11 Friedman, op. cit. 12 Ibid. 13 David Pollack, SANS Institute, 2004: http://www.sans.org/reading_room/whitepapers/physical/ implementing-robust-physical-security_1447
Sin no 6: POOR BUSINESS PRACTICES Data centre business management demands high levels of attention to detail, mirroring the high levels of environmental control required inside the equipment room. Among the many issues to consider are the operator s attitude to its customers compliance obligations, especially items such as corporate governance. A check on the business health of the data centre operator is also sensible and indicators include recognised credit ratings, revenue, profitability, reputation and size. Business quality > Quality management systems the operator complies with management standards such as ISO9001 > Security the operator complies with security standards such as ISO27001 and, where appropriate, PCI-DSS > Project management the operator applies project management principles such as PRINCE 2 > Corporate governance the operator complies with legislation such as Health & Safety and WEEE whilst acknowledging and providing support for customer compliance with wide ranging obligations like the Data Protection Act, The Companies Act, Sarbanes Oxley and MIFID How healthy is your operator? Among the many metrics to be examined during due diligence is financial stability. For example, you will want to know how well-managed the operator is, see details of their organisation and finances, and understand their appetite for commercial risk.
Sin no 7: Fire Protection Fire risk and the potential danger to life and property as well as down time resulting from such an event should be given very careful consideration when choosing a data centre facility. A complete fire protection strategy, together with a fully comprehensive fire protection system will help reduce the overall risk profile. Such a system will incorporate the key elements of prevention, detection and suppression and will aim to identify the presence of a fire, effectively communicate its existence and then contain and extinguish it, minimising risk to life, property and service downtime. Prevention > Construction - walls, floors, doors and ceilings to be made of suitable fire rated material in line with appropriate local standards and codes. ie, BS6266 14,NFPA 75 15 > Fire ratings - to comply with the relevant Tier requirements i.e. within UPS and battery rooms, a Tier III facility should offer >1 hour fire rating and at Tier IV should be >2 hour fire separation between other areas 16 Stopping fire in its tracks Access and egress points for cable entries and other services should be correctly fire stopped when transiting through different areas of the data centre, using appropriate devices such as intumescent collars. 14 BS6266:2011 Fire protection for electronic equipment installations. 15 NFPA 75 Standard for the Fire Protection of Information Technology. 16 ANSI / TIA 942 2005 Telecommunications Infrastructure Standard for Data Centres p.118 For more information call 0845 070 1997 or email enquiries@ssetelecoms.co.uk
Detection > Air sampling - smoke detection, sometimes referred to as a Very Early Smoke Detection Aparatus (VESDA) system, is usually described as a high powered photoelectric detector. Here, the air sampling systems use an advanced detection method with a very sensitive laser for rapid detection. > False alarms - ensure the detection system is tuned not to react to possible false alarms caused by other components in the data centre, as these can be costly in terms of service outage and material. > Key locations - intelligent spot type detectors are commonly placed below raised floors, on ceilings and within ceiling plenums. Placing detectors near the exhaust and the intake of CRAC units (computer room air conditioners), can also accelerate detection. Suppression > Gaseous agents various systems are available using either inert or halocarbon based gases. Clean agent systems extinguish fires by removing heat from them, whereas, inert gases essentially suffocate the fire by depriving it of oxygen. > System choice a wide range of gaseous brand and system types are available, such as FM200, Inergen, Argonite and Novec 1230. Check that the solution is current and uses allowable substances. > Water-based systems these can be used as an alternative or to supplement other suppression solutions, either full sprinkler or mist-based solutions which use much less water and can be less invasive. > Double knock these zoned systems provide added security against the damaging and costly effects of false alarms. Keeping our standards high Various EU and British Standards require certain levels of protection to be available in different environments. Within a data centre or computer room, aspirating smoke detection systems are often recommended, i.e. BS5839 17. 17 BS5839:2002 & 2013 Fire detection and fire alarm systems for buildings. For more information call 0845 070 1997 or email enquiries@ssetelecoms.co.uk
Conclusion Building and running a reliable data centre is a complex process with thousands of variables. However, beyond the basic ability to provide a cost effective home for a customer s IT equipment, the main business of a data centre operator is risk reduction and we hope that our Seven Deadly Sins have given you an insight into how you might go about identifying those risks, working out your organisation s appetite for risk and then taking your own position. If we ve helped you become a more informed buyer, then this ebook s job is done! Now you can recognise the main risks, the second ebook in our series of three should be a must-read. It will show you in an illustrated way how the 7 Deadly Sins can be mitigated, as well as providing suitable discussion topics for data centre providers and users. Have a think about your own organisation s attitude to risk, then we ll see you in ebook two. For more information call 0845 070 1997 or email enquiries@ssetelecoms.co.uk