Building a Tier 4 Data Center on a Tier 1 Budget

Transcription

1 Building a Tier 4 Data Center on a Tier 1 Budget A Walk Through of TIA-942 Peter P. Charland III, RCDD/NTS/WD, CET RTKL Associates Inc. March 27, 2008 BICSI South Central Region Meeting Denver, CO

2 TIA-942 History - Timeline 1960 s and 1970 s - Data centers limited to finance, telecommunications, and government sectors companies come together in NYC Forum for users and owners of data centers to share information Continues to develop into various groups and associations

3 TIA-942 History Moving Forward 1993 Site Uptime Network formed by The Uptime Institute, Inc. Network of member companies sharing information and collaborating to find solutions.

4 TIA-942 History Tiers Several systems developed to classify data centers with respect to architecture, electrical, mechanical, and telecommunications. Systems tended to be complex for nontechnical management. The Uptime Institute s four-tier system provides simple means to understand data center investments.

5 TIA-942 History A Beginning In April 2005, ANSI/TIA approves ANSI/TIA-942 Telecommunication Infrastructure Standard for Data Centers Based on four-tier system of The Uptime Institute

6 TIA Overview Spaces within a data center Architectural design elements of data center spaces MEP design elements Cabling and pathways

7 What s in your data center?

8 A Data Center - More than a computer room Computer room

9 A Data Center - More than a computer room Computer room Computer room TR

10 A Data Center - More than a computer room Computer room Computer room TR Operations center

11 A Data Center - More than a computer room Computer room Computer room TR Operations center Support staff offices

12 Data Center - More than a computer room Computer room Computer room TR Operations center Support staff offices Entrance facilities

13 A Data Center - More than a computer room Computer room Computer room TR Operations center Support staff offices Entrance facilities Loading docks, storage, burn-in rooms

14 A Data Center - More than a computer room Computer room Computer room TR Operations center Support staff offices Entrance facilities Loading docks, storage, burn-in rooms Secured yards generators & fuel

15 A Data Center - More than a computer room Computer room Computer room TR Operations center Support staff offices Entrance facilities Loading docks, storage, burn-in rooms Mechanical and electrical spaces

16 Data Center Spaces Entrance Rooms & Access Providers Main Distribution Area & edge devices Horizontal Distribution Areas Telecommunications Room Racks & Cabinets

17 Topologies within Data Center Typical Single Entrance Room One or more TRs Several Horizontal Distribution Areas

18 Topologies within Data Center Minimized Single Main Distribution Area Entrance facility Telecommunications room Horizontal distribution within data center

19 Topologies within Data Center Divergent Multiple Entrance Rooms One or more TRs Multiple Horizontal Distribution Areas with redundant backbone cabling

20 Architectural Elements Size Equipment: electrical and mechanical Ceiling heights Treatments Lighting Doors Floor loading

21 Data Center Cabling

22 Cabling Elements Service provider cables Backbone cables Horizontal cables Zone distribution

23 Cabling Elements ANSI/TIA/EIA-568-B.2 still applies!

24 Cabling Elements Data center can be a building Backbones within data center Centralized optical fiber cabling Horizontal cable reduced Redundancy

25 Generators UPS and batteries Bonding and grounding Power & environmental monitoring Emergency-poweroff (EPO) Power Elements

26 Mechanical Elements Cooling operations Environmental parameters Fire suppression

27 Redundancy - the start of Tiering Entrance pathways - OSP Access provider services Entrance facilities Main distribution areas (with care) Backbone cabling Horizontal cabling Mechanical and electrical systems

28 Concurrent Maintainability Systems can be maintained, upgraded, and tested without impacting normal operations.

29 Fault Tolerant System can handle any unplanned failure or planned outage without impacting normal operations including loads on support systems.

30 Tier 1 - N A starting point Need - Just enough

31 Architectural Plain box Tier 1 - N

32 Tier 1 - N Cabling/Equipment Single feeder pathway for access provider Single entrance room Single-path distribution No redundancy

33 Power Tier 1 - N Single power feeder UPS - single module or non-redundant modules Generator optional PDU s may be used

34 Mechanical Tier 1 - N Single cooling source Multiple cooling units to need immediate need Generator power, if available

35 Tier 1 - N Need - Just enough Failure of single unit or disruption for maintenance will disrupt operations

36 Tier 1

37 Tier 2 - N+1 The start of redundancy Need plus one

38 Architectural Tier 2 - N+1 Physical protection against disruption, natural and man-made Environmental conditioning improvements Loading dock

39 Tier 2 - N+1 Cabling/Equipment Two feeder pathways for access provider Single entrance room Redundant backbone cables within data center Critical equipment with redundant components

40 Power Tier 2 - N+1 Single power feeders with ATS UPS - redundant UPS modules (N+1) Generator sized as N Redundant PDU s, preferably from separate UPS systems Dual power cords for all equipment Grounding Infrastructure

41 Mechanical Tier 2 - N+1 Multiple units plus redundant unit Single piping pathways Generator power Air supplied via raised floor

42 Tier 2 - N+1 Need plus one Single pathway for systems Maintenance of critical power will disrupt operations Less susceptible to disruption from the failure of single unit

43 Tier 1 Tier 2

44 Tier 3-2N Doubling up Two complete systems for each needed: One active set of systems One standby set of systems

45 Architectural Tier 3-2N Specific protections against most disruption Physical separate of redundant systems Security checkpoints, access controls, and fences Site access improvements EMI protection 2 or more loading docks

46 Tier 3-2N Cabling/Equipment Two access providers or two points-ofpresence Redundant backbone cables & pathways within data center Hot, standby spare for all critical equipment

47 Power Tier 3-2N A mix of N+1 and 2N Dual utility feeders, one active & one inactive N+1 redundancy at all component levels Redundant power to mechanical systems Generator sized for equipment and mechanicals plus 1 Any component can be isolated Power monitoring

48 Mechanical Tier 3-2N Fully redundant cooling units Dual path piping system Redundant power Environmental monitoring

49 Tier 3-2N One active, one standby Redundant pathways, each able to carry full load Planned outages of systems without disruption to service Unplanned failure of one system without disruption to service Susceptible to disruption in failure/maintenance mode

50 Tier 1 Tier 2 Tier 3

51 Tier 4-2(N+1) Two of everything, all the time Two complete systems for each needed: Two active sets of systems Each set is N+1

52 Architectural Tier 4-2(N+1) Specific protections against all disruption Secured equipment yard

53 Tier 4-2(N+1) Cabling/Equipment Secondary distribution area for redundant routers & switches Secondary area isolated from main area for all systems Armored cables or conduit pathways Dual power supplies for electronic equipment

54 Power Tier 4-2(N+1) Utility feeder from two different substations Both feeders are active Primary and reserve UPS systems Generator sized for building load plus 1

55 Mechanical Tier 4-2(N+1) Alternative water storage for evaporative cooling systems

56 Tier 4-2(N+1) Two active, each able to carry full load Planned outages of systems without disruption to service Unplanned failures without disruption to service Redundant pathways, Very limited susceptibility to service disruptions

57 Tier 1 Tier 2 Tier 3 Tier 4

58 Tier Data Center Costs Annual Outage* Construction Cost 1 N 28.8 hrs $450 / sq. ft. 2 N hrs $600 / sq. ft. 3 2N 1.6 hrs $900 / sq. ft. 4 2(N+1) 0.8 hrs $1,100 / sq. ft. * Outage times based on actual field data by The Uptime Institute, Inc.

59 Growth, a planned cost? Right-sizing the initial design Not too big, not too small Total Cost of Ownership Realistic planning for growth

60 Word of caution There is no almost.

61 It is only a standard.

62 A different approach to tiering A compromise - Application tiering

63 A different approach to tiering A compromise - Application tiering Dual site data center and redundancy

64 A different approach to tiering A compromise - Application tiering Dual site data center and redundancy =?

65 Question Can you build a Tier 4 data center on a Tier 1 budget?

66 Well sort of? Answer

67 Concluding remarks TIA-942 provides an excellent basis of design, not a blueprint for design. Designers must educate owners. Designers must consider creative solutions = 4, but 1$ + 1$ 1$

68 Questions? Peter P. Charland III, RCDD/NTS/WD, CET RTKL Associates Inc.