Case study; A sustainable Green Data Centre, fact or fiction? Brian O Hora, BSc MBA MPM Networks & Infrastructure Manager Information Systems Services, Trinity College Dublin
200 Pearse Street Single UPS, no external bypass 20 low power racks Total IT/IS capacity 40kW Multiple contracts O Reilly building 4 racks No backup mains No fire suppression Split AC units Multiple tenants IT/IS capacity 10kW Background to need Other on campus hosting service providers and needs College Strategic Plan 2009/14 action item 6.10 make College a Green campus College high level risk register
Project trigger Procurement strategy Initially to include IT/IS Final excluding IT/IS Project risk management Contracting approach ICE standard contract D&B Procurement strategy and risk management Post project operational risk management approach D&B and maintenance for 4 years (alliance)
Pre project feasibility, proposal and initiation Challenges identified Single shared sub station Over capacity in 200PS DC External routes Indoor physical spaces Outdoor physical spaces Planning permission? Occupied working building Disruptive works window Site establishment Internal/external site access H & S
Project milestones Feb 2009 - May 2010 - Pre project feasibility, proposal and initiation June 2010 - Project planning Aug 31 st to March 31 st 2011 Procurement and selection phase, PQQ & ITT April 2011 - grant of planning permission pending appeal period & final grant May 13 th 2011 - final confirmation of funding availability and allocation July /Aug 2011 Detailed Design phase - prime contactor instructed to proceed with design including 4 th floor roof space Aug/Dec 2011 - Build phase, construction Jan/March 2012 Integration, testing, commissioning, training & familiarisation March 16 th 2011 Substantial completion & handover
Performance specification Green showcase minimum EU CoC compliance across load range Resilience and availability - Tier Level 3 desirable Scalability and modularity 25kW to 250kW in 25kW increments and two phases Phase 1-150kW IT load, all single upstream sized for phase 2-250kW IT load Phase 1 - min x16-20kw capable racks, plus passive wiring rack and adjacent active network equipment rack Energy monitoring; live PUE and detailed monitoring to PDU outlet level Supporting tools for IaaS; shared service & multi tenancy aligned, capacity management, fault management, performance management, accounting and security management
Procurement and Planning & Design phases Procurement restricted EU competition, PQQ and ITT and phases Q3 2010 PQQ, 17 responses to PQQ, 9 successful Q4 2010 ITT release and site visits scheduled Dec 2010 Jan 2011 Q1 Jan 2011 Planning application lodged for generator enclosure on roof and ITT closes, clarifications & selection starts Feb, preferred bidder appointed end Q1 Q2 2011 Contract negotiations, staged payments agreed, DCC planning award favourable, detailed design commences, focus on structural issue Q3 2011 Structural issue resolved, design completed, focus on long lead time items and disruptive works, rooms cleared & services relocated, site established
Construction phase Roof challenges Enabling works Surface condition Leak avoidance Access Heavy equipments - road closure notice PS opposite fire station
Construction phase Capacity issue in main DC - Additional 3 phase long autonomy UPS-B on general services - Installation of UPS-B also complicated due to capacity issue Impacts related to single shared sub station with main DC - Full day substation down time required to connect new facility -Failure of emergency backup system in main DC - reschedule - Impact on cost and schedule - Additional local and full functional testing requirements - Multiple failures of contractors generator on second connection attempt
Integration and commissioning phase Q1 2012 Formal witness testing and commissioning of all major systems Snagging Substantial completion agreement Security system integration & testing CCTV, door access control, intruder alarm Fire Alarm and VESDA system Familiarization training including (H & S fire detection gas suppression) End to end Management System & network integration
Management System
Management System
Management System
Management System
Primary outcomes Modular room - installed within the existing building to enclose the data centre, constructed of a powder coated metal skin internal & external and rockwool infill Raised floor, leak detection, no leak prevention x20 20kW IT racks plus one passive wiring plus adjacent network equipment rack, 25% uplift of rack space Overall power room power budget 150kW Dual 3 phase epdus per rack, all epdu outputs monitored ( 1,000) for power and kw hours Link from room passive wiring rack to main communications room 120 pair SM and 120 pair OM3 MM and 12 pair SM and MM and 24 CAT6A to each rack Dual high efficiency UPS, 20min autonomy per UPS at full phase 2 load, 10 year batteries 700kVA generator, 32 hr tank at full phase 2 load, dump tank at ground level Rear door heat exchangers, 18oC water temp in, 22-24oC return water temp, Standalone Dry Air Cooler, partial free cooling below 20-22oC, 100% free cooling at 15oC and below Operation of rear door heat exchangers and DAC
Operation of rear door heat exchangers
A sustainable Green Data Centre? Independently Verified load test results PUE measured (OAT 12-13.7 degrees) IT load 25kW 50kW 75kW 100kW 125kW 150kW PUE 1.36 1.17 1.15 1.16 1.13 1.18 Temp C 12 12.5 13 13.7 13 13.5 PUE below EU CoC over entire range and < 1.2 @ > 33% load, below 15oC OAT Broader definition of Green or sustainability includes carbon foot print of input power, facility PUE and efficiency of network/compute/storage Highly integrated UCS network/compute and storage selected as IaaS platform to populate facility, further encouraging virtualization and discouraging over provisioning Design driven by energy efficiency e.g. RDHC & no leak prevention Design options selection driven by energy efficiency - colour or racks, lighting Strategic Plan action item 6.10 make College a Green campus aligned, a Green showcase and shared services aligned, less control over input power sustainability A community shared services low latency HA facility offers opportunity to extend energy sustainability upstream to include low carbon foot print electrical supply
Brian O Hora, BSc MBA MPM Networks & Infrastructure Manager Information Systems Services, Trinity College Dublin Thank you