Federal Data Center Consolidation Initiative

Transcription

1 Federal Data Center Consolidation Initiative 2011 Data Center Consolidation Plan & Progress Report October 7, 2011 Page 1

2 1 Introduction Agency Goals for Data Center Consolidation Implementing Shared Services/Multi-tenancy Agency Approach, Rationale and Timeline Agency Governance Framework for Data Center Consolidation Cost-benefit Analysis Risk Management and Mitigation Acquisition Management Communications Strategy Progress FDCCI Consolidation Progress Cost Savings Page 2

3 1 Introduction The U.S. Department of Labor s (DOL) mission is to foster, promote and develop the welfare of the wage earners, job seekers, and retirees of the United States, to improve working conditions, advance opportunities for profitable employment, and assure work-related benefits and rights. The ten (10) infrastructure agencies are: Bureau of Labor Statistics (BLS) Employee Benefits Security Administration (EBSA) Employment & Training Administration (ETA) Job Corps Mine Safety & Health Administration (MSHA) Office of Administrative Law Judges (OALJ) Office of the Assistant Secretary for Administration & Management (OASAM) Office of Inspector General (OIG) Occupational Safety & Health Administration (OSHA) Office of Workers Compensation Programs (OWCP) The U.S. Department of Labor s Data Center Consolidation Initiative (DCCI) is part of our Information Technology (IT) strategy to adopt a consistent approach to managing IT infrastructure, IT assets and data throughout the Department. As part of our IT Modernization and Green IT initiatives, as defined in the Strategic Sustainability Performance Plan (SSPP), data center consolidation is a DOL-wide initiative aimed at reducing the total number and overall expense of DOL data centers, associated energy costs and physical space allocations. All agencies within the Department are involved in this program, and solutions to meet the objectives of this program will be Departmental rather than agency specific. To achieve these results the scope of the DCCI will include both on-premise and off-premise facilities. These facilities will support automated information processing and data storage operation on behalf of the entire Department including development, test, and production workloads. For example, facilities include Tier I-IV data centers, server rooms, server closets, office areas, Cloud environments, processing centers, hosting and co-location services. DOL will focus on the following strategic objectives: Page 3

4 Reduce the number of DOL owned / leased facilities; Leverage Federal platforms; Reduce the number of physical servers through virtualization; Increase average rack utilization; Increase energy and space efficiency of its facilities, and; Acquire energy efficient servers through its technology refresh program. DCCI will consider a range of factors to include best practices, cost, security, privacy, interoperability, Cloud computing, data portability, data integrity, human resources and leveraging government-wide initiatives. This plan will address the following factors: Downward pressure on Federal agency spending: A key strategy of the DOL plan will be to migrate a number of production facilities now operated in the Washington, DC Metro area by separate agencies or bureaus into one (1) or two (2) centers. This will provide greater opportunity for hardware sharing and server virtualization to drive down IT infrastructure spending. Seek alternatives to bureau-specific data center footprints: Across our geographic presence within the continental USA, DOL will seek a significant reduction of current data center sites with a solution(s) that will serve all agency requirements for production and will enforce best data center practices to control space, energy, server hardware usage and licensing costs. Offset additional expenditures for consolidation: DOL recognizes the DCCI will require up-front investments in terms of project costs and higher direct costs for data center operations as production activities move from the current locations at mixed use office buildings to a dedicated minimum Tier I data center facility. These increased costs will be somewhat offset by more efficient use of space, energy and computing resources, but additional funds will be required. Use Department-wide solutions for facilities remaining in-house : DOL is including the DCCI as a part of its comprehensive IT Modernization initiative which will re-architect DOL IT around the concepts of a unified, shared information environment with common data sources. This new architecture will enable enterprise-wide solutions to be developed more quickly and efficiently. 2 Agency Goals for Data Center Consolidation DOL will follow its well-defined Governance process to establish an optimum number of centrally managed DOL centers to modernize the technology infrastructure. The primary objectives of this initiative are to: Page 4

5 Reduce the overall number of data centers used in support of DOL s ongoing mission and to remain congruent with the IT Modernization goals in supporting the evolving Strategic Plans. By reducing the total number of data centers within DOL, the total cost of ownership for DOL should be reduced while at the same time producing more efficient and effective operations practices; Reduce energy consumption by migrating data center services into a Tier I facility configured to support the Green IT Initiative; Implement improved levels of service and availability through the use of Information Technology Infrastructure Library (ITIL) based Service Delivery and Service Support processes that include 24x7 monitoring and response within the consolidated data center environment; Improve Disaster Recovery / Continuity of Operations and maximize performance by balancing business systems across geographically dispersed data centers. 3 Implementing Shared Services/Multi-tenancy DCCI is a component of the DOL IT Modernization initiative facilitating the department s goal of an agile, centrally managed IT infrastructure that supports the program and business requirements across multiple DOL agencies. The Department has already undertaken a number of projects which have provided shared services. The consolidation of data centers will be done in conjunction with these other initiatives. DOL has a proven track record and well-established processes for implementing shared services. In 2006, the Department began consolidation of wide-area network (WAN) services for nine (9) major agencies that had previously maintained separate WAN infrastructures. This project, called DOLNet, was successfully completed using a single MPLS network with standardized service and support processes managed centrally through the DOL Enterprise Services Office (ESO). In addition, this offered outsourced, managed firewall services that are being used for Internet gateway protection as well as internal security for a number of DOL agencies. The consolidation of the DOL WAN into a single architecture will simplify the telecommunications requirements for the DCCI. Each of these projects contributes significantly towards a more unified, single IT infrastructure at DOL. DOL has also actively engaged with other Federal agencies for Cyber Security Services using the Department of Justice (DOJ) Cyber Security Assessment and Management (CSAM) tool for managing Federal Information Security Management Act (FISMA) compliance tasks. Currently, DOL uses the National Finance Center for payroll services and is procuring a full set of outsourced services from other Federal agencies to meet broader Human Resources Line of Page 5

6 Business (HRLoB) requirements. In addition, DOL has recently submitted a request for proposals to the four (4) government Human Resources (HR) Shared Service Center (SSC) providers to provide a new HR IT processing capability utilizing the shared service center approach. We anticipate migrating to the HR SSC provider in FY12. DOL also serves as a Federal service center for safety and health reporting, making its Safety and Health Information Management System (SHIMS) available to other Federal agencies. DOL has worked to develop multimedia services capable of transmitting webcasts to all DOL locations using its local and wide-area network infrastructure, enabling Department-wide broadcasts of key events including agency all-hands meetings, training and speeches from the Secretary of Labor. Other significant systems that have been implemented include a Department-wide e-property system for IT inventory and asset management, a centrally managed system to meet the Federal Domain Name Services Secured (DNSSEC) requirements, and a centrally-managed identity and access management service to meet Federal Homeland Security Presidential Directive - 12 (HSPD-12) mandates. The Department-wide implementation of Managed Trusted Internet Protocol Services (MTIPS) is underway to consolidate and reduce DOL Internet connections as well as increase the security of external connections. In FY11, DOL initiated an consolidation project and issued a Request for Information (RFI) for a Software as a Service (SaaS) Cloud solution. These efforts, combined with the consolidation of data centers, will provide a more uniform and efficient computing and telecommunications infrastructure. Each of the initiatives described above demonstrates how DOL is shifting IT investments toward more efficient architecture and technology that seeks a single, centrally managed solution that can provide a unified, consistent design while meeting program or agency-specific requirements for production processing. Each initiative contributes towards a single, uniform, target infrastructure for a more efficient IT environment. Data Center Consolidation is a key project which when executed will make further infrastructure projects easier to execute, with fewer resources required and is a major contributor towards a significant Department IT strategic goal. 4 Agency Approach, Rationale and Timeline The Department will move forward as one (1) unit across ten (10) major infrastructure service providers (BLS, OASAM, OALJ, OWCP, ETA, OSHA, MSHA, EBSA, OIG, and Job Corp). The segmented technical infrastructure that exists today will evolve to a more department centric model that is able to leverage resources across the environment for maximum effectiveness. This approach is not new for DOL and is consistent with the implementation of other DOL shared services such as the DOLNet WAN, Webcasting Services, IPv6, DNSSEC, ITIL Universal Services Management Tool (USMT), and MTIPS initiative. This multi-year transformation will not only Page 6

7 meet the stated objectives of the Federal Data Center Consolidation Initiative (FDCCI) but will strategically position DOL to meet its mission well into the future. To meet the stated goals of FDDCI as well as the DOL mission, the DOL has developed a threepronged approach with each strategy addressing a specific segment of the technical environment. The prioritization of the strategies is based on internal analysis and has full executive support. The three (3) strategies are: a) Strategy I - Production Facilities in the Metropolitan District of Columbia Area - Establish one (1) or two (2) DOL data centers for hosting production facilities in the metro area to meet minimum Tier I requirements. Consolidating the metro area data centers into one (1) or two (2) facilities that utilize best practices (hot aisle / cold aisle design, floor tile layout, etc.), will result in more efficient cooling and increased energy savings. This includes reducing the energy and space footprint through virtualization and evaluating Cloud offerings for DOL services and applications. This consolidation will result in approximately fifty percent (50%) of the DOL hardware infrastructure being relocated based on current inventory counts. Strategy I began in Q1FY11 with significant time spent conducting analysis of hosting options, selection of a hosting facility, and planning the actual move. The consolidation will occur in FY12 and FY13 as a sequence of physical moves, grouped by agency, followed by the decommissioning of the vacated space. Agency driven server virtualization will continue to occur although a brief hold on changes will be instituted just prior and following the actual moves. b) Strategy II - Non-Metropolitan DC Production, Regional, and Secondary Facilities - Establish an optimum number of DOL data centers for hosting production facilities in areas outside the DC metro area that align with continuity of operation facilities and balanced workloads for optimum performance. Consolidating the non-metro area data centers (regional and secondary facilities) into facilities that utilize best practices (hot aisle / cold aisle design, floor tile layout, etc.), will result in more efficient cooling and increased energy savings. This approach also includes reducing the energy and space footprint through virtualization and evaluating Cloud services for DOL applications. The objective is to standardize the facilities to gain operational efficiencies while reducing facility maintenance costs. Strategy II began in the second half of FY11 with analysis and planning of optimal infrastructure configurations of the sites. c) Strategy III - State Offices, Small Offices, and Processing Centers - Reduce the number of server closets across multiple DOL offices and states offices by assessing alternate technologies such as virtual desktops. This strategy will include an analysis and reengineering of the delivery mechanisms for state office services. Strategy III is planned to begin in FY12 and continue into FY15. DOL intends to establish strategic positioning for processing centers when an end-of-contract is reached. Improvements in operational efficiencies, remote server management, and on-site staff support are expected. Since most of the server closets are adequately controlled using routine office-type air conditioning, savings from reduced cooling are not anticipated. The DOL plan considers all options to achieve consolidation including use of existing DOL facilities that meet Tier I or better requirements: Page 7

8 In-house implementation: No existing DOL Tier I data center has sufficient space to accommodate the consolidated footprint / inventory. Expansion and retrofit of existing facilities is cost prohibitive with significant power infrastructure limitations and lack of diverse power feeds. Outsourced implementation: DOL has conducted extensive market research to identify as potential options data centers that have existing contractual arrangements with GSA or other Federal agencies including the Networx contract, and those available commercially. The DOL strategy, while executing the DCCI, is to continue to seek efficiencies, improve performance, enhance security and reduce costs through a number of tactical activities. These tactical activities include: Continued implementation of server virtualization. This will continue throughout all three (3) phases. While some agencies are further along, it is expected that all agencies will have reached a high degree of virtualization by the end of FY15; Cloud computing for agency applications. DOL will continue to research and assess the viability of Cloud alternatives for applications running on traditional hardware. The speed at which DOL agencies transition to the Cloud will be based on the service provider s ability to meet or exceed current capabilities, reduce operational costs, address security, privacy and records management requirements and meet the future needs of the Department; Decommission unused servers and storage hardware. As a result of virtualization and technology refresh, older, power inefficient hardware will be replaced with green IT hardware. Agencies will be strongly encouraged to consider the green IT characteristics (high efficiency power supplies for example) of any hardware before acquisition. Reduced power consumption with lower energy costs is expected; Enable power-saving features on existing hardware. Agencies will be strongly encouraged to leverage existing technologies that support reduced power consumption during periods of low activity. An operational strategy for implementing these features across agencies will be developed and provided to the technical staff; Optimize efficiency of existing DOL Tier I+ facilities. The exiting DOL Tier I+ agencies will be assessed for operational efficiency, opportunities for further rack / space utilization, and implementation of best practices including hot aisle / cold aisle design, airflow management, blanking unused rack positions, floor tile placement, active / occupancy sensors and minimizing air leaks in raised floor systems. A unified solution across the Department with fewer data center facilities and associated carbon footprints will meet DOL IT Modernization requirements as well as the goals of the FDCCI. Our approach provides many benefits to both DOL and ultimately the Federal Government by reducing energy consumption, reducing operation and capital expenses, improving security, Page 8

9 guaranteeing processing performance, implementing IT service management best practices in a 24x7 service environment and increasing business value. Supporting analysis for Strategy I spans the remainder of CY11 and part of CY12 per the current DCCI schedule. This includes identifying and prioritizing opportunities, defining consolidation projects, performing cost benefit and requirements analysis, and determining deployment strategy. 5 Agency Governance Framework for Data Center Consolidation The current IT governance framework at DOL will provide the oversight and internal mechanisms to assure successful implementation of the DCCI. This project will be managed by the Office of the Chief Information Officer (OCIO) using a federated approach that includes participation from all ten (10) major infrastructure service providers within DOL and an established IT governance organization at the Department level. In addition, key DOL executives will provide senior leadership for the DCCI. The Deputy Secretary of Labor, the Assistant Secretary for Administration and Management and the Chief Information Officer (CIO) of DOL define policy and effect decision-making for this initiative and have already provided direction and leadership on the DCCI to agency heads and IT executives. The DCCI project is managed by the Deputy CIO, overseeing a dedicated project team that is responsible for day-to-day activities. Additionally, a technical focus group was chartered for the DCCI with representation from all major IT infrastructure agencies. DOL established the Enterprise Implementation Committee (EIC) in May 2011 to engage senior agency business leaders in the implementation of cross-agency, IT Modernization initiatives, including those related to IT infrastructure, common services, and customer service programs. EIC members were appointed by their agency leadership based on their business knowledge and broad perspective with representation from every agency and office as well as regional and field personnel. The EIC is designed to assure IT Modernization and customer service initiatives are implemented in a manner that addresses agency impacts and provides effective support to business mission and operations. As a result, the EIC engages in IT Modernization implementation strategies to facilitate progress on Department-wide initiatives. The DOL Office of the CIO supports an established IT investment management framework including the DOL System Development Life Cycle Management (SDLCM), Capital Planning and Investment Control, IT Security and Enterprise Architecture. Throughout each phase of the DCCI project life-cycle each group will provide guidance and direction to ensure effective IT investment management, financial stewardship, IT security compliance and adherence to the target architecture to meet DOL business requirements. DOL has Enterprise Change Management processes in place and is expanding the scope of this activity to include Page 9

10 department-wide business applications. This follows IT Service Management (ITSM) principles for best practices and will support the actual data center migration activities. The DCCI project will develop and maintain all appropriate artifacts for an IT Modernization initiative including the timely execution of Cost-benefit Analysis, analysis of alternatives, a risk management plan including risk register, an acquisition management plan and a project management plan complete with a work breakdown structure for cost and time reporting. The communication plan will reflect the existing IT governance structure at DOL and will be augmented as necessary for the DCCI project so that all stakeholders are informed and have timely information to plan their activities for the project. External requirements for the DCCI will be addressed, including budget requests for IT Modernization investment beginning in CY12 that reflect the work being undertaken along with a multi-year work breakdown structure or master schedule reflecting the deliverables, milestones and tasks for this project. This schedule will be modified to reflect additional OMB reporting requirements as they are announced. The DOL DCCI project team has already reviewed current agency-specific activities and plans to assure their convergence with the goals of the DCCI and the Department s overall strategy. This includes data center renovations and expansion, hardware upgrades and server virtualization activities. It is anticipated the project team will establish periodic reporting for measuring progress in individual agencies against the DOL performance goals for DCCI. Specific performance metrics will be developed and tracked to measure progress against DCCI goals and those in Section 9 of the Federal Strategic Sustainability Performance Plan titled Energy and resource efficiency of data centers. 5.1 Cost-benefit Analysis (CBA) As discussed in Section 4 Agency Approach, Rationale, and Timeline, the DOL DCCI schedule demonstrates the current early stages of the program. DOL is in the SDLCM Conceptual Planning phase and gathering data to formulate a Cost-benefit Analysis that will meet the requirements and guidelines of the DOL IT Investment Management Framework and OMB Circular A-94. A first iteration of the DOL CBA has been circulated to management for review. The DCCI team continues to collect data and is in the process of refining the draft copy. Throughout the lifecycle of the project the CBA document will be updated as more information becomes available. 5.2 Risk Management and Mitigation DOL will use an integrated risk management and mitigation approach based on the department s SDLCM. This process ensures that the risk identification, analysis, prioritization, tracking, control and mitigation activities are accomplished systematically throughout the project lifecycle. During the earliest stages of conceptualization and planning and throughout Page 10

11 the duration of the project, the risk management process is used to identify scope, technical, cost and schedule risks that could have a negative impact on the successful project delivery. The DOL DCCI risk management process uses a risk register to capture, track and prioritize the individual project risks based on the probability and impact of risk materialization. To assist in this process, the register contains twenty-eight (28) risk areas that are used to identify specific project related risks based on comparative historical classifications. The DCCI project risk register is flexible enough to track all project, system, component and data center risks. Once the risk management plan and accompanying risk register have been created, populated and reviewed by executive management, an on-going tracking process is used to monitor, control and mitigate the project risks. This process will include: Weekly analysis of project risks by the project team and management Monthly meetings with agency technical and functional subject matter experts Quarterly review of the risk register to ensure that the risk information is complete, upto-date and prioritized During this review, an evaluation is conducted to determine the effectiveness of the mitigation strategy and to ascertain whether any adjustments need to be made. The DOL risk management process allows specific persistent risks to be identified and analyzed before reaching a triggering event or threshold requiring allocation of project resources necessary for formulation of mitigation and control strategies and to manage the overall project risk and risk reserves to an acceptable level. Based on both an initial and subsequent analysis of DCCI, the project includes the following risks that have been added to the DCCI risk register: Unplanned contract modifications and change orders for data center services Inability to get out of facilities, lease and term agreements System failures after consolidation (i.e. power supplies, etc.) Insufficient existing facilities to meet DOL requirements, delaying the consolidation schedule Agency resistance to the project Lack of suitable Cloud offerings to meet DOL requirements Lack of FISMA C&A for Cloud offerings Inability to accurately measure all baseline costs for existing data center facilities results in a CBA that does not show significant cost savings from consolidation Failure to meet objectives of the SSPP initiative through data center consolidation Not meeting estimated server reductions through agency-specific tasks for virtualization Additional resources, particularly technical skill sets such as networking, system engineering, and storage architecture, will be needed for the DCCI project starting with the analysis phase (FY11) Page 11

12 Significant DOL budget reductions will force delay or abandoning the entire consolidation effort 5.3 Acquisition Management The Department of Labor utilizes four (4) IT investment management processes combining to form the robust DOL IT Investment Management Framework. The SDLCM, Capital Planning and Investment Control, Security Life Cycle and Enterprise Architecture comprise the processes. Acquisition development is led by the DOL DCCI Project Manager (PM), with the dedicated and sustained assistance of the Integrated Project Team and the guidance of the DOL Office of Procurement Services. The PM oversees and manages the acquisition planning effort starting at the conceptual planning phase of the SDLCM process and throughout the life of the investment. Additionally, the PM ensures the acquisition process follows capital planning and acquisition guidelines and, whenever possible, incorporates Performance-Based Acquisition. The Federal Acquisition Regulation Part 7 requirements concerning WBS elements have been incorporated into the overall project. Referencing Section 4, the DOL DCCI project team anticipates following a number of high-level programmatic phases (Strategies I-III), each associated with its own acquisition activities and deliverables. 5.4 Communications Strategy A key component of the DCCI plan will be a robust communication plan that will play a critical role throughout the lifecycle of the DCCI project. The communication plan will include the specific details on the frequency, audience, media, format, source and type of information that will encompass a formal communication process tailored to the wide-ranging audience involved in the DCCI project: DOL Executive Leadership including the Deputy Secretary and CIO Weekly Management Meetings with DOL Deputy CIO Periodic presentations to the DOL Enterprise Implementation Committee DOL Agency IT leadership and Stakeholders DOL Data Center Managers FDCCI senior leadership and OMB DOL Business Operations Center (BOC) staff DOL Facilities Management staff DOL Performance Officer for the DOL SSPP Solution Providers / Vendors - DOL has established a focus group with the technical and management staff members of all ten (10) DOL agencies that meet on a monthly basis. The meetings are forums for in-depth discussions of the DCCI project goals including the benefits of virtualization and consolidation. Additionally, DOL agencies have one-on-one meetings with the Department CIO providing a platform for agencies to provide input to the DOL strategy, voice concerns as well as identify agency specific requirements and risks. With the combination of working group and individual agency meetings, DOL hopes to foster an open and collaborative Page 12

13 environment where agency stakeholders have a sense of ownership in the planning and implementation of the DCCI project. Face-to-face meetings, teleconferencing, electronic and print media are used to brief stakeholders on the status of the DCCI project, reinforce the goals of the project and highlight key milestones. In addition, DOL has developed more comprehensive software based tools to gather, catalogue, and analyze agency data. The addition of a Configuration Management Database and shared workspace will facilitate the communication and collaboration between the DCCI project team and agency stakeholders during the planning and execution of the three (3) DOL strategies. 6 Progress 6.1 FDCCI Consolidation Progress DOL continues towards its stated FDCCI goals. As was noted in the April 8, 2011 OMB submission, two (2) facilities were successfully closed in the first quarter of These facilities were EFAST1, located in Lawrence, Kansas, and OWCP Lees Summit, located in Lees Summit, Missouri. In the June 30, 2011 submission, it was noted three (3) facilities will be closing in the January to March 2012 timeframe. The three (3) DOL facilities are the OASAM Regional Seattle facility, the OSHA Regional Seattle facility and the OWCP Seattle Energy facility. These facilities will be consolidated to one (1) new Regional Seattle site. Based on current plans and activities underway, the consolidation of the three (3) Seattle facilities is expected to be completed by end of calendar year Upon successful completion, the process model for the Seattle consolidation will be implemented for future Strategy II and III consolidations. DOL continues to review the risks and benefits for commercial and Federal options for Strategy I consolidated tiered data centers. The CBA and associated documents are continually refined with the DCCI project team reevaluating and informing senior DOL management of updates or changes in the viable options. The FDCCI, in conjunction with the IT Modernization initiative, has provided DOL senior management additional visibility into the IT infrastructure. This visibility will likely translate into commodity services transforming from a traditional delivery model to a Cloud based delivery model. Efforts are currently underway to identify potential Cloud opportunities and by utilizing the FDCCI derived data, develop cost and decision models. 6.2 Cost Savings Cost savings for the FDCCI are currently within a range based on the existing data center environment as compared to several hosting options included in the Cost-benefit Analysis. Savings associated with the FDCCI will be based in a large part on the selected hosting option, agencies continued migration to virtualized and Cloud environments and the ability to meet scheduled milestones. It is expected savings information on Strategy I will be available in Q2FY12. Page 13