EXHIBIT 71. Bloomsburg University. Technology Disaster Recovery/Continuity Plan

Transcription

1 EXHIBIT 71 Bloomsburg University

2 APPENDIX E-Disaster Recovery/Continuity Plan Self Study-2008 BLOOMSBURG UNIVERSITY Bloomsburg University July 2007

3 Table of Contents EXECUTIVE SUMMARY... 3 SECTION I: PURPOSE... 4 SECTION II: DEFINITION OF DISASTER... 4 SECTION III: POLICY... 4 SECTION IV: TEAM DEFINITIONS... 5 SECTION V: TEAM IDENTIFICATION... 6 SECTION VI: RESPONSIBILITIES OF THE DISASTER PREVENTION TEAM SECTION VII: RESPONSIBILITIES OF THE DISASTER ASSESSMENT TEAM SECTION VIII: RESPONSIBILITIES OF THE OPERATIONS RECOVERY TEAM SECTION IX: RESPONSIBILITIES OF THE SITE SUPPORT TEAM SECTION X: RESPONSIBILITIES OF THE SITE RECOVERY TEAM SECTION XI: DISTRIBUTION OF DISASTER RECOVERY/CONTINUITY PLAN SECTION XII: REVISIONS TO THE DISASTER RECOVERY/CONTINUITY PLAN SECTION XIII: TESTING OF THE DISASTER RECOVERY/CONTINUITY PLAN SECTION XIV: EMERGENCY RESPONSE PROCEDURES SECTION XV: OFFICE OF TECHNOLOGY ORGANIZATION CHART SECTION XVI: TECHNOLOGY STAFF TELEPHONE NUMBERS SECTION XVII: VENDOR CONTACT LIST SECTION XVIII: ASSUMPTIONS

4 Executive Summary The has initiated a to provide recovery and continuous technology services in the event of a disaster. The two critical centers of technology on campus reside in Ben Franklin Hall (BFH) housing the computer center and in Waller Administrative Building (WAB) housing the campus central telephone equipment and outside connections to the Internet and off campus telephone services. If either of these areas should experience a disaster, campus technology services would be severely affected. The plan concentrates efforts toward restoring mission critical systems that directly affect the overall campus community. Notably the mainframe Student Information System (SIS), the campus data and voice network, key serves that provide , web services and centralized file storage, off campus connections to the Internet, and local/long-distance telephone services are all considered mission critical. In the event of a technology disaster, the plan documents a systematic response by the staff, a method to provide temporary operations, and a strategy to permanently restore services to pre-disaster status. The SIS system resides on a Unisys mainframe in the BFH Computer Center. Although the new SAP Campus Management module will incorporate SIS, it is anticipated the mainframe will be required for at least another three to four years. SIS provides mission critical functions such as student class scheduling, grading, financial aid and student billing. A contract has been established with SunGard Inc. to furnish a Hot Site for the mainframe during a disaster. This simply means that if the university mainframe is damaged beyond immediate repair, a comparable mainframe residing in a secure location in Philadelphia will be used to support all mainframe systems. The university is currently paying a maintenance fee to ensure that the Hot Site is available and additional fees will be incurred if a disaster is declared. During Hot Site operations, a permanent mainframe replacement will be secured and installed on the campus or mainframe operations will be outsourced until SAP is fully implemented. Redundant core network equipment and mission critical servers are currently located in both BFH and WAB. In the event of a disaster in either location the redundant network equipment or servers in the other location assumes processing for the entire campus. Telephone services are also considered critical. Plans are in progress to provide a second source of Internet and local/long-distance telephone services terminating in BFH by the fall of By providing local redundancy, temporary services continue until permanent replacements are installed. The detailed Technology Disaster Recover/Continuity Plan provided in this document outlines the steps that will be taken by the to restore university technology to normal operations after a disaster has occurred. 3

5 Section I: Purpose To establish and define the procedures that will insure the continued operation and/or the orderly recovery of the operations within a reasonable time period after a "technology disaster" has occurred. Section II: Definition of Technology Disaster Any unplanned or spontaneous event or condition rendering the organization unable to use its information or communication infrastructure, causing significant loss of essential services. Section III: Policy It is the responsibility of the Assistant Vice-President of Technology to declare a technology disaster, to assemble the Damage Assessment Team and to notify the President and Vice-Presidents. The Damage Assessment Team (DAT) determines the extent of the disaster and notifies the appropriate vendors affected. The Operations Recover Team (ORT), Site Support Team (SST), and Site Recovery Team (SRT) are notified and the teams activated. 4

6 Section IV: Team Definitions A. Disaster Prevention Team (DPT) maintains an environment that avoids disasters from occurring and makes preparations to effectively react to a disaster if one does occur. B. Damage Assessment Team (DAT) performs an evaluation of the disaster, notifies appropriate administrators and vendors, and activates other teams. C. Operations Recovery Team (ORT) maintains temporary operation of technology functions until permanent replacements are in place. D. Site Support Team (SST) provides direct support to users, modifies desktop computers and software to accommodate configuration changes and keeps users informed of the latest status. E. Site Recovery Team (SRT) makes necessary site preparations, secures replacement hardware, installs and tests replacement hardware. 5

7 Section V: Team Identification A. Disaster Prevention Team (DPT) 1. Directory of Technology (TEAM LEADER) 2. Director Administrative Applications 3. Manager Telecommunication Services 4. Manager Network Services 5. Supervisor of Operations B. Damage Assessment Team (DAT) 1. Asst. VP Technology (TEAM LEADER) 2. Directory of Technology (ALTERNATE TEAM LEADER) 3. Director Administrative Applications 4. Manager Telecommunication Services 5. Manager Network Services 6. Manager Technology Support Services 7. Staff member reporting the problem 8. Facilities Management representative C. Operation Recovery Team (ORT) 1. Directory of Technology (TEAM LEADER) 2. Director Administrative Applications (ALTERNATE TEAM LEADER) 3. Manager Telecommunication Services 4. Manager Network Services 5. Appropriate Specialty Team based on disaster a. Mainframe ORT b. Network ORT c. Server ORT d. Telecommunications ORT e. Food Services ORT 6

8 D. Site Support Team (SST) 1. Asst. VP Technology (TEAM LEADER) 2. Manager Technology Support Services (ALTERNATE TEAM LEADER) 3. Director Administrative Applications 4. Helpdesk Coordinator 5. Director of Communications 6. Appropriate Specialty Team based on disaster a. Mainframe SST b. Network SST c. Server SST d. Telecommunications SST e. Food Services SST 7. Other support the staff as required E. Site Recovery Team (SRT) 1. Manager Telecommunication Services (TEAM LEADER) 2. Manager Network Services (ALTERNATE TEAM LEADER) 3. Director Administrative Applications 4. Director Purchasing 5. Facilities Management representative 6. Appropriate Specialty Team based on disaster a. Mainframe SRT b. Network SRT c. Server SRT d. Telecommunications SRT e. Food Services SRT 7

9 F. Specialty Teams 1. ORT a. Mainframe (1) Supervisor Operations (2) Senior Systems Analyst (3) Student Systems Analyst (4) Database Administrator (5) Web Programmer (6) Transaction Coordinator b. Network (1) Network Administrator (2) Network Technician c. Servers (1) Four Network Technicians d. Telecommunications (1) Two Telecommunications Technicians e. Food Services (1) Dining Systems Administrator (2) Supervisor Operations (3) Asst Dir Admin & Techn, Residence Life 2. SST a. Mainframe (1) Senior Systems Analyst (2) Student Systems Analyst b. Network (1) Manager Network Services (2) Network Administrator c. Servers (1) Manager Network Services (2) Two Network Technicians d. Telecommunications (1) Manager Telecommunications Services (1) Telecommunications Technician e. Food Services (1) Dining Systems Administrator (2) Supervisor Operations (3) Asst Dir Admin & Techn, Residence Life 8

10 3. SRT a. Mainframe (1) Supervisor Operations (2) Senior Systems Analyst (3) Student Systems Analyst (4) Database Administrator (5) Web Programmer (6) Transaction Coordinator b. Network (1) Network Administrator (2) Network Technician c. Servers (1) Four Network Technicians d. Telecommunications (1) Two Telecommunications Technicians e. Food Services (1) Dining Systems Administrator (2) Supervisor Operations (3) Asst Dir Admin & Techn, Residence Life 9

11 Section VI: Responsibilities of the Disaster Prevention Team A. Follow off-site backup procedures. B. Maintain and distribute the Disaster Recovery/Continuity Plan documents. C. Maintain current documentation of the systems. D. Maintain backups for data and documentation. 1. Tapes a. Mainframe weekly backup tapes are stored in the University Police office area of the University Book Store building. b. Mainframe daily data base backup tapes are stored in the fireproof vault in BFH. c. Server backup tapes are stored in WAB 2. Documentation a. Disaster Recovery/Continuity Plan. b. Selected System Manuals. c. Staff list and telephone numbers. d. Vendor list and telephone numbers. e. Other necessary documentation. E. Schedule Testing Periods for Disaster Recovery/Continuity Plan. F. Security 1. Passwords a. Ensure that transaction passwords are changed every 90 days. b. Ensure that server administrator s passwords are changed every 90 days. 2. Access to BFH computer center. a. Will be controlled by a lock system. b. Will be limited to the staff and when needed Facility Management staff and vendors. c. Exceptions will be made on an individual basis by the Supervisor of Operations. 10

12 Section VII: Responsibilities of the Disaster Assessment Team A. Assemble as soon as possible: 1. The disaster assessment team will meet in the office of the Asst. VP of Technology. 2. If Ben Franklin is hazardous or is not usable and damage is confined to that building, the disaster assessment team will meet in the second floor conference room in the Waller Administration Building. Contact University Police for night entrance. 3. If none of the campus facilities are usable, it is presumed that the disaster is of such proportions that prompt recovery of the operations is not of paramount importance. B. Assess the extent of the disaster. 1. Notify the University President, Vice Presidents and the State System. 2. DAT will put the Disaster Recovery/Continuity Plan into effect. 3. DAT will contact appropriate staff and vendors 4. If the Mainframe is affected, DAT will notify the Hot Site vendor. 5. DAT will alter staff schedules, if necessary, to render the system operational. 11

13 Section VIII: Responsibilities of the Operations Recovery Team A. If the Mainframe is affected. 1. Make provisions for office and storage space for ORT at the Hot Site. 2. Make provisions for housing the ORT at the Hot Site. 3. Obtain and secure backup tapes and documentation from off-site storage sites. 4. Obtain transportation and travel to the Hot Site. 5. Put a stop to all development and freeze system software at Hot Site. 6. Make the Hot Site operational to a point that it can accept the disaster site's exec load within a period of 24 hours. 7. Establish telecommunications link between Hot Site and Bloomsburg. 8. Make necessary modifications and adjustments to programs and equipment to ensure the proper operation and performance to the system. 9. Enable users to have access to the system. 10. Monitor system security. 11. Decide the priority of batch processing. 12. Carry out normal operations responsibilities. 13. Maintain, monitor, and document operations at Hot Site. B. If the Network or Servers are affected. 1. Ensure that the redundant routers/switched and servers are processing as expected. 2. For non-mission critical servers that do not have a redundant server, secure a replacement server, configure the replacement, load data and connect to the network. 3. For the Oracle data base server, secure server from Hot Site vendor, configure the replacement, load data and connect to the network. C. If the Telecommunications Center is affected. 1. Configure off campus communications through a second demark in BFH. 2. Configure telephone switch in BFH to process limited telephone connections on the campus. 3. Configure BFH network router/switch to connect to the Internet. 12

14 D. If the Food Services servers are affected. 1. Secure replacement computers. 2. Configure replacement computers with Food Services software. 3. Connect to the campus network. 13

15 Section IX: Responsibilities of the Site Support Team A. Provide User Support 1. Notify all departments that recovery operations are in progress. 2. Support users with any questions that arise. 3. Make modifications to desktop units to properly interface with any changes to the infrastructure configuration. 4. Modify programs, if needed, to ensure proper operation. 5. Update users on recovery status throughout recovery operation. 6. Perform other duties as assigned. B. Monitor and document operations at Bloomsburg. 14

16 Section X: Responsibilities of the Site Recovery Team A. Determine if disaster site can be restored at current location. B. Determine if anything can be salvaged. C. Determine if any operations will be outsourced. D. Secure and prepare a recovery site location that meets equipment requirements. E. Equipment installation. 1. Contact vendors. 2. Purchase new equipment or contract lease or outsource agreements. 3. Assist in equipment installation. 4. Re-establish communication with the university network. 5. Test and debug system. F. Inform users of normal operation. G. Resume standard operating procedures and return to disaster prevention procedures. 15

17 Section XI: Distribution of Disaster Recovery/Continuity Plan A. Upon approval, the Disaster Recovery/Continuity Plan will be distributed to the following: 1. Members of the Disaster Recovery Teams 2. Provost, Vice-President Academic Affairs 3. On-site safe 4. Off-site safe B. When the Disaster Recovery/Continuity Plan is revised, it will be distributed to those mentioned in "A" above. Distribution will only take place when revisions are approved. 16

18 Section XII: Revisions to the Disaster Recovery/Continuity Plan A. The Disaster Recovery/Continuity Plan will be reviewed by the disaster prevention team annually. B. Revisions occasioned by new names, addresses, telephone numbers, new positions, equipment changes, office locations and any other related items, will be made. C. The revised Disaster Recovery/Continuity Plan will be distributed upon approval. D. The old Disaster Recovery/Continuity Plan will be removed from all distribution points. E. The old Disaster Recovery/Continuity Plan will be filed as part of the Disaster Recovery Plan history. 17

19 Section XIII: Testing of the Disaster Recovery/Continuity Plan A. Testing of the Hot Site will take place during the summer months annually. B. Testing of redundant server, network and Internet services will take place during the summer months annually. C. Additional testing will occur if: 1. The Disaster Prevention Team deems it necessary after the plan is reviewed. 2. Substantial revision of the Disaster Recovery/Continuity Plan occurs. 3. Major problems are encountered during the test period. D. Hot Site testing will cover the following activities: 1. Obtainment of backup tapes from the off-site storage location. 2. Travel to Hot Site. 3. Load system and data to Hot Site mainframe. 4. Make a parallel run of at least one of the high priority applications. 5. Test high profile transactions. 6. Analysis of the test by the Disaster Recovery Teams and recommendations for improvements to the plan. 7. Make necessary modification to the Disaster Recovery/Continuity Plan 8. Distribute any modifications made to the Disaster Recovery/Continuity Plan. E. All testing procedures, analysis and recommendations should be documented and filed with the Disaster Recovery/Continuity Plan. 18

20 Section XIV: Emergency Response Procedures In the event a disaster should occur at Bloomsburg University the following steps should be taken: A. If the disaster is self-determined as being life threatening: 1. Staff - Pull the fire alarm 2. Staff - Evacuate the building 3. Staff - Notify University Police as soon as possible 4. Staff - Notify Asst. VP Technology 5. Asst. VP Technology - Contact the Disaster Assessment Team B. If the disaster is self-determined as being non-life threatening: 1. Staff - Notify Asst. VP Technology 2. Staff - Notify University Police (if necessary) 3. Asst. VP Technology - Contact the Disaster Assessment Team C. If the disaster occurs during a non-staff shift: 1. University Police - Contact the Asst. VP Technology 2. Asst. VP Technology - Contact the Disaster Assessment Team 19

21 Section XV: Organization Chart 20

22 Section XVI: Technology Staff Telephone Numbers Last First Title Department Ext. Cell Adams Daymon Applications Developer II Applications Dev. & Operations 4101 Barnes Bill Library Network Admin. Andruss Library Bieber Glenn Director Bobak Regina Instructional Tech. Specialist Technology Support Services Burns Robert Senior Systems Analyst Applications Dev. & Operations 4667 Celli David Manager Technology Support Services Chappell Melissa Student Systems Analyst Applications Dev. & Operations 4677 Chiado Nancy Box Office Manager Performing Arts Facility 2795 Cosper Jeff Info Technology Technician Technology Support Services Donahoe Frances Financial/HR Systems Analyst Applications Dev. & Operations 4652 Donahoe James Residence Hall Support Technology Support Services Friez Cara Media Specialist Instructional Media Services Telecommunication Furman Greg Technician Telecommunications Gessner Jim Director Applications Dev. & Operations Hemrick Lisa Administrative Assistant 4079 Hoffman Doug Network & Systems Admin. Networking Systems Hoover Terrin Chief Engineer Instructional Media Services Josuweit Sam Network Manager Networking Systems Karnes Todd Telecommunication Technician Telecommunications Computer Operations Supervisor Applications Dev. & Operations Klinger Mike Kozlek Vince COB IT Support Analyst Technology Support Services Lewellyn Rob Applications Developer II Applications Dev. & Operations 4114 Long Mike Network OS Administrator Networking Systems Loss Doug Network Analyst Networking Systems Melnick Gary Manager Telecommunications Mills Melanie Help Desk Coordinator Technology Support Services Mohr Wayne Asst. VP of Technology Ohl Judy Software & Web Support Applications Dev. & Operations 5327 Phillips Janice PC/Unix Systems Analyst Technology Support Services Presswood Randall Director Performing Arts Facility Raynes Mark Database Administrator Networking Systems 4111 Rodenhaver Carla Secretary/Student Payroll Instructional Media Services Rymell John Operations Center/Waller Applications Dev. & Operations Schmitz Kim PC Systems Analyst Technology Support Services Stahler Cheryl Desktop Support/Software Technology Support Services Stettler Jason IMPS Technician Technology Support Services Telecommunication Technician Telecommunications Taylor Jeff VonBlohn Kyle Lab/Class Tech Technology Support Services 4842 Wolfgang Chris Applications Developer II Applications Dev. & Operations

23 Help Desk (MCHS) Technology Support Services 4357 Student Desk (MCHS) Technology Support Services & 4231 Student Desk (MCHS) Instructional Media Services Student Desk (SRH) Networking Systems 5260 Telephone Repairs (WAB) Telecommunications 4406 TSS After Hours Support Technology Support Services TSS Student Technology Support Services Fax (Lisa's Office/Library) 3066 Fax (Help Desk Area/MCHS) Technology Support Services 3846 Fax (Applications Dev. SRH) Applications Dev. & Operations 4377 Fax (Telecomm. Area WAB) Telecommunications 4900 Fax (Performing Arts HAAS) Performing Arts Facility 2015 Fax (Instructional Media MCHS) Instructional Medial Services

24 Section XVII: Vendor Contact List Vendor Name Contact Phone Cell Phone Anixter Ed Mulhern Avaya Tim Kelly Greg Nask tech sup Apple Ted Evans ext Cambridge Computer Jessica Robertson computer.com Cisco Scott Porter Gateway Jeff Sparr ext Hewlett- Packard Kevin Drumheller Integra Bus Center Kevin Drumheller KMSystems Pomeroy Brian Baker SMA Darrel Andrews SPS Jay Straigis SunGard William Reiter Level3 PA-Team Help Desk Elizabeth Zeigler Unisys Kevin Laskowski Verizon Robert Long

25 Section XVIII: Assumptions A. A disaster will not affect both the BFH Computer Center and the WAB Telecommunication Center at the same time. B. At least a few members of the staff, as the result of happenstance, will survive; even in the worst case. C. All of the staff, including management, operations, technical support staff, programming staff, and clerical support will have access to the Disaster Recovery/Continuity Plan so they will be capable of filling incapacitated staff, if required. D. Bloomsburg University administration may be required to secure technical staff to carry out the duties outlined in the Disaster Recovery/Continuity Plan in the event members of the current staff have been incapacitated by either injury or death. E. A command room will be available on campus to conduct the business of recovery. F. Users will perform manual operations while the services are not available, and will store appropriate data to be entered on the system once operations have resumed. G. All attempts will be made to restore to a configuration that is fully equivalent to the existing one; but users may be required to make changes to their jobs to run a slightly different configuration. H. A disaster recovery vendor is under contract to provide a Hot Site for the Unisys mainframe. Access to the Hot Site will be via the Internet. I. Compatible mainframe equipment at the Hot Site can be made available in time to restore a critical operational systems environment within two days. J. System backup tapes in the off-site location are secure and properly restore on the Hot Site mainframe. 24

26 K. The BFH computer center or WAB communications center would ultimately be restored either at the original site or at some other suitable location on the campus. L. Compatible replacement equipment can be made available in time to restore the operations systems environment within six weeks. M. Redundant core network equipment and servers in WAB will provide adequate services to the campus for functions such as , university web services, and Internet access in the event the disaster occurs in BFH. N. A second demark is permanently established in BFH or a temporary demark can be made in BFH for Internet and telephony services in the event the disaster occurs in WAB. O. An alternative voice switches is available in BFH to support limited telephony in the event the disaster occurs in WAB. 25