ADAMS STATE COLLEGE COMPUTING SERVICES DEPARTMENT INFORMATION TECHNOLOGY DISASTER RECOVERY PLAN Prepared Date: 30 June 2006 Revision Date: Rev 1: 2 October 2006
ASC IT Disaster Recovery Plan Document Change Page Revision Prepared Date Approved Date Reason for Change Original 30 June 2006 Draft IT Disaster Recovery Plan for Cabinet Review Revision 1 2 October 2006 Final IT DR Plan for Cabinet Approval 2
Adams State College Information Technology (IT) Disaster Recovery Plan Table of Contents Section 1.0 Introduction.....4 2.0 Objectives....4 3.0 Scope.....4 4.0 Assumptions 5 5.0 Definitions...6 6.0 General Disaster Response and Recovery Guidelines....6 7.0 IT Risk Assessment 7.1 Level 1 Computing Services Building and Central Computer Room..7 7.2 Level 2 ASC Telecommunications 12 7.3 Level 2 911 Emergency Services...16 7.4 Level 2 Network Services..18 7.5 Level 2 Cable Plant... 22 7.6 Level 3 File and Print Services..23 7.7 Level 3 Enterprise Resource Planning Services (Banner).....25 7.8 Level 3 Email Services..29 7.9 Level 3 Web Services 35 7.10 Level 3 Campus Card Services....37 7.11 Level 3 Residential Network Computing Services (Resnet)...39 7.12 Level 3 Academic Instructional Technology Classrooms...40 7.13 Level 3 Student Computer Laboratory Services...42 8.0 Maintenance of the IT Disaster Recovery Plan.44 9.0 Attachments Attachment A ASC Computing Services Contact List Attachment B ASC Campus Contact List Attachment C Vendor Contact Information 3
Attachment D Equipment and Software Inventory Attachment E Disaster Recovery Action Items & Improvement Recommendations 1.0 INTRODUCTION Adams State College (ASC) is a four year, Colorado, public college which, by statue, offers undergraduate Liberal Arts and Sciences, Teacher Preparation, and Business degree programs; a limited number of master s level programs; and two year transfer programs with a community college role and mission. Over time, Information Technology (IT) services have become critical to performing the educational mission of the college. As a result of this ever-increasing reliance on technology, IT services require a comprehensive Disaster Recovery Plan to assure these services can be re-established quickly and completely in event of a disaster. This Plan summarizes the results of a comprehensive risk analysis conducted for all IT services; it provides general steps that will be taken in event of a disaster to restore IT functions; and it provides recommendations for hardening of the IT infrastructure that require executive-level management approval and additional funding to implement. 2.0 OBJECTIVES The primary objective of this Disaster Recovery Plan is to help ensure college business continuity by providing the ability to successfully recover computer services in the event of a disaster. Specific goals of this plan relative to an emergency include: Detailing a general course of action to follow in the event of a disaster, Minimizing confusion, errors, and expense to the college, and Implementing a quick and complete recovery of services. Secondary objectives of this Plan are: Reducing risks of loss of services, Providing ongoing protection of institutional assets, and Ensuring the continued viability of this plan. 3.0 SCOPE This plan will only address the recovery of systems under the direct control of the Computing Services Department that are considered critical for business continuity. Also, given the uncertain impact of a given incident or disaster, it is not the intent of this document to provide specific recovery instructions for every system. Rather, this 4
document will outline a general recovery process which will lead to development of specific responses to any given incident or disaster. Three levels of risk, based on severity to campus operations, have been identified. A Level 1 risk is associated with the Computer Services building and central computer room which house the campus servers, router, PBX and serves as the primary hub for campus electronic and voice communications and connectivity. A Level 2 risk is associated with the campus network infrastructure and the telephone public exchange (PBX). The final risk level, Level 3, is associated with risks specific to unique applications or functionality. Though risk at all levels must be addressed for disaster recovery purposes, Level 1 risks will be given increased priority over other levels.. The same holds true for Level 2 versus Level 3 risks. The following major service areas are addressed in this plan: Level 1 - Computing Services Building & Central Computer Room Level 2 - Central Telephone Services Level 2-911 Emergency Services Level 2 - Network Infrastructure and Services Level 2 - Cable Plant Level 3 - File & Print Services Level 3 - ERP Services (Banner) Level 3 - Email Services Level 3 - Web Services Level 3 - Campus Card Services (1card) Level 3 - Student Residential Network Computing Services (RESNET) Level 3 - Technology Enhanced Classroom Support Level 3 - Student Computer Lab Services NOTE: All systems that are both necessary for the daily operations of ASC and the responsibility of the Computing Services Department are maintained under service contracts with the appropriate equipment vendors. 4.0 ASSUMPTIONS This disaster recovery plan is based on the following assumptions: The safety of students, staff, and faculty is of paramount; the safeguard of such will supersede concerns specific to hardware, software, and other recovery needs. Once an incident covered by this plan has been declared a disaster, the appropriate priority will be given to the recovery effort and the resources and support required as outlined in this IT Disaster Recovery Plan will be made available. Depending on the severity of the disaster, other departments/divisions on campus may be required to modify their operations to accommodate changes in system performance, computer availability and physical location until a full recovery has 5
been completed. The ASC Cabinet will encourage campus departments to have contingency or business continuity plans for their operations, which include operating without IT systems for an extended period of time. 5.0 DEFINITIONS The following definitions pertain to their use in this IT Disaster Recovery Plan: Backup/Recovery Tapes: Copies of all software and data located on the central servers, which are used to return the servers to a state of readiness and operation that existed shortly prior to the incident/disaster. Disaster: A significant or unusual incident that has long-term implications to business continuity and the ongoing operations of ASC. Incident: An event which impacts a specific IT service or server. Level 1 Risk: Risk associated with the most critical IT services/capabilities, based upon impact to the campus if the service or capability were lost. Level 2 Risk: Risk associated with critical IT services/capabilities, based upon impact to the campus if the service or capability were lost. Level 3 Risk: Risk associated with the loss of selected applications/functionality..6.0 GENERAL DISASTER RESPONSE & RECOVERY GUIDELINES 1. In the event of a disaster, the CIO will notify the three primary IT Disaster Recovery Teams; network, administrative and telecommunications (see Appendix A, IT Disaster Recovery Teams). 2. Appropriate steps will be taken to safeguard personnel and minimize damage to any related equipment and/or software. 3. A damage assessment will be conducted by each team and recommendations made to the CIO for recovery of impacted services. 4. Individuals required to assist in recovery of these services will be identified. The CIO will communicate this need to the VP for Finance & Administration (see Appendix B, ASC Campus Contact List). 5. The campus will be informed as to IT system degradation and restrictions on IT usage and/or availability. 6. The CIO will develop an overall IT recovery plan and schedule, focusing on highest priorities of the campus infrastructure, first, as defined by the Cabinet. 6
7. Necessary software and hardware replacement will be coordinated with vendors and the ASC Purchasing Office, as required (see Appendix C for vendor contact information and Appendix D for a list of critical equipment). 8. The CIO will oversee the recovery of campus IT services based on established priorities. 9. The CIO will ensure that IT recovery efforts are properly coordinated with other campus recovery efforts. 10. The CIO will communicate recovery status updates to the ASC Cabinet and campus at large. 11. The CIO will verify restoration of the IT infrastructure to pre-disaster functionality. 7.0 IT RISK ASSESSMENT 7.1 Level 1 - Computing Services (CS) Building and Central Computer Room 7.1.1 General The CS Building is a two story, concrete, structure located on the north end of campus behind the Rex Recreation Facility. The Computing Services staff, in its entirety, is housed in the facility on the 1 st and 2 nd floors. The Central Computer Room is located on the 2 nd floor of the Computing Services Facility. This room houses the main campus servers and router; the phone switch (PBX) and peripheral servers, such as voicemail and E-911. It is the location where all data and transmitted communications for Adams State College is redirected, combined, stored and retrieved. There is no off-site backup facility, currently identified, that could replace the functions of the Central Computer Room if it is rendered inoperable by an environmental or manmade disaster. 7.1.1 Risk Assessment 7.1.1.1 Physical/Security Risks The Computing Services Building can be accessed through three doors. Each door is keyed to a unique Computing Services key; a campus master key will not open the Computing Service building doors. There are a large number of windows on the 1 st floor of the building which are susceptible to breakage and possible unauthorized entry. Many of the windows have screws or bolts on the outside frames, allowing for potentially undetected intrusion into the building. There is no alarm or camera system for the building doors or windows. Periodically, in the evening, officers from the campus Public Safety Office will ensure that the building doors are secured. Entrance to the Central Computer room, located on the 2 nd floor, is through a single, locked door; keyed with Computing Services, unique key; the room is comprised of concrete walls with no windows. 7
The stairwell to the Central Computer Room is located by the rear entry door to Computing Services; periodically this rear entry door is propped open for equipment delivery, offering an opportunity for an unauthorized person to access the facility. There is no video surveillance inside the computer room. 7.1.1.2 Environmental Risks Rain o The CS building has a flat roof; the roof has leaked into the Central Computer Room in the past; o There are no environmental sensing devices installed in the Computer Room to detect water leakage. If a leak were to occur over a weekend, CS personnel may not be aware of it until the following Monday, possibly too late to mitigate equipment damage. Flooding Fire o First floor offices would be impacted in the event of flooding PBX batteries are located in the first floor battery room and would be ruined if flooding were to occur. o The Computer Room is located on the 2 nd floor of a concrete structure which protects it from flooding o There are no plumbing lines located above the computer room which could burst or leak o The building generator is located at ground level and is susceptible to a flooding risk o Though the building structure is concrete, it houses a large number of desktop computers, a paper storage area, a PBX battery room and individual cubicles which contain documents, books and equipment o The Computer Room contains large quantities of equipment, but minimal combustibles such as papers or documents widespread fire is not likely, however small, contained fires are possible in the wiring and equipment. o Within the Computer Room, the telecom wall is wood, plastic and PIC insulated wire it is the most flammable part of the room. o Storage of combustibles (cardboard, paper, plastics, liquids) is not allowed in the Computer Room o There is no fire suppression system in the Computer Room to reduce damage to equipment. 8
Extreme Temperatures o The Computing Services Building suffers from inadequate temperature regulation. Externally mounted, window, swamp coolers provide some cooling relief during the summer months; inadequate heating during the winter months has resulted in individual floor heaters being purchased for personnel cubicles o Primary and backup air conditioners are available to cool the Central Computer Room. Either system is capable of providing the necessary cooling for the room. Only the primary air conditioner is generator powered. The backup unit automatically fails over in the event the primary unit stops operating o Computer Room air conditioner units have heaters and the computers produce heat, so risk of too low a temperature is minimal. Natural Disasters (earthquake, tornado, high winds) Other o The CS building is a solidly constructed concrete structure which protects personnel and equipment from high winds. o The San Luis Valley region does not have a history of major earthquakes or tornados. o The building sealing is poor which has allowed birds to gain access to the interior of the building. If rodents are able to enter, as well, this has the potential to cause cabling or electronics damage. 7.1.1.3 Internal Systems Risk Power is provided to the CS building from Excel Energy through the regular power grid. The building has 3-phase power utilizing transformers to provide power for the air conditioners and multiple step-down transformers to provide power for equipment in the computer room. o The main building transformer and entry wiring is located on the west exterior wall of the Computing Services building. o The transformer and wiring is not protected by a locked enclosure and is susceptible to vandalism. Standby power is provided by a natural gas powered generator. o This generator provides power to the entire building except for the elevator which is 440 volts and one air conditioner which is 440 volts. These two items are not considered essential for disaster recovery. o The generator performs a self-test each week. Cutover testing is performed on a periodic basis. The generator is serviced annually. 9
o The generator is located at ground level and is not protected by a fence or other locked enclosure. It is susceptible to flooding and vandalism. Available computer room power is currently maxed out. Additional circuits must be freed up or installed to provide adequate power for additional server needs. o Service outages could occur if additional hardware is added to circuits that are already fully utilized Essential computers and equipment have battery UPS s to maintain power until the generator can run up in the event of a power outage. Many of the UPS s are operating well-beyond there recommended useful life and need to be replaced 7.1.1.4 External Systems Risk Operation of the Central Computing Room is highly dependent upon the external campus cable plant which provides fiber and copper lines to carry data and telecommunication services. o The cable plant was upgraded in CY2000 as part of the campus cable plant capital project The cable plant is estimated to have a ten year life expectancy o Copper wiring and fiber optic cable is run in underground conduit that can be accessed via manholes o The entry for all fiber optical cable and telephone cable is located at ground level on the west side of the Computing Services facility; it is not protected by any type of physical barrier to prevent damage due to vandalism and/or accident. 7.1.2 Recovery Planning Recovery decisions will be based on the extent of the damage to the CS building and central computing room. A backup computing facility does not currently exist, so if the central computing room remains habitable, every effort will be made to re-establish services in the same area. If the central computer room is not habitable, the Computing Services area that existed on the 1 st floor of the Richardson Hall building will be established as a backup computer facility. Adequate fiber, copper and power must be brought into the facility in order to bring up partial services to the campus. If it appears recovery of individual services will take longer than a week to restore, on a selective basis, services will be evaluated for possible out-sourcing to commercial organizations. 10
7.1.3 Preventative Measures The CS building and Central Computer Room are the single most important IT resources on the campus. Restoring this facility will be both expensive and time consuming. The current facility/room should be hardened to protect it from possible environmental or manmade damage. The following recommendations are made to protect this significant resource: o Install a building and computer room alarm and monitoring system both environmental, motion and video, with a remote-notification capability. o Construct a pitched roof to protect the computer room from possible water damage from rain or melting snow. o Improve building sealing to prevent access by birds, rodents, etc. o Designate additional storage areas outside of the CS building to reduce building clutter and reduce the amount of flammable material on-hand. Develop and document a power plan for the central computer room. Add additional electrical power and circuits to accommodate near-term and future equipment needs. Re-wire the backup air conditioner to allow generator operation for both air conditioners. Replace older UPS s and put all UPS s on a standard replacement cycle to ensure a seamless cutover to generator power, if and when, there are power failures. Protect the external building transformer and generator by protecting both with locked enclosures. Protect the fiber optic and telecom cable entry point via a physical barrier Provide better physical security for MDF s and wiring closets to preclude inadvertent or intentional damage. Establish a standby computer room on the 1 st floor of the RH building. o The initial focus of this effort should be to bring enough fiber, copper and power connectivity to this area to support a partial recovery of campus services in the event of a disaster to the central server room. Contact possible offsite service providers (commercial and educational) who could, on an interim basis, host critical campus services. 11
7.2. Level 2 - ASC Telecommunications 7.2.1 General Adams State College provides internal and external phone service through a Private Branch Exchange (PBX) telephone network used within the college. Use of a PBX saves the College from having to connect all of its telephone sets, separately, to the public telephone network. In addition to telephones, fax machines, modems and many other communication devices can be connected to a PBX. For this reason, all such devices are generally referred to as extensions. The ASC PBX has a redundant operating system with monitoring and trouble reporting equipment. However, it can and does experience problems. Most of the problems associated with the PBX are likely to cause partial phone outages or short-term inconvenience to customers. These problems can normally be fixed within a few hours. There are, however, some major problems that can occur and that take longer to isolate and repair due to multiple commercial companies being involved. In addition to basic telephone services, campus voice mail and call accounting services are also provided by ASC Telecommunications. o ASC uses the Repartee voice mail system Loss or the voicemail system would be a major inconvenience, but is not considered to be a critical loss to the campus. The voice mail system is no longer under warranty. o The call accounting system is an integral part of the telecommunications services. The interruption or temporary loss of this service would result in the loss of call records and other billing information. There are many PBX hardware manufacturers and models. Adams State currently uses a FUJITSU F-9600. Vendor contact information can be found in Appendix C of this document. This contact information is also found on the PBX, itself. The PBX equipment is installed on the 2 nd floor of the Computing Service s building in the Central Computing Room. There are several special circuits that the PBX utilizes to provide telecommunication related services: o There are four T-1 s that provide voice communication between the PBX and the external phone system. o There are two special circuits that are used for the campus E 911 system. o There are other special internal circuits that provide services and access to: Voice mail Call accounting Dial up services 911 Emergency Service 12
7.2.2 Risk Assessment 7.2.2.1 Physical/Security Risk See paragraph 7.1.1.1 Physical/Security Risks for the Computing Services Building and Central Computing Room. There is one dial-in channel to the PBX for remote maintenance and a Qwest phone circuit for alarm reporting. o Anyone attempting to access the PBX would need to know the phone numbers, passwords and have knowledge of the PBX program language. 7.2.2.2 Environmental Risk See paragraph 7.1.1.2, Environment Risks for Computing Services Building and Central Computing Room. PBX s are designed to function in a wide variety of environments with a temperature range of 41F to 104F being acceptable. 7.2.2.3 Internal Systems Risk The PBX operates at 48 volts DC through a power inverter, via normal grid power. During a power outage the PBX will operate off of a back-up battery system. o The back up batteries are located in a 1 st floor room in the Computing Services building and should provide up to 6 hours of system power o Computing Service has a standby generator that will provide electrical power during normal grid power outages. The generator is located at ground level and is not protected by a fence or other locked enclosure. It is susceptible to flooding and vandalism. Though the PBX is susceptible to hardware and system failures, it has a redundant control system and software which monitors the system operations. In addition, a removable hard drive system is used to maintain and store the software operating system. o In the case of a component or software failure the Critical-I monitoring system will contact the ASC PBX maintenance provider, Altura, who will in turn contact the onsite ASC Telecommunication technician. In most cases, a component failure will only affect a part of the system operation. Once a determination is made as to what component has failed, a replacement part 13
can be on- site in as little as 8 hours. If additional technical assistance is needed a certified technician could be dispatched from Altura s New Mexico office. The call accounting system and voice mail systems are dependent upon the operation of the PBX and are also susceptible to hardware and system failures. The voice mail system runs on an older server utilizing several software programs to perform its function. Hardware and or software failure would be the most likely cause of a problem. o The voice mail system uses the OS2 operating system, which is no longer vendor supported, and which will not run on current server hardware. o There is no maintenance agreement for the voicemail system.. The majority of call accounting problems will also be related to internal hardware or software problems. These problems can normally be fixed by the Adams State College telephone technician. In cases where hardware replacement or additional support is needed the system is covered by a maintenance contract with Altura Communications. 7.2.2.4 External System Risk The PBX utilizes special circuits (T-1 s) to provide connectivity to external phone systems. If these T-1 s become damaged, or if there is equipment failure at a distance point, ASC s capability to make or receive calls will be impacted. These T-1 s are maintained and serviced by Qwest, who is responsible for repair and restoration of service. Qwest contact information is located in Appendix C of this document. The PBX is dependent upon the cable plant copper wiring to provide ASC phone service. ASC is dependent upon Qwest Communications to provide long distance service. 7.2.3 Recovery Planning ASC maintains a PBX shared-maintenance agreement with Altura Communications. This agreement covers the PBX and call accounting system. Altura contact information is located in Appendix C. In most situations, PBX problems will be related to internal hardware or software problems o These problems usually affect only a few subscribers or pose an inconvenience. o These problems are normally repaired by the Adams State College telephone technician. 14
o In cases where hardware replacement is needed, Altura can have the necessary part on site with in 8 hours. o If the ASC on-site technician is unavailable, the Altura help desk can be contacted at 800-654-0715. The ASC site number is 05616. This contact information is also posted on the front of the PBX for quick reference. In the case of a catastrophic failure of the PBX, impact to the campus would be severe: o All phone and associated phone services would cease to function. o The ASC shared- maintenance agreement would not cover such a disaster o In the event of a total system failure, Altura Communications would be contacted and arrangements would be made to have another FUJITSU F- 9600 PBX or similar type PBX delivered, set up and made operational by a team of Altura technicians. This team most likely would respond within 8 to 16 hours from the New Mexico area. Once the situation was normalized, financial negotiations with Altura for the switch replacement would have to be conducted. Repairs to the voice mail and call accounting systems will initially be attempted by ASC s onsite telecommunications staff. o Additional technical support would be requested from Altura, if needed, for the call accounting system. o The voice mail vendor, WSTC Communications, would be contacted for additional technical support if the ASC technician was unable to make the necessary voice mail repairs 7.2.4 Preventative Measures Refer to the preventive measures called out for Computing Services Building and Central Computer Room (paragraph 7.1.3). In-place preventive measures include: o Maintaining an annual Shared Maintenance Agreement with Altura Communications. o Maintaining a certified ASC PBX technician. o Backing up PBX configuration changes to a hard drive that is then stored in the ES building safe. o Making tape backups of the voice mail system on a regular basis. This does not include stored voice mail messages. o Making tape backups of the call accounting system and data on a regular basis. 15
Additional preventive measures, to be considered, should include the following: o Installation of a fire suppression system in the central computer room. o Creating a crash kit with spare parts, such as digital or analog trunk cards to minimize PBX downtime. o Maintaining a spare server that will run the voice mail, OS2, operating system. o Develop a campus emergency communication strategy that assumes the PBX is inoperable. This strategy should: Consider providing cellular phones to key ASC personnel or departments. Consider the use of instant messaging as an internal campus communication option. Review the use of voice over IP (VOIP) as a possible PBX alternative. 7.3. Level 2 - ASC 911 Emergency Services 7.3.1 General Adams State College provides 911 and enhanced (E911) services through auxiliary equipment attached to the PBX. The 911 system is dependant upon the PBX to function. The Primary 911 reporting unit is located in the Computer Services Building Central Computing Room, with a secondary reporting unit located at ASC Public Safety. The 911 system provides two functions; the first is a stand alone emergency reporting system for the college. The second function acts in tandem by reporting 911 calls to the local city emergency center. The enhanced 911 service provides a physical location for emergency for 911 calls. This physical address is stored in a data base within the local 911 system and in a second database maintained at an external Qwest site. The 911 database is maintained by the ASC telecommunications technician 7.3.2 Risk Assessment 7.3.2.1 Physical/Security Risk See paragraph 7.1.1.1 physical/security Risks for the Computing Services Building and Central Computing Room. 7.3.2.2 Environmental Risk 16
See paragraph 7.1.1.2, Environment Risks for Computing Services Building and Central Computing Room. 73.2.3 Internal Systems Risk The 911 system operates on a desk top PC and utilizes several software programs to perform its function. Therefore, hardware and/or software failure will be the most likely problem to occur. The 911 system operates off of 110 volts provided through a UPS which operates off normal grid power. o During a power outage, the 911 system will operate off of a backup battery system. o Computing Service has a stand by generator that provides electrical power during grid power outages. So, electrical concerns are minimal. o In the event of an electrical disaster, with the generator and backup battery system there would be little or no impact. 7.3.2.4 External System Risk The 911 system utilizes two special circuits to provide connectivity to external phone systems. If these circuits become damaged, or if there is equipment failure at a distance point, ASC s capability to make 911 calls will be impacted. As these circuits are maintained and serviced by Qwest Communications, it will be responsible for repair and restoration of service. If there is an external failure of these special circuits the E911 system will fail to function. However, the telephone system would automatically take over the 911 system and provide basic 911 services. That is, 911 calls will go the 911 operator but E911 location information would not be sent. 7.3.3 Recovery Planning In most situations, 911 system problems will be related to internal hardware or software problems. These problems are normally repaired by the Adams State College telephone technician. Assistance is also available from Teletronics Technical Support at 1-800-444-7434, pin # 6809162891 If the 911 data base needs to be replaced, it can be downloaded from Qwest. This is considered our off site storage. In cases where hardware replacement is needed, the Altura help desk can be contacted at 800-654-0715; the ASC site number is 05616. This information is also posted on the front of the 911 system for quick reference. 17
7.3.4 Preventative Measures Refer to the preventive measures for Computing Services Building and Central Computer Room (paragraph 7.1.3). Current preventive measures include: o Maintaining an annual Shared Maintenance Agreement with Altura Communications o Maintaining a certified ASC PBX technician o Backing up 911 configuration changes to a hard drive that is then stored, offsite, in the ES building vault Other preventive measures to be considered include the following: o Installation of a fire suppression system in the central computer room 7.4 Level 2 Network Infrastructure and Services 7.4.1 General Network services are provided via the wired and wireless network infrastructure. Network services include a wide variety of functions, such as network/file storage (including the associated backup), printing, routing, switching, DNS and DHCP services, web/internet services, bandwidth allocation and monitoring, firewalls, etc. Network services are totally dependent on the campus cable plant and a widevariety of other commercial equipment including servers, switches, routers, wireless access points. Loss of network services impacts all other IT services. Although impact to telephone services is currently minor, dependencies could increase if the campus switches over to newer technologies, such as voice over IP (VOIP). 7.4.2 Risk Assessment 7.4.2.1 Physical/Security Risk With the exception of the cable plant infrastructure and switching electronics located in the campus wiring closets and individual building main distribution facilities (MDF s), all other equipment supporting network services is located in the Central Computing room located in the Computing Services building. o See paragraph 7.1.1.1 physical/security Risks for the Computing Services Building and Central Computing Room. 18
There is currently no offsite network data storage capability. Though selected data is backed up to tape (Banner, Novell, Linux) and stored offsite, data located on any disc backup system (web) would be lost if the Computing Services Computer Room was rendered inoperable. Telephone and data switching electronics are located in main distribution facilities (MDFs) and/or wiring closets located in each of the major campus buildings. o Though each closet is locked, in many cases, particularly in the residence halls, these closets are also used for miscellaneous storage and are accessed by other than Computing Services personnel. The risk for inadvertent damage and possible malicious damage is medium to high in these areas. Many closet environments are excessively dusty/dirty and suffer from significant humidity and temperature fluctuations. This can cause a higher than normal network electronic failure rate and reduce the lifetime of the copper network and telephone terminations/cabling. Wiring closet security is not up to industry standards. In many cases, doors that do have locks are warped and do not close properly. Also, access to the closets is not monitored or controlled. In some cases the ceilings are not hardened. Network printers are occasionally located in unsecured areas leaving them vulnerable to vandalism. 7.4.2.2 Environmental Risk See paragraph 7.1.1.2, Environment Risks for Computing Services Building and Central Computing Room. Wiring closets/mdf s are not environmentally controlled and subject the equipment to varying humidity and temperature extremes and exposure to excessive dirt and dust. There is a risk of equipment and cabling failure because of the lack of a reasonable operating environment. 7.4.2.3 Internal Systems Risk Hardware or software failure impacting individual network services is a significant risk. o Most network services do not have redundant hardware or failover systems in-place. There are numerous unique hardware items that represent potential single points of failure. o Equipment is used beyond its advertised/supported life due to budgetary constraints. Failed equipment will be replaced by spare, older, equipment obtained during equipment upgrade cycles. 19
Hiring experienced network engineers and technicians is nearly impossible, given the institutions salary limitations and ASC s remote location. o Adequate training and career growth opportunities must be provided to maintain ASC s current network technical staff Systems documentation, OS and configuration backup procedures, and training for backup personnel is accomplished on an ad hoc basis, resulting in differing levels of available documentation and competently trained personnel in the event of a major incident. All network equipment configurations are backed up nightly to a configuration change management server called Device Authority. This server is not backed up and resides on the first floor of Computing Services. Without establishing appropriate individual and group directory quotas, network storage availability could be exceeded, preventing any additional storage from occurring. Directory tree corruption could potentially require manual reinstallation of all network printer information for each individual device. Wiring closet UPS systems are not tested and/or replaced in a systematic manner. 7.4.2.4 External System Risk Campus Internet connectivity is dependent upon a single Qwest Communication s fiber optic pathway into the San Luis Valley. This pathway can and has been damaged, resulting in the loss of external campus connectivity. There are currently no secondary (backup) data trunking pathways between campus buildings. If current cable plant pathways are damaged, network services will be impacted. Hackers could attempt to launch denial of service attacks and/or attacks against network equipment and IOS and/or configuration files. 7.4.3 Recovery Planning Given the wide-variety of potential problems which could impact network services, the following generic recovery planning steps will be utilized to identify and resolve network problems: o Assess which network service or services have been lost. o Notify the campus, by whatever means available as to the service outage. o Trouble-shoot to isolate the cause of the service outage if necessary, contact the appropriate vendor for diagnostic support (see Appendix C for vendor contact information). 20