Patch Management Good Practice Guideline

Transcription

1 Programme NPFIT Document Record ID Key Sub-Prog / Project Infrastructure Security NPFIT-FNT-TO-INFR Prog. Director Chris Wilber Status Approved Owner James Wood Version 1.0 Author Gary Croft Version Date 02/10/2009 Patch Management Good Practice Guideline Crown Copyright 2010

2 Amendment History: Version Date Amendment History /03/2009 First draft for comment /04/2009 Second draft including comments /05/2009 Third draft including comments passed for final review /10/2009 Approved Forecast Changes: Anticipated Change When Annual Review October 2010 Reviewers: This document must be reviewed by the following: Name Signature Title / Responsibility Date Version Infrastructure Security Team James Wood Head of IT Security Approvals: This document must be approved by the following: Name Signature Title / Responsibility Date Version James Wood Head of IT Security 02/10/ Distribution: NHS Connecting for Health Information Governance Website Document Status: This is a controlled document. Whilst this document may be printed, the electronic version maintained in FileCM is the controlled copy. Any printed copies of the document are not controlled. 2

3 Related Documents: These documents will provide additional information. Ref no Doc Reference Number Title Version 1 NPFIT-SHR-QMS-PRP-0015 Glossary of Terms Consolidated.doc Latest 2 NPFIT-FNT-TO-IG-GPG-0033 Glossary of Security Terms ( rasec/gpg) Latest Glossary of Terms: List any new terms created in this document. Mail the NPO Quality Manager to have these included in the master glossary above [1]. Term Acronym Definition 3

4 Contents 1 About this Document Purpose Audience Content Disclaimer Introduction Background What is Patch Management? Functional Non-Functional Upgrades Vulnerability Disclosure Responsible Disclosure Partial Disclosure Full Disclosure No Disclosure What should be patched? Operating Systems Applications Firmware Patch Management Process Awareness Asset and Inventory Management Notifications WARP Vulnerability Management Systems Analysis Testing Preparation Change Management Emergency Change Management Deployment Monitoring

5 7 Other Considerations Mitigations Hardening Systems Patching Schedules Unsupported Systems Security Incident Management

6 1 About this Document 1.1 Purpose The purpose of this document is to offer advice and guidance relating to PatchManagement in NHS or other healthcare environments. Detailed technical knowledge of the techniques presented is not required. Guidance includes: - The reasons that good Patch Management is beneficial. Suggestions on implementing good Patch Management. 1.2 Audience This document has been written for readers within any NHS or healthcare provider organisation who have a general familiarity with IT applications and infrastructure issues. 1.3 Content This document comprises the following sections / topics: - Introduction Background What is Patch Management? Vulnerability Disclosure What should be patched? Patch Management Process Other Considerations 6

7 1.4 Disclaimer Reference to any specific commercial product, process or service by trade name, trademark manufacturer, or otherwise, does not constitute or imply its endorsement, recommendation, or favouring by NHS Connecting for Health. The views and opinions of authors expressed within this document shall not be used for advertising or product endorsement purposes. Any party relying on or using any information contained in this document and/or relying on or using any system implemented based upon information contained in this document should do so only after performing a risk assessment. It is important to note that a risk assessment is a prerequisite for the design of effective security countermeasures. A correctly completed risk assessment enables an NHS organisation to demonstrate that a methodical process has been undertaken which can adequately describe the rationale behind any decisions made. Risk assessments should include the potential impact to live services of implementing changes. This means that changes implemented following this guidance are done so at the implementers risk. Misuse or inappropriate use of this information can only be the responsibility of the implementer. 7

8 2 Introduction The following information provides a knowledge-based framework that will help maintain good practice values within an organisation. This guide details good practice, and by following it some of the consequences of non-compliance should be avoided. After reading this document the reader should understand: - The basics of Patch Management, the various types of techniques available to achieve this and their advantages, as well as any shortcomings in terms of security. Good practice for any NHS or healthcare provider organisation implementing Patch Management within its organisation, or building upon existing Patch Management approaches. 2.1 Background The NHS in England is composed of numerous individual organisations, ranging from large complex setups to smaller organisations with limited resource and capability. Due to the working practices between these organisations trust relationships may exist to interconnect the Wide Area Networks (WAN) or Local Area Networks (LAN) which could lead to resulting patch management approaches (or lack of) having a wider impact than the initial immediately affected organisation. N3 is a private Wide Area Network (WAN) and access is therefore strictly limited to authorised endpoints. Any organisation wishing to connect to N3 is responsible for ensuring that their N3 connection does not compromise the security measures already in place within the WAN. This is agreed to by signing the Information Governance Statement of Compliance 1 by all connected organisations. N3 faces numerous potential threats to security, possibly from inadequately protected partner networks, or connections to uncontrolled external networks such as the Internet. These threats are continually evolving in both strength and frequency. Therefore ongoing vigilance against these threats, and the maintenance of strict security standards, are essential to the continuing success of N3 which is considered UK Critical National Infrastructure (CNI)

9 3 What is Patch Management? Patch Management is the ability to implement patches in a timely manner using strategies and plans as appropriate to ensure systems and services continue to be available and secure for the purposes of the business function(s) they provide. Patching (sometimes also known as a fix ) is a technique used to correct a problem (often known as a bug ) in a computer program. This is typically done by obtaining a repair program from the vendor which the original computer program was purchased from. Patches can be summarised into the two categories: Functional Non-Functional 3.1 Functional This typically involves correcting features or functionality of a computer program. Functional patches may involve changes to the look and feel of certain areas of a product or areas that may not work as intended but which do not affect the overall security of the application. From a purist point of view, it may be noted that functional problems may affect the ability to undertake certain tasks which can align with availability targets. However, security generally would focus on more significant impacts such as complete unavailability e.g. denial of service, or integrity of the computer program rather than reduced functionality. If a system requires a functional patch in order to increase it s ability (or restore ability if lost following an upgrade or other reason) to undertake certain tasks or efficiencies, it would be considered good practice to implement this by following Service Management 2 disciplines such as Problem and Change Management. Additionally, when patching systems that have clinical use or functionality, the clinical impacts should be considered within a clinical safety assessment or similar methodology. 3.2 Non-Functional These types of patch are released for several purposes, many of which are security related. The focus of this document will therefore mainly consider these. In the case of patches released to repair or mitigate a specific security vulnerability found in a system in live service. The severity of the vulnerability generally dictates the timescale for availability, ultimately leading to deployment and subsequent correction of the vulnerability. The addition of the patch usually will not affect functionality or have affects to users of systems or services, however, this should be confirmed via pre-deployment testing and/or post implementation testing, where systems have 2 The IT Infrastructure Library (ITIL) v3 is considered to be the standard for Service Management. More information can be found here: 9

10 clinical impact or use, appropriate levels of testing rigour should be applied to all patches. 3.3 Upgrades The activity of upgrading a system, i.e. moving to a new release of computer code, may include many previous functional and non-functional patches, while at the same time adding new features or functionality not available previously. It is considered poor security practice to rely on upgrades to apply nonfunctional patches. It is most likely that patches will be released in the time before upgrades are available which could place significant risk on organisational assets. Once an upgrade is available this may negate the need to apply previous nonfunctional patches. Non-functional patches made available beyond this date should then be applied as and when they are released. Upgrades may comprise of interim release such as Services Packs which complement major releases of Microsoft Windows OS, or a move to a complete major release. 10

11 4 Vulnerability Disclosure For the purposes of the rest of this document, the majority of the content will be focused in respect to Non-Functional patches to potential information security vulnerabilities. These vulnerabilities may be disclosed in a number of ways or not at all. It is important to understand these concepts as they can influence risk management decisions. Vulnerabilities may be found by various types of individuals, groups and organisations and the methods used to disclose discovered vulnerabilities can vary considerably. It should also be noted that the availability of non-vendor patches or interim fixes may differ between disclosure methods and must be considered fully before any potential implementation. 4.1 Responsible Disclosure Vulnerabilities disclosed in this way are reported to the vendor(s) of the products concerned. The individual, group or organisation which discovered the vulnerability works with the vendor to ensure that the vulnerability is satisfactorily resolved. Using a responsible disclosure methodology, no details of the vulnerability are released publicly by either the vendor or the discoverer of the vulnerability until a corresponding patch is made available. This approach gives a window of opportunity to address the problem and release a patch to the affected communities. The vendor and the discoverer of the vulnerability both conduct a co-ordinated release of the patch and vulnerability details simultaneously once the patch is available. The discoverer of the vulnerability is credited publicly with discovering the vulnerability by the vendor. For an individual, group or organisation this can be good publicity. In the last few years, there has been much more of a focus on responsible disclosure for economic reasons. A number of organisations (such as TippingPoint 3 ) have started paying individual security researchers and groups for sole rights to a newly discovered vulnerability via bounty programmes and then working with the vendor of the affected product on the researcher or groups behalf. This allows researchers and groups to remain anonymous to the product vendor if they so wish but also allows them to get paid for their work. Previously, many security researchers worked on discovering vulnerabilities in products purely for the intellectual challenge of doing so and without any financial motive. Another argument for such bounty programmes is that they discourage potentially less ethical security researchers from selling discovered vulnerabilities to criminals who would use the vulnerabilities to compromise systems for financial gain. 3 The Zero Day Initiative from TippingPoint is one such example of a bounty programme. See: for more information. 11

12 4.2 Partial Disclosure The partial disclosure methodology is characterised by some high-level details of the vulnerability being divulged to the general public. This is often via a security researcher s web site or blog or possibly via one of the well known security mailing lists such as Bugtraq. 4 Via this route, the vendor is notified of the vulnerability as well (or is notified directly by the discoverer of the vulnerability) and will often produce a security bulletin detailing some mitigating actions which can be taken prior to a patch being released. If the mitigating actions are followed, they can often lead to a loss of functionality in the product containing the vulnerability. Therefore, a risk assessment should be performed to determine level of exposure to the vulnerability and vulnerability severity versus the consequence of lost functionality for users of the product. Some security researchers and groups follow the partial disclosure methodology because they feel it encourages the vendor to produce a patch for a vulnerability more quickly than if they notified the vendor directly. On occasions, the security researcher or group will give the vendor a time limit for producing the patch, which if not achieved by the vendor, will see the researcher or group release full details of the vulnerability to the public potentially creating risks with little or no mitigation immediately available. 4.3 Full Disclosure Security researchers and groups who practice the full disclosure methodology feel that all systems should be open to scrutiny and that information on vulnerabilities in systems should be made available to all. This for example allows customers to determine their level of exposure themselves and put mitigating actions in place without vendor timescales driving these activities. A downside is that by releasing full details to everyone, this information also makes it into the hands of more nefarious individuals. Vulnerabilities reported in this way are therefore reported to everyone at the same time via either security related mailing lists, security websites or blogs. This provides opportunities for potential attackers to exploit these vulnerabilities before vendors release patches and affected organisations are able to apply them. Upon discovering vulnerabilities disclosed in this way, a vendor will often produce guidance for customers in the form of a security bulletin which details possible mitigations which can be put in place prior to a patch being released. Note that potential mitigations may involve disabling certain functional aspects of a product. In these instances, a risk assessment should be performed to determine level of exposure to the vulnerability and vulnerability severity versus the consequence of lost functionality for users of the product. Other mitigations may be possible in the short term before a patch is made available. For example, technologies such as Intrusion Prevention Systems i may assist as interim measures however, they should not be solely relied upon. 4 Information about the Bugtraq mailing list: 12

13 4.4 No Disclosure In some circumstances vulnerabilities are developed often for financial gain. In this scenario an organisation or person may choose to use vulnerability for its own purposes (sometimes illegal) and not make it known to any outside communities. A slight variation on this theme is that the same organisation or person may choose to auction a discovered vulnerability to the highest bidder. This again could be used for illegal activities, or it may be purchased by the vendor of the system concerned or an intrusion prevention system provider to enhance its own products. A zero-day (or zero-hour) vulnerability is one that is unknown to both the vendor of the affected product and the general public and for which no security update exists. Zero-day vulnerabilities often receive active exploitation in the wild and exploitation of the vulnerability is often how they are discovered. Finally, once a patch is released, it is possible for an attacker to reverse engineer and analyse the patch to discover the full details of the vulnerability and subsequently develop techniques so as to be able to exploit any remaining un-patched systems, therefore timely patch management is very important. 13

14 5 What should be patched? 5.1 Operating Systems Sometimes referred to as OS, this is an interface between hardware and applications; it is responsible for the management and coordination of activities and the sharing of the limited resources of the computer. Examples of operating systems are Microsoft Windows and Linux. 5.2 Applications Some applications come included with an OS, while others are added as need arises. Common user applications include Microsoft Office applications and Adobe Acrobat, system applications may include Web servers and Databases. 5.3 Firmware These are typically coded instructions that are stored in permanent or semipermanent memory. An example of this would include a computer BIOS which is loaded prior to an OS or application. Other examples may include the firmware used on network devices or in medical devices that serve important functions. In the case of Medical Devices it is advisable to seek manufacturer instructions if in doubt of the clinical safety impacts of applying patches to medical devices. Further guidance is issue from the Medicines and Healthcare Products Regulatory Agency (MHRA) MHRA Device Alert information is available from the mhra.gov.uk website

15 6 Patch Management Process There are many variants of patch management processes, however typically they include several of the following areas: Awareness Monitoring Analysis Deployment Testing Preparation Figure 1 A Typical Patch Management Process These are discussed in the next sections. 15

16 6.1 Awareness It is important for an organisation to be aware of its systems and be aware when patching is required, there are several ways to achieve this. In complex environments several strategies may have to co-exist Asset and Inventory Management 6 It is a fundamental requirement for good patch management (and for many other disciplines such as financial and licensing) to have some form of record which reflects what information assets are deployed within the organisation. This should also be a living record to reflect its evolving nature. There are commercial utilities which can aid this effort and methodologies that can be worked towards. ITIL 7 recommends the use of a Configuration Management Database to track an organisations assets. If the management and maintenance of an asset is outsourced beyond that of the host organisation, steps should be taken to ensure that patch management is undertaken to continue to safeguard these assets. The potential and actual risks relating to these assets cannot be outsourced Notifications Many vendors offer mailing lists, and alerting bulletins that can be subscribed to, often free of charge. Independent organisations such as SANS Internet Storm Centre, 8 BugTraq 9 and PatchManagement.org 10 also provide similar services. In larger organisations it is commonplace for the ICT department to subscribe specific addresses which can be accessed by multiple staff for continuity WARP A Warning, Advice and Reporting Point can often be a good way to share advice and information on computer based threats and vulnerabilities. WARP s can be community specific and therefore the content available can be tailored even more to the environment of the organisations and users of it. Further information is available on the WARP website

17 6.1.4 Vulnerability Management Systems There are many commercial and some free systems that allow an organisation to raise awareness of potential patching issues through automated tools. Commercial examples of this are Qualys 12 or BigFix 13 Vulnerability assessment tools such as Nessus 14 or Microsofts Baseline Security Analyser 15, can sometimes be used to perform more ad hoc scans, which may or may not integrate into automated management products. 6.2 Analysis When an organisation becomes aware of a potential vulnerability requiring patching, it needs to analyse and decide the relevance and consequences within its specific environment(s) in relation to the systems and services in place Risk Management There are many risk management approaches and tools that can be used to aid analysis. The NHS has access to the ISF 16 Risk Assessment Tools and the ISO standards via the Information Governance Toolkit 17 Further Risk Management advice and guidance can be found from the NHS Connecting for Health website 18. In some environments it may be appropriate to have an easy to use scoring matrix to identify the impact and the probability of a particular vulnerability being exploited hsinforiskmgt 17

18 (1) Not Critical (2) (3) (4) Less Critical Moderate Critical Highly Critical Impact/Criticality (5) Extremely Critical (1) (2) (3) (4) (5) Improbable Remote Occasional Probable Frequent Probability Figure 2 An example of a Vulnerability Assessment Scoring Matrix In the above scoring matrix, a patching policy (as an example) could be adopted to patch vulnerabilities placed in the Red area within 24 hours, or as soon as possible. Green scored vulnerabilities may be patched in the normal business as usual patching schedule, and Yellow dealt with on a case-by-case basis. This is by no means the only way to score vulnerabilities and other approaches may be more suitable in a given environment. Vendors and independent parties like SysAdmin, Audit, Network Security Institute (SANS) may provide risk ratings to inform the reader and give an indication as to how quickly they believe the patch should be applied. There also may be different impacts and probabilities in particular business areas, therefore several assessments may have to be done to understand the full exposure to the organisation. 6.3 Testing Prior to deploying any patches it is advisable to test them in a non-live environment, such as a test environment, designed to mimic the live environment as much as is practically possible. 18

19 Patches should be obtained from reputable sources, in most instances this will be the vendor. The patch itself should be confirmed against published checksums or digitally signed to prove its authenticity. Unofficial patches can cause adverse affects or/and additional vulnerabilities. In some organisations it may not be feasible to maintain test environments, however it may be still possible to implement patches on lower impact systems to observe the results first, before deploying on more important systems this should be balanced in risk management terms as described in earlier sections in relation to the need to have the patch deployed in a timely manner. Again, in NHS organisations clinical safety may be a concern and specific guidance should be sought where applicable. 6.4 Preparation It is advisable to prepare well when rolling out patches, assuming a test environment has been used (as suggested in the previous section), this will given some confidence that adverse effects can be avoided. Testing however does not allow business impacts to be factored in and therefore other approaches are advisable to take into account some real world factors Change Management This is an activity recommended by ITIL to manage changes within an organisation. It could include a phased approach to the patch rollout, which would include advising system users and business functions of the need that this is to occur, which in turn can allow any problems or unforeseen items to be raised. It can also be used to co-ordinate activities during the deployment. A change management plan should also include steps for rollback, should problems be encountered Emergency Change Management Similar to Change Management, however this involves deployment within tight timescales and certain stages may be omitted. This could be required to patch a high potential impact vulnerability, it could also be required if a patch has had undesired consequences and requires rolling back. 6.5 Deployment Centralised Deployment In larger organisations this is often beneficial due to the number of devices that may require patching. It also means that patches can be obtained once either via the internet or other means and then distributed via the organisations LAN, which conserves often precious external bandwidth. 19

20 NHS Connecting for Health have negotiated a number of Enterprise Wide Agreements for NHS organisations in England (other terms may be available for Home Countries). Detail of the Novell ZenWorks product can be found here 19 Many NHS organisations also use Microsoft products. Detail regarding Microsoft Systems Management Server and Client Access Licensing can be found here 20 Microsoft also provides Windows Servers Update Services (WSUS). Network device vendors often provide bespoke management software to enable central deployment of patches that can have added benefits such as monitoring and alerting. For example Cisco provides CiscoWorks 21 software Auto-updates Some systems may be set to auto-update. When a patch becomes available it will be obtained (typically from the internet) and deployed automatically. This can be advantageous in some circumstances however it can also have undesirable results. This approach is also very bandwidth inefficient in large organisations. The N3 network provides a central N3 Internet Gateway for use by NHS organisations, a large percentage of its bandwidth is taken up by update/patching downloads from the internet. NHS organisations should make efforts to centralise their updates where practical to ensure patches are not downloaded multiple times within the same organisation and thus consuming bandwidth unnecessarily Single Instance patching In some circumstances it may be required to physically attend a device to deploy a patch, this is obviously more labour intensive than the central deployment or automated approaches. Reasons for using this approach may vary, however one reason could be the type of patch is to firmware or devices with limited network connectivity, which may mean this is the only realistic option available

21 6.5.4 Restarts or Service Stop Depending on the nature of the patch this may or may not be required. For more critical services it may be possible to place redundant devices into live temporarily before placing the patched devices back into live service. This should be described in associated Change Management plans and possibly done at non/least service affecting times to minimise disruption. Some patch management techniques allow a patch to be deployed, but not finalised until a restart is made. If a user is working on a recently patched device they may be given the opportunity to restart at a later time or whenever is convenient. This should be balanced against the risks of having an un-patched system left in that state for a longer amount of time. There are also occasional issues with system stability in systems which have been patched but have not yet been rebooted. It is not good security practice to allow end users to repeatedly delay the application of patches to their systems. Deployment systems such as Microsoft System Management Server and Novell ZenWorks allow users to defer the reboot of a patched system for a short period of time only e.g. up to four hours Devices that may not be on the network This concerns devices that may be patched by centralised or automated methods but are not on the organisations network at the point in time when deployment is needed, for example a laptop that is with a member of staff outside the office, or a desktop computer that is presently not switched on. Steps should be made to ensure that these types of devices are patched as soon as practically possible. Technologies such as Network Access Control (NAC) can ensure that when an un-patched device is reconnected to an organisations network it is picked up and the appropriate action is taken. 6.6 Monitoring Monitoring should take place following deployment of patches. It should also take account of any failed deployments which should be addressed as necessary. In certain instances the installation of components (software, hardware or other) can return devices to an un-patched state which should be monitored for and dealt with accordingly. It could also be useful within large organisations to develop metrics in relation to patching activities. These could be measured against business units or functions, to allow trends or patterns to be identified. It may also allow any gaps or failings to be more visible and remediation work can taken place to prevent this from reoccurring. This can be a time consuming exercise and commercial products can assist with these types of tasks such as Customer Relationship Management (CRM) systems or Vulnerability Management Systems. 21

22 7 Other Considerations This section contains a few additional considerations, which are relevant but do not directly apply to the previous body of this good practice guide. 7.1 Mitigations In certain situations like a zero day attack or for critical live systems, it may not be possible to apply a patch immediately in which case there may be mitigations that could be used. Sometimes when a vendor releases a patch there is an alternative stated as a workaround this many involve reconfiguring a device in a manner rendering the vulnerability either unlikely or no longer possible. There are also solutions that provide screening capabilities, for example Intrusion Prevention System providers, or devices can be placed in higher security areas such as DMZ s which can make certain vulnerability exploitation unlikely. See Good Practice Guides on IDS and IPS Technologies and Firewall Technologies Hardening Systems It is always advisable to limit exposure to potential vulnerabilities (particularly on critical systems) by removing unnecessary services or features Prevention is better than cure. For further information see System Hardening GPG 23. Additionally it is also good practice to prevent users installing unauthorised software which may come with its own vulnerabilities that are not picked up by the organisations ICT staff. Techniques including avoiding administrator access to systems or locking down USB ports may be appropriate in efforts to achieve this. 7.3 Patching Schedules It is good practice to embed patch management with the organisation as a Business as Usual (BAU) activity, Patching Schedules may go some way towards achieving this. It is worth considering that vendors may have differing approaches with regard to release of patches. While most will make critical patches available as soon as possible, other have chosen phased approaches for more routine patches, which include lower severity security patches. Microsoft for example, has a concept known as Patch Tuesday and releases the majority of its patches on the 2 nd Tuesday of each month. It is therefore advisable to correlate patching schedules around the release times to avoid exposure for longer periods of time

23 7.4 Unsupported Systems It is commonplace for a vendor to provide limited support and therefore patches to a product, often systems in use beyond this timeframe are referred to as end of life, although it may be possible in some circumstances to purchase extended support packages. This however does not mean that products reaching these phases cease to have vulnerabilities. More to the point it is unlikely a patch will be provided and this could have serious consequences. An organisation continuing to run unsupported systems does so at risk and should have a significant business reason for doing this, In addition, the organisation should have appropriate mitigations in place to reduce any risks to acceptable levels. 7.5 Security Incident Management The vast majority of this good practice guide has referred to patch management as a means to prevent potential vulnerabilities being exploited and thus impacting an organisations business functions. If for some reason a failing occurs in relation to patch management good Security Incident Management practices may be required. This should detail the measures necessary to deal with a security incident and return to a state of normality. 23