Information Security and Continuity Management Information Sharing Portal. Category: Risk Management Initiatives

Transcription

1 Information Security and Continuity Management Information Sharing Portal Category: Risk Management Initiatives Contact: Chip Moore, CISO State of North Carolina Office of Information Technology Services Project Initiation date: June 2010 Project Completion Data: August 2011 (Vulnerability system implemented and on-going use by agencies began) 1 Version 1.0

2 Executive Summary Vulnerability Management within state government is very difficult and takes too long. A leading security firm estimates that patching alone can deliver 80% protection ; however, this only works if the software patches are applied quickly and appropriately. The vulnerabilities for software and hardware are identified at different times of the year, pose different threats, and are typically announced either monthly or quarterly, based on the vendor. The effort to apply patches (fixes) to thousands of devices is immense. The longer a device is not patched, the greater the likelihood of compromise and a security incident. In North Carolina there is a division of responsibility for system patching among state agencies. The Office of Information Technology Services (ITS), the State s central IT services provider, segregates its vulnerability management between the operating system or platform and application vulnerabilities. ITS patches the platform vulnerabilities and the client agencies patch their application vulnerabilities. It was approached this way because the application vulnerabilities stand a much greater chance of breaking the business and the agencies own their respective business applications. Late in 2010, the Office of Information Technology (ITS) for the State of North Carolina embarked on a project that would leverage open source tools to take the scanning results from all platforms, load them into a centrally managed database, strip out the false positives, and deliver the results immediately to customers. The customers can then assess and validate the vulnerabilities. Once addressed, customers are able to update the findings and report back to ITS on the current status of any and all vulnerabilities. This has reduced the amount of time to market from weeks to days. We were able to accomplish this while saving money and retaining more historical information. 2 Version 1.0

3 Business Problem and Solution Description As government has rushed to provide data to its citizens and the use of distributed servers has exploded, this move has exposed more and more of government data to the World Wide Web. As Internet usage has expanded, each of these servers operating systems and applications represent an opportunity for compromise. These vulnerabilities continue to be discovered and patched. This is evident based on the fact that Microsoft provides monthly patching for its products. In a large enterprise, these vulnerabilities represent a huge target of opportunity for those who would do us harm and a real challenge for the agencies to keep current on patching. Based on the server landscape, thousands of devices may need to be patched. Automation to deploy the most significant patches and knowledge of risk posture are keys to an agency s success. A leading security vendor suggested that at least 80 percent of vulnerabilities can be addressed by a robust patching program. The issue for vulnerability management is that it is not a one-time event. It is recurring and needs to be executed in a defined, repeatable process. Based on the amount of sensitive or confidential data that most states hold, this must be a top priority for all states. Devices that need patching tend to be very dynamic and the nature of the network changes all of the time. For example, the state s cycle for vulnerability management is: 1. Vendor releases vulnerabilities and patches, at the same time. Most major vendors are on a monthly or quarterly cycle. 2. Vulnerability management companies distribute patch packages for products to their customers. 3. Enterprises evaluate the impact to their environment. This includes a decision on the risks, threat and livelihood of a particular vulnerability. 4. Enterprises scan their networks to see what the potential impact of the vulnerabilities could be, based on the number of devices using the platform or application. 5. The scan information is circulated to the offices responsible for the vulnerability. 6. The offices must then review the situation, based on the criticality of the vulnerability in their environment, and then decide if the fix will have any impact to the business. 7. Once these decisions are made, staff will start deploying the patches in a controlled method with feedback along the way. One way is to start with the development cycle one week; if no problems arise, then advance to apply 3 Version 1.0

4 patches to the preproduction environment; if no problems arise, then patch production. 8. This sequence is then repeated again and again, based on the cycle of the vendors. One problem with this approach is that it takes time, and there is never enough time in security. The security organization SANS estimates that an unpatched server on the Internet may be compromised in less than ten minutes, While this may be overly aggressive, real life has shown that a personal computer, server or piece of network equipment can be compromised in only a few days, not weeks. Another issue is that vulnerability management can be tedious, but it is critical. A robust vulnerability management program is essential. From the central IT organization, one of the biggest obstacles is getting the findings out to customers as quickly as possible. Time is critical and the data has to be valid. False positives need to be addressed (a false positive is a finding that the scanning software detects but is not really a potential vulnerability). Since you have to prioritize the vulnerabilities, you cannot be distracted by looking into vulnerabilities that do not exist on the network. These false positives need to be addressed once and then not addressed again. Resources are always an obstacle, since it is no small feat to scan thousands of machines and present the results for action to the community at large to start remediation. Customers must then decide the order in which to patch their devices. All parts of state government have resource constraints. The security office is often seen as overhead, and states are looking to reduce the budgets any way they can. This means the IT organizations are demanding efficient, timely and cost effective solutions. In North Carolina, it was recognized early on that the scanning and reporting process was laborious and the data was often stale by the time it was presented to customers. The time it took to get the information to the security offices of large agencies and then have them redistribute the data to the divisions within their agencies could be weeks. Oftentimes, by the time the divisions developed a plan to remediate an issue, the next set of vulnerability results would be on hand and the process would start all over. This does not even take into account the management and nuances of the particular vendor solutions that may have been acquired. As previously stated, there should be a huge sense of urgency to get these devices patched and remediated as quickly as possible without interrupting the business. Based on the way technology is deployed in the state, this may be no small task. 4 Version 1.0

5 After working for some time to get a repeatable process that reported results in a timely manner, we realized that we had to make the process as short as possible without adding more people or spending more money for a better solution. The security office within ITS solicited feedback from our staff and customers to see where we could improve. It became very clear that the timeliness of the reports, ease of use and the amount of manual intervention were all areas that needed to be addressed. We looked at commercial products and, while all would do similar tasks, they created different challenges. However, chief among the challenges was the cost to acquire and maintain them. The security office then looked for a more cost effective solution. We found it with an open-source content management solution called Drupal, Drupal looked like it would do all that we needed, and it was becoming popular in the federal space. We had the Linux and MySQL experience -- we just needed to get Drupal to work. At the same time, due to cost, we also dropped one of our two vulnerability management solutions. We are now using Nessus for all scans. We find it to be flexible, scalable and accurate. The architecture consists of three tiers: web, application server and a database server. A project plan was developed and agreed to by management. The only costs incurred were the acquisition of a small VMware server, which costs $ a month. We also purchased a VeriSign certificate to allow encryption in transit. The site is also limited to only IP addresses that exist on the state s network. The internal IT security staff is able to code, test and maintain the product. We were even able to incorporate the state s centrally managed access and authentication service. This meant that no new credentials had to be created. Everyone already had a user name and password -- we just had to grant them access to the system. We then surveyed the agencies as to who should have access to the vulnerability data. There was little or no training required. Individuals can only see the vulnerabilities assigned to their agency or division or application. ITS segregates its vulnerability management between the operating system or platform and application vulnerabilities. ITS patches the platform vulnerabilities and the agencies patch their application vulnerabilities. We approached it this way because the application vulnerabilities stand a much greater chance of breaking the business. This said, we are able to separate the platform and application vulnerabilities into different groups and assign them to different people so patches can be applied more efficiently. We conducted a staged rollout and asked for and received feedback along the way. This allowed us to deliver a solution that worked for the state agencies. 5 Version 1.0

6 Significance The significance of this project is that it streamlines IT operations and delivers timely and relevant information security communications to all executive branch agencies that have servers or devices under the control of ITS. This vastly improves the protection and control of the data that is entrusted to the state by the citizens. By its nature, information security is behind the technology; anything that can be done to accelerate the deployment of patches and provide the IT staff and the stakeholders with better information means that the state is better protected, faster. This project allowed North Carolina agencies to meet the target patching time, as stated in the Statewide Security Manual. Benefit of the Project This project has improved the security of vulnerable devices in the state of North Carolina and has vastly improved customer access to the data they need. The process is documented and repeatable. All of our customers and, by extension, the citizens of North Carolina have benefited from this project. The metrics have been impressive. It used to take up to six weeks to compile and distribute all of the vulnerabilities from each platform to the agencies. Now we have the results uploaded in less than two days. The agencies know they can always get the most up to date findings by going to the security web site. The site provides a wealth of information based on risks. We even provide information about the patch, risk ratings, common vulnerability exposure (CVE) reference, if one exists, and the common vulnerability scoring system (CVSS). The data can be sorted and presented in many different ways, including graphs. The data can also be exported to a spreadsheet. From 2011 to 2012, the number of outstanding vulnerabilities on our network went down almost 85 percent against all platforms scanned. Due to changes in staffing, staff leaving and reallocation of existing staff, we were able to repurpose three positions. The reallocation of the three positions saved the state over $250,000 annually. We were also able to eliminate our commercially provided vulnerability management solution. The greatest benefit of this project is that it is completely transferable, adaptable, extensible and cost effective. Other than the cost of the server and the SSL certificate, there is little incremental cost. Further, since Drupal is a content management system, 6 Version 1.0

7 we are looking to extend the solution to provide other relevant information to the state agency in the future. The innovation in this project is that it has taken two open-source solutions, Nessus and Drupal, integrated them with our statewide enterprise access and authentication service, and delivered a solution unlike any other known, free solution. This solution would rival commercial implementations, and it was all done by internal staff, working their normal jobs with little or no additional costs. Because Drupal is so robust, we can continue to define new processes or data to be included for consumption by our customers, all done securely. 7 Version 1.0