These materials are 2015 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.

Transcription

1

2

3 Threat Intelligence Norse Special Edition by Karen Scarfone, CISSP, ISSAP and Steve Piper, CISSP

4 Threat Intelligence For Dummies, Norse Special Edition Published by John Wiley & Sons, Inc. 111 River St. Hoboken, NJ Copyright 2015 by John Wiley & Sons, Inc., Hoboken, New Jersey No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without the prior written permission of the Publisher. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) , fax (201) , or online at Trademarks: Wiley, For Dummies, the Dummies Man logo, The Dummies Way, Dummies.com, Making Everything Easier, and related trade dress are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affiliates in the United States and other countries, and may not be used without written permission. Norse and the Norse logo are trademarks or registered trademarks of Norse Corporation. All other trademarks are the property of their respective owners. John Wiley & Sons, Inc., is not associated with any product or vendor mentioned in this book. LIMIT OF LIABILITY/DISCLAIMER OF WARRANTY: THE PUBLISHER AND THE AUTHOR MAKE NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE ACCURACY OR COMPLETENESS OF THE CONTENTS OF THIS WORK AND SPECIFICALLY DISCLAIM ALL WARRANTIES, INCLUDING WITHOUT LIMITATION WARRANTIES OF FITNESS FOR A PARTICULAR PURPOSE. NO WARRANTY MAY BE CREATED OR EXTENDED BY SALES OR PROMOTIONAL MATERIALS. THE ADVICE AND STRATEGIES CONTAINED HEREIN MAY NOT BE SUITABLE FOR EVERY SITUATION. THIS WORK IS SOLD WITH THE UNDERSTANDING THAT THE PUBLISHER IS NOT ENGAGED IN RENDERING LEGAL, ACCOUNTING, OR OTHER PROFESSIONAL SERVICES. IF PROFESSIONAL ASSISTANCE IS REQUIRED, THE SERVICES OF A COMPETENT PROFESSIONAL PERSON SHOULD BE SOUGHT. NEITHER THE PUBLISHER NOR THE AUTHOR SHALL BE LIABLE FOR DAMAGES ARISING HEREFROM. THE FACT THAT AN ORGANIZATION OR WEBSITE IS REFERRED TO IN THIS WORK AS A CITATION AND/OR A POTENTIAL SOURCE OF FURTHER INFORMATION DOES NOT MEAN THAT THE AUTHOR OR THE PUBLISHER ENDORSES THE INFORMATION THE ORGANIZATION OR WEBSITE MAY PROVIDE OR RECOMMENDATIONS IT MAY MAKE. FURTHER, READERS SHOULD BE AWARE THAT INTERNET WEBSITES LISTED IN THIS WORK MAY HAVE CHANGED OR DISAPPEARED BETWEEN WHEN THIS WORK WAS WRITTEN AND WHEN IT IS READ. For general information on our other products and services, or how to create a custom For Dummies book for your business or organization, please contact our Business Development Department in the U.S. at , contact [email protected], or visit For information about licensing the For Dummies brand for products or services, contact BrandedRights&Licenses@ Wiley.com. ISBN (pbk); ISBN Manufactured in the United States of America Publisher s Acknowledgments Some of the people who helped bring this book to market include the following: Project Editor: Jennifer Bingham Acquisitions Editor: Amy Fandrei Editorial Manager: Rev Mengle Business Development Representative: Karen Hattan Special help from Norse: Jeff Harrell, Brian Contos, Kurt Stammberger, Lisa Huff, Carlos Hermosillo, Jonathan Curtis, and Rita Chen

5 Table of Contents Introduction... 1 About This Book... 1 Foolish Assumptions... 1 Icons Used in This Book... 2 Beyond the Book... 2 Chapter 1: Understanding Threat Intelligence... 3 Defining Threat Intelligence... 4 Basic terminology... 4 Threat indicator types... 4 Threat indicator metadata... 5 Seeing Why TI Matters... 8 Improved attack prevention, detection, and response... 8 Expedited forensic investigations... 8 Inputs for risk assessments... 9 Understanding TI Delivery... 9 TI reports... 9 Machine readable TI (MRTI)... 9 Console based TI TI appliances Chapter 2: Gathering Threat Intelligence Data Gathering Locations Existing data feeds Internal customer networks External networks Automated Sources of TI Anonymous proxies Crawlers Free services Geolocation Honeypots IANA and Internet registries IRC P2P Chapter 3: Scoring Threat Intelligence Assessing Source Quality... 21

6 iv Threat Intelligence For Dummies Calculating Scores Aging Score history Using Scores Chapter 4: Supporting Incident Response Recognizing the Need to Support Incident Response Improving incident detection Reducing loads on existing devices Facilitating forensic investigations Using MRTI Using Console Based TI Using TI Reports Using Multiple TI Forms Together Manual mitigation Automatic mitigation Chapter 5: Strengthening Threat Mitigation Exploring Strategies for Strong Threat Mitigation Blocking attacks Improving catch rates Stopping advanced attacks Leveraging Existing Devices Architecture Pros of using existing devices Cons of using existing devices Using a Dedicated TI Appliance Architecture Pros of using a dedicated appliance Cons of using a dedicated appliance Chapter 6: Ten Buying Criteria for TI Solutions Automation Integration and Interoperability Frequency of MRTI Updates Metadata Richness Sophistication of Scoring Threat Coverage Visibility into Darknets Geolocation Accuracy Variety and Number of TI Sources Quality of TI Sources Glossary

7 Introduction It s harder every day for IT security professionals to fight off the latest attacks. Utilities such as antivirus software and intrusion prevention systems (IPSs) are often ineffective against today s advanced malware and emerging cyberthreats. Compromises often occur months before they re discovered, leaving massive damage in their wakes. Just think of all the recent stories in the media about major data breaches costing tens of millions of dollars apiece. What can you do to protect your organization? Now you can use carefully collected, meticulously maintained real time threat intelligence (TI) about the latest attacks to block them before they target your organization. No solution is foolproof, but TI can help close the gap with attackers. About This Book This book introduces you to TI services and products. These technologies use a wide variety of data gathering techniques to find out about current and emerging threats and to communicate that information to their customers. TI is rapidly becoming indispensable for securing organizations because it helps existing security controls work more efficiently. This book tells you what you need to know to adopt TI successfully. Foolish Assumptions When writing this book, we made some assumptions about you, the reader: You re an IT security professional working for a corporation, government agency, educational institution, or IT services firm.

8 2 Threat Intelligence For Dummies You have foundational knowledge of attack detection and prevention technologies such as firewalls, IPSs, and security information and event management (SIEM) products. You re concerned that your organization may be next to suffer a major data breach. Icons Used in This Book This book uses the following icons to indicate special content. You won t want to forget the information in these paragraphs. A Tip icon points out practical advice that can help you craft a better strategy, whether you re planning a purchase or setting up your software. Look out! When you see this icon, it s time to pay attention. You won t want to miss this cautionary information. Maybe you re one of those highly detailed people who really need to grasp all the nuts and bolts even the most techie parts. If so, these tidbits are right up your alley. Beyond the Book This book covers the basics of threat intelligence, but the best way to find out more about it is to look at actual data. To see a graphic representation of threat intelligence, visit ipviking.com. For more technical information on TI products and services from Norse, check out

9 Chapter 1 Understanding Threat Intelligence In This Chapter Defining threats and threat intelligence Knowing why threat intelligence matters Understanding the delivery of threat intelligence The use of threat intelligence (TI) to help organizations improve their security is rising steadily. For many years, TI was thought of as simply being IP and URL blacklists, but the concept has matured rapidly and expanded to include many parts of security infrastructures. You may have noticed that many of the security products in your organization already support the use of TI information, including next generation firewalls (NGFWs), intrusion prevention systems (IPSs), unified threat management (UTM) appliances, web proxies, load balancers, and security information and event management (SIEM) products. You may also be aware of TI services used to forensically investigate suspicious activity. You may have even heard of the latest TI technology: dedicated appliances specifically designed to detect and stop emerging threats. But do you know how these products and services work, let alone how to select the products and services that are best for your organization? This chapter explains the basics of TI. First, we explain what threats and TI are all about. Then we discuss why TI should be important to you. Finally, we explore delivery mechanisms the types of TI products and services that are available today.

10 4 Threat Intelligence For Dummies Defining Threat Intelligence This section explains the fundamentals of threat intelligence. Basic terminology As with many other terms in the IT security field, there s no consensus on the definition of threat, and many people use threats and attacks interchangeably. We use the following foundational terms in this book: Threat: An IT entity, such as a host or website, that is suspected of performing attacks. Attack: An instance of malicious activity. An example of an attack is malware directed at a target. Attacker: A person or group who attacks others. (Another term for attacker is threat agent.) Threat indicator: One or more related data points that imply heightened risk: for example, an IP address located in a specific country. Threat intelligence: Insights into threats gained by collecting evidence, such as observing attacks and studying the characteristics of attackers. TI comprises threat indicators and associated threat indicator metadata. Some people use the term hacker interchangeably with attacker, but others in the security community feel strongly that hacker has both positive and negative connotations. To prevent confusion, we use the term attacker to mean something negative. Threat indicator types Different TI products and services support varying types of threat indicators. The most fundamental component of a threat indicator is an IP address, which marks the identity of a potentially malicious host. In some cases, the IP address really points to a local network that a malicious host is using, such as when an infected machine is located behind a firewall implementing network address translation (NAT) services. Another basic component of a threat indicator is a URL, which

11 Chapter 1: Understanding Threat Intelligence 5 generally indicates a path to potentially malicious content, such as malware. And finally, domain names are also common factors in determining threat indicators. You may wonder why IP addresses are the most fundamental component of most threat indicators. Part of the reason is that IP addresses are readily visible in network traffic, so they re easy to collect. But there s more to the story. IP addresses are universally understood by existing network security controls such as firewalls, routers, and network IPSs. Also, because IP addresses are easy to extract from network traffic, it s less resource intensive to analyze them than to analyze other aspects of network traffic. Comparing an IP address against a list of known malicious addresses, for example, is much quicker than decrypting, unpacking, and analyzing the content of every packet at wire speeds. Threat indicator metadata A threat indicator by itself has some value, but it has much more value when it s accompanied by metadata. This metadata provides richer context for the threat indicator. Simply giving someone the IP address of a possible threat doesn t indicate the problem with that host, the source of the information, or the severity of the threat. The metadata for the threat indicator can provide all this context and more. Time stamp One of the simplest, yet most critical, pieces of metadata for a threat indicator is a time stamp, which indicates when the TI related to the threat indicator was collected. A time stamp can indicate, for example, whether a host that s distributing malware was doing so a minute ago or a week ago. Things change rapidly in the security realm, and yesterday s severe threat may be less worrisome today. Score Many TI feeds offer risk scores, which are measures of the relative maliciousness of a given IP address, URL, host, or executable. If a TI service has high confidence that an address is hosting malware, that address may get a score of 100 on a 0 to 100 scale. If an address was known to be malicious a few

12 6 Threat Intelligence For Dummies days ago but hasn t shown any malicious activity recently, it may receive a score of 50 on that same scale. See Chapter 3 for an in depth examination of how TI is scored, including how you can best use risk scores. Source Some sources of TI only aggregate existing information that is, they take free threat feeds from other parties and aggregate them into a single feed of their own. Other sources of TI do their own intelligence gathering, organically collecting threat information from customer networks or out in the wild, by monitoring and analyzing global network traffic, especially traffic on the darknets. (See the nearby sidebar Casting light on darknets. ) For more information on sources of TI and their relative quality and timeliness, see Chapter 2. Geolocation Most sources of TI provide geolocation information and use that information as a major factor in risk scoring. If many attacks are coming from a particular country or region, other activity from the same place is somewhat more likely to be malicious as well. Chapter 2 contains detailed information on geolocation. Category TI products and services categorize threats in many ways. The purpose of these categories is to provide insight into the nature of a threat. Here are some sample high level threat categories: Anonymous proxy: This host has been identified as a node on an anonymous proxy network, potentially indicating attackers intent to hide their points of origin. See Chapter 2 for more information on anonymous proxies. Bogon: The host s IP address hasn t been allocated by the Internet Assigned Numbers Authority (IANA) or assigned by an authorized registry such as the American Registry for Internet Numbers (ARIN), which means that it shouldn t be used on the Internet.

13 Chapter 1: Understanding Threat Intelligence 7 Bot: The host appears to be infected by malware and is under the control of attackers as part of a botnet. Botnet: The host appears to be a command and control node for a botnet, giving orders to bots directing them to attack other hosts, for example. Malware: The host is known to serve malware. An example is a web server with a URL that points to malware hosted on the web server itself. Passive DNS: The host s IP address appears in passive Domain Name System (DNS) records. Passive DNS records often capture information on attacker domains, such as those used for phishing. Each organization is likely to give different weights to these categories in its risk calculations. For example, malware category threats might be considered more dangerous than passive DNS threats. Based on the weights it assigns to different categories, an organization might decide to automatically block all activity involving current bogon, bot, botnet, and malware category hosts, but it might not block activity involving current passive DNS or anonymous proxy hosts. Organizations typically assign custom weights to these categories when calculating TI risk scores. Casting light on darknets The term darknets collectively refers to all the underground websites and other shadow IT resources available through the Internet. These resources are considered to be dark because they re not advertised on the regular Internet; they re typically hidden behind anonymous proxies to conceal users and administrators identities. They re also called dark because they re often associated with illegal activities. One of the goals of TI is to cast light on darknets to identify those hosts and networks most closely associated with darknet activity. Given up to date intelligence, many organizations err on the side of caution by proactively blocking all incoming and outgoing connections with darknets.

14 8 Threat Intelligence For Dummies Seeing Why TI Matters Why should you care about TI? Because the security of your organization s data and your own data increasingly depend on it. Using TI effectively also raises your organization s return on security investments, providing more bang for the buck when you use TI in combination with existing security controls and processes. With TI adoption increasing rapidly, it s likely that by the end of 2016, virtually every enterprise attack detection or prevention product will be able to ingest at least some live TI data feeds. The following sections give you more reasons to care about TI. Improved attack prevention, detection, and response Today s enterprise security controls for attack prevention, detection, and response firewalls, IPSs, and SIEMs struggle to identify many attacks because these attacks are highly customized. Signature based detection alone is no longer effective. You need to adopt a whole new way of identifying attacks, and that way is TI. TI doesn t replace your existing security controls; it enhances them. Take your enterprise firewall as an example. It could do a much better job of blocking malicious activity if it had an up to date list of IP addresses that are known to host malicious servers. TI can provide such a list, making your firewall more effective at stopping attacks while simultaneously taking a load off other security controls. Expedited forensic investigations After an incident occurs, the forensic investigation can take days or weeks, with a good chunk of this time being spent trying to determine who attacked the organization. By using TI services, you can quickly find out the threat history of an IP address or a URL and its threat categorizations. This information expedites investigations, such as by revealing that an attacking host is actually an innocent bystander being controlled by a botnet.

15 Chapter 1: Understanding Threat Intelligence 9 Inputs for risk assessments One of the toughest parts of performing an enterprise risk assessment is accurately portraying the nature of the threats against the organization. Fortunately, you can use TI services in conjunction with your security logs to categorize threats to your organization and their relative frequency. This data lets you see trends over time and helps you ensure that your risk assessment process takes the entire threat landscape into account. Understanding TI Delivery TI solutions can be delivered in several ways. The following sections discuss the major forms available today. TI reports One form of TI solution is a human readable TI report, created by human intelligence analysts who research a particular threat, then produce a white paper or booklike report on that threat. Most often, a TI report is created after an incident occurs to help determine the attacker s identity, motives, and methods. TI reports are particularly helpful to an organization that believes the same attacker might strike again. Because each report takes a deep dive into a single threat, a robust TI report can take days or weeks of staff time to compile, which tends to make TI reports very expensive. Also, because of the dynamic nature of threats and the time needed to compile a TI report, some of the information in a report may be outdated even before the report is complete. TI reports are best used in extraordinary circumstances, such as major data breaches. Otherwise, an organization can spend its money more effectively on other TI solutions. Machine readable TI (MRTI) Machine readable threat intelligence (MRTI) is, as the name implies, TI specifically formatted for automated use. An MRTI feed can become an input into a SIEM, firewall, IPS, or other

16 10 Threat Intelligence For Dummies security control to give it real time information about threat indicators. This data allows the security control to make better informed decisions about alerting on, blocking, or otherwise responding to a possible threat. The simplest and oldest form of MRTI is the blacklist, which traditionally is a list of threat indicators (such as IP addresses, executables, or hostnames) that are considered to be malicious. These blacklists typically lack metadata, so there s no context to explain why each entry is blacklisted. Without metadata, you have no way of knowing how old each entry is, what confidence you have in each entry, and so on. Newer forms of MRTI include metadata and are delivered in a dynamic feed, typically through an application programming interface (API). This API allows the security control using the MRTI to read in the MRTI in the appropriate manner perhaps reading in the entire feed or just reading in changes since the previous version was received. Because the MRTI feed is live, security controls can download updates every few minutes or even every few seconds. The volume of MRTI provided varies from vendor to vendor and even among use cases for a single vendor. In one extreme example, a TI customer may want metadata on as many hosts as possible, such as every host that s likely to be malicious. This metadata could be used to reject connection attempts from the hosts with the highest risk scores or to require additional security measures before connecting to hosts that score above a certain threshold. In another extreme example, a TI customer may want all the historical information associated with a particular IP address. This information could show whether a particular host has been compromised for years or only a few hours. A historical query could be part of the research that a customer does to compile a TI report. All sorts of cases fall between these two extremes, of course. You may want different breadths and depths of TI for different circumstances. You may want the worst million hosts (based on risk scores) to be blocked by your perimeter security controls, for example, and also want your incident response team to be able to drill down into any host s TI

17 Chapter 1: Understanding Threat Intelligence 11 history for investigative purposes. Or you may want to be alerted immediately if any of your organization s hosts suddenly appear in the MRTI feed with a high score. Console based TI Another form of TI solution is console based TI, which is essentially MRTI in a human friendly console interface. A classic example of console based TI is an interface that incident responders can query to gather information on a particular attacking IP address. Many security controls can convert MRTI to console based, human readable TI. This format isn t to be confused with the aforementioned human readable TI reports, which are narrative in nature. TI appliances The final form of TI solution is a dedicated TI appliance (see Figure 1-1). Developed by a TI vendor, a TI appliance typically requires a persistent connection to the vendor s data feeds. Then the appliance uses this MRTI out of band (to detect likely malicious activity) or inline (to detect and block such activity). A TI appliance may also include an interface for viewing console based TI. TI appliances are alternative or complementary solutions to existing enterprise security controls that take in MRTI feeds. It s common for existing controls to have fairly low limits on the amount of MRTI they can ingest and process because of performance concerns. A dedicated TI appliance, however, is designed to ingest large amounts of MRTI and to receive and process frequent (or even continuous) MRTI updates. Figure 1-1: Sample TI appliance from Norse.

18 12 Threat Intelligence For Dummies Also, a dedicated TI appliance can process a great deal more traffic than its enterprise counterparts such as firewalls, IPSs, and SIEM solutions because it s designed simply to examine network traffic for known threat indicators (such as IP addresses) and then handle that traffic according to simple policies. A sample policy could block all activity involving bots and botnets, for example, and permit but record all activity involving anonymous proxies. A TI appliance can take the load off other enterprise controls by blocking traffic before it reaches those controls, thereby reducing the amount of money a company has to spend on scaling those controls. This is often the factor that makes TI such a compelling value, because high quality TI can improve your security return on investment across many different traditional security controls simultaneously. TI can enable controls to drop malicious activity at the network perimeter before the activity even reaches enterprise firewall or IPS devices. Estimates of the size of the TI market vary, but analysts commonly value it at hundreds of millions of dollars a year and growing rapidly. Some analysts predict that in the next few Sizing the TI market years, the market for TI solutions will be well over $1 billion a year, with most enterprises adopting TI solutions as part of their foundational security suites.

19 Chapter 2 Gathering Threat Intelligence In This Chapter Understanding the sources of TI Reviewing automated intelligence Not all threat intelligence (TI) is created equal. Some sources of TI tend to produce higher quality results than others, and some sources specialize in finding particular categories of threats. Being educated on the subject of TI gathering is important, because it lets you make better decisions about acquiring TI products and services, as well as do your own assessment of the TI data that your organization receives. It s also a lot of fun (for geeks, anyway) to dive into the details! Data Gathering Locations TI is gathered from many locations, and each vendor uses a varied combination of locations, so different TI vendors produce somewhat different data sets. You should be familiar with the major data gathering locations and the relative pros and cons of each type so that you can evaluate service offerings.

20 14 Threat Intelligence For Dummies Existing data feeds Some TI vendors don t perform their own data collection and analysis. Rather, they reuse existing data feeds, often freely available, that they acquire from other parties. The big problem with this technique is that the vendor rarely knows how accurate and timely the information in each feed is; it simply merges a bunch of feeds, which doesn t necessarily make the situation any better. In fact, this practice usually muddles the metadata and destroys the context of each piece of information. Generally, reusing existing data feeds is recommended only if you can verify the integrity of each feed. Integrity includes both the initial composition of the data feed and its maintenance over time, such as dropping items that are no longer applicable. Internal customer networks Another common source of TI is the internal network of a TI product/service customer. Typically, the vendor installs TI products in a company s infrastructure and enables these products to monitor the company s internal networks at key points. This monitoring has two purposes: Detecting suspicious activity entering, transiting, or exiting the customer s network Sharing this information (sanitized, of course) with other customers to alert them to threats that they may see in the near future Unfortunately, this approach has some significant drawbacks: The vendor can see a customer s sensitive security information, such as which vulnerabilities an attacker has exploited and which of its hosts are compromised. The vendor is responsible for sanitizing all the information it collects. What happens if your TI vendor makes a mistake and accidentally reveals your compromise to your competitors? What happens if your TI vendor itself is compromised? Consider these questions before acquiring a TI product that involves monitoring your own networks.

21 Chapter 2: Gathering Threat Intelligence 15 External networks Another way to gather TI is to monitor external networks that is, key places on the Internet. We call them external networks to differentiate them from internal customer networks, described in the preceding section. External network monitoring allows a TI vendor to find attack traffic in many places on the Internet, not just on customer networks. This technique is more proactive than waiting until customers get attacked, and it provides a much more comprehensive picture of current threats, identifying thousands or even millions of attacking hosts around the world. The downside of monitoring external networks is the relative cost. A TI vendor deploys equipment and software in many locations, depending on how comprehensive a system the customer wants, and then monitors and maintains those deployments. The more points that are monitored, the more thorough and detailed the TI is, but the more expensive the system is to compile and maintain. The dynamic nature of TI Threat intelligence is incredibly dynamic. TI feeds need to be refreshed around the clock to stay relevant and accurate. Nowadays, many threats arise, spread, and disappear in hours or even minutes. This dynamism is a major reason why it can be counterproductive for a TI vendor to reuse existing data feeds instead of generating new feeds. Imagine that you re using data feeds without timing information. How do you know that the information was collected seconds or minutes ago, not weeks or months ago? You face two significant problems: Some of the threat information you collect is no longer accurate. A victimized host may have been cleaned up and patched since your data was collected, making it benign instead of malicious. You re missing out on information about the latest threats, so you have no way of stopping them. The best way to address these challenges is to ensure that you re always getting the latest TI in your data feeds.

22 16 Threat Intelligence For Dummies Automated Sources of TI Automated sources of TI are more easily used by your existing security controls compared to manual sources because the automated sources can be automatically updated. Automation allows vendors to detect new threats and communicate them to customers in a matter of minutes or even seconds and in security, a single second can make a difference. In this part of the chapter, we look at the most common automated sources of TI. A robust machine readable TI (MRTI) feed should include all these automated sources of TI, and more. A single source provides only a small piece of the overall picture. See Chapter 1 for details on MRTI. Anonymous proxies As described in Chapter 1, an anonymous proxy is a node on an anonymous proxy network. Such a network is designed to anonymize users computing activity so that their actions can t be traced to their origin. For this reason, many criminals use anonymous proxy networks to conceal their identities when they use the Internet to commit crimes. A single anonymous proxy network such as Tor may contain thousands of proxies. The identities of these proxies are known only to hosts that are using the proxy network. As a result, some TI vendors maintain a presence on these networks so that they can identify participating proxies. Just because a host uses an anonymous proxy network doesn t mean that it s performing malicious activity but the likelihood that malicious activity is associated with that host is much higher than average. Crawlers A crawler is a program that methodically scans web servers and other hosts to identify the content that they serve to others. Crawling every web server in the world would be

23 Chapter 2: Gathering Threat Intelligence 17 much too resource intensive for TI vendors comparable in scope to what major Internet search companies such as Google might do. Instead, a TI vendor can use crawlers to scan hosts that are suspected of harboring malicious content or to look for content related to an attack. Suppose that a server has been the source of several attacks detected over the past few days. It would be a good use of resources to crawl this server to determine whether the server is still distributing malware. Another effective use of crawlers is to check hosts on darknets and other suspicious networks for leaked confidential information, such as credit card numbers and medical records. Identification of such leaks indicates that the host in question is being used to store stolen information. Both these uses of crawlers create high confidence that the host is being used to perform malicious activities and that TI customers should avoid conducting incoming or outgoing transactions with it. Free services Some TI vendors provide free services to the Internet community as a whole, such as free and free Domain Name System (DNS) hosting. The latter service is particularly popular with attackers, who often need to acquire and stage a domain name in a hurry so that they can launch a phishing attack and then tear down the site in a matter of hours. By offering free services and monitoring their use carefully, TI vendors can detect malicious activity involving these services. Geolocation Geolocation generally refers to identifying the physical location of something. In the context of TI, geolocation refers to determining where a particular host is located based on its IP address or hostname and other characteristics. The granularity of this location may vary widely across TI products and services. Some products may report only the

24 18 Threat Intelligence For Dummies country of origin, whereas others also report the state or city; still others provide GPS coordinates with resolution of feet or meters. Geolocation information serves a few important purposes: It can identify hosts in countries that don t need to interact with the organization. Some organizations that have a limited clientele and particularly high security needs choose to block all activity involving certain geographic locations, but this posture tends to be rather extreme. TI customers commonly factor geolocation information into risk scores. Suppose that most of the attacks against an organization come from a particular city. When new suspicious activity involving that city is observed, it may make sense to give those threat indicators higher scores than usual because of their association with the city. Geolocation information can be used to correlate threats. Initially, threats may appear to come from the same geographic region, but a more detailed analysis may indicate that they re coming from the same building, suggesting a single attacker. Honeypots For many years, security researchers and security minded organizations have used honeypots hosts whose only purpose is to lure attackers and record their activities for analysis to detect malicious activity. An attacker (or malware acting on behalf of the attacker) scans a honeypot, thinks that it s a legitimate host, and attacks it. Although the honeypot may be configured to appear to contain valuable resources, it doesn t; it s simply a trap designed to record malicious activity. Honeypots are obvious ways to collect high fidelity TI. A vendor can set up honeypots at various places on the Internet, and any hosts that interact with the honeypots are considered suspicious. Analyzing honeypot activity can confirm whether suspicious activity is accidental (perhaps a user inadvertently transposed numbers in an IP address) or intentional (an attacker may have exploited vulnerabilities to gain unauthorized access to the honeypot host), or whether malware is propagating from host to host.

25 Chapter 2: Gathering Threat Intelligence 19 Using honeypots for your organization may sound like a great idea, but some organizations have policies that explicitly prohibit their use. Make sure to check with your organization s legal counsel before deploying honeypots. IANA and Internet registries The Internet Assigned Numbers Authority (IANA) oversees IP addresses globally (for more on this, see Chapter 1). IANA controls the high level allocation of IPv4 and IPv6 addresses by delegating the allocation of chunks of address space to authorized Internet registries, each of which then allocates IP addresses in a different region of the world. Only addresses that have been allocated by IANA to a registry and then allocated by that registry to an organization (such as an Internet service provider, company, or educational institution) may be used on the Internet. The use of other addresses on the Internet, such as reserved private addresses (such as the 10 net), indicates spoofed traffic, misconfigured network devices, a darknet, or other suspicious activity. IRC Another common source of TI is the Internet Relay Chat (IRC) protocol. IRC, which has been around for many years, provides a text based chat mechanism that works over the Internet in a standardized way. People around the world use thousands of IRC servers to communicate with one another. Unfortunately, attackers take advantage of the communications infrastructure provided by IRC servers. Many botnets leverage IRC channels to communicate with their bots, for example. By monitoring these channels, TI vendors can identify hosts that are bots or botnet command and control nodes, which give orders to and collect information from bots. P2P Peer to peer (P2P) networks are used for many reasons, both benign and malicious. The basic purpose of P2P is to allow people to share files. Unfortunately, many people use P2P to share illegal content, such as pirated music and software. P2P

26 20 Threat Intelligence For Dummies networks are also used to share data stolen during attacks on organizations. TI vendors can monitor P2P networks to determine which hosts are involved in providing or acquiring illegal content. Human sources of intelligence This chapter focuses almost exclusively on automated sources of intelligence. This focus isn t meant to downplay the value of human sources of intelligence only to recognize the fact that threats change so quickly that only automated sources have any chance of keeping up. A properly trained and skilled human, given access to MRTI, can expand on that information to produce a highly detailed report about a threat. Before MRTI was widely available, threat analysts had to rely solely on their own data gathering skills. One case involved an attack from a single IP address; two analysts working together traced that attack to a particular group and monitored that group s known communications channels on the Internet in an attempt to detect other imminent attacks.

27 Chapter 3 Scoring Threat Intelligence In This Chapter Assessing the quality of TI sources Understanding how risk scores are calculated Seeing how to use risk scores Threat intelligence (TI) isn t of any value if you can t make it actionable. It s one thing to know that a certain host has shown some malicious characteristics in the past; it s quite another thing to quantify the current level of confidence in the host s malicious nature. In this chapter, we dig into this quantification of confidence, which we simply call scoring. Every TI vendor has its own proprietary methods of scoring, but all methods are based on the same principles. Assessing Source Quality As we discuss in Chapter 2, TI comes from many possible sources. Quality is going to vary from source to source. Information from free services, for example, generally isn t as accurate as information from honeypots when it comes to identifying threats. Therefore, you d expect free services information to carry less weight in an overall score than honeypot information. For a single source, it s certainly possible to have different degrees of confidence in the different threat indicators it collects. Take geolocation data, for example. There may be a high degree of certainty about the location of one attacker but less confidence in the location of another, so you d expect to

28 22 Threat Intelligence For Dummies give more weight positive or negative to the data that has more certainty. Timeliness also matters. Recent information generally has greater weight than older information, all else being equal. What makes scoring so challenging, however, is that all else isn t always equal. The quality of sources can vary in many ways. How are you supposed to judge what s more important: a 2 minute old finding from Source A or a 6 hour old finding of a different type from Source B? Scores are used so that the TI vendor can make these comparisons and ultimately give you the user of the TI a rational foundation for making decisions. Calculating Scores As we mention in the preceding section, calculating scores is challenging. Scores are really a measure of relative risk posed by other hosts. Risk measurement has long been a controversial topic in security circles. There s no consensus, or even anything approaching consensus, about how to measure security risk except that risk measurement involves many factors. No magic formula exists for calculating risk scores. Each TI vendor has its own way of analyzing TI and determining the relative priority of each factor. Think of the factors as variables in a giant formula. Dozens or hundreds of variables could be used in a single formula, with high level categories such as these: Geography and geolocation Routing changes and IP registration validity Domain Name Service (DNS) reverse lookup for IP address Frequency of searches for a threat indicator Threat category (bot, anonymous proxy, and so on) Risk scoring does involve some common practices, however, and the TI products and services you use should follow them. We discuss these practices in the following sections.

29 Chapter 3: Scoring Threat Intelligence 23 Aging Score aging is the process of reassessing a score after a given period of time. Suppose that you see clearly malicious activity from a particular host, so you assign that host a score of 100 on a 0 to 100 scale. Now imagine that it s a week later, and you ve seen no subsequent malicious activity from this host. Are you still completely confident that the host is malicious? Maybe the host is a victim that has been cleaned up, or maybe the IP address has been reassigned. To reflect this gradual loss of confidence in the data, it s absolutely critical that risk scores be aged. Recent data, including lack of observations of new malicious activity, should be given greater weight than older data. Aging should occur repeatedly during the first few days after malicious activity is observed and can occur less often during subsequent days unless new malicious activity is observed, in which case the aging process should be restarted for the indicator in question. Score history Closely related to the aging process is maintaining a score history a record of all the previous scores for a particular indicator. Score history varies widely from vendor to vendor. Some vendors don t provide any history; others have data going back months or even years for each indicator. Score history itself can be a form of indicator metadata. Suppose that one host has no score history for the past four years except for a single brief period. Now suppose that another host has a long record of malicious activity. It probably makes sense to treat the second host as a higher risk than the first host, because it s been a malicious actor consistently for years on end. The other host may have suffered a brief compromise but doesn t show a pattern of malicious behavior.

30 24 Threat Intelligence For Dummies Using Scores You ve got a source of risk scores, and it s taking aging and score history into account. Now what? You re ready to use the scores, which means that you need to set a threshold. A threshold is the lowest score that matters to your organization. Every organization has a different threshold that makes sense based on its risk tolerances and the combination of threats it faces. TI vendors often suggest a threshold, but organizations should be prepared to alter it. Imagine an organization with low risk tolerance, such as a financial institution. It may choose to block financial transactions for a wide range of scores because of the possibility that these transactions are fraudulent. By contrast, an organization with high risk tolerance may permit transactions for all but the highest scoring indicators. Setting a threshold involves determining what level of false positives and false negatives is acceptable how many benign transactions can be blocked and how many malicious transactions can be permitted. Generally, setting a threshold that lowers false positives increases false negatives, and vice versa. When you deploy TI technology, you may need to adopt a high threshold at first and gradually lower it as evidence indicates its appropriateness. You can find out more about risk management from experts in fields such as insurance and the actuarial sciences. Countless methodologies and formulas are available, not to mention many books, standards, and other documents. One standard of particular interest is International Organization for Standardization (ISO) 31000, which provides foundational, Risk research non IT specific information on risk management. ISO is available at standards/iso31000.htm. For IT specific information, the National Institute of Standards and Technology (NIST) provides a Risk Management Framework page at gov/groups/sma/fisma/ framework.html.

31 Chapter 4 Supporting Incident Response In This Chapter Understanding the need for TI to support incident response Using MRTI to enhance incident detection and response Using console based TI to speed incident investigations Using TI reports to dig deep into an attacker s characteristics Using multiple forms of TI to optimize incident response This chapter shows you how to apply threat intelligence (TI) principles to real world situations. Incident response is a critical capability for organizations, because security incidents can cause untold damage in a short time. Think of all the major data breaches you ve heard about. Now think about how different things would have been if those breaches had been stopped before sensitive data was stolen. TI is invaluable for incident response. The forms of TI most often used to support incident response efforts are machinereadable TI (MRTI), console based TI, and TI reports. Used alone at times and in combination at other times, each of these forms of TI plays its own role in incident response. Recognizing the Need to Support Incident Response TI is critical to incident response because it provides information you can t get in any other way. It can tell you

32 26 Threat Intelligence For Dummies how suspicious a certain host is, for example, and it can tell you the threat life story of a host of interest. This section discusses the most common reasons to use TI to support incident response. Improving incident detection TI can improve the detection of incidents in multiple ways: TI can speed incident detection by indicating which suspicious activity is linked to known malicious hosts. An example is having MRTI ingested by your security information and event management (SIEM) system so that when analysts review suspicious log entries, they have context provided by the MRTI. TI can find compromised systems within your own organization, such as malware infected hosts that are acting as bots. You can identify these hosts by regularly checking MRTI for your organization s external IP addresses. MRTI can provide real time information about attacks on other organizations. This information allows you to block the hosts at the source of these attacks from accessing your networks, thereby preventing breaches before they occur. Strictly speaking, this information isn t incident detection, but it reduces the noise that analysts have to filter to find serious incidents. Reducing loads on existing devices MRTI can reduce the loads on existing network security infrastructure devices, such as firewalls, intrusion prevention systems (IPSs), and SIEM appliances. All traffic that has highscoring indicators can be dropped automatically, for example. This strategy relieves your security devices of the task of indepth inspection of malicious traffic, thus optimizing the performance of those devices.

33 Chapter 4: Supporting Incident Response 27 Facilitating forensic investigations TI can reduce the workload of your incident response team, especially in forensic investigations, which involve locating the source of an attack and finding out more about that source. TI is a natural aid in this process particularly console based TI for normal cases and TI reports for special situations. Using MRTI MRTI can aid incident response in several ways. Figure 4-1 shows one common architecture for incident response, which works as follows: 1. MRTI is transferred regularly from a TI vendor to your SIEM solution. 2. An attacker launches attacks against your organization, and the SIEM records this activity. 3. The SIEM does its standard analysis of events and correlates suspicious activity with the attacker s IP address, which was included in the latest MRTI update. 4. The SIEM acts appropriately to stop the activity, such as alerting a human to intervene or directing the firewall to block the connection. Figure 4-1: Sample architecture for using MRTI for incident response.

34 28 Threat Intelligence For Dummies Another common use of MRTI is to prevent incidents from occurring. See Chapter 5 for more information. Using Console Based TI Console based TI and MRTI use similar architectures. All that s added in console based TI is an analyst console, which can interact with the SIEM server. Should an analyst want to gather more information about a particular host or URL that the server reports as suspicious, he or she can issue a query through the console, and the console displays a score and metadata history for that host or URL. This information can be invaluable in identifying the attacker and his or her likely motive. The console doesn t have to be linked to the SIEM; it can be stand-alone, interacting directly with the TI vendor s servers. Although many organizations have both an MRTI service and a console based TI service, others have just one or the other. Single service organizations tend to have less mature threat intelligence gathering infrastructure. Using TI Reports TI reports provide the most detailed information available about a particular threat. TI reports may leverage some MRTI and console based TI, but for the most part, they re compiled manually, in narrative format. They re particularly helpful for providing insights into an attacker s background, motivations, capabilities, interest in the organization, and the like. Using Multiple TI Forms Together Organizations often find that incident response is most efficient and effective when they combine multiple forms of TI. Continuous analysis may occur behind the scenes via MRTI, with humans using console based TI and TI reports as merited to support particular investigations. The results of the combined analysis can be used to mitigate threats.

35 Chapter 4: Supporting Incident Response 29 Ideally, mitigation stops threats before they cause damage. In reality, mitigation often involves cleaning up after an incident, such as stopping additional malicious activity. Manual mitigation Manual mitigation is the most basic form of mitigation. In this process, an analyst suggests mitigation measures, such as asking a firewall administrator to block all connections involving a particular IP address. The sole advantage of manual mitigation over automatic mitigation (see the next section) is that manual mitigation is potentially less likely to cause benign activity to be blocked assuming that the analyst is skilled and properly trained. In manual mitigation, the analyst can access consolebased TI and (in some cases) TI reports to gather more information about an attacker before deciding how to handle the threat. Manual mitigation has several disadvantages. For one, it s very slow; an attack may be over and the damage done before manual mitigation can begin. Also, it s quite easy for an attacker to evade manual mitigation. If all traffic from one IP address is blocked, the attacker can simply change the host s IP address or use a different host. Automatic mitigation The alternative to manual mitigation is automatic mitigation, in which the network security infrastructure is empowered to stop detected threats without human decision making and intervention. A simple example is a network IPS configured to block any attacks that it detects. This IPS could be receiving MRTI feeds and using this information to identify attacks with a greater degree of certainty. More complex automatic mitigation is possible. A SIEM could be configured to analyze events from a wide variety of security controls, for example, and to correlate these events to identify threats. SIEMs commonly receive MRTI feeds (as in the IPS example discussed previously) so that they can determine when an incident is occurring with greater accuracy.

36 30 Threat Intelligence For Dummies When the SIEM identifies an incident, in some environments it can reconfigure selected security controls such as networkbased firewalls and IPSs to block related activity. Chapter 5 discusses automatic mitigation in more detail. Empowering incident response with TI A large power and energy utility faced a variety of challenges in responding to security incidents, such as reducing the number of incidents and saving time in incident investigation. The utility already had a basic network security infrastructure (firewalls, IPSs, antivirus servers, and the like), as well as incident response processes and personnel. What it was missing was TI to make that security infrastructure function more effectively and efficiently. The company selected Norse s ( Darklist and DarkViking services. Darklist provided MRTI scoring for millions of malicious hosts; DarkViking provided on demand long term scores and metadata history for hosts. These services integrated easily into the utility s existing infrastructure. The number of incidents dropped immediately, because the security infrastructure used MRTI to block the most suspicious connection attempts automatically. Incident response speed also increased. Malicious activity previously hidden by SSL/TLS was readily identified by suspicious IP addresses. Likewise, insiders carrying out malicious activity were detected when they used suspicious external hosts. A final benefit of using the Norse services was that the metadata captured in the MRTI was highly useful for training junior analysts to understand and compare potential threats.

37 Chapter 5 Strengthening Threat Mitigation In This Chapter Understanding how TI strengthens threat mitigation Mitigating threats through an existing perimeter device Mitigating threats through a dedicated TI appliance As discussed in Chapter 4, mitigation involves stopping threats. This chapter focuses on using automatic mitigation to prevent threats. By strengthening threat mitigation, you can move from a mind-set of containing damage and investigating successful breaches to a mind-set of stopping threats before they succeed. We don t mean to imply that threat intelligence (TI) can prevent all incidents from occurring. It can t. Nothing can. But TI can help. Exploring Strategies for Strong Threat Mitigation Strong threat mitigation reduces the effect and damage caused by successful compromises, as discussed in the following sections.

38 32 Threat Intelligence For Dummies Blocking attacks Machine readable TI (MRTI) can be invaluable for preventing attacks from hitting your organization. Assuming that the MRTI vendor monitors many points on the Internet, this process can work in a few ways: The vendor sees many attacks in progress against other organizations and provides your organization timely MRTI about these attacks. With this information, your network security infrastructure can stop the same hosts from attacking your organization by blocking their connection attempts. (We like to call this process community immunity.) The vendor detects bots and botnets (see Chapter 1), which an attacker could use at any time for malicious purposes, such as attacking other hosts. The vendor lists the compromised hosts in its MRTI feed to your organization, which then can block connection attempts from these hosts. The vendor detects hosts running as anonymous proxies and lists them in its MRTI feed. This information allows your organization to prohibit any connection attempts from these proxies because they may involve malicious activity. Depending on the sources used by your MRTI vendor, you may have additional ways of blocking attacks. Ask prospective MRTI vendors about their blocking capabilities. Chapter 6 provides a few specific features to consider. Improving catch rates Catch rates refer to the likelihood of identifying attacks in progress. MRTI can be very helpful in improving your organization s catch rates, because it provides additional information about hosts that may be attempting to attack your environment. Suppose that your security information and event management (SIEM) system is analyzing events to identify potential incidents. MRTI certainly would help your SIEM make better decisions, because the SIEM could access the

39 Chapter 5: Strengthening Threat Mitigation 33 score of each host of concern. A high score may raise the profile of a particular event, perhaps bringing it to the attention of a human analyst. Also, a high score may trigger automatic mitigation, such as blocking communications from a suspected attacking host or a suspected victim host so that any compromise of that host can t spread. Firewalls, intrusion prevention systems (IPSs), and other security devices can use score information to detect attacks in progress, but SIEMs are the most common consumers of MRTI feeds. Stopping advanced attacks Another reason to strengthen threat mitigation is to stop advanced attacks cyberthreats that evade your traditional signature based security defenses, because they ve never been seen before. Advanced attacks are often associated with advanced persistent threats (APTs) and zero day attacks and are delivered through a variety of means, including spearphishing, drive by downloads, and watering hole sites. Some advanced attacks can be detected by security solutions designed to analyze suspicious files in the safety of a sandbox. However, these products typically trigger alerts after malware has reached its intended target following completion of sandbox analysis. By then, the endpoint is already infected. What you really need is something that can help stop advanced attacks before they have the chance to compromise their targets. That something is MRTI. By using MRTI to block potential bad actors hosts using anonymous proxies, hosts participating in botnets, hosts observed attacking other hosts, and so on you can prevent all attacks from these hosts, including advanced attacks. No security control is 100 percent effective. MRTI can significantly reduce the success of advanced attacks, but attackers may be able to evade inclusion in some MRTI feeds by camouflaging their activity, such as by frequently changing the hosts they use. Also, insider threats may not be networkbased, so network security controls may not detect them.

40 34 Threat Intelligence For Dummies Leveraging Existing Devices The traditional method of using MRTI to strengthen threat mitigation involves feeding that MRTI into existing perimeter devices, such as a firewall or an IPS that blocks incoming and outgoing traffic. Architecture Figure 5-1 shows a common architecture, with the organization s firewall at its network perimeter. Figure 5-1: Sample architecture for an existing perimeter device. This architecture works as follows: 1. The MRTI vendor sends regular data feeds to the firewall, which ingests them automatically. 2. An attacker targets the organization and attacks one of the organization s hosts. 3. The attack attempts to pass through the firewall protecting the host, but the firewall recognizes the source IP address as being high scoring in the latest MRTI feed that is, scoring above the blocking threshold. 4. The firewall blocks the incoming connection, preventing the attack. A unified threat management (UTM) device or another network security infrastructure device that takes the place of a firewall can use the same architecture.

41 Chapter 5: Strengthening Threat Mitigation 35 Pros of using existing devices This architecture has several positive aspects, including the following: By using MRTI information, the perimeter device can block connection attempts from known malicious hosts, preventing attacks before they start. The perimeter device can use MRTI data to analyze existing connections. If a connection from a newly identified malicious host is found, the connection can be terminated before more damage is done. This architecture takes the load off other network security devices. Blocking malicious connection attempts means that IPSs and other security devices have fewer attacks to handle. In some organizations that use this type of architecture, security devices don t need to be replaced as quickly as before because their workloads are lighter. Cons of using existing devices Unfortunately, importing MRTI into existing perimeter devices isn t always viable. Some devices can t ingest MRTI feeds, which is clearly a showstopper. The most common problem with leveraging existing perimeter devices, however, is resource limitations, particularly with older devices: Perimeter devices may be able to hold only a small number of indicators from an MRTI feed because of hardcoded limitations. Suppose that an organization wants to block everything with a score above 90 (on a 100 point scale). The firewall may hold only 10,000 indicators, which wouldn t even include all the indicators with a score of 100. Although an old perimeter device may be able to hold millions of indicators, checking every connection against the indicator list may take too much processing power. This limitation could slow the performance of the perimeter device, causing unacceptable levels of packet latency. On the other hand, using MRTI to winnow out malicious traffic before in depth analysis is performed on that traffic may actually save processing power.

42 36 Threat Intelligence For Dummies The phrase millions of indicators may sound extreme, but it s not. Millions of hosts throughout the Internet are compromised or acting suspiciously at any given time. The highest quality MRTI feeds attempt to identify as many of these hosts as possible. When evaluating prospective vendors, ask about the comprehensiveness of their MRTI feeds. Existing perimeter devices may not be able to keep up with MRTI feed updates, depending on how the feed is delivered and how often updates occur. Delivering the entire MRTI feed over and over to a device may be impractical; instead, application programming interfaces (APIs) are used to deliver incremental updates. Many perimeter devices don t have the limitations we discuss in this section, in which case there are no major drawbacks to using MRTI with them. Taking stock of incidents A major stock exchange with traditional network security controls wanted to reduce the number of incidents occurring on its systems. After analyzing incident patterns, the stock exchange decided to incorporate MRTI into its security infrastructure. It acquired a subscription to the Norse ( Darklist service, which provides MRTI scores and metadata for millions of suspicious hosts. Integrating the MRTI feeds into the existing infrastructure was straightforward. The stock exchange chose to have its IPS sensors deployed inline, outside the perimeter firewalls, as the recipients of the feeds. These sensors blocked both incoming and outgoing connection attempts from hosts listed in the MRTI feeds, based on a threshold of 95 on a 100 point scale. Over the course of a year, the IPS sensors using Norse Darklist blocked 12 million malicious source addresses from establishing connections to the stock exchange s systems, and firewall connections were reduced by 20 percent. Also, the stock exchange identified internal hosts that were already infected with malware and blocked these hosts from communicating with botnet servers, thus limiting the damage caused by these infections.

43 Chapter 5: Strengthening Threat Mitigation 37 Using a Dedicated TI Appliance The alternative to importing MRTI into existing perimeter devices is deploying a dedicated TI appliance a specialized device that ingests MRTI feeds as frequently as they re available (often every few seconds). The appliance monitors connection attempts to determine whether they involve a threat indicator score at or above the blocking threshold, and it blocks suspicious connections to prevent attacks and limit damage. The appliance also checks existing connections against the latest MRTI updates to see whether the connection is related to a new indicator. Architecture TI appliances typically are deployed in front of or behind perimeter firewall devices the same places where you d deploy IPS sensors. Figure 5-2 shows one common architecture, with the appliance behind the firewall. Figure 5-2: Sample architecture for a dedicated TI appliance. This architecture works as follows: 1. The MRTI vendor sends regular data feeds to the appliance, which ingests them automatically. 2. An employee within the organization tries to connect to an external website that happens to house malware.

44 38 Threat Intelligence For Dummies 3. The connection attempt reaches the appliance, which identifies the destination IP address as being an infected website in the latest MRTI feed. 4. The appliance blocks the outgoing connection attempt, preventing the malware from infecting the internal host. This architecture also blocks incoming connection attempts from malicious hosts. Pros of using a dedicated appliance This architecture offers the same advantages as using MRTI with existing security devices (see Pros of using existing devices earlier in this chapter). An additional, major advantage of using a dedicated appliance is that it places no additional burden on any existing perimeter devices. In fact, the appliance typically reduces the burden on these devices because it blocks suspicious connection attempts and connections. Another benefit of a dedicated appliance is that it is designed to utilize all the TI data and metadata. Existing security devices typically can use some, but not all, of the data and metadata. Cons of using a dedicated appliance The drawback, of course, of using a dedicated appliance is the cost of acquiring, installing, and maintaining the appliance itself. This cost may be amplified for organizations with redundant, high-availability network designs. However, most organizations realize a positive return on investment by prolonging the replacement of their existing security devices for performance or efficacy reasons, as described in the preceding section.

45 Chapter 6 Ten Buying Criteria for TI Solutions In This Chapter Evaluating TI products and services Reviewing buying criteria This chapter discusses what to seek and what to avoid when you re evaluating dedicated threat intelligence (TI) appliances, machine readable TI (MRTI) feeds, and consolebased TI offerings. Here are ten attributes to consider. Automation The vast majority of data collection and analysis performed by a TI vendor should be done in real time, via automation. If humans are involved in doing a substantial amount of the analysis, the TI won t be available to customers quickly enough to stop emerging threats. Furthermore, your TI vendor should automatically update its TI appliances, TI consoles, and MRTI feeds throughout the day. No human action should be required to receive TI updates. Integration and Interoperability A TI product or service is of little value if it can t be readily used with other security controls, such as security information and event management (SIEM) servers, intrusion prevention

46 40 Threat Intelligence For Dummies systems (IPSs), firewalls, unified threat management (UTM) devices, and/or custom security applications and databases. Be wary of TI offerings that can only be used with one or two vendors hardware or software, because you may end up locked into solutions that you don t necessarily want in the long term. Instead, look for TI solutions that offer a robust application programming interface (API) with all the functionality that you need. The API should be well documented so that it s easy to integrate into existing security controls. Frequency of MRTI Updates Be sure to consider two aspects of update frequency: How often the vendor updates its TI: Find out how often each indicator s score and metadata is updated within the MRTI feed. If you re considering a console based TI service, find out how often its TI is updated. How the vendor provides updates: Find out how often your organization s systems SIEMs, firewalls, IPSs, TI appliances, and the like will receive MRTI updates. The more often MRTI is updated, the smaller the threat window will be. Ideally, updated MRTI is delivered to customers live, or at least every few minutes. Metadata Richness The metadata associated with the TI offering you choose should provide a wealth of contextual information that can be used for better decision making. This information should be usable in both automated policies and human analysis, because some use cases will present gray areas that require a human analyst to intervene. Metadata fields that you should expect in every TI offering include risk scores, threat types, time stamps, GPS independent geolocation, and indicator categories. Additional metadata may be helpful but usually isn t necessary.

47 Chapter 6: Ten Buying Criteria for TI Solutions 41 Sophistication of Scoring One of the most important criteria for evaluating TI products and services is scoring sophistication. The best TI vendors use actuarial techniques adapted from the insurance industry to derive mathematically sound risk scores. Consider how many variables and which variables are considered in the scoring. Odds are that vendors won t reveal all the details on their scoring (let alone their algorithms), but they should be willing to share basic information about the variables they consider. Another important factor in risk scoring is aging. As discussed in Chapter 3, you should avoid products and services that don t incorporate aging in their calculations because threat data becomes inaccurate quickly. Threat Coverage TI is like astronomy: No telescope can cover the entire sky in all wavelengths all the time. Different astronomical observatories focus on gathering different information sets (for instance, planet hunting or x rays emitted from black holes). Likewise, no TI vendor can provide 100 percent coverage of all possible threats on the Internet. Even if a TI vendor got close to 100 percent coverage by deploying millions of monitoring points all over the world, it s unlikely that every single possible path on the Internet could be monitored at all times for all threats. Also, because new threats emerge all the time, a window would always be open between the emergence of a new threat and its detection. All that said, the TI vendor you select should provide broad threat coverage on a global scale. You can use a variety of ways to quantify coverage, including the following: Percentage of Internet IP addresses monitored Terabytes of traffic monitored per day Number of honeypots deployed and maintained

48 42 Threat Intelligence For Dummies Visibility into Darknets TI vendors vary in their visibility into darknets, such as in the identification of anonymous proxies and the malicious hosts hidden behind these proxies. Darknets are major sources of attacks, so any TI product or service that doesn t include TI derived from darknets is significantly deficient. Geolocation Accuracy As discussed in Chapter 2, GPS independent geolocation is an important part of TI. The more accurate geolocation information is, the more powerful it is for identifying threats and quantifying their relative risk. Look for highly granular, up to date geolocation information when selecting TI products. Variety and Number of TI Sources Having a wide variety and large number of TI sources provides a much more comprehensive picture of threats. Chapter 2 discusses several categories of automated sources of TI. Ideally, your MRTI feed should include numerous sources from each category. TI vendors may not be willing to share all the details on their TI sources, for obvious reasons, but they should be willing to disclose the types of sources they use and to provide rough estimates of the number of sources in some categories. It would be reasonable for a TI vendor to say that it monitors hundreds of thousands of anonymous proxies, for example. Quality of TI Sources Although the variety and number of TI sources are important distinctions among products, so is the quality of the TI sources. If the sources aren t producing high quality results, there s little point in using them.

49 Glossary anonymous proxy: A node on an anonymous proxy network. See also anonymous proxy network. anonymous proxy network: A network designed to anonymize computing activity so that users actions can t be traced back to their origins. Tor is an example of an anonymous proxy network. attack: An instance of malicious activity. An example of an attack is malware directed at a target. attacker: A person or group that attacks others; also called a threat agent. blacklist: A list of IP addresses, executables, or hosts that are considered to be malicious. Traditional blacklists typically lack metadata. catch rate: The likelihood of identifying malicious traffic or an attack in progress. console based threat intelligence (TI): MRTI behind a humanfriendly console interface. See also machine readable threat intelligence (MRTI). crawler: A program that methodically scans web servers and other hosts to identify the content they serve to others. darknet: An underground (often illicit) website or other hidden IT resource available on the Internet. Typically, darknets are hidden behind anonymous proxies. See also anonymous proxy. geolocation: In the context of TI, determining a host s location without trusting the host s self professed GPS coordinates. TI geolocation uses triangulation based on pings from trusted servers with known locations. honeypot: A host designed to lure attackers and record their activities for analysis.

50 44 Threat Intelligence For Dummies machine readable threat intelligence (MRTI): High volume threat intelligence specifically formatted for high speed (automated) use. MRTI typically consists of indicators and indicator metadata, which provide context. See also threat intelligence (TI). metadata: In the context of TI, data about threat indicators, such as time stamps, scores, sources, and categories. See also threat indicator. risk score: An actuarial calculation of the relative maliciousness of a given IP address, URL, host, or executable. A risk score can be based on thousands of variables. Also called a score. score aging: The process of reassessing a risk score after a given period of time. See also risk score. score history: A record of all the previous risk scores for a particular threat indicator. See also risk score and threat indicator. threat: An IT entity, such as a host or website, suspected of performing attacks. threat indicator: One or more related data points that imply heightened risk: for example, an IP address located in a specific country. Also called an indicator. threat intelligence (TI): Insights into threats gained by collecting evidence, such as observing attacks and studying the characteristics of attackers. threshold: A lower limit risk score configured by an organization to block threats or trigger alerts whenever the limit is met or exceeded. See also risk score. TI appliance: A dedicated hardware device designed to ingest real time MRTI and use it to detect and block malicious activity. See also machine readable threat intelligence (MRTI). TI report: A white paper or booklike report on a particular threat, written by human TI analysts.

51

52