THE UNITED STATES NAVAL WAR COLLEGE. National Security Affairs Department

Transcription

1 THE UNITED STATES NAVAL WAR COLLEGE National Security Affairs Department Theater Security Decision Making Course Extract from ANALYZING INTELLIGENCE: NATIONAL SECURITY PRACTITIONER'S PERSPECTIVES by James A. Bruce and Roger Z. George "Intelligence Analysis: What is it - What Does it Take?" Reprinted by permission from Analyzing Intelligence: National Security Practitioner's Perspectives, by James A. Bruce and Roger Z. George, pp. 1-6; Copyright 2014 Georgetown University Press. Published by Georgetown University Press. TSDM Policy 19-2

2 CHAPTER I Intelligence Analysis What Is It-and What Does It Take? JAMES B. BRUCE AND ROGER Z. GEORGE [The takedown of Osama bin Laden was] a perfect fusion of intelligence collection, intelligence analysis, and military operations. -Robert M. Gates, Sixty Minutes, May 15, 2011 n that often hidden nexus between collecting intelligence and conducting operations, I analysis, as former analyst, CIA director, and defense secretary Gates reminds us, plays a vital role. When analysis is good, it improves the chances that operations or policy will succeed. When analysis is not good, it degrades the chances of successful operational or policy outcomes. If your job is to produce intelligence analysis, you have powerful incentives to get it right. This book is about getting it right. Producing good analysis is hard. There are plenty of obstacles in its path. Good analysts often overcome them. Empowered by solid training, education, and experience, and perhaps seasoned with previous analytic failures and other "teaching moments" on their path to professionalization, they understand how to improve the odds favoring good analysis. The contributors to this book have all trod that path. What Is Analysis? Analysis, good or bad, is about producing judgments, forecasts, and insights. A word about each: judgment. Analysis, as explained in a recent authoritative study, is "an exercise in judgment under conditions of uncertainty." 1 A judgment is a conclusion or inference based on analysis of incomplete and uncertain information, with some generally bounded probability (never really known and not always stated) of being correct, but also with some chance of being wrong. Analytic judgments can be expressed with some degree of probability ("there is a 70 percent chance that...'') or a statement of confidence ("we have moderate confidence that X is... "). Analytic judgments are typically expressed without accompanying statements of probability or confidence.

3 2 Intelligence Analysis Intelligence analysts who concluded that the obscure compound in Abbottabad, Pakistan, housed Osama bin Laden, and who also forecast that he would be there during the time of the planned takedown, were exercising judgment under uncertainty. No one had conclusive proof or "smoking gun evidence" that the compound was his or that he would actually be there when the raid was conducted. These were judgments. The process that produced them was analysis. This analysis was based in part on good intelligence collection, the reliability of which was also critical to the successful takedown operation. In examples discussed later in this chapter, we examine two specific judgments under uncertainty: the state of the Soviet nuclear missile launch readiness during the 1962 Cuban Missile Crisis and whether weapons of mass destruction (WMDs) were present in Iraq in In the first instance, analysis was good; in the second, it was poor. Throughout this book, the authors seek to explain how to achieve the good and avert the poor. Forecast. A forecast is a judgment about the future. Warning intelligence is composed principally of forecasts, which are not "predictions." Customers of intelligencepolicymakers and military leaders-care especially about what will happen and less about what has happened already, which is too late to influence. Through their policy initiatives and military operations, they are trying to accomplish something that will reduce threats or otherwise enhance US national security. Thus they need better understanding about what is coming, rather than about what has happened. Intelligence analysis that produces good forecasts helps them achieve that. Good forecasting is hard and the record is mixed. For example, if policymakers do not know that a staunch ally in the Middle East is about to be overthrown by virulent anti-american forces, then they cannot take actions to prevent it or even mitigate its effects, as happened in Iran in Some aspects of the Arab Spring of 2011 were reliably forecast, and some were not. Still, policymakers are often successfully warned in advance, such as in the 1967 Arab-Israeli War, outbreaks of military tensions between India and Pakistan, and the breakup of Yugoslavia-all cases where analysts provided sound forecasts that were helpful to policymakers. Analysts are often more successful when forecasting multiple scenarios, with indicators expected to accompany each. These also help policymakers anticipate alternative outcomes and hypothesize how different policy options might influence different outcomes. Insight. When customers of intelligence receive analysis that offers a fresh, new perspective, or when it causes them to think about a hard problem in a new way-even if it does not present any new information-they appreciate the insight that analysis has brought them. Analytic insights are less about facts than they are about contextualizing them. For example, it is insightful for policymakers and military commanders to learn when a conventional war is metamorphosing into a counterinsurgency war, as happened in Iraq in The earlier they can get this kind of insight, the better they can adjust to a changing situation. This particular insight about counterinsurgency in Iraq came late in the game for policymakers and military planners. That it was provided much sooner than it was acted upon illustrates a separate problem for analysis: Pdncymakers do not have

4 Intelligence Analysis 3 to use intelligence. Th'ey make their decisions for lots of reasons, and sometimes intelligence-even when at its best-makes no difference at all. Subsequently, we explain this complex intelligence-policy relationship in greater depth. 3 It affects the use that intelligence customers make-or don't make-of the judgments, forecasts, and insights that analysts provide them. Analysis:The Cognitive Part of Intelligence Slightly more than half a century ago, the American scholar and pioneering intelligence analyst Sherman Kent lamented that the US intelligence community (IC) lacked a professional literature. 4 Serving as the head of the Central Intelligence Agency's (CIA's) Office of National Estimates, Kent hoped to define and develop a professional intelligence analysis discipline, noting that academic professions could not operate without an understanding of the field or a comparable body of knowledge. Today, though there is a large body of writing on intelligence generally, critiques to advance analysis still seem in short supply. To be sure, many critical writers have concentrated on the past and current failings of intelligence or seek to expose sensational operations to excite or infuriate the public, while former intelligence officers and policy officials are often tempted to put the record "straight" as they see it. However, both approaches have typically neglected assessing the discipline of "intelligence analysis" or adding to the collective knowledge of what constitutes sound analytic principles and practices. Is there even a professional discipline known as "intelligence analysis?" Considerable effort has been devoted to defining what is meant by the general term "intelligence;' which surely encompasses analysis as one part of a multifaceted process of gaining specific, often s~cret information for government use. 5 Analysis is the thinking part of the intelligence process, or as the former career analyst and senior official Douglas MacEachin has phrased it, "intelligence is a profession of cognition." 6 It is all about monitoring important countries, trends, people, events, and other phenomena, and identifying patterns or anomalies in behavior and cause-effect relationships among key factors that explain past outcomes and might point to future developments with policy implications for the United States. Another key founder of CIA analytic practices and principles has phrased it succinctly: "The mission of intelligence analysts is to apply in-depth substantive expertise, all-source information, and tough-minded tradecraft to produce assessments that provide distinctive value-added to policy clients' efforts to protect and advance U.S. security interests." 7 Analysis is just one part but, ultimately in our view, the decisive part of the intelligence process that produces "decision advantage" for policymakers. 8 The typical diagram of the intelligence cycle found in figure 1.1 exemplifies how many see the intelligence process. It starts with identifying what the customer needs (requirements) and ends with delivering the intelligence (dissemination) to satisfy those needs. 9 Despite its simplification of what is a very complex process, this conceptualization does underline the analyst's pivotal role in transforming information provided by various collection systems into

5 4 Intelligence Analysis Analysis and Production Dissemination Types of Intelligence Human Intelligence Signals Intelligence Imagery and Geospat1al Intelligence Measurement and Signature Intelligence Open-Source Intelligence Other Technical Intelligence Customers The nation's leaders, policymakers, armed ~' ~~~-=--~.,! forces, homeland defense, Ill and law enforcement Requirements Figure I. I Intelligence Cycle Source: Adapted from a briefing. The lnte/11gence Community, available at the director of national intelligence website ( judgments, forecasts, and insights for the policy customer. Whether that information is good, bad, or somewhere in between, the analyst must transform it into value-added information that is relevant and useful for the policymaker in a way that provides him or her with decision-making advantages not otherwise available. This analysis comes in a variety of forms. Traditionally one thinks of products "finished intelligence" analyses-that are printed and distributed to select government users. This definition of analysis conveys, however, a mechanistic and also somewhat linear process, which figure 1.1 represents. The "production line" metaphor conjures up an image of analysts writing, reviewing, editing, and publishing an assessment, and then moving onto the next question or task. While this is true at a basic level, the process is vastly more complex. In reality, the cognitive part of analysis is more akin to a computer model that has been collecting and interpreting incoming data and constantly reassessing how new data might change not only the findings but also the computer model being used to organize and interpret the data. The forms that analysis can take, then, are not limited to the printed or electronic word or graphic. As often, "analysis" occurs when analysts interact with policymakers over the telephone, through , during a videoconference, or at a meeting. This form of intelligence support has been referred to as "analytic transactions.'' Though impossible to quantify, perhaps tens of thousands of such transactions occur yearly. 10 Moreover, the sharing of data, hypotheses, interpretations, and questions among analysts and other nongovernment experts is possibly where the most insightful cognition is occurring, rather than on the page of a finished assessment or a PowerPoint slide. \ \

6 lnte/11gence Analysis 5 The Professional Analyst-What It Takes The analytic process, then, must be understood as demanding more than just a well-educated individual who can write concisely. The complete intelligence analyst must combine the skills of historian, journalist, research methodologist, collection manager, and skeptic. He or she must synthesize the skills of a subject-matter expert (SME) with those of an expert in intelligence itself At a minimum, the fully qualified analyst must demonstrate a very unique skill set combined from the two groups below: Skills of a university-trained SME: Substantive mastery of specific subject-matter content and good understanding of its relevance and implications for US national security policies. Skills in the use of social science research methods to organize and evaluate opensource data and information, including open-source intelligence (OS INT) research skills. Research imagination and scientific rigor to generate ~nd test hypotheses, most often with qualitative data. Skills of an expert in the conduct of intelligence: Collection. An understanding of three of the four clandestine collection disciplines most important to one's analytic portfolio: human intelligence (HUMINT), signals intelligence (SIGINT), geospatial intelligence (GEOINT), and measurement and signatures intelligence (MASINT). Cognition and tradecrafr. An understanding of the cognitive processes inherent in conducting analysis, how the cognitive function may affect the analytic process itself, and how structured analytic tradecrafr can mitigate cognitive error, including why openness to contrary opinion and "alternative" analysis also improve analytic outcomes. Foreign denial and deception (D&D ). An understanding of how US adversaries and "intelligence targets" develop countermeasures to collection efforts in order to deny the information sought or to manipulate that information for deception purposes, and the analytic implications of successful D&D countermeasures. Learning from practice. The capacity to admit error and especially to learn from it, including "best practices" that will give valid and reliable results over the long haul. Collaboration. The ability to work successfully in collaborative environments, both virtually and directly, with intelligence professionals from diverse agencies and with customers of analysis. Thus what distinguishes an intelligence analyst from a subject-matter expert outside the intelligence community are not the first three characteristics, which are shared with many international affairs specialists, although these attributes are required in intelligence. Indeed, the IC needs to further develop these important skills in its analysts.

7 6 Intelligence Analysis SMEs are expected to be well versed in the history, politics, culture, and language of many countries or be technical experts in a wide variety of areas. They may also be plugged into US policy deliberations and even involved in advising government officials on the optimal policies to adopt. And many foreign affairs specialists have methodological expertise. Where the intelligence analyst distinguishes himself or herself is not only in having these characteristics but also in having the other five. Fully professional analysts must be experts on how to use intelligence-collection capabilities and know their limitations, and be both imaginative and rigorous in considering explanations for missing, ambiguous, and often contradictory or deceptive data. At the same time, they need to be skilled in methodologically reliable analytic tradecraft and be able to be self-critics of their own biases and expectations of what the data show. And, most important, they must be open to changing their minds, learning from experience, and consciously asking the question "If I'm wrong, how might I need to modify the way I am analyzing the problem?" Professional growth requires the ability to learn from both errors and successes-one's own as well as others' -and mastering best practices that will improve the odds of successful analysis. Finally, the increased use of group-based structured analytic techniques, and the new IC environment emerging after the Intelligence Reform and Terrorism Prevention Act of 2004 (IRTPA) that created the Office of the Director of National Intelligence (ODNI), increased the importance of collaboration among analysts within their agencies and among then. Analysis today is a highly collaborative activity, and good skills in analytic collaboration will enhance the odds of success. A Growing Analysis Literature As of 2014-nearly fifty years after Kent's lament-the body of scholarly writing on intelligence analysis remains surprisingly thin but growing. It is true that academics and intelligence professionals have seen a fast-growing literature on intelligence in recent years. Yet very few have exclusively addressed intelligence analysis, and fewer still have treated it comprehensively. This is surprising given the importance of the subject and the thousands of professionals who practice the craft daily throughout the sixteen agencies in the US intelligence community. Yet a survey of the literature on US intelligence analysis over the past several decades yields only a few scholars and practitioners who have addressed the pitfalls in analysis and paths to improve it. In his Psychology of Intelligence Analysis, practitioner Richards Heuer has made a seminal contribution to improving analysis by incorporating knowledge and insights from the field of cognitive psychology. 11 Foundational to later advances in structured analytic tradecraft, this work is complemented by an ethnographic study of analysis by Rob Johnston. After Heuer, Johnston's data-rich Analytic Culture in the U.S. Intelligence Community identified numerous problems in the conduct of analysis IC-wide and proposed social science-based ways to improve it. 12 Formet CIA analyst Stephen

8 14 Intelligence Analysis operational-or worse, publicly announced by an emboldened Nikita Khrushchev as a strategic fair accompli with an accompanying ultimatum-american policymakers would have faced a very different and far less favorable set of options. The timeliness attribute is at the heart of warning intelligence, where analysis plays every bit as critical a role as collection because both must work for warning to succeed. In spite of a flawed estimate in September that failed to anticipate the Soviet gambit, the timely and successful U-2 overflights in October, and the trenchant and accurate analysis that followed, show the Cuban Missile Crisis to have been an outstanding intelligence-warning and crisis-support success. WMDs in Iraq: Confronting Intelligence Failure As the successful Cuban Missile Crisis case shows, intelligence can provide unique value added to policymaking through special collection, insightful analysis, strictly objective policy relevance, and timeliness. But failure is also part of the record. If intelligence always worked as effectively as it did during the Cuban Missile Crisis, there would be no controversy over whether it was worth the billions it costs every year, over the episodic clamor for intelligence reform, or especially over its putative value added for policymakers. Intelligence failures are disquieting. They shake the confidence of those who argue that the intelligence community is worth the investment and the risk that it takes to provide the most useful information to policymakers. An especially disturbing failure was the erroneous estimates ofwmds in Saddam Hussein's Iraq. The now well-known October 2002 NIE on Iraq made major errors in assessing Iraq's WMD programs. This NIE erroneously judged that Iraq had stockpiled as much as five hundred tons of chemical weapons (CW) and had an ongoing CW program; that Iraq had an active biological weapons (BW) program with BW agent stored there, along with mobile l}w labs; that Iraq was reconstituting its nuclear weapons program; that Iraq had a program of unmanned aerial vehicles that was probably capable of delivering BW agent to foreign shores, including to the United States; and that Iraq had missiles whose range exceeded permissible limits under United Nations (UN) sanctions.40 Only the last of the five major judgments (on missiles) proved to be correct. Four were completely wrong. Estimates-correct or not-so closely tied to a US decision to take military action are necessarily in the spotlight and rightly so. But even if this estimate had not been central to the debate over the Iraq invasion, it would still merit attention because of what it uncovered about the state of US intelligence analysis as the twenty-first century began. Why were the key findings so wrong? 41 Briefly, it was a significant collection failure because both human and technical intelligence collectors had failed to penetrate Iraq's WMD programs and because collection had also provided some wrong and misleading information. It was also a significant analysis failure. Reviewing the record, we find that analysts were more dependent on faulty collection than they comprehended, failed to question their past assumptions, and drew erroneous ee>nclusions from dated,

9 Intelligence Analysis 15 Collectors HUMINT SIGINT IMINT or GEOINT MASI NT OSINT Producers of "raw" intelligence Analysts Briefed and written analytic products Producers of "finished" intelligence Customers Policymakers, anned forces, homeland defense, and law enforcement Users of raw and finished intelligence Potential for analytic error through ambiguous, deceptive, contradictory, and missing infonnation Potential for analytic error through policy bias, adoption of policymaker biases, or politicization Potential for analytic error through "mind-set," faulty assumptions, poor tradecraft. or epistemology Figure 1.2 From Collection to Customer: The Analyst's Pivotal Role wrong, and poor information. 42 In short, on two key measures of unique value addedspecial collection and expert analysis-intelligence failed almost completely. Whether it also failed a third key test, strict objectivity, remains a matter of dispute. Major inquiries by the Senate Select Committee on Intelligence and another by the Silberman-Robb WMD Commission have given analysts a clean bill of health. Both concluded that they had found no evidence whatever of politicization-that is, that policymakers had not apparently influenced intelligence judgments favorable to the war decision. 43 But other observers think this is a more complex and nuanced problem, and even if there was no obvious arm-twisting by policymakers, anticipation caused by the omnipresent war prep-. arations surely distorted analysis. 44 As figure 1.2 illustrates, then, the possibility for analytic errors can occur in three critical areas: where there is poor or missing information, where unchallenged mind-sets or assumptions exist, or where bias may interfere with analytic objectivity. These three areas are explored throughout this book. Analyzing Intelligence Analysis Whether we focus on missiles in Cuba in 1962, on WMDs in Iraq forty years later, or on other major successes and failures of analysis, the central questions are whether the vast amount of intelligence analysis is succeeding or failing to give policymakers what they most need and how professionalizing analysis can help in this regard. This book draws on the individual and collective experience of many intelligence experts-most of whom have enjoyed long careers as successful analysts themselves, some as senior managers of analysts and others who are scholars of the issues we pose here. The book explains how analysis has been conducted and how it can improve.

10 16 Intelligence Analysis We examine how intelligence analysis has evolved since its origins in the middle of the last century, including attention to its traditions, culture, and track record. We examine how analysis supports the most senior national security and military policymakers, how analysts must deal with the perennial challenges of analytic bias and foreign denial and deception, and how they must become masters rather than victims of an ever-changing collection environment. We propose new ways to address perennial issues in warning analysis and emerging analytic issues such as domestic intelligence and homeland defense, and we suggest new approaches to supporting national strategy and new forms of analytic outreach in a global intelligence environment. We introduce specific new ideas for using structured analytic tradecraft for both strategic- and tactical-level analysis and for developing self-corrective techniques to improve analytic reliability. We examine the evolution and enhancements in analysis since the creation of the Office of the Director of National Intelligence in 2005 and the importance of building a common analytic culture in sixteen agencies. We also underline much-needed improvements in analyst training and education, as well as further professional development for a new generation of intelligence analysts, which can get the IC closer to the standards of a maturing discipline. If this book can illuminate the less well-known or poorly understood attributes and issues of the intelligence analysis process and then point to promising ways to improve it, we believe it can help to raise the quality and reliability of analysis. Simply put, our principal objective in the following chapters is to provide a better understanding of analysis for both the producers and users of intelligence. Notes 1. National Research Council, Intelligence Analysis for Tomorrow: Advances ftom the Behavioral and Social Sciences (Washington, DC: Nation;i.l Academies Press, 2011), There were many judgm.ents made in both the Cuban Missile Crisis and the Iraq WMD case. Some were incorrect, notably the September 1962 national intelligence estimate on Soviet offensive weapons in Cuba that underestimated the possibility of intermediate-range nuclear ballistic missiles on the island and the flawed 2002 estimate of nonexistent chemical and biological weapon stockpiles in Iraq. Other judgments were correct, such as those about the: state of readiness of Soviet weapons on the island (once discovered) and whether Saddam Hussein's missiles exceeded allowable range limits, as well as the: intelligence community's forecasts regarding the post-saddam regional and domestic conditions. As in most intelligence controversies, analysts are never entirely correct or entirely wrong. 3. See, for example, chaps. 5-7 and chap. 13 on warning. 4. Sherman Kent, "The Need for an Intelligence Literature," reprinted in Sherman Kent and the Board of National Estimates, ed. Donald Steury (Washington, DC: Center for the Study of Intelligence, CIA, 1994), Michael Warner, "Wanted: A Definition of Intelligence; Studies in Intelligence 46, no. 1 (2002): Douglas MacEachin, "Strategic Analysis," in Transforming US. Intelligence, ed. Jennifer E. Sims and Burton Gerber (Washington, DC: Georgetown University Press, 2006), 117.