Security Metrics A Beginner's Guide Caroline Wong Mc Graw Hill New York Chicago San Francisco Lisbon London Madrid Mexico City Milan New Delhi San Juan Seoul Singapore Sydney Toronto
Contents FOREWORD ACKNOWLEDGMENTS INTRODUCTION xxi xxv xxvii PART I Why Security Metrics? 1 Why Measure Security? 3 Purpose of an Information Security Program 4 Define a Mission Statement and a Charter for the Information Security Program 5 Evaluate the Components of an Information Security Program 7 Review the Predictive Security Model 9 Benefits of a Security Metrics Program 16 A Lesson for Security Metrics from the Traffic Safety Industry 16 Measurement Provides Visibility 19 Measurement Educates and Provides a Common Language 19 Measurement Enables Improvement 20 Why Are Security Metrics So Hard to Do? 25 xi
Xii Security Metrics: A Beginner's Guide 2 Why Security Metrics Are Needed Now 27 Security Work Is Never Finished: Technology Changes and Moore's Law 28 Verizon Business 2009 Data Breach Investigations Report 29 Symantec Global Internet Security Threat Report 30 Ernst & Young's 12th Annual Global Information Security Survey 31 More on the Increasing Sophistication of Attacks 32 Malware 32 Botnets 33 New Developments in Information Security 35 The Increasing Importance of Application Security 36 The Cloud 37 Targeted Attacks 38 The Impact of Social Networking 39 Profile of a Hacker 39 The "Old" Profile of a Hacker 40 Today's Hacker 40 Today's "Security Best Practices" Are Not Good Enough 43 A Good Starting Point for Strategy 44 Controls and Standards 46 Applying Metrics to Best Practices 49 PART II Essential Components of an Effective Security Metrics Practitioner 3 Analytics 55 What Are Security Analytics? 57 "Who Cares?" Test 57 Visualization 59 What You See May Not Be What You Get 59 Multiple Metrics for a More Complete Picture 61 Bundling Interpretation and Metrics 63 Do I Need a PhD in Math? 63 Leverage Analytic Patterns Developed by Others 64 Cool the PhD: You Can Leverage Patterns Without It! 65 Use the Trend Analysis Pattern 66 Examples of Applying Analytic Patterns 69 Example 1: Trend Analysis Microsoft Vulnerabilities 69 Example 2: Hypothesis Testing 72 Example 3: Trend Comparison 77
Contents xiii Example 4: Data Sample Effect 80 Example 5: Telling a Story 81 4 Commitment to Project Management 91 Information Security Culture Project Management 94 Brief Objective Statement 96 Type of Change 96 Proposed Start Date and End Date 96 Roles and Responsibilities 97 Project Name 100 Problem Statement 101 Solution Statement 101 Team Priority or Principle Supported 102 Project Scope 103 Project Description 103 Change Details and Impacts 104 Risks of Not Implementing This Project 104 Dependencies or Risks 104 Metrics/Success Measures 106 Major Deliverables and Deadlines 107 Required Budget 107 Information Security Resources Required 109 Other Resources Required 110 Example Application of the Project Management Methodology 110 Run-the-Business Activities 116 92 PART III Decide What to Measure 5 Identify Core Competencies, Information Security Work, and Resourcing Options 121 Evaluating Security Core Competencies for Metrics Projects 122 Spectrum of Information Security Work 125 Sustain 126 Establish 129 Build 132 Leveraging the Outsourcing and Offshoring Models 134 Benefits 134 Concerns 135 How to Manage the Risk 136 Additional Recommendations 138
xiv Security Metrics: A Beginner's Guide 6 identify Targets 1*1 Revisiting Objectives of an Information Security Metrics Program 142 Identifying What's Important 143 Compliance 143 Highest Risk 145 Business-Enabling Security Practices 148 Identifying What's Broken 148 Process Improvement 149 Technology Improvement 150 Audit Findings 155 Identifying What's Basic 155 Identifying What Needs to Be Discussed 157 Identifying What's New 158 Technology Evaluations 158 Cloud Provider Metrics and Evaluations 159 PART IV Get Started 7 Define Project Objectives 163 Training for a Marathon 164 Mapping a Target to a Benefit 166 Defining the Objective of a Security Metrics Project 167 Objective Desired Direction 167 Metrics Project Distance 170 Metrics Project Timeline 170 Lessons Learned 171 Setting Baselines 172 Initial Buy-in from Stakeholders 173 8 Define Your Priorities 177 A Real-World Prioritization Example 178 Why Is It Important to Prioritize? 180 Advantages of Effective Prioritization 181 Factors to Consider 182 Compliance 182 Risk Reduction 183 Threat Analysis 184 Alignment with Top Business Objectives 185 Specific Prioritization Factors for Security Metrics Projects 185 How to Prioritize 189 Prioritization Representations 189 Phase 1: Brainstorming and List Generation 189
Contents XV Phase 2: Top-Down Prioritization 190 Phase 3: Comprehensive Review 191 Phase 4: Draw the Line 192 Publication and Recurring Reviews 193 9 Identify Key Messages and Key Audiences 195 Why Stakeholder Engagement Is Important 196 Stakeholder Engagement 197 What's This Person's Area of Responsibility? Why Is This Person Important to Information Security? 197 What's Valuable to This Key Stakeholder? 199 What Are Their Security Needs? 199 For What Purpose Do You Need Their Buy-In? What Do You Need Them to Approve? 199 What Information Do You Need from This Person? 200 Examples 200 Key Audience: Chief Executive Officer 200 Key Audience: Chief Financial Officer 202 Key Audience: Chief Risk Officer 203 Key Audience: Chief Technology Officer 205 Key Audience: Business Unit Leader 206 Key Audience: Chief Information Officer 207 Key Audience: Director of Physical Security 208 Key Audience: Director of Human Resources 210 Chapter Summary 211 10 Obtain Buy-in from Stakeholders 215 What Is Buy-In and Why Do You Need It? 216 Preparing for a Buy-In Discussion with Stakeholders 217 Understanding Your Part 217 Understanding Your Stakeholders 221 The Steering Committee 223 Meeting, Explaining, Asking, Documenting 224 Documentation and Commitment 224 PART V Toolkit 11 Automation 229 Automation: Benefits 230 Automation: Workflow 231 Design: Hypothesize and Strategize 233 Collect: Extract, Cleanse, Transform, Merge, and Load Datasets 238
Security Metrics: A Beginner's Guide Calculate: Slice, Dice, and Model 244 Communicate: Visualize, Annotate, Publish 245 Orchestrate: Deploy, Schedule, Execute, and Coordinate 246 12 Analysis Technologies and a Case Study 251 Automation: Technologies 252 Design 255 Collect 257 Calculate 259 Communicate 261 Orchestrate 263 Case Study 264 Spreadsheet Chaos 265 Homegrown Solution 267 Purpose-Built Product Solution 269 PART VI Creating the Best Environment for Healthy Metrics 13 Define a Communications Strategy 275 What Do You Want to Communicate? 276 Keep Your Message Consistent 277 Know Your Audience 279 Acquisitions 279 International Audience 280 Communicate Well 281 Information Security Is Complex: Visual Aids and Remote Technology 281 Media Training 282 Share More 283 Why Not Share? 283 A Few Good Reasons Why Sharing Helps More Than It Hurts 284 Communication Formats 285 The 1:1 285 The Committee 286 Additional Tips on Communicating Effectively 287 14 Drive an Action Plan: The Importance of Project Management 289 Role of the Project Manager 290 Managing Change 291 Reporting 291 Meetings 293
Contents XVii Decision Making 293 Brief Objective Statement 294 Roles and Responsibilities 294 Problem Statement 296 Solution Statement 296 Team Priority or Principle Supported 297 Project Scope 297 Change Details and Impacts 298 Risks of Not Implementing This Project 298 Dependencies or Risks 298 Metrics and Success Measures 298 Major Deliverables and Deadlines 299 Budget 299 Information Security Resources 299 Reporting Formats 300 Status Reporting 300 PART VII Secret Sauce: Lessons Learned from an Enterprise Practitioner 15 Improving Data Quality and Presentation 305 Data Cleansing 307 Making Data Accurate 307 Making Data Complete 310 Making Data Consistent 310 Making Data Unambiguous 311 Reporting Data from Multiple Systems 311 Raw Data Generators 312 Ticketing Systems 312 Asset Management Systems 313 Consistent Meaning 314 Data, Processes, and People 314 Working with Stakeholders to Perform Data Cleansing 316 Fix the Process, and Then Automate 317 Don't Wait for Perfect Data Before Reporting 319 16 Resourcing and Security Metrics Projects 321 Resourcing Options 323 Security Team Resourcing 324 Outsourcing to Obtain Metrics 325
XVlH Security Metrics: A Beginner's Guide Leveraging Politics and Competition 325 Metrics as Justification for More Resources 327 Incident Response Metrics 328 Security Consulting Metrics 329 Report Quickly 330 PART VIII Looking Forward 17 Security Metrics for Cloud Computing 335 Cloud Computing Defined 336 Characteristics 337 Service Models 337 Deployment Models 338 Cloud Business Drivers 340 PART IX The New Normal 342 Security Metrics vs. Cloud Security Metrics 344 Cloud Security Alliance 346 CSA Cloud Metrics Working Group Template 349 CSA Cloud Metrics Working Group Lifecycle 353 Final Thoughts 354 Appendix and Glossary Appendix Templates and Checklists 359 Chapter 1: The Three Benefits of a Security Metrics Program 360 Chapter 2: Best Practice Analysis 360 Chapter 5: Request for Proposal 361 Chapter 6: Metrics for High Risk Areas 361 Metrics for Process Improvement 362 Metrics for Security Technology 362 Metrics for Non-Security Technology 364 Metrics for What's Basic? 366 Metrics for New Technology 366 Chapter 7: Meeting with Stakeholders 367 Chapter 8: Basic Prioritization Questions 367 Risk Reduction Questions 368 Business Objective Alignment Questions 368 Chapter 9: Identifying Key Audiences and Key Messages 369 Stakeholder Analysis 369
Contents xix Examples of Key Audiences, Key Messages, and Security Metrics 370 Analysis: What Do You Need? 371 Chapter 17: Template for Completely and Unambiguously Defining a Metric 372 Glossary 375 Index 381