Software Assurance: Enabling Security Automation and Software Supply Chain Risk Management. Commerce. National Defense. Today Everything s Connected

Transcription

1 Software Assurance: Enabling Security Automation and Software Supply Chain Risk Interdependencies Between Physical & Cyber Infrastructures: Convergence of Safety, Security and Dependability Considerations -- Need for secure/resilient software applications August 30, 2012 Joe Jarzombek, PMP, CSSLP Director for Software Assurance National Cyber Security Division Homeland Security Today Everything s Connected Your System is attackable Commerce National Defense Public/Private Collaboration Efforts for Security Automation and Software Supply Chain Risk Next SwA Forum meets Sep 2012 at MITRE, McLean, VA When this Other System gets subverted Homeland through un-patched vulnerability, a misconfiguration, or an application Security weakness Interdependencies Between Physical & Cyber Convergence of Safety, Security and Dependability Considerations In an era riddled with asymmetric cyber attacks, claims about system reliability and safety must include provisions for built-in security of the enabling software High Reliance on Software Security Feature Buffer Overflow (CWE-120) Exploit (CAPEC-123) SQL Injection (CWE-89) Exploit (CAPEC-66) Built-in Security enables Resilience Exploitable Software Weaknesses are sources for future Zero-Day Attacks 1

2 Security is a Requisite Quality Attribute: Vulnerable Software Enables Exploitation Rather than attempt to break or defeat network or system security, hackers are opting to target application software to circumvent security controls. 75% of hacks occurred at application level 90% of software attacks were aimed at application layer (Gartner & Symantec, June 2006) most exploitable software vulnerabilities are attributable to non-secure coding practices (and not identified in testing). Functional correctness must be exhibited even when software is subjected to abnormal and hostile conditions Software applications with exploitable vulnerabilities & weaknesses SECURITY Software applications with exploitable vulnerabilities & weaknesses In an era riddled with asymmetric cyber attacks, claims about system reliability, integrity & safety must include provisions for built-in security of the enabling software. Challenges in Mitigating Risks Attributable to Exploitable Software and Supply Chains Complexity hampers ability to determine and predict code behavior; so any assurance claims for security/safety-critical applications are limited. Without adequate diagnostic capabilities and commonly recognized standards from which to: discern product assurance; benchmark process capabilities, and assert claims about the assurance of products, systems and services, provenance and pedigree of supply chain actors become a more dominant consideration for security/safety-critical applications: Enterprises and Users lack requisite transparency for more informed decision-making for mitigating risks; Favoring domestic suppliers does not necessarily address assurance in terms of capabilities to deliver secure/safe components, systems or software-reliant services. Software Assurance Addresses Exploitable Software: Outcomes of non-secure practices and/or malicious intent Exploitation potential of vulnerability is independent of intent Defects EXPLOITABLE SOFTWARE Unintentional Vulnerabilities Malware Intentional Vulnerabilities High quality can reduce security flaws attributable to defects; yet traditional S/W quality assurance does not address intentional malicious behavior in software *Intentional vulnerabilities: spyware & malicious logic deliberately imbedded (might not be considered defects) Software Assurance (SwA) is the level of confidence that software functions as intended and is free of vulnerabilities, either intentionally or unintentionally designed or inserted as part of the software throughout the life cycle.* From CNSS Instruction 4009 National Information Assurance Glossary (26APR2010) Challenges in Mitigating Risks Attributable to Exploitable Software and Supply Chains (cont.) Several needs arise: Need internationally recognized standards to support security automation and processes to provide transparency for more informed decision-making for mitigating enterprise risks. Need Assurance to be explicitly addressed in standards & capability benchmarking models for organizations involved with security/safety-critical applications. Need more comprehensive diagnostic capabilities to provide sufficient evidence that code behavior can be well understood to not possess exploitable or malicious constructs. Need rating schemes for software products and supplier capabilities. Understanding the Threat and Controlling the Attack One who knows the enemy and knows himself will not be endangered in a hundred engagements. One who does not know the enemy but knows himself will sometimes be victorious; sometimes meet with defeat. One who knows neither the enemy nor himself will invariably be defeated in every engagement. The Art of War, Sun Tzu An appropriate defense can only be established if one knows its weaknesses and how it could be attacked; thus controlling attack surface/vectors Software Assurance Forum, Joe Jarzombek Challenges in Mitigating Risks Attributable to Exploitable Software and Supply Chains (cont.) Enterprises seek comprehensive capabilities to: Avoid installing software with MALWARE pre-installed. MAEC Determine that no publicly reported VULNERABILITIES CVE remain in code prior to operational acceptance, and that future discoveries of common vulnerabilities and exposures can be quickly patched. Determine that exploitable software WEAKNESSES that put the users most at risk are mitigated prior to operational acceptance. CWE 2

3 Challenges in Preventing and Responding to Cyber Incidents Silos of operation Proprietary reporting formats Needs arise: Need standards to support security automation and processes to support exchange of information and cyber indicators relative to incident management and response. Software Assurance End State Objectives Government, in collaboration with industry / academia, raised expectations for product assurance with requisite levels of integrity and security: Helped advance more comprehensive software assurance diagnostic capabilities to mitigate risks stemming from exploitable vulnerabilities and weaknesses; Collaboratively advanced use of software security measurement & benchmarking schemes Promoted use of methodologies and tools that enabled security to be part of normal business. Acquisition managers & users factored risks posed by the software supply chain as part of the trade-space in risk mitigation efforts: Information on suppliers process capabilities (business practices) would be used to determine security risks posed by the suppliers products and services to the acquisition project and to the operations enabled by the software. Information about evaluated products would be available, along with responsive provisions for discovering exploitable vulnerabilities, and products would be securely configured in use. Suppliers delivered quality products with requisite integrity and made assurance claims about the IT/software safety, security and dependability: Relevant standards would be used from which to base business practices & make claims; Qualified tools used in software lifecycle enabled developers/testers to mitigate security risks; Standards and qualified tools would be used to certify software by independent third parties; IT/software workforce had requisite knowledge/skills for developing secure, quality products. Enabling Software Supply Chain Transparency DHS Software Assurance Program Overview Program established in response to the National Strategy to Secure Cyberspace - Action/Recommendation 2-14: DHS will facilitate a national public-private effort to promulgate best practices and methodologies that promote integrity, security, and reliability in software code development, including processes and procedures that diminish the possibilities of erroneous code, malicious code, or trap doors that could be introduced during development. DHS Program goals promote the security and resilience of software across the development, acquisition, and operational life cycle DHS Software Assurance (SwA) program is scoped to address: Trustworthiness - No exploitable vulnerabilities or malicious logic exist in the software, either intentionally or unintentionally inserted, Dependability (Correct and Predictable Execution) - Justifiable confidence that software, when executed, functions as intended, Survivability- If compromised, damage to the software will be minimized; it will recover quickly to an acceptable level of operating capacity; it s rugged ; Conformance Planned, systematic set of multi-disciplinary activities that ensure processes/products conform to requirements, standards/procedures. See Wikipedia.org for Software Assurance - CNSS Instruction No. 4009, "National Information Assurance Glossary," Revised 2006, defines Software Assurance as: "the level of confidence that software is free from vulnerabilities, either intentionally designed into the software or accidentally inserted at anytime during its lifecycle, and that the software functions in the intended manner". Software Assurance Forum & Working Groups* encourage the production, evaluation and acquisition of more secure and resilient software through targeting: People Developers and users education & training Processes Sound practices, standards, & practical guidelines for secure software development Build Security In - and SwA community resources & info clearinghouse Technology Security test criteria, measurement, diagnostic tools, common languages & enumerations, SwA Research & Development Products and Contributions SwA Common Body of Knowledge (CBK) & Glossary Organization of SwSys Security Principles/Guidelines SwA Developers' Guide on Security-Enhancing SDLC SwA Curriculum Project: Masters and Undergraduate Software Security Assurance State of the Art Report Systems Assurance Guide (via DoD and NDIA) SwA-related standards ISO/IEC JTC1 SC7/27/22, IEEE CS, OMG, TOG, & CMM-based Assurance Acquisition Software security improvements through due-diligence questions, specs and guidelines for acquisitions/ outsourcing Practical Measurement Framework for SwA/InfoSec Making the Business Case for Software Assurance SwA Metrics & Tool Evaluation (with NIST) SwA Ecosystem w/ DoD, NSA, NIST, OMG & TOG NIST Special Pub 500 Series on SwA Tools Security Automation Enumerations & Languages (CVE, OVAL, CWE, CAPEC, MAEC, CybOX, STIX), scoring systems, and risk analysis capabilities SwA in Acquisition: Mitigating Risks to Enterprise Software Project for SwA SOAR * SwA Forum is part of Cross-Sector Cyber Security Working Group (CSCSWG) established under auspices of the Critical Infrastructure Partnership Advisory Council (CIPAC) that provides legal framework for participation. DHS NCSD Software Assurance (SwA) Program Advances security and resilience of software throughout the lifecycle; focuses on reducing exploitable software weaknesses and addresses means to improve capabilities that routinely develop, acquire, and deploy resilient software. Serves as a focal point for interagency public-private collaboration to enhance development and acquisition processes, capability benchmarking and rating schemes to address software security needs. Hosts interagency Software Assurance Forums, working groups and training to provide public-private collaboration in advancing software security and providing publicly available resources. Provides collaboratively developed, peer-reviewed information resources on Software Assurance, via journals, guides & on-line resources suitable for use in education, training, and process improvement. Provides input and criteria for leveraging international standards and maturity models used for process improvement and capability benchmarking of software suppliers and acquisition organizations. Enables software security automation and measurement capabilities through use of common indexing and reporting capabilities for malware, exploitable software weaknesses, cyber indicators and attacks which target software. Collaborates with national & international standards organizations and industry to create standards, metrics and certification mechanisms from which products and tools could be qualified for software security verification. Manages programs for Malware Attribute Enumeration Classification (MAEC), Common Weakness Enumeration (CWE), Common Attack Pattern (CAPEC) & Cyber Observable expression (CybOX). Manages programs for Common Vulnerabilities & Exposures (CVE) and Open Vulnerability & Assessment Language (OVAL) that provide information feeds for continuous monitoring, security content automation, vulnerability databases, and security/threat alerts from many organizations Homeland Security Cybersecurity and Communications DHS Software Assurance (SwA) Outreach & Awareness Co-sponsor SwA working group sessions, semi-annual SwA Forum, for government, academia, and industry to facilitate ongoing publicprivate collaboration. Provide SwA presentations, workshops, and tracks at conferences Co-sponsor SwA issues of CROSSTALK to spread the word Sep 2008 issue on Application Security Mar/Apr 2009 issue on Reinforcing Good Practices Sep/Oct 2009 issue on Resilient Software Mar/Apr 2010 issue on System Assurance Sep/Oct 2010 issue on Game Changing Tools & Practices Mar/Apr 2011 issue on Rugged Software Sep/Oct 2011 issue on Protecting against Predatory Practices Mar/Apr 2012 issue on Securing a Mobile World Sep/Oct 2012 issue on Resilient Cyber Ecosystem Mar/Apr 2013 issue on Supply Chain Risk Collaborate with standards organizations, consortiums, professional societies, education/training initiatives in promoting SwA Provide free SwA resources via BuildSecurityIn website to promote secure development methodologies (since Oct 05) Host SwA Community Resources & Information Clearinghouse via (since Dec 07) 18 3

4 Architecture and Design Considerations for Secure Software SwA Pocket Guide* The IEEE Guide to the Software Engineering Body of Knowledge (SWEBOK) defines the design phase as both the process of defining the architecture, components, interfaces, and other characteristics of a system or component and the result of [that] process. The software design phase: is the software engineering life cycle activity where software requirements are analyzed in order to produce a description of the software s internal structure that will serve as the basis for its implementation. consists of the architectural design and detailed design activities that follow the software requirements analysis phase and precedes software implementation in the SDLC. This pocket guide includes the following topics: Basic Concepts Design Principles for Secure Software Architecture and Threat Modeling Secure Design Patterns Architectural-level Patterns Design-level Patterns Secure Session Design and Architectural Considerations for Mobile Applications Formal Methods and Architectural Design Design Review and Verification Key Architecture and Design Practices for Mitigating Exploitable Software Weaknesses Questions to Ask Developers *Download FREE SwA Pocket Guides at Life-Cycle Standards View Categories (ISO/IEC and 12207) Organization Project Governance Processes Engineering Project Strategy and policy Technical Processes Processes Stakeholder Requirements Definition Enterprise risk management Compliance Project Planning Requirements Business case Attack modeling (misuse and abuse cases) Project Assessment and Data and information classification Supply Chain Control Risk-based derived requirements Assurance case Sw security requirements management Project-Enabling Processes Architectural Design Secure Sw architectural design Life Cycle Model Risk-based architectural analysis Project Support Secure Sw detailed design and analysis Infrastructure Processes SwAecosystem Implementation Enumerations, languages, and Decision Secure coding and Sw construction repositories Security code review and static analysis Formal methods Project Portfolio Risk Threat Assessment Integration Human Resource Sw component integration SwAeducation Configuration Risk analysis of Sw reuse components SwAcertification and training Recruitment Verification & Validation Information Risk-based test planning Quality Security-enhanced test and evaluation Dynamic and static code analysis Measurement Penetration testing Independent test and certification Agreement Processes Transition Acquisition Secure distribution and delivery Outsourcing Secure software environment (secure configuration, Agreements application monitoring, code signing, etc) Risk-based due diligence Supplier assessment Operations and Sustainment Supply Operation Incident handling and response Maintenance Defect tracking and remediation Vulnerability and patch management Version control and management Disposal 20 Software Reuse Processes Domain Engineering Reuse Asset Reuse Program Software Support Processes Sw Documentation Sw Quality Assurance Sw Configuration Sw Verification & Sw Validation Sw Review Sw Audit Sw Problem Resolution Apply Key Principles & Practices Table 1- Adapted from Enhancing the Development Life Cycle to Produce Secure Software General Principle Key Practices Principle Design Conformance Minimize the number of Principle of least privilege Minimizes the number of actors in the system granted high levels of privilege, and the high-consequence targets amount of time any actor holds onto its privileges. Separation of privileges, duties, and roles Ensures that no single entity (human or software) should have all the privileges required to modify, delete, or destroy the system, components and resources. Separation of domains This practice makes separation of roles and privileges easier to implement. Don t expose vulnerable or Keep program data, executables, and Reduces the likelihood that an attacker who gains access to program data will easily high-consequence configuration data separated locate and gain access to program executables or control/configuration data. components Segregate trusted entities from untrusted Reduces the exposure of the software s high-consequence functions from its high-risk entities functions, which can be susceptible to attacks. Minimize the number of entry and exit points Reduces the attack surface. Assume environment data is not trustworthy Reduces the exposure of the software to potentially malicious execution environment components or attacker-intercepted and modified environment data. Use only trusted interfaces to environment This practice reduces the exposure of the data passed between the software and its resources environment. Deny attackers the means Simplify the design This practice minimizes the number of attacker-exploitable vulnerabilities and to compromise weaknesses in the system. Hold all actors accountable This practice ensures that all attacker actions are observed and recorded, contributing to the ability to recognize and isolate/block the source of attack patterns. Timing, synchronization, and sequencing Modeling and documenting timing, synchronization, and sequencing issues will reduce should be simplified to avoid issues. the likelihood of race conditions, order dependencies, synchronization problems, and deadlocks. Make secure states easy to enter and This practice reduces the likelihood that the software will be allowed to inadvertently vulnerable states difficult to enter enter a vulnerable state. Design for controllability This practice makes it easier to detect attack paths, and disengage the software from its interactions with attackers. Caution should be taken when using this approach since it can open a whole range of new attack vectors. Design for secure failure Reduces the likelihood that a failure in the software will leave it vulnerable to attack. Software Assurance (SwA) Pocket Guide Series SwA in Acquisition & Outsourcing Software Assurance in Acquisition and Contract Language Software Supply Chain Risk and Due-Diligence SwA in Development Integrating Security into the Software Development Life Cycle Key Practices for Mitigating the Most Egregious Exploitable Software Weaknesses Risk-based Software Security Testing Requirements and for Secure Software Architecture and Design Considerations for Secure Software Secure Coding and Software Construction Security Considerations for Technologies, Methodologies & Languages SwA Life Cycle Support SwA in Education, Training and Certification Secure Software Distribution, Deployment, and Operations Code Transparency & Software Labels Assurance Case Secure Software Environment and Assurance EcoSystem SwA Measurement and Information Needs Making Software Security Measurable Practical Measurement Framework for SwA and InfoSec SwA Business Case and Return on Investment Conduct Design Reviews Verification activities during the design phase includes: Structured inspections, conducted on parts or views of the high-level design throughout the design phase; Independent verification and validation (IV&V) reviews; A preliminary design review conducted at the end of the architecture design phase and before entry into the detailed design phase; and A critical design review added at designated points in the software development lifecycle. Design Verification The design should be verified considering the following criteria: The design is correct and consistent with and traceable to requirements; The design implements the proper sequence of events, inputs, outputs, interfaces, logic flow, allocation of timing and sizing budgets, error definition, isolation, and recovery; The selected design can be derived from requirements; and The design implements safety, security, and other critical requirements correctly as shown by suitably rigorous methods. SwA Pocket Guides and SwA-related documents are collaboratively developed with peer review; they are subject to update and are freely available for download via the DHS Software Assurance Community Resources and Information Clearinghouse at (see SwA Resources) 4

5 Leverage Attack Patterns CAPEC Common Attack Pattern Enumeration and Classification: capec.mitre.org Attack Patterns: Blueprint for creating a specific type of attack Abstracted common attack approaches from the set of known exploits Capture the attacker s perspective to aid software developers, acquirers and operators in improving the assurance profile of their software Use Attack Patterns to: Guide definition of appropriate policies Guide creation of appropriate security requirements (positive and negative) Provide context for architectural risk analysis Guide risk-driven secure code review Provide context for appropriate security testing Provide a bridge between secure development and secure operations Software Security Test Techniques throughout SDLC See details in FREE SwA Pocket Guide on Software Security Testing at Penetration Testing can enhance pre-deployment test outcomes and identify post-release exploit points Ask Questions Ask the designers/developers of the software: How does software assure that state information has not been tampered with? How does the software prevent attackers from repeatedly triggering a synchronized critical section? Are misuse cases utilized during the design process? If so, what modeling techniques are used to define and handle misuse cases? How does the software perform access control checks? How does the application ensure that only authorized personnel are accessing sensitive information? How does the software assure that data transmitted over a communications channel cannot be intercepted and/or modified? Are formal methods incorporated into the architectural design to improve software security? If so, what methods are used and why? How does the server-side security check for bypasses in the client-side validation? What techniques are used to prevent this? How does the server ensure that the Session ID has not been stolen or manipulated? How does team prevent weaknesses in initialization of variables and objects? Secure Coding Preparing to Write Secure Code Secure Coding Principles Secure Coding Practices Secure Memory and Cache Secure Error and Exception Handling What to Avoid Questions to Ask Developers Are compiler warnings disabled in code being delivered? Software Security Testing vs Security Requirements Testing Software security testing is not the same as testing the correctness and adequacy of security functions implemented by software, which are most often verified through requirements-based testing that: cannot fully demonstrate that software is free from exploitable weaknesses / vulnerabilities. is not the best approach to determining how software will behave under anomalous and hostile conditions. Download FREE SwA Pocket Guide on Software Security Testing at Key Practices for Mitigating the Most Egregious Exploitable Software Weaknesses Identifies mission/business risks attributable to the respective weaknesses;identifies common attacks that exploit those weaknesses, and provides recommended practices for preventing the weaknesses. CWE focuses on stopping vulnerabilities at the source by educating designers, programmers, and QA/testers on how to eliminate all too-common mistakes before software is even shipped. CWE Top-N lists serve as tools for education, training and awareness to help programmers prevent the kinds of vulnerabilities that plague the software industry. Software consumers could use the same list to help them to ask for more secure software. Software managers and CIOs can use the CWE list as a measuring stick of progress in their efforts to secure their software. 5

6 Many SwA Resources Focus On Development Assurance for CMMI Define Business Goals Development Organization DO 1 Establish the assurance resources to achieve key business objectives DO 2 Establish the environment to sustain the assurance program within the organization Acquisition and Supplier AM 1 Select, manage, and use effective suppliers and third party applications based upon their assurance capabilities. Assurance Process Reference Model Development Project DP 1 Identify and manage risks due to vulnerabilities throughout the product and system lifecycle DP 2 Establish and maintain assurance support from the project DP 3 Protect project and organizational assets Prioritize funds and manage risks Development Engineering DE 1 Establish assurance requirements DE 2 Create IT solutions with integrated business objectives and assurance DE 3 Verify and Validate an implementation for assurance Enterprise Assurance Support ES 1 Establish and maintain organizational culture where assurance is an integral part of achieving the mission ES 2 Establish and maintain the ability to support continued delivery of assurance capabilities ES 3 Monitor and improve enterprise support to IT assets Created to facilitate Communication Across An Organization s Multi-Disciplinary Stakeholders Courtesy of Michele Moss, BAH, SwA Processes & Practices Enable Resilient Technology Sustained environment to achieve business goals through technology Security-Enhanced Process Improvements Organizations that provide security engineering & risk-based analysis throughout the lifecycle will have more resilient software products / systems. Build Security In throughout the lifecycle Attack Modeling Abuse Cases Plan Secure S/W Requirements Engineering Security Requirements Risk Assessment Requirements and Use Cases Risk Secure Design Principles & Practices Design Design Review Risk-based Test Plans Security Design Reviews Architecture and Detailed Design Leverage Software Assurance resources (freely available) to incorporate in training & awareness Modify SDLC to incorporate security processes and tools (should be done in phases by practitioners to determine best integration points) Secure Programming Practices Code Review Build Test / Validation of Security & Resilience Static/Dynamic Application Security Testing Code and Testing Secure Distribution/ Deployment Risk Deploy PenetrationSecurity Ops & Testing Vulnerability Mgt S/W Support Scanning & Remediation Field Deployment and Feedback Documentation for Secure Use & Configuration Organizational Process Assets cover: governance, policies, standards, training, tailoring guidelines Avoid drastic changes to existing development environment and allow for time to change culture and processes Make the business case and balance the benefits Retain upper management sponsorship and commitment to producing secure software. * Adopted in part from Software Assurance: Mitigating Supply Chain Risks (DHS NCSD SwA); What to Test from a Security Perspective for the QA Professional (Cigital) and Neutralizing the Threat: A Case Study in Enterprisewide Application Security Deployments (HP Fortify Software & Accenture Security Technology Consulting) 3 The DHS SwA Processes and Practices Working Group has synthesized the contributions of leading government and industry experts into a set of high-level goals and supporting practices (an evolution of the SwA community s Assurance Process Reference Model) The goals and practices are mapped to specific industry resources providing additional detail and real world implementation and supporting practices Assurance Focus for CMMI Building Security In Maturity Model Open Software Assurance Maturity Model CERT Resilience Model CMMI for Acquisition CMMI for Development CMMI for Services SwA Community s Assurance Process Reference Model Initial Mappings SwA Community s Assurance Process Reference Model - Self Assessment SwA Community s Assurance Process Reference Model Mapping to Assurance Models Other valuable resources that are in the process of being mapped include NIST IR 7622: DRAFT Piloting Supply Chain Risk Practices for Federal Information Systems NDIA System Assurance Guidebook Microsoft Security Development Lifecycle SAFECode Mission/Business Process Understand Your Business Requirements for Assurance Understand Assurance-Related Process Capability Expectations Process Improvement Lifecycle - A Process for Achieving Assurance Measure Your Results Organization Support Information System Build or Refine and Execute Your Assurance Processes Look to Standards for Assurance Process Detail It can be used by acquirers, suppliers and integrators as a to tool to discuss areas of strength and weakness What assurance goals are being met? What practices are being implemented? Who are the suppliers and how are they managing risk? SwA Community Assurance Process Reference Model Self Assessment In the following table, all references to assurance are intended to include system and software assurance, and cyber security in support of the business/mission functions supported by systems and software. Goal Practice Practice Implementation Notes Level Development Engineering Understand the operating environment and define the operating constraints for mission and information assurance within the environments of system development. DE 1 Establish assurance requirements Develop customer mission and information assurance requirements Define product and product component assurance requirements Identify operational concepts and associated scenarios for intended and unintended use and associated assurance considerations Identify appropriate controls for integrity and availability of the system to in support of organizational objectives Analyze assurance requirements Balance assurance needs against cost benefits Obtain Agreement of risk for assurance level Adapted from: Paul Croll, Computer Sciences Corporation, August

7 The Process Reference Model For Assurance Process Reference Model for Assurance Goals and Practices September 2010 In the following table, all references to assurance are intended to include system and software assurance, information assurance, and cyber security in support of the business/mission functions supported by systems and software. Goal Practice List Development Engineering Understand the operating environment and define the operating constraints for mission and information assurance within the environments of system development. Develop customer mission and information assurance requirements Define product and product component assurance requirements Identify operational concepts and associated scenarios for intended and unintended use and associated DE 1 Establish assurance assurance considerations requirements Identify appropriate controls for integrity and availability of the system to in support of organizational objectives Analyze assurance requirements Balance assurance needs against cost benefits Feedback The SwA Checklist is available on the DHS SwA Community Resources and Information Clearinghouse website alongside the Assurance PRM Self-Assessment: The Processes and Practices Working Group welcomes feedback on your experiences using the SwA Checklist in the field. Obtain Agreement of risk for assurance level It can be used as a navigation tool to guide SwA implementation efforts More Information You have been asked to ensure that the OWASP Top Ten (an assurance coding Standard) are not in the Code You can look at the OSAMM for guidance on how to do it Additional details on the SwA Checklist are discussed in the March 2011 issue of CrossTalk The Journal of Defense Software Engineering: Intended Use of the SwA Checklist Useful to an organization that is currently, or soon will be, acquiring or developing software, wants to improve its SwA capabilities, and is interested in using a maturity model or model components Organizations can use the SwA Checklist to: Guide their own development Evaluate vendor capabilities The SwA Checklist can facilitate: Establishing an assurance baseline Learning more about current software assurance best practices Guiding their selection of the most appropriate model components Business Case for Software Assurance April 2009 SwA Report provides background, context and examples: Motivators Cost/Benefit Models Overview Measurement Risk Prioritization Process Improvement & Secure Software Globalization Organizational Development Case Studies and Examples 7

8 IT/software security risk landscape is a convergence between defense in depth and defense in breadth Practical Measurement Framework for Software Assurance and Information Security Security Measurement Resources Oct 08 Feb 09 May 09 Enterprise Risk and Governance are security motivators Acquisition could be considered the beginning of the lifecycle; more than development In the digital age, sovereignty is demarcated not by territorial frontiers but by supply chains. Dan Geer, CISO In-Q-Tel Oct 2008 Software Assurance provides a focus for: -- Secure Software Components, -- Security in the Software Life Cycle, -- Software Security in Services, and -- Software Supply Chain Risk Software Assurance Curriculum Project Vol I: Master of Software Assurance Reference Curriculum In Dec 2010 the IEEE Computer Society and the ACM recognized the Master of Software Assurance (MSwA) Reference Curriculum as a certified master s degree program in SwA the first curriculum to focus on assuring the functionality, dependability, and security of software and systems. Vol II: SwA Undergraduate Course Outlines see download the PDF version of the report CMU/SEI-2010-TR-019 Vol III: Master of SwA Course Syllabi Vol IV: Community College Education Report on Integrating the MSwA Reference Curriculum into Model Curriculum and Guidelines for Graduate Degree Programs in Information Systems provides reference and guidance material. To facilitate implementation, the MSwA project team is offering assistance, free of charge, to educational institutions looking to launch an MSwAdegree program. For more information,go to * Acquisition Program Supplier Supply chain introduces risks to American society that relies on Federal Government for essential information and services. 30 Sep 2005 changes to Federal Acquisition Regulation (FAR) focus on IT Security Focuses on the role of contractors in security as Federal agencies outsource various IT functions. Scope of Supplier Expansion and Foreign Involvement graphic in DACS Secure Software Engineering, July 2005 article Software Development Security: A Risk Perspective synopsis of May 2004 GAO report Defense Acquisition: Knowledge of Software Suppliers Needed to Manage Risks Software Assurance Professional Competency Model Specialty Areas* Software Assurance & Security Engineering Information Assurance Compliance Vulnerability Assessment & Cyber Threat Systems Requirements Planning Systems Security Architecture Strategic Planning & Policy Development Technology Research and Development Education and Training Knowledge * Specialty Areas aligned with Framework for National Initiative for Cybersecurity Education Risk (Enterprise <=> Project): Shared Processes & Practices Different Focuses Enterprise-Level: Regulatory compliance Changing threat environment Business Case Program/Project-Level: Cost Schedule Performance Software Supply Chain Risk traverses enterprise and program/project interests 1. Insert and enforce software assurance requirements in contracts. 2. Review IT security policies to ensure that all users of organizational networks and data comply with the strictest security policies possible with respect to the mission. 3. Determine how much risk the organization can afford and who is accountable for that risk. 8

9 Thousands of downloads from open libraries with documented vulnerabilities Source: Maximizing Benefits and Mitigating Risks of Open Source Components in Application Development, by Sonatype Software Security Assurance: Not just a good idea Many people responsible for protecting most critical infrastructure facilities have felt comfortable about security of their systems. Facilities rely on industrial control systems (ICS) -- custom-built suites of systems that control essential mechanical functions of power grids, processing plants, etc -- usually not connected to the Internet, also known as "air-gapped." Many industry owners, operators and regulators believed that this security model provided an infallible, invulnerable barrier to malicious cyber attacks from criminals and advanced persistent threat (APT) adversaries. National Defense Authorization Act (NDAA) -- which included a focus on software security (in Section 932, Strategy on Computer Software Assurance) -- serves as first cybersecurity law of 2011 and requires the U.S. Dept of Defense to develop a strategy for ensuring the security of software applications. Software Security Assurance, a set of practices for ensuring proactive application security, is key to making applications compliant with this new law. How Stuxnet Demonstrates That Software Assurance Equals Mission Assurance: The rules of the game have changed, by Rob Roy, Federal CTO of Fortify, an HP Company Even after vulnerabilities are discovered and patches made available, many developers use the flawed, non-patched version of reused components Who makes risk decisions? Who inherits the residual risk? Who owns the residual risk attributable to exploitable software? Source: Maximizing Benefits and Mitigating Risks of Open Source Components in Application Development, by Sonatype Comprehensive Program Protection 5/1/2012 Page 53 Program Protection Plan Outline and Guidance as Expected Business Practice Signed by Principal Deputy, USD(AT&L) on July 18, 2011 What s in the DoD Policy Memo? Every acquisition program shall submit a PPP for Milestone Decision Authority review and approval at Milestone A and shall update the PPP at each subsequent milestone and the Full- Rate Production decision. Expected business practice, effective immediately, and reflected in upcoming DoDI and DAG updates The PPP is the Single Focal Point for All Security Activities on the Program Distribution Statement A Cleared for public release by OSR on 4/25/2012, SR Case # 12-S-1841 applies. NRC Regulatory Guidance on Cyber Security Software Assurance Methods Countermeasure Selection NRC Regulatory Guide 5.71, Cyber Security Programs for Nuclear Facilities, Section C.12 in Appendix C, System and Service Acquisition Directly relates to current NRC guidance on cyber security in the supply chain and SDLC of an ICS regulated by the agency. Section C.12.2 Supply Chain Protection control drill down to the vendor level with requirements accountability for the RG 5.71 control baseline (Appendices B&C). Section C.12.3 Trustworthiness requires developers employ software quality and validation methods to minimize flawed or malformed software; requires all tools to undergo commercial certification process Section C.12.5 Developer Security Testing See NRC Regulatory Guidance on Cyber Security Development Process Apply assurance activities to the procedures and structure imposed on software development Operational System Implement countermeasures to the design and acquisition of end-item software products and their interfaces Development Environment Apply assurance activities to the environment and tools for developing, testing, and integrating software code and interfaces Additional Guidance in PPP Outline and Guidance Program Protection Process Distribution Statement A Cleared for public release by OSR on 4/25/2012, SR Case # 12-S-1842 applies. 5/1/12 Page 54 9

10 (HW, SW, Firmware) (HW, SW, Firmware) Protection Measures can be in Specifications and/or Processes Strategic Guidance (OSD/JCS) Joint Concepts (COCOMs) Generic RFP Language is Available CBA Program Protection in Engineering Discipline ICD Protect Capability from Supply Chain/System Design Exploit Supply Chain Risk Software Assurance Information Assurance Protect Advanced Technology Capability from Foreign Collection/Design Vulnerability Anti-Tamper Export Control Intel/CI/Security TD Phase RFP MDD Materiel Solution AoA Focus Scope of Protection SEP MS A Technology Development CDD ASR SRR SFR PDR CDR PPP EMD Phase RFP Production Contract FRP Decision or MS B MS C FDD Review Program Protection at SE Technical Reviews SEP PPP Pre-EMD Review SEP PPP Engineering & Manufacturing Development SEP CPD PPP Production and Deployment PPP Emphasizing Use of Affordable, Risk-based Countermeasures O&S Integrated Process to Manage Security Risks Foreign Collection Design Vulnerability Supply Chain Exploit/Insertion International Community System Assurance Activities ISO/IEC System and Software Engineering Systems and Software Assurance Establishes common assurance concepts, vocabulary, integrity levels and lifecycle ISO/IEC IT Security Techniques Supplier Relationships Establishes techniques between acquirer and supplier for supply chain risk management International Council on Systems Engineering (INCOSE) Systems Security Engineering (SSE) working group established to develop SSE updates to INCOSE SE Handbook The Open Group (TOG) The Open Trusted Technology Provider Framework (O-TTPF) - open standard that codifies best practices across the entire lifecycle covering: Product Development Secure Engineering Supply Chain Integrity Program Protection Process 5/1/12 Page 55 Distribution Statement A Cleared for public release by OSR on 4/25/2012, SR Case # 12-S-1842 applies. Comprehensive Program Protection 5/1/2012 Page 58 Distribution Statement A Cleared for public release by OSR on 4/25/2012, SR Case # 12-S-1841 applies. Tiered Supply Chain Problem (Notional Example) We are engaged with many parts of the Community for Software Assurance-related standardization System Integrator (e.g. Weapon System Platform) Software Router COTS S/W Custom Developer Open Source S/W 2nd Tier Supplier Foreign Suppliers OEM for Router Add-On Cards Supplier Foreign Coders Design Tools Web Auction Radar Supplier Threat can reside several layers down from System Integrator How is it shipped? How is it verified and validated? How is it physically protected? Do you execute a Blind Buy? 2nd Tier Supplier Manage Risks Criticality Schedule Cost 3rd Tier COTS Supplier FPGA A/D Converter Custom ASIC 1 st Tier Supplier 2 nd Tier Supplier 3 rd Tier Supplier 4 th Tier Supplier Program Protection Process 5/1/12 Page 56 Distribution Statement A Cleared for public release by OSR on 4/25/2012, SR Case # 12-S-1842 applies. Input Results: Criticality Results Mission Critical Functions Logic-Bearing Components System Impact (I, II, III, IV) Rationale Mission 1 CF 1 Processor X II Redundancy CF 2 SW Module Y I Performance Mission 2 CF 3 SW Algorithm A II Accuracy CF 4 FPGA 123 I Performance Supplier Risk Results Supplier Critical Components Supplier 1 Processor X Supplier Risk FPGA 123 Supplier Risk Findings Supplier 2 SW Algorithm A Cleared Personnel SW Module Y Cleared Personnel Vulnerability Assessment Results 3rd Tier Custom Development Critical Components Identified Exploitability (I, II, III, IV) System Impact (HW, SW, Exposure Firmware) Vulnerabilities Vulnerability 1 Low Low Processor X II Vulnerability 4 Medium Low Vulnerability 1 High High Vulnerability 2 Low Low SW Module Y I Vulnerability 3 Medium Medium Vulnerability 6 High Low SW Algorithm A None Very Low II Very Low Vulnerability 1 Low High FPGA 123 I Vulnerability 23 Low High Risk Mitigation and Countermeasure Options PPP Methodology Consequence of Losing Mission Capability Very High High Moderate Low Very Low Likelihood of Losing Mission Capability Near Certainty (VH) Highly Likely (H) Likely (M) Low Likelihood (L) Not Likely (VL) Likelihood Likelihood Initial Risk Posture R2 Consequence R2 Consequence R2 R1 R1 Risk Mitigation Decisions R1 ISO/IEC JTC1 SC22: ISO/IEC Technical Report (TR) Information technology -- Programming languages -- Guidance to avoiding vulnerabilities in programming languages through language selection and use. Published in October As published, the document includes language-independent summaries of nearly 70 classes of vulnerabilities. The working group is already drafting the 2 nd Edition of the report which will add information specific to individual programming languages. Use recent versions of compilers and DO NOT disable compiler warning flags SC7: ISO/IEC 15026, System and Software Assurance Publication of the standard, by both ISO/IEC and IEEE, in spring Program Protection Process 5/1/12 Page 57 Distribution Statement A Cleared for public release by OSR on 4/25/2012, SR Case # 12-S-1842 applies. 10

11 ISO/IEC/IEEE 15026, System and Software Assurance Other standards providing details of selected SW processes Source: J. Moore, SC7 Liaison Report, IEEE Software and Systems Engineering Standards Committee, Executive Committee Winter Plenary Meeting, February ISO/IEC12207: Life cycle processes for Software ISO/IEC24748: Guide to Life Cycle ISO/IEC 15289: Document - ation Interoperation ISO/IEC 16326: Project Mgmt ISO/IEC 15939: Measure - ment ISO/IEC 16085: Risk Mgmt ISO/IEC15288: Life cycle processes for systems Common vocabulary, process architecture, and process description Other standards providing details of selected system processes + conventions ISO/IEC15026: Additional practices for higher assurance systems System and software assurance focuses on the management of risk and assurance of safety, security, and dependability within the context of system and software life cycle Terms of Reference changed: ISO/IEC JTC1/SC7 WG7, previously System and Software Integrity SC7 WG9 61 Executive Summary 1. Introduction 1.1 Background 1.2 Purpose and Scope 1.3 Audience Acquisition Official Defined 1.4 Document Structure 1.5 Risk-Managed Software Acquisition Process 2. Planning Phase 2.1 Needs Determination, Risk Categorization, & Solution Alternatives 2.2 SwA Requirements 2.3 Acquisition Plan and/or Acquisition Strategy 2.4 Evaluation Plan and Criteria 2.5 SwA Due Diligence Questionnaires 3. Contracting Phase 3.1 Request for Proposals Work Statement Terms and Conditions Instructions to Suppliers Certifications Prequalification 3.2 Proposal Evaluation 3.3 Contract Negotiation 3.4 Contract Award 4. Implementation and Acceptance Phase 4.1 Contract Work Schedule 4.2 Change Control 4.3 Risk Plan 4.4 Assurance Case 4.5 Independent Software Testing 4.6 Software Acceptance SwA Acquisition & Outsourcing Handbook 5. Follow-on Phase 5.1 Support and Maintenance Risk Assurance Case Transition to Ops Other Change Considerations 5.2 Disposal or Decomissioning Appendix A/B Acronyms/Glossary Appendix C An Imperative for SwA in Acquisition Appendix D Software Due Diligence Questionnaires Table D-1. COTS Proprietary Software Questionnaire Table D-2. COTS Open-Source Software Questionnaire Table D-3. Custom Software Questionnaire Table D-4. GOTS Software Questionnaire Table D-5. Software Services Appendix E Other Examples of Due Diligence Questionnaires Appendix F Sample Language for the RFP and/or Contract F.1 Security Controls and Standards F.2 Securely Configuring Commercial Software F.3 Acceptance Criteria F.4 Certifications F.5 Sample Instructions to Offerors Sections F.6 Sample Work Statement Sections F.7 Open Web Application Security Project F.8 Certification of Originality Appendix H References ISO/IEC/IEEE Assurance Case Set of structured assurance claims, supported by evidence and reasoning (arguments), that demonstrates how assurance needs have been satisfied. Shows compliance with assurance objectives Provides an argument for the safety and security of the product or service. Built, collected, and maintained throughout the life cycle Derived from multiple sources System, Software, or Work Product Make the case for adequate quality/ assurance of the justify belief in Quality / Assurance Factor Claims Arguments Evidence is developed for Quality / Assurance Case supports Quality / Assurance Subfactor Sub-parts A high level summary Justification that product or service is acceptably safe, secure, or dependable Rationale for claiming a specified level of safety and security Conformance with relevant standards & regulatory requirements The configuration baseline Identified hazards and threats and residual risk of each hazard / threat Operational & support assumptions Attributes Clear Consistent Complete Comprehensible Defensible Bounded Addresses all life cycle stages Software Supply Chain Risk and Due-Diligence -- Table 1 SwA Concern Categories SwA Concern Categories Risks Purpose for Questions Software History and Licensing Development Process Software Security Training and Awareness Planning and Requirements Architecture and Design Software Development Built-in Software Defenses Component Assembly Testing Software Manufacture and Packaging Installation Assurance Claims and Evidence Support Software Change Timeliness of Vulnerability Mitigation Individual Malicious Behavior Security Track Record Financial History and Status Organizational History Foreign Interests and Influences Service Confidentiality Policies Operating Environment for Services Security Services and Monitoring 65 SwA Acquisition & Outsourcing Handbook Software Assurance in Acquisition: Mitigating Risks to the Enterprise Version 1.0, Oct 2008, available for community use; published by National Defense University Press, Feb 2009 Software Supply Chain Risk and Due-Diligence -- Table 1 SwA Concern Categories SwA Concern Risks Purpose for Questions Categories Software History The software supplier s development practice in To address supply chain concerns and identify and Licensing using code of unknown origin may be unable to risks pertaining to history/pedigree of software produce trustworthy software. during any and all phases of its life cycle that should have been considered by the supplier. Development If supplier project management does not perceive To determine whether project management Process the value of SwA and enforce best practices, they enforces software assurance related best will not be consistently implemented. practices. Software Security Developers unaware of software assurance best To determine whether training of developers in Training and practices are likely to implement software with SwA best practices is a supplier policy and Awareness security flaws (making it more susceptible to attack). practice. Planning and If nonfunctional requirements (security, quality, To determine whether the supplier s Requirements safety) are not specified, developers will not requirements analysis process explicitly implement them. addresses SwA requirements. Architecture and The software may be designed without considering To determine how security is considered during Design security or minimization of exploitable defects. the design phase. Software If developers lack qualified tools or if personnel are To ascertain that the supplier has and enforces Development allowed to inappropriately access or change policies and SwA practices in the development configuration items in the development environment, of software that use secure software then delivered software might have unspecified development environments to minimize risk features. The supplier might lack sufficient process exposures. capability to deliver secure products, systems or services. Built-in Software Defenses The software may lack preventive measures to help it resist attack effectively and proactively. To ensure that capabilities are designed to minimize the exposure of the software s vulnerabilities to external threats and to keep the software in a secure state regardless of the input and parameters it receives from its users or environment. 11

12 Software Supply Chain Risk and Due-Diligence -- Table 1 SwA Concern Categories SwA Concern Categories Risks Purpose for Questions Component Assembly Insufficient analysis of software components To ensure that the software components are used to assemble larger software packages thoroughly vetted for their security properties, may introduce vulnerabilities to the overall secure behaviors, and known types of package. weaknesses that can lead to exploitable vulnerabilities. Testing Software released with insufficient testing To determine whether the appropriate set of may contain an unacceptable number of analyses, reviews, and tests are performed exploitable defects. on the software throughout the life cycle which evaluate security criteria. Software Manufacture Vulnerabilities or malicious code could be To determine how the software goes through and Packaging introduced in the manufacturing or packaging the manufacturing process, how it is process. packaged, and how it remains secure. Installation The software may not install as advertised To ensure the supplier provides an and the acquirer may not get the software to acceptable level of support during the function as expected. installation process. Assurance Claims and Supplier assurance claims (with supporting To determine how suppliers communicate Evidence evidence) may be non-existent or their claims of assurance; ascertain what the insufficiently verified. claims have been measured against, and identify at what levels they will be verified. Support Supplier ceases to supply patches and new To ensure understanding of supplier policy for releases prior to the acquirer ending use of security fixes and when products are no software. Vulnerabilities may go unmitigated. longer supported. Software Change Weak change control procedures can corrupt To determine whether software changes are software and introduce new security adequately assessed and verified by supplier vulnerabilities. management. Timeliness of Sometimes it can be extremely difficult to To ensure security defects and configuration Vulnerability Mitigation make a software supplier take notice and errors are fixed properly and in a timely repair software to mitigate reported fashion. vulnerabilities. Table 2- Questions for COTS (Proprietary & Open Source), GOTS, and Custom Software No. Question 11 Does the license/contract restrict the licensee from discovering flaws or disclosing details about software defects or weaknesses with others (e.g., is there a gag rule or limits on sharing information about discovered flaws)? 12 Does the license/contract restrict communications or limit the licensee in any potential communication with third-party advisors about provisions for support (e.g., is there a gag rule or limits placed on the licensee that affect ability to discuss contractual terms or breaches) regarding the licensed or contracted product or service? 13 Does software have a positive reputation? Does software have a positive reputation relative to security? Are there reviews that recommend it? 14 Is the level of security where the software was developed the same as where the software will operate? Development Process 15 What are the processes (e.g., ISO 9000, CMMI, etc.), methods, tools (e.g., IDEs, compilers), techniques, etc. used to produce and transform the software (brief summary response)? 16 What security measurement practices and data does the company use to assist product planning? 17 Is software assurance considered in all phases of development? Explain. 18 How is software risk managed? Are anticipated threats identified, assessed, and prioritized? COTS Open- Source COTS Proprietary COTS Open- Source GOTS Custom Software Supply Chain Risk and Due-Diligence -- Table 1 SwA Concern Categories SwA Concern Categories Risks Purpose for Questions Individual Malicious A developer purposely inserts malicious code, To determine whether the supplier has and Behavior and supplier lacks procedures to mitigate risks enforces policies to minimize individual from insider threats within the supply chain. malicious behavior. Table 1 SwA Concern Categories -- (with interests relevant to security and privacy) SwA Concern Categories Risks Purpose for Questions Service Confidentiality Without policies to enforce client data confidentiality/ To determine the service provider s Policies privacy, acquirer s data could be at risk without confidentiality and privacy policies and service supplier liability. ensure their enforcement. Security Track Record A software supplier that is unresponsive to known software vulnerabilities may not mitigate/patch vulnerabilities in a timely manner. To establish insight into whether the supplier places a high priority on security issues and will be responsive to vulnerabilities they will need to mitigate. Table 3 - Questions for Hosted Applications No. Questions Financial History and Status A software supplier that goes out of business will be unable to provide support or mitigate product defects and vulnerabilities. To identify documented financial conditions or actions of the supplier that may impact its viability and stability, such as mergers, selloffs, lawsuits, and financial losses. Service Confidentiality Policies 1 What are the customer confidentiality policies? How are they enforced? 2 What are the customer privacy policies? How are they enforced? Organizational History Foreign Interests and Influences There may be conflicting circumstances or competing interests within the organization that may lead to increased risk in the software development. There may be controlling foreign interests (among organization officers or from countries) with malicious intent to the users country or organization planning to use the software. To understand the supplier s organizational background, roles, and relationships that might have an impact on supporting the software. To help identify supplier companies that may have individuals with competing interests or malicious intent to a domestic buyer/user. 3 What are the policies and procedures used to protect sensitive information from unauthorized access? How are the policies enforced? 4 What are the set of controls to ensure separation of data and security information between different customers that are physically located in the same data center? On the same host server? Operating Environment for Services 5 Who configures and deploys the servers? Are the configuration procedures available for review, including documentation for all registry settings? Service Confidentiality Policies Operating Environment for Services Security Services and Monitoring Without policies to enforce client data confidentiality/ privacy, acquirer s data could be at risk without service supplier liability. Operating environment for the services may not be hardened or otherwise secure. Insufficient security monitoring may allow attacks to impact services. To determine the service provider s confidentiality and privacy policies and ensure their enforcement. To understand the controls the supplier has established to operate the software securely. To ensure software and its operating environment are regularly reviewed for adherence to SwA requirements through periodic testing and evaluation. 7 What are the data backup policies and procedures? How frequently are the backup procedures verified? 11 What are the agents or scripts executing on servers of hosted applications? Are there procedures for reviewing the security of these scripts or agents? 12 What are the procedures and policies used to approve, grant, monitor and revoke access to the servers? Are audit logs maintained? 13 What are the procedures and policies for handling and destroying sensitive data on electronic and printed media? 15 What are the procedures used to approve, grant, monitor, and revoke file permissions for production data and executable code? Table 2- Questions for COTS (Proprietary & Open Source), GOTS, & Custom Software No Question 1 Can the pedigree of the software be established? Briefly explain what is known of the people and processes that created the software. 2 Explain the change management procedure that identifies the type and extent of changes conducted on the software throughout its life cycle. 3 What type of license(s) are available for the open source software? Is it compatible with other software components in use? Is indemnification provided, and will the supplier indemnify the purchasing organization from any issues in the license agreement? Explain. 4 Is there a clear chain of licensing from original author to latest modifier? Describe the chain of licensing. 5 What assurances are provided that the licensed software does not infringe upon any copyright or patent? Explain. 6 Does the company have corporate policies and management controls in place to ensure that only corporate-approved (licensed and vetted) software components are used during the development process? Explain. COTS Proprietary GOTS Custom 7 Are licensed software components still valid for the intended use? 8 Is the software in question original source or a modified version? 9 Has the software been reviewed to confirm that it does not infringe upon any copyright or patent? 10 How long has the software source been available? Is there an active user community providing peer review and actively evolving the software? More Due-Diligence Questions Relevant to Acquisition & Outsourcing Relevant to deliberate actions that are controllable and preventable by developers that have security implications Were any compiler warnings disabled for the software being delivered? Relevant to hosted applications and services Cloud computing, XXXX_as a Service, SOA, Seeking more examples from security aware community 12

13 Recent Resources for SCRM NIST IR The Open Group Trusted Technology Provider Framework SCRM processes, tools and techniques: Enhance security through the implementation of system security engineering (e.g. criticality analysis and defensive engineering practices) throughout the system life cycle. Optimize acquisition and contracting to define requirements and source selection criteria that reduce supply chain risk, give preference to vendors that minimize supply chain risk in verifiable ways, and evaluate security equally with other desirable factors, such as low cost, rapid deployment, or new features. Implement acquisition processes to document and monitor risk mitigation methods and requirements and provide for the update of documentation throughout the system lifecycle. Best Practices, Tools and Techniques References General SCRM References The following documents provide systems security engineering guidance and detailed risk management best practice for use in commercial or government systems. Draft NISTIR 7622, Piloting Supply Chain Risk for Federal Information Systems, June National Defense Industrial Association (NDIA) System Assurance Committee Engineering for System Assurance. SCRM References from the Department of Defense The following documents describe SCRM best practice for NSS, provide guidance on the successful implementation of SCRM pilots that incorporate all-source threat information, summarize the DoD pilot experience, and identify trusted suppliers of integrate circuits as accredited by the Defense Microelectronic Agency. Key Practices and Implementation Guide for the DoD Comprehensive National Cybersecurity Initiative 11: Supply Chain Risk Pilot Program. February 25, Concept of Operations for the DoD Comprehensive National Cybersecurity Initiative 11: Supply Chain Risk Pilot Program. August 25, Draft Comprehensive National Cybersecurity Initiative (CNCI) DoD Supply Chain Risk (SCRM) Pilot Program Report, November 30, 2010 List of Trusted Integrated Circuits (IC) Suppliers available at SCRM References from the Department of Homeland Security The following documents and the assessment tool provide guidance for civilian Departments and agencies guidance for the successful implementation of a SCRM pilot. Used with the NISTIR 7622, which identifies key practices, the following documents enable the development and operation of systems to manage supply chain risks. Concept of Operations for the Civilian Agency Pilot Program (CAPP) Template for a SCRM Pilot Plan of Action and Milestones SCRM Capability Assessment Tool Supply Chain Risk (SCRM) processes, tools and techniques: Numerous SCRM processes, tools and techniques facilitate the implementation of SCRM USG-wide. Departments and Agencies shall adopt and tailor these recommended SCRM processes, tools, and techniques, and apply them to the procurement and operation of mission critical elements within NSS, to include those which: Control the quality, configuration, and security of software, hardware, and systems throughout their lifecycles, including commercial elements or sub-elements. Detect the occurrence, reduce the likelihood of occurrence, and mitigate the consequences of products containing counterfeit elements or malicious functions. Develop requirements or capabilities to detect the occurrence of vulnerabilities within custom and commodity hardware and software through enhanced test and evaluation. Best Practices, Tools and Techniques References Software Assurance Community documents from [email protected] Software Assurance in Acquisition and Contract Language ( Software Supply Chain Risk and Due Diligence ( Polydys, Mary L. and Wisseman, Stan., Software Assurance in Acquisition: Mitigating Risks to the Enterprise, A Reference Guide for Security-Enhanced Software Acquisition and Outsourcing, Information Resources College Occasional Paper, National Defense University Press, Washington, D.C. February ( Polydys, Mary L. and Wisseman, Stan. (2007, May). "Software Assurance: Five Essential Considerations for Acquisition Officials." CrossTalk The Journal of Defense Software Engineering, Vol. 20, No. 5. ( Robert J. Ellison, John B. Goodenough, Charles B. Weinstock, Carol Woody Evaluating and Mitigating Software Supply Chain Security Risks, May 2010 ( Goertzel, Karen, Theodore Winograd, et al. for Department of Homeland Security and Department of Defense Data and Center for Software. Enhancing the Development Life Cycle to Produce Secure Software: A Reference Guidebook on Software Assurance, October ( Bob Ellison, CERT, Software Engineering Institute and Carol Woody, CERT, Software Engineering Institute, Considering Software Supply Chain Risks, CrossTalk The Journal of Defense Software Engineering,, September/October 2010 ( BSI/version/1/part/4/data/1009EllisonWoody.pdf?branch=main&language=default) Bob Ellison, CERT, Software Engineering Institute and Carol Woody, CERT, Software Engineering Institute, Supply-Chain Risk : Incorporating Security into Software Development, March 2010 ( 13

14 Best Practices, Tools and Techniques References Industry Standards for SCRM EIA Standard for Preparing an Electronic Component Plan IDEA-STD-1010 Diminishing Manufacturing Sources and Material Shortages (DMSMS) Guidebook SAE-AS9120 Quality Systems for Aerospace Product Distributors SAE-AS5553 Counterfeit Electronic Parts; Avoidance, Detection, Mitigation and Disposition Federal IT Security References The following documents provide a foundation of federal information technology security practices or provide detailed guidance specific to managing risks inherent in the information technology product or services supply chain. CNSS Instruction No. 1253, Security Categorization and Control Selection for National Security Systems, October 2009 NIST Special Publication Revision 3, Recommended Security Controls for Federal Information Systems and Organizations, August 2009 (includes updates as of ). NIST Special Publication Revision 1, Guide for Applying the Risk Framework to Federal Information Systems: A Security Life Cycle Approach, February There are many definitions of weakness -- in this context A(software) weakness is a property of software/ systems that, under the right conditions, may permit unintended / unauthorized behavior. *Common Weakness Enumeration (CWE) There are many definitions of vulnerability -- in this context: A(software) vulnerability is a collection of one or more weaknesses that contain the right conditions to permit unauthorized parties to force the software to perform unintended behavior (a.k.a. is exploitable ) *Common Vulnerabilities and Exposures (CVE) * Part of ITU-T Cyber Information Exchange (CYBEX) series 1500; co-sponsored by DHS NCSD Mitigating Software Risks Avoid installing software with MALWARE pre-installed. Determine that no publicly reported VULNERABILITIES remain in code prior to operational acceptance, and that future discoveries of common vulnerabilities and exposures can be quickly patched. Determine that exploitable software WEAKNESSES that put the users most at risk are mitigated prior to operational acceptance. MAEC CVE CWE Software Assurance Software Assurance (SwA) is the level of confidencethat software functions as intended and is is free of from vulnerabilities, either intentionally or unintentionally designed or inserted as part of the software throughout the life cycle.* Derived From: CNSSI-4009 Automation Languages, enumerations, registries, tools, and repositories throughout the Lifecycle Including design, coding, testing, deployment, configuration and operation Malware (malicious software) Automation is one piece Malwareis software designed to disrupt computer operation, gather sensitive information, or gain unauthorized access to computer systems. Malware is a general term used to describe any kind of software or code specifically designed to exploit a computer, or the data it contains, without consent. The expression is a general term used by computer professionals to mean a variety of forms of hostile, intrusive, or annoying software. It is software, and can also appear in the form of script or code. Malware includes computer viruses, worms, trojans, spyware, most rootkits, and other malicious programs (and can include unwanted programs, such as adware). In law, malware is sometimes known as a computer contaminant, for instance in the legal codes of several U.S. states of the SwA puzzle. 14

15 Many DHS, DoD, and NIST sponsored efforts are key to changing how software-based systems are developed, deployed & operated securely. These are (or are becoming used in) international standards OCIL ARF CCI CAG ITU-T GUI intruder tools Solutions Also Emerged Over Time stealth /advanced scanning techniques widespread attacks on DNS infrastructure automated probes/scans executable code attacks (against browsers) automated widespread attacks network mgmt. diagnostics sniffers propagation of malicious code widespread attacks using NNTP to distribute attack DDoS attacks binary encryption increase in tailored worms sophisticated command & control Attack Sophistication diffuse spyware anti-forensic techniques home users targeted distributed attack tools increase in wide-scale Trojan horse distribution CIEL CCv4 hijacking sessions back doors disabling audits Internet social engineering attacks password cracking widespread denial-ofservice attacks www attacks Windows-based remote controllable Trojans (Back Orifice) techniques to analyze code for vulnerabilities without source code packet spoofing automated probes/scans exploiting known vulnerabilities burglaries password guessing 1980 s 1990 s 2000 s 2010 s Making Security Measurable (MSM): You Are Here Architecting Security with Information Standards for COIs Asset Vulnerability Configuration Threat Software Assurance Enterprise Security Threat Design Design Vulnerabilities System Development System Certification Intrusion Detection Incident Deploy Build Assess Test Exploits Attacks Change Trust Identity Central Reporting Test Deploy Malware CWE, CAPEC, CWSS, CWRAF CPE, CCE, OVAL, OCIL, XCCDF, AssetId, ARF CVE, CWE, CAPEC, MAEC, CybOX, IODEF 2012 MITRE 2012 MITRE Cyber Threats Emerged Over Time stealth /advanced scanning techniques propagation of malicious code widespread attacks using NNTP to distribute attack DDoS attacks binary encryption increase in tailored worms widespread attacks on DNS infrastructure GUI intruder tools network mgmt. diagnostics automated probes/scans executable code attacks (against browsers) automated widespread attacks sophisticated command & control Attack Sophistication Asset Vulnerability Configuration Threat diffuse spyware sniffers anti-forensic techniques home users targeted distributed attack tools increase in wide-scale Trojan horse distribution System Development System Certification Intrusion Detection Incident hijacking sessions back doors disabling audits Internet social engineering attacks password cracking widespread denial-ofservice attacks www attacks Windows-based remote controllable Trojans (Back Orifice) techniques to analyze code for vulnerabilities without source code Change Trust Identity Central Reporting packet spoofing automated probes/scans exploiting known vulnerabilities burglaries password guessing 1980 s 1990 s 2000 s 2010 s 15

16 Asset Inventory Configuration Guidance Vulnerability Threat Intrusion Detection Incident Standardization Efforts leveraged by the Security Content Automation Protocol (SCAP) What IT systems do I have in my enterprise? CPE (Platforms) What known vulnerabilities do I need to worry about? CVE (Vulnerabilities) What vulnerabilities do I need to worry about right now? CVSS(Scoring System) Operations Security Processes How can I configure my systems more securely? CCE (Configurations) How do I define a policy of secure configurations? XCCDF (Configuration Checklists) Assessment of System Development, Integration, & Sustainment Activities and Certification & Accreditation How can I be sure my systems conform to policy? How can I be sure the operation of my systems conforms to policy? OVAL(Assessment Language) OCIL(Interactive Language) What weaknesses in my software could be exploited? CWE (Weaknesses) What attacks can exploit which weaknesses? CAPEC (Attack Patterns) Operational Enterprise Networks How can we recognize malware & share that info? What observable behavior might put my enterprise at risk? MAEC (Malware Attributes) CybOX(Cyber Observables) Development & Sustainment Security Processes Trust Enterprise IT Change Identity Centralized Reporting Enterprise IT Asset What events should be logged, and how? How can I aggregate assessment results? CEE (Events) ARF (Assessment Results) 2012 MITRE Asset Inventory Configuration Guidance Vulnerability Threat Intrusion Detection Incident Standardization Efforts focused on mitigating risks and enabling faster incident response What IT systems do I have in my enterprise? CPE (Platforms) What known vulnerabilities do I need to worry about? CVE (Vulnerabilities) CPE/ SWID/ OVAL/ ARF CCE/ OVAL/OCIL/ XCCDF/CPE /SWID/CCS S/ARF CVE/CWE/ CVE/CWE/ CVE/CWE/ CVSS/CCE/ CVSS/CCE/ CVSS/CCE/ OVAL/OCIL/ OVAL/OCIL/XC OVAL/OCIL/ARF/X XCCDF/CPE/ CDF/CPE/CAPE CCDF/CPE/CAPEC CWSS/SWID C/MAEC/CybO /MAEC/CEE/CybO X/SWID X/SWID Operations Security Processes What vulnerabilities do I need to worry about right now? How can I configure my systems more securely? CVSS(Scoring System) CCE (Configurations) How do I define a policy of secure configurations? XCCDF (Configuration Checklists) Assessment of System Development, Integration, & Sustainment Activities and Certification & Accreditation How can I be sure my systems conform to policy? How can I be sure the operation of my systems conforms to policy? OVAL(Assessment Language) OCIL(Interactive Language) What weaknesses in my software could be exploited? CWE (Weaknesses) What attacks can exploit which weaknesses? CAPEC (Attack Patterns) CWE/CAPEC/ CWSS/MAEC/ OVAL/OCIL/X CCDF/CCE/CP E/ARF/SWID/S AFES/SACM Development & Sustainment Security Processes Trust CVE/CWE/ CVSS/CCE/ CCSS/OVAL/XCCDF /OCIL/ CPE/CAPEC/ MAEC/CWSS/ CEE/ARF/ SWID/CybOX Enterprise IT Identity Change Operational Enterprise Networks CVE/CWE/CVSS/CCE/CCSS/OVAL/OCIL/XC CDF/CPE/CAPEC/MAEC/CWSS/CEE/ARF/S WID/CybOX Centralized Reporting Enterprise IT Asset How can we recognize malware & share that info? What observable behavior might put my enterprise at risk? What events should be logged, and how? How can I aggregate assessment results? MAEC (Malware Attributes) CybOX(Cyber Observables) CEE (Events) ARF (Assessment Results) 2012 MITRE Cyber Ecosystem Standardization Efforts What IT systems do I have in my enterprise? CPE (Platforms) What known vulnerabilities do I need to worry about? CVE (Vulnerabilities) What vulnerabilities do I need to worry about right now? How can I configure my systems more securely? CVSS(Scoring System) CCE (Configurations) Asset Inventory Configuration Guidance Vulnerability Threat Intrusion Detection Incident How do I define a policy of secure configurations? How can I be sure my systems conform to policy? How can I be sure the operation of my systems conforms to policy? What weaknesses in my software could be exploited? What attacks can exploit which weaknesses? How can we recognize malware & share that info? What observable behavior might put my enterprise at risk? What events should be logged, and how? How can I aggregate assessment results? XCCDF (Configuration Checklists) OVAL(Assessment Language) OCIL(Interactive Language) CWE (Weaknesses) CAPEC (Attack Patterns) MAEC (Malware Attributes) CybOX(Cyber Observables) CEE (Events) ARF (Assessment Results) 2012 MITRE Assessment of System Development, Integration, & Sustainment Activities and Certification & Accreditation CW E/CAPEC/C W SS/MAEC/OV AL/OCIL/XCCD F/CCE/CPE/AR F/SW ID/SAFES/ SACM Development & Sustainment Security Processes Trust CPE/ SW ID/ OVAL/ ARF Enterprise IT Change CCE/ OVAL/OCIL/ XCCDF/CPE /SW ID/CCS S/ARF CVE/CW E/CVS S/CCE/CCSS/O VAL/XCCDF/O CIL/ CPE/CAPEC/M AEC/CW SS/CE E/ARF/SW ID/C ybox CVE/CW E/ CVSS/CCE/ OVAL/OCIL/X CCDF/CPE/ CW SS/SWID Identity CVE/CW E/ CVSS/CCE/ OVAL/OCIL/XC CDF/CPE/CAPE C/MAEC/CybO X/SW ID Operations Security Processes Centralized Reporting CVE/CW E/ CVSS/CCE/ OVAL/OCIL/ARF/ XCCDF/CPE/CAP EC/MAEC/CEE/C ybox/sw ID Operational Enterprise Networks CVE/CW E/CVSS/CCE/CCSS/OVAL/OCIL/XC CDF/CPE/CAPEC/MAEC/CWSS/CEE/ARF/S W ID/CybOX Enterprise IT Asset 16

17 Knowledge Repositories Mitigating Risk Exposures Asset Definition Asset Inventory CPE/SW ID/OVAL Configuration Guidance Configuration Guidance XCCDF/OVAL/ CCE/CCSS/OCIL Vulnerability Alert CVE/CW E/O VAL/CVSS/ CVRF Vulnerability Responding to Security Threats Threat Alert CVE/CW E/CVS S/CAPEC/MAE C/CybOX Threat Indicator Sharing Intrusion Detection IODEF, CVE, CPE, MAEC, CEE, CybOX, RID, RID-T Incident Report Incident CYBEX, CW E, IODEF, OVAL, CVE, CPE, CVSS, MAEC, CEE, CW SS, CybOX, RID, RID-T SwA-funded Cyber Ecosystem Standardization What IT systems do I have in my enterprise? CPE (Platforms) What known vulnerabilities do I need to worry about? CVE (Vulnerabilities) What vulnerabilities do I need to worry about right now? CVSS(Scoring System) How can I configure my systems more securely? CCE (Configurations) OVAL/XCCDF /OCIL/CCE/C CSS/CPE/ SW ID/ARF System & Software Assurance Guidance/ Requirements CW E/CAPEC/ SBVR/CW SS/ MAEC Assessment of System Development, Integration, & Sustainment Activities and Certification & Accreditation CPE/ SW ID/ OVAL/ ARF CCE/ OVAL/OCIL/ XCCDF/CPE /SW ID/CCS S/ARF CVE/CW E/ CVSS/CCE/ OVAL/OCIL/XCC DF/CPE/ CW SS/SWID CVE/CW E/ CVSS/CCE/ OVAL/OCIL/XCCD F/CPE/CAPEC/MA EC/CybOX/SW ID CVE/CW E/ CVSS/CCE/ OVAL/OCIL/ARF/X CCDF/CPE/CAPEC /MAEC/CEE/CybOX /SW ID Operations Security Processes How do I define a policy of secure configurations? XCCDF (Configuration Checklists) How can I be sure my systems conform to policy? OVAL(Assessment Language) How can I be sure the operation of my systems conforms to policy? OCIL(Interactive Language) What weaknesses in my software could be exploited? CWE (Weaknesses) What attacks can exploit which weaknesses? CAPEC (Attack Patterns) How can we recognize malware & share that info? MAEC (Malware Attributes) CW E/CAPEC/C W SS/MAEC/OV AL/OCIL/XCCD F/CCE/CPE/AR F/SW ID/SAFES/ SACM Development & Sustainment Security Processes Trust CVE/CW E/CVS S/CCE/CCSS/O VAL/XCCDF/O CIL/ CPE/CAPEC/M AEC/CW SS/CE E/ARF/SW ID/C ybox Enterprise IT Change Identity Centralized Reporting Operational Enterprise Networks CVE/CW E/CVSS/CCE/CCSS/OVAL/OCIL/XC CDF/CPE/CAPEC/MAEC/CWSS/CEE/ARF/S W ID/CybOX Enterprise IT Asset What observable behavior might put my enterprise at risk? What events should be logged, and how? How can I aggregate assessment results? CybOX(Cyber Observables) CEE (Events) ARF (Assessment Results) 2011 MITRE Knowledge Repositories Asset Definition CPE/SW ID/OVAL Configuration Guidance XCCDF/OVAL/ CCE/CCSS/OCIL Vulnerability Alert CVE/CW E/O VAL/CVSS/ CVRF Threat Alert CVE/CW E/CVS S/CAPEC/MAE C/CybOX Indicator Sharing IODEF, CVE, CPE, MAEC, CEE, CybOX, RID, RID-T Incident Report CYBEX, CW E, IODEF, OVAL, CVE, CPE, CVSS, MAEC, CEE, CW SS, CybOX, RID, RID-T Standardization Efforts leveraged by the Security Content Automation Protocol (SCAP) What IT systems do I have in my enterprise? What known vulnerabilities do I need to worry about? CPE (Platforms) CVE (Vulnerabilities) Asset Inventory Configuration Guidance Vulnerability Threat Intrusion Detection Incident What vulnerabilities do I need to worry about right now? How can I configure my systems more securely? CVSS(Scoring System) CCE (Configurations) OVAL/XCCDF /OCIL/CCE/C CSS/CPE/ SW ID/ARF System & Software Assurance Guidance/ Requirements CW E/CAPEC/ SBVR/CW SS/ MAEC Assessment of System Development, Integration, & Sustainment Activities and Certification & Accreditation CPE/ SW ID/ OVAL/ ARF CCE/ OVAL/OCIL/ XCCDF/CPE /SW ID/CCS S/ARF CVE/CW E/ CVSS/CCE/ OVAL/OCIL/XCC DF/CPE/ CW SS/SWID CVE/CW E/ CVSS/CCE/ OVAL/OCIL/XCCD F/CPE/CAPEC/MA EC/CybOX/SW ID CVE/CW E/ CVSS/CCE/ OVAL/OCIL/ARF/X CCDF/CPE/CAPEC /MAEC/CEE/CybOX /SW ID Operations Security Processes How do I define a policy of secure configurations? XCCDF (Configuration Checklists) How can I be sure my systems conform to policy? OVAL(Assessment Language) How can I be sure the operation of my systems conforms to policy? OCIL(Interactive Language) What weaknesses in my software could be exploited? CWE (Weaknesses) What attacks can exploit which weaknesses? CAPEC (Attack Patterns) How can we recognize malware & share that info? MAEC (Malware Attributes) CW E/CAPEC/C W SS/MAEC/OV AL/OCIL/XCCD F/CCE/CPE/AR F/SW ID/SAFES/ SACM Development & Sustainment Security Processes Trust CVE/CW E/CVS S/CCE/CCSS/O VAL/XCCDF/O CIL/ CPE/CAPEC/M AEC/CW SS/CE E/ARF/SW ID/C ybox Enterprise IT Change Identity Centralized Reporting Operational Enterprise Networks CVE/CW E/CVSS/CCE/CCSS/OVAL/OCIL/XC CDF/CPE/CAPEC/MAEC/CWSS/CEE/ARF/S W ID/CybOX Enterprise IT Asset What observable behavior might put my enterprise at risk? What events should be logged, and how? How can I aggregate assessment results? CybOX(Cyber Observables) CEE (Events) ARF (Assessment Results) 2011 MITRE Cyber Ecosystem Standardization Efforts What IT systems do I have in my enterprise? CPE (Platforms) What known vulnerabilities do I need to worry about? CVE (Vulnerabilities) What vulnerabilities do I need to worry about right now? CVSS(Scoring System) How can I configure my systems more securely? CCE (Configurations) Efforts focused on mitigating risks and enabling more robust continuous monitoring and faster incident response What IT systems do I have in my enterprise? What known vulnerabilities do I need to worry about? What vulnerabilities do I need to worry about right now? How can I configure my systems more securely? CPE (Platforms) CVE (Vulnerabilities) CVSS(Scoring System) CCE (Configurations) How do I define a policy of secure configurations? XCCDF (Configuration Checklists) How do I define a policy of secure configurations? XCCDF (Configuration Checklists) How can I be sure my systems conform to policy? OVAL(Assessment Language) How can I be sure my systems conform to policy? OVAL(Assessment Language) How can I be sure the operation of my systems conforms to policy? OCIL(Interactive Language) How can I be sure the operation of my systems conforms to policy? OCIL(Interactive Language) What weaknesses in my software could be exploited? CWE (Weaknesses) What attacks can exploit which weaknesses? CAPEC (Attack Patterns) How can we recognize malware & share that info? MAEC (Malware Attributes) What weaknesses in my software could be exploited? CWE (Weaknesses) What attacks can exploit which weaknesses? CAPEC (Attack Patterns) How can we recognize malware & share that info? MAEC (Malware Attributes) What observable behavior might put my enterprise at risk? CybOX(Cyber Observables) What observable behavior might put my enterprise at risk? CybOX(Cyber Observables) What events should be logged, and how? CEE (Events) What events should be logged, and how? CEE (Events) How can I aggregate assessment results? ARF (Assessment Results) How can I aggregate assessment results? ARF (Assessment Results) 2011 MITRE 2011 MITRE 17

18 Vulnerability Type Trends: A Look at the CVE List ( ) Fighting Botnets and Malware Standardized ways of characterizing botnets, bots, and their behaviors, and providing indications of bot infection is an important element in developing voluntary best practices in botnet detection, notification and mitigation. The information conveyed to end-users should be reliable, unambiguous, and not confusing. Given that there may be multiple providers (ISPs and other security services providers) notifying end-users, it will be beneficial if they provide the same types of information and use a common terminology; represented using structures that are easily consumed by machines. Sponsored by DHS NPPD NCSD: 1. The Malware Attribute Enumeration and Characterization (MAEC) is a language for describing malware (include bot) attributes and behaviors. 2. The Cyber Observables Expression (CybOX) is a language for capturing observable events and indicators at various levels of infrastructure, from the endpoint up to the higher levels of the network. In this context, the usage of standardized cyber security languages developed expressly for such purposes can help eliminate the roadblocks to such information sharing through their definition of a common terminology and structure, thus essentially solving the issue of what information to share and how to share it. As such, these standardized languages can facilitate the exchange of information between providers to promote improved practices for detecting, notifying, and mitigating botnets. Since these standardized languages are represented using structures that are easily consumed by machines, they can enable the providers to directly provide information that could be used by applications on end-user systems to detect and mitigate botnet activity. When used in combination, MAEC and CybOX provide a powerful and flexible way of characterizing bots and botnets. On the outset, CybOX can be used to capture the specific network observable events, such as command and control mechanisms, that may be associated with a botnet. After further analysis is performed, MAEC can integrate this CybOX data into a characterization of the attributes and behaviors exhibited by the bots utilized by the botnet. This MAEC/CybOX data can then be exchanged in a machine consumable format between other providers for further detection and mitigation of the botnet infrastructure, and can also be utilized as a mechanism for detecting the bot components on end-user systems. There s a good deal of interest in MAEC and CybOX as evidenced by industry participation in the MAEC and CAPEC mailing lists, 240 and 190 participants, respectively. In addition several vendors have plans to generate information in MAEC and CybOX after the next versions are released MITRE Knowledge RepositoriesMitigating Risk Exposures Asset Definition CPE/SW ID/OVAL Asset Inventory OVAL/XCCDF /OCIL/CCE/C CSS/CPE/ SW ID/ARF Develop System & Software Assurance Guidance/ Requirements CW E/CAPEC/ SBVR/CW SS/ MAEC Responding to Security Threats Configuration Guidance Vulnerability Alert Threat Alert CVE/CW E/O VAL/CVSS/ CVRF XCCDF/OVAL/ CCE/CCSS/OCIL Indicator Sharing CVE/CW E/CVS S/CAPEC/MAE C/CybOX CYBEX, CW E, IODEF, OVAL, CVE, CPE, CVSS, MAEC, CEE, CW SS, CybOX, RID, RID-T Respond Configuration Guidance Vulnerability Threat Intrusion Detection Identify CPE/ SW ID/ OVAL/ ARF Incident Report IODEF, CVE, CPE, MAEC, CEE, CybOX, RID, RID-T CCE/ OVAL/OCIL/ XCCDF/CPE /SW ID/CCS S/ARF CVE/CW E/ CVSS/CCE/ OVAL/OCIL/XCC DF/CPE/ CW SS/SWID CVE/CW E/ CVSS/CCE/ OVAL/OCIL/XCCD F/CPE/CAPEC/MA EC/CybOX/SW ID Incident CVE/CW E/ CVSS/CCE/ OVAL/OCIL/ARF/X CCDF/CPE/CAPEC /MAEC/CEE/CybOX /SW ID Operations Security Processes Assessment of System Development, Integration, & Sustainment Activities and Certification & Accreditation Deploy CVE/CW E/CVS S/CCE/CCSS/O VAL/XCCDF/O CIL/ CPE/CAPEC/M AEC/CW SS/CE E/ARF/SW ID/C ybox CW E/CAPEC/C W SS/MAEC/OV AL/OCIL/XCCD F/CCE/CPE/AR F/SW ID/SAFES/ SACM Development & Sustainment Security Processes Trust Enterprise IT Change Operational Enterprise Networks CVE/CW E/CVSS/CCE/CCSS/OVAL/OCIL/XC CDF/CPE/CAPEC/MAEC/CWSS/CEE/ARF/S W ID/CybOX Identity Centralized Reporting Enterprise IT Asset Removing and Preventing the Vulnerabilities Requires More Specific Definitions CWEs Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting ) (79) Improper Neutralization of Script-Related HTML Tags in a Web Page (Basic XSS) (80) Improper Neutralization of Script in an Error Message Web Page (81) Improper Neutralization of Script in Attributes of IMG Tags in a Web Page (82) Improper Neutralization of Script in Attributes in a Web Page (83) Improper Neutralization of Encoded URI Schemes in a Web Page (84) Doubled Character XSS Manipulations (85) Improper Neutralization of Invalid Characters in Identifiers in Web Pages (86) Improper Neutralization of Alternate XSS Syntax (87) Improper Restriction of Operations within the Bounds of a Memory Buffer (119) Buffer Copy without Checking Size of Input ('Classic Buffer Overflow ) (120) Write-what-where Condition (123) Out-of-bounds Read (125) Improper Handling of Length Parameter Inconsistency (130) Improper Validation of Array Index (129) Return of Pointer Value Outside of Expected Range (466) Access of Memory Location Before Start of Buffer (786) Access of Memory Location After End of Buffer (788) Buffer Access with Incorrect Length Value 805 Untrusted Pointer Dereference (822) Use of Out-of-range Pointer Offset (823) Access of Uninitialized Pointer (824) Expired Pointer Dereference (825) Path Traversal (22) Relative Path Traversal (23) Path Traversal: '../filedir' (24) Path Traversal: '/../filedir' (25) < more here > Path Traversal: '...//' (34) Path Traversal: '.../...//' (35) Absolute Path Traversal (36) Path Traversal: '/absolute/pathname/here (37) Path Traversal: '\absolute\pathname\here (38) Path Traversal: 'C:dirname (39) Path Traversal: '\\UNC\share\name\' (Windows UNC Share) (40) 2011 MITRE Develop Identify Respond Deploy 2011 MITRE 2011 MITRE 18

19 Cyber Ecosystem Standardization Efforts What IT systems do I have in my enterprise? CPE (Platforms) What known vulnerabilities do I need to worry about? CVE (Vulnerabilities) What vulnerabilities do I need to worry about right now? CVSS(Scoring System) How can I configure my systems more securely? CCE (Configurations) How do I define a policy of secure configurations? XCCDF (Configuration Checklists) How can I be sure my systems conform to policy? OVAL(Assessment Language) How can I be sure the operation of my systems conforms to policy? OCIL(Interactive Language) What weaknesses in my software could be exploited? CWE (Weaknesses) What attacks can exploit which weaknesses? CAPEC (Attack Patterns) How can we recognize malware & share that info? MAEC (Malware Attributes) What observable behavior might put my enterprise at risk? CybOX(Cyber Observables) What events should be logged, and how? CEE (Events) How can I aggregate assessment results? ARF (Assessment Results) Leverage Common Weakness Enumeration (CWE) to mitigate risks to mission/business domains CWE is a formal list of software weakness types created to: Serve as a common language for describing software security weaknesses in architecture, design, or code. Serve as a standard measuring stick for software security tools targeting these weaknesses. Provide a common baseline standard for weakness identification, mitigation, and prevention efforts. Some Common Types of Software Weaknesses: Buffer Overflows, Format Strings, Etc. Structure and Validity Problems Common Special Element Manipulations Channel and Path Errors Handler Errors User Interface Errors Pathname Traversal and Equivalence Errors Authentication Errors Resource Errors Insufficient Verification of Data Code Evaluation and Injection Randomness and Predictability 2011 MITRE CVE 1999 to MITRE 2011 MITRE CVE Vendor/Industry Engagement Making Security Measureable : measurablesecurity.mitre.org Sponsored by DHS with MITRE as technical lead Open, community efforts that are freeto use Resources provided for voluntary adoption XML-based 2011 MITRE Some important things to note 19

20 Differing levels of maturity Notional W Effort CVE OVAL CWE CAPEC CWE CCR CWSS CWRAF Maturity Very Mature Very Mature Mature Somewhat Mature Brand-new Brand-new Brand-new W d We encourage you to get involved in these communities W d : The set of all discovered software weaknesses in W What is the context? Where can automation help - today? W Notional V W d What problems are we trying to solve? Where do we start? V: The set of all vulnerabilities in W S: The set of all software in existence at some point in time S Notional W Notional W V V d W d W: The set of all instances of software weaknesses in S V d : The set of all discoveredvulnerabilities in V 20

21 Notional Notional W W V W d V W d V d V d What does the future hold? Maybe just some things get worse? W S Notional W Notional V W d V W d V d V d We know it s notthis, at least not in the near-term One reasonable near-term goal W Notional Increase in the percentage of weaknesses that are discovered W W d Notional V V d W d Decreased number of vulnerabilities V V d Increase in the percentage of vulnerabilities that are discovered Maybe the problem grows unbounded? Is this really better? Yes 21

22 For the software we re responsible for W Notional A 4 W 4 W d A 3 W 3 How do we identify these? Weaknesses we really care about which weaknesses are most important? A 2 A 1 A 0 W 2 W 1 W 0 Diagram from: Vulnerability of Energy Delivery Control Systems, INL, September 2011 For the software we re responsible for Notional For the software we re responsible for W Notional Attacks that target these weaknesses V W d Weaknesses we really care about V d V cve How do we identify these? how can those weaknesses be attacked? Vulnerabilities identified with a CVE are a good starting point where should we start? Weaknesses and attacks relevant here Dictionaryof publicly-disclosed vulnerabilities with unique identifiers and here and here CVE ID Status Description References CVE entries feed into NVD Note: Each CVE entry is the result of expert analysis to verify, de-conflict and de-duplicate public vulnerability disclosures and here Maybe different than those relevant here Diagram from: Vulnerability of Energy Delivery Control Systems, INL, September ,258 entries (as of last week) assert(cve!= Bug_Database); Common Vulnerabilities and Exposures (CVE) 22

23 National Vulnerability Database (NVD) Prioritizing weaknesses to be mitigated CVE Entry CVSS Scores Affected Platforms Root-cause Weaknesses (CWE s) References to Advisories References to Mitigations References to Tools OVAL-based Checks NVD OWASP Top 10 CWE/SANS Top 25 U.S. government repository of standards-based vulnerability management data Lists are a good start but they are designed to be broadly applicable website: nvd.nist.gov We would like a way to specify priorities based on business/mission risk Dictionary of software weakness types Idaho National Labs Vulnerability CWE ID Name Description Alternate Names Applicable Platforms Applicable Languages Technical Impacts Potential Mitigations Observed Instances (CVE s) Related Attack Patterns (CAPEC s) Examples Plus much, much more 860+ entries in a tree-structure Common Weakness Enumeration (CWE) For the software we re responsible for W Notional W d Weaknesses we really care about How do we identify these? which weaknesses are most important? 23

24 Common Weakness Risk Framework (CWRAF) Technical Impacts 1. Modify data 2. Read data 3. DoS: unreliable execution 4. DoS: resource consumption 5. Execute unauthorized code or commands 6. Gain privileges / assume identity 7. Bypass protection mechanism 8. Hide activities Weightings W1=0 W2=0 W3=10 W4=4 W5=10 W6=0 W7=0 W8=0 Layers 1. System 2. Application 3. Network 4. Enterprise Technical Impact Scorecard Multiple pieces we ll focus on Vignettes CWRAF: Technical Impact Scorecard Common Weakness Risk Framework (CWRAF) How do I identifywhich of the 900+ CWE s are most important for my specific business domain, technologies and environment? Common Weakness Scoring System (CWSS) How do I rankthe CWE s I care about according to my specific business domain, technologies and environment? Application System Network Enterprise For each layer and each technical impact MD RD UE RC EA GP BP HA 8 3 assign a weighting from 0 to 10 How do I identify and score weaknesses important to my organization? Leveraging Vignettes in Cyber Security Standardization for Key ICT Applications in various Domains CWRAF: Technical Impact Scorecard MD RD UE RC EA GP BP HA Application System Network Enterprise These weightings can now be used to evaluate individual CWE s based on each CWE s Technical Impacts Common Weakness Risk Assessment Framework uses Vignettes with Archetypes to identify top CWEs in respective Domain/Technology Groups Note: Values for illustrative purposes only 24

25 Note: Values for illustrative purposes only MD RD UE RC EA GP BP HA Application System Network Enterprise CWE-78 Technical Impacts CWSS Formula 95 Notional CWSS Score for CWE-78 for this vignette Common Weakness Scoring System (CWSS) How do you score weaknesses using CWSS? 1. Establish weightings for the vignette 2. CWSS scoring engine processes each relevant CWE entry and automatically scores each CWE based on vignette definition 3. CWE dictionary presented in priority order based on vignette-driven CWSS scores 4. Organization now has their own customized Top N list of critical weaknesses for this vignette <CWE ID= 1 <CWE ID= 2 <CWE ID= Vignette Technical Impact Scorecard CWSS Scoring Engine Step 1 is only done once the rest is automatic 3 4 CWE-89: 99 CWE-238: 92 CWE-6: 83 CWE-45: 56 CWE-721: 44 CWE-482: 31 CWE-754: 0 CWE-73: 0 CWSS Score CWE 97 CWE CWE CWE CWE CWE CWE CWE CWE CWE CWE CWE CWE CWE CWE CWE-131 CWSS Scoring Engine User-defined cutoff W W d Most Important Weaknesses Vignette CWRAF/CWSS in a Nutshell How do you score weaknesses discovered in code using CWSS? Tool 3 2 Line 23: CWE-109 Line 72: CWE-84 Line 104: CWE-482 Line 212: CWE-9 Line 213: CWE-754 Source Code 1 Vignette Technical Impact Scorecard CWSS Scoring Engine 4 1. Establish weightings for the vignette 2. Run code through analysis tool(s) 3. Tools produce report of CWE s found in code 4. CWSS scoring engine automaticallyscores each CWE based on vignette definition Step 1 is only done once the rest is automatic Line 212: CWE-9: 99 Line 72: CWE-84: 79 Line 23: CWE-109: 56 Line 104: CWE-482: 31 Line 213: CWE-754: 0 Common Weakness Risk Framework (CWRAF) and Common Weakness Scoring System (CWSS) Organizations that have declared plans to work on CWRAF Vignettes and Technical Scorecards to help evolve CWRAF to meet their customer's and the community's needs for a scoring system for software errors. Common Weakness Risk Framework (CWRAF) and Common Weakness Scoring System (CWSS) Organizations that have declared plans to support CWSS in their future offerings and are working to help evolve CWSS to meet their customer's and the community's needs for a scoring system for software errors. 25

26 Tool A Tool B Tool C CWE Coverage Claims Representation (CCR) Set of CWE s tool claimsto cover Most Important Weaknesses (CWE s) Which static analysis tools find the CWE s I care about? What types of attacks should I test my system against? CWSS Score CWE 97 CWE CWE CWE CWE CWE CWE CWE CWE CWE CWE CWE CWE CWE CWE CWE-131 CWSS Scoring Engine CWE CWE-79 CWE-78 W W d Most Important Weaknesses Related CAPEC ID s CAPEC-232, CAPEC-106, CAPEC-19, CAPEC-108, CAPEC-15, CAPEC-43, CAPEC-6, Common Attack Pattern Enumeration and Classification CWRAF/CWSS Provides Risk Prioritization for CWE throughout Software Life Cycle Enables education and training to provide specific practices for eliminating software fault patterns; Enables developers to mitigate top risks attributable to exploitable software; Enables testing organizations to use suite of test tools & methods (with CWE Coverage Claims Representation) that cover applicable concerns; Enables users and operation organizations to deploy and use software that is more resilient and secure; Enables procurement organizations to specify software security expectations through acquisition of software, hosted applications and services. Your Problem: How can you know that software you install or build is secure? Developers are often not trained in security, and thus let dangerous code slip into software. Most system owners likely have less understanding of these issues. So, you need tools and techniques to help you find and eliminate weaknesses that might get built into your software with the developer workforce that you have. Common Attack Pattern Enumeration and Classification (CAPEC) Dictionary of attack types (mostly software) CAPEC ID Name Description Attack Prerequisites Indicators of Attack Examples Related Weaknesses (CWE s) Mitigations Plus much, much more 386 patterns, organized by categories, with views Proposed Tools Lists of common weaknesses and attacks. Tools built to cover these can help you know your staff is considering all known weaknesses. Training developers about these can prevent weaknesses from being built in. Tools to check for weaknesses. Running these on source code and/or compiled code can help you find weaknesses to remove. Then, you can verify that they were removed. Metrics to assign priority to issues found. These metrics can help you adjust your level of rigor to the risks and impact-level of the specific system. Decide how much risk (if any) to accept. 26

27 Using the Tools Train developers to avoid the worst problems. Find security weaknesses in the code Manual design/code review Automated static analysis tools (for source code) Eliminate those weaknesses Test software against attacks Automated dynamic application analysis tools (for executable code) Repeat as necessary Tools to check for weaknesses Common Weaknesses Methods Manual design/code review Automated static analysis of code Often Combined based on CWE Tools/Resources Developers trained to avoid/locate weaknesses and attacks. An appropriate automated tool. Source Code. An analyst to help interpret results. DHS Software Assurance (SwA) Program can help your organization locate training for developers and analysts, as well as potential tools. What software should you focus on? Sadly, attackers can attack even unimportant software on a machine or network to get to more important (but less vulnerable software). So, to some adequate extent, you must protect even your low-impact systems, to protect your higher impact systems/information. Lists of common weaknesses and attacks: Common Attack Pattern Enumeration and Classification (CAPEC) Source: Academia, private sector, public sector Content: Dictionary of over 400 attacks Some examples: address spoofing, brute-force encryption/password attacks, man-in-the-middle attacks, etc. Typical info: attack methods/examples, common mitigation strategies, links to related weaknesses Typical Use: DynamicTool purchasers can use the list to ensure their tools attack the full range of weaknesses. Typical Use: Pen Testers can get guidance on how to attack known weaknesses to verify that they are not present. Lists of common weaknesses and attacks: Common Weakness Enumeration (CWE) Source: Academia, private sector, public sector. Content: Dictionary of over 600 typesof common software weaknesses. Some examples: buffer overflow, command/sql injection, missing/weak authentication, etc. Typical info: code examples, common mitigation examples, links to related attacks Typical Use: COTS vendors require SW developers to avoid CWEs as part of their work. It s much cheaper not to have to remove issues later. Typical Use: There is commercial training for developers on how to avoid adding weaknesses to their code. Typical Use: Tool purchasers can use the list to ensure their tools look for the full range of weaknesses. Tools to check for weaknesses Attack Paths Methods Manual penetration testing Automated dynamic analysis of applications (Tools like??) Automated Web-testing (Tools like Web-Inspect ) Tools/Resources White-hat hackers trained to locate/exploit weaknesses. Often Combined based on CAPEC An appropriate automated tool. Executable Code. An analyst to help interpret results. An appropriate automated tool. Access to the website. An analyst to help interpret results. DHS Software Assurance (SwA) Program can help your organization locate training for penetration testers and analysts, as well as potential tools. 27

28 Common Weakness Scoring System (CWSS) Source: DHS -MITRE Content: Method to score (rank) weaknesses Similar to the National Vulnerability Databases CVSS. Typical Use: Rank weaknesses so you can decide which ones to address first. DHS Software Assurance (SwA) Program can help your organization understand how to use CWSS to rank weaknesses. automation can help Common Weakness Enumeration (CWE) Common Attack Pattern Enumeration and Classification (CAPEC) CWE Coverage Claims Representation (CCR) Common Weakness Enumeration (CWE) Common Weakness Risk Framework (CWRAF) Common Weakness Scoring System (CWSS) Common Attack Pattern Enumeration and Classification (CAPEC) CWE Coverage Claims Representation (CCR) Common Vulnerabilities and Exposures (CVE) Open Vulnerability Assessment Language (OVAL) Malware Attribute Enumeration and Characterization (MAEC) Cyber Obersvables expression (CybOX) When should I focus on Weaknesses and Vulnerabilities? SwA and Operational Resilience Concept Focus on Weaknesses A type of defect that may be exploitable. Design Source code Object code Binaries Keep Weaknesses from becoming vulnerabilities Deployed system Focus on Vulnerabilities Something in code that can actually be exploited. How do we prevent this next time? Resilience Requirements Compliance Measurement and OVAL SCAP Monitoring Incident and Control Enterprise Focus B P M N Vulnerability and Resolution Are we being attacked? C A P E C Who is attacking and what do they want? Controls Applied to Asset Are we at risk? Adapted from September 2010 SwA Forum, CERT RMM for Assurance, Lisa Young, SEI Courtesy of Michele Moss Putting it all Together What weaknesses are most important? CWE CWSS What types of attacks exploit those weaknesses? Does the system contain any of those weaknesses? Does my testing cover all of those weaknesses? CAPEC Free Resources and Events SwA Working Group Sessions: Nov MITRE in McLean, VA SwA Forum Next session: Sep, MITRE in McLean, VA SwA Websites: [email protected] Making Security Measureable: measurablesecurity.mitre.org See Language for sharing exchange of indicators and coorelation of incident information -- Cyber Observables expression (CybOX) at 28

29 Creating Attack-Aware Software Applications with Real-Time Defenses AppSensor provides the ability to have a much richer range of responses; the guidance lists 14 types of response. Specific examples for each type show how the types can be customized for each application s context. Application Defensive Measures that detect malicious activity Attack-Aware Detection I recognize that software has become a foundation of our modern world. AppSensor Project, The Rugged Software MANIFESTO Focus on Resilience and Survivability - If compromised, damage to the software will be minimized, and it will recover quickly to an acceptable level of operating capacity; it is rugged I recognize the awesome responsibility that comes with this foundational role. ruggedsoftware.org I am rugged -and more importantly, my code is rugged. I recognize that my code will be used in ways I cannot anticipate, in ways it was not designed, and for longer than it was ever intended. 29

30 I recognize that my code will be attacked by talented and persistent adversaries who threaten our physical, economic, and national security. I am rugged because I assure my code will support its mission. I recognize these things -and I choose to be rugged. I am rugged because my code can face these challenges and persist in spite of them. I am rugged because I refuse to be a source of vulnerability or weakness. I am rugged, not because it is easy, but because it is necessary and I am up for the challenge. 30

31 Typical Reactions to Software Assurance Awareness When an awareness is reached by on organization, one of several responses is usually taken: Ignore the problem (aka head in the sand) Undertake a paper exercise of policy and process that ultimately has no direct effect on the security of the software (aka lipstick on a pig) Assess and remediate the individual exploited application (aka band-aid) Seek to address the root problems by investigation and adoption of individual tactical application security practices such as penetration testing, static code analysis, security testing, etc (aka treating individual symptoms) Address the issue comprehensively though strategic thought and action (aka treating the disease) ruggedsoftware.org 184 Key Role of Application Security Risk in the Cybersecurity Game Ultimate goal is to prevent security vulnerabilities from ever entering software Multi-Perspective Application Security Risk : A Toolbox Approach Reality is they are already there and even new code from security-aware developers needs to be checked Application security risk analysis is the practice of: checking software for weaknesses/vulnerabilities characterizing the risk they pose identifying and prioritizing mitigations Adopted f rom Sean Barnum presentation at SSTC 2012 HS SEDI FFRDC is managed and operated by The MITRE Corporation for DHS. 185 Varying Perspectives of Security issues are becoming increasingly critical to organizations More and more enterprises are becoming aware of the importance of software assurance as an element of their broader security focus This awareness typically comes from one of three sources: The exploitation and breach of an individual fielded application An external mandate from senior management or an external governing entity that the issue must be addressed Internal epiphany or evolution of understanding static source code static binary code dynamic application scanning application penetration testing application data security fuzzing complexity composition & pedigree etc

32 Varying Capabilities of Perspectives Different perspectives are effective at finding different types of weaknesses Some are good at finding the cause and some at finding the effect Static Code Penetration Test Data Security Code Review Cross-Site Scripting (XSS) X X X SQL Injection X X X Architecture Risk Insufficient Authorization Controls X X X X The Gestalt of Multi-perspective Better situational awareness Reinforce confidence in findings of each perspective Combine the assurance of dynamic analysis with the detail of structure analysis to plan effective mitigation of high-criticality risk Broken Authentication and Session X X X X Information Leakage X X X Improper Error Handling X Insecure Use of Cryptography X X X Cross Site Request Forgery (CSRF) X X Denial of Service X X X X Coding Practices X Poor X Automating Perspectives Automation should be leveraged wherever possible but should be combined with focused manual analysis Automated tools will find the low-hanging fruit much faster than manual analysis can Manual analysis will find less obvious and occasionally high-risk issues The Challenges of Integrated Multiperspective Varying perspectives have different drivers and priorities based on context Differing perspectives treat location of issue differently making correlation a challenge Each tool for each perspective has its own reporting schema Need for a unified findings schema (SAFES) Current State of the Practice Most organizations undertaking application security risk analysis only perform one or maybe two analysis perspectives and those are done as independent processes often by separate teams If developer-centric organization, typically start with static analysis If test-centric, typically start with application scanning and penetration testing If information assurance or data-centric, typically start with data security scanning The Need for Standards in Effective Integration Always make sure comparing apples to apples Weakness Common Weakness Enumeration (CWE) Attack Common Attack Pattern Enumeration and Classification (CAPEC) Vulnerability Common Vulnerabilities and Exposures (CVE) Technical Context Common Platform Enumeration (CPE)

33 A Recommended Baseline for Multiperspective To effectively assess the security risk of an application, an assessment methodology should at a minimum include the following perspectives: Static source code analysis Application scanning & penetration testing Application data security analysis Application Data Security Analyzing the security concerns of how an application accesses and manages its database Strengths Analyzes a live, fully configured system rather than just source code Good at catching really bonehead mistakes (they are more common than you think) Helps mitigate both insider and external threats Limitations Only as good as what you tell it to look for Does not understand semantics of data (can use limited proxies) Multi-perspective integration value Confirmation of likely weaknesses as vulnerabilities Better contextual info about nature and severity of weaknesses Improved understanding of likelihood of weaknesses being exploitable Increases accuracy of forensic data Improved data flow policies Improved Access Control Static Source Code Value of Aligning Multiple Perspectives Analyze code without executing it Strengths Fast compared to manual code review Fast compared to testing Complete, consistent coverage of source code (all paths) Brings security knowledge with it Limitations Only analyzes the source code you feed it Doesn t find everything Architecture errors Bugs you re not looking for System administration mistakes User mistakes False positives Multi-perspective integration value Actual location of the weakness in code Identify issues to target with penetration testing Identify co-influencing weaknesses within relevant contexts Null Pointer Dereference Threading Issues Issues in Dead Code Insecure Crypto Functions Application Logic Issues Static Total Potential Security Issues (CWEs) Dynamic Reduce false positives Map Exploited Issues to Code Environment Configuration Issues Issues in integrations of modules Runtime Privileges Issues Protocol Parser/SerializerIssues Issues in 3rd party components SQL Injection Cross Site Scripting HTTP Response Splitting OS Commanding LDAP Injection 194 Application Scanning & Penetration Testing Practical Example: USAF ASACoE Security testing (black box) of applications through simulated attacks Strengths Simulates the actual risk (attacker s action) Tests full software stack Low false positives Mature technology Limitations Only as good as what you scan (crawling limitations) limited to the test cases executed Must run tests often to stay protected Can only be performed once code is runable Risky to run on production applications Cannot identify the actual source of the problem, only the symptom Multi-perspective integration value Confirming that weaknesses are vulnerable Mapping penetration scans to locations in source code Mapping data security findings to injection findings, privilege issues, etc. Application Software Assurance Center of Excellence (ASACoE) The Focal Point for Air Force Software Assurance (SwA) capability with the goal of reducing softwareinduced risk from Air Force applications

34 Overview of Triage Assessment Process Establish buildable source code and executable test or operational environment Run static source code analysis scan Run web application scan Run application data security scan Prioritize results analysis Eliminate obvious false positives Correlate results of different tools to confirm vulnerabilities or eliminate false positives Conduct remaining analysis Characterize and classify findings Create integrated findings report Adorn integrated report with mitigation advice for findings 199 automation can help Common Weakness Enumeration (CWE) Common Attack Pattern Enumeration and Classification (CAPEC) CWE Coverage Claims Representation (CCR) Common Weakness Enumeration (CWE) Common Weakness Risk Framework (CWRAF) Common Weakness Scoring System (CWSS) Common Attack Pattern Enumeration and Classification (CAPEC) CWE Coverage Claims Representation (CCR) Common Vulnerabilities and Exposures (CVE) Open Vulnerability Assessment Language (OVAL) Malware Attribute Enumeration and Characterization (MAEC) Cyber Obersvables expression (CybOX) ASACoE Rationale for Multi-perspective Approach Air Force is looking to maximize its understanding of security risk in all areas of its applications (interfaces, business logic, data tier, etc.) ASACoE recognizes the difficulty and complexity of analyzing application security tool scan results ASACoE wants to provide as much context and guidance as possible to developers for mitigation and remediation 200 IT/Software Supply Chain is a National Security & Economic Issue Adversaries can gain intimate access to target systems, especially in a global supply chain that offers limited transparency Advances in science and technology will always outpace the ability of government and industry to react with new policies and standards National security policies must conform with international laws and agreements while preserving a nation s rights and freedoms, and protecting a nation s self interests and economic goals; Forward-looking policies can adapt to the new world of global supply chains; Standards for automation, processes, and products must mature to better address supply chain risk management, systems/software assurance, and the exchange of information and indicators for cyber security; Assurance Rating Schemes for software products and organizations are needed. IT/software suppliers and buyers can take more deliberate actions to security-enhance their processes, practices and products to mitigate risks Government & Industry have significant leadership roles in solving this Individuals can influence the way their organizations adopt security practices Globalization will not be reversed; this is how we conduct business To remain relevant, standards and capability benchmarking measures must address assurance mechanisms needed to manage IT/Software Supply Chain risks. Summary and Conclusions Software Assurance analysis is increasingly becoming a high priority and is maturing in its capability Varying perspectives of analysis are available, each with their own unique value Blending multiple perspectives together yields better overall coverage and an integrated gestalt It is real and possible to begin pursuing this approach today Commerce National Defense Public/Private Collaboration Efforts for Security Automation and Software Supply Chain Risk 201 Next SwA Forum meets Sep 2012 at MITRE, McLean, VA 34

35 Next SwA Forum at MITRE, McLean, VA Sep 2012 Joe Jarzombek, PMP, CSSLP Director for Software Assurance National Cyber Security Division Department of Homeland Security (703) LinkedIn SwA Mega-Community Working for Homeland Security The DHS Office of Cybersecurity and Communications (CS&C) serves as the national focal point for securing cyber space and the nation s cyber assets. CS&C is actively seeking top notch talent in several areas including: Software assurance Information technology Telecommunications Program management Public affairs To learn more about CS&C and potential career opportunities, please visit USAJOBS at 35