Software & Supply Chain Assurance: A Historical Perspective of Community Collaboration
|
|
|
- Naomi Berry
- 9 years ago
- Views:
Transcription
1 Software & Supply Chain Assurance: A Historical Perspective of Community Collaboration Joe Jarzombek, PMP, CSSLP Director for Software & Supply Chain Assurance Stakeholder Engagement & Cyber Infrastructure Resilience Cyber Security & Communications Enabling Enterprise Resilience through Security Automation, Software Assurance, and Supply Chain Risk Management
2 FOR WORKSHOPS AND OVERVIEW DISCUSSIONS DOD SOFTWARE ASSURANCE INITIATIVE: Mitigating Risks Attributable to Software Countering Threats that Target Software in Systems and Networks Workshop Out-Briefs Sep1, 2004 The first pubic meeting FOR WORKSHOPS AND OVERVIEW DISCUSSIONS
3 FOR WORKSHOPS AND OVERVIEW DISCUSSIONS Action Required to Address Material Weakness in Software FOR WORKSHOPS AND OVERVIEW DISCUSSIONS SOFTWARE ASSURANCE INITIATIVE PROVIDES: Proactive transformation of processes to mitigate risks attributable to software Sustainable capability to counter threats to software-enabled DoD systems and networks Dramatic Increase in Mission Risk
4 FOR WORKSHOPS AND OVERVIEW DISCUSSIONS Software Assurance Forum FOR WORKSHOPS AND OVERVIEW DISCUSSIONS SW Assurance is managed as part of: the DoD Information Assurance (IA) Strategy and the DHS National Cyber Security Strategy WG1 - Security Process Capability (improvement & evaluation), WG2 - Software Product Evaluation (product focused), WG3 - Counter Intelligence (CI) Threat Assessment Support WG4 - Acquisition/Procurement and Industrial Security, and WG5 - User Identification & Prioritization of Protected Assets WG6 - Workforce Education and Training
5 PITAC* Findings Relative to Needs for Secure Software Engineering & Software Assurance Commercial software engineering today lacks the scientific underpinnings and rigorous controls needed to produce high-quality, secure products at acceptable cost. Commonly used software engineering practices permit dangerous errors, such as improper handling of buffer overflows, which enable hundreds of attack programs to compromise millions of computers every year. In the future, the Nation may face even more challenging problems as adversaries both foreign and domestic become increasingly sophisticated in their ability to insert malicious code into critical software. Recommendations for increasing investment in cyber security provided to NITRD Interagency Working Group for Cyber Security & Information Assurance R&D * President s Information Technology Advisory Committee (PITAC) Report to the President, Cyber Security: A Crisis of Prioritization, February 2005 identified top 10 areas in need of increased support, including: secure software engineering and software assurance and metrics, benchmarks, and best practices [Note: PITAC is now a part of PCAST]
6 Software Assurance Addresses Exploitable Software: Outcomes of non-secure practices and/or malicious intent Exploitation potential of vulnerability is independent of intent Defects EXPLOITABLE SOFTWARE Unintentional Vulnerabilities Malware Intentional Vulnerabilities High quality can reduce security flaws attributable to defects; yet traditional S/W quality assurance does not address intentional malicious behavior in software *Intentional vulnerabilities: spyware & malicious logic deliberately imbedded (might not be considered defects) Software Assurance (SwA) is the level of confidence that software functions as intended and is free of vulnerabilities, either intentionally or unintentionally designed or inserted as part of the software throughout the life cycle.* From CNSS Instruction 4009 National Information Assurance Glossary (26APR2010)
7 DHS Software Assurance Program Overview Program established in response to the National Strategy to Secure Cyberspace - Action/Recommendation 2-14: DHS will facilitate a national public-private effort to promulgate best practices and methodologies that promote integrity, security, and reliability in software code development, including processes and procedures that diminish the possibilities of erroneous code, malicious code, or trap doors that could be introduced during development. DHS Program goals promote the security and resilience of software across the development, acquisition, and operational life cycle DHS Software Assurance (SwA) program is scoped to address: Trustworthiness - No exploitable vulnerabilities or malicious logic exist in the software, either intentionally or unintentionally inserted, Dependability (Correct and Predictable Execution) - Justifiable confidence that software, when executed, functions as intended, Survivability - If compromised, damage to the software will be minimized, and it will recover quickly to an acceptable level of operating capacity; Conformance Planned, systematic set of multi-disciplinary activities that ensure processes/products conform to requirements, standards/procedures. See Wikipedia.org for Software Assurance - CNSS Instruction No. 4009, "National Information Assurance Glossary," Revised 2006, defines Software Assurance as: "the level of confidence that software is free from vulnerabilities, either intentionally designed into the software or accidentally inserted at anytime during its lifecycle, and that the software functions in the intended manner". 7
8 Disciplines Contributing to Software Assurance* Information Assurance Project Mgt Systems Engineering Software Acquisition Software Assurance Software Engineering *Test & Evaluation Safety & Security In Education and Training, Software Assurance could be addressed as: A knowledge area extension within each of the contributing disciplines; A stand-alone CBK drawing upon contributing disciplines; A set of functional roles, drawing upon a common body of knowledge; allowing more in-depth coverage dependent upon the specific roles. Intent is to provide framework for curriculum development and evolution of contributing BOKs * See Notes Page view for contributing BOK URLs and relevant links *Info Systems Security Eng The intent is not to create a new profession of Software Assurance; rather, to provide a common body of knowledge: (1) from which to provide input for developing curriculum in related fields of study and (2) for evolving the contributing 8 disciplines to better address the needs of software security, safety, dependability, reliability and integrity.
9 Assurance relative to Trust Managing Effects of Unintentional Defects in Component or System Integrity Managing Consequences of Unintentional Defects Quality Safety TRUST Security Managing Consequences of Attempted/Intentional Actions Targeting Exploitable Constructs, Processes & Behaviors
10 * Acquisition Program Supplier Supply chain introduces risks to American society that relies on Federal Government for essential information and services. 30 Sep 2005 changes to Federal Acquisition Regulation (FAR) focus on IT Security Focuses on the role of contractors in security as Federal agencies outsource various IT functions. Scope of Supplier Expansion and Foreign Involvement graphic in DACS Secure Software Engineering, July 2005 article Software Development Security: A Risk Management Perspective synopsis of May 2004 GAO report Defense Acquisition: Knowledge of Software Suppliers Needed to Manage Risks 10
11 Risk Management (Enterprise <=> Project): Shared Processes & Practices // Different Focuses Enterprise-Level: Regulatory compliance Changing threat environment Business Case Program/Project-Level: Cost Schedule Performance Software Supply Chain Risk Management traverses enterprise and program/project interests 11
12 Software Assurance End State Objectives Government, in collaboration with industry / academia, raised expectations for product assurance with requisite levels of integrity and security: Helped advance more comprehensive software assurance diagnostic capabilities to mitigate risks stemming from exploitable vulnerabilities and weaknesses; Collaboratively advanced use of software security measurement & benchmarking schemes Promoted use of methodologies and tools that enabled security to be part of normal business. Acquisition managers & users factored risks posed by the software supply chain as part of the trade-space in risk mitigation efforts: Information on suppliers process capabilities (business practices) would be used to determine security risks posed by the suppliers products and services to the acquisition project and to the operations enabled by the software. Information about evaluated products would be available, along with responsive provisions for discovering exploitable vulnerabilities, and products would be securely configured in use. Suppliers delivered quality products with requisite integrity and made assurance claims about the IT/software safety, security and dependability: Relevant standards would be used from which to base business practices & make claims; Qualified tools used in software lifecycle enabled developers/testers to mitigate security risks; Standards and qualified tools would be used to certify software by independent third parties; IT/software workforce had requisite knowledge/skills for developing secure, quality products. Enabling Software Supply Chain Transparency 12
13 DHS NCSD Software Assurance (SwA) Program Through public-private collaboration promotes security and resilience of software throughout the lifecycle; focused on reducing exploitable software weaknesses and addressing means to improve capabilities that routinely develop, acquire, and deploy resilient software products. Collaboratively advancing software-relevant rating schemes Serves as a focal point for interagency public-private collaboration to enhance development and acquisition processes and capability benchmarking to address software security needs. Hosts interagency Software Assurance Forums, Working Groups and training to provide public-private collaboration in advancing software security and providing publicly available resources. Provides collaboratively developed, peer-reviewed information resources on Software Assurance, via journals, guides & on-line resources suitable for use in education, training, and process improvement. Provides input and criteria for leveraging international standards and maturity models used for process improvement and capability benchmarking of software suppliers and acquisition organizations. Enables software security automation and measurement capabilities through use of common indexing and reporting capabilities for malware, exploitable software weaknesses, and common attacks which target software. Collaborates with the National Institute of Standards and Technology, international standards organizations, and tool vendors to create standards, metrics and certification mechanisms from which tools can be qualified for software security verification. Manages programs to facilitate the adoption of Malware Attribute Enumeration Classification (MAEC), Common Weakness Enumeration (CWE), and Common Attack Pattern Enumeration and Classification (CAPEC). Homeland Security Cybersecurity and Communications 13
14 Challenges in Mitigating Risks Attributable to Exploitable ICT/Software & Supply Chains Several needs arise: Need internationally recognized standards to support security automation and processes to provide transparency for informed decision-making in mitigating enterprise risks. Need comprehensive diagnostic capabilities to provide sufficient evidence that code behavior can be understood to not possess exploitable or malicious constructs. Need Assurance to be explicitly addressed in standards & capability benchmarking models for organizations involved with security/safety-critical applications. Need rating schemes for ICT/software products and supplier capabilities.
15 Mitigating Risks Attributable to Exploitable Software and Supply Chains Enterprises seek comprehensive capabilities to: Avoid accepting software with MALWARE pre-installed. Determine that no publicly reported VULNERABILITIES remain in code prior to operational acceptance, and that future discoveries of common vulnerabilities and exposures can be quickly patched. Determine that exploitable software WEAKNESSES that put the users most at risk are mitigated prior to operational acceptance or after put into use. MAEC CVE CWE
16 DHS Software Assurance Program Structure * As part of the DHS risk mitigation effort, the SwA Program seeks to reduce software vulnerabilities, minimize exploitation, and address ways to improve the routine development of trustworthy software products and tools to analyze systems for hidden vulnerabilities. The SwA framework encourages the production, evaluation and acquisition of better quality and more secure software; leverages resources to target the following four areas: People education and training for developers and users Processes sound practices, standards, and practical guidelines for the development of secure software Technology diagnostic tools, cyber security R&D and measurement Acquisition due-diligence questionnaires, contract templates and guidelines for acquisition management and outsourcing * July 28, 2006 statement of George Foresman, DHS UnderSecretary for Preparedness, before the U.S. Senate Committee on Homeland Security and Governmental Affairs, Subcommittee on Federal Financial Management, Government Information, and International Security 16
17 Software Assurance Forum & Working Groups* encourage the production, evaluation and acquisition of better quality and more secure software through targeting People Developers and users education & training Processes Sound practices, standards, & practical guidelines for secure software development Build Security In - and SwA community resources & info clearinghouse SwA Common Body of Knowledge (CBK) & Glossary Organization of SwSys Security Principles/Guidelines SwA Developers' Guide on Security-Enhancing SDLC Software Security Assurance State of the Art Report Systems Assurance Guide (via DoD and NDIA) SwA-related standards ISO/IEC JTC1 SC7/27/22, IEEE CS, OMG, TOG, & CMM-based Assurance Technology Security test criteria, diagnostic tools, common enumerations, SwA R&D, and SwA measurement Products and Contributions Acquisition Software security improvements through due-diligence questions, specs and guidelines for acquisitions/ outsourcing Practical Measurement Framework for SwA/InfoSec Making the Business Case for Software Assurance SwA Metrics & Tool Evaluation (with NIST) SwA Ecosystem w/ DoD, NSA, NIST, OMG & TOG NIST Special Pub 500 Series on SwA Tools Common Weakness Enumeration (CWE) dictionary Common Attack Pattern Enumeration (CAPEC) SwA in Acquisition: Mitigating Risks to Enterprise Software Project Management for SwA SOAR * SwA Forum is part of Cross-Sector Cyber Security Working Group (CSCSWG) established under auspices of the Critical Infrastructure Partnership Advisory Council (CIPAC) that 17 provides legal framework for participation.
18 SwA Collaboration for Content & Peer Review BSI focuses on making Software Security a normal part of Software Engineering SwA Community Resources and Information Clearinghouse (CRIC) focuses on all contributing disciplines, practices and methodologies that advance risk mitigation efforts to enable greater resilience of software/cyber assets. The SwA CRIC provides a primary resource for SwA Working Groups. Where applicable, SwA CRIC & BSI provide relevant links to each other.
19 Process Agnostic Lifecycle Since 3 Oct 2005 Architecture & Design Architectural risk analysis Threat modeling Principles Guidelines Historical risks Modeling tools Resources Code Code analysis Assembly, integration & evolution Coding practices Coding rules Code analysis Resources Test Security testing White box testing Attack patterns Historical risks Resources Requirements Requirements engineering Attack patterns Resources Touch Points & Artifacts Fundamentals Risk management Project management Training & awareness Measurement SDLC process Business relevance Resources System Penetration testing Incident management Deployment & operations Black box testing Resources Key Best (sound) practices Foundational knowledge Tools Resources 19
20 See for information
21 Security-Enhanced Capabilities: Mitigating Risks to the Enterprise With today s global software supply chain, Software Engineering, Quality Assurance, Testing and Project Management must explicitly address security risks posed by exploitable software. Traditional processes do not explicitly address software-related security risks that can be passed from projects to using organizations. Mitigating Supply Chain Risks requires an understanding and management of Suppliers Capabilities, Products and Services Enterprise risks stemming from supply chain are influenced by suppliers and acquisition projects (including procurement, SwEng, QA, & testing). IT/Software Assurance processes/practices span development/acquisition. Derived (non-explicit) security requirements should be elicited/considered. More comprehensive diagnostic capabilities and standards are needed to support processes and provide transparency for more informed decision-making for mitigating risks to the enterprise Free resources are available to assist personnel in security-enhancing contracting, outsourcing and development activities (see 21
22 Need for Rating Schemes Rating of Software products: Supported by automation Standards-based Rules for aggregation and scaling Verifiable by independent third parties Labeling to support various needs (eg., security, dependability, etc) Meaningful and economical for consumers and suppliers Rating of Suppliers providing software products and services Standards-based or model-based frameworks to support process improvement and enable benchmarking of organizational capabilities Credential programs for professionals involved in software lifecycle activities and decisions 22
23 Content for Curricula Development Software Assurance: A Curriculum Guide to the Common Body of Knowledge to Produce, Acquire, and Sustain Secure Software, updated Oct 2007 Toward an Organization for Software System Security Principles and Guidelines, Version 1.0, IIIA Technical Paper Feb 2008 Both collaboratively developed through the Software Assurance Working Group on Workforce Education and Training
24 0. INTRODUCTION 0.1/0.2 PURPOSE / SCOPE 0.3 REASONING UNDERLYING THE ORGANIZATION 0.4 ORGANIZATION OF REMAINDER OF DOCUMENT Toward an Organization for Software System Security Principles and Guidelines 1. THE ADVERSE 1.1. LIMIT, REDUCE, OR MANAGE VIOLATORS 1.2. LIMIT, REDUCE, OR MANAGE BENEFITS TO VIOLATORS OR ATTACKERS 1.3. INCREASE ATTACKER LOSSES 1.4. INCREASE ATTACKER UNCERTAINTY 2. THE SYSTEM 2.1. LIMIT, REDUCE, OR MANAGE VIOLATIONS 2.2. IMPROVE BENEFITS OR AVOID ADVERSE EFFECTS ON SYSTEM BENEFITS 2.3. LIMIT, REDUCE, OR MANAGE SECURITY-RELATED COSTS 2.4. LIMIT, REDUCE, OR MANAGE SECURITY-RELATED UNCERTAINTIES 3. THE ENVIRONMENT 3.1. NATURE OF ENVIRONMENT 3.2. BENEFITS TO AND FROM ENVIRONMENT 3.3. LIMIT, REDUCE, OR MANAGE ENVIRONMENT-RELATED LOSSES 3.4. LIMIT, REDUCE, OR MANAGE ENVIRONMENT-RELATED UNCERTAINTIES 4. CONCLUSION 5. APPENDIX A: PRINCIPLES OF WAR 6. APPENDIX B: PURPOSE-CONDITION-ACTION-RESULT MATRIX 7/8. BIBLIOGRAPHY / ACKNOWLEDGEMENTS
25 25
26 Software Security Engineering: A Guide for Project Managers Organized for Project Managers Derives material from DHS SwA Build Security In web site Provides a process focus for projects delivering softwareintensive products and systems Published in May
27 State of the Art Report July 2007 FREE publicly available resource provides a comprehensive look at efforts to improve the state of Software Security Assurance: describes the threats and common vulnerabilities to which software is subject; presents the many ways in which the S/W Security Assurance problem is being framed and understood across government, industry, and academia; describes numerous methodologies, best practices, technologies, and tools currently being used to specify, design, and implement software that will be less vulnerable to attack, and to verify its attackresistance, attack-tolerance, and attack-resilience; offers a large number of available resources from which to learn more about principles and practices that constitute Software Security Assurance; provides observations about potentials for success, remaining shortcomings, and emerging trends across the S/W Security Assurance landscape. Free via The SOAR reflects output of efforts in the DoD-DHS Software Assurance Forum and Working Groups that provide collaborative venues for stakeholders to share and advance techniques and technologies relevant to software security.
28 Reference Resource on Software Assurance Describes how to integrate security principles and practices in software development life cycle Addresses security requirements, secure design principles, secure coding, risk-based software security testing, and secure sustainment Provides guidance for selecting secure development methodologies, practices, and technologies Collaboratively developed/updated via SwA Forum working groups Released Oct 2008 by DACS Free, available for download via DACS & DHS SwA Community Resources & Information Clearinghouse
29 Fundamental Practices for Secure Software Development: A Guide to the Most Effective Secure Development Practices in Use Today, Oct 8, 2008 Common security-related elements of software development methodologies Security requirements help drive design, code handling, programming, and testing activities Secure Programming practices: Minimize unsafe function use Use the latest compiler toolset Use static and dynamic analysis tools Use manual code review on high-risk code Validate input and output Use anti-cross site scripting libraries Use canonical data formats Avoid string concatenation for dynamic SQL Eliminate weak cryptography Use logging and tracing Test to validate robustness and security Fuzz testing Penetration testing & third party assessment Automated test tools (in all development stages) Code Integrity and Handling Least privilege access, Separation of duties, Persistent protection, Compliance management; Chain of custody & supply chain integrity. Documentation (about software security posture & secure configurations)
30 SwA Acquisition & Outsourcing Handbook Software Assurance in Acquisition: Mitigating Risks to the Enterprise Version 1.0, Oct 2008, available for community use published by National Defense University Press, Feb 2009
31 Executive Summary 1. Introduction 1.1 Background 1.2 Purpose and Scope 1.3 Audience Acquisition Official Defined 1.4 Document Structure 1.5 Risk-Managed Software Acquisition Process 2. Planning Phase 2.1 Needs Determination, Risk Categorization, & Solution Alternatives 2.2 SwA Requirements 2.3 Acquisition Plan and/or Acquisition Strategy 2.4 Evaluation Plan and Criteria 2.5 SwA Due Diligence Questionnaires 3. Contracting Phase 3.1 Request for Proposals Work Statement Terms and Conditions Instructions to Suppliers Certifications Prequalification 3.2 Proposal Evaluation 3.3 Contract Negotiation 3.4 Contract Award 4. Implementation and Acceptance Phase 4.1 Contract Work Schedule 4.2 Change Control 4.3 Risk Management Plan 4.4 Assurance Case Management 4.5 Independent Software Testing 4.6 Software Acceptance SwA Acquisition & Outsourcing Handbook 5. Follow-on Phase 5.1 Support and Maintenance Risk Management Assurance Case Management Transition to Ops Other Change Management Considerations 5.2 Disposal or Decomissioning Appendix A/B Acronyms/Glossary Appendix C An Imperative for SwA in Acquisition Appendix D Software Due Diligence Questionnaires Table D-1. COTS Proprietary Software Questionnaire Table D-2. COTS Open-Source Software Questionnaire Table D-3. Custom Software Questionnaire Table D-4. GOTS Software Questionnaire Table D-5. Software Services Appendix E Other Examples of Due Diligence Questionnaires Appendix F Sample Language for the RFP and/or Contract F.1 Security Controls and Standards F.2 Securely Configuring Commercial Software F.3 Acceptance Criteria F.4 Certifications F.5 Sample Instructions to Offerors Sections F.6 Sample Work Statement Sections F.7 Open Web Application Security Project F.8 Certification of Originality Appendix H References
32 Software Assurance (SwA) Pocket Guide Series SwA in Acquisition & Outsourcing Software Assurance in Acquisition and Contract Language Software Supply Chain Risk Management and Due-Diligence SwA in Development * Risk-based Software Security Testing Requirements and Analysis for Secure Software Architecture and Design Considerations for Secure Software Secure Coding and Software Construction Key Practices for Mitigating the Most Egregious Exploitable Software Weaknesses * All include questions to ask developers SwA Life Cycle Support SwA in Education, Training and Certification SwA Pocket Guides and SwA-related documents are collaboratively developed with peer review; they are subject to update and are freely available for download via the DHS Software Assurance Community Resources and Information Clearinghouse at (see SwA Resources)
33 Security-Enhanced Process Improvements Organizations that provide security engineering & risk-based analysis throughout the lifecycle will have more resilient software products / systems. Build Security In throughout the lifecycle Attack Modeling Secure S/W Requirements Engineering Secure Design Principles & Practices Secure Programming Practices Test / Validation of Security & Resilience Secure Distribution/ Deployment Documentation for Secure Use & Configuration Abuse Cases Security Requirements Risk Analysis Design Review Risk-based Test Plans Code Review Static/Dynamic Analysis Risk Analysis Penetration Testing Security Ops & Vulnerability Mgt Plan Risk Assessment Requirements and Use Cases Security Design Reviews Architecture and Detailed Design Design Build Application Security Testing Code and Testing S/W Support Scanning & Remediation Field Deployment and Feedback Deploy Organizational Process Assets cover: governance, policies, standards, training, tailoring guidelines Leverage Software Assurance resources (freely available) to incorporate in training & awareness Modify SDLC to incorporate security processes and tools (should be done in phases by practitioners to determine best integration points) Avoid drastic changes to existing development environment and allow for time to change culture and processes Make the business case and balance the benefits Retain upper management sponsorship and commitment to producing secure software. * Adopted in part from Software Assurance: Mitigating Supply Chain Risks (DHS NCSD SwA); What to Test from a Security Perspective for the QA Professional (Cigital) and Neutralizing the Threat: A Case Study in Enterprisewide Application Security Deployments (Fortify Software & Accenture Security Technology Consulting) 33
34 We are engaged with many parts of the Community for Software Assurance-related standardization
35 ISO/IEC/IEEE 15026, System and Software Assurance ISO/IEC24748: Guide to Life Cycle Management Other standards providing details of selected SW processes Source: J. Moore, SC7 Liaison Report, IEEE Software and Systems Engineering Standards Committee, Executive Committee Winter Plenary Meeting, February ISO/IEC12207: Life cycle processes for Software ISO/IEC 15289: Document - ation Interoperation ISO/IEC 16326: Project Mgmt ISO/IEC 15939: Measure - ment ISO/IEC15288: Life cycle processes for systems Other standards providing details of selected system processes + ISO/IEC15026: Additional practices for higher assurance systems ISO/IEC 16085: Risk Mgmt Common vocabulary, process architecture, and process description conventions System and software assurance focuses on the management of risk and assurance of safety, security, and dependability within the context of system and software life cycle Terms of Reference changed: ISO/IEC JTC1/SC7 WG7, previously System and Software Integrity SC7 WG9 35
36 ISO/IEC/IEEE Assurance Case Set of structured assurance claims, supported by evidence and reasoning (arguments), that demonstrates how assurance needs have been satisfied. Shows compliance with assurance objectives Provides an argument for the safety and security of the product or service. Built, collected, and maintained throughout the life cycle Derived from multiple sources System, Software, or Work Product Sub-parts A high level summary Justification that product or service is acceptably safe, secure, or dependable Rationale for claiming a specified level of safety and security Conformance with relevant standards & regulatory requirements The configuration baseline Identified hazards and threats and residual risk of each hazard / threat Operational & support assumptions Make the case for adequate quality/ assurance of the justify belief in Quality / Assurance Factor Claims Arguments Evidence is developed for Quality / Assurance Case supports Quality / Assurance Subfactor Attributes Clear Consistent Complete Comprehensible Defensible Bounded Addresses all life cycle stages 36
37 Life-Cycle Standards View Categories (ISO/IEC and 12207) Organization Governance Processes Strategy and policy Enterprise risk management Compliance Business case Supply Chain Management Project-Enabling Processes Life Cycle Model Management Infrastructure Management SwA ecosystem Enumerations, languages, and repositories Project Portfolio Management Human Resource Management SwA education SwA certification and training Recruitment Quality Management Agreement Processes Acquisition Outsourcing Agreements Risk-based due diligence Supplier assessment Project Project Management Processes Project Planning Project Assessment and Control Assurance case management Project Support Processes Decision Management Risk Management Threat Assessment Configuration Management Information Management Measurement Engineering Technical Processes Stakeholder Requirements Definition Requirements Analysis Attack modeling (misuse and abuse cases) Data and information classification Risk-based derived requirements Sw security requirements Architectural Design Secure Sw architectural design Risk-based architectural analysis Secure Sw detailed design and analysis Implementation Secure coding and Sw construction Security code review and static analysis Formal methods Integration Sw component integration Risk analysis of Sw reuse components Verification & Validation Risk-based test planning Security-enhanced test and evaluation Dynamic and static code analysis Penetration testing Independent test and certification Transition Secure distribution and delivery Secure software environment (secure configuration, application monitoring, code signing, etc) Operations and Sustainment Software Reuse Processes Domain Engineering Reuse Asset Management Reuse Program Management Software Support Processes Sw Documentation Management Sw Quality Assurance Sw Configuration Management Sw Verification & Sw Validation Sw Review Sw Audit Sw Problem Resolution Supply Operation Incident handling and response Maintenance Defect tracking and remediation Vulnerability and patch management Version control and management Disposal 37
38 Many DHS sponsored efforts are key to changing how software-based systems are developed, deployed and operated securely. OCIL ARF CCI CAG ITU-T CCv4 CIEL
39 The Landscape of Cyber Security Standardization Efforts Standard Processes Standard Formats & Concepts Common Collections/Reference Resources IT Cyber Security IT Cyber Security IT Cyber Security Pre- Deployment Phase 24748: Guide to Life Cycle Management 12207: Life cycle processes for SW 16326: Project Mgmt 15939: Measurement 16085: Risk Management 15288: Life cycle processes for systems 15026: Additional practices for higher assurance systems Common Criteria ISO/IEC SC22 collection of language standards OMG KDM - Knowledge Discovery Metamodel OMG SBVR - Symantec Business Vocabulary and Rules PL vulnerabilities OMG SAEM SW Assurance Evidence Metamodel OMG ARG Argumentation Metamodel X.CWE X.CAPEC SWEBOK CWE CAPEC SWEBOK Security KA ISSA CCLSP Assurance-related questions SE2004 curriculum Curriculum proposals ABET accreditation CSDP Assurancerelated questions Post- Deployment Operations Phase ITIL SP and 53a SP SP X.CVE X.CVSS X.OVAL DNS GRC Roundtable FDCC SCAP NVD CVE CVSS X.XCCDF X.CCE X.CPE OVAL XCCDF CCE X.CWE X.CAPEC X.CEE CPE CWE CAPEC X.MAEC X.CYBIEF CEE MAEC
40 THE GOAL Qualified system and SW engineers applying sound processes using appropriate assurance tools delivered and deployed securely all based on a commonly understood nomenclature aware of emerging assurance issues adapted for assurance considerations to produce demonstrably sound software and operated securely about currently known threats, problems and solutions. Measuring Cyber Security SOAR SWA SOAR SWA CBOK Guide to life cycle management SW and systems assurance Prog Language vulnerabilities Common Criteria OMB FDCC/SCAP SWEBOK Security KA System LC processes SW LC processes Programming language standards of SC22 and others Supply chain studies SP and 53a SE2004 curriculum Curriculum proposals Documentation Measurement Process considerations Assurance case NIST Checklists Secure Configuration Guides ABET accreditation IEEE CSDP Assurancerelated questions Risk management Management OMG Models for the assurance case X.CWE, X.CAPEC X.CEE, X.MAEC X.CVE, X.CVSS, X.CPE, X.CCE, X.OVAL, X.XCCDF X.CYBIEF ISSA CCLSP Assurancerelated questions NIST , NIST NVD, CVE, OVAL, XCCDF, CVSS, CPE, CCE, CWE, CAPEC, CEE, MAEC
41 Many SwA Resources Focus On Development Assurance for CMMI
42 Process Improvement Lifecycle - A Process for Achieving Assurance Mission/Business Process Understand Your Business Requirements for Assurance Measure Your Results Information System Build or Refine and Execute Your Assurance Processes Understand Assurance-Related Process Capability Expectations Organization Support Look to Standards for Assurance Process Detail Adapted from: Paul Croll, Computer Sciences Corporation, August 2007
43 The Assurance PRM Is A Holistic Framework Define Business Goals Development Organization DO 1 Establish the assurance resources to achieve key business objectives DO 2 Establish the environment to sustain the assurance program within the organization Development Project DP 1 Identify and manage risks due to vulnerabilities throughout the product and system lifecycle DP 2 Establish and maintain assurance support from the project DP 3 Protect project and organizational assets Prioritize funds and manage risks Enterprise Assurance Support ES 1 Establish and maintain organizational culture where assurance is an integral part of achieving the mission ES 2 Establish and maintain the ability to support continued delivery of assurance capabilities ES 3 Monitor and improve enterprise support to IT assets Acquisition and Supplier Management AM 1 Select, manage, and use effective suppliers and third party applications based upon their assurance capabilities. Development Engineering DE 1 Establish assurance requirements DE 2 Create IT solutions with integrated business objectives and assurance DE 3 Verify and Validate an implementation for assurance Enable Resilient Technology Sustained environment to achieve business goals through technology Created to facilitate Communication Across An Organization s Multi-Disciplinary Stakeholders Courtesy of Michele Moss, BAH, SwA Processes & Practices
44 The DHS SwA Processes and Practices Working Group has synthesized the contributions of leading government and industry experts into a set of high-level goals and supporting practices (an evolution of the SwA community s Assurance Process Reference Model) The goals and practices are mapped to specific industry resources providing additional detail and real world implementation and supporting practices Assurance Focus for CMMI Building Security In Maturity Model Open Software Assurance Maturity Model CERT Resilience Management Model CMMI for Acquisition CMMI for Development CMMI for Services SwA Community s Assurance Process Reference Model Initial Mappings SwA Community s Assurance Process Reference Model - Self Assessment SwA Community s Assurance Process Reference Model Mapping to Assurance Models Other valuable resources that are in the process of being mapped include NIST IR 7622: DRAFT Piloting Supply Chain Risk Management Practices for Federal Information Systems NDIA System Assurance Guidebook Microsoft Security Development Lifecycle SAFECode
45 Our Assurance Capability Framework Enables Communication Project leadership and team members need to know where and how to contribute Policy Processes for Assurance Methodologies For Achieving Assurance Assurance PRM defines the goals and practices needed to achieve SwA Assurance for CMMI defines the Assurance Thread for Implementation and Improvement of Assurance Practices that are assumed when using the CMMI-DEV Detailed Criteria Understanding gaps helps suppliers and acquirers prioritize organizational efforts and funding to implement improvement actions
46 DHS Software & Supply Chain Assurance Outreach Co-sponsor SSCA Forum & WGs for government, academia, and industry to facilitate ongoing public-private collaboration. Provide SwA presentations, workshops, and tracks at conferences Co-sponsor issues of CROSSTALK to spread the word Sep/Oct 2009 issue on Resilient Software Mar/Apr 2010 issue on System Assurance Sep/Oct 2010 issue on Game Changing Tools & Practices Mar/Apr 2011 issue on Rugged Software Sep/Oct 2011 issue on Protecting against Predatory Practices Mar/Apr 2012 issue on Securing a Mobile World Sep/Oct 2012 issue on Resilient Cyber Ecosystem Mar/Apr 2013 issue on Supply Chain Risk Management Sep/Oct 2013 issue on Securing the Cloud Mar/Apr 2014 issue on Mitigating Risks from Counterfeit & Tainted Products Collaborate with standards organizations, consortiums, professional societies, education/training initiatives in promoting SwA Provide free SwA resources via BuildSecurityIn website to promote secure development methodologies (since Oct 05) Host SSCA Community Resources & Information Clearinghouse via 46
47 Business Case for Software Assurance April 2009 SwA Report provides background, context and examples: Motivators Cost/Benefit Models Overview Measurement Risk Prioritization Process Improvement & Secure Software Globalization Organizational Development Case Studies and Examples
48 Security Measurement Resources Oct 08 Feb 09 May 09 Practical Measurement Framework for Software Assurance and Information Security Oct 2008
49 Measurement Guidance: Purpose To provide a practical framework for measuring software assurance achievement of SwA goals and objectives within the context of individual projects, programs, or enterprises. Making informed decisions in the software development lifecycle related to information security compliance, performance, and functional requirements/controls Facilitate adoption of secure software design practices Mitigate risks throughout the System Development Lifecycle (SDLC) and ultimately reduce the numbers of vulnerabilities introduced into software code during development Determining if security/performance/trade-offs have been defined and accepted Assessing the trustworthiness of a system. Can be applied beyond SwA to a variety of security-related measurement efforts to help facilitate risk-based decision making through providing quantitative information on a variety of aspects of organization s security related performance. 49
50 Software Assurance Ecosystem: The Formal Framework The value of formalization extends beyond software systems to include related software system process, people and documentation Process Docs & Artifacts Requirements/Design Docs & Artifacts Reports Risk Analysis, etc) Process, People & Documentation Evaluation Environment Some point tools to assist evaluators but mainly manual work Claims in Formal SBVR vocabulary Evidence in Formal SBVR vocabulary Large scope requires large effort Software System / Architecture Evaluation Many integrated & highly automated tools to assist evaluators Claims and Evidence in Formal vocabulary Combination of tools and ISO/OMG standards Standardized SW System Representation In KDM Large scope capable (system of systems) Iterative extraction and analysis for rules Process, People, documentation Evidence Formalized Specifications Software system Technical Evidence Executable Specifications Claims, Arguments and Evidence Repository - Formalized in SBVR vocabulary - Automated verification of claims against evidence - Highly automated and sophisticated risk assessments using transitive inter-evidence point relationships Hardware Environment Software System Artifacts IA Controls Protection Profiles CWE
51 Software Assurance Curriculum Project Vol I: Master of Software Assurance Reference Curriculum In Dec 2010 the IEEE Computer Society and the ACM recognized the Master of Software Assurance (MSwA) Reference Curriculum as a certified master s degree program in SwA the first curriculum to focus on assuring the functionality, dependability, and security of software and systems. Vol II: SwA Undergraduate Course Outlines see to download the PDF version of the report CMU/SEI-2010-TR-019 Vol III: Master of SwA Course Syllabi Vol IV: Community College Education Report on Integrating the MSwA Reference Curriculum into Model Curriculum and Guidelines for Graduate Degree Programs in Information Systems provides reference and guidance material. To facilitate implementation, the MSwA project team is offering assistance, free of charge, to educational institutions looking to launch an MSwA degree program. For more information,go to
52 Software & Supply Chain Assurance Strategy CRR with ExtDep Contract Reviews Business Due-Diligence CFIUS Reviews Etc. Support Response Security Enumerations & Languages, Information Sharing and Analysis Enable Automation Publications, Websites, FedVTE, Training & Education Working Groups, Forums Outreach & Collaborate Manage Supply Chain Risks For Software, Hardware and Services US Fed Dept Mgmt & IT Acquisition Homeland Security Influence Policy Policies implement standards Standards are a foundation of good policy Influence Standards ISO/IEC JTC1 The Open Group OMG ITU-T CYBEX NIST Spec Pubs SP , 160 & 53 Office of Cybersecurity and Communications
53 Security Feature Cross-site Scripting (XSS) Attack (CAPEC-86) Improper Neutralization of Input During Web Page Generation (CWE-79) SQL Injection Attack (CAPEC-66) Improper Neutralization of Special Elements used in an SQL Command (CWE-89) Exploitable Software Weaknesses (CWEs) are exploit targets/vectors for future Zero-Day Attacks 2012 MITRE 53
54 Software Assurance Software Assurance (SwA) is the level of confidence that software functions as intended and is free of vulnerabilities, either intentionally or unintentionally designed or inserted as part of the software throughout the life cycle.* Derived From: CNSSI-4009 Automation Languages, enumerations, registries, tools, and repositories throughout the Lifecycle Including design, coding, testing, deployment, configuration and operation
55 Cyber Threats Emerged Over Time stealth /advanced scanning techniques widespread attacks on DNS infrastructure GUI intruder tools hijacking sessions back doors disabling audits Internet social engineering attacks password cracking network mgmt. diagnostics automated probes/scans executable code attacks (against browsers) automated widespread attacks sniffers propagation of malicious code widespread attacks using NNTP to distribute attack widespread denial-ofservice attacks DDoS attacks www attacks binary encryption increase in tailored worms sophisticated command & control techniques to analyze code for vulnerabilities without source code Attack Sophistication diffuse spyware anti-forensic techniques home users targeted distributed attack tools increase in wide-scale Trojan horse distribution Windows-based remote controllable Trojans (Back Orifice) password guessing exploiting known vulnerabilities packet spoofing burglaries automated probes/scans 1980 s 1990 s 2000 s 2010 s
56 Solutions Also Emerged Over Time stealth /advanced scanning techniques widespread attacks on DNS infrastructure GUI intruder tools hijacking sessions back doors disabling audits Internet social engineering attacks password cracking network mgmt. diagnostics automated probes/scans executable code attacks (against browsers) automated widespread attacks sniffers propagation of malicious code widespread attacks using NNTP to distribute attack widespread denial-ofservice attacks DDoS attacks www attacks binary encryption increase in tailored worms sophisticated command & control techniques to analyze code for vulnerabilities without source code Attack Sophistication diffuse spyware anti-forensic techniques home users targeted distributed attack tools increase in wide-scale Trojan horse distribution Windows-based remote controllable Trojans (Back Orifice) password guessing exploiting known vulnerabilities packet spoofing burglaries automated probes/scans 1980 s 1990 s 2000 s 2010 s
57 Architecting Security with Information Standards for COIs Asset Management Vulnerability Management Configuration Management Threat Management System Development System Certification Intrusion Detection Incident Management Change Management Trust Management Identity Management Central Reporting 2012 MITRE
58 Knowledge Repositories Mitigating Risk Exposures Responding to Security Threats Asset Definition Configuration Guidance Vulnerability Alert Threat Alert Indicator Sharing Incident Report CPE/SWID/OVAL XCCDF/OVAL/ CCE/CCSS/OCIL CVE/CWE/ OVAL/CVSS /CVRF CVE/CWE/CVSS/ CAPEC/MAEC/ CybOX IODEF, TAXII, CVE, CPE, MAEC, CEE, CybOX, STIX, RID, RID-T CYBEX, CWE, CVE IODEF, OVAL, CPE, CVSS, MAEC, CEE, CWSS, STIX, TAXII, CybOX, RID, RID-T Asset Inventory Configuration Guidance Analysis Vulnerability Analysis Threat Analysis Intrusion Detection Incident Management OVAL/XCCDF /OCIL/CCE/C CSS/CPE/ SWID/ARF Supply Chain Risk Mgt, System & Software Assurance Guidance/ Requirements CWE/CAPEC/ SBVR/CWSS/ MAEC, SCOX Assessment of System Development, Integration, & Sustainment Activities and Certification & Accreditation CPE/ SWID/ OVAL/ ARF CCE/ OVAL/OCIL/ XCCDF/CPE /SWID/CCS S/ARF CVE/CWE/ CVSS/CCE/ OVAL/OCIL/XCC DF/CPE/ CWSS/SWID CVE/CWE/ CVSS/CCE/ OVAL/OCIL/XCCD F/CPE/CAPEC/MA EC/CybOX/SWID CVE/CWE/ CVSS/CCE/ OVAL/OCIL/ARF/X CCDF/CPE/CAPEC /MAEC/CEE/CybOX /SWID Operations Security Management Processes CWE/CAPEC/CWSS/ MAEC/OVAL/OCIL/X CCDF/CCE/CPE/ARF /SWID/SAFES/SACM CVE/CWE/CVSS/ CCE/CCSS/OVA L/XCCDF/OCIL/ CPE/CAPEC/MA EC/CWSS/CEE/A RF/SWID/CybOX Operational Enterprise Networks CVE/CWE/CVSS/CCE/CCSS/OVAL/OCIL/XC CDF/CPE/CAPEC/MAEC/CWSS/CEE/ARF/S WID/CybOX/STIX Development & Sustainment Security Management Processes Trust Management Enterprise IT Change Management Identity Management Centralized Reporting Enterprise IT Asset Management
59 Cyber Ecosystem Standardization Efforts What IT systems do I have in my enterprise? What known vulnerabilities do I need to worry about? What vulnerabilities do I need to worry about right now? How can I configure my systems more securely? How do I define a policy of secure configurations? How can I be sure my systems conform to policy? How can I ensure operation of my systems conforms to policy? What weaknesses in my software could be exploited? What attacks can exploit which weaknesses? How can we recognize malware & share that info? What observable behavior might put my enterprise at risk? How can I share threat information? What events should be logged, and how? How can I aggregate assessment results? CPE (Platforms) CVE (Vulnerabilities) CVSS (Scoring System) CCE (Configurations) XCCDF (Configuration Checklists) OVAL (Assessment Language) OCIL (Interactive Language) CWE (Weaknesses) CAPEC (Attack Patterns) MAEC (Malware Attributes) CybOX (Cyber Observables) STIX (Structure Threat Information) CEE (Events) - ARF (Assessment Results) Many standards are XML-based; enabling automation of information exchange Several standards support multiple enterprise cybersecurity functions 2011 MITRE
60 The Structured Threat Information expression A framework/data model to standardize the representation of cyber threat intelligence Builds on existing languages/models where possible Provides a structure that enables: Consistent semantics Automated interpretation Advanced analysis Enables the expression of relationships between entities within the framework: E.g. threat actor A uses TTP B which can be detected via indicator C 60
61 STIX: Primary Components What activity are we seeing? What threats should I look for? Where has this threat been seen? What does it do? What weaknesses does it exploit? Why does it do this? Who is responsible for this threat? What can I do about it? 61
62 62
63 Kill Chain Exploit Targets Courses of Action Why were they doing it? Why should you care about it? What exactly were they doing? What you are looking for Where was it seen? Who was doing it? What were they looking to exploit? What should you do about it?
64 STIX Architecture Why were they doing it? Why should you care about it? What you are looking for What exactly were they doing? What could/should have been done to harden the attack surface/vector to prevent Where was it seen? the target from being exploitable? Who was doing it? What were they looking to exploit? What should you do about it?
65 Leverage Common Weakness Enumeration (CWE) to mitigate risks to mission/business domains CWE is a formal list of software weakness types created to: Serve as a common language for describing software security weaknesses in architecture, design, or code. Serve as a standard measuring stick for software security tools targeting these weaknesses. Provide a common baseline standard for weakness identification, mitigation, and prevention efforts. Some Common Types of Software Weaknesses: Buffer Overflows, Format Strings, Etc. Structure and Validity Problems Common Special Element Manipulations Channel and Path Errors Handler Errors User Interface Errors Pathname Traversal and Equivalence Errors Authentication Errors Resource Management Errors Insufficient Verification of Data Code Evaluation and Injection Randomness and Predictability cwe.mitre.org
66 Security Automation Pipework 66 CVE enabling reporting and patching of vulnerabilities CWE identifying and mitigating root cause exploitable weaknesses CybOX cyber observables and supply chain exploit indicators CAPEC schema attack patterns and software exploits Homeland Security Office of Cybersecurity and Communications
67 CWRAF/CWSS Provides Risk Prioritization for CWE throughout Software Life Cycle Enables education and training to provide specific practices for eliminating software fault patterns; Enables developers to mitigate top risks attributable to exploitable software; Enables testing organizations to use suite of test tools & methods (with CWE Coverage Claims Representation) that cover applicable concerns; Enables users and operation organizations to deploy and use software that is more resilient and secure; Enables procurement organizations to specify software security expectations through acquisition of software, hosted applications and services.
68 When should I focus on Weaknesses and Vulnerabilities? Concept Design Source code Object code Binaries Deployed system Focus on Weaknesses Focus on Vulnerabilities A type of defect that may be exploitable. Keep Weaknesses from becoming vulnerabilities Something in code that can actually be exploited.
69 Software Assurance (SwA) Competency Model, March 2013 Developed to support the following uses: Provide employers of SwA personnel with a means to assess the SwA capabilities of current and potential employees. Offer guidance to academic or training organizations: develop SwA courses to support the needs of organizations that are hiring and developing SwA professionals. Enhance SwA curricula guidance by providing information about industry needs and expectations for competent SwA professionals. Provide direction and a progression for the development and career planning of SwA professionals. Provide support for professional certification and licensing.
70 ICT/software security risk landscape is a convergence between defense in depth and defense in breadth Risk shifts to end-points; Enterprise Risk Management and Governance are security motivators Acquisition could influence the lifecycle; more than development Software & Supply Chain Assurance provides a focus for: -- Resilient Software and ICT Components, -- Security in the Component Life Cycle, -- Software Security in Services, and -- Supply Chain Risk Management (mitigating risks of counterfeit & tainted products)
71 What Are We Protecting? Program Protection Planning DODI Update Technology What: Leading-edge research and technology Who Identifies: Technologists, System Engineers Components What: Mission-critical elements and components Who Identifies: System Engineers, Logisticians What: Information about applications, processes, capabilities and end-items Who Identifies: All Information ID Process: CPI Identification Threat Assessment: Foreign collection threat informed by Intelligence and Counterintelligence assessments Countermeasures: AT, Classification, Export Controls, Security, Foreign Disclosure, and CI activities Focus: Keep secret stuff in by protecting any form of technology ID Process: Criticality Analysis Threat Assessment: DIA SCRM TAC Countermeasures: SCRM, SSE, Anticounterfeits, software assurance, Trusted Foundry, etc. Focus: Keep malicious stuff out by protecting key mission components ID Process: CPI identification, criticality analysis, and classification guidance Threat Assessment: Foreign collection threat informed by Intelligence and Counterintelligence assessments Countermeasures: Information Assurance, Classification, Export Controls, Security, etc. Focus: Keep critical information from getting out by protecting data Protecting Warfighting Capability Throughout the Lifecycle Note: Program Protection Planning Includes DoDI 8500 series Comprehensive Program Protection 5/1/2012 Page 71 Distribution Statement A Cleared for public release by OSR on 4/25/2012, SR Case # 12-S-1841 applies.
72 SSCA Focus on Tainted Components Mitigating risks attributable to exploitable non-conforming constructs in ICT Tainted products are those that are corrupted with malware, or exploitable weaknesses & vulnerabilities that put users at risk Enable scalable detection and reporting of tainted ICT components Leverage/mature related existing standardization efforts Provide Taxonomies, schema & structured representations with defined observables & indicators for conveying information: o Tainted constructs: Malicious logic/malware (MAEC), Exploitable Weaknesses (CWE); Vulnerabilities (CVE) o Attack Patterns (CAPEC) Catalogue Diagnostic Methods, Controls, Countermeasures, & Mitigation Practices Publicly reported weaknesses and vulnerabilities with patches accessible via National Vulnerability Database (NVD) sponsored by DHS & hosted by NIST Exploitable weakness TAINTED [exploitable weakness, vulnerability, or malicious construct] Unpatched Vulnerability COUNTERFEIT Malware Exploitable weakness Unpatched Vulnerability AUTHENTIC DEFECTIVE Components can become tainted intentionally or unintentionally throughout the supply chain, SDLC, and in Ops & sustainment *Text demonstrates examples of overlap
73 Scope of UL Cybersecurity Assurance Program UL CAP focus on Network-Connectable Devices Addresses known vulnerabilities (CVSS+) at the time of certification (i.e. CVEs catalogued in the NVD) Performs baseline weakness assessment for potential zero day vulnerabilities (CWSS and rankings from other organizations) Addresses known malware at time of certification Addressing the most relevant CWEs, establishes a baseline to mitigate weaknesses that, if otherwise exploited, could be vectors of attack; becoming zero-day vulnerabilities * Example of programs using CVE, CWE, CWSS, CVSS, CAPEC, etc 73
74 ICT/Software & Supply Chain Assurance is a National Security & Economic Issue Adversaries can gain intimate access to target systems, especially in a global supply chain that offers limited transparency Advances in science and technology will always outpace the ability of government and industry to react with new policies and standards National security policies must conform with international laws and agreements while preserving a nation s rights and freedoms, and protecting a nation s self interests and economic goals; Forward-looking policies can adapt to the new world of global supply chains; Standards for automation, processes, and products must mature to better address supply chain risk management, systems/software assurance, and the exchange of information and indicators for cyber security; Assurance Rating Schemes for ICT/software products and suppliers are needed. ICT/software suppliers and buyers can take more deliberate actions to security-enhance their processes, practices and products to mitigate risks Government & Industry have significant leadership roles in solving this Individuals can influence the way their organizations adopt security practices Globalization will not be reversed; this is how we conduct business To remain relevant, standards and capability benchmarking measures must address assurance mechanisms needed to manage IT/Software Supply Chain risks.
75 Homeland Security National Defense Commerce & Standards General Services Public-Private Collaboration Efforts for Security Automation, Software Assurance, and Supply Chain Risk Management
76 Software & Supply Chain Assurance: Enabling Enterprise Resilience through Security Automation, Software Assurance, and Supply Chain Risk Management Joe Jarzombek, PMP, CSSLP Director for Software & Supply Chain Assurance Cyber Security & Communications (until 31 Dec 2015) Mitigating Risk Exposures Attributable to Exploitable ICT Products and Services
Software & Supply Chain Assurance: Mitigating Risks Attributable to Exploitable ICT / Software Products and Processes
Software & Supply Chain Assurance: Mitigating Risks Attributable to Exploitable ICT / Software Products and Processes Joe Jarzombek, PMP, CSSLP Director for Software & Supply Chain Assurance Stakeholder
Software Assurance: Enabling Enterprise Resilience through Security Automation and Software Supply Chain Risk Management
Software Assurance: Enabling Enterprise Resilience through Security Automation and Software Supply Chain Risk Management Joe Jarzombek, PMP, CSSLP Director for Software Assurance Cyber Security & Communications
Secure Content Automation Protocol (SCAP): How it is increasingly used to automate enterprise security management activities
Secure Content Automation Protocol (SCAP): How it is increasingly used to automate enterprise security management activities Sean Barnum [email protected] September 2011 Overview What is SCAP? Why SCAP?
Software Assurance: Enabling Security Automation and Software Supply Chain Risk Management. Commerce. National Defense. Today Everything s Connected
Software Assurance: Enabling Security Automation and Software Supply Chain Risk Interdependencies Between Physical & Cyber Infrastructures: Convergence of Safety, Security and Dependability Considerations
Reducing risks in the Software Acquisition Life Cycle
Reducing risks in the Software Acquisition Life Cycle Stan Wisseman 6 December 2007 Agenda Buyers vs. Sellers Need for enhancing acquisition process with SwA considerations DHS SwA Initiative Overview
Software Assurance in Acquisition and Contract Language
Software Assurance in Acquisition and Contract Language Acquisition & Outsourcing, Volume I Version 1.1, July 31, 2009 Software Assurance (SwA) Pocket Guide Resources This is a resource for getting started
SAFECode Security Development Lifecycle (SDL)
SAFECode Security Development Lifecycle (SDL) Michael Howard Microsoft Matthew Coles EMC 15th Semi-annual Software Assurance Forum, September 12-16, 2011 Agenda Introduction to SAFECode Security Training
Security Content Automation Protocol for Governance, Risk, Compliance, and Audit
UNCLASSIFIED Security Content Automation Protocol for Governance, Risk, Compliance, and Audit presented by: Tim Grance The National Institute of Standards and Technology UNCLASSIFIED Agenda NIST s IT Security
Adding a Security Assurance Dimension to Supply Chain Practices
Adding a Security Assurance Dimension to Supply Chain Practices John Whited, CISSP, CSSLP Randall Brooks, CISSP, CSSLP Raytheon Company Session ID: GRC-401 Session Classification: Intermediate Agenda What
Continuous Monitoring
Continuous Monitoring The Evolution of FISMA Compliance Tina Kuligowski [email protected] Overview Evolution of FISMA Compliance NIST Standards & Guidelines (SP 800-37r1, 800-53) OMB Memorandums
WHITE PAPER ON SECURITY TESTING IN TELECOM NETWORK
WHITE PAPER ON SECURITY TESTING IN TELECOM NETWORK DATE OF RELEASE: 27 th July 2012 Table of Contents 1. Introduction... 2 2. Need for securing Telecom Networks... 3 3. Security Assessment Techniques...
Cybersecurity Resources
Assessment Resources Cybersecurity Resources Cyber Resiliency Review (CRR) is a DHS assessment tool that measures the implementation of key cybersecurity capacities and capabilities. The goal of the CRR
FREQUENTLY ASKED QUESTIONS
FREQUENTLY ASKED QUESTIONS Continuous Monitoring 1. What is continuous monitoring? Continuous monitoring is one of six steps in the Risk Management Framework (RMF) described in NIST Special Publication
CDM Vulnerability Management (VUL) Capability
CDM Vulnerability Management (VUL) Capability Department of Homeland Security Office of Cybersecurity and Communications Federal Network Resilience Vulnerability Management Continuous Diagnostics and Mitigation
PASTA Abstract. Process for Attack S imulation & Threat Assessment Abstract. VerSprite, LLC Copyright 2013
2013 PASTA Abstract Process for Attack S imulation & Threat Assessment Abstract VerSprite, LLC Copyright 2013 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Release of the Draft Cybersecurity Procurement Language for Energy Delivery Systems
Release of the Draft Cybersecurity Procurement Language for Energy Delivery Systems Energy Sector Control Systems Working Group Supporting the Electricity Sector Coordinating Council, Oil & Natural Gas
Actions and Recommendations (A/R) Summary
Actions and Recommendations (A/R) Summary Priority I: A National Cyberspace Security Response System A/R 1-1: DHS will create a single point-ofcontact for the federal government s interaction with industry
DoD Software Assurance (SwA) Overview
DoD Software Assurance (SwA) Overview Tom Hurt Office of the Deputy Assistant Secretary of Defense for Systems Engineering NDIA Program Protection Summit / Workshop McLean, VA May 19, 2014 May 19, 2014
Protect Your Organization With the Certification That Maps to a Master s-level Education in Software Assurance
Protect Your Organization With the Certification That Maps to a Master s-level Education in Software Assurance Sponsored by the U.S. Department of Homeland Security (DHS), the Software Engineering Institute
Homeland Security Perspectives: Cyber Security Partnerships and Measurement Activities
16 Oct 2012 Homeland Security Perspectives: Cyber Security Partnerships and Measurement Activities Bradford Willke Cyber Security Advisor, Mid Atlantic Region National Cyber Security Division (NCSD) Office
Developing Secure Software in the Age of Advanced Persistent Threats
Developing Secure Software in the Age of Advanced Persistent Threats ERIC BAIZE EMC Corporation DAVE MARTIN EMC Corporation Session ID: ASEC-201 Session Classification: Intermediate Our Job: Keep our Employer
DEVELOPING SECURE SOFTWARE
DEVELOPING SECURE SOFTWARE A FOUNDATION FOR CLOUD AND IOT SECURITY Eric Baize @ericbaize Senior Director, Product Security Office EMC Corporation Chairman of SAFECode CSA EMEA Congress November 2015 1
Computer System Security Updates
Why patch? If you have already deployed a network architecture, such as the one recommended by Rockwell Automation and Cisco in the Converged Plantwide Ethernet Design and Implementation Guide (http://www.ab.com/networks/architectures.html),
Software Development: The Next Security Frontier
James E. Molini, CISSP, CSSLP Microsoft Member, (ISC)² Advisory Board of the Americas [email protected] http://www.codeguard.org/blog Software Development: The Next Security Frontier De-perimiterization
How To Monitor Your Entire It Environment
Preparing for FISMA 2.0 and Continuous Monitoring Requirements Symantec's Continuous Monitoring Solution White Paper: Preparing for FISMA 2.0 and Continuous Monitoring Requirements Contents Introduction............................................................................................
IT Risk Management: Guide to Software Risk Assessments and Audits
IT Risk Management: Guide to Software Risk Assessments and Audits Contents Overview... 3 Executive Summary... 3 Software: Today s Biggest Security Risk... 4 How Software Risk Enters the Enterprise... 5
Promoting Application Security within Federal Government. AppSec DC November 13, 2009. The OWASP Foundation http://www.owasp.org
Promoting Application Security within Federal Government AppSec DC November 13, 2009 Dr. Sarbari Gupta, CISSP, CISA Founder/President Electrosoft [email protected] 703-437-9451 ext 12 The Foundation
The introduction covers the recent changes is security threats and the effect those changes have on how we protect systems.
1 Cyber-attacks frequently take advantage of software weaknesses unintentionally created during development. This presentation discusses some ways that improved acquisition practices can reduce the likelihood
Experience the commitment WHITE PAPER. Information Security Continuous Monitoring. Charting the Right Course. cgi.com 2014 CGI GROUP INC.
Experience the commitment WHITE PAPER Information Security Continuous Monitoring Charting the Right Course May 2014 cgi.com 2014 CGI GROUP INC. During the last few months of 2013, six federal agencies
Penetration Testing Guidelines For the Financial Industry in Singapore. 31 July 2015
For the Financial Industry in Singapore 31 July 2015 TABLE OF CONTENT 1. EXECUTIVE SUMMARY 3 2. INTRODUCTION 4 2.1 Audience 4 2.2 Purpose and Scope 4 2.3 Definitions 4 3. REQUIREMENTS 6 3.1 Overview 6
CONTINUOUS DIAGNOSTICS BEGINS WITH REDSEAL
CONTINUOUS DIAGNOSTICS BEGINS WITH REDSEAL WHAT IS CDM? The continuous stream of high profile cybersecurity breaches demonstrates the need to move beyond purely periodic, compliance-based approaches to
C ETS C/ETS: CYBER INTELLIGENCE + ENTERPRISE SOLUTIONS CSCSS / ENTERPRISE TECHNOLOGY + SECURITY
CSCSS / ENTERPRISE TECHNOLOGY + SECURITY C/ETS: CYBER INTELLIGENCE + ENTERPRISE SOLUTIONS CENTRE FOR STRATEGIC CSCSS CYBERSPACE + SECURITY SCIENCE CSCSS / ENTERPRISE TECHNOLOGY + SECURITY GROUP Information
White Paper. Understanding NIST 800 37 FISMA Requirements
White Paper Understanding NIST 800 37 FISMA Requirements Contents Overview... 3 I. The Role of NIST in FISMA Compliance... 3 II. NIST Risk Management Framework for FISMA... 4 III. Application Security
How To Use A Policy Auditor 6.2.2 (Macafee) To Check For Security Issues
Vendor Provided Validation Details - McAfee Policy Auditor 6.2 The following text was provided by the vendor during testing to describe how the product implements the specific capabilities. Statement of
How To Ensure Your Software Is Secure
Software Assurance Countermeasures in Program Protection Planning MARCH 2014 Deputy Assistant Secretary of Defense for Systems Engineering and Department of Defense Chief Information Officer Washington,
CDM Software Asset Management (SWAM) Capability
CDM Software Asset Management (SWAM) Capability Department of Homeland Security Office of Cybersecurity and Communications Federal Network Resilience Table of Contents 1 PURPOSE AND SCOPE... 2 2 THREAT
Defense-in-Depth Strategies for Secure, Open Remote Access to Control System Networks
Defense-in-Depth Strategies for Secure, Open Remote Access to Control System Networks A look at multi-vendor access strategies Joel Langill TÜV FSEng ID-1772/09, CEH, CPT, CCNA Security Consultant / Staff
Telecom Testing and Security Certification. A.K.MITTAL DDG (TTSC) Department of Telecommunication Ministry of Communication & IT
Telecom Testing and Security Certification A.K.MITTAL DDG (TTSC) Department of Telecommunication Ministry of Communication & IT 1 Need for Security Testing and Certification Telecom is a vital infrastructure
BMC Client Management - SCAP Implementation Statement. Version 12.0
BMC Client Management - SCAP Implementation Statement Version 12.0 BMC Client Management - SCAP Implementation Statement TOC 3 Contents SCAP Implementation Statement... 4 4 BMC Client Management - SCAP
Promoting Application Security within Federal Government. AppSec DC November 13, 2009. The OWASP Foundation http://www.owasp.org
Promoting Application Security within Federal Government AppSec DC November 13, 2009 Dr. Sarbari Gupta, CISSP, CISA Founder/President Electrosoft [email protected] 703-437-9451 ext 12 The Foundation
Cisco Security Optimization Service
Cisco Security Optimization Service Proactively strengthen your network to better respond to evolving security threats and planned and unplanned events. Service Overview Optimize Your Network for Borderless
Build (develop) and document Acceptance Transition to production (installation) Operations and maintenance support (postinstallation)
It is a well-known fact in computer security that security problems are very often a direct result of software bugs. That leads security researches to pay lots of attention to software engineering. The
ENERGY SECTOR CYBERSECURITY FRAMEWORK IMPLEMENTATION GUIDANCE
ENERGY SECTOR CYBERSECURITY FRAMEWORK IMPLEMENTATION GUIDANCE JANUARY 2015 U.S. DEPARTMENT OF ENERGY OFFICE OF ELECTRICITY DELIVERY AND ENERGY RELIABILITY Energy Sector Cybersecurity Framework Implementation
Department of Homeland Security Federal Government Offerings, Products, and Services
Department of Homeland Security Federal Government Offerings, Products, and Services The Department of Homeland Security (DHS) partners with the public and private sectors to improve the cybersecurity
U.S. Office of Personnel Management. Actions to Strengthen Cybersecurity and Protect Critical IT Systems
U.S. Office of Personnel Management Actions to Strengthen Cybersecurity and Protect Critical IT Systems June 2015 1 I. Introduction The recent intrusions into U.S. Office of Personnel Management (OPM)
Cyber Security Metrics Dashboards & Analytics
Cyber Security Metrics Dashboards & Analytics Feb, 2014 Robert J. Michalsky Principal, Cyber Security NJVC, LLC Proprietary Data UNCLASSIFIED Agenda Healthcare Sector Threats Recent History Security Metrics
7 Homeland. ty Grant Program HOMELAND SECURITY GRANT PROGRAM. Fiscal Year 2008
U.S. D EPARTMENT OF H OMELAND S ECURITY 7 Homeland Fiscal Year 2008 HOMELAND SECURITY GRANT PROGRAM ty Grant Program SUPPLEMENTAL RESOURCE: CYBER SECURITY GUIDANCE uidelines and Application Kit (October
Principles for Software Assurance Assessment. A Framework for Examining the Secure Development Processes of Commercial Technology Providers
A Framework for Examining the Secure Development Processes of Commercial Technology Providers PRIMARY AUTHORS: Shaun Gilmore, Senior Security Program Manager, Trustworthy Computing, Microsoft Corporation
High Level Cyber Security Assessment 2/1/2012. Assessor: J. Doe
2/1/2012 Assessor: J. Doe Disclaimer This report is provided as is for informational purposes only. The Department of Homeland Security (DHS) does not provide any warranties of any kind regarding any information
Get Confidence in Mission Security with IV&V Information Assurance
Get Confidence in Mission Security with IV&V Information Assurance September 10, 2014 Threat Landscape Regulatory Framework Life-cycles IV&V Rigor and Independence Threat Landscape Continuously evolving
Security in the Network Infrastructure - DNS, DDoS,, etc.
Security in the Network Infrastructure - DNS, DDoS,, etc. GTER, São Paulo December 8, 2006 Steve Crocker, [email protected] Russ Mundy, [email protected] Proactive Security Build security into the infrastructure
Information Technology Engineers Examination. Information Security Specialist Examination. (Level 4) Syllabus
Information Technology Engineers Examination Information Security Specialist Examination (Level 4) Syllabus Details of Knowledge and Skills Required for the Information Technology Engineers Examination
Design Specification for IEEE Std 1471 Recommended Practice for Architectural Description IEEE Architecture Working Group 0 Motivation
Design Specification for IEEE Std 1471 Recommended Practice for Architectural Description IEEE Architecture Working Group 0 Motivation Despite significant efforts to improve engineering practices and technologies,
ICT Security Cybersecurity CYBEX Overview of activities in ITU-T with focus on Study Group 17
ICT Security Cybersecurity CYBEX Overview of activities in ITU-T with focus on Study Group 17 TSB Briefing to the Regional Offices, 28 Feb 2011 Martin Euchner Advisor of ITU-T Study Group 17 [email protected]
Cybersecurity The role of Internal Audit
Cybersecurity The role of Internal Audit Cyber risk High on the agenda Audit committees and board members are seeing cybersecurity as a top risk, underscored by recent headlines and increased government
Sharing Data / Information
Trustworthy Software Initiative (T S I) Sharing Data / Information Update Briefing for TF-CSIRT Thursday 26 th September 2013, London, UK Ian Bryant TSI Technical Director DMU/CSC/TS/2013/156 v1.0 2013-09-26
Middle Class Economics: Cybersecurity Updated August 7, 2015
Middle Class Economics: Cybersecurity Updated August 7, 2015 The President's 2016 Budget is designed to bring middle class economics into the 21st Century. This Budget shows what we can do if we invest
Federal Desktop Core Configuration (FDCC)
Federal Desktop Core Configuration (FDCC) Presented by: Saji Ranasinghe Date: October, 2007 FDCC Federal Desktop Core Configuration (FDCC) Standardized Configuration with Hardened Security Settings to
Defending Against Data Beaches: Internal Controls for Cybersecurity
Defending Against Data Beaches: Internal Controls for Cybersecurity Presented by: Michael Walter, Managing Director and Chris Manning, Associate Director Protiviti Atlanta Office Agenda Defining Cybersecurity
Global Cyber Range (GCR) Empowering the Cybersecurity Professional (CyPro)
Global Cyber Range (GCR) Empowering the Cybersecurity Professional (CyPro) NICE Conference 2014 CYBERSECURITY RESILIENCE A THREE TIERED SOLUTION NIST Framework for Improving Critical Infrastructure Cybersecurity
Is Penetration Testing recommended for Industrial Control Systems?
Is Penetration Testing recommended for Industrial Control Systems? By Ngai Chee Ban, CISSP, Honeywell Process Solutions, Asia Pacific Cyber Security Assessment for Industrial Automation Conducting a cyber-security
Cyber Resilience Implementing the Right Strategy. Grant Brown Security specialist, CISSP @TheGrantBrown
Cyber Resilience Implementing the Right Strategy Grant Brown specialist, CISSP @TheGrantBrown 1 2 Network + Technology + Customers = $$ 3 Perfect Storm? 1) Increase in Bandwidth (extended reach) 2) Available
White Paper. Automating Your Code Review: Moving to a SaaS Model for Application Security
White Paper Automating Your Code Review: Moving to a SaaS Model for Application Security Contents Overview... 3 Executive Summary... 3 Code Review and Security Analysis Methods... 5 Source Code Review
Using the HITRUST CSF to Assess Cybersecurity Preparedness 1 of 6
to Assess Cybersecurity Preparedness 1 of 6 Introduction Long before the signing in February 2013 of the White House Executive Order Improving Critical Infrastructure Cybersecurity, HITRUST recognized
Understanding How They Attack Your Weaknesses: CAPEC Sean Barnum MITRE
Understanding How They Attack Your Weaknesses: CAPEC Sean Barnum MITRE HS SEDI is a trademark of the U.S. Department of Homeland Security (DHS). The Long-established Principal of Know Your Enemy One who
Compliance Guide ISO 27002. Compliance Guide. September 2015. Contents. Introduction 1. Detailed Controls Mapping 2.
ISO 27002 Compliance Guide September 2015 Contents Compliance Guide 01 02 03 Introduction 1 Detailed Controls Mapping 2 About Rapid7 7 01 INTRODUCTION If you re looking for a comprehensive, global framework
Introduction to NICE Cybersecurity Workforce Framework
Introduction to NICE Cybersecurity Workforce Framework Jane Homeyer, Ph.D., Deputy ADNI/HC for Skills and Human Capital Data, ODNI Margaret Maxson, Director, National Cybersecurity Education Strategy,
Cybersecurity Strategic Talent Management. March, 2012
Cybersecurity Strategic Talent Management March, 2012 Cyber Operations - Starts with People Exploit Intel Attack Cyber Operations Defend Enablers 2 Talent Management Challenge Mission: Attract, Develop,
Manage Vulnerabilities (VULN) Capability Data Sheet
Manage Vulnerabilities (VULN) Capability Data Sheet Desired State: - Software products installed on all devices are free of known vulnerabilities 1 - The list of known vulnerabilities is up-to-date Desired
NASA OFFICE OF INSPECTOR GENERAL
NASA OFFICE OF INSPECTOR GENERAL OFFICE OF AUDITS SUITE 8U71, 300 E ST SW WASHINGTON, D.C. 20546-0001 April 14, 2016 TO: SUBJECT: Renee P. Wynn Chief Information Officer Final Memorandum, Review of NASA
CS 2 SAT: The Control Systems Cyber Security Self-Assessment Tool
INL/CON-07-12810 PREPRINT CS 2 SAT: The Control Systems Cyber Security Self-Assessment Tool ISA Expo 2007 Kathleen A. Lee January 2008 This is a preprint of a paper intended for publication in a journal
HP Application Security Center
HP Application Security Center Web application security across the application lifecycle Solution brief HP Application Security Center helps security professionals, quality assurance (QA) specialists and
Juniper Networks Secure
White Paper Juniper Networks Secure Development Lifecycle Six Practices for Improving Product Security Copyright 2013, Juniper Networks, Inc. 1 Table of Contents Executive Summary...3 Introduction...3
Attachment A. Identification of Risks/Cybersecurity Governance
Attachment A Identification of Risks/Cybersecurity Governance 1. For each of the following practices employed by the Firm for management of information security assets, please provide the month and year
ICBA Summary of FFIEC Cybersecurity Assessment Tool
ICBA Summary of FFIEC Cybersecurity Assessment Tool July 2015 Contact: Jeremy Dalpiaz Assistant Vice President Cyber Security and Data Security Policy [email protected] www.icba.org ICBA Summary
Enterprise Security Tactical Plan
Enterprise Security Tactical Plan Fiscal Years 2011 2012 (July 1, 2010 to June 30, 2012) Prepared By: State Chief Information Security Officer The Information Security Council State of Minnesota Enterprise
Suggested Language to Incorporate System Security Engineering for Trusted Systems and Networks into Department of Defense Requests for Proposals
Suggested Language to Incorporate System Security Engineering for Trusted Systems and Networks into Department of Defense Requests for Proposals JANUARY 2014 Deputy Assistant Secretary of Defense for Systems
Cybersecurity Enhancement Account. FY 2017 President s Budget
Cybersecurity Enhancement Account FY 2017 President s Budget February 9, 2016 Table of Contents Section 1 Purpose... 3 1A Mission Statement... 3 1.1 Appropriations Detail Table... 3 1B Vision, Priorities
FFIEC Cybersecurity Assessment Tool
Overview In light of the increasing volume and sophistication of cyber threats, the Federal Financial Institutions Examination Council 1 (FFIEC) developed the Cybersecurity Tool (), on behalf of its members,
Standards & Practices for the software and system engineers /
Standards & Practices for the software and system engineers / professionals John Walz [email protected] IEEE Computer Society 1 st VP IEEE Software & Systems Engineering i Standards d Committee Systems
The Software Supply Chain Integrity Framework. Defining Risks and Responsibilities for Securing Software in the Global Supply Chain.
The Software Supply Chain Integrity Framework Defining Risks and Responsibilities for Securing Software in the Global Supply Chain July 21, 2009 Editor Stacy Simpson, SAFECode Contributors Dan Reddy, EMC
Software Application Control and SDLC
Software Application Control and SDLC Albert J. Marcella, Jr., Ph.D., CISA, CISM 1 The most effective way to achieve secure software is for its development life cycle processes to rigorously conform to
Enterprise Cybersecurity Best Practices Part Number MAN-00363 Revision 006
Enterprise Cybersecurity Best Practices Part Number MAN-00363 Revision 006 April 2013 Hologic and the Hologic Logo are trademarks or registered trademarks of Hologic, Inc. Microsoft, Active Directory,
elearning for Secure Application Development
elearning for Secure Application Development Curriculum Application Security Awareness Series 1-2 Secure Software Development Series 2-8 Secure Architectures and Threat Modeling Series 9 Application Security
A Systems Engineering Approach to Developing Cyber Security Professionals
A Systems Engineering Approach to Developing Cyber Security Professionals D r. J e r r y H i l l Approved for Public Release; Distribution Unlimited. 13-3793 2013 The MITRE Corporation. All rights reserved.
Network Security and Vulnerability Assessment Solutions
Network Security and Vulnerability Assessment Solutions Unified Vulnerability Management It s a known fact that the exponential growth and successful exploitation of vulnerabilities create increasingly
DRAFT. Practical Measurement Framework for Software Assurance and Information Security. Version 1.0
DRAFT Practical Measurement Framework for Software Assurance and Information Security Version 1.0 October 1, 2008 Acknowledgements The Department of Defense (DoD), Department of Homeland Security (DHS),
OCIE CYBERSECURITY INITIATIVE
Topic: Cybersecurity Examinations Key Takeaways: OCIE will be conducting examinations of more than 50 registered brokerdealers and registered investment advisers, focusing on areas related to cybersecurity.
Software Sustainability Challenges for Acquisition, Engineering, and Capability Delivery in the Face of the Growing Cyber Threat
2012 Systems and Software Technology Conference Software Sustainability Challenges for Acquisition, Engineering, and Capability Delivery in the Face of the Growing Cyber Threat Paul R. Croll Fellow CSC
