Evaluating the Quality of Software Engineering Performance Data

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Evaluating the Quality of Software Engineering Performance Data"

Transcription

1 Evaluating the Quality of Software Engineering Performance Data James Over Software Engineering Institute Carnegie Mellon University July 2014

2 Copyright 2014 Carnegie Mellon University This material is based upon work funded and supported by Industry cost recovery under Contract No. FA C-0003 with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded research and development center sponsored by the United States Department of Defense. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of Industry cost recovery or the United States Department of Defense. NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN AS-IS BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. This material has been approved for public release and unlimited distribution. This material may be reproduced in its entirety, without modification, and freely distributed in written or electronic form without requesting formal permission. Permission is required for any other use. Requests for permission should be directed to the Software Engineering Institute at Carnegie Mellon is registered in the U.S. Patent and Trademark Office by Carnegie Mellon University. SM and TSP SM are service marks of Carnegie Mellon University. DM

3 Managing Software Development Managing software engineering is challenging due to the nature of the work. Intuition The ability to understand something immediately without the need for conscious reasoning Counterintuitive Contrary to intuition or commonsense, but often true 3

4 Management Visibility Data can provide visibility into a software project s past and future performance. Did the project meet key milestones? Is the project on schedule? Are costs under control? How does project performance compare to other projects? Will the software work when released? 4

5 Can We Trust the Data? Manufacturing data standard operational definitions machines measure, collect, and report precise and accurate Software engineering project data lack of standards and operational definitions In God we trust, all others bring data - Deming people often measure, collect, and report imprecise and inaccurate 5

6 TSP Measurement Framework TSP Measures Work Product Size Time on Task Defects Resource Availability Schedule Five direct measures Team and team member data Estimated during planning Measured while working Evaluated weekly and when task completed phase completed component completed cycle completed project completed 6

7 Measurement Context Supports Evaluation The five direct measures include a reference to Who team and team member What project, product or component, size measure, process, process phase, and task When project cycle, calendar, time of day The five direct measures have dependencies that can be evaluated to test for internal consistency, for example: Work or tasks are ordered and time on task cannot overlap Defects are found while performing a task and the timestamp and time to fix cannot exceed the start and stop time for the task Product size, time to perform a task, and defects injected and removed are generally correlated 7

8 Performance Evaluation Process Import Data Data Quality Check Baseline Benchmark Produce Certificate Assess submission Store data Add to catalog Consistency Analysis Statistical Analysis Falsified values Project Facts Product Facts Process Facts Projects Organizations 8

9 TSP Data Database 1 Projects 114 Time Log Entries 100,466 Defect Log Entries 10,860 Tasks 73,000 Components 8,412 Database 2 Projects 109 Time Log Entries 103,023 Defect Log Entries 18,408 Tasks 11,499 Components 7,464 9

10 Example Data Quality Tests Check for missing or incorrect value time log entry without a start or stop date defect log entry without a reference to the phase where the defect was discovered Check for internal inconsistency Time log entry outside of the project start and end date, or overlaps with another entry, or violates process sequencing Defect log entry timestamp inconsistent with outside the start and end date for the associated task Check for statistical anomalies and expected distributions Test the distribution of each direct measures to expected values Outlier evaluation Intentional and unintentional incorrect values 10

11 Time Log Leading Digit Analysis The TSP Time Log tracks time spent on tasks in the plan. Time Log Leading Digit Analysis 35.00% Accurate data are important. How many task hours this week? How many task hours to complete a module or component? Tracking in real time improves accuracy. Frequency 30.00% 25.00% 20.00% 15.00% 10.00% 92.3% Expected Actual 5.00% The leading digit analysis compares data to the ideal distribution. 0.00% Leading Digit 11

12 Defect Log and Size Log Analysis Defect Log Leading Digit Analysis Size Log Leading Digit Analysis 40.00% 45.00% 40.00% 35.00% 30.00% 35.00% 30.00% 66.7% 25.00% 90.2% Frequency 25.00% 20.00% 15.00% 10.00% Expected Actual Frequency 20.00% 15.00% 10.00% Expected Actual 5.00% 5.00% 0.00% % Leading Digit (fix time) Leading Digit 12

13 Baseline and Benchmark Data 13

14 Team Size Mean = 8.2 Median = Frequency Team Size

15 Mean Team Member Weekly Task Hours Mean = 10.3 Median = Frequency Mean Team Member Weekly Task Hours

16 Productivity Mean = 10.3 Median = 7.1 Frequency Productivity (LOC/Hr)

17 Plan Vs. Actual Hours for Completed Parts 10,000 9,000 8,000 R² = ,000 Actual Hours for Completed Parts 6,000 5,000 4,000 3,000 2,000 1, ,000 4,000 6,000 8,000 10,000 Plan Hours for Completed Parts 17

18 Plan Task Hours Vs. Actual Task Hours R² = Actual Task Hours Plan Task Hours 18

19 Defect Density Median of Defects Per KLOC DLD Review 2.2 Code Review 5.2 Code Inspection 3.3 Unit Test 3.8 Build/Integration Test 0.7 System Test Defects Per KLOC (Median) 19

20 Summary Management should be part instinct and part fact-based, but the lack of data and facts in software engineering slows learning. Deming urged us to trust data more than our intuition, but can the data be trusted? This presentation has shown a measurement system that can be evaluated to determine the accuracy of project data. has built-in checks and balances to guard against intentional and unintentional errors. produces data that supports estimating, planning, tracking, baselines, and benchmarking. 20

21 Questions? 21

22 Presenter Information Presenter James Over Technical Director Software Solutions Division Telephone: U.S. Mail Software Engineering Institute Customer Relations 4500 Fifth Avenue Pittsburgh, PA USA Web Customer Relations Telephone: SEI Phone: SEI Fax:

Applying Software Quality Models to Software Security

Applying Software Quality Models to Software Security Applying Software Quality Models to Software Security Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 Carol Woody, Ph.D. April 21, 2015 Copyright 2015 Carnegie Mellon University

More information

Moving Target Reference Implementation

Moving Target Reference Implementation CYBER SECURITY DIVISION 2014 R&D SHOWCASE AND TECHNICAL WORKSHOP Moving Target Reference Implementation Software Engineering Institute, Carnegie Mellon University Andrew O. Mellinger December 17, 2014

More information

Merging Network Configuration and Network Traffic Data in ISP-Level Analyses

Merging Network Configuration and Network Traffic Data in ISP-Level Analyses Merging Network Configuration and Network Traffic Data in ISP-Level Analyses Timothy J. Shimeall, Ph.D. Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 Presentation Title

More information

Contracting Officer s Representative (COR) Interactive SharePoint Wiki

Contracting Officer s Representative (COR) Interactive SharePoint Wiki Contracting Officer s Representative (COR) Interactive SharePoint Wiki James Smith Andy Boyd Software Solutions Conference 2015 November 16 18, 2015 Copyright 2015 Carnegie Mellon University This material

More information

Resolving Chaos Arising from Agile Software Development

Resolving Chaos Arising from Agile Software Development Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 523 Author Date High Level Alternatives Approach. Blame the Agile development process, fire the folks who are controlling it and

More information

Data Management Maturity (DMM) Model Update

Data Management Maturity (DMM) Model Update Data Management Maturity (DMM) Model Update Rawdon Young November 2012 Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 Contents / Agenda The DMM SEI Observations on Core

More information

2012 CyberSecurity Watch Survey

2012 CyberSecurity Watch Survey 2012 CyberSecurity Watch Survey Unknown How 24 % Bad is the Insider Threat? 51% 2007-2013 Carnegie Mellon University 2012 Carnegie Mellon University NO WARRANTY THIS MATERIAL OF CARNEGIE MELLON UNIVERSITY

More information

Model Checking Distributed Software

Model Checking Distributed Software Model Checking Distributed Software Sagar Chaki September 19, 2014 Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 Model Checking and Me 1997 : Ed visits IIT Kharagpur Just

More information

Exploring the Interactions Between Network Data Analysis and Security Information/Event Management

Exploring the Interactions Between Network Data Analysis and Security Information/Event Management Exploring the Interactions Between Network Data Analysis and Security Information/Event Management Timothy J. Shimeall CERT Network Situational Awareness (NetSA) Group January 2011 2011 Carnegie Mellon

More information

VoIP in Flow A Beginning

VoIP in Flow A Beginning VoIP in Flow A Beginning Nathan Dell CERT/NetSA 2013 Carnegie Mellon University Legal Copyright 2013 Carnegie Mellon University This material is based upon work funded and supported by the Department of

More information

Electricity Subsector Cybersecurity Capability Maturity Model (ES-C2M2) (Case Study) James Stevens Senior Member, Technical Staff - CERT Division

Electricity Subsector Cybersecurity Capability Maturity Model (ES-C2M2) (Case Study) James Stevens Senior Member, Technical Staff - CERT Division Electricity Subsector Cybersecurity Capability Maturity Model (ES-C2M2) (Case Study) James Stevens Senior Member, Technical Staff - CERT Division James Stevens is a senior member of the technical staff

More information

Elasticsearch, Logstash, and Kibana (ELK)

Elasticsearch, Logstash, and Kibana (ELK) Elasticsearch, Logstash, and Kibana (ELK) Dwight Beaver dsbeaver@cert.org Sean Hutchison shutchison@cert.org January 2015 2014 Carnegie Mellon University This material is based upon work funded and supported

More information

Assurance Cases for Design Analysis of Complex System of Systems Software

Assurance Cases for Design Analysis of Complex System of Systems Software Assurance Cases for Design Analysis of Complex System of Systems Software Presented at AIAA Infotech@Aerospace Conference Software Assurance Session 8 April 2009 Stephen Blanchette, Jr. Problem: SoS are

More information

A Systematic Method for Big Data Technology Selection

A Systematic Method for Big Data Technology Selection A Systematic Method for Big Data Technology Selection John Klein Software Solutions Conference 2015 November 16 18, 2015 Copyright 2015 Carnegie Mellon University This material is based upon work funded

More information

Cyber Intelligence Workforce

Cyber Intelligence Workforce Cyber Intelligence Workforce Troy Townsend Melissa Kasan Ludwick September 17, 2013 Agenda Project Background Research Methodology Findings Training and Education Project Findings Workshop Results Objectives

More information

Kurt Wallnau Senior Member of Technical Staff

Kurt Wallnau Senior Member of Technical Staff Engineering Realistic Synthetic Insider Threat (Cyber-Social) Test Data Kurt Wallnau Senior Member of Technical Staff Dr. Kurt Wallnau joined the SEI in 1993. He joined CERT Science of Cyber-Security (SoCS)

More information

Common Testing Problems: Pitfalls to Prevent and Mitigate

Common Testing Problems: Pitfalls to Prevent and Mitigate : Pitfalls to Prevent and Mitigate AIAA Case Conference 12 September 2012 Donald Firesmith Software Engineering Institute (SEI) Carnegie Mellon University Pittsburgh, PA 15213 Clarification and Caveat

More information

Architecture Patterns for Mobile Systems in Resource- Constrained Environments

Architecture Patterns for Mobile Systems in Resource- Constrained Environments Architecture Patterns for Mobile Systems in Resource- Constrained Environments Grace A. Lewis Advanced Mobile Systems Initiative May 2, 2013 Minneapolis, MN USA Motivation First responders, soldiers and

More information

Network Flow Analysis in Information Security Strategy

Network Flow Analysis in Information Security Strategy Network Flow Analysis in Information Security Strategy Timothy J. Shimeall, Ph.D. Situational Awareness Team January, 2015 2015 Carnegie Mellon University Copyright 2014 Carnegie Mellon University This

More information

Agile Development and Software Architecture: Understanding Scale and Risk

Agile Development and Software Architecture: Understanding Scale and Risk Agile Development and Software Architecture: Understanding Scale and Risk Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 Robert L. Nord SSTC, April 2012 In collaboration

More information

Penetration Testing Tools

Penetration Testing Tools Penetration Testing Tools Ken van Wyk January 2007 ABSTRACT: This article provides a primer on the most commonly used tools for traditional penetration testing. (A related article provides an overview

More information

The Personal Software Process SM (PSP SM )

The Personal Software Process SM (PSP SM ) The Personal Software Process SM (PSP SM ) Watts S. Humphrey November 2000 TECHNICAL REPORT CMU/SEI-2000-TR-022 ESC-TR-2000-022 Pittsburgh, PA 15213-3890 The Personal Software Process SM (PSP SM ) CMU/SEI-2000-TR-022

More information

Supply-Chain Risk Management Framework

Supply-Chain Risk Management Framework Supply-Chain Risk Management Framework Carol Woody March 2010 Scope of SEI Work Context Significantly reduce the risk (any where in the supply chain) that an unauthorized party can change the behavior

More information

Understanding High Maturity Organizations

Understanding High Maturity Organizations Understanding High Maturity Organizations Donna K. Dunaway, Charles V. Weber, Mark C. Paulk, Will Hayes, and Mary Beth Chrissis Carnegie Mellon University Pittsburgh, PA 15213-3890 Capability Maturity

More information

Monitoring Trends in Network Flow for Situational Awareness

Monitoring Trends in Network Flow for Situational Awareness Monitoring Trends in Network Flow for Situational Awareness SEI CERT NetSA 2011 Carnegie Mellon University NO WARRANTY THIS MATERIAL OF CARNEGIE MELLON UNIVERSITY AND ITS SOFTWARE ENGINEERING INSTITUTE

More information

UFO: Verification with Interpolants and Abstract Interpretation

UFO: Verification with Interpolants and Abstract Interpretation : Verification with Interpolants and Abstract Interpretation and Sagar Chaki Software Engineering Institute Carnegie Mellon University Aws Albarghouthi, Yi i and Marsha Chechik University of Toronto A

More information

CMMI for SCAMPI SM Class A Appraisal Results 2011 End-Year Update

CMMI for SCAMPI SM Class A Appraisal Results 2011 End-Year Update CMMI for SCAMPI SM Class A 2011 End-Year Update Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 1 Outline Introduction Current Status Community Trends Organizational Trends

More information

Overview. CMU/SEI Cyber Innovation Center. Dynamic On-Demand High-Performance Computing System. KVM and Hypervisor Security.

Overview. CMU/SEI Cyber Innovation Center. Dynamic On-Demand High-Performance Computing System. KVM and Hypervisor Security. KVM and Hypervisor Security David Shepard and Matt Gaston CMU/SEI Cyber Innovation Center February 2012 2012 by Carnegie Mellon University. Published SEI PROPRIETARY INFORMATION. Distribution: Director

More information

Getting Started with Service- Oriented Architecture (SOA) Terminology

Getting Started with Service- Oriented Architecture (SOA) Terminology Getting Started with - Oriented Architecture (SOA) Terminology Grace Lewis September 2010 -Oriented Architecture (SOA) is a way of designing, developing, deploying, and managing systems it is neither a

More information

Experiences in Migrations of Legacy Systems

Experiences in Migrations of Legacy Systems Experiences in Migrations of Legacy Systems Bill Wood, Mike Gagliardi, and Phil Bianco Software Solutions Conference 2015 November 16 18, 2015 Copyright 2015 Carnegie Mellon University This material is

More information

Extending AADL for Security Design Assurance of the Internet of Things

Extending AADL for Security Design Assurance of the Internet of Things Extending AADL for Security Design Assurance of the Internet of Things Presented by Rick Kazman, PhD Team: Carol Woody (PI), Rick Kazman, Robert Ellison, John Hudak, Allen Householder Software Engineering

More information

Software Architecture for Big Data Systems. Ian Gorton Senior Member of the Technical Staff - Architecture Practices

Software Architecture for Big Data Systems. Ian Gorton Senior Member of the Technical Staff - Architecture Practices Software Architecture for Big Data Systems Ian Gorton Senior Member of the Technical Staff - Architecture Practices Ian Gorton is investigating issues related to software architecture at scale. This includes

More information

Architectural Implications of Cloud Computing

Architectural Implications of Cloud Computing Architectural Implications of Cloud Computing Grace Lewis Research, Technology and Systems Solutions (RTSS) Program Lewis is a senior member of the technical staff at the SEI in the Research, Technology,

More information

Network Monitoring for Cyber Security

Network Monitoring for Cyber Security Network Monitoring for Cyber Security Paul Krystosek, PhD CERT Network Situational Awareness 2006 Carnegie Mellon University What s Coming Up The scope of network monitoring Cast of characters Descriptions

More information

Copyright 2014 Carnegie Mellon University The Cyber Resilience Review is based on the Cyber Resilience Evaluation Method and the CERT Resilience

Copyright 2014 Carnegie Mellon University The Cyber Resilience Review is based on the Cyber Resilience Evaluation Method and the CERT Resilience Copyright 2014 Carnegie Mellon University The Cyber Resilience Review is based on the Cyber Resilience Evaluation Method and the CERT Resilience Management Model (CERT-RMM), both developed at Carnegie

More information

Buyer Beware: How To Be a Better Consumer of Security Maturity Models

Buyer Beware: How To Be a Better Consumer of Security Maturity Models Buyer Beware: How To Be a Better Consumer of Security Maturity Models SESSION ID: GRC-R01 Julia Allen Software Engineering Institute Carnegie Mellon University jha@sei.cmu.edu Nader Mehravari Software

More information

Risk Management Framework

Risk Management Framework Risk Management Framework Christopher J. Alberts Audrey J. Dorofee August 2010 TECHNICAL REPORT CMU/SEI-2010-TR-017 ESC-TR-2010-017 Acquisition Support Program Unlimited distribution subject to the copyright.

More information

Using Domain Name Registrant Information To Identify Malicious Domains

Using Domain Name Registrant Information To Identify Malicious Domains Using Domain Name Registrant Information To Identify Malicious Domains Mark Langston Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 Do Bad Actors Use Fake Addresses? Copyright

More information

Abuse of CPE Devices and Recommended Fixes

Abuse of CPE Devices and Recommended Fixes Abuse of CPE Devices and Recommended Fixes Dr. Paul Vixie (Farsight Security, Inc.) Chris Hallenbeck (US-CERT, DHS) Jonathan Spring (CERT/CC, Carnegie Mellon) August 7, 2014 Black Hat USA 2014 2014 Carnegie

More information

Network Analysis with isilk

Network Analysis with isilk Network Analysis with isilk Presented at FloCon 2011 Ron Bandes CERT Network Situational Awareness (NetSA) Group 2011 Carnegie Mellon University 2011 Carnegie Mellon University NO WARRANTY THIS MATERIAL

More information

Automated Provisioning of Cloud and Cloudlet Applications

Automated Provisioning of Cloud and Cloudlet Applications Automated Provisioning of Cloud and Cloudlet Applications Secure and Assured Mobile Computing Components Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 Jeff Boleng, PhD

More information

Software Security Engineering: A Guide for Project Managers

Software Security Engineering: A Guide for Project Managers Software Security Engineering: A Guide for Project Managers Gary McGraw Julia H. Allen Nancy Mead Robert J. Ellison Sean Barnum May 2013 ABSTRACT: Software is ubiquitous. Many of the products, services,

More information

emontage: An Architecture for Rapid Integration of Situational Awareness Data at the Edge

emontage: An Architecture for Rapid Integration of Situational Awareness Data at the Edge emontage: An Architecture for Rapid Integration of Situational Awareness Data at the Edge Soumya Simanta Gene Cahill Ed Morris Motivation Situational Awareness First responders and others operating in

More information

Assurance in Service-Oriented Environments

Assurance in Service-Oriented Environments Assurance in Service-Oriented Environments Soumya Simanta Research, Technology, and System Solutions (RTSS) Program Software Engineering Institute Carnegie Mellon University Pittsburgh 15232 28 th October,

More information

$100 SiLK Network Flow Sensor

$100 SiLK Network Flow Sensor $100 SiLK Network Flow Sensor Ron Bandes John Badertscher Dwight Beaver 1 Copyright 2014 Carnegie Mellon University This material is based upon work funded and supported by the Department of Defense under

More information

A Study of Systems Engineering Effectiveness. Building a Business Case for Systems Engineering

A Study of Systems Engineering Effectiveness. Building a Business Case for Systems Engineering Building a Business Case for Systems Engineering NO WARRANTY THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN AS-IS" BASIS. CARNEGIE MELLON UNIVERSITY MAKES

More information

CMMI: What do we need to do in Requirements Management & Engineering?

CMMI: What do we need to do in Requirements Management & Engineering? Colin Hood Page 1 of 11 : What do we need to do in Requirements Management & Engineering? Colin Hood HOOD Group February 2003 : What do we need to do in Requirements Management & Engineering?... 1 1 Abstract...

More information

The CERT Top 10 List for Winning the Battle Against Insider Threats

The CERT Top 10 List for Winning the Battle Against Insider Threats The CERT Top 10 List for Winning the Battle Against Insider Threats Dawn Cappelli CERT Insider Threat Center Software Engineering Institute Carnegie Mellon University Session ID: STAR-203 Session Classification:

More information

Sustaining Operational Resiliency: A Process Improvement Approach to Security Management

Sustaining Operational Resiliency: A Process Improvement Approach to Security Management Sustaining Operational Resiliency: A Process Improvement Approach to Security Management Author Richard A. Caralli Principle Contributors James F. Stevens Charles M. Wallen, Financial Services Technology

More information

Building Resilient Systems: The Secure Software Development Lifecycle

Building Resilient Systems: The Secure Software Development Lifecycle Building Resilient Systems: The Secure Software Development Lifecycle Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213, PhD Technical Director, CERT mssherman@sei.cmu.edu

More information

Department of Homeland Security Cyber Resilience Review (Case Study) Matthew Butkovic Technical Manager - Cybersecurity Assurance, CERT Division

Department of Homeland Security Cyber Resilience Review (Case Study) Matthew Butkovic Technical Manager - Cybersecurity Assurance, CERT Division Department of Homeland Security Cyber Resilience Review (Case Study) Matthew Butkovic Technical Manager - Cybersecurity Assurance, CERT Division Matthew Butkovic is a Technical Manager Cybersecurity Assurance

More information

The Capability Maturity Model for Software, Version 1.1

The Capability Maturity Model for Software, Version 1.1 The Capability Maturity Model for Software, Version 1.1 Mark C. Paulk xxx 1998 Carnegie Mellon University Pittsburgh, PA 15213-3890 Sponsored by the U.S. Department of Defense. 1997 by Carnegie Mellon

More information

CMMI Version 1.2. SCAMPI SM A Appraisal Method Changes

CMMI Version 1.2. SCAMPI SM A Appraisal Method Changes Pittsburgh, PA 15213-3890 CMMI Version 1.2 SCAMPI SM A Appraisal Method Changes SM CMM Integration, IDEAL, and SCAMPI are service marks of Carnegie Mellon University. Capability Maturity Model, Capability

More information

Operationally Critical Threat, Asset, and Vulnerability Evaluation SM (OCTAVE SM ) Framework, Version 1.0

Operationally Critical Threat, Asset, and Vulnerability Evaluation SM (OCTAVE SM ) Framework, Version 1.0 Operationally Critical Threat, Asset, and Vulnerability Evaluation SM (OCTAVE SM ) Framework, Version 1.0 Christopher J. Alberts Sandra G. Behrens Richard D. Pethia William R. Wilson June 1999 TECHNICAL

More information

Steve Masters (SEI) SEPG North America March 2011. 2011 Carnegie Mellon University

Steve Masters (SEI) SEPG North America March 2011. 2011 Carnegie Mellon University Using Organizational Business Objectives to Guide a Process Improvement Program Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 (SEI) SEPG North America March 2011 Agenda

More information

Introduction to the OCTAVE Approach

Introduction to the OCTAVE Approach Introduction to the OCTAVE Approach Christopher Alberts Audrey Dorofee James Stevens Carol Woody August 2003 Pittsburgh, PA 15213-3890 Introduction to the OCTAVE Approach Christopher Alberts Audree Dorofee

More information

Configuring and Monitoring SharePoint Servers

Configuring and Monitoring SharePoint Servers Configuring and Monitoring SharePoint Servers eg Enterprise v5.6 Restricted Rights Legend The information contained in this document is confidential and subject to change without notice. No part of this

More information

Building a Body of Knowledge for ICT Supply Chain Risk Management

Building a Body of Knowledge for ICT Supply Chain Risk Management Building a Body of Knowledge for ICT Supply Chain Risk Management Dan Shoemaker Nancy Mead May 2013 ABSTRACT: This paper proposes a set of Supply Chain Risk Management (SCRM) activities and practices for

More information

CLOUDFOUNDRY.ORG FOUNDATION SOFTWARE GRANT AND CORPORATE CONTRIBUTOR LICENSE AGREEMENT ( AGREEMENT )

CLOUDFOUNDRY.ORG FOUNDATION SOFTWARE GRANT AND CORPORATE CONTRIBUTOR LICENSE AGREEMENT ( AGREEMENT ) CLOUDFOUNDRY.ORG FOUNDATION SOFTWARE GRANT AND CORPORATE CONTRIBUTOR LICENSE AGREEMENT ( AGREEMENT ) Thank you for your interest in the CloudFoundry.org Foundation, Inc. (the Foundation ). In order to

More information

Software Assurance vs. Security Compliance: Why is Compliance Not Enough? Carol Woody, Ph.D.

Software Assurance vs. Security Compliance: Why is Compliance Not Enough? Carol Woody, Ph.D. Software Assurance vs. Security Compliance: Why is Compliance Not Enough? Carol Woody, Ph.D. Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 2012 Carnegie Mellon University

More information

The Key to Successful Monitoring for Detection of Insider Attacks

The Key to Successful Monitoring for Detection of Insider Attacks The Key to Successful Monitoring for Detection of Insider Attacks Dawn M. Cappelli Randall F. Trzeciak Robert Floodeen Software Engineering Institute CERT Program Session ID: GRC-302 Session Classification:

More information

The Team Software Process SM (TSP SM )

The Team Software Process SM (TSP SM ) The Team Software Process SM (TSP SM ) Watts S. Humphrey November 2000 TECHNICAL REPORT CMU/SEI-2000-TR-023 ESC-TR-2000-023 Pittsburgh, PA 15213-3890 The Team Software Process SM (TSP SM ) CMU/SEI-2000-TR-023

More information

CMMI for Acquisition, Version 1.3

CMMI for Acquisition, Version 1.3 CMMI for Acquisition, Version 1.3 CMMI-ACQ, V1.3 CMMI Product Team Improving processes for acquiring better products and services November 2010 TECHNICAL REPORT CMU/SEI-2010-TR-032 ESC-TR-2010-032 Software

More information

CERT Resilience Management Model (CERT -RMM) V1.1: NIST Special Publication 800-66 Crosswalk

CERT Resilience Management Model (CERT -RMM) V1.1: NIST Special Publication 800-66 Crosswalk CERT Resilience Management Model (CERT -RMM) V1.1: NIST Special Publication 800-66 Crosswalk Lisa R. Young, Software Engineering Institute Ma-Nyahn Kromah, SunGard Availability Services October 2013 TECHNICAL

More information

SOA for Healthcare: Promises and Pitfalls

SOA for Healthcare: Promises and Pitfalls SOA for Healthcare: Promises and Pitfalls Dennis B. Smith dbs@sei.cmu.edu SOA in Health Care Conference: Value in a Time of Change Chicago, IL USA June 3, 2009 Agenda Healthcare IT Challenges SOA: The

More information

An Application of an Iterative Approach to DoD Software Migration Planning

An Application of an Iterative Approach to DoD Software Migration Planning An Application of an Iterative Approach to DoD Software Migration Planning John Bergey Liam O Brien Dennis Smith September 2002 Product Line Practice Initiative Unlimited distribution subject to the copyright.

More information

Enable File and Folder Auditing

Enable File and Folder Auditing Enable File and Folder Auditing Publication Date: Feb 9, 2015 EventTracker 8815 Centre Park Drive Columbia MD 21045 www.eventtracker.com About this Guide: This guide will help the end user to enable auditing

More information

The Personal Software Process (PSP) Tutorial

The Personal Software Process (PSP) Tutorial The Personal Software Process (PSP) Tutorial Watts Humphrey / Jim Over Speaker: Daniel M. Roy (STPP, visiting scientist SEI) Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213

More information

Arcade Game Maker Pedagogical Product Line: Marketing and Product Plan

Arcade Game Maker Pedagogical Product Line: Marketing and Product Plan Arcade Game Maker Pedagogical Product Line: Marketing and Product Plan Arcade Game Team July 2003 Unlimited distribution subject to the copyright. This work is sponsored by the U.S. Department of Defense.

More information

CRR Supplemental Resource Guide. Volume 5. Incident Management. Version 1.1

CRR Supplemental Resource Guide. Volume 5. Incident Management. Version 1.1 CRR Supplemental Resource Guide Volume 5 Incident Management Version 1.1 Copyright 2016 Carnegie Mellon University This material is based upon work funded and supported by Department of Homeland Security

More information

Edge Analytics: Analysis of Social Media to Support Tactical Users

Edge Analytics: Analysis of Social Media to Support Tactical Users Edge Analytics: Analysis of Social Media to Support Tactical Users Bill Anderson & Keegan Williams Software Solutions Conference 2015 November 16 18, 2015 Copyright 2015 Carnegie Mellon University This

More information

Structuring the Chief Information Security Officer Organization

Structuring the Chief Information Security Officer Organization Structuring the Chief Information Security Officer Organization December 1, 2015 Julia Allen Nader Mehravari Cyber Risk and Resilience Management Team CERT Division Software Engineering Institute Carnegie

More information

CMMI for Development, Version 1.3

CMMI for Development, Version 1.3 Carnegie Mellon University Research Showcase @ CMU Software Engineering Institute 11-2010 CMMI for Development, Version 1.3 CMMI Product Team Follow this and additional works at: http://repository.cmu.edu/sei

More information

Pipeliner CRM Phaenomena Guide Sales Pipeline Management. 2015 Pipelinersales Inc. www.pipelinersales.com

Pipeliner CRM Phaenomena Guide Sales Pipeline Management. 2015 Pipelinersales Inc. www.pipelinersales.com Sales Pipeline Management 2015 Pipelinersales Inc. www.pipelinersales.com Sales Pipeline Management Learn how to manage sales opportunities with Pipeliner Sales CRM Application. CONTENT 1. Configuring

More information

Insider Threat Control: Using a SIEM signature to detect potential precursors to IT Sabotage. CERT Insider Threat Center

Insider Threat Control: Using a SIEM signature to detect potential precursors to IT Sabotage. CERT Insider Threat Center Insider Threat Control: Using a SIEM signature to detect potential precursors to IT Sabotage CERT Insider Threat Center April 2011 NOTICE: THIS TECHNICAL DATA IS PROVIDED PURSUANT TO GOVERNMENT CONTRACT

More information

Configuring and Monitoring Virtual Server 2005

Configuring and Monitoring Virtual Server 2005 Configuring and Monitoring Virtual Server 2005 eg Enterprise v5.6 Restricted Rights Legend The information contained in this document is confidential and subject to change without notice. No part of this

More information

CERT Virtual Flow Collection and Analysis

CERT Virtual Flow Collection and Analysis CERT Virtual Flow Collection and Analysis For Training and Simulation George Warnagiris 2011 Carnegie Mellon University Report Documentation Page Form Approved OMB No. 0704-0188 Public reporting burden

More information

CMMI for Development, Version 1.3

CMMI for Development, Version 1.3 CMMI for Development, Version 1.3 CMMI-DEV, V1.3 CMMI Product Team Improving processes for developing better products and services November 2010 TECHNICAL REPORT CMU/SEI-2010-TR-033 ESC-TR-2010-033 Software

More information

Software Assurance Competency Model

Software Assurance Competency Model Software Assurance Competency Model Thomas Hilburn, Embry-Riddle Aeronautical University Mark Ardis, Stevens Institute of Technology Glenn Johnson, (ISC) 2 Andrew Kornecki, Embry-Riddle Aeronautical University

More information

T-Check in System-of-Systems Technologies: Cloud Computing

T-Check in System-of-Systems Technologies: Cloud Computing T-Check in System-of-Systems Technologies: Cloud Computing Harrison D. Strowd Grace A. Lewis September 2010 TECHNICAL NOTE CMU/SEI-2010-TN-009 Research, Technology, and System Solutions (RTSS) Program

More information

Service Measurement Index Framework Version 2.1

Service Measurement Index Framework Version 2.1 Service Measurement Index Framework Version 2.1 July 2014 CSMIC Carnegie Mellon University Silicon Valley Moffett Field, CA USA Introducing the Service Measurement Index (SMI) The Service Measurement Index

More information

Software Solutions for the Telecommunications Industry

Software Solutions for the Telecommunications Industry Software Solutions for the Telecommunications Industry A Unique Framework NCT software platform the building block Framework Architecture Framework Architecture NCT - Fault Management Fault Registration

More information

CERT Resilience Management Model (RMM) v1.1: Code of Practice Crosswalk Commercial Version 1.1

CERT Resilience Management Model (RMM) v1.1: Code of Practice Crosswalk Commercial Version 1.1 CERT Resilience (RMM) : Code of Practice Crosswalk Commercial Version 1.1 Kevin G. Partridge Lisa R. Young October 2011 TECHNICAL NOTE CMU/SEI-2011-TN-012 CERT Program Unlimited distribution subject to

More information

Capability Maturity Model Integration (CMMI ) Overview

Capability Maturity Model Integration (CMMI ) Overview Pittsburgh, PA 15213-3890 Capability Maturity Model Integration ( ) Overview SM CMM Integration, SCAMPI, SCAMPI Lead Appraiser, and SEI are service marks of Carnegie Mellon University., Capability Maturity

More information

Trends and New Directions in Software Architecture

Trends and New Directions in Software Architecture Trends and New Directions in Software Architecture Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 Chief Scientist, Software Solutions Division SEI Fellow Copyright 2015

More information

Advance the state of the practice. Exercise your skills with other top software engineering professionals.

Advance the state of the practice. Exercise your skills with other top software engineering professionals. Carnegie Mellon Advance the state of the practice. AFFILIATE PROGRAM Join research projects on the leading edge. Exercise your skills with other top software engineering professionals. OUR MISSION is to

More information

QUADRO POWER GUIDELINES

QUADRO POWER GUIDELINES QUADRO POWER GUIDELINES DA-07261-001_v03 July 2015 Application Note DOCUMENT CHANGE HISTORY DA-07261-001_v03 Version Date Authors Description of Change 01 June 6, 2014 VL, SM Initial Release 02 June 2,

More information

DoD Software Migration Planning

DoD Software Migration Planning DoD Software Migration Planning John Bergey Liam O Brien Dennis Smith August 2001 Product Line Practice Initiative Technical Note CMU/SEI-2001-TN-012 Unlimited distribution subject to the copyright. The

More information

Allscripts Professional EHR

Allscripts Professional EHR for Allscripts Professional EHR Remote Desktop (RDP) Setup and Installation Guide April, 2015 ii Copyright 2010 2015 NoteSwift, Inc. All Rights Reserved. NoteSwift is a trademark of NoteSwift, Inc., in

More information

Solutions for Microsoft Project Server and Microsoft Dynamics GP Timesheet Integration

Solutions for Microsoft Project Server and Microsoft Dynamics GP Timesheet Integration Solutions for Microsoft Project Server and Microsoft Dynamics GP Timesheet Integration White Paper Date: May 2011 http://www.daxdata.com/project-server.php 1 Table of Contents Introduction... 3 The Solution

More information

IBM FlashSystem. SNMP Guide

IBM FlashSystem. SNMP Guide IBM FlashSystem SNMP Guide IBM FlashSystem SNMP Guide Note Before using this information and the product it supports, read the information in Notices on page 9. This edition applies to IBM FlashSystem

More information

Data Rights for Proprietary Software Used in DoD Programs

Data Rights for Proprietary Software Used in DoD Programs Data Rights for Proprietary Software Used in DoD Programs Julie Cohen Bonnie Troup (The Aerospace Corporation) Henry Ouyang (The Aerospace Corporation) April 2010 TECHNICAL NOTE CMU/SEI-2010-TN-014 Acquisition

More information

Deriving Software Security Measures from Information Security Standards of Practice

Deriving Software Security Measures from Information Security Standards of Practice Deriving Software Measures from Standards of Practice Julia Allen Christopher Alberts Robert Stoddard February 2012 2012 Carnegie Mellon University Copyright 2012 Carnegie Mellon University. This material

More information

Add-On Module Advanced Recurring Charges

Add-On Module Advanced Recurring Charges 0 Add-On Module Advanced Recurring Charges LICENCE 1 The Revelation Package (herein after referred to as "the Package") is protected by both South African Copyright law and International Treaty Provisions.

More information

The Software Engineering. Today and in the Future. Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213

The Software Engineering. Today and in the Future. Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 The Software Engineering Institute t (SEI): Today and in the Future Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 Paul D. Nielsen 1 May 2008 Software Engineering Institute

More information

Quest InTrust for Active Directory. Product Overview Version 2.5

Quest InTrust for Active Directory. Product Overview Version 2.5 Quest InTrust for Active Directory Product Overview Version 2.5 Copyright Quest Software, Inc. 2006. All rights reserved. This guide contains proprietary information, which is protected by copyright. The

More information

Comments on Software Quality by Watts S. Humphrey Fellow, Software Engineering Institute Carnegie Mellon University Pittsburgh, PA

Comments on Software Quality by Watts S. Humphrey Fellow, Software Engineering Institute Carnegie Mellon University Pittsburgh, PA Comments on Software Quality by Watts S. Humphrey Fellow, Software Engineering Institute Carnegie Mellon University Pittsburgh, PA Summary This paper reviews the software industry s current approach to

More information

BMC Remedy Action Request System 7.0 Open Source License Agreements

BMC Remedy Action Request System 7.0 Open Source License Agreements March 2006 BMC Remedy Action Request System 7.0 Open Source License Agreements Copyright 1991 2005 BMC Software, Inc. All rights reserved. BMC, the BMC logo, all other BMC product or service names, BMC

More information

Configuring and Monitoring Event Logs

Configuring and Monitoring Event Logs Configuring and Monitoring Event Logs eg Enterprise v5.6 Restricted Rights Legend The information contained in this document is confidential and subject to change without notice. No part of this document

More information

DUAL MONITOR DRIVER AND VBIOS UPDATE

DUAL MONITOR DRIVER AND VBIOS UPDATE DUAL MONITOR DRIVER AND VBIOS UPDATE RN-07046-001_v01 September 2013 Release Notes DOCUMENT CHANGE HISTORY RN-07046-001_v01 Version Date Authors Description of Change 01 September 30, 2013 MD, SM Initial

More information