Slide 1 Supply Chain Management of Open Source Software used within Software Development Lifecycle Author: Roderick Koch Co-Author: Kym Watkin-Statham http://www.sentar.com/ Secure Sw. Dev. Lifecycle with a focus on OSS.
Slide 2 What is Open Source Software (OSS)? DoD CIO Memo Clarifying Guidance Regarding OSS, 16 Oct 2009, To effectively achieve its missions, the Department of Defense must develop and update its software-based capabilities faster than ever, to anticipate new threats and respond to continuously changing requirements. The use of Open Source Software (OSS) can provide advantages in this regard. Open Source Software is software for which the human-readable source code is available for use, study, reuse, modification, enhancement, and redistribution by the users of that software. In other words, OSS is software for which the source code is open. http://dodcio.defense.gov/todayincio/freeopensourcesoftware.aspx 2 DoD see s the advantages of using OSS and supports its use. OSS is human readable source code available of study, modification, enhancement and redistribution.
Slide 3 Commercial or OSS Which is the greater risk? Fallacy that more eyes make the code safer Developers like to build, not audit and fix others developer s code. Does your Commercial Software Vendor or Open Source Foundation have a process for secure software development and evaluation testing? Do they provide visibility into the process and results? Foreign Developer participation means that it s foreign developed software But.US companies also outsource development to foreign developers. 3
Slide 4 Growth of OSS usage http://docs.ismgcorp.com/files/external/wp_fsisac_third_party_software_security_working_group.pdf 4 Demand for Open Source components is skyrocketing. An 800% increase over the last five years. Today typically 80-90% of a software project is OSS components with 10-20 % being custom code providing the glue. Could you provide your customer a list of all OSS used on your project? What you did to evaluate it for licensing and vulnerabilities? How did it s use get approved?
Slide 5 Widespread Vulnerabilities in OSS Organizations regularly consume outdated, flawed, or insecure components http://img.en25.com/web/sonatypeinc/%7b39fd4cdf-c16f-4c4f-9d3f-f7302f176941%7d_clm_whitepaper.pdf 5 Organizations regularly consume outdated, flawed, or insecure components, which can introduce significant risk.
Slide 6 Complexity OSS Component complexity exacerbates the problem. Organizations lack actionable security, quality and licensing information. http://img.en25.com/web/sonatypeinc/%7b39fd4cdf-c16f-4c4f-9d3f-f7302f176941%7d_clm_whitepaper.pdf 6 Components are enormously complex; each one is made up of hundreds of sub-assemblies (e.g. class files). Class files are commonly shared among components. It is difficult and time consuming for developers to research and determine security, quality and licensing characteristics for all of the components they use to assemble their applications. To do this for direct dependencies is hard enough; to extend this research to all component dependencies is beyond reason. Even if research is conducted, it is difficult to take action because it is not integrated directly in the tools that developers use and problems are found much later in the lifecycle. Given the pressure to deliver applications quickly, developers are forced to take a chance when they select components exposing the organization to risk.
Slide 7 Control Process for OSS in SDLC http://img.en25.com/web/sonatypeinc/%7b39fd4cdf-c16f-4c4f-9d3f-f7302f176941%7d_clm_whitepaper.pdf 7 OSS Projects innovate rapidly and release frequently. However, there is no update notification infrastructure for open source components.
Slide 8 Software Security Development Lifecycle (SSDL) Software assurance begins with code quality and evidence of that quality. You can assume a software defect found during the development of a product may require $1 to remedy. If the defect escapes the development phase and enters the independent testing phase the cost will be approximately $100 to remedy. If the defect escapes the independent testing phase and makes it into production the cost will be approximately $1,000 to remedy. If sensitive data is lost or attackers make the software do things they are not supposed to do, through exploitation of a known software weakness, the costs of this defect may exceed many thousands of dollars to repair if repair is even possible and the impact could go well beyond anything that money can represent. http://www.bsimm.com/ 8 BSIMM (pronounced bee simm ) = Building Security In Maturity Model, v = vendor, a subset of the whole. The BSIMM is designed to help you understand, measure, and plan a software security initiative. The BSIMM is a study of real-world software security initiatives organized so that you can determine where you stand with your software security initiative and how to evolve your efforts over time.
Slide 9 How to judge the good from the bad OSS Project or Foundation? Approval from independent third party evaluator Does the Project or Foundation have; Good commercial company backing? Popularity in open source communities? Committer activity levels? Good Documentation and support system? Good Change Management and Intellectual Property (IP) controls? Good bug tracking system, historic time to fix vulnerabilities, current vulnerabilities? 9 Approval from independent evaluator like Black Duck, Veracode, Coverity, Sonatype, all offer some information of OSS for free, and more for a fee.
Slide 10 How to judge the good from the bad OSS? Black Duck http://www.ohloh.net/ 10
Slide 11 How to judge the good from the bad OSS? Black Duck http://www.ohloh.net/ 11
Slide 12 OSS vulnerability tracking OSS Vulnerability Tracking http://cve.mitre.org/cve/ (Common Vulnerability and Exposure) http://web.nvd.nist.gov/view/vuln/search (National Vulnerability DB http://osvdb.org/ (Open Source Vulnerability DB) http://www.us-cert.gov/ (US Computer Emergancy Readiness Team) https://www.cybercom.mil/default.aspx (DoD CAC required) https://www.google.com/ (Open Search for Vulnerabilities) Most OSS foundations and project have a bug tracking system No standard for reporting & responding Library version control and dependencies can get sticky The vulnerability is fixed in the new version, but it breaks other functionality, therefore, you still need to run the older vulnerable library. 12
Slide 13 Component Lifecycle Management (CLM) A new approach in the market is Component Lifecycle Management (CLM) which offers the ability to enforce policies in the development process. The benefits of this approach include: Provides a central facility for active risk assessment and management across development environments & teams Informs and governs the software supply chain by validating, authenticating, securely delivering, and monitoring components security popularity and licensing information throughout the development lifecycle. It offers developer-friendly policy enforcement and early flaw detection and prevention. Ensures the security and integrity of the components that make up critical applications by providing a complete component and application bill-of-materials inventory and a fast-path to discovering and fixing at-risk applications. Reduce operating costs since the cost of ripping out obsolete components from existing applications is high assuming the older versions can be identified in the first place. 13 A new approach is Component Lifecycle Management (CLM). For example, if a development team inadvertently downloads an obsolete OSS component, CLM can apply a method of breaking the build when that library is submitted, enforcing the use of a more current version. CLM informs the developers and security staff which components have risky vulnerabilities and which ones do not.
Slide 14 Component Lifecycle Management (CLM) Sonatype CLM for IDE in Eclipse lets developers make the best component choices early in the development cycle. http://www.sonatype.org/ 14 Point out bottom right shows current version popularity [black your version], License, Security Alerts Center describes the Vulnerabilities with CVE or OSVDB IDs.
Slide 15 Component Lifecycle Management (CLM) Integration with Content Integration servers enforces policy at build times. Sonatype Demo - https://www.youtube.com/watch?v=dcpra4snjj8 15
Slide 16 Component Lifecycle Management (CLM) Dashboards and reports provide a complete view of global risk with drill-down detail to drive action. http://www.sonatype.org/ 16
Slide 17 Component Lifecycle Management (CLM) Newly discovered threats are continuously reported against your inventory of components to ensure sustaining trust throughout your software supply chain. Sonatype Demo - https://www.youtube.com/watch?v=dcpra4snjj8 17
Slide 18 Obtain approval for use of OSS Who or Where in the organization does the final approval lie? Don t ask, don t tell policy has worked in the past, why change it? Internal program approval PM or Change Control Board CISO, ISSM, IAM External customer s approval process What does the contract say about OSS? Is there a formal external approval required; Contracting Agent, CISO, DAA Approved Products List 18 Who s the Sheriff in your town? Who can give final approval to use OSS? Talk the bullets. Approved Product List? Example the Army has a CoN List.
Slide 19 This? or This? How do developers structure and manage the source code? Melting Pots or Buckets of accountability? 19
Slide 20 Code from Multiple Sources Custom Developed 15% Legacy Harvested Code Others? Subcontractors 5% 10% Final Project Code Open Source Software 55% Commercial Software 5% Government Provided 10% FOR OFFICIAL USE ONLY (FOUO) 20
Slide 21 Software Security Development Lifecycle (SSDL) - Security Architecture Review http://www.bsimm.com/ 21
Slide 22 Potential Threats within the SDLC License Violations (aka. IP) Injection of Malicious Functionality Design Flaw in Security Functionality Poor Coding Practice http://xkcd.com/1354/ 22
Slide 23 What s in memory? Memory will contain juicy information, such as passwords or decrypted messages from other clients, secret key from X.509 certificates. Sending another heartbeat message leaks another 64KB, so rinse and repeat to scour the victim's system for goodies. How long? OpenSSL's implementation of TLS heartbeats was committed to the project's source code 61 minutes to midnight on Saturday, 31 December, 2011. 23 It was out there 2 years before official discovery by Google and Codenomicon. What bad is that this vulnerability will not show up on audit log files. It s following normal accepted behavior.
Slide 24 Was Heartbeat a security design flaw? IETF RFC 6520 defines the Heartbeat Extension for Transport Layer Security (TLS) and Datagram Transport Layer Security (DTLS) allowing the usage of keep-alive functionality without performing a renegotiation and a basis for Path Maximum Transmission Unit (PMTU) discovery. The necessary probe packets are the HeartbeatRequest messages. Jim Humphreys - Senior Security Analyst at Tangible Security posted his opinion on LinkedIn: The problem is that the standard does NOT REQUIRE that the recipient of a Heartbeat Request Message verify that the length of the data in the Payload field be exactly the same length as specified in the Heartbeat Message Length field. While this may seem obvious, its usually the obvious (but unstated) that leads to problems in fully specifying a protocol. FOR OFFICIAL USE ONLY (FOUO) 24
Slide 25 IETF RFC 6520 4. Heartbeat Request and Response Messages The Heartbeat protocol messages consist of their type and an arbitrary payload and padding. struct { HeartbeatMessageType type; uint16 payload_length; opaque payload[heartbeatmessage.payload_length]; opaque padding[padding_length]; } HeartbeatMessage; The total length of a HeartbeatMessage MUST NOT exceed 2^14 or max_fragment_length when negotiated as defined in [RFC6066]. type: The message type, either heartbeat_request or heartbeat_response. payload_length: The length of the payload. payload: The payload consists of arbitrary content. padding: The padding is random content that MUST be ignored by the receiver. The length of a HeartbeatMessage is TLSPlaintext.length for TLS and DTLSPlaintext.length for DTLS. Furthermore, the length of the type field is 1 byte, and the length of the payload_length is 2. Therefore, the padding_length is TLSPlaintext.length - payload_length - 3 for TLS and DTLSPlaintext.length - payload_length - 3 for DTLS. The padding_length MUST be at least 16. FOR OFFICIAL USE ONLY (FOUO) The sender of a HeartbeatMessage MUST use a random padding of at least 16 bytes. The padding of a received HeartbeatMessage message MUST be ignored. If the payload_length of a received HeartbeatMessage is too large, the received HeartbeatMessage MUST be discarded silently. When a HeartbeatRequest message is received and sending a HeartbeatResponse is not prohibited as described elsewhere in this document, the receiver MUST send a corresponding HeartbeatResponse message carrying an exact copy of the payload of the received HeartbeatRequest. If a received HeartbeatResponse message does not contain the expected payload, the message MUST be discarded silently. If it does contain the expected payload, the retransmission timer MUST be stopped. http://tools.ietf.org/html/rfc6520) 25
Slide 26 What if RFC 6520 had required The Heartbeat Request Message to verify that the length of the data in the Payload field be exactly the same length as was specified in the Heartbeat Message Length field or discard without reply. What if developers were trained on secure coding practices What if a static code scan was performed to check for bad coding practice What if a fuzz or application penetration test was performed? http://www.theregister.co.uk/2014/04/09/heartbleed_explained/ FOR OFFICIAL USE ONLY (FOUO) 26
Slide 27 Software Security Development Lifecycle (SSDL) - Open Source Security & Static Code Scans http://www.bsimm.com/ 27
Slide 28 Did you really get the source code? Developers like to take the easy path Downloading compiled code (binaries or.jars) rather then source code. Downloading everything but only using a portion An example from Apache Lucene Downloading and compiling the source code can be difficult Usually downloading with (Git, Subversion, Mercurial) Providence & Chain of Custody (Verifying Hashes & Signatures) Jan 6 15:39:19 2014 openssl-1.0.1f.tar.gz (MD5) (SHA1) (PGP sign) 28
Slide 29 Heartbeat - Was it a coding oversight? hbtype = *p++; /* message type is popped into the hbtype variable, the pointer is incremented by one byte */ n2s(p, payload); /* the n2s() procedure writes the 16-bit length of the heartbeat payload to the variable payload and increments the pointer by two bytes. */ pl = p; /* Then pl becomes a pointer to start of payload */ /* Enter response type, length and copy payload */ *bp++ = TLS1_HB_RESPONSE; s2n(payload, bp); memcpy(bp, pl, payload); The Fix: hbtype = *p++; n2s(p, payload); if (1 + 2 + payload + 16 > s->s3->rrec.length) return 0; /* silently discard */ pl = p; http://www.theregister.co.uk/2014/04/09/heartbleed_explained/ http://git.openssl.org/gitweb/?p=openssl.git;a=blob;f=ssl/t1_lib.c;h=a2e2475d136f33fa26958fd192b8ace158c4899d#l3969 FOR OFFICIAL USE ONLY (FOUO) 29
Slide 30 Would a static scan find Heartbeat vulnerability? http://www.openssl.org/source/ 30 OpenSSL-1.0.1f 134 Folders, 2,361 Files
Slide 31 Why didn t Cppcheck catch this? http://cppcheck.sourceforge.net/ 31 Static scan shows 7 Stylistic warnings and no errors. memcpy() is not a App Sexc Dev STIG banned function Place to use a static code scan is for your developers to run one against their own code during code check-in process.
Slide 32 Why didn t Cppcheck catch this? 32 21 instances of memcpy() in this file alone. Difficulty of tracing variables feeding a function if you were not the author. Certainly code that implements communication protocols should get a high level of review.
Slide 33 Limitations of Static Source Code Scans Cannot scan binaries.class are compiled JRE runtime binaries, not source code. They are difficult to scan with accuracy..so.a.dll.exe High false positive counts Importing entire 3rd party projects brings in many files that will never be use. This drives up the count of High and Moderate findings. For example: Lucene: 61 high findings (.class files) 33 Static Code Scans can catch lots of bad code. They primarily focus on Availability and Performance issues. It is best to use them early in the SDLC.
Slide 34 Software Security Development Lifecycle (SSDL) - Static/Dynamic Binary Scanners 34
Slide 35 Static/Dynamic Binary Scans A determination of software vulnerability density for a specific version of software at a point in time provided through a third party administered process. This analysis is done against the software s binaries not the source code. It can catch additions made within the compile process Probably better at catching Malware 35
Slide 36 Static/Dynamic Binary Scans What to look for: Minimum; CWE/CVE ID & Vulnerability Definition, Folder File & Line #, Risk Level Better; CWE/CVW link, click through to code (IDE integration), Exploitability http://www.veracode.com/ 36
Slide 37 Static/Dynamic Binary Scans What to look for: Your scanner should help you prioritize fixes 37
Slide 38 What to look for: Static/Dynamic Binary Scans Your scanner should help you track remediation progress 38
Slide 39 Software Security Development Lifecycle (SSDL) - Dynamic/Fuzz/Penetration Testing http://www.bsimm.com/ 39
Slide 40 Dynamic/Fuzz/Pen Testing A software testing technique, often automated or semi-automated, that involves providing invalid, unexpected, or random data to the inputs of a computer program. The program is then monitored for exceptions such as crashes, or failing built-in code assertions or for finding a potential memory leak. Fuzzing is commonly used to test for security problems in software or computer systems. File formats and network protocols are the most common targets of testing, but any type of program input can be fuzzed. Interesting inputs include environment variables, keyboard and mouse events, and sequences of API calls. Even items not normally considered "input" can be fuzzed, such as the contents of databases, shared memory, or the precise interleaving of threads. For the purpose of security, input that crosses a trust boundary is often the most interesting. For example, it is more important to fuzz code that handles the upload of a file by any user than it is to fuzz the code that parses a configuration file that is accessible only to a privileged user. Codenomicon, a commercial vendor of fuzz testing tools, was testing their Safeguard extension feature for protocol testing and discovered the Heartbleed flaw and reported to CERT http://en.wikipedia.org/wiki/fuzz_testing https://www.brighttalk.com/webcast/288/109899?autoclick=true 40 What is Fuzz testing? Pick key areas like; network protocols, file formats, user inputs. Fuzz test anything that crosses the trust boundary.
Slide 41 Hire a Bounty Hunter? 41 Bounty Hunters Talk about the Hacker Black Market Talk about Google and other Corp. paying a bounty of up to $4,000 per bug.
Slide 42 Key Takeaways 42
Slide 43 Questions? Contact: Rick.Koch@Sentar.com 43 Thank ISSA & Cyber Huntsville Questions? Rick.Koch@Sentar.com
Slide 44 Web References 1. DoD CIO guidance on Open Source Software http://dodcio.defense.gov/todayincio/freeopensourcesoftware.aspx 2. Video on Open Source component vulnerability https://www.youtube.com/watch?feature=player_embedded&v=cjtptrmhre8 3. Title: Financial Services, ISAC working group white paper on 3 rd party software security : http://docs.ismgcorp.com/files/external/wp_fsisac_third_party_software_security_worki ng_group.pdf 4. http://blog.sonatype.com/2010/12/now-available-central-download-statistics-for-ossprojects/ 5. Building Security In Maturity Model http://www.bsimm.com 6. Black Duck Hub on OSS http://www.ohloh.net 7. Cartoon explanation of Heartbleed http://xkcd.com/1354/ 8. SSL Heart Beat FRC http://tools.ietf.org/html/rfc6520) 9. OpenSSL source code download http://www.openssl.org/source/ 10. http://www.theregister.co.uk/2014/04/09/heartbleed_explained 11. Comparison of free and open-source software licenses: http://en.wikipedia.org/wiki/comparison_of_free_software_licenses 12. http://www.ciosummits.com/media/pdf/solution_spotlight/cignex_open_source.pdf 44
Slide 45 Book References Title: The Art of Software Security Assessment: Identifying and Preventing Software Vulnerabilities. Author: Mark Down Title: Secure Coding Standards, Software Engineering Institute, Carnegie Mellon: The CERT C Secure Coding Standard The Cert C++ Secure Coding Standard The CERT Oracle Secure Coding Standard for Java 45