Rolling out an Effective Application Security Assessment Program. Jason Taylor, CTO

Transcription

1 Rolling out an Effective Application Security Assessment Program Jason Taylor, CTO

2 About Security Innovation Authority in Application Security 10+ years of research and assessment Security testing methodology adopted by Microsoft, Adobe, Symantec, SAP Authors of 16 books Helping Organizations Create Secure Applications STANDARDS TRAINING ASSESSMENT

3 Agenda Role of assessments in an application security program Creating a risk-based tiered assessment model Risk ranking in practice Vulnerability remediation & metrics

4 The 3 Pillars of Secure Software Development Role of Assessments Standards Align development activities with policies, compliance, requirements Set expectations for your teams (in/outsource) Require education and continued assessments for success Education Builds knowledge needed to apply standards & assessments correctly Provide the means to test, verify, and measure all related activities Assessment Evaluates applications and SDLC against standards Provide a continuous improvement feedback model Results drive policy, standards, education and tools usage improvements

5 Assessments and the Application Security Continuum Secure at the Source Protect in Play Find and Fix Education Remediation Secure Coding Standards Existing Security Web Application Firewalls Application Whitelisting Vulnerability Scanning o Static/Dynamic Analysis Manual Assessments These strategies should not be mutually exclusive leveraging all three will have an exponential effect in risk reduction

6 Enterprise Assessment Program Goals Improve Vulnerability Management Regular, iterative testing ensures continually-improving test results and will catch vulnerabilities more quickly Measure risk of each application through vulnerability discovery and remediation metrics Discover trends and weaknesses that can be used to improve the overall AppSec program through standards and training Optimize Frequency and Depth of Testing Match level of testing and analysis to application criticality Ensures high risk applications get more attention and low-risk applications are not over tested Manage and Optimize Costs Predictable cost Investment matched to level of risk

7 Best Practices for Enterprise Assessments Use automated tools for heavy lifting Find known vulnerabilities faster than humans Adopt when you have the skills to use properly Be sure tools are integrated into SDLC and used at key checkpoints Complement with manual efforts Necessary to find deeply rooted, business logic, and other vulnerabilities that tools can not find Be sure to leverage a threat model to focus on high-risk areas Support vulnerability remediation Problem isn t solved when found, only when corrected properly Match test efforts with your organization s ability to remediate Measure effectiveness

8 Agenda Role of assessments in an application security program Creating a risk-based tiered assessment model Risk ranking in practice Vulnerability remediation & metrics

9 Why Risk-Rank? Helps your organization to Quantitatively categorize application assets Plan assessment and mitigation activities cost effectively Ensure prioritization is based on real- business risk Inappropriate security assessments are costly Deep inspection on all applications is neither feasible nor necessary Spending time on a low-priority application while a high-risk one remains vulnerable can be devastating (data breach, DDOS, etc.) Allows you to understand risk-based options for managing the security of inherited applications Remove, replace, take off-line, implement compensating controls

10 Taking a risk-based approach to assessments Conventional approaches to application security not risk-based Typically no more than automated vulnerability scanning that look for some pre-determined set of common vulnerabilities Frequently fail to address each application s unique code-, system- and workflow-level vulnerabilities Provides little practical guidance on prioritizing defect remediation Does not enable the creation of a roadmap to guide enterprise application security posture improvements The majority of application security programs focus on:* Automated security testing during development (41%) Secure coding standards that are adhered to (32%) A secure SDLC process improvement plan (30%) * The State of Application Security Maturity Ponemon Institue & Security Innovation, 2013

11 Application Security Risk Ranking Risk can be based upon: Data classification Business criticality of the service provided Exposure level (who the application is exposed to) Identify and prioritize application risk based on Business impact Data criticality Compliance mandates Operational risk Security threats Attack surface Some applications have very little exposure, while others are exposed to large numbers of users over the Internet. Some are connected to other enterprise systems, databases or web services, while others are more isolated and harder to access. Exposure (e.g. Internet vs. Intranet)

12 Application Security Risk Management Model Yields guidance on asset classification Delivers clear and concise way to measure risk Provides clear and actionable guidance on how to react to application security vulnerabilities Asset Threat Vulnerability Mitigation What are you trying to protect? What are you afraid of happening? How could the threat occur? What is currently reducing the risk? Probability Impact What is the impact to the business? How likely is the threat given the controls? Well-Formed Risk Statement

13 The Value of Threat Modeling Threat modeling assists the risk management process by helping you think through common problems: Too many applications; too little time Inappropriate security testing efforts A highly insecure application that is currently risk-ranked low because you don t realize how vulnerable it is to attack A high risk-ranked application may be very secure because of compensating controls you already have in place. To know your Enemy, you must become your Enemy. -Sun Tzu

14 Critical Success Factors in AppSec Risk Management Understand which applications introduce the greatest risk Understand what mitigating controls are necessary Aligning the level the testing and analysis with the criticality of the application Know what to do with security assessment results Make sure risk-ranking framework is: Transparent so decisions and calculations can be explainable Adaptable so each group can apply unique drivers, goals or resources Practical so you end up with something that works

15 AppSec Risk Analysis Considerations Threats can be inherited from systems dependencies and connectivity hackers can leverage non-critical applications to get to critical applications Evaluate all 3 rd party code/applications and COTS There is no standard formula risk tolerance and data classification is contextual to each organization Legacy applications inherently carry higher risk most code was written without security in mind If it s not managed code, you know there are known vulnerabilities that you have to look for, e.g., buffer overflows Business Criticality is driving factor when determining which applications to secure and level of regular assessment needed

16 Building your Application Inventory Goal is to have a complete list of all applications deployed Automated tools can help gather Inventory for COTS applications Custom-built or in-house developed applications are trickier Can be as simple as an spreadsheet or document, or as complex as nested mind maps or visual models Needs to serve as a living document and be kept up to date Be sure to include to-be-developed applications Make sure it is accessible to those who need to view and update Important to document 3 rd party software No access to source code = dependence on vendor to fix vulnerabilities May be viewed as higher risk due to lack of control

17 Defining your Inventory Approach Define and communicate your strategy for the process Decisions regarding manual vs. automated approaches Use of developer surveys Rollout by business unit, geography, etc. Create and maintain a central repository to maintain data Determine ownership of master list Include risk ranking, application description, business owner, etc. Start to think about application risk tiers How many groups do you need? How do you differentiate between groups? This is important for when you start doing data classification and mapping security test levels to each tier (more on this later)

18 Inventory List sample of documented information Software application name and vendor (if 3 rd party) Purpose and description of each application Business areas/functions supported Date originally implemented, last major change, current version IT owner Number of users Technical specs Operating System Code base size, location, and development languages used Deployed environment Dependencies (other applications, middleware, etc.)

19 Taking Inventory Start of the Classification Process Classifying your applications by type and function Interfaces and Users Internet-facing application? Customer-facing application? Partner-facing applications? Internal- or partner-facing web services? Customer Relationship Management (CRM) system? Type of Data PCI Credit Card PII e.g. Home Address HIPAA medical data Other customer sensitive data Other business sensitive data

20 Now a Bit Deeper on the Technical Specs Underlying Technology Infrastructure (OS, hardware, etc) Platform (.NET, Java, PHP, etc) Database (SQL, Oracle, etc) Current Security State Previous Assessment Activity (type, date, etc.) Known Vulnerabilities (high, medium, low) Existing Protections in place (IDS/IPS, WAF)

21 Where To Start? The Questionnaire. Objective is to determine: How do you define risk? What attributes make applications more or less risky? How do 3 rd party applications fit into the risk model? Make it detailed; keep it simple. Binary answers, if possible Avoid ambiguity

22 Risk Ranking - Questionnaire Application development team completes questionnaire to assist in the determination of the risk rating Arrange questions in categories, such as Users, Architecture, Data Classification, Compliance Requirements, etc. See following slides for samples The weight assigned to an individual question depends on your business objectives and risk tolerance Leverage threat modeling to assist here Some prefer a quantitative approach, e.g., Give each question a numeric score/weight (not all questions are equal) Add all scores to determine the tier of each application

23 Risk-Ranking: Users What type of user access does your application offer? Internal, external [Internet-facing], both, neither? Are there different user roles? e.g., admin, editor, user/viewer? Are there anonymous users? Is this application designed to aid Management or Board Members in decision making?

24 Risk-Ranking: Architecture Is native code executed on the client device? e.g., activex in a web app, assembly for embedded device, etc. Where will your application be deployed? Does application implement any kind of authentication? If yes, provide additional details Does application implement any kind of authorization? If yes, provide additional details Is this application a plug-in or extension for another application? If yes, include additional details on what all applications it will be working with Is there connectivity with other applications?

25 Risk-Ranking: Data Classification What type of data is contained in your application? Social security numbers, driver's license number, financial information (including partial credit card #s), medical information, health insurance information, usernames and passwords, etc. Could the compromise of data managed by your application lead to any of the following? Loss of critical campus operations Negative financial impact (money lost, value of the data) Damage to organization reputation Potential for regulatory or legal action Violation of organizational information security policies or principles Does application manage any classified or patented data? Important for firms that need to protect IP, copyrighted materials, research data sets

26 Risk-Ranking Questionnaire - Sample

27 Determining what the risk ranking means The application tiers represents the degree of risk your application presents Determine what policies you want in place for High, Medium, and Low risk applications What security activities are required? What level of training and security aptitude is required on the team? What assessment processes should be used? What tools and technologies should be implemented? Use the risk classification to define levels of investment for each level so that resources are used efficiently

28 Defining Tiers * Customer-facing applications would include internet-facing applications as well as applications that reside on mobile or in-home devices

29 Tier SDLC Policy ^ Dynamic testing only necessary for Web-based applications

30 Application Security Requirements - Example Application Tier High Medium Low Application Security Requirements Security Champion Full security training curriculum Security design and coding standards Threat model Design review Code review (automated and manual analysis) 3 rd party penetration test (automated and manual analysis) Privacy review Deployment review Security awareness training Security design and coding standards Threat model Code review (automated scanning) Penetration test (automated scanning) Deployment review (as appropriate) Penetration test (automated scanning)

31 Agenda Role of assessments in an application security program Creating a risk-based tiered assessment model Risk ranking in practice Vulnerability remediation & metrics

32 Risk Ranking in Practice Application #1 Application 1 helps customer to collect names and addresses for customer s newsletters. Data is stored in a shared database within the data center. This application was built by a third party, however customer owns all source code, maintenance, and rights to the application Data Sensitivity (1) - Full Names, addresses Lifespan (3) This application does not have an EOL set Compliance (0) This application does not have any compliance requirements Customer or Internet Facing (2) This application is hosted on the shared Virtual Server within the DMZ, it accesses a shared Database in the data center. Risk Rank 2

33 Risk Ranking in Practice Application #4 Application 4 is an internal support help-deck application for customer ticketing on a specific product. Customer records are stored in a central database within the data center; however, it is only customer name and address. This application was built in-house and the product it supports will be end-of-life d within the next 9-12 months. Data Sensitivity (1) Full Names, addresses Lifespan (1) This application has an EOL <12 months Compliance (1) This application has minimal internal compliance requirements Customer or Internet Facing (1) This application is hosted on an internal server and is not accessible externally. Risk Rank 1

34 Risk Ranking in Practice Application #7 Application 7 is an operational e-commerce application. It was built by a 3rd party to sell customer s products. Once the data has been collected it is stored in an encrypted database. Data collected is sensitive, and must be treated as such. Data Sensitivity (3) Full names, addresses, account numbers, credit card information Lifespan (3) This application does not have an EOL set Compliance (2) PCI, PII Customer or Internet Facing (3) This application is hosted on a dedicated Virtual Server within the DMZ, is Internet-facing, and accesses a database in a co-located data center. Risk Rank 3

35 Risk-Ranking in Practice Summary of Analysis

36 Agenda Role of assessments in an application security program Creating a risk-based tiered assessment model Risk ranking in practice Vulnerability remediation & metrics

37 Application Vulnerability Management Not all vulnerabilities are equal, so it s important to have defined criteria for vulnerability criticality designation and remediation Verify Is this a real problem? When in doubt, assume its real Understand What s worst possible damage? What is the fix? Analyze What's the risk to fix vs. the risk to defer? Look for other ways to minimize risk, even if its only for short term Look for root causes and drive policy/process change Prioritize Threat Modeling, STRIDE and Microsoft DREAD Data asset classification Criticality definitions (critical, high, medium, low)

38 Managing Assessment Findings Understand what you ve found - security isn t always obvious This will crash can be understood very quickly This can potentially be used to make your device stop responding takes more thought Start with the most critical problems Use your own vulnerability classification, STRIDE/DREAD, or tool ranking Keep in mind that tool rating does not reflect true risk to YOUR organization Use stack trace functionality to determine root cause & exploitability Requires some manual code review skills to do data/control flow analysis Watch for duplicates Sometimes you can get a ton of results from a single root cause, but there are multiple paths to exploit. One fix can get them all. Be careful with your fix Can create new issues or push problem deeper into code path Consider if WAF/AWL can mitigate critical/difficult to fix vulnerabilities

39 Closing the Loop Developer creates a weakness If correctly tooled, developer is alerted to weakness before check-in If not Sandbox / Dev test finds vulnerability Developer needs to analyze, determine root weakness, fix System test finds vulnerabilities Development team needs to model vulnerabilities, determine root causes, establish SDL policies and processes Individual developers assigned vulnerabilities to diagnose, fix Customer finds vulnerabilities How will you respond?

40 Tracking and Trending Nothing Can Ever Be Lost Weaknesses and vulnerabilities need to have their own lifecycle Losing a report can cost you your company Tools need to help this process, not hinder it Trending Open vulnerabilities Time to fix vulnerabilities by type and severity Number of vulnerabilities discovered over time High or critical vulnerabilities in production systems Time to fix vulnerabilities New vs. existing vulnerabilities Security Incidents Root Cause Analysis What issues give your developers (and customers) the most trouble? SDLC phase in which vulnerabilities are being discovered

41 Risk & Vulnerability Reporting

42 Security Risk Rating by Business Unit

43 Coding / Implementation Metrics % of software components subject to static and dynamic code analysis % of defects discovered in each SDLC phase % of defects caused by problems in: Requirements Design Implementation Deployment

44 Source Code Analysis Metrics Lines of Code and other similar measurements # of static analysis warnings # of static analysis false positives % of false positives # of repeat security bugs # of p0 security bugs in codecomplete drops to QA # of p0 security bugs pre-prod % of code re-used from other projects % of code from third parties (libraries) Source:

45 Finding and Preventing Vulnerabilities Secure at the Source Information Security Standards Secure Coding Standards Secure Development Process Education Find and Fix Vulnerability Scanning Penetration Testing Public Disclosure Fix is not simply addressing specific vulnerabilities But systemic issues (i.e. SDLC, lack of skills) that are enabling vulnerabilities It s not just about secure code, but secure requirements and design 70% of vulnerabilities are introduced before a line of code is written Design vulnerabilities are not just the most costly to fix, but often the most difficult Prevention requires knowing WHAT to do and HOW to do it

46 Conclusion A risk-based approach is an effective way to manage security across large number of applications AppSec risk management depends on analysis of data criticality and attack surface Risk tiers set of application security policies, standards, SDLC activities and training Enforce for each application Recurring assessment saves cost, improves efficiency, and reduces time to fix Some applications need deep assessments; some do not Identify which so you are not under/over testing & spending Continue to reassess application criticality to determine if still applicable

47 Application Security Solutions Professional Services Managed Enterprise Application Security Testing Software Assessment SDLC Optimization & Compliance IT Attack Simulation Copy of Presentation securityinnovation.com Computer Based Training 100+ Technical & Awareness courses Secure Design, Coding, Testing.NET, Java, C/C++, C#, PHP, Mobile, Cloud, OWASP, PCI, Database Secure Development Standards 3,500 code snippets, attacks, how-to s, checklists, etc. Meets PCI requirements 6.3, 6.5 and 6.6

48 Managed Application Security Testing (MAST) Decreased cost 20% to 30% reduction in cost over stand-alone testing services Online portal to facilitate reporting, scheduling, and remediation assistance Download reports for compliance or integrate into defect mgmt or GRC system Integrates with TEAM Academy training platform Accurate results Vulnerabilities are verified and false positives removed Faster remediation - Prescriptive platform and technology specific guidance provided

49 Questions/Comments? Copy of Presentation Technical Question Contact Sales

50 Gathering Vulnerability Information Tools Source code analyzers Dynamic application scanners Attack Surface Analyzer (MS) Threat Modeling Tool (MS) File Fuzzing Tools (MS) Regex Fuzzer (MS) Visual Studio Code Analysis Sonar, Jira, Panopticode Findbugs, PMD, Coverity, Klocwork Systems System inventory System vulnerability scans Qualys, Nessus Logs Application logs Web server logs Proxy logs System logs Firewall logs Splunk Other Bug tickets Audit reports Incident reports Sprint, Product backlogs 56

51 Assessment Program Key Performance Indicators (KPI s) KPIs align metrics to strategic business goals Weighted Risk Trend Weighted risk score over time, iterations of development Defect Remediation Window How long a defect takes to remediate to a fixed state Rate of Defect Recurrence Rate at which a defect is re-introduced over the life of an application Specific Coverage Metric Total addressable attack surface of the applications functionality tested Security to Quality defect Ratio # of defects in a testing cycle logged as security defects as a ratio to all quality defects

52 Measuring Effectiveness of your Assessment Program Necessary for assessing your organization s security posture Helps Senior Management and development teams to more readily measure the effectiveness of the Application Security program Measurements can be used to track program s overall progress Are any disturbing trends showing up? If deviations are appearing, are they within acceptable limits? How can we improve prioritization and remediation of issues Even impartial and incomplete metrics may be able to highlight areas of weakness that could be improved Weak or deficient controls Unproductive processes Failure to regularly measure and understand the consequences could be result in serious compliance issues

53 Defect Metrics Number of defects observed in a software product is a measure of software quality Number of security design changes required Number of security related errors detected by code inspections Number of code changes necessitated by security related aspects Aged backlog of bugs in a production product Measure rate of bug closure Measure rate that the most critical bugs get fixed