Web Application security testing: who tests the test? Ainārs Galvāns Application Penetration Tester www.exigenservices.lv
About myself Functional testing Leading test group Reporting to client Performance testing HTTP level analysis Behavior modelling Security testing Application security Threat analysis Formal QA Certificates: ISTQB, CTT+, CPTE 2 www.exigenservices.lv
Why we test, what we test? 3 www.exigenservices.lv
Why we test, what we test? Specification Design Code What they needed 4 www.exigenservices.lv
Test levels - theory UAT System Integration Unit What they needed Specification Design Code 5 www.exigenservices.lv
Different test levels Dev. User Test 6 www.exigenservices.lv
Because bugs escape testing mistakes DEV Test time User changes skills 7 www.exigenservices.lv
QA approach: test to reduce bugs Functional testing practice: bug metrics Defect Removal Efficiency (DRE) = bugs before delivery / total number of bugs Defect density = defects / number of code lines passing security testing is not an indication that no flaws exist 8 www.exigenservices.lv
How much testing is enough? Functional testing experience: Coverage problem part of test strategy IEEE Std 829, Standard for test documentation Security testing is different: OWASP testing standard (minimal tests) ISO 2700*standard (minimal controls) Guides on: testing, coding, code review 9 www.exigenservices.lv
How much testing is enough? - experience Test guides: i.e. by OWASP ~100 attack types B.B. Test tools: thousands of attacks My own experience Average internal audit : Perhaps 3-5 person automated days testing is the choice? Singe attack type per field take up to 2 hours I don t test everything I don t test app., I validate assumptions 10 www.exigenservices.lv
Recent movement in Functional Testing Industry leaders re-defining term test Automated Checks VS Sapient Tests Some checks are hard(er) to automate check known Automated security test tools checks to see if the application is vulnerable to attacks that s not hard to automate Tools that help sapient (manual) testing 11 www.exigenservices.lv
Risk assessment: issue -> threat DREAD: prioritize issues based on sum of Damage potential Reproducibility (prerequisites) Exploitability (knowledge/tools required) Affected users Discoverability (e.g. risk of getting caught) Alternatives exist, such as CVSS Assumes we know the vulnerability Assumes we know all ways to exploit it (now or in future) 12 www.exigenservices.lv
Experience: missing threat analysis Audit Development 13 www.exigenservices.lv
Different security testing levels Application VS Perimeter* Code VS network Threat VS vectors Internal VS External Interfaces VS business Techniques VS risks * PCI: Application Layer VS Network Layer Testing 14 www.exigenservices.lv
Risk assessment: threat -> issue Step 1 OWASP Application Threat Modeling Decompose the application Step 2 Step 3 Determine (and rank) threats Set countermeasures & mitigation TEST it 15 www.exigenservices.lv
Mitigation strategies Performance monitoring Utilization = % of CPU, memory, network, etc. Response times, Session count, fault ratios, Security monitoring Analyze authentication (login) failures Analyze authorization failures (server side) Analyze XSS attack attempts 16 www.exigenservices.lv
Missed bug = reported by user? most of security incidents are discovered and reported only months after the initial intrusion or data compromise. OWASP: The CISO Guide Exception: DOS attack 17 www.exigenservices.lv
Backdoor mitigation Backdoors Application code (created on purpose) Created by hacker (i.e. stolen admin password) Mitigation analyze attacks not vulnerabilities Security Intelligence, SIEM Honeypot Users could help mitigate Inform user about last login User intelligence (i-bank: transaction history) Change passwords on regular basis 18 www.exigenservices.lv
Industry accepted as well as my own new and even crazy RECOMMENDATIONS 19 www.exigenservices.lv
Analysis of discovered vulnerabilities (findings) Adopt PCI recommendations per vulnerability : Define whether/how exploitable Risk ranking Adopt PCI overall analysis recommendations: Describe restriction imposed Use PCI Test report evaluation checklists Analyze risk of vulnerabilities missed 20 www.exigenservices.lv
Comprehensive testing Demand standard security Protect from script kiddies Discourage hackers (i.e. do not encourage) Distinguish comprehensive testing Documenting test coverage, not just bugs Hire two independent auditors to compete Penalties for missed security threats Developer testing 21 www.exigenservices.lv
Mitigation: intrusion detection Example:XSS protection options Turtle: do not allow Accepted: do not allow and notify admin Honeypot: allow, temporarily move to vault 22 www.exigenservices.lv
Gartner Top 10 Strategic Technology Trends for 2015 Risk-Based Security and Self-Protection While 100% security solutions aren t feasible, advanced risk assessment and mitigation will come into play in the next few years. Security will move away from perimeter defense to multifaceted approaches test educate 23 www.exigenservices.lv
QUESTIONS? Contact: Ainārs Galvāns Aplication PenTester, Exigen Services Latvia ainars.galvans@exigenservices.com J.Daliņa iela 15 Rīga, LV-1013, Latvia phone +371 6707 2976 mobile +371 2943 2698 www.exigenservices.lv 24 www.exigenservices.lv