Early Vulnerability Detection for Supporting Secure Programming Luciano Sampaio - lsampaio@inf.puc- rio.br Alessandro Garcia - afgarcia@inf.puc- rio.br OPUS Research Group LES DI PUC- Rio - Brazil OPUS Research Group
Introduction Secure programming is the practice of writing programs that are resistant to attacks by malicious people or programs; Check for SQL Injection. Check for XSS. Check for Cookie Poisoning. Security vulnerability or just vulnerability is a flaw that can be exploited to allow an attacker to cause unintended operations. 2
Introduction In 2001 a single vulnerability in Microsoft s Internet Information Server, cost an estimated $2 billion in repair, lost productivity, and support (pcworld - 2001);! It is almost never present in computer science courses;! 86% of audited websites has at least 1 serious security vulnerability (WhiteHat - 2013);! Developers usually find out about vulnerabilities too late. 3
Top 10 Vulnerabilities Open Web Application Security Project (OWASP) 2003, 2004, 2007, 2010, 2013;! OWASP Top 10 2013 From 8 datasets from 7 companies with over 500,000 vulnerabilities; 01 (SQL/Command) Injection; 02 Broken Authentication and Session Management; 03 Cross- Site Scripting (XSS); 04 Insecure Direct Object References; 05 Security Misconfiguration; 06 Sensitive Data Exposure; 07 Missing Function Level Access Control; 08 Cross- Site Request Forgery (CSRF); 09 Using Known Vulnerable Components; 10 Unvalidated Redirects and Forwards. 4
Denial of Service XSS (Cross-Site Scripting) SQL Injection Cookie Poisoning Information Leakage Security Misconfiguration 5 5
Tools can help On average 17% cost savings (Dejan Baca - 2008);! Manual Inspection Daunting and error prone;! Static Analysis! It analyzes the application by examining the source code without executing it Dynamic Analysis Exact location of the problem; Higher rate of false positives; IBM AppScan, Fortify, Lapse+, CodePro, ASIDE, It analyzes the application during runtime Usually can not inform the exact location of the problem; Low rate of false positives; Fortify,. 6
Limitations Late Detection;! High rate of false positives;! Do not provide possible solutions. 7
Creates Limitations >> Late Detection Developer Source code Runs Report Generates Plug-in 8
Limitations >> High rate of false positives 20-30% - (Nadeem 2012). 9
Limitations >> High rate of false positives >> Pattern Matching ASIDE s false positive of XSS. ASIDE s false negative of SQL Injection. 10
Limitations >> Do not provide possible solutions 11
Motivation and Objectives Developers might not have the necessary knowledge and even if they have, it is a complex task;! There are available tools. However, they do not fully support secure programming We call them "security vulnerability pointers ;! Help developers create secure software. 12
Proposal >> Early Detection Possible solutions Creating Developer Source code String login = request.getparameter("login"); String sql = "SELECT * FROM USER WHERE LOGIN = '" + login + "'"; Statement statement = conn.createstatement(); ResultSet resultset = statement.executequery(sql); request.setattribute("login", login); Report Runs Generates Plug-in 13
Proposal >> Early Detection ASIDE ESVD 14
Pattern Matching vs Data Flow Analysis 15
Data Flow Analysis >> Context Insensitive vs Context Sensitive CodePro ESVD CodePro ESVD 16
Data Flow Analysis >> Infinite Loop ESVD 17
Data Flow Analysis >> What we do not understand Containers;! Reflection;! InnerClasses ESVD Dillig, I., Dillig, T. and Aiken, A. (2011). Precise reasoning for programs using containers. ACM SIGPLAN Notices Tripp, O., Pistoia, M., Fink, S. J., Sridharan, M. and Weisman, O. (2009). TAJ: Effective Taint Analysis of Web Applications. In Proceedings of the 2009 ACM SIGPLAN conference on Programming language design and implementation - PLDI 09., PLDI 09. ACM Press. http://doi.acm.org/10.1145/1542476.1542486. 18
Evaluation >> Empirical Studies Rate of False Positive Benchmark on 5 open- source projects and 1 custom- made project; Early Detection effectiveness Participants were asked to create a code using our tool. 19
Evaluation >> Benchmark Blueblog Personalblog WebGoat Roller Pebble NCO Version 1.0 1.2.6 5.4 0.9.9 2.6.4 1.0 Number of packages 22 10 24 70 100 49 Number of classes 38 38 159 283 743 84 Number of methods 227 253 1.453 2.704 3.445 517 Lines of Code 2.200 2.933 24.483 34.301 36.709 6.048 Number of Vulnerabilities 18 148 488 521 440 77 Applications 20
Benchmark >> Analyzed vulnerabilities Nr Vulnerability ASIDE Lapse+ CodePro ESVD 1 Command Injection 0 1 1 1 2 Cookie Poisoning 1 1 1 1 3 Cross- Site Scripting (XSS) 1 1 1 1 4 HTTP Response Splitting 0 1 1 1 5 LDAP Injection 0 1 1 1 6 Log Forging 1 1 1 1 7 Path Traversal 0 1 1 1 8 Reflection Injection 0 0 1 1 9 Security Misconfiguration 0 0 1 1 10 SQL Injection 1 1 1 1 11 XPath Injection 0 1 1 1 Total 4 9 11 11 21
Benchmark >> BlueBlog Precision Recall F1 Score ASIDE 0,26 0,61 0,36 CodePro 0,80 0,22 0,35 Lapse+ 0,41 0,72 0,52 ESVD 1,00 0,44 0,62 22
Benchmark >> PersonalBlog Precision Recall F1 Score ASIDE 0,87 0,40 0,55 CodePro 0,75 0,02 0,04 Lapse+ 0,83 0,24 0,37 ESVD 0,97 0,78 0,87 23
Benchmark >> PersonalBlog - False Positive 24
Benchmark >> WebGoat Precision Recall F1 Score ASIDE 0,49 0,71 0,58 CodePro 0,50 0,09 0,15 Lapse+ 0,68 0,65 0,67 ESVD 0,79 0,41 0,54 25
Benchmark >> WebGoat - False Positive 26
Benchmark >> Roller Precision Recall F1 Score ASIDE 0,30 0,12 0,17 CodePro 0,78 0,09 0,16 Lapse+ 0,41 0,17 0,24 ESVD 0,99 0,89 0,94 27
Benchmark >> Roller - False Positive 28
Benchmark >> Pebble Precision Recall F1 Score ASIDE 0,50 0,36 0,42 CodePro 0,71 0,06 0,11 Lapse+ 0,46 0,27 0,34 ESVD 0,95 0,63 0,75 29
Benchmark >> Pebble - False Positive 30
Benchmark >> NCO Precision Recall F1 Score ASIDE 0,71 0,35 0,47 CodePro 0,36 0,05 0,09 Lapse+ 0,39 0,42 0,40 ESVD 0,39 0,61 0,47 31
Benchmark >> NCO - False Positive 32
Benchmark >> Summary Precision Recall F1 Score % False Positive ASIDE 0,48 0,39 0,43 51,78% CodePro 0,62 0,07 0,13 37,62% Lapse+ 0,55 0,36 0,43 44,73% ESVD 0,88 0,66 0,75 11,70% 33
Benchmark >> Summary Blueblog Personalblo WebGoat Roller Pebble NCO Total % LDAP Injection 0 g 0 0 0 0 0 0 Vulnerabilit 0,0% Command Injection 0 0 2 0 0 0 2 0,1% XPath Injection 0 0 1 0 0 0 1 0,1% Hard- code content 0 0 4 0 0 0 4 0,2% Security 0 4 3 2 0 0 9 0,5% Misconfiguration HTTP Response 4 0 0 4 1 1 10 0,6% Splitting Cookie Poisoning 0 0 6 2 3 0 11 0,7% Reflection Injection 0 5 1 1 11 1 19 1,1% String concatenation 0 0 22 1 0 0 23 1,4% Information Leakage 0 0 8 16 7 0 31 1,8% SQL Injection 0 16 33 10 0 0 59 3,5% Cross- Site Scripting 2 11 5 59 16 6 99 5,9% Path Traversal 3 0 31 38 36 1 109 6,4% Log Forging 2 56 70 284 185 0 597 35,3% Input not sanitized 7 56 302 104 181 68 718 42,4% Total 18 148 488 521 440 77 1692 100,00% 34
Benchmark >> Memory 1000" 900" Memory' 800" 700" 600" 500" 400" 300" 200" Blueblog" Personalblog" NCO" WebGoat" Roller" Pebble" 100" 0" ASIDE" CodePro" Lapse+" ESVD" Blueblog Personalblog NCO WebGoat Roller Pebble ASIDE 272,46 123,12 142,12 486,24 395,32 472,4 CodePro 314,14 89,24 148,54 254,3 189,5 267 Lapse+ 462,56 425,6 351,06 456,92 324,34 521,4 ESVD 209,82 257,64 465,42 556,02 725,02 961,7 35
Benchmark >> Time 10:04,80$ 08:38,40$ Time% 07:12,00$ 05:45,60$ 04:19,20$ 02:52,80$ Blueblog$ Personalblog$ NCO$ WebGoat$ Roller$ Pebble$ 01:26,40$ 00:00,00$ ASIDE$ CodePro$ Lapse+$ ESVD$ Blueblog Personalblog NCO WebGoat Roller Pebble ASIDE 00:01,61 00:01,74 00:02,71 00:06,54 00:04,12 00:10,58 CodePro 00:26,91 00:35,93 00:37,76 01:42,91 09:34,71 05:13,30 Lapse+ 00:22,46 00:21,49 00:28,27 00:25,73 00:20,39 01:15,11 ESVD 00:00,68 00:00,77 00:07,34 00:22,63 00:43,89 06:11,49 36
Experiment 2 groups of participants (students and professionals), divided in 2 groups (Early Detection and Late Detection): Both using ESVD; ESVD modified version;! 2 questionnaires (1 before and 1 after): Understand the participant s profile and collect feedback; 37
Experiment Asked them to develop some functionalities of a small system: Initial project and basic jsp pages already created; Login and logout; Add, Update, Delete and List comments;! Record their screen, audio and Eclipse s interactions: ScreenFlow; Rabbit- eclipse; 38
Experiment >> Participants Early Late Total Total Student 10 10 20 34 Professional 7 7 14 16# 14# 12# 10# 8# 6# 4# 2# 0# 15# Quan/ty# 12# Student# 7,25# 1,67# Average#of#Years# Professional# 39
Experiment >> Completed tasks Participant Id Task 1 Task 2 Task 3 Task 4 Task 5 1 1 0 0 0 0 2 1 0 0 0 0 3 1 1 0 0 0 4 1 0 0 0 0 5 1 0 0 0 0 10 1 1 1 0 0 13 1 0 0 0 0 14 1 0 0 0 0 15 1 0 0 0 0 16 1 1 0 0 0 18 1 1 0 0 0 19 1 1 0 0 0 24 1 0 0 0 0 25 1 0 0 0 0 32 1 1 0 0 0 33 1 0 1 0 0 34 1 1 1 1 1 17 7 3 1 1 40
Experiment >> Participants - Final numbers Early Late Total Total Student 10 10 20 34 Professional 7 7 14 Early Late Total Total Student 5 3 8 17 Professional 5 4 9 41
Experiment >> Participant s profile 42
Experiment >> Number of vulnerabilities Added Removed Left Early Late Early Late Early Late Professional 23 9 6 1 17 8 Student 11 6 3 0 8 6 49 10 39 Vulnerability Added Removed Left Cookie Poisoning 1 0 1 HTTP Response Splitting 1 1 0 SQL Injection 3 1 2 Log forging 5 3 2 Cross- Site Scripting 13 3 10 Misconfiguration 26 2 24 Total 49 10 39 43
The plug- in ESVD - Early Security Vulnerability Detector - 0.3.6;! Download at: https://marketplace.eclipse.org/content/early- security- vulnerability- detector- esvd/! A project containing several security vulnerabilities: http://www.inf.puc- rio.br/~lsampaio/plugin/ early_vulnerability_detector/latest/webdemo.zip! More info at: http://thecodemaster.net/ 44
The plug- in >> Menu 45
The plug- in >> Preferences Page 46
The plug- in >> Preferences Page 47
The plug- in >> Preferences Page 48
The plug- in >> UI 49
The plug- in >> Provide possible solutions 50
The plug- in >> Provide possible solutions 51
Thank you! Luciano Sampaio - lsampaio@inf.puc- rio.br Alessandro Garcia - afgarcia@inf.puc- rio.br OPUS Research Group LES DI PUC- Rio - Brazil OPUS Research Group
Questions? LES DI PUC- Rio - Brazil OPUS Research Group