Early Vulnerability Detection for Supporting Secure Programming

Transcription

1 Early Vulnerability Detection for Supporting Secure Programming Luciano Sampaio - lsampaio@inf.puc- rio.br Alessandro Garcia - afgarcia@inf.puc- rio.br OPUS Research Group LES DI PUC- Rio - Brazil OPUS Research Group

2 Introduction Secure programming is the practice of writing programs that are resistant to attacks by malicious people or programs; Check for SQL Injection. Check for XSS. Check for Cookie Poisoning. Security vulnerability or just vulnerability is a flaw that can be exploited to allow an attacker to cause unintended operations. 2

3 Introduction In 2001 a single vulnerability in Microsoft s Internet Information Server, cost an estimated $2 billion in repair, lost productivity, and support (pcworld );! It is almost never present in computer science courses;! 86% of audited websites has at least 1 serious security vulnerability (WhiteHat );! Developers usually find out about vulnerabilities too late. 3

4 Top 10 Vulnerabilities Open Web Application Security Project (OWASP) 2003, 2004, 2007, 2010, 2013;! OWASP Top From 8 datasets from 7 companies with over 500,000 vulnerabilities; 01 (SQL/Command) Injection; 02 Broken Authentication and Session Management; 03 Cross- Site Scripting (XSS); 04 Insecure Direct Object References; 05 Security Misconfiguration; 06 Sensitive Data Exposure; 07 Missing Function Level Access Control; 08 Cross- Site Request Forgery (CSRF); 09 Using Known Vulnerable Components; 10 Unvalidated Redirects and Forwards. 4

5 Denial of Service XSS (Cross-Site Scripting) SQL Injection Cookie Poisoning Information Leakage Security Misconfiguration 5 5

6 Tools can help On average 17% cost savings (Dejan Baca );! Manual Inspection Daunting and error prone;! Static Analysis! It analyzes the application by examining the source code without executing it Dynamic Analysis Exact location of the problem; Higher rate of false positives; IBM AppScan, Fortify, Lapse+, CodePro, ASIDE, It analyzes the application during runtime Usually can not inform the exact location of the problem; Low rate of false positives; Fortify,. 6

7 Limitations Late Detection;! High rate of false positives;! Do not provide possible solutions. 7

8 Creates Limitations >> Late Detection Developer Source code Runs Report Generates Plug-in 8

9 Limitations >> High rate of false positives 20-30% - (Nadeem 2012). 9

10 Limitations >> High rate of false positives >> Pattern Matching ASIDE s false positive of XSS. ASIDE s false negative of SQL Injection. 10

11 Limitations >> Do not provide possible solutions 11

12 Motivation and Objectives Developers might not have the necessary knowledge and even if they have, it is a complex task;! There are available tools. However, they do not fully support secure programming We call them "security vulnerability pointers ;! Help developers create secure software. 12

13 Proposal >> Early Detection Possible solutions Creating Developer Source code String login = request.getparameter("login"); String sql = "SELECT * FROM USER WHERE LOGIN = '" + login + "'"; Statement statement = conn.createstatement(); ResultSet resultset = statement.executequery(sql); request.setattribute("login", login); Report Runs Generates Plug-in 13

14 Proposal >> Early Detection ASIDE ESVD 14

15 Pattern Matching vs Data Flow Analysis 15

16 Data Flow Analysis >> Context Insensitive vs Context Sensitive CodePro ESVD CodePro ESVD 16

17 Data Flow Analysis >> Infinite Loop ESVD 17

18 Data Flow Analysis >> What we do not understand Containers;! Reflection;! InnerClasses ESVD Dillig, I., Dillig, T. and Aiken, A. (2011). Precise reasoning for programs using containers. ACM SIGPLAN Notices Tripp, O., Pistoia, M., Fink, S. J., Sridharan, M. and Weisman, O. (2009). TAJ: Effective Taint Analysis of Web Applications. In Proceedings of the 2009 ACM SIGPLAN conference on Programming language design and implementation - PLDI 09., PLDI 09. ACM Press. 18

19 Evaluation >> Empirical Studies Rate of False Positive Benchmark on 5 open- source projects and 1 custom- made project; Early Detection effectiveness Participants were asked to create a code using our tool. 19

20 Evaluation >> Benchmark Blueblog Personalblog WebGoat Roller Pebble NCO Version Number of packages Number of classes Number of methods Lines of Code Number of Vulnerabilities Applications 20

21 Benchmark >> Analyzed vulnerabilities Nr Vulnerability ASIDE Lapse+ CodePro ESVD 1 Command Injection Cookie Poisoning Cross- Site Scripting (XSS) HTTP Response Splitting LDAP Injection Log Forging Path Traversal Reflection Injection Security Misconfiguration SQL Injection XPath Injection Total

22 Benchmark >> BlueBlog Precision Recall F1 Score ASIDE 0,26 0,61 0,36 CodePro 0,80 0,22 0,35 Lapse+ 0,41 0,72 0,52 ESVD 1,00 0,44 0,62 22

23 Benchmark >> PersonalBlog Precision Recall F1 Score ASIDE 0,87 0,40 0,55 CodePro 0,75 0,02 0,04 Lapse+ 0,83 0,24 0,37 ESVD 0,97 0,78 0,87 23

24 Benchmark >> PersonalBlog - False Positive 24

25 Benchmark >> WebGoat Precision Recall F1 Score ASIDE 0,49 0,71 0,58 CodePro 0,50 0,09 0,15 Lapse+ 0,68 0,65 0,67 ESVD 0,79 0,41 0,54 25

26 Benchmark >> WebGoat - False Positive 26

27 Benchmark >> Roller Precision Recall F1 Score ASIDE 0,30 0,12 0,17 CodePro 0,78 0,09 0,16 Lapse+ 0,41 0,17 0,24 ESVD 0,99 0,89 0,94 27

28 Benchmark >> Roller - False Positive 28

29 Benchmark >> Pebble Precision Recall F1 Score ASIDE 0,50 0,36 0,42 CodePro 0,71 0,06 0,11 Lapse+ 0,46 0,27 0,34 ESVD 0,95 0,63 0,75 29

30 Benchmark >> Pebble - False Positive 30

31 Benchmark >> NCO Precision Recall F1 Score ASIDE 0,71 0,35 0,47 CodePro 0,36 0,05 0,09 Lapse+ 0,39 0,42 0,40 ESVD 0,39 0,61 0,47 31

32 Benchmark >> NCO - False Positive 32

33 Benchmark >> Summary Precision Recall F1 Score % False Positive ASIDE 0,48 0,39 0,43 51,78% CodePro 0,62 0,07 0,13 37,62% Lapse+ 0,55 0,36 0,43 44,73% ESVD 0,88 0,66 0,75 11,70% 33

34 Benchmark >> Summary Blueblog Personalblo WebGoat Roller Pebble NCO Total % LDAP Injection 0 g Vulnerabilit 0,0% Command Injection ,1% XPath Injection ,1% Hard- code content ,2% Security ,5% Misconfiguration HTTP Response ,6% Splitting Cookie Poisoning ,7% Reflection Injection ,1% String concatenation ,4% Information Leakage ,8% SQL Injection ,5% Cross- Site Scripting ,9% Path Traversal ,4% Log Forging ,3% Input not sanitized ,4% Total ,00% 34

35 Benchmark >> Memory 1000" 900" Memory' 800" 700" 600" 500" 400" 300" 200" Blueblog" Personalblog" NCO" WebGoat" Roller" Pebble" 100" 0" ASIDE" CodePro" Lapse+" ESVD" Blueblog Personalblog NCO WebGoat Roller Pebble ASIDE 272,46 123,12 142,12 486,24 395,32 472,4 CodePro 314,14 89,24 148,54 254,3 189,5 267 Lapse+ 462,56 425,6 351,06 456,92 324,34 521,4 ESVD 209,82 257,64 465,42 556,02 725,02 961,7 35

36 Benchmark >> Time 10:04,80$ 08:38,40$ Time% 07:12,00$ 05:45,60$ 04:19,20$ 02:52,80$ Blueblog$ Personalblog$ NCO$ WebGoat$ Roller$ Pebble$ 01:26,40$ 00:00,00$ ASIDE$ CodePro$ Lapse+$ ESVD$ Blueblog Personalblog NCO WebGoat Roller Pebble ASIDE 00:01,61 00:01,74 00:02,71 00:06,54 00:04,12 00:10,58 CodePro 00:26,91 00:35,93 00:37,76 01:42,91 09:34,71 05:13,30 Lapse+ 00:22,46 00:21,49 00:28,27 00:25,73 00:20,39 01:15,11 ESVD 00:00,68 00:00,77 00:07,34 00:22,63 00:43,89 06:11,49 36

37 Experiment 2 groups of participants (students and professionals), divided in 2 groups (Early Detection and Late Detection): Both using ESVD; ESVD modified version;! 2 questionnaires (1 before and 1 after): Understand the participant s profile and collect feedback; 37

38 Experiment Asked them to develop some functionalities of a small system: Initial project and basic jsp pages already created; Login and logout; Add, Update, Delete and List comments;! Record their screen, audio and Eclipse s interactions: ScreenFlow; Rabbit- eclipse; 38

39 Experiment >> Participants Early Late Total Total Student Professional # 14# 12# 10# 8# 6# 4# 2# 0# 15# Quan/ty# 12# Student# 7,25# 1,67# Average#of#Years# Professional# 39

40 Experiment >> Completed tasks Participant Id Task 1 Task 2 Task 3 Task 4 Task

41 Experiment >> Participants - Final numbers Early Late Total Total Student Professional Early Late Total Total Student Professional

42 Experiment >> Participant s profile 42

43 Experiment >> Number of vulnerabilities Added Removed Left Early Late Early Late Early Late Professional Student Vulnerability Added Removed Left Cookie Poisoning HTTP Response Splitting SQL Injection Log forging Cross- Site Scripting Misconfiguration Total

44 The plug- in ESVD - Early Security Vulnerability Detector ;! Download at: security- vulnerability- detector- esvd/! A project containing several security vulnerabilities: rio.br/~lsampaio/plugin/ early_vulnerability_detector/latest/webdemo.zip! More info at: 44

45 The plug- in >> Menu 45

46 The plug- in >> Preferences Page 46

47 The plug- in >> Preferences Page 47

48 The plug- in >> Preferences Page 48

49 The plug- in >> UI 49

50 The plug- in >> Provide possible solutions 50

51 The plug- in >> Provide possible solutions 51

52 Thank you! Luciano Sampaio - lsampaio@inf.puc- rio.br Alessandro Garcia - afgarcia@inf.puc- rio.br OPUS Research Group LES DI PUC- Rio - Brazil OPUS Research Group

53 Questions? LES DI PUC- Rio - Brazil OPUS Research Group