Security testing: a key challenge for software engineering. Yves Le Traon, yves.letraon@uni.lu Professor, Univ of Luxembourg

Transcription

1 Security testing: a key challenge for software engineering Yves Le Traon, yves.letraon@uni.lu Professor, Univ of Luxembourg 1

2 Objectives of the presentation - Promote applied research in systematic security testing - How? - Results of three research experiments - Lessons learnt - Open challenges 2

3 The perimeter of the talk Hacker Misuse the application by : - Exploiting flaws in the application Exploit vulnerabilities: -code injection - Bypass attacks - HTML JavaScript Flash Java Applets Requests Responses Information System JSP PHP DB Security mechanisms Client Web app. firewall Server 3

4 Overview Testing security? About XSS, web browsers and regression testing About mobile apps attack surface (Android) About internal information system security Emerging security testing challenges 4

5 About testing security 5

6 - 6 - Looking for bugs and other errors

7 Software testing: cost and trust Testing Design for testability (2) Design for trust Detecting inconsistencies between implementation and specification Testing Testing (1) Reliability Em Real Decrease Decrease with reg 7

8 Classical testing issues About XSS, web browsers and regression testing 8

9 Regression testing: From chaos to order 9

10 Software regression failures Non regression Chaotic Regression version 1 Version 2 Version 3 Version 4 Version 5 Version6 versions 10

11 Security and sofware engineering Program understanding/reverse engineering About mobile apps attack surface (Android) 11

12 Software testing vs. Security Testing Book: Open Close Read Functional testing: It works as expected Robustness testing: Then should still work Security testing: Very particular robustness Then should still work 12

13 Security testing is two fold Tester as a hacker (Pentest) Tester of the Security policy Security mechanisms Requests Responses Security mechanisms Client Web app. firewall Server Information System

14 About XSS, web browsers and regression testing Erwan Abgrall PhD Kereval, France Sylvain Gombault researcher telecom Bretagne 15

15 Attacking process Victim (web browser) Attacker Web server XSS attack Vector executed? Client request Generates the web page embedding an attack vector Compromised Web browser Attacker takes control XSS TestDriver 16

16 Vector / Payload / Attack Vector: piece of HTML code enabling JavaScript code execution Payload: The javascript code to be executed <script>alert(1)</script> Attack: Injection that makes the server generate the vector ></input><script>alert(1)</ script> 17

17 Attack surface of a web browser Code within a web browser that can be run by an attacker The set of executable vectors in a given browser XSS Test Driver aims at exercising this code 18

18 Selection of XSS test vectors Referenced vectors XSS Cheat Sheet : html5 xss cheat sheet: UTF-7 XSS Cheat Sheet: Final benchmark can be found New vectors generation {html4tag} X {property} X {JScall} 6 new vectors combinations 19

19 Example 1 svg based xss <g> onload <svg xmlns=" onload="javascript:%(eval_payload)s"></g></s vg> 20

20 Example 2 SVG chameleon behavior via embedded XSLT version="1.0"?> <?xmlstylesheet type="text/xml" href="#stylesheet"?> <!DOCTYPE doc [ <!ATTLIST xsl:stylesheet id ID #REQUIRED>]> <svg xmlns=" <xsl:stylesheet id="stylesheet" version="1.0" xmlns:xsl=" <xsl:template match="/"> <iframe xmlns=" src="javascript:%(eval_payload)s"></iframe> </xsl:template> </xsl:stylesheet> <circle fill="red" r="40"></circle> </svg> 21

21 Evading known signatures: Root vs. obfuscated vectors ROOT <script>alert(1)</script> OBFUSCATED <DIV STYLE="width:expression(eval(String.from CharCode(97,108,101,114,116,40,39,120, 115,115,39,41,32)));"> 22

22 Evading known signatures: Root vs. obfuscated vectors v1 v2 v3 v82 v83 v84 V3-with-comment1 V3-with-comment2 V3-with-commentn V3-with-comment2-encoded1 V3-with-comment2-encoded2 V3-with-comment2-encodedp Noxiousness of an obfuscated XSS Noxiousness of a root XSS 23

23 24 Web browser test mechanism

24 25

25 Are mobiles protected? NO! Web browser is the n 1 application for smartphones These browsers inherit the defects from their parent browsers Webkit / Gecko / Presto They even access more information Data URI & specific functionalities HTML Storage Saved Identifiers, since typing password is tedious on mobiles Camera API, Vibrator API, Contacts API WebAPI 26

26 27 Attack surface over time

27 Regression is about deltas Ideally: convergence In practice Vn Vn+1 Vn Vn+1 Delta(Vn, Vn+1) 28

28 Opera Opera variation 0 29

29 Internet Explorer IE 4.01 IE 5.01 IE 5.5 IE 6.0 IE 7.0 (beta 3) IE 8.0 IE 9.0 (beta) 30

30 31 Netscape

31 32 Mozilla

32 Firefox

33 34 Chrome

34 35 Android browser

35 One step further: Test for counter attacking A web browser differs from one another by its many features, one of them being its specific sensitivity to XSS attack vectors. identifying a fake user-agent determining the exact nature of an attacker's web browser for protecting and possibly counter-attacking. 36

36 Counter-attacking process Victim (web browser) UA(IE8.0) Attacker FakeUA(IE8.0) XSS Tests Web server Are you who you pretend? Chrome1.997! XSS attack with payload 37

37 Test Method for attacker identification Use the reaction of a given web browser to such known XSS vectors as a signature identifies it precisely (family and version) Test driven web browser fingerprinting 38

38 Results the exact version of a web browser (out of 77) can be determined thanks to its signature (71% of accuracy). 6 XSS test vectors are sufficient to quickly determine the exact family a web browser belongs to, with an accuracy of 98.6 % 39

39 Lessons learnt and challenges No obvious systematic regression testing strategy for security Urgent need for A tooled environment to systematically run regression tests An updated benchmark of XSS vectors Research Challenges Automate the generation of test vectors Collaborative FuzzTesting: Shazzer Possibly MBT? Using dynamic tests to detect and identify an attack 40

40 About mobile apps attack surface (Android) Alexandre Bartel PhD - UL Jacques Klein Researcher - UL 41

41 Permission-based architectures 42

42 Android Overview (1/2) Android = Software Stack 43

43 Android Overview (2/2) Android = permission based system Every application has a list of permissions Each permission controls access to a specific resource Android 2.2 declares in total 142 high-level permissions. Developers write the permission list Application 1 INTERNET CAMERA READ_SMS Example of Permission List 44

44 Permission Gap Permission Gap = {Declared Permission Set} - {Inferred Permission Set} Consequence: the attack surface is larger Example: attacker exploits a buffer overflow in a C library -> he could take advantage of the permission which is declared but not used by the application. -> How often do applications present a permission gap? 45

45 Code Static analysis: control flowgraph 46

46 Permission-Based Security Model

47 Android framework evaluation Android v2.2 bytecode and obtained a matrix M composed of 3957 methods We identified 4852 permissions checks in the framework Google maintains more than 4000 enforcement points Not necessarily programmed in a systematic way Documentation and maintenance issues 48

48 Case study : 1355 Android applications (Alternative Markets, Nov. 2011) 3.91% 12.47% 3.99% 6.57% 9.52% 8.71% 9.52% 16.24% 11.88% 1.85% 4.13% 5.76% 5.46% games communications news sports travel health entertainment reference system multimedia finance shopping productivity 358 / 1355 have a permission gap (26.4 %) 49

49 Evaluation of the Android Framework Matrix Comparison with testing [Felt,2011] At least 3 permissions were missing Combining results from testing (underapproximation) and static analysis (overapproximation) may yield correct results 51

50 ublic void onactivityresult(int, int, android.content.intent); Code: Testing and localizing suspicious byte code 0: iload_1 1: tableswitch{ //999 to : 28; default: 20 } 20: aload_0 21: iload_1 22: iload_2 23: aload_3 24: invokespecial #378; //Method android/app/activity.onactivityresult:(iilandroid/content/intent;)v 27: return 28: iload_2 29: bipush -1 31: if_icmpne 20 34: aload_3 35: ldc_w #380; //String android.intent.extra.ringtone.picked_uri 38: invokevirtual #384; //Method android/content/intent.getparcelableextra:(ljava/lang/string;)landroid/os/parcelable; 41: checkcast #386; //class android/net/uri 44: astore 4 46: aload 4 48: ifnull 20 51: aload_0 52: iconst_1 53: aload 4 55: invokestatic #96; //Method SoundboardActivity android/media/ringtonemanager.setactualdefaultringtoneuri:(landroid/content/context;ilandroid/net/uri;)v 58: goto 20 public void onadclick(com.mobclix.android.sdk.mobclixadview); Code: 0: ldc #43; //String SoundboardActivity 2: ldc_w #390; //String Ad clicked! 5: invokestatic #393; //Method android/util/log.v:(ljava/lang/string;ljava/lang/string;)i 8: d/content/context;ilandroid/net/uri;) istore_2 9: return RingtoneManager.setActualDefaultRingtoneUri:(Landroi OK! public void oncreate(android.os.bundle); Code: 0: aload_0 1: aload_1 2: invokespecial #397; //Method android/app/activity.oncreate:(landroid/os/bundle;)v 5: aload_0 6: ldc_w #398; //int : invokevirtual #401; //Method setcontentview:(i)v 12: aload_0 13: aload_0 14: putfield #67; //Field mcontext:landroid/app/activity; 17: aload_0 18: invokestatic #406; //Method com/mobclix/android/sdk/mobclix.oncreate:(landroid/app/activity;)v 21: aload_0 22: ldc_w #398; //int

51 Android Inter component Communication Given one Android app : We use data-flow analysis to Compute a list of components the app. Communicate with Compute an interface showing how other apps. Could communicate with the app. Each app can be described with those 2 lists Data-flow Analysis Having this map opens the door to : Detect Intents which can be intercepted by other applications In general the map can be used to detect all kinds of ICC vulnerabilities Detect Application Collusion (apps which share permissions)

52 Application Collusion GPS INTERNET READ_CONTACT

53 Conclusion Android security and research challenges Could we understand how to improve Android apps security? Static analysis allows reducing the attack surface Combining testing and static analysis Locate the suspicious byte code? Detect malware Combining testing with static analysis 55

54 About internal information system security 56

55 In a nutshell security policy architecture PEP1 Security policy Test cases Validate security mechanisms PEP2 PDP PEPn PEP: Policy Enforcement Point PDP: Policy Decision Point Research questions: How to evaluate test quality? How to generate security test cases? Secure Application 3/22/2013 Tejeddine Tejeddine Mouelhi Mouelhi - MUTATION'12 - SnT 58

56 Security policy: rights and duties Access Control Rules Express permissions or prohibitions for users to access some resources of the system Based on an Access control models (RBAC, OrBAC, MAC, DAC,...) Permission(Library, Teacher, Borrow, Book, WorkingDays) Obligation policies About usages/duties The doctor should examine a patient within 20 minutes 59

57 Obligation Management Start Create duty Oblig Abstract Instantiation End(Ca) Concrete End(Ca) Start(Ca) Fulfilled Inactive Active Fulfilled Start(Cv) Inactive Violated End(Ca) Violated Fulfilled Fulfilled Violated End(Ca) 60

58 Security policy mutation analysis Access control Policy: R1 R2 Ri. Initial policy Access control Policy: R1 Access control Policy: Access control Policy: R2 R1 R3 R2 R1 Ri. R3 R2 Ri. R3 Ri. Mutants implementation Test cases Components (Security mechanisms, Access control) Use Communicate (Input validation, security protections) Collaborate DB Components (Security mechanisms, Access control) Use Application 61

59 Testing Access Control Reuse functional tests Code coverage CR1: 1 test case per declared rule CR2: 1 test case per concrete rule 62

60 Case studies #classes #methods #LOC LMS (Library Management System) VMS (Virtual Meeting Server) ASMS (Auction Sale Management System)

61 Number of mutants Operator category Op. LMS ASMS VMS Basic Mutation operators Type changing Parameter changing Hierarchy changing PPR PRP RRD CRD RPD APD Rule adding operator ANR Total

62 Mutation score Functional test cases are not sufficient 100% 78% 100% 100% 100% 87% 73% 69% 65% Functional CR1 49% CR2 0% LMS ASMS VMMS Overall mutation scores with basic security mutants

63 Mutation score Testing non-explicit rules is harder Functional 100% CR1 CR2 80% 60% 40% 20% 0% LMS ASMS VMMS Mutation scores with all mutants

64 Conclusion and research challenges A qualification process Some challenges Test generation for security policies Formal models MBT Combinatorial testing Test performances vs. security Regression testing when the security policy evolves

65 Emerging research challenge Putting the cloud under pressure 68

66 Tests must scale too: Peer-to-peer Load Testing Test methodology Normal load Distributed Denial of Service Attacks (DDoS) 69

67 Tests must scale too: Peer-to-peer Load Testing Test methodology Normal load Distributed Denial of Service Attacks (DDoS) 70

68 Overall conclusion Many open challenges Testing as a hacker Models for generating new vectors No test adequacy criteria Regression testing Testing IDS and security components Testing a security policy Already some adequacy criteria Formal modelling MBT Testing a cloud robustness Design for testable security Model-driven security 71

69 «intelligently react to abnormal situations and ensure the quality of the information» (P1 conclusion) Questions? 72

70 6th IEEE conference in software testing, verification and validation Luxembourg, 2013 Deadline: september 17 73