Analyzing Android Applications with Soot and FlowDroid

Transcription

1 Analyzing Android Applications with Soot and FlowDroid Eric Bodden SECURE SOFTWARE ENGINEERING GROUP

2 Agenda Why analyzing Android apps? Brief history of Soot Inter-procedural analysis with Heros Features added by FlowDroid Improving FlowDroid s results by deobfuscating apps with Harvester 2 SECURE SOFTWARE ENGINEERING GROUP

3 Why analyzing Android Apps? SECURE SOFTWARE ENGINEERING GROUP

4 4 [ICSE 15]

5 [BlackHat Europe 15] [McAfee s Threat Report Q4 2015] 5 SECURE SOFTWARE ENGINEERING GROUP

6 Other good reasons To optimize performance To lower power consumption To map the app landscape Your own topic here 6 SECURE SOFTWARE ENGINEERING GROUP

7 The Soot framework SECURE SOFTWARE ENGINEERING GROUP

8 What is Soot? a free compiler infrastructure, written in Java (LGPL) was originally designed to analyze and transform Java bytecode original motivation was to provide a common infrastructure with which researchers could compare analyses (points-to analyses) has been extended to include decompilation, visualization, Android support, inter-procedural analysis support, etc. etc. 8

9 What is Soot? Soot has many potential applications: used as a stand-alone tool (command line or Eclipse plugin) extended to include new IRs, analyses, transformations and visualizations as the basis of building new special-purpose tools 9

10 Soot - Past and Present Started in with the development of coffi by Clark Verbrugge and some first prototypes of Jimple IR by Clark and Raja Vallée-Rai First publicly-available versions of Soot 1.x were associated with Raja s M.Sc. thesis New contributions and releases have been added by many researchers from around the world Currently maintained by my research group at Darmstadt and Paderborn 10

11 Soot - Past and Present Soot 1.x has been used by many research groups for a wide variety of applications. Has also been used in several compiler courses. Last version was Soot 2.0 and the first version of the Eclipse Plugin were released - June JIT for PLDI Soot 2.3.0: Java 5 support Soot 2.4.0: partial Reflection support Soot 2.5.0: Better points-to analyses etc. Since then: continuous nightly builds, no official releases (and none planned) 11

12 Soot & me (2003) 12

13 Soot & me (2006) 13

14 14

15 The first thing Soot does... Parse method (as source code or bytecode) and convert into control-flow graph (CFG) Nodes: Simplified statements Edges: Possible control flows between such statements 15

16 Example y=x y=x; if(p) if(p) x=y; else z=2; x=y z=2 b=y; a=z; b=y a=z 16

17 In general, CFG is an over-approximation if(isprime( )).. if(!isprime( )).. y=x y=x; if(p) if(p) x=y; if(!p) z=2; b=y; a=z; x=y z=2 if(!p) b=y depending on how complex predicate p is, cannot infer that branches are mutually exclusive a=z 17

18 Intermediate Representation: Jimple Jimple = like Java, but simple Combines the best of both worlds Local variables, like in source code no stack operations Special variables for this and parameters Only simple statements, never nested 18

19 Jimple IR void foo() { Main this; double d1, d2, temp$0; int i1; void foo() { double d1 = 3.0; double d2 = 2.0; int i1 = (int) (d1*d2); bar(this,i1); this Main; d1 = 3.0; d2 = 2.0; temp$0 = d1 * d2; i1 = (int) temp$0; virtualinvoke this.<main: void bar(main,int)>(this, i1); return; all variables explicitly declared, even this 19

20 Jimple IR void foo() { Main this; double d1, d2, temp$0; int i1; void foo() { double d1 = 3.0; double d2 = 2.0; int i1 = (int) (d1*d2); bar(this,i1); this Main; d1 = 3.0; d2 = 2.0; temp$0 = d1 * d2; i1 = (int) temp$0; virtualinvoke this.<main: void bar(main,int)>(this, i1); return; special references for this and parameters 20

21 Jimple IR void foo() { Main this; double d1, d2, temp$0; int i1; void foo() { double d1 = 3.0; double d2 = 2.0; int i1 = (int) (d1*d2); bar(this,i1); this Main; d1 = 3.0; d2 = 2.0; temp$0 = d1 * d2; i1 = (int) temp$0; virtualinvoke this.<main: void bar(main,int)>(this, i1); return; no stack operations; instead assignments 21

22 Jimple IR void foo() { Main this; double d1, d2, temp$0; int i1; void foo() { double d1 = 3.0; double d2 = 2.0; int i1 = (int) (d1*d2); bar(this,i1); this Main; d1 = 3.0; 1:n d2 = 2.0; temp$0 = d1 * d2; i1 = (int) temp$0; virtualinvoke this.<main: void bar(main,int)>(this, i1); return; complex statements broken down at most one reference on left-hand side, at most two references on right-hand side 22 => three-address code

23 Jimple IR void foo() { Main this; double d1, d2, temp$0; int i1; void foo() { double d1 = 3.0; double d2 = 2.0; int i1 = (int) (d1*d2); bar(this,i1); this Main; d1 = 3.0; d2 = 2.0; temp$0 = d1 * d2; i1 = (int) temp$0; virtualinvoke this.<main: void bar(main,int)>(this, i1); return; method calls fully resolved, 23 explicit this reference

24 Whole-program analysis The most important data structure to conduct an interprocedural whole-program analysis is a call graph. A call graph is a static abstractions of all method calls that a program may execute at runtime methods are nodes calls are edges flow-insensitive (no execution order) 24

25 A simple call graph public class Main implements Observer { public static void main(string[] args) { Main m = new Main(); Subject s = new Subject(); s.addobserver(m); s.modify(); Main.main Subject.<init> Subject.addObserver public void update(observable o, Object arg) { System.out.println(o+" notified me!"); Subject.modify static class Subject extends Observable { public void modify() { setchanged(); notifyobservers(); Observable.setChanged Main.update Observable.notifyObservers 25

26 Problem: Polymorphic calls import java.util.*; Main.makeCollection public class Main { public static void main(string[] args) { Collection c = makecollection(args[0]); c.add("soso"); static Collection makecollection(string s) { if(s.equals("list")) { return new ArrayList(); else { return new HashSet(); Main.main ArrayList.add? HashSet.add 26

27 Constructing call graphs Soot supports a number of call-graph construction algorithms CHA: Class Hierarchy Analysis RTA: Rapid Type Analysis VTA: Variable Type Analysis new: special support for libraries [Reif et al., FSE 16] Spark: Andersen-style subset-based analysis 27 SECURE SOFTWARE ENGINEERING GROUP

28 Inter-procedural analysis with Heros SECURE SOFTWARE ENGINEERING GROUP

29 Inter-procedural analysis with Heros Soot itself has templates to define intraprocedural analyses Also comprises algorithms for constructing call graphs But has no template mechanism for interprocedural traversals Heros adds this support by implementing the IFDS and IDE frameworks 29 SECURE SOFTWARE ENGINEERING GROUP

30 1995 Thomas Reps Susan Horwitz Mooly (Shmuel) Sagiv IFDS Framework 30

31 I F D S Framework a b f(a) f(b) = f(a b) {a,b distributive Subset problem Many applications: Alias-, Taint-, Typestate-, Shape-Analysis... Idea: reduce inter-procedural analysis problem to pure graph reachability 31

32 Example Program Reps, Horwitz, Sagiv

33 Super Program graph 1995 Reps, Horwitz, Sagiv 33

34 Idea: Specialize graph Program Super Graph Flow Functions = Exploded Super Graph 34

35 Flow functions as edges 1995 Reps, Horwitz, Sagiv Requires finite (or at least enumerable) domain 35

36 Exploded Super Graph normal call return call-to- return Reps, Horwitz, Sagiv

37 On-the-fly construction Much more efficient to create super graph on the fly (exploded or not) Start with program s entry point(s) For all transitive successors compute super edges as they are needed see [Naeem, Lhotak, CC 2010] 37

38 RHS Solver Generic solver for IFDS (and IDE) problems Fully context sensitive Can run in time linear in size of graph. O(E D^3) E: number of super-graph edges D: size of finite domain Decides reachability efficiently by computing summary edges between method-start 38

39 RHS 95 data-flow facts Static Super-graph edge Computed Summary Edge statements d holds at s if there is a summary edge from s startof(methodof(s),d ) to (s,d) for some d 39 0 d d

40 RHS 95 flow functions f(b) must be distributive: f(a) f(a) f(b) = f(a b) f(a b) 40

41 method calls procedure p afterwards call method-local p decision: b holds at s if there is ret a psummary edge from startof(methodof(s),d ) to (s,b) for some d 0 a b 0 c d 41

42 Ex.: Information-flow analysis void main() { int x = secret(); int y = 0; y = foo(x); print(y); int foo(int p) { return p; 42

43 IFDS Info-flow analysis 43

44 IFDS in Soot public interface IFDSTabulationProblem<N,D,M> { FlowFunctions<N,D,M> flowfunctions(); Multimap<M,D> initialseeds(); D zerovalue(); InterproceduralCFG<N,M> interproceduralcfg(); default implementation exists, uses CallGraph class internally 44

45 IFDS in Soot public interface FlowFunctions<N, D, M> { public FlowFunction<D> getnormalflowfunction(n curr, N succ); public FlowFunction<D> getcallflowfunction( N callstmt, M destinationmethod); public FlowFunction<D> getreturnflowfunction( N callsite, M calleemethod, N exitstmt, N returnsite); public FlowFunction<D> getcalltoreturnflowfunction( N callsite, N returnsite); 45

46 IFDS in Soot public interface FlowFunction<D> { Set<D> computetargets(d source); 46

47 What FlowDroid brings to the table SECURE SOFTWARE ENGINEERING GROUP

48 sources sinks code analysis report potential privacy leaks

49 SMS/MMS Location Calendar Contact sources sinks code analysis report potential privacy leaks SuSi [NDSS 14]

50 sources sinks code analysis report potential privacy leaks SuSi [NDSS 14] SMS/MMS Bluetooth NFC Internet

51 sources sinks code analysis report potential privacy leaks [PLDI 14] FlowDroid

52 void main() { 7 a = new A(); b = a.g; b.f foo(a); sink(b.f); a.g.f 6 5 void foo( z ) { x = z.g; w = source(); x.f x.f = w; 3 x.f 4 1 z.g.f w 2 Will it leak? 52

53 Two main requirements maximize true warnings minimize false warnings 53

54 and do it fast! maximize true warnings minimize false warnings 54

55 maximizing true warnings Recall

56 How do apps actually execute? 56

57 Activity lifecycle Activity starts oncreate() onstart() onrestart() onresume() Activity is running onpause() Also: services broadcast receivers content providers fragments onstop() ondestroy() Activity is shut down 57

58 public class MyActivity extends Activity { private static String URL = protected void oncreate(bundle savedinstancestate) { TelephonyManager telephonymanager = (TelephonyManager) getsystemservice(context.telephony_service); String imei = telephonymanager.getdeviceid(); URL = protected void onstart(){ try{ URL url = new URL(URL); HttpURLConnection conn = (HttpURLConnection) url.openconnection(); conn.setrequestmethod("get"); conn.setdoinput(true); // Starts the query conn.connect(); catch(exception ex){ 58

59 Dummy main method Activity starts oncreate() onstart() onresume() Activity is running onpause() onstop() onrestart() boolean p = true; MyActivity act = new MyActivity(); while(p) { act.oncreate(null); while(p) { act.onstart(); while(p) { act.onresume(); act.onpause(); //more to come ondestroy() Activity is shut down 59

60 Dummy main method so-called opaque predicate must be constructed such that analysis will definitely not evaluate it to true/false boolean p = true; MyActivity act = new MyActivity(); while(p) { act.oncreate(null); while(p) { act.onstart(); while(p) { act.onresume(); act.onpause(); //more to come 60

61 Dummy main method boolean p = System.currentTimeMillis()>0; MyActivity act = new MyActivity(); while(p) { act.oncreate(null); while(p) { act.onstart(); while(p) { act.onresume(); act.onpause(); //more to come 61

62 Next problem, callbacks clicking a button causes a callback to be called 62

63 Next problem, callbacks which is something a security analysis might want to be aware of 63

64 Android defines callbacks in a special XML file <Button android:id="@+id/button1" android:layout_width="wrap_content" android:layout_height="wrap_content" android:layout_alignparenttop="true" android:layout_centerhorizontal="true" android:layout_margintop="185dp" android:text="@string/button" android:onclick="sendmessage"/> 64

65 boolean p = System.currentTimeMillis()>0; MyActivity act = new MyActivity(); while(p) { act.oncreate(null); while(p) { act.onstart(); while(p) { act.onresume(); act.sendmessage(...) act.onpause(); //more to come <Button android:id="@+id/button1" android:layout_width="wrap_content" android:layout_height="wrap_content" android:layout_alignparenttop="true" android:layout_centerhorizontal="true" android:layout_margintop="185dp" android:text="@string/button" android:onclick="sendmessage"/> Problem: not all UI elements are registered using XML files 65

66 Button b = (Button) findviewbyid(r.id.button2); b.setonclicklistener(new View.OnClickListener() { public void onclick(view v) { SmsManager sm = SmsManager.getDefault(); String number = " "; sm.sendtextmessage(number, null, imei, null, null); //sink ); 66

67 b.setonclicklistener(..) boolean p = System.currentTimeMillis()>0; MyActivity act = new MyActivity(); while(p) { act.oncreate(null); while(p) { act.onstart(); while(p) { act.onresume(); act.sendmessage(...) act.onpause(); //more to come new MyListener().onClick(..) Iterate until no new callback registrations are discovered 67

68 minimize false warnings maximize Precision

69 Minimizing false warnings through highly precise analysis FlowDroid s analysis is Fully Context sensitive k-field sensitive (default: k=5) k-object sensitive (default: k=5) Flow sensitive 69 SECURE SOFTWARE ENGINEERING GROUP

70 Problem: aliases may be created before the taint is created! x = new A(); y = new A(); y = x; x.f = privatekey(); need to look into the past print(y.f); 70

71 Forward Taint Analysis Backward Alias Analysis x.f=a Work Queue a y.f x=y Work Queue x.f CountingThreadPoolExecutor terminates not before both queues are empty 71

72 void main() { a = new A(); 7 b = a.g; b.f foo(a); sink(b.f); a.g.f void foo( z ) { x = z.g; w = source(); x.f = w; 3 x.f x.f 1 z.g.f w 2 For a more general approach see our ECOOP 16 paper on the Boomerang pointer analysis 72

73 Deobfuscating apps with Harvester [NDSS 16] SECURE SOFTWARE ENGINEERING GROUP

74 Current Android malware public static void gdadbjrj(string paramstring1, String paramstring2) throws Exception{ // Get class instance Class clz = Class.forName( gdadbjrj.gdadbjrj("vrif3+in9a.ata3rynd1bcvrv]af") ); Object localobject = clz.getmethod( gdadbjrj.gdadbjrj("]a9mafvm.9")).invoke(null); // Get method name String s = gdadbjrj.gdadbjrj( BaRIta*9caBBV]a"); // Build parameter list Class c = Class.forName( gdadbjrj.gdadbjrj("vrif3+invttnsari+r]kr9ar9")); Class[] arr = new Class[] { nglpsq.cbhgc, nglpsq.cbhgc, nglpsq.cbhgc, c, c ; // Get method and invoke it clz.getmethod(s, arr).invoke(localobject, paramstring1, null, paramstring2, null, null); SmsManager.sendTextMessage(...) 74

75 sendtextmessage(num, text) Harvester Class.forName(className) sendtextmessage( , loc_other ) sendtextmessage( , loc_us ) Class.forName( SmsManager ) 75

76 if(build.fingerprint.startswith("generic")) Static Analysis? msg = AES.decrypt("1234","fri$ds\&S") nr = "00" Dynamic Analysis? No! if(simcountryiso().equals("us")) No! nr += "4242" nr += "8888" sendtextmessage(nr, msg)... 76

77 Static Analysis + Dynamic Analysis 77

78 if(build.fingerprint.startswith("generic")) msg = AES.decrypt("1234","fri$ds\&S") nr = "00" Environment if(simcountryiso().equals("us")) nr += "4242" nr += "8888" sendtextmessage(nr, msg)... 78

79 if(build.fingerprint.startswith("generic")) msg = AES.decrypt("1234","fri$ds\&S") nr = "00" Environment if(simcountryiso().equals("us")) nr += "4242" nr += "8888" sendtextmessage(nr, msg)... 79

80 if(build.fingerprint.startswith("generic")) msg = AES.decrypt("1234","fri$ds\&S") nr = "00" Environment if(simcountryiso().equals("us")) nr += "4242" nr += "8888" sendtextmessage(nr, msg)... 80

81 x if(build.fingerprint.startswith("generic")) msg = AES.decrypt("1234","fri$ds\&S") nr = "00" Environment if(simcountryiso().equals("us")) nr += "4242" nr += "8888" sendtextmessage(nr, msg)... 81

82 x if(build.fingerprint.startswith("generic")) msg = AES.decrypt("1234","fri$ds\&S") nr = "00" Environment if(simcountryiso().equals("us")) nr += "4242" nr += "8888" sendtextmessage(nr, msg)... 82

83 x if(build.fingerprint.startswith("generic")) msg = AES.decrypt("1234","fri$ds\&S") nr = "00" Environment if(simcountryiso().equals("us")) nr += "4242" nr += "8888" sendtextmessage(nr, msg)... 83

84 x if(build.fingerprint.startswith("generic")) msg = AES.decrypt("1234","fri$ds\&S") nr = "00" Environment if(executor_1) nr += "4242" nr += "8888" sendtextmessage(nr, msg)... 84

85 main() { Callee1(false); Callee1(true); Callee1(boolean EXECUTOR_1) { msg = AES.decrypt("1234","fri$ds\&S") nr = "00" if(executor_1) nr += "4242" nr += "8888" Log(nr, msg) sendtextmessage(nr, msg) 85

86 Harvester enables de-obfuscation Class c = Class.forName(gdadbjrj.gdadbjrj( VRIf3+InVTTnSaRI+R]KR9aR9 ));... Class c = Class.forName("SmsManager"); SmsManager.sendTextMessage(a, b, c, d, e); 86

87 Recall: Precision: 87% 100% Efficiency: < 3 minutes 16,799 Malware Samples we manually looked into 12 samples Our approach works pretty well since none designs a malware that is combined to an environment variable Interesting findings: Premium-rate numbers C&C messages URLs (URIs) Encryption key for WhatsApp data and more 87

88 Effect of Harvester Harvester replaces conditionals by toolcontrolled variables Then runs multiple dynamic execution, one for each combination of boolean values some cases optimized away Dynamic analysis then reads out the requested dynamic values during execution Can improve FlowDroid s results, e.g. by resolving reflective calls 88

89 Results of Harvester Works very well on all current malware we tested Recovery rate for values of interest: >90% Usually results within seconds Can be used to improve results of static analyses by adding to the call graph 89

90 64 Test apps Complex data structures Callbacks, Lifecycle Field and Object Sensitivity Inter-app communication Reflection Implicit flows

91 DroidBench, obfuscated with DexGuard? = correct warning, = missed leak ple circles in one row: multiple leaks exp 91 SECURE SOFTWARE ENGINEERING GROUP

92 Summary Soot provides Jimple IR and general static-analysis infrastructure Heros adds inter-procedural analysis support FlowDroid adds modeling of lifecycle and highly precise taint analysis Harvester aids Soot/FlowDroid-based analyses through static/dynamic obfuscation 92 SECURE SOFTWARE ENGINEERING GROUP

93 Prof. Dr. Eric Bodden Chair for Software Engineering Heinz Nixdorf Institut Zukunftsmeile Paderborn Telefon: eric.bodden@uni-paderborn.de SECURE SOFTWARE ENGINEERING GROUP