AndroSSL: A Platform to Test Android Applications Connection Security

AndroSSL: A Platform to Test Android Applications Connection Security François Gagnon, Marc-Antoine Ferland, Marc-Antoine Fortier, Simon Desloges, Jonathan Ouellet, and Catherine Boileau Cybersecurity Research Lab Cégep de Sainte-Foy, Québec, Canada, frgagnon@cegep-ste-foy.qc.ca http://www.cegep-ste-foy.qc.ca/cybersecurity Abstract. Developing secure mobile applications is not an easy task; especially when dealing with SSL/TLS since very few developers possess experience with those protocols. This paper presents AndroSSL, an automated platform to assess the security of (SSL/TLS) connections established by Android applications. AndroSSL assists mobile application developers by testing their applications for man-in-the-middle attacks, and, successful, pinpoints the reason why the application is vulnerable. Keywords: Privacy, SSL, MitM, Android, Test-Bed, Automated Experiment. 1 Introduction Mobile application developers are facing a new and difficult security challenge. While traditional web applications, common in the desktop world, rely on web browsers to manage secure communications, each mobile application must deal with this element on its own. Establishing a secure channel using the SSL/TLS protocol [3] requires the client to check the validity of the SSL 1 certificate received from the server. An application accepting an invalid certificate would allow an attacker to impersonate the real server through a man-in-the-middle (MitM) attack. The last decade has shown that validating an SSL certificate is a difficult and errorprone task. Even big players (e.g., web browser developers) have a hard time getting it right (see 2 CVE-2008-4989, CVE-2009-1358, CVE-2009-2510, CVE-2009-3046, CVE-2010-1378, CVE-2014-1266). It would be unrealistic to believe all (even most) mobile application developers will rise to the task easily; especially since a lot of mobile applications are developed by non-expert programmers (much less security specialists). AndroSSL is a framework aiming to help mobile developers test their applications against connection security flaws. It relies on virtualization to provide a low cost and highly automated platform. Moreover, by offering a wide range of tests (in several different contexts), it is meant to provide detailed information regarding vulnerabilities in the certificate validation process. 1 The terms SSL and TLS are used interchangeably throughout this paper. 2 http://www.cve.mitre.org/

The paper is structured as follows: Section 2 provides an overview of AndroSSL, while Section 3 details the methodology used to mount MitM attacks. Section 4 discusses related work. Finally, Section 5 concludes with a summary of our findings and an opening on upcoming future work. 2 Experiment Test-Bed The objective of AndroSSL is to perform automated SSL MitM attacks against Android applications. When a MitM attack succeeds, it stands as proof that the application is vulnerable. Automation makes AndroSSL interesting for non-security experts (e.g., mobile app developers), since AndroSSL users do not have to worry about the inner working of security protocols. To automated its process, AndroSSL uses the AVP test-bed which relies heavily on virtualization to accomplish its mission. Android virtual devices (AVD) running on the Google QEMU emulator are used to host the applications to be tested, while standard virtual machines running in VMWare Workstation are used to host the required custom servers (e.g., DNS, Gateway, Fake SSL server). AndroSSL will run on any machine capable of running the Android Development Kit. When testing the security of an application, AndroSSL will go through eight steps: 1. Load a scenario describing the experiment. 2. Configure the network services needed (e.g., DNS, Fake SSL server). 3. Start an AVD. 4. Install the targeted Android application and launch it. 5. Start recording different aspects of the experiment (e.g., network traffic, Android internal log (logcat), screenshots of the AVD). 6. Interact with the application to trigger a legitimate login attempt. 7. Serve a fake 3 SSL certificate to the application. 8. Analyze experiment logs to determine if a security flaw was successfully exploited. The experiment specifications (scenario) is given to the system through an XML file. The scenario indicates which AVDs to use for the experiment, what information is to be recorded (e.g., network traffic, Logcat) and what actions are to be performed by the AVDs. During the execution phase, AVP manipulates AVDs through the instrumentation of various Android virtualization technologies. It executes the AVDs with their applications and then collects experiment data. AVP supports a wide range of actions and data collection capabilities. Regarding AndroSSL, the most important actions are: start/stop AVDs 4 and install/start applications. AVP supports several data gathering capabilities; those leveraged by AndroSSL are: network traffic recording and Android logcat recording. Once the experiment is over, post-analysis of the log files can be performed, see Section 3.2. 3 Meaning a certificate that should be considered invalid from the application s point of view. 4 With the possibility to use snapshots.

2.1 User Simulation The most challenging requirement of AndroSSL towards its test-bed is the need for proper user-simulation to lead an application to a specific state in order to trigger remote (possibly secure) connections. Our approach has been to manually pre-record (only once for each app) the sequence of actions leading the application to initiate a secure connection, then replay this action sequence automatically (as many times as needed) in AndroSSL experiments. Although this introduces manual intervention, it does not deviate from the objective of AndroSSL to help developpers test their apps automatically. Indeed, not every developer has the knowledge to mount a MitM attack or to analyze and interpret the network traffic and logs, but, it will be easy for them to record their intended action sequence. 3 Methodology The methodology behind AndroSSL has two components: the various scenarios used to attack the tested applications and the analysis of collected data to determine whether the attack was successful or not. Since some results will be discussed, a description of the experiment dataset comes first. 3.1 Dataset AndroSSL has been used to test 90 Android applications against 6 different attack scenarios. All these applications required Internet access and were downloaded from the top 500 most popular of each categories on the Google Play Store. 3.2 Analysis Three components are considered during the automated analysis of a run (that is, one application tested against one attack scenario): SSL server logs, network traffic and Logcat. When the SSL handshake is completed with a fake server, the success for the MitM is confirmed. Hence, the server logs are a good source of information. Network traffic is analyzed to find the plain password or a standard unsalted hash. This method allows to applications not using SSL to exchange credentials. Custom or salted hashes are more difficult to handle. In these cases, a heuristic search for keywords could be used, but manual confirmation would be required. Initially, logcat was not intended to be part of our result analysis. During the development phase, a search query was run by mistake on the logcat output file and returned surprising results. The password we used was broadcasted inside the AVD for a number of apps (10 out of 90). It turns out that those apps were relying on an API to query the server. The API will (or won t) use SSL properly to send the credentials to the server; this is also tested by AndroSSL. The broadcast of credentials inside an Android device is problematic as another installed application could manage to receive those broadcasts 5 (e.g., the CatLog Logcat Reader App 6 ). 5 Since Android 4.3, root privileges are required to access logcat. 6 https://play.google.com/store/apps/details?id=com.nolanlawson.logcat

Fig. 1. Static URL Network Configuration 3.3 Attack Scenarios The chosen apps have been tested against 6 attack scenarios. These scenarios, as well as the results, are presented below. 3.3.1 Actual Server The first scenario consists of letting the application connect to its actual server without any attempts to attack the app. No particular network configuration is required here. No meaningful results were expected. However, analysis of the network traffic turned out to be quite rich: 9 apps send the user s password in clear to the server. Anyone capturing the network traffic would catch the credentials. 1 app hashes (SHA-256 unsalted) the user s password before sending it to the server un-encrypted. 3.3.2 Static URL Two attack scenarios build the certificate using a static hostname. That is, the hostname is not related to the URL requested by the application. The network configuration schema of Figure 1 is used for these scenarios. The app queries a fake DNS (step 1) which will redirect the traffic to our fake SSL server (step 2). Every SSL connection will be served using the same static certificate (step 3). Two test cases rely on a static certificate. 3.3.2.1 Self-Signed This scenario is the easiest MitM involving SSL to perform for an attacker: create a self-signed certificate for a random hostname and use that certificate for a MitM attack. Figure 1 shows the network configuration underlying this attack scenario. Applications vulnerable to this attack are broken at two levels: chain validation (the certificate is self-signed) and source validation (the certificate is not for the expected hostname). 8 apps were vulnerable to this attack: 2 hash the password with MD5 (unsalted), the other 6 do not hash. 3.3.2.2 Valid This scenario is also easy to perform for an attacker: obtain a valid certificate for a domain you really own (15 minutes and 15 dollars) and use that certificate for a MitM attack. Being a static case, the network configuration of Figure 1 is used again here. Applications vulnerable to this attack are broken at source validation (the certificate chain is legitimate, but it is not for the expected hostname). The same 8 apps as in the previous case fell for this attack.

Fig. 2. Dynamic URL Network Configuration 3.3.3 Dynamic URL Three attack scenarios build the certificate using a dynamic hostname. That is, the hostname is related to the URL requested by the application. For the test cases requiring a dynamic certificate (i.e., where the certificate s hostname matches the one requested by the application), a more complex network configuration is deployed, see Figure 2. The DNS server attributes sequential IP addresses by incrementing the last byte by one at every request (step 1 in Figure 2). The first request receives IP 100.100.100.1, the second gets 100.100.100.2, and so on. Then, a Gateway (IPTables 7 ) will redirect SSL traffic towards the fake SSL server, but to a specific port (steps 2 and 3). For instance, IP 100.100.100.1 would get port 11001, while IP 100.100.100.255 would get 11255; the last three digits of the port match the last byte of the IP. When the SSL server receives a request on port 11xyz, it can associate this request with the URL originally requested to the DNS server (port 11xyz implies IP 100.100.100.xyz, which can then be matched to a URL in the DNS memory). Hence, the server is able to dynamically construct an SSL certificate with the expected hostname (step 4). 3.3.3.1 Self-Signed This scenario requires generating a self-signed certificate at runtime for the specific hostname requested by the app. Applications vulnerable to this attack are broken at chain validation (the certificate is self-signed). 12 applications fell for the attack, including the 8 discussed in the previous (static url) attacks. The 4 new vulnerable applications validate the hostname properly but not the chain (none of them hashes the password). The others were already known to be broken both for hostname and chain validation. 3.3.3.2 Signed by Self-Signed A self-signed certificate is generated once. Then, for each request a certificate is generated for the corresponding hostname and signed by the self-signed. Applications vulnerable to this attack are broken at chain validation. The same 12 apps as in the previous case fell for this attack. 3.3.3.3 Signed by Real A real certificate is first obtained for a controlled URL (like in Section 3.3.2.2). Then, for each request a certificate is generated for the corresponding 7 http://ipset.netfilter.org/iptables.man.html

hostname and signed by the real certificate. Applications vulnerable to this attack are broken at chain validation for a specific check: a real end-user certificate should not have the authority to sign other certificates (it is neither a root nor an intermediate certificate; it is a leaf). 12 applications fell for the attack, the same 12 as in the previous scenario. 3.3.4 Other Scenarios Since there are so many subtleties when validating SSL certificates, other scenarios will be created to test different flaws (e.g., expired certificates). 4 Related Work [6] performed a manual analysis of popular security libraries and concluded that SSL certificate validation is completely broken in non-browser software. Their analysis is not targeted towards Android applications and is fully manual. Hence, this method is not appropriate to average developers testing their apps. [4] proposed an automated approach relying on static analysis of Android application source code to find potential SSL certificate validation problems. An automated static analysis of 13 500 Android applications by their Mallodroid tool flagged 1 074 of them as being potentially vulnerable to MitM attacks. Furthermore, manual MitM attacks were performed on a subset of 100 potentially vulnerably apps. This analysis confirmed 41 vulnerable to MitM. Although the static analysis is automatic, manual work by a security expert is needed to avoid false positives (apps flagged as being vulnerable while they cannot be exploited). Again, not entirely suitable for a developer seeking to test his apps. [1] proposes a technique to automatically test certificate validation in SSL libraries. They target libraries implementations and not (Android) applications. Having correct SSL libraries is essential because apps using a broken library risk inheriting its vulnerabilities. However, having perfect libraries does not mean no vulnerabilities in applications as developers can misuse libraries introducing vulnerabilities. Hence testing applications and SSL libraries for vulnerabilities is complementary. [7] manually tested 100 Android Apps for MitM attacks. 32 accepted invalid certificate opening the door to a MitM attack. Their conclusion is that "...[there is a] need to give developers more effective tools that can help them detect and fix issues before the app is in production...". [5] proposes a way to improve the security of mobile applications by changing development practices related to SSL connections handling. Although this major step would certainly have a positive outcome, exhaustive testing will always play an important role (especially in security sensitive areas where new flaws are to be discovered). Google released the nogotofail [2] tool to test client side SSL certificate validation. A few difference with AndroSSL are worth mentioning. nogotofail provides no automation for user interaction on the client side. The user interaction has to be done (and repeated) entirely manually. Having no automation for user interaction means nogotofail is not limited to Android applications; it can be used to test any client. nogotofail is not limited to client-side validation as it includes bugs like Heartbleed, POODLE and gotofail. It seems that two scenarios are built in nogotofail to test for client-side certifi-

cate validation: one static (Invalid Hostname Certificate [2]) and one dynamic (hinted in [8]). It is not clear how easily nogotofail can be extended with other scenarios. [9] proposed an approach similar to AndroSSL where the experiment is fully automated and no security experts are required. However, the methodology of SMV-Hunter differs from AndroSSL making the two approaches complementary. SMV-Hunter s fully automated support is a great advantage, especially for large scale analysis. However, when SMV-Hunter fails (complex UI controls, invalid heuristic path), AndroSSL could take over to provide a more precise control of the user interaction. AndroSSL provides a more exhaustive set of test scenarios; using a variety of certificates and different network configurations (while SMV-Hunter tests for a single attack scenario). Having multiple scenarios allows AndroSSL to provide precise information regarding the certificate validation vulnerabilities in an application. For instance, a few applications tested were successfully attacked in the dynamic URL setup while the attempts failed in the static URL setup. This leads to the conclusion that those applications validate the certificate s hostname, but do not properly validate the signature chain. 5 Conclusion The main lesson we learned from our experimentation is that mobile application developers are not aware of the security challenges they face. Some applications, not even relying on SSL, are just too easy to attack. Tools such as AndroSSL should facilitate security testing for mobile app developers. 5.1 Summary Results Table 1. Result Summary Network Configurations Test Cases Vulnerable Apps Nb Actual Server Logcat 3, 4, 34, 35, 36, 37, 54, 72, 73 9 Network 3, 34, 35, 36, 37, 54, 63, 67, 70, 71 10 Static URL Self-Signed Valid 14, 16, 46 H, 53 H, 64, 68, 87, 89 8 Self-Signed Dynamic URL Signed by Self-Signed 9, 14, 16, 25, 26, 46 H, 53 H, 64, 68, 87, 89, 90 12 Signed by Real Total 25 Table 1 provides a summary of the problems found automatically by AndroSSL when testing 90 popular Android applications. For each test case, the list of vulnerable apps is given (each app being represented by a numeric ID). Apps in bold were vulnerable to a single experiment (this illustrates the need for a multi test-cases approach), and subscript H indicates apps hashing the password client side (no salt). The apps appearing more than once (e.g., 14 and 34) have multiple weaknesses (14 does not validate the certificate hostname nor the chain while 34 sends the clear password into the Android log and also on the network). Overall, AndroSSL determined that 25

apps have a vulnerability. If we remove the apps that are just sending the password into the Android log 8, we are still left with 22. 10 apps sent the user credential through the network over a non-encrypted channel. Another 12 apps have a broken SSL certificate validation procedure (8 accepting certificate with an invalid hostname and an invalid chain while the other 4 accept certificates with an invalid chain as long as they have the proper hostname). 5.2 Future Work Several tasks are planned to continue the development of AndroSSL: Adding new network configurations is a definitive priority, particularly to have the possibility to hijack SSL connections only (and target only 1 URL) while letting non-ssl connections flow towards the legitimate server. This will provide a better understanding regarding the conditions required for a successful MitM attack. This should allow for more successful attacks. Adding new test cases (i.e., new "fake" certificates) is an ongoing task as there are many subtle cases worth testing. References 1. Brubaker, C., Jana, S., Ray, B., Khurshid, S., Shmatikov, V.: Using frankencerts for automated adversarial testing of certificate validation in SSL/TLS implementations. In: proceedings of the 2014 IEEE Symposium on Security and Privacy (SP) (2014) 2. Brubaker, C., Klyubin, A., Condra, G.: nogotofail. https://github.com/google/ nogotofail (2014) 3. Dierks, T., Rescorla, E.: Rfc5246 tls v1.2. https://tools.ietf.org/html/rfc5246 (2008) 4. Fahl, S., Harbach, M., Muders, T., Smith, M., Baumgartner, L., Freisleben, B.: Why Eve and Mallory love Android: An analysis of Android SSL (in)security. In: Proceedings of the 2012 ACM Conference on Computer and Communications Security (CSS 12). pp. 50 61 (2012) 5. Fahl, S., Harbach, M., Perl, H., Koetter, M., Smith, M.: Rethinking SSL development in an appified world. In: Proceedings of the 2013 ACM Conference on COmputer and Communications Security (CSS 13). pp. 49 60 (2013) 6. Georgiev, M., Iyengar, S., Jana, S., Anubhai, R., Boneh, D., Shmatikov, V.: The most dangerous code in the world: Validating SSL certificate in non-browser software. In: Proceedings of the 2012 ACM Conference on COmputer and Communications Security (CSS 12). pp. 38 49 (2012) 7. Onwuzurike, L., Cristofaro, E.D.: Danger is my middle name: Experimenting with SSL vulnerabilities in Android apps. In: proceedings of the 2015 ACM WiSec (2015) 8. Sillars, D.: Using nogotofail to find issues with your https connections. http: //developerboards.att.lithium.com/t5/at-t-developer-program-blogs/usingnogotofail-to-find-issues-with-your-https-connections/ba-p/39891 (2015) 9. Sounthiraraj, D., Sahs, J., Lin, Z., Khan, L., Greenwood, G.: SMV-Hunter: Large scale, automated detection of SSL/TLS man-in-the-middle vulnerabilities in Android apps. In: proceedings of the 2014 Network and Distributed System Security Symposium (NDSS 14) (2014) 8 Although this is problematic and should be addressed, we could consider this flaw as more difficult to exploit.