Censorship In The Wild: A Measurement-based Analysis of Internet Filtering in Syria

Transcription

1 Censorship In The Wild: A Measurement-based Analysis of Internet Filtering in Syria Emiliano De Cristofaro (University College London) Joint work with: A. Chaabane, T. Chen, A. Friedman, M. Cunche, D. Kaafar Tech Report: 1 / 24

2 Censorship Censorship: The practice of suppressing ideas and information that certain individuals, groups, or government officials may find objectionable, dangerous, or detrimental Often aims to restrict freedom of speech, control knowledge available to masses and enforce ethical/religious principles Internet proliferation has prompted increased electronic censorship Many countries worldwide have some Internet filtering programs See reports from, e.g., OpenNet Initiative, RWB, Freedom House Note that many Western countries (including UK, Australia, France, Italy) also perform non-negligible filtering 2 / 24

3 Motivation Research community has increasingly focused on studying Internet censorship and proposing countermeasures Alas, understanding of filtering processes is inherently limited as analysis is almost exclusively based on probing 1 Both challenging and risky to conduct active measurements 2 Only a limited number of requests can be observed, and only a limited view of policies is possible 3 Hard to assess the extent of the censorship Unsurprisingly, real-world datasets very hard to come by 3 / 24

4 Roadmap 2011 leak of 600GB logs from 7 Blue Coat SG-9000 proxies Deployed in Syria to monitor and filter Internet traffic at a country-scale a unique opportunity to provide a detailed snapshot of a real-world censorship ecosystem We present a measurement-based analysis of the Blue Coat logs, uncovering a relatively stealthy, yet quite targeted, censorship based on four main characteristics: 1 IPs blocking entire subnets 2 Domains blocking specific websites 3 Keywords denying all requests with a certain keyword 4 Category targeting specific content 4 / 24

5 The Dataset Leaked by Telecomix, a hacktivist group during OpSyria: action de surveillance de la censure d Internet en Syria 600 GB of logs from Blue Coat appliances (54GB compressed) Obtained from an unprotected FTP server Made publicly available by Telecomix 5 / 24

6 Blue Coat proxies SG-9000 model, used for filtering and proxying Used by the Syrian Telecommunications Establishment (STE) Installed between Internet backbone and Syrian network 6 / 24

7 Ethics Consideration Even though data was public, we are aware of its sensitivity, thus we: Encrypted all data (and backups) at rest Did not re-distribute the logs Never obtained/extracted users personal information, but only analyzed aggregated traffic statistics Obtained IRB approval from INRIA and NICTA Out of the scope to discuss ethics of leaked data for research purposes (see Egelman et al. s FC 11 panel) 7 / 24

8 Datasets Description and Notation 1/2 Table : Datasets description. Dataset Notation # Requests Period # Proxies Full D full 751,295,830 July 22-23,31, 2011 August 1-6, Sample (4%) D sample 32,310,958 July 22-23, 2011 August 1-6, User D user 6,374,333 July Denied D denied 47,452,194 July 22-23,31, 2011 August 1-6, Million Requests Covering 9 days, Summer 2011 Logs from 7 different Blue Coat appliances 8 / 24

9 Dataset Description and Notation 2/2 Table : Description of a few relevant fields from the logs. Field name Description cs-host Hostname or IP address (e.g., facebook.com) cs-uri-scheme Scheme used by the requested URL (mostly HTTP) cs-uri-port Port of the requested URL cs-uri-path Path of the requested URL (e.g., /home.php) cs-uri-query Query of the requested URL (e.g.,?refid=7&ref=nf fr& rdr) cs-uri-extension Extension of the requested URL (e.g., php, flv, gif,...) cs-user-agent User agent (from request header) cs-categories Categories to which the requested URL has been classified c-ip Client s IP address (removed or anonymized) s-ip The IP address of the proxy that processed the client s request sc-status Protocol status code from the proxy to the client (e.g., 200 for OK) sc-filter-result Content filtering result: DENIED, PROXIED, or OBSERVED x-exception-id Exception raised by the request (e.g., policy denied, dns error). Set to - if no exception was raised. 9 / 24

10 Classification Allowed (x-exception-id = - ): allowed and served (no exception) Denied (x-exception-id - ): not served, either due to network error or censorship 1 Censored (x-exception-id {policy denied, policy redirect}): based on censorship policy 2 Error (x-exception-id { -, policy denied, policy redirect}): due to network error Proxied (sc-filter-result = PROXIED): response is in the cache; request can be either allowed or denied 10 / 24

11 Exception statistics Table : Statistics of different decisions and exceptions in the D sample dataset. sc-filter-result x-exception-id Classification Sample (D sample ) OBSERVED Allowed 30,140,158 (93.28%) PROXIED (total) Proxied 151,554 (0.47%) DENIED (total) Denied 2,019,246 (6.25%) tcp error 947,083 (2.93%) internal error 636,335 (1.97%) invalid request 115,297 (0.36%) unsupported protocol 28,769 (0.09%) dns unresolved hostname Error 6,247 (0.02%) dns server failure 2,235 (0.01%) unsupported encoding 6 (0.00%) invalid response 1 (0.00%) policy denied 283,197 (0.88%) Censored policy redirect 76 (0.00%) 11 / 24

12 Top allowed and censored domains Table : Top-10 allowed Domains. Domain # Of Requests Percentage google.com 2.26M 7.51% gstatic.com 1.03M 3.44% xvideos.com 876, % facebook.com 769, % microsoft.com 740, % fbcdn.net 654, % windowsupdate.com 652, % google-analytics.com 553, % doubleclick.net 518, % msn.com 498, % ytimg.com 470, % mediafire.com 392, % yahoo.com 320, % Table : Top-10 censored Domains. Domain # Of Requests Percentage facebook.com 68, % skype.com 23, % metacafe.com % live.com 18, % google.com 18, % zynga.com 16, % yahoo.com 16, % wikimedia.org 13, % fbcdn.net 12, % ceipmsn.com 6, % conduitapps.com 5, % msn.com 3, % conduit.com 3, % 12 / 24

13 Types of censored requests % Of Request Streaming Media Search Engines Internet Services Content Server Portal Sites Games Instant Messaging Education/Reference General News Other NA Anonymizers Business Social Networking Software/Hardware Online Shopping Shareware/Freeware P2P/File Sharing Entertainment Government/Military Figure : Distribution of censored traffic (D sample ) using categories from McAfee s TrustedSource. NA denotes not available, and Other is used for categories with less than 1K requests. 13 / 24

14 Understanding the Censorship Censorship seems to be based on: 1 Category 2 URL 3 keywords 4 IP Two censorship actions: 1 Simple denied 2 Request redirection Analysis requires: Reverse engineering Identifying keywords/url/ips used to filter requests 14 / 24

15 Category-based Filtering Category-based Filtering: Relate to the cs-categories field (not McAfee s) Classification of the requested page Local database or online tool Only three categories found in the logs: Unknown; None; Blocked Site Blocked Site: A custom-made category Targets specific Facebook pages Triggers redirection of the request 15 / 24

16 URL and keyword based filtering I Table : Top-10 domains suspected to be censored (from D sample ). Domain Censored Allowed Proxied skype.com 23,558 (8.32%) 0 (0.00%) 39 (0.03%) metacafe.com 19,257 (6.80%) 0 (0.00%) 49 (0.03%) wikimedia.org 13,506 (4.77%) 0 (0.00%) 143 (0.09%).il 2,609 (0.92%) 0 (0.00%) 370 (0.24% ) amazon.com 2,356 (0.83%) 0 (0.00%) % aawsat.com 2,180 (0.77%) 0 (0.00%) 230 (0.15%) jumblo.com 1,158 (0.41%) 0 (0.00%) 0 (0.00%) jeddahbikers.com 907 (0.32%) 0 (0.00%) 5 (0.00%) islamway.com 702 (0.25%) 0 (0.00%) 16 (0.01%) badoo.com 614 (0.22%) 0 (0.00%) 25 (0.02%) Media sharing/ IM / Chat : skype.com, metacafe, wikimedia.org Social network: badoo.com Political/Religious/News: islamway.com, aawsat.com,... Shopping: amazon.com 16 / 24

17 URL and keyword based filtering II Table : The list of 5 keywords identified as censored (in D sample ). Keyword Censored Allowed Proxied proxy 194,539 (68.68%) 0 (0.00%) 1,106 (0.73%) hotspotshield 5,846 (2.06%) 0 (0.00%) 24 (0.02%) ultrareach 2,290 (0.81%) 0 (0.00%) 436 (0.29%) israel 2,267 (0.80%) 0 (0.00%) 25 (0.02%) ultrasurf 2,073 (0.73%) 0 (0.00%) 468 (0.31%) Filtering anti-censorship technologies and Israel Proxy triggers many false positives Facebook: like buttons, social plugins,... xhamster: ad proxy.php 17 / 24

18 IP-based filtering I Table : Censorship ratio for top censored countries in D IPv4. Country Censorship # Censored # Allowed Ratio (%) Israel ,191 72,416 Kuwait Russian Federation ,161 United Kingdom , ,387 Netherlands ,206 7,077,371 Singapore ,768 Bulgaria ,786 Israel is by far the most IP-based censored country 18 / 24

19 Social Media Censorship Table : Top-10 censored social networks in D sample. Social network # Censored # Allowed # Proxied facebook.com 68,782 (24.28%) 769,555 (2.55%) 3,942 (2.60%) badoo.com 614 (0.22%) 0 (0.00%) 25 (0.02%) netlog.com 438 (0.15%) 0 (0.00%) 100 (0.07%) linkedin.com 308 (0.11%) 7,019 (0.02%) 75 (0.05%) hi5.com 124 (0.04%) 9,301 (0.03%) 20 (0.01%) skyrock.com 117 (0.04%) 270 (0.00%) 3 (0.00%) twitter.com 7 (0.00%) 115,502 (0.38%) 585 (0.39%) livejournal.com 1 (0.00%) 818 (0.00%) 0 (0.00%) ning.com 1 (0.00%) 1,886 (0.01%) 5 (0.00%) last.fm 0 (0.00%) 1,777 (0.01%) 1 (0.00%) Censored: badoo.com, netlog.com Not censored : facebook.com, twitter.com, linkedin.com, / 24

20 Facebook Table : Top Facebook pages of the Blocked Site category in D full. Facebook page # Censored # Allowed # Proxied Syrian.Revolution Syrian.revolution syria.news.f.n.n ShaamNews fffm barada.channel DaysOfRage Syrian.R.V YouthFreeSyria sooryoon Freedom.Of.Syria SyrianDayOfRage Censored Facebook pages related to Syrian Revolution 20 / 24

21 Anti-censorship technology in use Tor Tor relays not censored Majority of the Tor traffic allowed Reportedly blocked in 2012 Proxies & VPN Peer-to-Peer Networks (e.g., BitTorrent) Used to download censored software, such as Skype Google Cache Used to access censored sites 21 / 24

22 Ethics Consideration (revisited) Even though data was public, we are aware of its sensitivity, thus we: Encrypted all data (and backups) at rest Did not re-distribute the logs Never obtained/extracted users personal information, but only analyzed aggregated traffic statistics Obtained IRB approval from INRIA and NICTA Out of the scope to discuss ethics of leaked data for research purposes (see Egelman et al. s FC11 panel) Our work provides an accurate view/comprehensive analysis of large-scale censorship, thus, can be beneficial to entities on either side of censorship, but: We believe it helps understand the technical aspects of an actual censorship ecosystem, underlying technologies, policies, etc... We hope it can facilitate the design of censorship-evading tools 22 / 24

23 Conclusion 1/2 Presented a measurement-based analysis of Internet filtering in Syria, based on 600GB worth of logs from 7 Blue Coat filtering proxies Provided a detailed, first-of-a-kind snapshot of Syrian censorship practices Uncovered the presence of a relatively stealthy yet quite targeted filtering, which relied on 1 IP addresses to block access to entire subnets 2 Domains to block specific websites 3 Keywords to target specific content 23 / 24

24 Conclusion 2/2 We discovered: Keyword-based censorship produces some collateral damage (e.g., blocking all requests w/ keyword proxy ) IM heavily censored Social media censorship limited to specific pages Some ill-configured rules Anti-censorship technologies going through Some rules blocked a lot of ads Filtering architecture has evolved since 2011 New appliances have been purchased Tor has been blocked Tech Report: (And don t forget PETS, 24 / 24