Firewall on Demand Multidomain

Firewall on Demand Multidomain S E C U R I T Y V I A B G P F L O W S P E C & A W E B P L A T F O R M Leonidas Poulopoulos GRNET NOC Wayne Routly DANTE Jeffrey Haas JUNIPER Firewall on Demand Multidomain Internet2 Global Summit, Apr 9 2014

Firewall on Demand S E C U R I T Y V I A B G P F L O W S P E C & A W E B P L A T F O R M L e o n i d a s P o u l o p o u l o s l e o p o u l @ n o c. g r n e t. g r G R N E T N O C (@l e o p o u l ) Firewall on Demand Multidomain Internet2 Global Summit, Apr 9 2014

GRNET NOC Staff: 15 Network: 120 devices (40 routers/80 switches) Juniper-based network Presence: 90 cities Clients: ~100 Upstream: GÉANT Firewall on Demand Multidomain 3 Internet2 Global Summit, Apr 9 2014

DDoS Illustrated DDoS attack launched from compromised systems (bots) IX UPSTREAM NREN Victim DDoS attack traffic consumes network capacity DDoS attack targets applications and services Firewall on Demand Multidomain 4 Internet2 Global Summit, Apr 9 2014

DDoS facts 400 Gbps 309 <1 1 3 10 17 24 40 49 100 60 60 02 03 04 05 06 07 08 09 10 11 12 13 14 Source: Arbor Networks Inc. & Cloudflare Firewall on Demand Multidomain 5 Internet2 Global Summit, Apr 9 2014

Staying alive acls, firewall filters RTBH BGP flowspec Firewall on Demand Multidomain 6 Internet2 Global Summit, Apr 9 2014

BGP FLOWSPEC IETF AND JUNIPER ROADMAP Jeffrey Haas <jhaas@juniper.net>

BGP FLOWSPEC BGP Flowspec was originally defined in RFC 5575 and has been part of JUNOS since version 7.3. It permits layer 4 (TCP and UDP) firewall filters to be distributed in BGP on both a intradomain and inter-domain basis. Flowspec was originally defined to assist in mitigation of DDoS attacks. Deployments may use native configuration to distribute the filters. Several DDoS mitigation environments will generate the filters in support of their detection and mitigation tools. 8 Copyright 2014 Juniper Networks, Inc. www.juniper.net

CURRENT IETF WORK draft-ietf-idr-bgp-flowspec-oid Formally permits IBGP origination of BGP flowspec routes without requiring a longest-match for validation. In practice, operators have been using policy knobs to permit similar behaviors for nonebgp originated flowsec. draft-haas-idr-flowspec-redirect-rt-bis Clarifies some issues in RFC 5575 for the Redirect to VRF Route- Target. As currently documented, it s not possible to have a fully compatible BGP Flowspec implementation. 9 Copyright 2014 Juniper Networks, Inc. www.juniper.net

CURRENT IETF WORK draft-ietf-idr-flowspec-redirect-ip adds some exciting features to BGP flowspec: Permit redirection of traffic to a specific IP address rather than requiring tunneling via VRF. Permit the copying of traffic in a similar fashion. Some issues with the feature encoding and precedence of rules are being worked out currently. New draft expected soon. draft-ietf-idr-flow-spec-v6 Provide for support for IPv6 in flowspec. Necessary changes include: (Limited) Support for Next Header. Flow Label support Ambiguous case of Traffic Class with regard to ECN still under debate. 10 Copyright 2014 Juniper Networks, Inc. www.juniper.net

INTO THE REALM OF SPECULATIVE FICTION BGP Flowspec provides a convenient encoding mechanism to permit Layer3+ traffic filters be distributed. Future facing work, such as Software Defined Networking (SDN), Service Chaining/Network Function Virtualization or Interface to the Routing System (I2RS) may be able to leverage flowspec as a mechanism to distribute custom forwarding behaviors. 12 Copyright 2014 Juniper Networks, Inc. www.juniper.net

BGP community flow vs. RTBH vs. ACLs ACLS Distributed across the network Closer to the source Fine-grained even on core/backbone networks Multidomain easy propagation towards the upstream via BGP Easy automation & integration BGP RTHB Flowspec: enhancement of RTBH Does not affect all traffic to victim Less coarse More actions Separate NLRI Firewall on Demand Multidomain 13 Internet2 Global Summit, Apr 9 2014

Firewall on Demand NEED FOR BETTER TOOLS TO MITIGATE TRANSIENT ATTACKS GRANULARITY: Per-flow level ACTION: Drop, rate-limit, redirect SPEED: 1-2 orders of magnitude quicker EFFICIENCY: closer to the source, multi-domain AUTOMATION: integration with other systems MANAGEABILITY: status tracking, web interface Firewall on Demand Multidomain 14 Internet2 Global Summit, Apr 9 2014

FoD Architecture Shibboleth Django MVC User Interface Long Polling (Gevent) Job Queue (Celery/Beanstalk) OPEN SOURCE https://code.grnet.gr/projects/flowspy http://flowspy.readthedocs.org https://fod.grnet.gr Caching Layer (Memcached) Network Config to XML proxy (nxpy) Python NETCONF client (ncclient) NETCONF ebgp ebgp ibgp ibgp Firewall on Demand Multidomain 15 Internet2 Global Summit, Apr 9 2014

FoD Screenshots more during demo Firewall on Demand Multidomain 16 Internet2 Global Summit, Apr 9 2014

How it works Single domain Customer s NOC logs in web tool (shibboleth) & describes flows and actions Destination validated against customer s IP space A dedicated router is configured (NETCONF) to advertise the route via BGP flowspec Dynamic firewall filters are implemented on all routers Attack is mitigated upon entrance End of attack: Removal via the tool, or auto-expire Firewall on Demand Multidomain 17 FoD UPSTREAM IX GRNET Client Client Web ebgp NETCONF ibgp Internet2 Global Summit, Apr 9 2014

GRNET FoD usage examples 2.5years 20Tbytes 100rules 40users 20peers Firewall on Demand Multidomain 18 Internet2 Global Summit, Apr 9 2014

What now? Idea! BGP is by nature MULTIDOMAIN Deploy FoD in a MULTIDOMAIN Environment GÉANT and its peering NRENs Firewall on Demand Multidomain 19 Internet2 Global Summit, Apr 9 2014

Firewall on Demand A Multi-Domain Implementation Wayne Routly Security Manager DANTE connect communicate collaborate

GÉANT : Who What How Pan-European Network..Transit Network.ISP 30 Physical Pops 50,000 km network infrastructure on 44 routes 100Gb/s 100s TB of Data 15+ Million IPs 100+ Workstations Truly Global (50 million users) 10,000 institutions Interconnects European NRENs - 40 Commercial & Commodity Traffic connect communicate collaborate 21

Today Little bit of DDoS on the side.. NTP, DNS, SMTP. Amplification Attacks 2k DDoS Events (183 pm) 298 vs 929.. 1k in 2014, average 300 connect communicate collaborate 22

Today DDoS Events CyNet Target: The University of Cyprus (www.ucy.ac.cy) Port Ranges: 0, 2070 and 3475 Multiple Source IP s and source AS s. Attack peak: Over 13G over 1G link connect communicate collaborate 23

Today DDoS Events CyNet [2] Destination AS 3268 Traffic Date Seen Dst IP Addr Flows (%) Packets (%) Bytes (%) 2013-09-02 04:58 194.42.x.x 124919(97.2) 440.6 M(99.2) 517.4 G(99.5) 2013-09-02 04:59 82.116.x.x 129( 0.1) 143000( 0.0) 154.3 M( 0.0) 2013-09-02 05:00 194.42.x.x 128( 0.1) 244000( 0.1) 12.3 M( 0.0) 2013-09-02 04:59 194.42.x.x 114( 0.1) 57000( 0.0) 10.5 M( 0.0) 2013-09-02 04:59 82.116.x.x 90( 0.1) 239500( 0.1) 311.4 M( 0.1) 2013-09-02 04:59 194.42.x.x 81( 0.1) 40500( 0.0) 8.7 M( 0.0) Destination ports for 194.42.x.x Date Seen Dst Port Flows (%) Packets (%) Bytes (%) 2013-09-02 04:58 2070 47268(37.8) 144.2 M(32.7) 182.4 G(35.3) 2013-09-02 04:58 0 46315(37.1) 260.0 M(59.0) 295.4 G(57.1) 2013-09-02 04:58 3475 29714(23.8) 31.3 M( 7.1) 39.2 G( 7.6) 2013-09-02 04:58 771 1348( 1.1) 4.3 M( 1.0) 243.6 M( 0.0) 2013-09-02 04:58 769 145( 0.1) 516000( 0.1) 29.0 M( 0.0) 2013-09-02 04:58 2816 55( 0.0) 199500( 0.0) 16.7 M( 0.0) 2013-09-02 04:58 1024 30( 0.0) 114500( 0.0) 6.4 M( 0.0) connect communicate collaborate 24

Today DDoS Events GRNET DNS Amplification Attack Target: GRNET Port Ranges: 53 (DNS) Multiple Source IP s & Source AS s. Attack peak: 20G over 10G link Date first seen Dst IP Addr Flows (%) Packets (%) Bytes (%) 2013-03-13 09:34 194.177.211.x 35531( 7.8) 36.1 M(11.3) 53.5 G(11.9) 2013-03-13 09:34 194.177.211.x 34632( 7.6) 35.6 M(11.1) 52.6 G(11.7) 2013-03-13 09:33 194.177.211.x 34469( 7.6) 35.3 M(11.1) 52.2 G(11.6) 2013-03-13 09:33 194.63.239.x 49621(11.0) 31.8 M(10.0) 44.3 G( 9.9) 2013-03-13 09:33 194.63.239.x 48220(10.6) 27.1 M( 8.5) 36.7 G( 8.2) 2013-03-13 09:33 194.63.239.x 39278( 8.7) 26.1 M( 8.2) 36.5 G( 8.1) connect communicate collaborate 25

Uhm..Now What connect communicate collaborate

Today Security Changes - Audits connect communicate collaborate

Strategy security solutions that simplify the improvement of the security status quo connect communicate collaborate 28

Requirements - Defining It must be easy to use It must be ENHANCE security Must deliver MEASURABLE VALUE REDUNDANCY must be incorporated into existing processes accepted by all participants. conform to BEST PRACTICES & STANDARDS Must be SCALABLE. connect communicate collaborate 29

GÉANT Security Complete Security Solution - NSHaRP It is a mechanism to quickly and effectively inform affected users of incidents detected transiting the GÉANT network dynamically. It adds value by serving as an extension to an NRENs CERT, by adding visibility to incidents targeting or originating from your network Innovative and Unique - Caters for different types of requirements.is a process that will enhance GÉANT backbone security and will extend the NRENs ability to protect their infrastructure. connect communicate collaborate 30

Firewall on Demand But Why? better tools to mitigate transitory attacks and anomalies Better in terms of Granularity: Per-flow level Source/Dest IP/Ports, protocol type, DSCP, TCP flag Action: Drop, rate-limit, redirect Speed: More responsive (Seconds / Minutes vs. Hours / Days) Efficiency: Closer to the source, Multi Domain Automation: Integration with other systems (NSHaRP) Manageability connect communicate collaborate 31

Firewall on Demand Tomorrow NSHaRP Customer or GN NOC logs into web tool and describes flows and actions Flow destination is validated against the customer s IP space Dedicated router is configured to advertise the route via BGP flowspec ibgp propagates the tuples to all GEANT routers. Dynamic firewall filters are implemented on all routers Attack is mitigated (dropped, rated-limited) upon entrance End of attack: Removal via the tool, or auto-expire NREN A Credit: Andreas Polyrakis, GRNET FoD LEVEL3 GEANT NREN B Customer connect communicate collaborate 32

Firewall on Demand Roadmap Phase 1 - Test Flow Spec on GN Athens Router - Test Propagation to GN Gateways Phase 2 - Deploy Flow Spec Server - Web Interface - Pilot Phase 2 (b) - Processes - API - Production Service Today 6 Months 12 Months connect communicate collaborate 33

GÉANT Tests GÉANT Flowspec CARNet Attacker Flowspec Flowspec GRNET FoD Victim Click Apply 6 seconds later Firewall on Demand Multidomain 34 Internet2 Global Summit, Apr 9 2014

FoD multidomain principles FoD setup & deploy by every interested domain/nren Multidomain FoD deployed in GÉANT Multidomain FoD authentication: edugain Multidomain FoD authorization: peer address space GÉANT accepts BGP flowspec rules from domains Policies/filters per peering based on rule dest. addr. User belongs to a domain/institution/nren :: Peer Peer is assigned an administrative IPv4 address space Rule creation with destination address/network only inside the user s Peer assigned address space Firewall on Demand Multidomain 35 Internet2 Global Summit, Apr 9 2014

FoD multidomain deployment scenarios Possible mitigation with RTBH, ACL ACL Flowspec RTBH GÉANT Flowspec NREN Victim Flowspec Flowspec m FoD Flowspec Flowspec FoD Legitimate Traffic Flows Malicious Traffic Flows Flow spec rule propagation BGP Peering Flow spec rules Firewall on Demand platform Flowspec NREN Flowspec Flowspec FoD Firewall on Demand Multidomain 36 Internet2 Global Summit, Apr 9 2014

Current Status GRNET in production since end of 2011 Tests: Multihop BGP peering with PSNC Interest/Evaluation from BELNET GÉANT BGP flowspec enabled in all core devices Successful tests between GRNET and GÉANT Multiple scenarios tested Iperf between Croatia and Greece Gone in 6 seconds In production by April 2015 Firewall on Demand Multidomain 37 Internet2 Global Summit, Apr 9 2014

Extensions FoD {single,multi}-domain interfaces to other tools/platforms REST API XMPP client/server ØMQ extensions Filter counters/graphs NETCONF Juniper UtilityMIB Ipv6 support (Whenever available) Firewall on Demand Multidomain 38 Internet2 Global Summit, Apr 9 2014

Can I deploy/try/test it? Open source project FoD : https://code.grnet.gr/projects/flowspy Docs: https://flowspy.readthedocs.org Ask for a demo account PEER WITH US! Firewall on Demand Multidomain 39 Internet2 Global Summit, Apr 9 2014

Demo time attaaaaack! Firewall on Demand Multidomain 40 Internet2 Global Summit, Apr 9 2014

Questions? 42: The Answer to the Ultimate Question of Life, The Universe, and Everything. Douglas Adams, The Hitchhiker's Guide to the Galaxy Firewall on Demand Multidomain 41 Internet2 Global Summit, Apr 9 2014

Thank you Leonidas Poulopoulos GRNET NOC Wayne Routly DANTE Jeffrey Haas JUNIPER Firewall on Demand Multidomain Internet2 Global Summit, Apr 9 2014