Data Transfer Network (TPN)

Transcription

1 The Science DMZ: A Network Design Pa8ern for Data- Intensive Science Jason Zurawski zurawski@es.net Science Engagement Engineer, ESnet Lawrence Berkeley National Laboratory New Mexico Technology in EducaCon (NMTIE) November 19 th, 2014

2 Overview ESnet Overview Science DMZ MoCvaCon and IntroducCon Science DMZ Architecture Network Monitoring Data Transfer Nodes & ApplicaCons Science DMZ Security User Engagement Wrap Up 2 ESnet Science Engagement (engage@es.net) - 11/19/14

3 SC Supports Research at More than 300 Institutions Across the U.S ESnet at a Glance High- speed naconal network, opcmized for DOE science missions: conneccng 40 labs, plants and facilices with >100 networks $32.6M in FY14, 42FTE older than commercial Internet, growing twice as fast $62M ARRA grant for 100G upgrade: Universities DOE laboratories The Office of Science supports: 27,000 Ph.D.s, graduate students, undergraduates, engineers, and technicians 26,000 users of open-access facilities 300 leading academic institutions 17 DOE laboratories transicon to new era of opccal networking world s first 100G network at concnental scale 8 Culture of urgency: 4 awards in past 3 years R&D100 Award in FY13 5 out of 5 for customer sacsfaccon in last review Dedicated staff to support the mission of science 3 ESnet Science Engagement (engage@es.net) - 11/19/14

4 4 ESnet Science Engagement - 11/19/14

5 Network as Infrastructure Instrument US R&E (DREN/Internet2/NLR) CANADA (CANARIE) ASIA-PACIFIC (ASGC/Kreonet2/ TWAREN) RUSSIA AND CHINA (GLORIAD) CANADA (CANARIE) LHCONE FRANCE (OpenTransit) CERN (USLHCNet) ASIA-PACIFIC (KAREN/KREONET2/ NUS-GP/ODN/ REANNZ/SINET/ TRANSPAC/TWAREN) SEATTLE PNNL RUSSIA AND CHINA (GLORIAD) ASIA-PACIFIC (BNP/HEPNET) AUSTRALIA (AARnet) LATIN AMERICA CLARA/CUDI SUNNYVALE ASIA-PACIFIC (ASCC/KAREN/ KREONET2/NUS-GP/ ODN/REANNZ/ SINET/TRANSPAC) LBNL SACRAMENTO SLAC BOISE US R&E (DREN/Internet2/ NASA) US R&E (NASA/NISN/ USDOI) DENVER US R&E (DREN/Internet2/ NISN/NLR) AMES CHICAGO KANSAS CITY FNAL ANL BOSTON BNL NEW YORK PPPL WASHINGTON DC JLAB US R&E (Internet2/ NLR) CERN CANADA (CANARIE) EUROPE (GÉANT/ NORDUNET) ASIA-PACIFIC (SINET) ORNL AUSTRALIA (AARnet) ALBUQUERQUE NASHVILLE EUROPE (GÉANT) ATLANTA LATIN AMERICA (AMPATH/CLARA) LATIN AMERICA (CLARA/CUDI) El PASO HOUSTON US R&E (DREN/Internet2/ NISN) Vision: ScienCfic progress will be completely unconstrained by the physical locacon of instruments, people, computaconal resources, or data. 5 ESnet Science Engagement (engage@es.net) - 11/19/14

6 Overview ESnet Overview Science DMZ MoCvaCon and IntroducCon Science DMZ Architecture Network Monitoring Data Transfer Nodes & ApplicaCons Science DMZ Security User Engagement Wrap Up 6 ESnet Science Engagement (engage@es.net) - 11/19/14

7 Mo8va8on Networks are an essencal part of data- intensive science Connect data sources to data analysis Connect collaborators to each other Enable machine- consumable interfaces to data and analysis resources (e.g. portals), automacon, scale Performance is criccal ExponenCal data growth Constant human factors Data movement and data analysis must keep up EffecCve use of wide area (long- haul) networks by sciencsts has historically been difficult 7 ESnet Science Engagement (engage@es.net) - 11/19/14

8 Tradi8onal Big Science 8 ESnet Science Engagement (engage@es.net) - 11/19/14

9 Big Science Now Comes in Small Packages 9 ESnet Science Engagement (engage@es.net) - 11/19/14

10 Understanding Data Trends 100PB 10PB 1PB Small collaboration scale, e.g. light and neutron sources Medium collaboration scale, e.g. HPC codes A few large collaborations have internal software and networking organizations Data Scale 100TB 10TB Large collaboration scale, e.g. LHC 1TB 100GB 10GB Collaboration Scale 10 ESnet Science Engagement (engage@es.net) - 11/19/14

11 Data Mobility in a Given Time Interval This table available at: 11 ESnet Science Engagement (engage@es.net) - 11/19/14

12 The Central Role of the Network The very structure of modern science assumes science networks exist: high performance, feature rich, global scope What is The Network anyway? The Network is the set of devices and applicacons involved in the use of a remote resource This is not about supercomputer interconnects This is about data flow from experiment to analysis, between facilices, etc. User interfaces for The Network portal, data transfer tool, workflow engine Therefore, servers and applicacons must also be considered What is important? Ordered list: 1. Correctness 2. Consistency 3. Performance 12 ESnet Science Engagement (engage@es.net) - 11/19/14

13 TCP Ubiquitous and Fragile Networks provide conneccvity between hosts how do hosts see the network? From an applicacon s perspeccve, the interface to the other end is a socket CommunicaCon is between applicacons mostly over TCP TCP the fragile workhorse TCP is (for very good reasons) Cmid packet loss is interpreted as congescon Packet loss in conjunccon with latency is a performance killer Like it or not, TCP is used for the vast majority of data transfer applicacons (more than 95% of ESnet traffic is TCP) 13 ESnet Science Engagement (engage@es.net) - 11/19/14

14 A small amount of packet loss makes a huge difference in TCP performance Local (LAN) With loss, high performance beyond metro distances is essentially impossible InternaConal Metro Area Regional ConCnental Measured (TCP Reno) Measured (HTCP) Theoretical (TCP Reno) Measured (no loss) 14 ESnet Science Engagement (engage@es.net) - 11/19/14

15 Working With TCP In Prac8ce Far easier to support TCP than to fix TCP People have been trying to fix TCP for years limited success Like it or not we re stuck with TCP in the general case PragmaCcally speaking, we must accommodate TCP Sufficient bandwidth to avoid congescon Zero packet loss Verifiable infrastructure Networks are complex Must be able to locate problems quickly Small footprint is a huge win small number of devices so that problem isolacon is tractable 15 ESnet Science Engagement (engage@es.net) - 11/19/14

16 PuMng A Solu8on Together EffecCve support for TCP- based data transfer Design for correct, consistent, high- performance operacon Design for ease of troubleshoocng Easy adopcon is criccal Large laboratories and universices have extensive IT deployments DrasCc change is prohibicvely difficult Cybersecurity defensible without compromising performance Borrow ideas from tradiconal network security TradiConal DMZ Separate enclave at network perimeter ( Demilitarized Zone ) Specific locacon for external- facing services Clean separacon from internal network Do the same thing for science Science DMZ 16 ESnet Science Engagement (engage@es.net) - 11/19/14

17 The Science DMZ Superfecta Engagement Partnerships EducaCon & ConsulCng Resources & Knowledgebase Engagement with Network Users Data Transfer Node High performance Configured for data transfer Proper tools Dedicated Systems for Data Transfer Performance TesCng & Measurement perfsonar Enables fault isolacon Verify correct operacon Widely deployed in ESnet and other networks, as well as sites and facilices Network Architecture Science DMZ Dedicated locacon for DTN Proper security Easy to deploy - no need to redesign the whole network 17 ESnet Science Engagement (engage@es.net) - 11/19/14 17 ESnet Science Engagement 2014, (engage@es.net) Energy Sciences - Network 11/19/14

18 Overview ESnet Overview Science DMZ MoCvaCon and IntroducCon Science DMZ Architecture Network Monitoring Data Transfer Nodes & ApplicaCons Science DMZ Security User Engagement Wrap Up 18 ESnet Science Engagement - 11/19/14

19 Abstract or Prototype Deployment Add- on to exiscng network infrastructure All that is required is a port on the border router Small footprint, pre- produccon commitment Easy to experiment with components and technologies DTN prototyping perfsonar tescng Limited scope makes security policy excepcons easy Only allow traffic from partners Add- on to produccon infrastructure lower risk 19 ESnet Science Engagement (engage@es.net) - 11/19/14

20 Science DMZ Design PaPern (Abstract) Border Router perfsonar Enterprise Border Router/Firewall WAN 10G 10GE perfsonar Clean, High-bandwidth WAN path 10GE Site / Campus access to Science DMZ resources Science DMZ Switch/Router 10GE Site / Campus LAN 10GE High performance Data Transfer Node with high-speed storage Per-service security policy control points perfsonar 20 ESnet Science Engagement (engage@es.net) - 11/19/14

21 Local And Wide Area Data Flows Border Router perfsonar Enterprise Border Router/Firewall WAN 10G 10GE perfsonar Clean, High-bandwidth WAN path 10GE Site / Campus access to Science DMZ resources Science DMZ Switch/Router 10GE Site / Campus LAN 10GE High performance Data Transfer Node with high-speed storage Per-service security policy control points perfsonar High Latency WAN Path Low Latency LAN Path 21 ESnet Science Engagement (engage@es.net) - 11/19/14

22 Support For Mul8ple Projects Science DMZ architecture allows mulcple projects to put DTNs in place Modular architecture Centralized locacon for data servers This may or may not work well depending on insctuconal policcs Issues such as physical security can make this a non- starter On the other hand, some shops already have service models in place On balance, this can provide a cost savings it depends Central support for data servers vs. carrying data flows How far do the data flows have to go? 22 ESnet Science Engagement (engage@es.net) - 11/19/14

23 Mul8ple Projects Border Router Enterprise Border Router/Firewall WAN 10G 10GE perfsonar Clean, High-bandwidth WAN path 10GE Site / Campus access to Science DMZ resources Science DMZ Switch/Router 10GE Site / Campus LAN Project A DTN Project B DTN Per-project security policy control points perfsonar Project C DTN 23 ESnet Science Engagement (engage@es.net) - 11/19/14

24 Supercomputer Center Deployment High- performance networking is assumed in this environment Data flows between systems, between systems and storage, wide area, etc. Global filesystem oqen Ces resources together PorCons of this may not run over Ethernet (e.g. IB) ImplicaCons for Data Transfer Nodes Science DMZ may not look like a discrete encty here By the Cme you get through interconneccng all the resources, you end up with most of the network in the Science DMZ This is as it should be the point is appropriate deployment of tools, configuracon, policy control, etc. Office networks can look like an aqerthought, but they aren t Deployed with appropriate security controls Office infrastructure need not be sized for science traffic 24 ESnet Science Engagement (engage@es.net) - 11/19/14

25 Supercomputer Center Border Router Firewall WAN Routed perfsonar Offices perfsonar Virtual Circuit Core Switch/Router Front end switch Front end switch perfsonar Data Transfer Nodes Supercomputer Parallel Filesystem 25 ESnet Science Engagement - 11/19/14

26 Supercomputer Center Data Path Border Router Firewall WAN Routed perfsonar Offices perfsonar Virtual Circuit Core Switch/Router Front end switch Front end switch perfsonar Data Transfer Nodes Supercomputer Parallel Filesystem High Latency WAN Path Low Latency LAN Path High Latency VC Path 26 ESnet Science Engagement - 11/19/14

27 Development Environment One thing that oqen happens is that an early power user of the Science DMZ is the network engineering group that builds it Service prototyping Deployment of test applicacons for other user groups to demonstrate value The produccon Science DMZ is just that produccon Once users are on it, you can t take it down to try something new Stuff that works tends to a8ract workload Take- home message: plan for mul=ple Science DMZs from the beginning at the very least you re going to need one for yourself The Science DMZ model easily accommodates this 27 ESnet Science Engagement (engage@es.net) - 11/19/14

28 Science DMZ Flexible Design PaPern The Science DMZ design pa8ern is highly adaptable to research Deploying a research Science DMZ is straighrorward The basic elements are the same Capable infrastructure designed for the task Test and measurement to verify correct operacon Security policy well- matched to the environment, applicacon set is strictly limited to reduce risk Connect the research DMZ to other resources as appropriate The same ideas apply to supporcng an SDN effort Test/research areas for development TransiCon to produccon as technology matures and need dictates One possible trajectory follows 28 ESnet Science Engagement (engage@es.net) - 11/19/14

29 Science DMZ Separate SDN Connec8on Border Router perfsonar Enterprise Border Router/Firewall WAN High performance routed path Site / Campus access to Science DMZ resources perfsonar SDN Science DMZ Switch/Router SDN SDN Path Production Science DMZ Switch/Router Site / Campus LAN perfsonar Research DTN Per-service security policy control points Science DMZ Connections perfsonar Production DTN 29 ESnet Science Engagement (engage@es.net) - 11/19/14

30 Science DMZ Produc8on SDN Connec8on Border Router perfsonar Enterprise Border Router/Firewall WAN perfsonar Research Science DMZ Switch/Router SDN High performance routed path SDN Path Site / Campus access to Science DMZ resources Production SDN Science DMZ Switch/Router Site / Campus LAN perfsonar Research DTN Per-service security policy control points Science DMZ Connections perfsonar Production DTN 30 ESnet Science Engagement (engage@es.net) - 11/19/14

31 Science DMZ SDN Campus Border Border Router perfsonar Enterprise Border Router/Firewall WAN perfsonar Research Science DMZ Switch/Router High performance multi-service path Site / Campus access to Science DMZ resources Production SDN Science DMZ Switch/Router Site / Campus LAN perfsonar Research DTN Per-service security policy control points Science DMZ Connections perfsonar Production DTN 31 ESnet Science Engagement (engage@es.net) - 11/19/14

32 Common Threads Two common threads exist in all these examples AccommodaCon of TCP Wide area porcon of data transfers traverses purpose- built path High performance devices that don t drop packets Ability to test and verify When problems arise (and they always will), they can be solved if the infrastructure is built correctly Small device count makes it easier to find issues MulCple test and measurement hosts provide mulcple views of the data path perfsonar nodes at the site and in the WAN perfsonar nodes at the remote site 32 ESnet Science Engagement (engage@es.net) - 11/19/14

33 Mul8ple Ingress Flows, Common Egress Hosts will typically send packets at the speed of their interface (1G, 10G, etc.) Instantaneous rate, not average rate If TCP has window available and data to send, host sends uncl there is either no data or no window Hosts moving big data (e.g. DTNs) can send large bursts of back- to- back packets This is true even if the average rate as measured over seconds is slower (e.g. 4Gbps) On microsecond Cme scales, there is oqen congescon Router or switch must queue packets or drop them 10GE 10GE DTN traffic with wire-speed bursts Background traffic or competing bursts 10GE 33 ESnet Science Engagement - 11/19/14

34 Router and Switch Output Queues Interface output queue allows the router or switch to avoid causing packet loss in cases of momentary congescon In network devices, queue depth (or buffer ) is oqen a funccon of cost Cheap, fixed- config LAN switches (especially in the 10G space) have inadequate buffering. Imagine a 10G data center switch as the guilty party Cut- through or low- latency Ethernet switches typically have inadequate buffering (the whole point is to avoid queuing!) Expensive, chassis- based devices are more likely to have deep enough queues Juniper MX and Alcatel- Lucent 7750 used in ESnet backbone Other vendors make such devices as well - details are important Thx to Jim: h8p://people.ucsc.edu/~warner/buffer.html This expense is one driver for the Science DMZ architecture only deploy the expensive features where necessary 34 ESnet Science Engagement (engage@es.net) - 11/19/14

35 Output Queue Drops Common Loca8ons WAN Site Border Router Site Core Switch/Router 10GE 10GE Inbound data path Outbound data path Department Core Switch Department uplink to site core constrained by budget or legacy equipment 1GE 1GE Common locations of output queue drops for traffic outbound toward the WAN Wiring closet switch Common location of output queue drops for traffic inbound from the WAN Cluster data transfer node 10GE 10GE Outbound data path 1GE 1GE 1GE Department cluster switch 1GE 1GE 1GE Workstations 32+ cluster nodes 35 ESnet Science Engagement - 11/19/14

36 Overview ESnet Overview Science DMZ MoCvaCon and IntroducCon Science DMZ Architecture Network Monitoring Data Transfer Nodes & ApplicaCons Science DMZ Security User Engagement Wrap Up 36 ESnet Science Engagement - 11/19/14

37 Performance Monitoring Everything may funccon perfectly when it is deployed Eventually something is going to break Networks and systems are complex Bugs, mistakes, SomeCmes things just break this is why we buy support contracts Must be able to find and fix problems when they occur Must be able to find problems in other networks (your network may be fine, but someone else s problem can impact your users) TCP was intenconally designed to hide all transmission errors from the user: As long as the TCPs concnue to funccon properly and the internet system does not become completely parcconed, no transmission errors will affect the users. (From RFC793, 1981) 37 ESnet Science Engagement (engage@es.net) - 11/19/14

38 SoX Network Failures Hidden Problems Hard failures are well- understood Link down, system crash, soqware crash TradiConal network/system monitoring tools designed to quickly find hard failures Soq failures result in degraded capability ConnecCvity exists Performance impacted Typically something in the path is funcconing, but not well Soq failures are hard to detect with tradiconal methods No obvious single event SomeCmes no indicacon at all of any errors Independent tescng is the only way to reliably find soq failures 38 ESnet Science Engagement - 11/19/14

39 Sample SoX Failures Rebooted router with full route table normal performance Gradual failure of optical line card Gb/s degrading performance repair one month 39 ESnet Science Engagement - 11/19/14

40 Tes8ng Infrastructure perfsonar perfsonar is: A widely- deployed test and measurement infrastructure ESnet, Internet2, US regional networks, internaconal networks Laboratories, supercomputer centers, universices A suite of test and measurement tools A collaboracon that builds and maintains the toolkit By installing perfsonar, a site can leverage over 1100 test servers deployed around the world perfsonar is ideal for finding soq failures Alert to existence of problems Fault isolacon VerificaCon of correct operacon 40 ESnet Science Engagement (engage@es.net) - 11/19/14

41 perfsonar Deployment Footprint 41 ESnet Science Engagement - 11/19/14

42 Lookup Service Directory Search: hpp://stats.es.net/servicesdirectory/ 42 ESnet Science Engagement - 11/19/14

43 perfsonar Dashboard: hpp://ps- dashboard.es.net 43 ESnet Science Engagement - 11/19/14

44 Overview ESnet Overview Science DMZ MoCvaCon and IntroducCon Science DMZ Architecture Network Monitoring Data Transfer Nodes & ApplicaCons Science DMZ Security User Engagement Wrap Up 44 ESnet Science Engagement - 11/19/14

45 Dedicated Systems Data Transfer Node The DTN is dedicated to data transfer Set up specifically for high- performance data movement System internals (BIOS, firmware, interrupts, etc.) Network stack Storage (global filesystem, Fibrechannel, local RAID, etc.) High performance tools No extraneous soqware Limita=on of scope and func=on is powerful No conflicts with configuracon for other tasks Small applicacon set makes cybersecurity easier 45 ESnet Science Engagement - 11/19/14

46 Data Transfer Tools For DTNs Parallelism is important It is oqen easier to achieve a given performance level with four parallel conneccons than one conneccon Several tools offer parallel transfers, including Globus/GridFTP Latency interaccon is criccal Wide area data transfers have much higher latency than LAN transfers Many tools and protocols assume a LAN Workflow integracon is important Key tools: Globus Online, HPN- SSH 46 ESnet Science Engagement (engage@es.net) - 11/19/14

47 Data Transfer Tool Comparison In addicon to the network, using the right data transfer tool is criccal Data transfer test from Berkeley, CA to Argonne, IL (near Chicago). RTT = 53 ms, network capacity = 10Gbps. Tool Throughput scp: 140 Mbps HPN patched scp: 1.2 Gbps qp 1.4 Gbps GridFTP, 4 streams 5.4 Gbps GridFTP, 8 streams 6.6 Gbps Note that to get more than 1 Gbps (125 MB/s) disk to disk requires properly engineered storage (RAID, parallel filesystem, etc.) 47 ESnet Science Engagement (engage@es.net) - 11/19/14

48 Overview ESnet Overview Science DMZ MoCvaCon and IntroducCon Science DMZ Architecture Network Monitoring Data Transfer Nodes & ApplicaCons Science DMZ Security User Engagement Wrap Up 48 ESnet Science Engagement - 11/19/14

49 Science DMZ Security Goal disentangle security policy and enforcement for science flows from security for business systems RaConale Science data traffic is simple from a security perspeccve Narrow applicacon set on Science DMZ Data transfer, data streaming packages No printers, document readers, web browsers, building control systems, financial databases, staff desktops, etc. Security controls that are typically implemented to protect business resources oqen cause performance problems SeparaCon allows each to be opcmized 49 ESnet Science Engagement (engage@es.net) - 11/19/14

50 Performance Is A Core Requirement Core informacon security principles ConfidenCality, Integrity, Availability (CIA) Oqen, CIA and risk micgacon result in poor performance In data- intensive science, performance is an addiconal core mission requirement: CIA à PICA CIA principles are important, but if performance is compromised the science mission fails Not about how much security you have, but how the security is implemented Need a way to appropriately secure systems without performance compromises 50 ESnet Science Engagement (engage@es.net) - 11/19/14

51 Placement Outside the Firewall The Science DMZ resources are placed outside the enterprise firewall for performance reasons The meaning of this is specific Science DMZ traffic does not traverse the firewall data plane Packet filtering is fine just don t do it with a firewall Lots of heartburn over this, especially from the perspeccve of a convenconal firewall manager Lots of organizaconal policy direccves mandacng firewalls Firewalls are designed to protect converged enterprise networks Why would you put criccal assets outside the firewall??? The answer is that firewalls are typically a poor fit for high- performance science applicacons 51 ESnet Science Engagement (engage@es.net) - 11/19/14

52 Firewall Capabili8es and Science Traffic Firewalls have a lot of sophisccacon in an enterprise se}ng ApplicaCon layer protocol analysis (HTTP, POP, MSRPC, etc.) Built- in VPN servers User awareness Data- intensive science flows typically don t match this profile Common case data on filesystem A needs to be on filesystem Z Data transfer tool verifies credencals over an encrypted channel Then open a socket or set of sockets, and send data uncl done (1TB, 10TB, 100TB, ) One workflow can use 10% to 50% or more of a 10G network link Do we have to use a firewall? 52 ESnet Science Engagement (engage@es.net) - 11/19/14

53 Firewalls As Access Lists When you ask a firewall administrator to allow data transfers through the firewall, what do they ask for? IP address of your host IP address of the remote host Port range That looks like an ACL to me! No special config for advanced protocol analysis just address/port Router ACLs are be8er than firewalls at address/port filtering ACL capabilices are typically built into the router Router ACLs typically do not drop traffic permi8ed by policy 53 ESnet Science Engagement (engage@es.net) - 11/19/14

54 Security Without Firewalls Data intensive science traffic interacts poorly with firewalls Does this mean we ignore security? NO! We must protect our systems We just need to find a way to do security that does not prevent us from ge}ng the science done Key point security policies and mechanisms that protect the Science DMZ should be implemented so that they do not compromise performance Traffic permi8ed by policy should not experience performance impact as a result of the applicacon of policy 54 ESnet Science Engagement (engage@es.net) - 11/19/14

55 Firewall Performance Example Observed performance, via perfsonar, through a firewall: Almost 20 Cmes slower through the firewall Observed performance, via perfsonar, bypassing firewall: Huge improvement without the firewall 55 ESnet Science Engagement (engage@es.net) - 11/19/14

56 If Not Firewalls, Then What? Intrusion DetecCon Systems (IDS) One example is Bro h8p://bro- ids.org/ Bro is high- performance and ba8le- tested Bro protects several high- performance naconal assets Bro can be scaled with clustering: h8p:// ids.org/documentacon/cluster.html Other IDS solucons are available also Nerlow and IPFIX can provide intelligence, but not filtering Openflow and SDN Using Openflow to control access to a network- based service seems pre8y obvious This could significantly reduce the a8ack surface for any authenccated network service This would only work if the Openflow device had a robust data plane 56 ESnet Science Engagement (engage@es.net) - 11/19/14

57 If Not Firewalls, Then What? (2) Aggressive access lists More useful with project- specific DTNs If the purpose of the DTN is to exchange data with a small set of remote collaborators, the ACL is pre8y easy to write Large- scale data distribucon servers are hard to handle this way (but then, the firewall ruleset for such a service would be pre8y open too) LimitaCon of the applicacon set One of the reasons to limit the applicacon set in the Science DMZ is to make it easier to protect Keep desktop applicacons off the DTN (and watch for them anyway using logging, nerlow, etc take violacons seriously) This requires collaboracon between people networking, security, systems, and sciencsts 57 ESnet Science Engagement (engage@es.net) - 11/19/14

58 Collabora8on Within The Organiza8on All stakeholders should collaborate on Science DMZ design, policy, and enforcement The security people have to be on board Remember: security people already have policcal cover it s called the firewall If a host gets compromised, the security officer can say they did their due diligence because there was a firewall in place If the deployment of a Science DMZ is going to jeopardize the job of the security officer, expect pushback The Science DMZ is a strategic asset, and should be understood by the strategic thinkers in the organizacon Changes in security models Changes in operaconal models Enhanced ability to compete for funding Increased insctuconal capability greater science output 58 ESnet Science Engagement (engage@es.net) - 11/19/14

59 Overview ESnet Overview Science DMZ MoCvaCon and IntroducCon Science DMZ Architecture Network Monitoring Data Transfer Nodes & ApplicaCons Science DMZ Security User Engagement Wrap Up 59 ESnet Science Engagement - 11/19/14

60 Challenges to Network Adop8on Causes of performance issues are complicated for users. Lack of communicacon and collaboracon between the CIO s office and researchers on campus. Lack of IT expercse within a science collaboracon or experimental facility User s performance expectacons are low ( The network is too slow, I tried it and it didn t work ). Cultural change is hard ( we ve always shipped disks! ). ScienCsts want to do science not IT support The Capability Gap 60 ESnet Science Engagement (engage@es.net) - 11/19/14

61 Requirements Reviews h8p:// requirements/network- requirements- reviews/ The purpose of these reviews is to accurately characterize the near- term, medium- term and long- term network requirements of the science conducted by each program office. The reviews a8empt to bring about a network- centric understanding of the science process used by the researchers and sciencsts, to derive network requirements. We have found this to be an effec=ve method for determining network requirements for ESnet's customer base. 61 ESnet Science Engagement (engage@es.net) - 11/19/14

62 High Energy Physics Biological and Environmental Research Photo courtesy of LBL Photo courtesy of JGI Photo courtesy of NIST Advanced ScienCfic CompuCng Research Basic Energy Science Photo courtesy of LBL Nuclear Physics Fusion Energy Sciences Photo courtesy of SLAC Photo courtesy of PPPL 62 ESnet Science Engagement - 11/19/14

63 How do we know what our scien8sts need? Each Program Office has a dedicated requirements review every three years Two workshops per year, a8endees chosen by science programs Discussion centered on science case studies Instruments and FaciliCes the hardware Process of Science science workflow Collaborators Challenges Network requirements derived from science case studies + discussions Reports contain requirements analysis, case study text, outlook 63 ESnet Science Engagement (engage@es.net) - 11/19/14

64 2013 BER Sample Findings: Environmental Molecular EMSL frequently needs to ship physical copies of media to users when Sciences data sizes exceed a few GB. More often than not, this is due to lack of Laboratory bandwidth or storage resources at the user's home institution. (EMSL)

65 Overview ESnet Overview Science DMZ MoCvaCon and IntroducCon Science DMZ Architecture Network Monitoring Data Transfer Nodes & ApplicaCons On the Topic of Security User Engagement Wrap Up 65 ESnet Science Engagement - 11/19/14

66 Wrapup The Science DMZ design pa8ern provides a flexible model for supporcng high- performance data transfers and workflows Key elements: AccommodaCon of TCP Sufficient bandwidth to avoid congescon Loss- free IP service LocaCon near the site perimeter if possible Test and measurement Dedicated systems Appropriate security Support for advanced capabilices (e.g. SDN) is much easier with a Science DMZ 66 ESnet Science Engagement (engage@es.net) - 11/19/14

67 The Science DMZ in 1 Slide Consists of three key components, all required: FricCon free network path Highly capable network devices (wire- speed, deep queues) Virtual circuit conneccvity opcon Security policy and enforcement specific to science workflows Located at or near site perimeter if possible Dedicated, high- performance Data Transfer Nodes (DTNs) Hardware, operacng system, libraries all opcmized for transfer Includes opcmized data transfer tools such as Globus Online and GridFTP Performance measurement/test node perfsonar Engagement with end users Details at h8p://fasterdata.es.net/science- dmz/ 2013 Wikipedia 67 ESnet Science Engagement (engage@es.net) - 11/19/14

68 Links ESnet fasterdata knowledge base h8p://fasterdata.es.net/ Science DMZ paper h8p:// final.pdf Science DMZ list h8ps://gab.es.net/mailman/liscnfo/sciencedmz perfsonar h8p://fasterdata.es.net/performance- tescng/perfsonar/ h8p:// 68 ESnet Science Engagement - 11/19/14

69 Thanks! Jason Zurawski Science Engagement Engineer, ESnet Lawrence Berkeley National Laboratory New Mexico Technology in EducaCon (NMTIE) November 19 th, 2014