1 International Center for Advanced Internet Research at Northwestern University, 750 N. Lake Shore Drive, Suite 600, Chicago, IL 60611, USA

Transcription

1 OpenFlow Services for Science: An International Experimental Research Network Demonstrating Multi-Domain Automatic Network Topology Discovery, Direct Dynamic Path Provisioning Using Edge Signaling and Control, Integration With Multipathing Using MPTCP Joe Mambretti 1 Jim Chen 1 Fei Yeh 1, Te-Lung Liu, 2, Mon-Yen Luo 3 Chu- Sing Yang 4, Ronald van der Pol 5, Artur Barczyk 6 Sander Boele 5, Freek Dijkstra 5, Gerben van Malensteinz 7 1 International Center for Advanced Internet Research at Northwestern University, 750 N. Lake Shore Drive, Suite 600, Chicago, IL 60611, USA 2 National Center for High-Performance Computing No. 7, R&D 6th Rd., Hsinchu Science Park, Hsinchu City, Taiwan 3 National Kaohsiung University of Applied Sciences 415 Chien Kung Road, Sanmin District, Kaohsiung 80778, Taiwan 4 National Cheng-Kung University No.1, University Road, Tainan City 701, Taiwan 5 SARA, Science Park 140, 1098 XG Amsterdam, The Netherlands 6 California Institute of Technology, c/o CERN, 1211 Geneva Switzerland 7 SURFnet, Radboudkwartier 273, 3511 CK Utrecht, The Netherlands Abstract- Large scale data intensive science requires global collaboration and sophisticated high capacity data management. Traditionally, mismatches between the requirements of large scale science and the restrictions of generally deployed networks have been problematic. However, the emergence of more flexible networking, for example, using techniques based on OpenFlow, provides opportunities to address these issues because these techniques enable a high degree of network customization and dynamic provisioning. Also, these techniques enable large scale facilities to be created that can be used to prototype new architecture, services, protocols, and technologies. Consequently, a number of research organizations in several countries with major HPC facilities have been working together to design and implement a persistent international experimental research facility that can be used to prototype, investigate, and test network innovations for large scale global science. For SC12, this international experimental network facility will be extended to from sites across the world to the conference showfloor, and it will be used to support several testbeds and to showcase a series of complementary demonstrations of techniques that could be useful to large scale science applications, including multi-domain automatic network topology discovery, virtual networking and tunneling over WANs, dynamic path provisioning using edge signaling and control, and multipath provisioning using MPTCP. I. Introduction Large scale data intensive science requires global collaboration and sophisticated high capacity data management, including transporting high volume stream comprised of extremely large files to and from remote sites around the world. Also, large scale science applications require dynamic interconnections among many resources at multiple remote sites. For many science research activities, such connections are best provided directly by edge devices, applications, and processes and not by external central network operations. Traditionally, because of mismatches between these and related requirements of large scale science and the restrictions of generally deployed networks, optimal use of networking for science has been problematic. The emergence of more flexible networking, for example, using techniques based on OpenFlow, [1,2] provides opportunities to address these issues because it allows for a high degree of network customization and dynamic provisioning. Instead of compromising application performance because of network limitations, it is possible to make dynamic

2 adjustments to meet the requirements of the applications, even across global WANs using edge processes. To create such new services, architecture, and technologies, a number of research organizations in several countries with major HPC facilities have been designing and implementing a persistent international experimental research facility that can be used to prototype, investigate, and test network innovations for large scale global science. The multi-layer, dynamic provisioning potentials, and distributed nature of this facility presents interesting challenges and also major opportunities for addressing many types of research topics. For example, the facility can be used to create multiple different testbeds simultaneously. Although experimental Openflow programmable network testbeds around the world have diverse requirements and provide for different implementations, such testbeds can be interlinked and shared for joint experimental and demonstration activities. Openflow provides a flexible framework that can be used by researchers who would like to create specialized networks for individualized research experiments, while using a more general shared infrastructure. For SC12, this international experimental network facility will be extended to from sites across the world to the conference showfloor, and it will be used to support several testbeds and a series of complementary demonstrations. These demonstrations will showcase OpenFlow based techniques for science, including multi-domain automatic network topology discovery, programmable WAN tunneling, dynamic path provisioning using edge signaling and control, and multipath provisioning using MPTCP. The MPTCP demonstration, which builds on a successful SC11 OpenFlow demonstration, [3] is described in detail in a separate SC12 paper.[4] One series of these OpenFlow demonstrations will showcase capabilities for international multi-domain automatic network topology discovery. International science collaborations must be supported by multiple services provided by many networks in many different domains. Today, almost all OpenFlow networks have been implemented are within a single domain. In order for international science applications to take advantage of OpenFlow based networks and services, multi-domain techniques will have to be created. Several demonstrations planned for SC12 will demonstrate multi-domain automatic network topology discovery over heterogeneous networks. This technique can be used in conjunction with multipathing with MPTCP, for example, to determine what resources are available to use for large scale file transfers. Global science requires the close integration of resources around the world. Another set of related demonstrations will showcase dynamic path provisioning using direct edge signaling and control, using techniques for closely integrating programmable networks and computational and storage clouds, to support large scale scientific visualization based on highly distributed data repositories, rendering facilities, instrumentation, and visualization facilities. These demonstrations will show how OpenFlow based techniques can be used to create many types of ad hoc specialized networks, including high resolution scientific visualization using large scale data streamed live from multiple remote sites. Another complementary demonstration, is described separately in more detail in another SC12 paper, will showcase multipath provisioning, using MPTCP to illustrate how extremely large data files can be transported efficiently by ensuring maximum utilization of all available network capacity, including using multiple paths. Currently, large data sets are transferred using one or a few large capacity flows over multiple paths by using traditional hash based load balancing. However, hash based load balancing does not work well with only a few flows. This demonstration shows how MPTCP and multipathing in an OpenFlow enabled network is significantly more efficient. Another advantage of using MPTCP is that applications do not need to be adapted before using this technique. An illustration here depicts the international testbeds, sites, and SC12 booths that will be used for these demonstrations.

3 II. International Multi-Domain Automatic Network Topology Discovery (MDANTD) Currently, almost all OpenFlow systems have been implemented within a single domain. However, international science applications and collaborations must be supported by many networks in many different domains under continually changing conditions. Consequently, these paths, the topologies that they comprise, and their state information must be dynamic not static. To optimally support such applications and collaborations, techniques are required for large scale, international, multi-domain automatic network topology discovery (MDANTD). These techniques must be designed to anticipate a dynamically continually changing flow of information about the availability and location of highly distributed network resources. Also, to quickly convey this information, it must have a capability for being visualized, including options for edge visualization. At SC12 several demonstrations will showcase capabilities for international MDANTD based on OpenFlow. This technique can be used in conjunction with other emerging OpenFlow and NOX [5, 6] controller techniques, such as multipathing with MPTCP to determine what resources are available to use for large scale file transfers. NOX is an opensource OpenFlow controller that provides a platform for creating software tools (usually developed in C++ and/or Python supporting an interface) for controlling, monitoring, and analyzing networks. Although NOX provides tools that allow for admission control, direct control of network flows, discovery, access to state information such as topology and path status implementations, it is oriented to single domain implementations. The MDANTD technique is comprised of three primary processes. The first is an information gathering and distribution process using LLDP (IEEE Link Layer Discovery Protocol). [7] Through this process the NOX obtains the NOX IP and port information sends that information to a discovery process. This is an ongoing state information communication process through which LLDP packets, including these informational messages, are continually created and forwarded to OpenFlow switches. The second process leads to the creation of mapping tables. When LLDP packets are received by the discovery process, key information such as source ID, source port, and a descriptive text string are obtained. Then, the discovery process communicates with other NOX processes to establish links with source nodes. The discovery process also sends the source ID and the descriptive text string to the topology process. The topology process uses that information to create a mapping table and an ID table. These tables are used to determine relationships between NOX IPs and IDs. The third process creates complete multi-domain OpenFlow topology graphs, through intercommunication among the topology process, LAVI (an extensible backend created by the OpenFlow community for network visualization), the translate process, and ENVI (Extensible Network Visualization and Control Framework, the GUI frontend to LAVI). In a series of steps through the translate process, the frontend ENVI sends an information request message to the backend LAVI, which gathers all IDs from the ID table within the current topology. Using those IDs, LAVI interrogates the mapping table in topology to obtain the NOX IPs and ports related to individual IDs and then sends a JSON message containing NOX IPs, ports and link information to the translate process to prepare the JSON message for ENVI. Using, the topology information, ENVI can create a complete topology graph. This process can be accomplished dynamically and continuously. III. Virtual Networking Using OpenVswitch and Tunneling Over WANs A collaborative effort between the National Kaohsiung University of Applied Science (KUAS) and National Cheng Kung University (NCKU) designed, implemented, and has been experimenting with an international heterogonous testbed tunneling network, which has been operational for over two years. This testbed has supported international research projects, including security research using distributed EMUlab facility in Taiwan [8] cluster, LLDP inter domain topology discovery, international virtual network over distributed netfpga, [9] and other projects. For SC12, this prototype network will be extended to the conference using international network testbed facility. The early generations of this tunneling network, are described here along with experiments and demonstrations planned before and during SC12. This demonstration will showcase an international virtualized network testbed based on a programmable Open Virtual Switch architecture. This testbed will

4 use the Open Virtual Switch with a dynamically configurable tunneling mechanism, which is highly programmable. Using this technique, multiple VMs within different private cloud systems at different sites can be interconnected even at sites across the world. Currently, investigations for control frameworks for this technique are also being investigated to enable better coordination among this federated virtual OVS networks. Previously, virtual networks were created using a tunneling mechanism (including L3 over L2 paths) among KUAS, NCKU, and icair. However, more recently a virtual network has been created using Layer-2 networks over wide-area production networks based on interconnected OpenVswitches. This new approach provides a more flexible, controllable programmable path creation and management mechanism, especially because it can be used to integrate existing programmable networking protocols mechanisms such as STP (spanning tree protocol) and OpenFlow using NOX as a controller. A mechanism for implementing processes integrating GRE tunnel port settings and NOX has recently been demonstrated. For SC12, this OpenVswitch based international testbed will be extended to the showfloor within the icair and NCHC booths, with international connections interlinking private clouds at KUAS and NCKU. IV International Multi-Domain Provisioning Using Distributed Direct Dynamic Edge Signaling and Control Global science requires the close integration of resources around the world. A set of related demonstrations will showcase dynamic path provisioning using edge signaling and control, using techniques for closely integrating programmable networks and computational and storage clouds. Scientific research communities often require dynamic, continually changing interconnections among multiple highly distributed resource and collaboration sites. Furthermore, to provide for these dynamic topologies and related service, they cannot depend on traditional external operation procedures and processes. They require a simple platform that will place all necessary controls on devices in research labs so that the researchers, applications or edge processes can directly control network resources, even across global networks. A prototype of such a platform for easily integrating network resources within science workflows has been designed and implemented, and this platform will be demonstrated at SC12. These demonstrations will show how OpenFlow based techniques can be used to create many types of ad hoc specialized networks, including for large scale high resolution scientific visualization using large scale data at multiple remote sites. Specifically, these demonstrations will show how this platform can be used to support large scale HPC simulation and scientific visualization based on highly distributed resources, including data repositories, rendering facilities, instrumentation, computational clusters, storage devices, and visualization facilities. These demonstrations are related to a sequence of demonstrations that are being designed and showcased as part of GENI (Global Environment for Network Innovation) large scale testbed activities. These demonstrations will show how using dynamically programmable networks closely integrated with computational and storage clouds, it is possible to provide capabilities that can be used to create interactive simulation/visualization instruments to significantly improve this traditional process. An interactive real-time simulation/visualization instrument will include: a) distributed back-end MPI rendering clusters and storage, b) a web front end to setup control parameters for rendering and display the result, c) a customized web server to pipe rendering results to users efficiently, d) a program to check the rendering result and submit new jobs if the results had not been already assembled. For these demonstrations, these web interfaces will be used to dynamically identify the sites around the world, where the simulation images located, to convert the request and to send the request to the appropriate host over the private international network, and interactively visualize the simulation over a private network specifically designed for the demonstrations. Visualization is particularly important for nanotechnology science and engineering because those disciplines are focused on objects at nanometer scale. The science simulation/visualization examples that will be used are: a) Single/Double Slit Light Simulation at Nano-Macro Scale, b) Nano-Pattern Formation/Self Assembly, Photonic Band-Gap, Optical Pulse incident on Nano-particles. These topic areas are related to those that are creating new customized materials for optical components and that are designing materials that can use light to take the place of electric current used for sub component communication systems. V. Network Requirements

5 These demonstrations will use 10 Gbps paths on the international experimental network facility that has been established using the Global Lambda Integrated Facility (GLIF), and on various national and international networks, including TWAREN, SURFnet, the National Lambda Rail, CaveWave, C- Wave, and others, interconnected by the StarLight International/National Communication Exchange Facility in Chicago. At SC12, these demonstrations will use SCInet connections for WAN termination and for 10 Gbps path among several booths including those from the Open Cloud Consortium/iCAIR, the Dutch Consortium, NCHC, and Caltech. VI. Desired Outcomes These demonstrations will show how prototypes of large scale, multi-site distributed networks can be designed and implemented as separate environments based on virtualization techniques within a larger infrastructure using control and management tools that can be directly utilized by edge processes, applications and experiments. This approach anticipates creating fairly persistent large scale distributed international environments within which there are multiple programmable elements, including network resources encapsulated as software objects that are advertised and can be discovered and incorporated through ad hoc, dynamic processes with other types of tools and functions. Such resource within a large scale distributed facility can be used to experiment with and prototype new techniques for HPC communities. The research activities showcased through these demonstrations are providing network programming tools that are a) directed at meeting the needs of large scale data intensive flows for science research applications b) targeted to the capabilities of high performance communications facilities with adjunct controllers and c) can be integrated with science research workflows. Computational resources for edge processes used by the demonstrations will be based in part on high performance computational clusters and HPC cloud environments. VII. Relevance to HPC Community Many large scale highly distributed data intensive science research environments have network requirements that are distinctly different from those of more general purpose networks. Such environments often require close control of specific large scale flows among multiple sites that are managed not as general purpose flows but as key components within structured science workflows. Often these types of data intensive flows provide the most optimal results when they are separated from other types of flows, using direct edge process control and segmentation and partitioning techniques. Many HPC applications and distributed environments can be optimized by enabling network resources to be integral components of those environments, directly managed by HPC workflow tools, instead of noncontrolled external resources. These planned demonstrations will show how such environments can be created using OpenFlow as a base platform, or foundation, enabling such direct integration, with a focus on large scale L2 flows. Opportunities to design, implement, and operate such environments will increase as large capacity networks (e.g., 100 Gbps networks) are established, providing additional options for segmenting resources to address specialized as opposed to general purpose requirements. VIII Acknowledgements Support for icair participation in this research is provided by the US National Science Foundation through the Office of Cyberinfrastructure International Research Network Connections (IRNC) program and through the Computer and Information Science and Engineering (CISE) Global Environment for Network Innovations (GENI) program, including the International GENI (igeni) program. Support for the National Center for High-Performance Computing (NCHC), Taiwan, the National Kaohsiung University of Applied Science (KUAS), and National Cheng Kung University (NCKU) KUAS/NCKU is provided by various national science research programs. The work of SARA and SURFnet is made possible by the GigaPort program of the Dutch Government and the G eant3 project of the European Commission. The activities at CERN and the California Institute of Technology are supported by multiple EU and US funding agencies, including the US Department of Energy s Office of Science VIII References [1] N. McKeown, T. Anderson, H. Balakrishnan, G. Parulkar, L. Peterson Princeton University, J. Rexford, S. Shenker, J. Turner, Enabling Innovation In Campus Networks, ACM SIGCOMM Computer Communication Review archive, Volume 38 Issue 2, April 2008, pp [2]

6 [3] R. van der Pol, S. Boele, F. Dijkstra, J. Mambretti, J. Chen, F. I. Yeh, M. Savoie, B. Ho, L. Sun, Monitoring and Troubleshooting OpenFlow Slices with an Open Source Implementation of IEEE 802.1ag, detail.php?evid=rsand109, November 2011 [4] R. van der Pol, Sander Boele, Freek Dijkstra, Artur Barczyky, Gerben van Malensteinz, Jim Hao Chen, and Joe Mambretti, Multipathing with MPTCP and OpenFlow, forthcoming, Proceedings SC12, 2012 [5] N. Gude, T. Koponen, J. Pettit, B. Pfaff, M. Casado, N. McKeown, and S. Shenker, NOX: Towards an Operating System for Networks, CCR Online, July 2008 [6] NOX OpenFlow Controller, nox.html [7] Link Layer discovery Protocol (LLDP), IEEE 802.1AB, [8] [9]