Software Defined Exchange (SDX) and Software Defined Infrastructure Exchange (SDIX) Vision and Architecture Tom Lehman, Brecht Vermeulen, Marshall Brinn, Niky Riga, Larry Landweber DRAFT DO NOT DISTRIBUTE Contents Abstract... 1 1. Desired Characteristics... 2 2. SDX/SDIX Control Software... 4 3. SDX/SDIX Policies... 5 4. Development and Deployment Considerations... 8 5. References... 9 6. Acknowledgements... 9 Appendix A: SDX/SDIX Deployment Considerations... 1 Appendix B: GEC22 Demo Setup... 2 Abstract Software Defined Exchange (SDX) is a relatively new concept around on which there is currently much discussion, concept development, and debate regarding the purpose and architecture for this new paradigm. These discussions are in an early stage and there is no agreement in the community regarding what services should be provided, or what is the proper hardware/software architecture. This document presents one vision for a SDX functional model and architecture. Extension of this model to a multi-resource Software Define Infrastructure Exchange (SDIX) capability is also discussed. A key objective of this vision and architecture work is to initiate development activities to realize a prototype SDX/SDIX system. 11/18/2014 DRAFT DO NOT DISTRIBUTE 1
1. Desired Characteristics The definition and vision for a SDX is actively under discussion by the network research community and the concepts span a very wide range. One view of a SDX is as an enhanced internet or network exchange Point (XP), where the XP switch is replaced with an SDN-capable device. This enhancement will allow for a much richer set of policies and peering technologies. An SDN enabled XP could allow for richer, Figure 1 Range of SDX Ideas and Use Cases complex, and custom peering configurations which could span Layer 3, Layer 2, Layer 1, and flow level parameters. Applying these features sets to enhance Layer 3/BGP peering at an Internet exchange Point (IXP) is one active area of research. Extending this model beyond just network elements to include host, computation, and storage resources, leads to an expanded view of this paradigm referred to as Software Defined Infrastructure exchange (SDIX). The SDIX vision starts with the richer network connectivity options enabled by SDX and augments this further with host, computation, and storage resources available for use by the exchange point peers. In between these two approaches there are a variety of deployment options that are also summarized in the SDX workshop summary [2], and shown in Figure 1. We envision a development and deployment progression where SDXs will evolve into full-fledged SDIXs. This will allow support a range of these ideas and allow parallel deployment for experimentation and exchange point service isolation. 11/18/2014 DRAFT DO NOT DISTRIBUTE 2
Figure 2 Gradual SDX Deployment This gradual SDX deployment will include increasing levels of complexity and supported functionality as depicted on Figure 2. For purposes of discussion in the remainder of this paper, we refer to SDX as an exchange point facility that has only network elements. The term SDIX will be used to refer to an exchange point facility that has network, host, compute, and storage resources. Additionally, we envision SDXs as a place of peering between multiple administrative domains and between different providers and that should be autonomous and independent. Based on the above discussions, we note the following observations and requirements for a SDX/SDIX: 1. Include SDN infrastructure. This seems kind of self-evident but included for completeness, an SDX/SDIX should include SDN resources (e.g. OpenFlow, or some other automated control for network elements.) 2. Include Storage and Compute for SDIX: To be able to support the wide range of ideas presented in Figure 1 an SDIX should include host, compute, and storage resources. 3. Be sliceable: It should be able to allow participants to peer at different levels (Layer 3 BGP peering, Layer 2 Ethernet Circuit peering, etc). One can think of this as allowing the prototype SDX/SDIX to support multiple virtual SDXs/SIDXs running at the same time and peers can choose the level of peering they would like to participate at. a. Dynamic slices. Participants should be able to peer dynamically and temporarily and thus reserve resources at the SDX/SDIX only on a need-only basis. Future reservations will also be very nice to have to allow peers to plan future reservations. 4. Operated by a third party. SDXs/SDIXs are places of peering of multiple providers (network providers, service providers, etc) and should be operated by or as independent third parties. 5. Authentication. Since SDXs/SDIXs are envisioned to be places were not only peers with physical presence there can participate but also peers that have a 11/18/2014 DRAFT DO NOT DISTRIBUTE 3
virtual presence (e.g. only using/renting SDX s infrastructure), there should be a way to authenticate participating peers. For more details see section 3 6. Policy. Different peers need to be able to express and enforce peering policies. In a sense SDX is not only about using software to control network flows but also to control the behavior of the exchange itself. Since this is a larger topic see Section 3 for more details. 7. Provide Monitoring. An SDX/SDIX should provide the capability of monitoring of its performance and operation to participating peers. It should also allow peers to monitor the slices they participate in. 8. Security. A wide range of security infrastructure and capabilities should be integral components of an SDX/SDIX. Some of these security mechanisms should be utilized by the SDX/SDIX operator to ensure the integrity of the exchange point infrastructure. In addition, there should be security features and infrastructure which can be utilized by the exchange point peers for the purposed on per peering instantiation monitoring and assurance. 9. Brokering. An optional functionality of an SDIX might be the brokering or proxying of resources. This should make it easier for users and tools to talk to single points and getting an overview of all resources (can be a policed view depending on the identity). Reservation/provisioning of those resources however goes directly through the responsible Aggregate Manager. There is no delegation of resources to the SDIX broker to avoid fragmented usage. 2. SDX/SDIX Control Software Based on the characteristics described on Section 1, we believe that the GENI software and systems provides an excellent starting point to build a prototype SDX/SDIX control system. In particular, resources within a specific SDX/SDIX can be managed as a GENI Aggregate. A logical scenario would be a one-to-one mapping between a single SDX/SDIX and a specific GENI Aggregate Manager (AM) instance. However, since a single SDX/SDIX may have many resource types, it is also possible that a single SDX/SDIX may include multiple GENI AMs, each dedicated to a subset of the total set of resources. Both of these models are valid, and specific deployments will be largely driven by software development considerations associated with the specific resources types located within a specific SDX/SDIX. A high level view of this vision is shown in Figure 3. 11/18/2014 DRAFT DO NOT DISTRIBUTE 4
Figure 3 High Level SDX/SDIX Concept 3. SDX/SDIX Policies By choosing the prototype SDX/SDIX control software to be based on the GENI Software, we have already satisfied many of the requirements identified in Section 1. However, Authorization, Authentication, and Policy application is one key area where additional work will be needed in order to satisfy the unique characteristics of exchange points SDX/SDIX, can be thought of as providing two kinds of services, both subject to policy: 1. An aggregate manager with its own resources (host, computation, storage, networking) 2. A peering manager facilitating and providing connections between exchange point connectors. 11/18/2014 DRAFT DO NOT DISTRIBUTE 5
The first case is not different from other aggregate managers: they can and should limit the resources they allocate to the right parties, schedules and quantities based on policy and resource availability. The second case, that of a peering manager, includes some characteristics that are unique to exchange points and require some enhanced feature definition and development. Today s manually operated exchange point inform us regarding some of the goals and feature sets we desire for future SDX/SDIX facilities. A key parameter of current exchange points is that they are policy free. This expression is meant in a limited context to reflect the fact that the policy for peering across an exchange point is largely driven by the exchange point connectors, and not the exchange point operator. That is, if two connectors indicate their desire for a connection across the exchange point fabric, the role of the exchange point operator is to instantiate that connection. The exchange point infrastructure generally does not play a role in maintaining or enforcing policies that are unique to the connectors. For instance, the exchange point operator may have the following roles with respect to the establishment of a bilateral peering across the exchange point infrastructure: Accept a request for a connection establishment across the exchange point fabric. Authenticate the entity making the request, and verify that requesting entity does have the rights to utilize the exchange point resources involved For a bilateral peering instantiation, this will typically require authenticated requests, or some other indication of approval, from the two entities involved. Overall, the peering service purpose is to verify that all sides to a peering setup have approved a specific instantiation. That is, a peering service will do whatever its connectors ask, as long as all of the affected connectors ask or otherwise indicate approval. This is one aspect of being policy free. After peering service is instantiated, then the exchange point operator is out of the loop with respect to what the traffic or peering activity occurs across the exchange point. This is another aspect of being policy free. One objective is to extend this model of limited policy involvement to the SDX/SDIX design. However, we may need some additional feature sets in order to automate the multi-connector authorization and SDX/SDIX resource access rights check. Toward this goal, we identify multiple methods to accomplish this in the context of a future SDX/SDIX; Coordinated-peering and Uncoordinated-peering. These are described in more detail below. These protocols are, themselves, a matter of SDX /SDIX policy: i.e. some SDX/SDIX may use one, or the other, or different ones in different circumstances based on policy. 11/18/2014 DRAFT DO NOT DISTRIBUTE 6
a. Coordinated-peering: The two peering parties provide their requests to peer to the SDX/SDIX who decides whether to provide a credential to each party indicating that they may create their side of the peering point. Such a credential is then required by each peering partner to provision its end of the peering exchange. In this way the SDX/SDIX can assure that these two and only these two parties can enter into this peering exchange. We make the distinction between two different types of coordinated peering that has to do with the policy enforcement of a peering agreement: i. Passive coordinated-peering. The SDX/SDIX is the implementer of the peering but not the enforcer of the policy. An example of how this might work is for example that a peering request needs to be cryptographically approved ; i.e. signed, by both peers before the SDX goes ahead with the reservation. ii. Active coordinated-peering. In this case the policies of each peer are communicated to the SDX/SDIX and the SDX/SDIX is not only implementing peering requests but it also ensures that the peering does not violate the policy of any participating site. b. Uncoordinated-peering: Each peering party requests their end of the peering point and the SDX decides whether to provide it. No additional SDX/SDIX-minted credential required; no checking on who is or will be listening on the other side. The decision of WHETHER to allow a given party to receive a peering connection is based on SDX/SDIX policy, the attributes of the requestor and the current state of allocations by the requestor at the SDX/SDIX. The SDX/SDIX may maintain a white list or black list of parties who are allowed to obtain peering points. The SDX/SDIX may maintain a white list or black list of pairs of entities who may enter into peering relationships. The SDX/SDIX may limit the number of peering points, the number of peering partners and the amount of allocated bandwidth allowed to any given entity (by individual requestor or by federation authority). The SDX/SDIX may manage resource schedules so that resources may be allocated in the future (and provisioned in the future). In such a case, policy limits on metrics over a time window (e.g. bandwidth-hours) may be applied by SDX/SDIX policy The SDX/SDIX must provide real-time monitoring of the links flowing through the exchange and apply policies to limit or shutdown connections that violate certain quota-based policies. 11/18/2014 DRAFT DO NOT DISTRIBUTE 7
4. Development and Deployment Considerations A key objective of this vision and architecture work is to initiate development activities to realize a prototype SDX/SDIX system. Toward this goal, we identify the following feature sets that an initial SDX/SDIX deployment should include: 1. at Layer 2 for dynamic VLAN stitching between peers (automate what exchange points like MANLAN, WIX, StarLight do manually now). 2. SDX-BGP provider. Support deployment of the Georgia-Tech BGP SDX model described at [5]. Other possible applications and capabilities that should be addressed include: 3. Service provider peering (e.g. allow Netflix to determine routing policies for their traffic) 4. Support deployment of multi-domain flowspace firewall (Internet2) 5. Allow application to do traffic steering over multiple paths. 6. Setup a multi-connection peering in an SDX/SDIX with OpenFlow/SDN functionality where a peer can define flowspaces and advanced routing policies. For example a participant might want traffic from subnet x to go to provider X and traffic from subnet Y to go to provider Y. This also gives the capability of application traffic steering. 7. Multi-SDX/SDIX Coordination: The global infrastructure of SDX/SDIX will be highly distributed and hopefully federated. Technologies and mechanisms will be needed to instantiate exchange point services that span multiple SDXs/SDIXs to realize multi-domain SDX/SDIX features sets. The next phase in this work will be to develop a detailed design for the SDX/SDIX control software. This will largely revolve around changes needed to the GENI Aggregate Manager, Clearinghouse, Attribute Based Access Control (ABAC), and Client Tool systems. Once the SDX/SDIX architecture and software design has been agreed upon there will be additional documents to describe the technical implementation details. Another key factor with regards to SDX/SDIX designs is the environment in which they will be deployed. As discussed in this document, in many ways we are looking to utilize the SDN/SDIX paradigm, in combination with extensions to the GENI Software and Systems, to enhance the high performance exchange points we see deployed today in the Research & Education (R&E) and commercial infrastructures. Facilities such as the Washington International Exchange (WIX) and the Manhattan Landing (MANLAN) are typical examples from the R&E space. A high performance, not blocking, switch fabric, combined with rich network and content provider connectivity are the key features of a high performance exchange point. The objective of this SDX/SDIX development project is to develop systems that are applicable to this environment. Appendix A shows a possible progression from prototype to 11/18/2014 DRAFT DO NOT DISTRIBUTE 8
production deployment of an SDX/SDIX capability. Appendix B shows a prototype target for GEC22. 5. References [1] http://groups.geni.net/geni/wiki/sdxandsdiworkshop [2] http://groups.geni.net/geni/attachment/wiki/sdxandsdiworkshop/sdx Workshop Outbrief - Draft.pdf [3] http://groups.geni.net/geni/attachment/wiki/sdxandsdiworkshop/chip Elliott SDX Workshop.pptx [4] http://groups.geni.net/geni/attachment/wiki/sdxandsdiworkshop/tom Lehman SDX Workshop.pdf [5] http://groups.geni.net/geni/attachment/wiki/sdxandsdiworkshop/laurent Vanbever SDX Workshop.pdf 6. Acknowledgements Tom Lehman was supported by the U.S. NSF and GPO via an award to the University of Maryland/Mid-Atlantic Crossroads. Marshall Brinn, Niky Riga and Larry Landweber were supported by the U.S. NSF via an award to the GPO. Brecht Vermeulen was supported by the European Commission through the Fed4FIRE project. 11/18/2014 DRAFT DO NOT DISTRIBUTE 9
Appendix A: SDX/SDIX Deployment Considerations This appendix describes the environment and infrastructure that may be used for the development and testing of the SDX/SDIX systems described in this document. In addition, a vision is presented for how the SDX/SDIX systems could be deployed at established exchange points such as the Washington International Exchange (WIX) and the Manhattan Landing (MANLAN). Deployment of these features sets within established facilities such as WIX and MANLAN is described as a potential goal. This is presented here simply as a starting point for discussion with the operators/owners of these critical infrastructures. The plans include use of the generally available services and features sets from the Internet2 Advanced Layer2 Network Service (AL2S) and the GEANT AutoBAHN network. The services that will be leveraged are currently available from these operators, or ones that should be deployed in the near future. A short summary of these features sets in included below: Internet2 AL2S Services: FlowSpace Firewall (FSFW) o Network Virtualization (NV) Service Open Exchange Software Suite (OESS) o Dynamic Provisioning of point-to-point or multi-point Ethernet VLAN based topologies GENI Aggregate Manager (AM) o GENI RSpec provisioning method for network resources GEANT AutoBAHN Services: Network Service Interface (NSI) Connection Service o Dynamic Provisioning of point-to-point Ethernet VLAN based circuits There are six figures included in this appendix. A description for each of these figures is provided below: Figure A-1. SDX Development Topology This figure shows the infrastructure and topology to be utilized for initial development and testing of a distributed SDX system. This topology is focused just on the SDX, or network resource only, phase of this work. MAX, BBN, and iminds will utilize local resources to emulate a SDX facility. The MANLAN and GEANT segments included statically mapped VLAN based resources. A-1
The OESS, FSFW, GENI AM (Standard) control elements are the production versions of those systems which are currently available. The GENI AM (SDX-Version) is the GENI AM software with the extensions discussed in the main body of this document. Figure A-2. SDIX Development Topology This figure shows an addition of a GENI Rack to the previously described SDX Development Topology. This represents the transition from and SDX to and SDIX. There are multiple architectural issues that need further study associated with this transition. For instance, it is not clear if there should be one or multiple GENI AM (SDX-Version) deployed on a per SDIX basis. Figure A-3. SDX Deployment This figure shows a possible SDX deployment at real exchange points such WIX and MANLAN. In addition, a possible deployment of a GENI AM interface on the GEANT AutoBAHN service is shown. Figure A-4. SDIX Deployment This figure shows an addition of a GENI Rack to the previously described SDX Deployment Topology. This represents the transition from and SDX to and SDIX. Figure A-5. Distributed, Directly Connected Multi-SDIX Topology This figures shows an additional option for how the dynamic provisioning services offered by Internet2 AL2S and GEANT AutoBAHN could be leveraged. The effect is to present a distributed, directly connected SDIX topology. Figure A-6. Distributed, Directly Connected Multi- SDX and SDIX Topology This figures shows an additional option for how the network virtualization feature set of Internet2 AL2S could be leveraged. The intent is to utilize virtualized slices across Internet2 AL2S to create a dedicated SDX topology for the purpose of interconnecting the distributed SDIX facilities. A key distinction between this option and that shown in Figure A-5, is that here the GENI AM (SDX-Version) would be talking OpenFlow directly to the Internet2 AL2S switches (via the FlowSpace Firewall). A similar virtualization use is shown for the GEANT AutoBAHN infrastructure. It is not clear if this service will be offered on GEANT AutoBAHN. Appendix B: GEC22 Demo Setup At GEC22 we are planning to demonstrate the operation of a prototype SDIX deployment. We are targeting to have implemented a prototype SDIX and deploy a setup as depicted in Figure A-2. This will include two SDIXs running on the US side and one running at Europe. At this stage we are going to employ existing GENI and FIRE infrastructure at the depicted sites but are planning to enhance the software to enable policy-based peering. A-2
Test Resource EPCC BonFire Testbed i2cat OFELIA Island BBN SDX MANLAN GEANT AutoBAHN Internet2 AL2S iminds SDX NETMODE NITOS MAX SDX GENI AM (Standard) Exchange Point Switch Cloudlab GENI Test Resource GENI AM (SDX-Version) OESS FSFW Figure A-1. SDX Development Topology iminds Virtual Wall 1 FIRE resources A-3
Test Resource EPCC BonFire Testbed i2cat OFELIA Island BBN SDIX MANLAN GEANT AutoBAHN Internet2 AL2S iminds SDIX NETMOD NITOS MAX SDIX GENI AM (Standard) Exchange Point Switch Cloudlab GENI Test Resource GENI AM (SDX-Version) OESS FSFW Figure A-2. SDIX Development Topology iminds Virtual Wall 1 FIRE resources A-4
BBN GENI Cloudlab Manhattan Landing (MAN LAN) SDX Internet2 AL2S Washington International Exchange (WIX) SDX GEANT AutoBAHN MAX Figure A-3. SDX Deployment EPCC BonFire Testbed NETMOD GENI AM (SDX-Version) OESS FSFW iminds SDX i2cat OFELIA Island NITOS iminds Virtual Wall 1 FIRE GENI AM (Standard) Exchange Point Switch A-5
BBN GENI Cloudlab Manhattan Landing (MAN LAN) SDIX Internet2 AL2S GEANT AutoBAHN MAX Washington International Exchange (WIX) SDIX Figure A-4. SDIX Deployment EPCC BonFire Testbed iminds SDIX NETMOD GENI AM (SDX-Version) OESS FSFW i2cat OFELIA Island NITOS iminds Virtual Wall 1 FIRE GENI AM (Standard) Exchange Point Switch A-6
BBN GENI Cloudlab EPCC BonFire Testbed i2cat OFELIA Island Manhattan Landing (MAN LAN) SDIX GEANT AutoBAHN Internet2 AL2S iminds SDIX NETMOD NITOS MAX Washington International Exchange (WIX) SDIX GENI AM (Standard) Exchange Point Switch GENI AM (SDX-Version) OESS FSFW Figure A-5. Distributed, Directly Connected Multi-SDIX Topology iminds Virtual Wall 1 FIRE A-7
BBN GENI Cloudlab EPCC BonFire Testbed i2cat OFELIA Island Manhattan Landing (MAN LAN) SDIX GEANT AutoBAHN SDX iminds SDIX Internet2 AL2S SDX NETMOD NITOS MAX Washington International Exchange (WIX) SDIX GENI AM (Standard) Exchange Point Switch GENI AM (SDX-Version) OESS FSFW Figure A-6. Distributed, Directly Connected Multi- SDX and SDIX Topology iminds Virtual Wall 1 FIRE A-8