Building Scalable, Open, Programmable and Application Centric Data Center with Cisco ACI 林 瑝 錦 / Jerry Lin Cisco Systems 2015 July
Data Center Demands For the Cloud-Era Bare Metal VM Density and Server I/0 Multi-Cloud Big Data 75% physical servers 1 10G LAN on Motherboard 2 ~45% of DC Multi-Hypervisor 3 IP Traffic 25% CAGR 4 Lower TCO Workload Flexibility Agility Compliance/Security
Customer Spending $B Impact of Server Virtualization on Network Complexity $250 $200 $150 $100 $50 $0 Server utilization improved by almost 4x Server CapEx spend dropped However VM related OpEx significantly increased Server Spending Standalone Servers - Mgnt & Admin Virtual Servers - Mgnt & Admin Power & Cooling Expense Increased OpEx is attributed to network optimization to VM s to deliver application SLA s Source: ZK Research 96 97 98 99 00 01 02 03 04 05 06 07 08 09 10 11 12 13 Source: IDC, 2011 New Economic Model for the Datacenter 3
Orchestration in the IT world
Overloaded Network Constructs Basic Network Policy SLAs L4-7 Services Subnet Subnet Subnet VLAN VLAN VLAN Network constructs are overloaded with unintended functionality.
Application Language Barriers Developers Infrastructure Teams Application Tiers Provider / Consumer Relationships VLANs Subnets Protocols Ports Developer and infrastructure teams must translate between disparate languages.
Industry Standards and Forums 802.1 Overlay Networking Projects SDN WG Technical Advisory Group, Working Groups: Config, Extensibility, Futures/FPMOD/OF2.0 Open Network Research Center at Stanford University Open Source Cloud Computing project Initiatives: Quantum (Folsom release) Donabe ETSI SGI on Network Function Virtualization Overlay Working Groups: NVO3, L2VPN, TRILL, L3VPN, LISP, PWE3 API Working Groups/BOFs NETCONF, ALTO, CDNI, XMPP, SDNP, I2AEX Controller Working Groups: PCE, FORCES Protocol Working Groups: IDR, IS-IS, PF, MPLS, CCAMP, BFD New working group: I2RS Interface to the Routing System
Network Programmability Models 1 Programmable APIs 2a Pure SDN 2b Hybrid SDN Applications Applications Applications Applications 3 Overlays Networks Applications (Network Mgmt, Monitoring, ) Vendorspecific APIs Vendorspecific APIs Controller Vendorspecific APIs Controller Vendorspecific APIs Virtual Switch Overlays CLI, SNMP, Netflow, Vendor Specific (e.g. onepk) OpenFlow, PCEP, I2RS Vendor Specific (e.g. onepk) OpenFlow, PCEP, I2RS Vendor Specific (e.g. onepk) Overlay Protocols (e.g. VXLAN) Control Plane Control Plane Control Plane Control Plane Data Plane Data Plane Data Plane Data Plane Overlays Data Plane Openstack and Network Overlays Apply to All Models (Physical/Virtual) Custom Features Can Be Built
We Listened To You! Availability with Multi-tenant scale Secure Multitenant cloud Low latency, High Transaction Processing Capabilities 40G Aggregation Scale and Performance Scale 10/40/100G End-host scale for v4/v6 Scalable multi-tenancy Open Policy Automation Multi-Tenant Security Future Proof Investment in SDN, Open Solution, 10/40G Telemetry Investment Protection CTC Programmability, Telemetry, Troubleshooting Open, Extensible Framework, Multi-hypervisor Support Solution based on Openness, Service Agility Physical AND Virtual Agility, Automation
Application Centric Infrastructure Embracing SDN and Going Beyond Rapid Deployment of Applications onto Networks with Scale, Security and Full Visibility Integrated GBP VXLAN Overlay ACI FABRIC GROUP-BASED POLICIES CONTROLLER Best SDN Controller Interop 2015 ACI http://www.interop.com/lasvegas/specialevents/best-of-interop-awards.php
ACI Fabric Industry s most efficient fabric 1G/10G/40G edge - High density 40G spine (100G capable) Routed fabric Optimal IP Forwarding Bridging (L2) and Routing (L3) of VXLAN/NVGRE/VLAN at scale No x86 GW s Physical & Virtual Full visibility into virtual and physical Common operations from Hypervisor to Compute, To Fabric, to WAN
Virtual Overlay Networks Drive Cloud Readiness Unprecedented Infrastructure Flexibility Without planning, physical networks can introduce obstacles to VM migration POD POD Resource sharing over larger resource pools can optimize costs Goal is to reduce management complexity and integrate physical and virtual (consistent management, visibility, policies, etc..) L2 / VLAN MOBILITY CONSTRAINTS
Virtual Overlay Networks Drive Cloud Readiness Unprecedented Infrastructure Flexibility POD VIRTUAL NETWORK OVERLAY POD Virtual Network Overlays remove network complexity, increase scale VXLAN tunnels provide logical isolation of network traffic vpath-enabled services provide location independence of services and consistency for apps independent of location SECURITY POLICIES ENFORCED INDEPENDENT OF LOCATION
Application Centric Infrastructure Fabric Flat Hardware Accelerated Network ACI Fabric Full abstraction, de-coupled from VLANs and Dynamic Routing, low latency, built-in QoS Flexible Insertion Fabric Port Services Every device is one hop away, microsecond latency, no power or port availability constraints, ease of scaling Hardware filtering and bridging; default gateway; seamless service insertion, service farm aggregation Unified Management and Visibility ACI Controller manages all participating devices, change control and audit capabilities Files Users Logical Endpoint Groups by Role Heterogeneous clients, servers, external clouds; fabric controls communication
ACI Fabric Integrated Overlay Data Path - Encapsulation Normalization IP Fabric Using VXLAN Tagging Normalized Encapsulation Any to Any VTEP VXLAN IP Payload Localized Encapsulation VXLAN VNID = 5789 802.1Q VLAN 50 VXLAN VNID = 11348 NVGRE VSID = 7456 All traffic within the ACI Fabric is encapsulated with an extended VXLAN header External VLAN, VXLAN, NVGRE tags are mapped at ingress to an internal VXLAN tag Forwarding is not limited to, nor constrained within, the encapsulation type or encapsulation overlay network External identifies are localized to the Leaf or Leaf port, allowing re-use and/or translation if required Outer IP Outer IP 802.1Q NVGRE VXLAN Eth MAC Eth IP IP IP Eth IP Normalization of Ingress Encapsulation Payload Payload Payload Payload Payload
ACI Goal: Common Policy and Operations Framework Cloud Cloud Admin Application Admin Web Tier External Zone App Tier LICATION DB Tier Security Admin DMZ Trusted Zone SECURITY DB Tier Network Admin 16
ACI Goal: Common Policy and Operations Framework Cloud Cloud Admin Application Admin LICATION External Zone Security Admin DMZ Trusted Zone SECURITY DB Tier Network Admin COMMON POOL OF RESOURCES 17
Application Policy Model and Instantiation Application policy model: Defines the application requirements (application network profile) Web Tier Application Client App Tier Storage DB Tier Storage Policy instantiation: Each device dynamically instantiates the required changes based on the policies VM VM VM VM VM VM VM 10.2.4.7 10.9.3.37 10.32.3.7 All forwarding in the fabric is managed through the application network profile IP addresses are fully portable anywhere within the fabric Security and forwarding are fully decoupled from any physical or virtual network attributes Devices autonomously update the state of the network based on configured policy requirements
Defining Application Logic Through Policy Applications and Conversations Users Web Farm App Servers DB Farm Application communication can be defined as who is allowed to talk to whom. Communication between objects on the network can be thought of as one or two way conversations (monologue/dialogue.)
Building ACI Contracts Filter Action TCP Port 80 Permit Label Web Access Subject Filter Action Label Subjects are a combination of A filter, an action and a label Contract 1 Actions are policy options: Permit the traffic Subject 1 Block the traffic Subject 2 Redirect the traffic Log the traffic Subject 3 Copy the traffic Mark the traffic (DSCP/CoS) Contracts are groups of subjects which define communication between source and destination EPGs. The defined policy encompasses traffic handling, quality of service, security monitoring and logging.
Cisco ACI Layer 4-7 Service Integration Application Profile EXTERNAL Policy WEB Policy Policy DB WEB WEB WEB DB DB DB Terminal: Input1 Terminal: Output1 Service Graph: WebGraph Service Graph: appgraph Func: Firewall Func: Load Balancer Func: Load Balancer
ACI Fabric Powered with Group-Based Policies Outside (Tenant VRF) Policies Web Policies App Policies Connectivit y Firewall Filter Filter LB QoS Filter DB Application Network Profile ACI Fabric Scale-Out Penalty-Free Overlay APIC APIC APIC
ACI is a network Fabric which provides a new communication abstraction model Single Point of Orchestration Different administrative groups use same interface, high level of object sharing Application Policy Infrastructure Controller (APIC) Policy Contract Users Files All TCP/UDP: Accept, Redirect UDP/16384-32767: Prioritize All Other: Drop Create Contracts Between Endpoint Groups Port-level rules: drop, prioritize, push to service chain; reusable templates ACI Fabric Enforce Ingress Fabric Rules Hardware rules on each port, security in depth, embedded QoS Single Pass Services Define Endpoint Groups Security administrator defines generic templates in APIC, availed to contract creation Service Graph Files Users Any endpoints anywhere within the fabric, virtual or physical
Hypervisor Integration with ACI F/W EPG WEB APIC Application Network Profile L/B VM VM VM EPG WEB PORT GROUP PORT GROUP DB PORT GROUP EPG DB Relationship is formed between APIC and Virtual Machine Manager (VMM) ACI Fabric implements policy on Virtual Networks by mapping Endpoints to EPGs Endpoints in a Virtualized environment are represented as the vnics VMM applies network configuration by placement of vnics into: Port Groups (VMWare), VM Networks (Hyper-V) Networks (OpenStack) EPGs are exposed to the VMM as a 1:1 mapping to Port Groups, VM Networks or OpenStack Networking.
ACI Hypervisor Integration: VMWare DVS Virtual Distributed Switch APIC Admin VM Admin WEB PORT GROUP PORT GROUP DB PORT GROUP WEB DB WEB DB vcenter HYPERVISOR HYPERVISOR VIRTUAL 12 34 56 78 9 Cisco Attach Learn Automatically Create Instantiate Push Policy APIC location Application VDS Port Hypervisor VMs, and Groups (Lazy) Map of VMware Assign ESX to Policy EPG VDS Host to To vcenter Port through Groups Initial LLDP Handshake
Open: APIC Programming Interfaces Automation Hypervisor Management OVM Enterprise Monitoring Systems Manageme nt Orchestration Frameworks Open REST APIs Support Integration With Any Software Applications NORTHBOUND PROGRAMMABILITY LAYER APIC OpFlex: Open Fabric Attached Device API Supports Integration with Any Network Device SOUTHBOUND PROGRAMMABILITY LAYER
OpFlex A Flexible, Extensible Policy Protocol OPFLEX is a new extensible policy resolution protocol designed for declarative management of any datacenter infrastructure. Unlike legacy protocols such as OVSDB, OPFLEX was designed to offer: APIC Policies Who can talk to whom What about Topology control Ops stuff Declarative resolution Push + Pull API support Abstract policies rather than device-specific configuration Opflex Agent Opflex Agent Opflex Agent Opflex Agent Flexible, extensible definition of using XML / JSON Support for any device vswitch, physical switch, network services, servers, etc. Opflex Proxy Legacy API Opflex Agent Firewall Opflex Agent Hypervisor Switch Opflex Agent ADC http://tools.ietf.org/html/draft-smith-opflex-00
Cisco Virtual Networks Support Multiple Cloud Stacks Cloud Portal and Orchestration L4-7 Virtual Network Infrastructure L2-3 Hypervisor Computing Platform Physical Network vcloud Director/ DynamicOps WAAS vsphere System Center Cloud Network Services vpath Nexus 1000V Hyper-V UCS Unified Fabric (Nexus 2000 7000) Cisco UCS Director OpenStack and Partners ASA 1000V VSG NAM NetScaler Imperva Multiple (Hyper-V, KVM, ) Storage Platform
Policy Driven ACI Summary External Network POLICY WEB POLICY POLICY DB Application Virtualization APIC Networking Physical HYPERVISOR HYPERVISOR HYPERVISOR
Why ACI? The data center is both Virtual and Physical Enterprise Scale and Performance requires hardware acceleration A SINGLE architecture to deliver performance, programmability, agility and reduced complexity An Application Centric Policy Model that dynamically defines the network fabric by means of the application requirements An AUTOMATED network fabric for both virtual and physical workloads and services
Delivering Business Outcomes Example: Cisco IT with ACI (Based On Projections) Greater Business Agility Lower Capital Expenses Reduced Costs/ Complexity Lower Operating Cost Resource Optimization 58% 25% 21% 45% 10 20% Reduce Network Provisioning CAPEX Reduction Reduce Management Costs Reduce Power and Cooling Costs Compute and Storage Optimization *Based on Cisco IT Projections Cisco IT has already gained cost efficiencies through UCS. These are incremental savings with ACI.
Cisco ACI Takeaways Cisco Application Centric Infrastructure Fixed Workloads Variable Workloads SPEED SECURITY TELEMETRY POLICY NETWORK and SERVICES Delivered in minutes INHERENT Security and INTEGRATION Rich TELEMETRY & Application HEALTH SCORE Policy-based deployment/governance Physical & virtual OPEN and AGNTIC