Plant-Floor and Enterprise Network Convergence Jerry Lucas, Cisco Systems jerrluca@cisco.com Making Factory Automation Networks Secure
Agenda Trend Plant-Floor and Enterprise Network Convergence Rockwell Automation & Cisco Systems Alliance Converged Plantwide Ethernet Architectures Convergence-Ready Network Solutions Advantages of EtherNet/IP Manufacturing Security Overview 2
What you will learn Trends in Plant-Floor and Enterprise Network Convergence Technology enablers and business drivers Cultural and organizational convergence How the Rockwell Automation and Cisco Systems Alliance are helping customers with their technology, network and cultural convergence Products/Services Education Series Webcasts Reference Architectures A Layered Network Security Approach Overview Security Trends Defense in-depth: A Layered Approach 3
Convergence with the Internet of Things (IoT) Addresses Priorities Proprietary Networks Ruggedized Infrastructure Ethernet/IP Optimized for Industrial Applications IoT Architectures Connected Manufacturing evolution of Operational Technology Enterprise IT Manufacturing Operations Why converge networks? Time to market Security: perimeter no longer viable Simplicity and Flexibility: maintenance and management Problem resolution Voice, video, data collaboration Control of plant performance Remote talent Standards convergence open systems Future proofing
Producing Tangible Business Benefits for Manufacturing Companies Best-in-Class Manufacturers Top 20% 8 hours of downtime per year (99.91% Uptime) 11% total cost of ownership reduction for industrial network 90% Overall Equipment Effectiveness (OEE) +25% operating margin vs. corporate plan 67% Converged Industrial Ethernet Adoption Rate Middle 50% Downtime: 36 hours/year OEE: 80% Bottom 30% Downtime: 135 hours/year OEE: 60% 33% Converged Industrial Ethernet Adoption Rate Source: Aberdeen Group 2012
Plant-Floor and Enterprise Convergence Trend Technology Convergence Enterprise-wide Systems Supplier OEM More Enterprise Integration Corporate Headquarters Other Plant Customer Plant-wide Systems More Applications (control disciplines) Receiving Batching/ Blending Processing More Assets (things) Material Handling Packaging Connected Control Room Utilities Collaboration Shipping More Lower Total Cost of Ownership Faster Time to Market Better Asset Optimization Broader Risk Management 6
Technology Convergence Changing Automation Networks Back-office Mainframes and Servers Corporate Network Office Applications, Internetworking, Data Servers, Storage Back-office Mainframes and Servers Corporate Network Office Applications, Internetworking, Data Servers, Storage Control Network Gateway Human Machine Interface PC Based Controllers Programmable Logic Controllers Robotics Motors, Drives, Actuators Control Network Device Level Network Ethernet Traditional Sensors and Other Input/Output Devices Robotics Human Machine Interface PC Based Controllers Ethernet-based Sensors and Other Input/Output Devices Motors, Drives, Actuators Programmable Logic Controllers Ethernet Automation equipment vendors are implementing Ethernet-based protocols as an replacement of traditional fieldbus networks
Converged Plantwide Ethernet (CPwE) Enterprise Network Levels 4-5 Demilitarized Zone (DMZ) Separation between Control & Enterprise Networks Manufacturing Zone Level 3 Interconnection between Cell Zones, Server Farms, and DMZ Cell Zone Levels 0-2 Network Connection for PLCs, HMIs, I/Os, & Drives
Plant-wide Network Convergence Trend Technology Convergence Successful Plant-wide Network Convergence Requires Collaboration Simplification Innovation 9
Plant-Floor and Enterprise Requirements Similarities and Differences Enterprise (IT) Requirements So, what are the similarities and differences? Plant-Floor (Industrial) Requirements 10
Plant-Floor and Enterprise Requirements Similarities and Differences Plant-Floor Requirements Network Technology Standard IEEE 802.3 Ethernet and proprietary (non-standard) versions Standard IETF Internet Protocol (IPv6) and proprietary (non-standard) alternatives Industrial application layer protocols - e.g. CIP, Modbus TCP Local Area Network (LAN); smaller frames for control traffic Network availability Switch-Level and Device-Level Topologies Ring Topology is predominant for both, Redundant Star for switch topologies is emerging Standard IEEE, IEC and vendor specific Layer 2 resiliency protocols Enterprise Requirements Network Technology Standard IEEE 802.3 Ethernet Standard IETF Internet Protocol (IPv4 and IPv6) Standard application layer protocols e.g. SIP, SNMP, DNS, RTP, SSH Wide Area Network (WAN) and LAN; larger packets and frames Network availability Switch-Level topologies Redundant Star Topology is predominant Standard IEEE, IETF, and vendor specific Layer 2 and Layer 3 resiliency protocols 11
Plant-Floor and Enterprise Requirements Similarities and Differences Plant-Floor Requirements Switches Managed and Unmanaged Layer 2 is predominant Traffic types Information, control, safety, motion, time synchronization, energy management Performance Low Latency, Low Jitter Data Prioritization QoS Layer 2 & 3 IP Addressing Static Security Emerging: open by default, must close by configuration and architecture Inconsistent industrial security policies Enterprise Requirements Switches Managed Layer 2 and Layer 3 Traffic types Voice, Video, Data Performance Low Latency, Low Jitter Data Prioritization QoS Layer 3 IP Addressing Dynamic Security Pervasive Strong policies 12
Plant-Floor and Enterprise Requirements Similarities and Differences Plant-Floor Requirements Wireless Autonomous point solutions Mobile equipment (emerging) and personnel (prevalent) Computing Industrial Hardened Panel Mount Computers and Monitors Desktop Notebook 19 Rack Server Din Rail Mount Virtualization Emerging, becoming prevalent Environment Plant-floor Control Room Enterprise Requirements Wireless Centrally managed and autonomous Mobile personnel BYOD Guest access Computing Desktop, Notebook Tablets 19 Rack Server and Blade Server Unified Computing Systems (UCS) Virtualization Widespread Environment Data Center Data Communication Closet IDF - Intermediate Distribution Frame 13
Plant-Floor and Enterprise Requirements Policies - Similarities and Differences Focus Precedence of Priorities Types of Data Traffic Access Control Implications of a Device Failure Threat Protection Upgrades Plant-Floor Network 24/7 Operations, High OEE Availability Integrity Confidentiality Converged Network of Data, Control, Information, Safety and Motion Strict Physical Access Simple Network Device Access Production is Down ($$ s/hour or Worse) Isolate Threat but Keep Operating Scheduled During Downtime Enterprise Network Protecting Intellectual Property and Company Assets Confidentiality Integrity Availability Converged Network of Data, Voice and Video Strict Network Authentication and Access Policies Work-around or Wait Shut Down Access to Detected Threat Automatically Pushed During Uptime 14
Plant-Floor and Enterprise Requirements Switching - Similarities and Differences Industrial Ethernet Switches Industrial hardened Panel or DIN mount Managed or unmanaged IT Switches Campus, Data Center 19 rack mount e.g. 1RU Managed 15
Plant-Floor and Enterprise Requirements Network Topology - Similarities and Differences Switch and Device-level Topologies Controllers, Drives, and Distributed I/O Cell/Area Zone Redundant Star Flex Links Cisco Catalyst 3750 StackWise Switch Stack Ring Resilient Ethernet Protocol (REP) Cisco Catalyst 3750 StackWise Switch Stack Star/Bus Linear Cisco Catalyst 3750 StackWise Switch Stack Cisco Catalyst 2955 HMI HMI Controller HMI HMI Controllers Controllers Controllers, Drives, and Distributed I/O Cell/Area Zone Cell/Area Zone Controllers, Drives, and Distributed I/O Cell/Area Zone Controllers, Drives, and Distributed I/O Cell/Area Zone 16
Plant-Floor and Enterprise Requirements Cisco Validated Design - Similarities and Differences Cisco Validated Designs (CVD) consist of systems and solutions that are designed, tested, and documented to facilitate and improve customer deployments. These designs incorporate a wide range of technologies and products into a portfolio of solutions that have been developed to address the business needs of our customers. Cisco Validated Designs are organized by solution areas and will list one, two or all three primary types of documents: Design Guides System Assurance Guides Application Deployment Guides 17
Plant-Floor and Enterprise Requirements Network Management - Similarities and Differences Plant Engineering Information Technology Cisco Network Assistant FactoryTalk View, Faceplates SNMP and IP sweeps Establish early dialogue with your RSLogix, Add-on Profile IT counterparts Command Line Interface Device Manager Cisco Prime http://ab.rockwellautomation.com/networks-and- Communications/Stratix-5700-Ethernet-Switches 18
Best Practices for Network, Technology, Organizational and Cultural Convergence IT and Plant-Floor Engineering collaboration and sharing of best practices on: Standardization of design and technology System architecture design Protocols and services Service and support models Industrial Security Policy Consult reference architectures, reference models and industry standards: Network Segmentation Network services Domains of Trust An open, two-way dialogue is critical! 19
Rockwell Automation and Cisco Alliance Technology, Network, Cultural and Organizational Convergence Common Technology View: Achieve flexibility, visibility and efficiency through a single system architecture, using open, industry standard networking technologies, such as EtherNet/IP Converged Plantwide Ethernet Architectures: Plant-Floor focused reference architectures, comprised of Rockwell Automation and Cisco expertise, provide a foundation to successfully deploy the latest technologies optimized for both automation and IT professionals. Joint Product and Solution Collaboration: Stratix 5000 and 8000 families of Industrial Ethernet managed switches combine the best of both Rockwell Automation and Cisco to address IT and Plant-Floor priorities People and Process Optimization: Services and education to facilitate Plant-Floor and IT convergence, successful architecture deployment and efficient operations, so that critical resources can focus on increasing innovation and productivity 20
The Value in Bringing the Information Together Laboratory Information Management Systems Performan ce Production Scheduling Alarms/Events Quality Systems HMIs Control Systems Data Historians Other Database Systems Computerized Maintenance Management Systems You need robust Infrastructure Solutions to deliver the information fast, reliably and securely!
Industrial Network Security Trends Established Industrial Security Standards International Society of Automation ISA/IEC-62443 (Formerly ISA-99) Industrial Automation and Control Systems (IACS) Security Defense-in-Depth IDMZ Deployment National Institute of Standards and Technology NIST 800-82 Industrial Control System (ICS) Security Defense-in-Depth IDMZ Deployment Department of Homeland Security / Idaho National Lab DHS INL/EXT-06-11478 Control Systems Cyber Security: Defense-in-Depth Strategies Defense-in-Depth IDMZ Deployment A secure application depends on multiple layers of protection. Industrial security must be implemented as a system.
Defense-in-Depth Multiple Layers to Protect the Network and Defend the Edge Physical Security Network Security Computer Hardening Application Security Device Hardening
Strategic Alliance 24
STI Solution Technology Integrator 25
Cisco and Rockwell Automation Alliance Cultural Convergence Education Series Webcasts What every IT professional should know about Plant-Floor Networking What every Plant-Floor Engineer should know about working with IT Industrial Ethernet: Introduction to Resiliency Fundamentals of Secure Remote Access for plant-floor Applications and Data Securing Architectures and Applications for Network Convergence IT-Ready EtherNet/IP Solutions Available Online http://www.ab.com/networks/architectures.html 26
Reference Architectures Converged Plantwide Ethernet Architectures Rockwell Automation and Cisco Systems Collaboration Content relevant to both IT Network Engineers and Plant-Floor Control System Engineers Built on Technology and Industry Standards Recommendations and Design Guidance Documented configuration settings Cisco Validated Design Future-ready 27
Industry Standards Future-Ready Your Design Technology IEEE 802.3 - standard Ethernet 1588 - Precision Time Protocol (PTP) IETF - standard Internet Protocol (IP) IEC - International Electrotechnical Commission ODVA - Common Industrial Protocol (CIP) Manufacturing Purdue Reference Model for Control Hierarchy ISA-95 - Enterprise-Control System Integration ISA-99 Industrial Automation and Control Systems (IACS) Security NIST 800-82 Industrial Control System Security Built on Industry Standards 28
Plant-wide Network Architectures Logical Model Structure and Hierarchy Logical Model Converged Plantwide Ethernet (CPwE) 29
Campus Network Model Structure and Hierarchy Offers hierarchal modular topology Building blocks Easier to grow, understand and troubleshoot Creates small domains - clear demarcations and segmentation Fault domain (e.g. Layer 2 loops), broadcast domain, domains of trust (security) Multi-tier switch model Core Aggregates Distribution Switches Backbone of Network DMZ Connectivity Distribution Aggregates Access Switches Provides Layer 3 Services Access Aggregates Industrial Automation and Control System (IACS) Devices Provides Layer 2 Services Core Distribution Access 30
Logical Framework Converged Plantwide Ethernet (CPwE) Architectures Layer 2 Access Switch Layer 3 Distribution Switch Layer 3 Building Block Catalyst 3750 StackWise Switch Stack Cell/Area Zones Levels 0 2 Level 2 HMI Drive Controller Layer 2 Building HMIBlock I/O Media & Connectors Cell/Area Zone #1 Redundant Star Topology Flex Links Resiliency Rockwell Automation Stratix 8000 Layer 2 Access Switch Layer 2 I/O Building Block Controller Level 1 Controller Cell/Area Zone #2 Ring Topology Resilient Ethernet Protocol (REP) Controller The Cell/Area zone is a Layer 2 network for a functional area of the plant-floor. Key network considerations include: Structure and hierarchy using smaller Layer 2 building blocks Logical segmentation for traffic management and policy enforcement (e.g. QoS, Security) to accommodate time-sensitive applications HMI Drive HMI I/O Cell/Area Zone #3 Bus/Star Topology Level 0 Drive Drive Layer 2 Building Block 31
Logical Framework Converged Plantwide Ethernet (CPwE) Architectures Plant-Floor and Enterprise network convergence Plant engineer and IT network engineer collaboration Plant-wide EtherNet/IP Architectures Hierarchical segmentation Scalability Resiliency Traffic management Policy enforcement Security policies Defense-in-depth Secure remote access ERP, Email, Wide Area Network (WAN) Patch Management Remote Gateway Services Application Mirror AV Server FactoryTalk Application Servers View Historian AssetCentre, Transaction Manager FactoryTalk Services Platform Directory Security/Audit Data Servers Drive Controller HMI I/O Cell/Area Zone #1 Redundant Star Topology Flex Links Resiliency Remote Access Server I/O Gbps Link for Failover Detection Firewall (Active) Catalyst 6500/4500 Rockwell Automation Stratix 8000 Layer 2 Access Switch Controller I/O Firewall (Standby) Cisco ASA 5500 Catalyst 3750 StackWise Switch Stack HMI Drive Cell/Area Zone #2 Ring Topology Resilient Ethernet Protocol (REP) Demilitarized Zone (DMZ) Cisco Catalyst Switch I/O Cell/Area Zone #3 Bus/Star Topology Enterprise Zone Levels 4 and 5 Plant Firewall: Inter-zone traffic segmentation ACLs, IPS and IDS VPN Services Portal and Terminal Server proxy Industrial Zone Site Operations and Control Level 3 Network Services DNS, DHCP, syslog server Network and security mgmt HMI Cell/Area Zones Levels 0 2 Controller Drive 32
Networking Design Considerations EtherNet/IP Considerations Recommendations and guidance to help reduce Latency and Jitter, to help increase data Availability, Integrity and Confidentiality, and to help design and deploy a Robust, Secure and Future-Ready EtherNet/IP network infrastructure Robust Physical Layer Segmentation Resiliency Protocols and Redundant Topologies Time Synchronization Prioritization - Quality of Service (QoS) Multicast Management Convergence-Ready Solutions Security - Defense-in-Depth Scalable Secure Remote Access 33
Convergence-Ready Network Solutions Plant-wide Networks Partner Solution(s) e.g. OEM Industrial Plant-wide Systems Use of an industrial Ethernet protocol, such as EtherNet/IP, that fully utilizes standard Ethernet and IP as the industrial network infrastructure. Common network infrastructure devices asset utilization Future-ready - sustainability IP addressing schema: Class - address range, subnet, default gateway (routability) Implementation conventions static/dynamic, hardware/software configurable, NAT/DNS (who manages?) Use of industrial managed switches Network services such as loop prevention Integration between the network infrastructure and the control system configuration, management, diagnostics/troubleshooting 34
Convergence-Ready Network Solutions Plant-wide Networks Use of Network Services Segmentation Virtual LANs (VLANs) Structured hierarchy using Layer 2 and Layer 3 switching Topology Data prioritization - quality of service (QoS) Availability loop prevention, resilient topologies and protocols Multicast management Security stance Physical access, port security, access control lists, FactoryTalk Security Alignment with emerging industrial automation and control system (IACS) security standards such as ISA-99 and NIST 800-82 Time Synchronization Services IEEE 1588 Precision Time Protocol (PTP) Grand Master, Boundary Clock, Transparent Clock CIP Sync applications CIP Motion applications 35
EtherNet/IP Advantage Summary Single Network Technology for: Discrete Control, Process Control, Batch Control, Configuration, Information/Diagnostics, Safety Control, Time Synchronization, Motion Control and Energy Management Non industrial network traffic Voice, Video and Data Established 300+ Vendors, over 5,000,000 nodes ODVA: Cisco Systems and Rockwell Automation are principal members Standard IEEE 802.3 Ethernet and IETF TCP/IP Protocol Suite IT friendly Future-ready Sustainable; Industry Standards Optimized Asset Utilization Common network infrastructure assets Common troubleshooting tools (assets) and skills/training (human assets) for Enterprise (IT) and Plant-Floor (Industrial) networks Reduces asset management requirements thus supporting lean initiatives 36
Additional Material Cisco and Rockwell Automation Alliance Websites http://www.ab.com/networks/architectures.html Design Guides Converged plant-wide Ethernet (CPwE) Application Guides Fiber Optic Infrastructure Application Guide Education Series http://www.ab.com/networks/architectures.html Whitepapers Top 10 Recommendations for plant-wide EtherNet/IP Deployments Securing Manufacturing Computer and Controller Assets Production Software within Manufacturing Reference Architectures Achieving Secure Remote Access to plant-floor Applications and Data 37
Plant-wide Benefits of EtherNet/IP Seminar Making Factory Automation Networks Secure Follow ROKAutomation on Facebook & Twitter. Connect with us on LinkedIn. www.rockwellautomation.com