Industrial Communication. Corporate IT vs. Industrial Ethernet

Transcription

1 Industrial Communication Whitepaper Corporate IT vs. Industrial Ethernet

2 Contents Introduction... 3 Network Architecture... 4 Performance... 5 Environmental Factors... 8 Configuration and Maintenance... 9 Conclusion

3 Introduction As Ethernet grows increasingly popular as an Industrial application protocol, many choices exist relating to the nature of infrastructure equipment for a particular application. In migrating to Industrial Ethernet, how much can be lifted from the Business IT world rather than integrating a purpose-designed Industrial Ethernet regime? The Ethernet protocol has traditionally been within the realm of the IT department. Potential conflict can arise when engineering staff begin placing what are essentially field level devices until recently unrelated to mainstream IT onto the corporate Ethernet based network. Where different application priorities must coexist on a common network, how should the resulting compromise be decided? Network designs used by IT for business solutions generally follow a well established methodology and the equipment has been optimized for the environmental and application requirements of business. While IT personnel understand Ethernet and the corporate network very well, they may not be aware of the differences and requirements of that same network standard running in an industrial environment. For example, Industrial Ethernet has several performance and redundancy requirements which set an unusually high benchmark for reliability and recovery in the event of a fault. In a business setting, if network congestion causes a file server or server to hang for a few seconds, this does not typically present a problem to end-users. With hundreds of clients sharing the same resources, business network congestion often goes unnoticed. Should an Industrial Ethernet application experience a similar delay, the result could manifest itself as extended production downtime. With network characteristics so greatly dependent on the structural components and the way in which they are set up, let s examine some key differences between traditional business Ethernet infrastructure and its industrial counterpart. 3

4 Network Architecture Businesses share many similarities regarding network architectural requirements. Typical offices, cubicles, and meeting rooms drive port density requirements to the edge of the network. Starting from a floor plan of the building, overlapping circles representing a 90m horizontal cabling radius are located to ensure complete coverage for present and future requirements. Within each of those circles there are closets or rooms identified to locate Ethernet switch stacks ensuring that no Ethernet cable will exceed the 100m length limit. Stackable switches, located within that radius, have the advantage of accommodating additional ports as the business grows or changes. The closets they are located in house standard 19 rack frames sharing the climate control of the building. These are termed Intermediate Distribution Frames (IDFs). The IDFs located throughout the building or campus are then connected back to the Main Distribution Frame or MDF, typically a raised floor computer room which houses the network core. This core comprises critical resources of the business including servers, routers, and the network interface to the Internet or Intranet. The media used to connect the IDF to the MDF is typically high bandwidth Gigabit Ethernet multi-mode fiber that accommodates an inter-connection and does not exceed a 2km distance limitation. Taken overall, this approach represents a virtual business infrastructure standard. For industrial applications, port density varies widely. Some applications require just a few Ethernet ports in each location and those applications may be several hundred meters apart. Unlike offices, industrial applications have very different node density requirements. Placing a corporate standard 24-port Ethernet switch in a location that only requires four ports is expensive and wasteful. Add to that a possible need for a fiber optic interface that can span 3km or more to the next switch and you end up with a rather different infrastructure requirement not typically met with a business class component. 4

5 Performance An even more obvious difference between Business and Industrial Ethernet relates to performance criteria. The business performance requirements are driven by whether the application is clientserver, such as connectivity to a database or a Voice-over-IP (VoIP) phone connecting to VoIP PBX, versus a core network resource such as an server. Client-server application classes are more critical in terms of connectivity and this drives the benchmark requirement for performance as well as redundancy. Such client server applications tend to be more sensitive to packet loss or delays, but can still maintain connectivity for several seconds before terminating a session. Applications such as replication from servers to the desktop client can tolerate more substantial delays without impact. Because file server resources for VoIP PBX, database, , and enterprise are typically located at the network core, redundancy and performance requirements tend to be focused on this same area rather than at the edge of the network where the users actually are. There is also a matter of network redundancy. For Business applications, should a key infrastructure device such as a switch or router experience device or media failure, recovery within seconds is considered acceptable. Redundancy methods employed include the Spanning Tree Protocol (STP), or the evolved derivative, Rapid STP (RSTP). The timers used in these failover protocols do not resolve values below a second: healing the network by finding an alternative route typically takes from between a second to a minute. While this accommodates resources such as , web, Internet, Intranet, and file services, there is the minor inconvenience of perhaps dropping a phone call or having to log back into a database when the session was terminated. In any event, the productivity lost is measured in seconds or minutes and only on an individual user basis. Should there be device or media failure, the impact to the business is localized depending on the activities at the time and limited to individual users or resources. Compare this to the industrial application with essentially different traffic patterns and sensitivity to recovery time in the event of a switch or media failure. While business applications steer traffic from the edge of the network to core resources, Industrial Ethernet applications tend to be peer-topeer. Data must be exchanged between peer devices at the network edge so that programmed logic can respond to upstream or downstream changes, as well as share those changes with SCADA, HMI, I/O devices and other PLCs. The peer-to-peer nature of Industrial Ethernet drives communications and redundancy requirements rather than the edge-to-core requirements used by business/it. These Industrial Ethernet essential requirements may be defined as: Should a media failure occur, the recovery time does not affect the production application. The packet rate of communications must be supported by the infrastructure. Regarding the first requirement, media recovery taking longer than a second is unacceptable in most Industrial applications. Any interruption longer than this would usually stop production leading to downtime. Redundant Ethernet ring topologies provide sub-second recovery time and substantially address the problem. Other topologies such as dual homing in conjunction with Rapid Spanning Tree may also provide an acceptable solution. Media or device failure also takes on another dimension in an industrial application. While business systems typically operate during business hours, many industrial processes are continuous. Media failure out of hours may require expensive correction. In terms of the second essential requirement, Ethernet switches and devices now support full duplex communications. Any device or switch can transmit and receive simultaneously and the bandwidth available is 100Mbps. Fast Ethernet is more than sufficient. In fact, the bandwidth exceeds the capability of the devices to completely consume it. Additional traffic management 5

6 services such as Quality of Service reinforce efficient and reliable message delivery. Broadcast Rate Limiting is also employed to prevent excessive broadcasts from congesting device queues that would delay automation message processing. While business Ethernet switches of course offer such features, the industrial counterparts have evolved by making these attributes prominent and more easily managed using a web browser so that engineering or maintenance can use them. Performance differences between business and Industrial applications reveal that packet rate, packet size, and bandwidth use differ significantly and are summarized in Table 1. Table 1: Performance Differences Between Business and Industrial Applications Graphing test application packet captures for both industrial and business edge devices illustrates the difference. In Figures 1 and 2, the business application trend is in black and the automation application trend is in red. Business applications have packet burst rate requirements that define the upper limit bandwidth requirement of the infrastructure. The available 100Mbps bandwidth from the network edge to its core can be easily saturated when multiple users send large packet bursts. The data volume represents the larger packet sizes used by business, for example, when loading web pages or network printing. 6

7 Industrial applications on the other hand, have a higher, steady state packet rate with lower overall data volume due to the much smaller packet size as shown in Figures 3 and 4. Industrial Ethernet packet sizes are typically less than 500 bytes. These smaller packets interleave more effectively than larger packets and therefore reduce the likelihood of congestion. Since most Industrial Ethernet applications use cyclic, steady state communications, packet bursts are rare. Coupling the packet sizes and packet rates together reveals the average bandwidth occupancy over a five minute test capture period. As Fig. 5 illustrates, the overall utilization is under 2 Mbps and this is typical for many Industrial Ethernet applications. Though some voices may express concern about the high volume of communications in Industrial Ethernet, the reality is that 100Mbps full duplex Ethernet is quite sufficient for most applications. For business applications, applying increased bandwidth generally yields better performance. For Industrial Ethernet applications, additional bandwidth has little effect. Most automation edge devices do not have the processing speed and memory of business desktop PCs and servers. The key factors for industrial applications are efficiency and reliability. 7

8 Environmental Factors Business infrastructure Ethernet switches are typically installed in temperature, air quality, and humidity controlled areas with filtered power, standard surge suppressed outlets, and UPS backup the event of a power failure. There is less consideration for power redundancy in terms of dual power supplies or separate feeds for edge switches. Core switches are more likely to be equipped with redundant power supplies and this is reflected in the premium price of the core business switch. Business Ethernet switches are equipped with fans to cool the switch mother board. Should moisture or dust accumulate on the system board, the board could overheat and fail. Since the stackable edge switches are designed to accommodate typically ports should an edge switch fail due to environmental factors, the impact to the business is localized, affecting only those users attached. This hardware philosophy extends to many business edge switch designs. Cost considerations dictate that redundant power is not even an option due to the relatively localized user impact of component failure. Since the switch stacks at the edge of the IDF employ standard plug grounding, and office installation cable trays physically avoid EMI/RFI- producing sources such as HVAC and fluorescent lighting ballasts, there is less consideration given to the use of shielded horizontal cabling. This assists cost control when deploying or locating IDF closets. Slightly below the line but important none the less, corporate IT expects to replace edge switch equipment as Ethernet bandwidth increased on three to five year intervals to accommodate new technology and services. This has subtly affected the corporate attitude to MTBF in favor of providing higher bandwidth to the desktop edge through hardware upgrade on a regular basis. The historic example of Moore s Law drives business Ethernet switch OEM product life cycle and development. Industrial Ethernet systems are expected to keep working for substantially longer in much harsher environments. Fans that can draw in dust to the detriment of the enclosed electronics can threaten premature failure never convenient and often expensive. For this reason, fanless convection cooling is widely preferred by Industrial Ethernet switch OEMs (and sensible users). Industrial systems should require minimal upkeep. Environmental factors such as temperature, dust, debris, or humidity have to be properly designed for. System architects must be prepared to see production yields cost savings in the light of a cost increment on the switch itself. Industrial switch OEMs have responded by testing and certifying their product offers to survive these environments with long MTBF cycles. Electro-magnetic and radio frequency-emitting devices are common in industrial environments. Using shielded media and positive grounding not only protects the network infrastructure itself, it also reduces the risk that data packets will be corrupted during transit. The integrity of each packet is critically important to prevent delay as well as relying on higher layer protocols such as TCP to manage retransmission and error recovery mechanisms. 8

9 Configuration and Maintenance Business has highly evolved Help Desk and Network Operations support systems for configuring, managing, and maintaining the network. Problem reports are typically dispatched through a call center, which prioritizes each request and assigns it to the appropriate group or individual. Often these people share their time with projects related to deployment or roll-outs, or learning a new application while endeavoring to support it. When rolling up industrial applications and their problems into this type of organization, support request may get prioritized wrongly. If a VP cannot get their for example, that may have a much higher priority than a person on the production line reporting that a packaging machine is down and is holding up the entire line. Help Desk personnel may respond to requests more towards the political hierarchy than the bottom line business impact. Industrial network operators usually require immediate assistance in the event of a problem. Using general Help Desk personnel is a risk, particularly if they are unfamiliar with Automation applications. Responsibility for keeping the industrial network in service normally falls to the maintenance department. Maintenance usually has a fraction of the Ethernet knowledge held by the Help Desk. Industrial Ethernet OEMs have responded by developing simple tools, methods, and interfaces to allow the maintenance person to resolve the problem. Special setup and diagnostic features with web interfaces simplify the configuration options, or allow discovery and primary configuration of a switch by simply powering it up and plugging into the network with little or no reliance on the IT Help Desk. Fast replacement in the event of a device failure is essential for industrial applications and has led to simple methods for replacing a device. Should a switch or I/O device fail, maintenance can replace the device and the configuration often with just a screwdriver by matching settings on rotary switches. Signal interfaces can also help in alerting network problems. Ethernet switch housekeeping signals can be wired directly to PLC I/O modules and monitored for errors in logic. Should an error occur, this can be reported to an operator via the HMI console. All of this recommends Industrial Ethernet switches for use in industrial applications. Conclusion Industrial Ethernet continues to evolve and mesh with the traditional business Ethernet environment. Engineering system architects would do well to partner with IT to find complementary technical solutions and cross training resources to support industrial application according to their need. Business IT should also recognize the importance of Ethernet infrastructure devices suited to the task and that for Industrial Ethernet, the corporate standard for Ethernet is not necessarily best applied in an Industrial environment. 9

10 > Make the most of your energy Schneider Electric Industries SAS 35, rue Joseph Monier Due to possible changes in standards and equipment, the features described in this document in the form F Rueil-Malmaison of text and images are subject to confirmation by Schneider Electric. FRANCE Whitepaper by Michael Roche Tél. : +33 (0) Design: Schneider Electric 10