BUILDING BLOCKS FOR A SECURE REAL-TIME COMMUNICATION AND COMPUTING INFRASTRUCTURE FOR INDUSTRY 4.0

Transcription

1 INDUSTRIAL COMMUNICATION FOR FACTORIES BUILDING BLOCKS FOR A SECURE REAL-TIME COMMUNICATION AND COMPUTING INFRASTRUCTURE FOR INDUSTRY 4.0 WHITE PAPER

2 2 3 EXECUTIVE SUMMARY IMPRINT The flagship project Industrial Communication for Factories tations. An Industry 4.0 ICT architecture can be built up from (IC4F) is working on a secure communication and computing a collection of interoperating building blocks. Building Blocks for a Secure Real-Time Communication infrastructure with real-time capabilities for Industry 4.0. The and Computing Infrastructure for Industry 4.0 White Paper Version 1.0 (April 2018) Industrial Communication for Factories (IC4F). Published by the partners of the project Industrial Communication for Factories (IC4F). Internet: info@ic4f.de project is part of the PAiCE program of the German Federal Ministry for Economic Affairs and Energy (BMWi). The IC4F project develops a RAMI 4.0-compliant reference architecture for industrial communication. Specifically, on a high abstraction level, IC4F considers two layers: In order to define building blocks, available and upcoming technologies in the field of ICT, applications, and data must be analyzed. This includes technology domains such as cloud computing in an industrial environment, virtualization and industrial edge computing, 5G radio and 5G core network, big and fast data analytics, as well as artificial intelligence and machine learning algorithms. The analysis The ICT infrastructure layer provides wireless or wired includes mechanisms for secure and reliable connectivity in EDITORIAL TEAM: Erich Zielinski, Fraunhofer Heinrich Hertz Institute, Berlin, Germany Felix Beierle, Technische Universität Berlin, Germany Hans-Werner Bitzer, Deutsche Telekom AG, Bonn, Germany Knut Drachsler, GPS Gesellschaft für Produktionssysteme GmbH, Stuttgart, Germany Bernd Holfeld, Fraunhofer Heinrich Hertz Institute, Berlin, Germany Harald Klaus, Deutsche Telekom AG, Bonn, Germany Mathias Mormul, Universität Stuttgart, Germany Andreas Müller, Robert Bosch GmbH, Renningen, Germany Karoline Saatkamp, Universität Stuttgart, Germany Christian Schellenberger, Technische Universität Kaiserslautern, Germany Julius Schulz-Zander, Fraunhofer Heinrich Hertz Institute, Berlin, Germany Slawomir Stanczak, Fraunhofer Heinrich Hertz Institute, Berlin, Germany Edwin Sutedjo, Nokia Solutions and Networks, Munich, Germany PICTURE CREDITS: Nico ElNino istock (Title) pressmaster Fotolia (p. 11) scandinaviastock Fotolia (p. 12) sdecoret Fotolia (p. 16) vectorfusionart Fotolia (p. 19) PhonlamaiPhoto istock (p. 27) access to all kinds of objects on the shop floor and connects them with cloud resources in the different network domains. The application and data layer includes factory applications, data models, data management, data analytics and data visualizations, as well as artificial intelligence and machine learning algorithms. Moreover, an overarching security framework protects both layers. production, secure wireless communication and processes, massive sensor data analysis, and (virtual) network elements like secure gateways. This whitepaper briefly outlines the following four specific use cases and describes how they can be implemented based on the technologies and the building block approach developed by IC4F: Remote machine access Automated Guided Vehicles (AGVs) Matthias Wieland, Universität Stuttgart, Germany Alexander Willner, Technische Universität Berlin, Germany Florian Zeiger, Siemens AG, Munich, Germany Marc Zimmermann, Technische Universität Kaiserslautern, Germany The IC4F project proposes building blocks, which allow implementing Industry 4.0 use cases in an efficient and flexible manner and to realize use case patterns with similar Massive wireless sensor networks Mobile operation & control with ultra-reliable machine communication requirements. These building blocks define packages of CONTACT: Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute, HHI, Einsteinufer 37, Berlin, Germany functionalities to meet business needs. The building blocks are described by templates, which include a description of the functionality and context, exposed public interfaces, During the project, the IC4F consortium will present a proofof-concept implementation in real-world industrial environments for relevant use cases, including the four use cases Layout: LoeschHundLiepold Kommunikation GmbH interoperability, service parameters, and possible implemen- mentioned above.

3 4 5 CONTENTS 1. INTRODUCTION Executive Summary 3 1. Introduction 5 2. Analysis of Use Cases 8 3. Challenges and Requirements for the Building Block Approach New Technologies and Functionalities for the Smart Factory Cloud Computing and Virtualization G The Communication Network for the Cloud Era Data Driving the Smart Factory Security Protecting the Smart Factory Reference Architecture and Building Block Approach The IC4F Reference Architecture The IC4F Building Block Approach Application of Building Blocks in Demo Scenarios Remote Machine Access Automated Guided Vehicles Massive Wireless Sensor Networks Mobile Cooperation and Control with Ultra-Reliable Machine Communication About the IC4F Project 32 References, Abbreviations 34 German politics and leading industry associations together with research and development from companies and academia provide the foundation for concerted action in digitizing the industrial production process. Combining the organizational assets of all stakeholders will accelerate the speed at which the goal of fully connected factories of the future can be reached. In addition to its global goal, the project was also devised with Germany s competitiveness in mind. Germany has a unique landscape of small and medium-sized enterprises (SMEs) that account for about 95 % of the entrepreneurial forces. German companies are internationally recognized for their innovative products and for their efficient use and production of both tools and technologies for industrial production worldwide. Accordingly, continuously improving and enhancing Germany s leading position in smart production and cutting-edge products as well as the return of production facilities from foreign sites are goals pursued by the Industry 4.0 initiative that is governed by public and private research projects. Implementing the vision of Industry 4.0 requires a holistic view of the underlying infrastructure a type of industrial Internet/Intranet taking into account technical possibilities adapted to industrial requirements. This new industrial communication infrastructure that enables platforms and applications will become an important economic factor. Just recently, the telecom industry radically transformed itself by merging communication with information technologies. Communication technology performance has increased significantly over time, virtualization is reducing costs, and the fifth generation of mobile networks (5G) is expected to generate even more momentum. The use of ICT technologies in the automation and manufacturing domain, including the required adaptations to industry requirements, will bring tremendous benefits: Boost performance of production facilities thanks to tight monitoring and configuration of equipment, e.g. condition monitoring, predictive maintenance, digital twin of the factory in real-time. Close alignment of production and business processes, e.g., product customization and hyper-personalization through flexible (re-)configuration of production facilities. Connectivity of all objects in a heterogeneous environment and supporting both standardized and proprietary interfaces (interworking based on standards). Improved collaboration and increased confidence between business partners along the value chain (e.g., suppliers, distributors, tier-x) by quality-assured, secure and in-time connectivity inside factories within and across factory boundaries. Convergence between Operational Technology (OT), Information Technology (IT), and Communication Technologies (CT) seems to be the way to reach the goals set for the digitization of industrial production, and several initiatives

4 6 7 worldwide have started to work towards this goal, e.g., the Industrial Internet Consortium (IIC) or the Plattform Industrie 4.0. The flagship project Industrial Communication for Factories (IC4F), as part of the PAiCE program of the German Federal Ministry for Economic Affairs and Energy (BMWi), is working on a RAMI 4.0-compliant reference architecture for industrial communication systems and is creating building blocks for industrial communication systems that can be used in factories. The proposed building block approach addresses SMEs as well as large enterprises, providing them with a basis to develop tailor-made solutions for future Industry 4.0 use cases. IC4F applies the proposed approach and validates technologies, along with interworking and integration in different and representative demonstrators. Figure 1.1. visualizes the focus and the goal of the IC4F project, i.e., the convergence of OT, IT and CT. OT Efficiency Consistency, Continuity Safety Digital Twin in Real-Time Industrial Edge Cloud CT Mobility Collaboration Security Performance,e.g., Real-Time 5G, TSN, Virtualization,... IC4F PROJECT FIGURE 1.1.: FOCUS OF THE IC4F PROJECT. The IC4F approach is based on a thorough analysis of industrial use cases from three main sources: the IC4F project partners, the Industry 4.0 platform, and the workshops conducted together with industrial application partners of IC4F s user forum. Use cases of interest are identified on the basis of criteria such as demanding industrial requirements, clear request for new technologies, and enabling of new business opportunities. IT Flexibility Cost Reduction Security Fast Data, auto. Deployment Mobile Edge Cloud As a result, the ICT and compute infrastructure have to meet challenges and requirements along the following lines: Every object on the shop floor gets connected Objects become mobile the shop floor goes wireless Artificial Intelligence (AI) in production Fast and reliable communication for machine and process control digital twin of the factory in real-time Automated deployment and operations Comprehensive and scalable secure communication and data handling in industrial domains and processes To implement use cases in these domains, we analyze available and upcoming technologies in the field of ICT, applications and data. In particular, cloud computing in an industrial environment, virtualization and industrial edge computing, 5G (radio access and core network), analytics with big and fast data, and AI technologies are investigated. Functionalities, performance parameters and interfaces that enable new use cases are emphasized. The analysis also includes existing and innovative mechanisms for secure connectivity in production, secure wireless communication and secure processes as well as massive sensor data analysis. In doing so, we take into account new functionalities (e.g., role dependent and task-dependent data handling, scalable security services) and dedicated (virtual) network elements like security gateways and industrial edge clouds. Our aim is to establish an architecture that is able to describe the overall ICT and compute infrastructure for specific use cases. To this end, we link the functionalities described above to different levels of the architecture so that use case patterns with similar requirements can be addressed. The functionalities are viewed as building blocks on different architectural levels and this is referred to as the IC4F building block approach. The highest level (reference architecture) is very similar to generally accepted approaches like RAMI 4.0 and IIRA. However, the building block approach enables the step-by-step implementation of specific use cases. UNLICENSED AND SUBLICENSED SPECTRUM EDGE CLOUD NEW IIo T AUTHENTICATION MECHANISMS FACTORY A FACTORY B MOBILE DEVICES FACTORY FIGURE 1.2.: HIGH-LEVEL REPRESENTATION OF THE COMMUNICATION DOMAINS ADDRESSED BY IC4F. Finally, we will present four examples of uses cases based on the technologies and the building block approach assessed by IC4F: Remote machine access Automated guided vehicles Massive wireless sensor networks Mobile operation & control with ultra-reliable machine communication PUBLIC 4G/5G NETWORK PRIVATE 4G/5G LOCAL WIRELESS ACCESS POINT INTEGRATED HIGH ACCURACY INDOOR POSITIONING (HAIP) ENTERPRISE CLOUD PRIVATE 4G/5G BASE STATION WITH A LOCAL GATEWAY, INTEGRATED PAAS, IIo T PLATFORM AND ANALYTICS AUTOMATION GATEWAY (OPERATIONAL TECHNOLOGY) APPLICATION PROGRAMMING INTERFACE (ERP/MES/PM/CIM/CAX) CERTIFICATE AUTHORITY FOR INDUSTRIAL COMMUNICATION PUBLIC/PRIVATE HYBRID CLOUD INTERNET PUBLIC CLOUD MULTI-OPERATOR ENVIRONMENT REAL-TIME REMOTE MAINTENANCE AND CONTROL END-TO-END (E2E) INDUSTRIAL SLICE Q o S VIA PUBLIC INFRASTRUCTURE REMOTE OPERATIONS CENTER The IC4F consortium is planning to implement a proof-ofconcept in an industrial environment for relevant use cases including the four described above. The overall picture of the addressed domains of industrial communication is highlighted in Figure 1.2.

5 ANALYSIS OF USE CASES VALUE CHAIN INTEGRATION PRODUCTION INFORMATION TRANSPARENCY A major objective of the IC4F project is to help enterprises to implement the industrial use cases enabled by new technologies. Many definitions for the term industrial use case can be found in literature. We prefer the definition put forward by Cockburn [1]: A use case captures a contract between the stakeholders of a system about its behavior and describes the system s behavior under various conditions as it responds to a request from one of the stakeholder. We used the following sources as a basis for analyzing industrial use cases: Use cases from the IC4F application partners and associated partners. The IC4F User Forum, which includes more than 30 members from academia and industry, for discussing Industry 4.0 use cases and future solutions. The Plattform Industrie 4.0 [2], which includes hundreds of use cases that were filtered for analysis according to the field of production and logistics. For our use case analysis, we especially considered new, innovative use cases representing a trend in the field of Industry 4.0. We also included use cases with demanding industrial requirements to communication technologies beyond the state-of-the-art in our investigations. Based on the key priorities mentioned by EFFRA [3], IC4F defined four use case clusters in order to structure the use cases to be analyzed (shown in Figure 2.1.): The Value Chain Integration cluster which includes optimized processes and new business models along the industrial value chain. The Production Information Transparency cluster which focuses on the digital twin of processes and conditions in the factory for improving productivity and efficiency. The Versatile Production cluster which deals with production for user-specific products (e.g., lot size of one) and products with a short lifecycle. The Augmented Worker cluster which supports humans as actors in the field of production through assistance systems. As a result of the discussion with industrial users especially, the use cases listed below that combine several characteristics are expected to increase their performance in this context or will only then be enabled: Use cases that include mobile smart objects which need to exchange data with other objects and which cannot be wired easily (e.g., transport vehicles, mobile robots, rotating machine components). Use cases that need real-time transfer of high data volumes (e.g., acoustic or video data or data from a swarm of numerous sensors) between different locations/companies. Use cases that need ultra-high reliable wireless data (safety and low latency requirements). Use cases where wireless exchange of data at a high security level plays an important role. VERSATILE PRODUCTION FIGURE 2.1.: FOUR USE CASE CLUSTERS WITH RELEVANCE TO NEW INDUSTRIAL COMMUNICATION TECHNOLOGIES. The analysis within IC4F provides a clear picture of the concerns of stakeholders in future Industry 4.0 use cases and alignes the results of IC4F s work on architecture with the relevant audience at SMEs and large enterprises. The stakeholder concerns recorded also allow the IC4F project to derive and prioritize requirements, directly influencing the design of the IC4F implementations. Project results also include prototypes showing the proof-of-concept in representative real-world demonstrators. The IC4F demonstrators focus on the clusters Value Chain Integration, Production Information Transparency, and Versatile Production with AUGMENTED WORKER clear mapping (cf. Section 6 for a more detailed description of IC4F demonstrators): Value Chain Integration is represented by the Automated Guided Vehicles use case. Production Information Transparency is represented by the Remote Machine Access use case and the Massive Wireless Sensor Networks use case. Versatile Production is represented by the Mobile Cooperation & Control with Ultra-Reliable Machine Communication use case.

6 CHALLENGES AND REQUIRE- MENTS FOR THE BUILDING BLOCK APPROACH From the previous discussion of the use cases, it becomes communication point of view, wireless connections should obvious that use cases come in different shapes and siz- be used to avoid the spatial constraints of fixed cabling. es. Likewise, there are also many different ways to tackle underlying communication requirements. In this section, the High Bandwidth for Video use cases will be revisited to identify common denominators Outside the industrial context, the main performance charac- and a set of generic requirements that will drive technology teristic typically associated with a wireless network is band- selection and architecture for industrial networks. width, i.e., the amount of data transferred per time. While could be disrupted resulting, for instance, in the need for a factory network will consist of a hierarchy of compute re- use cases with high bandwidth requirements, such as video machine safety stop. Low and predictable latency is addressed sources that are located so that the different needs in terms Everything becomes Connected surveillance, may also exist in a factory, bandwidth as such is by ultra-reliable low-latency communication. Besides factory of speed and spatial requirements can be covered. An essential property Industry 4.0 will be a new communica- not expected to be a main driver in industrial networks. automation, tools that use Augmented Reality (AR) depend tion pattern. While a high degree of automation is already heavily on low latency in order to achieve the targeted level Sharing Infrastructure between Use Cases and Tenants state of the art in factories, Industry 4.0 adds the ability to High Device Density for Sensor Networks of usability and experience. In a real factory setup, several use cases owned and operat- seamlessly exchange data between the factory network One goal of the industrial factory network is the ability to ed by different business entities and with different communi- and the rest of the enterprise. Ubiquitous connectivity and obtain deep insights into production processes by gather- Hierarchical Infrastructure to Support Different Use Cases cation requirements will run on the same physical infrastruc- easy data exchange and access will be established between ing and analyzing data from many sensors. The number of In addition to wireless transmission, the timing requirements ture. The difficulty in these multi-tenant scenarios is how to the internet, the intranet, and the shop floor. This will pave sensors that can be connected simultaneously is an impor- of a use case must also include data processing. If a use case optimize two contradicting properties. On the one hand, re- the way for tighter integration between factory control and tant performance parameter. The energy consumed by the requires low latency between event and action, process- sources should be pooled ( shared ) between different use business processes. wireless connection should be minimized in order to enable ing will have to be executed as close to wireless access as cases and tenants to enable the best-possible utilization of a long battery lifetime. This is where truly wireless sensors possible. Collocation of access node and compute resource resources. On the other hand, resources should be isolated The Shop Floor goes Wireless without a wired power supply become feasible. is referred to as edge computing. It is used for communica- and dedicated to allow use case-specific optimizations and A close interlock between business and factory only makes tion and processing needs of objects connected to the same ensure that resources are available when needed. The con- sense when the factory can adapt to different business Fast and Reliable Communication for Machine Control edge computing instance, i.e., for a rather limited spatial cept of network slicing allows the virtual network embedding needs also in the physical world. If, for instance, a new In factory automation, the amount of data to be transferred is area only, such as a shop floor. Use cases that utilize data in a common physical network. product is to be launched, production will be executed by typically low, but the time between sending a message and from objects distributed over a larger spatial area require flexible robots, creating a new production island on demand reception of the message (referred as latency) is of uttermost can benefit from processing hierarchical cloud infrastructures. Automated Deployment and Operation rather than restructuring the entire static factory line. importance. Predictability of latency allowing constant cycle In some use cases, both requirements may even co-exist. Factory networks are complex. A manifold of use cases re- To exploit the possibilities of seamless communication times within a production network is even more important Sensor data is utilized in edge computing to enable shop sulting in different requirements, a rich choice of technology between machine control and business processes, physical than low absolute latency. With higher, but predictable laten- floor automation, and the same data can be used together options, and various possibilities for deploying these on a vir- flexibility on the shop floor is needed in order to allow for cy, a production process can still operate at a lower speed. with data from other shop floors, e.g., for analytics-based tualized hierarchical infrastructure will have to be considered. the free flow of production equipment and material. From a In the case of unpredictable latency, the entire production process optimization in a central cloud. This means that a Furthermore, an industrial network is not static.

7 NEW TECHNOLOGIES AND FUNCTIONALITIES FOR THE SMART FACTORY The IC4F project analyzes available and upcoming technolo- lenges since edge computing resources in today s approach- gies in the field of ICT, applications, and data. es are still located away from production and shop floor environments (introducing additional constraints with respect All of the above factors change over time and factory networks need to adapt to these changes. A high degree Security In the context of Industry 4.0, security is becoming even 4.1. Cloud Computing and Virtualization to real-time requirements), or do not support the industrial communication protocols required in OT. of automation is therefore a very important requirement for more important. In the past, automation networks were iso- the setup and operation of factory networks. The employed lated from the rest of the world, thus offering rather limited Cloud Computing in Industrial Environments The concept of an industrial edge cloud introduces a heter- ICT automation framework (not be confused with the points of attack. With the expansion of the Internet to the The majority of the Industry 4.0 use cases [2] discussed aim ogeneous resource pool for processing power and virtual- cyber-physical automation taking place on the shop floor) cyber-physical domain, attack scenarios familiar from the In- for flexible production and optimized efficiency through ad- ization (NFV, virtual networks, virtual working environments) must comprise deployable ( virtualized ) functions used to ternet are becoming relevant. An intruder does not have to vanced data analytics, so that these use cases depend heav- on the shop floor. Of course, the resource pool needs to build the factory network, a deployment system that pushes be inside the factory in order to launch an attack. Instead, a ily on cloud computing capabilities. Today, state-of-the-art support key industry requirements, such as stringent QoS these functions on the infrastructure, and an orchestration hacker can launch the attack via a cloud system and corrupt solutions connect machine data sources to industrial cloud requirements, redundancy concepts, safety features, or framework that generates the required communication links or even hijack a production environment from there. To pre- backend systems and much effort goes into establishing industrial communication protocols. Industrial edge clouds between these functions. vent scenarios like these, security must be an integral part of communication solutions that follow standards that comply therefore allow for an efficient use of shared resources in OT an industrial network where communication only takes place with industrial requirements, e.g., OPC UA. Current research environments with a strong focus on safe, secure and reliable FUNCTIONAL OPERATIONAL between verified identities and where end-to-end protection is used. Furthermore, a fine granular access management is now exploring the integration of backend and edge-cloud systems in an industrial context in order to enable seamless industrial processes. CONNECT EVERYTHING AUTOMATION system is needed to limit access to resources to eligible entities only. interaction of on-site cloud deployments (e.g., industrial edge clouds) and industrial backend cloud systems. Physical resources are available to different stakeholders/ actors who can use their own virtual resources according WIRELESS ACCESS MULTI-TENANCY Compatibility with Legacy and Heterogeneous Virtualization & Industrial Edge Computing to the given agreements, but without interfering with other actors resource assignments, and virtualization offers a suit- DISTRIBUTION FOR LOCAL HIGH-SPEED CENTRALIZATION FOR GLOBAL SCALE SECURITY COMPATIBILITY Environments Although a consistent and uniform rollout of an industrial network according to the described ideal requirements is desirable, the reality is sure to be different. Existing equipment, purchased before the dawn of Industry 4.0, will have Edge computing approaches in service provider infrastructures and IT/communication networks leverage the processing power available at the edge of the network, e.g., by providing processing power and/or storage close to the edge of networks. able trade-off between resource pooling (shared use of physical resources) and isolation (stakeholders can use assigned logical resources independent of each other). Mapping feature requests from Industry 4.0 use cases to industrial cloud concepts shows that scenarios also foresee a service setup to continue to operate together with native Industry 4.0 across cloud instances, e.g., services, virtual tenant networks FIGURE 3.1.: MAIN FUNCTIONAL AND OPERATIONAL REQUIREMENTS FOR INDUSTRIAL NETWORKS equipment. Mapping existing mobile edge computing approaches to the industry domain reveals unanswered questions and chal- or virtual work spaces connecting resources from industrial edge clouds to traditional enterprise or public clouds.

8 G The Communication Network for the Cloud Era Cellular mobile networks have been driven by the needs of human communications evolving from voice and data communication networks provided by 2G and 3G networks towards the mobile web in LTE. While LTE already includes some elements, such as narrowband communication, that target communication between machines, 5G [4] is especially designed for the Internet of Things and to fulfill the need of vertical industries. It consists of a 5G New Radio (NR) interface and enhancements to the core network, needed 5GC. While 5G NR provides the technical basis in terms of the performance needed for the wireless transmission link, 5GC with its service-based architecture enables the agile and intent-driven deployment of the network according to the requirements of specific use cases. 5G Performance The performance of 5G systems can be summarized as follows: embb - enhanced Mobile Broadband: data volumes reach 10 Tbps/km² and peak rates of 10 Gbps mmtc - massive Machine Type Communication: high IoT device density of 1 million/km 2 and optimized energy consumption targeted at 10 % of LTE reference URLLC - ultra-reliable low-latency communication: one-way latency below 1 ms, reliability of five 9 s and high mobility 5G New Radio Design Principles With regard to the wireless transmission of data, the above goals for performance are to be achieved with the following main technical design principles (among others): Increase overall wireless link capacity: New spectrum options from approx. 400 MHz to 100 GHz in licensed and unlicensed bands will be available and utilized by ultra-small up to macro cells. Decrease latency: Very short packet lengths can be used. Increase reliability: The same data is submitted in a redundant fashion using multiple channels (referred to as diversity), utilizing, for instance, different frequency bands, antennas or access points. The latter point is especially important with a view to reliability when devices are handed over from one cell to another. From an architectural point of view, the access points are split into two components called Remote Unit (RU) and Central Unit (CU). While the RUs hold the radio interface, the CUs are responsible for controlling the radio resources from several RUs. The CUs can be deployed as virtualized functions on an industrial edge cloud, for instance, and in this way complements the service-based architecture of the 5GC, which is explained below. 5G Core Service-Based Architecture The essence of 5G s service based architecture (SBA) can be described by the following three principles: Following the paradigm of software-defined networking (SDN), network elements are completely decoupled into software and hardware. The software parts are provided in the form of virtualized network functions (VNFs) as part of the network function virtualization (NFV) concept. They are developed following cloud native design patterns (like micro services or stateless operations) and are thus well-suited for deployment on edge or central clouds (referred to as a 5G multi-layer cloud architecture). Dynamic interaction between network functions, which replace the static point-to-point connections between network elements in traditional networks, is achieved through service-based interfaces that use HTTP 2.0 transport. The new 5G Network Repository Function (NRF) takes care of service registration and discovery. The network exposure functions offer Application Programming Interfaces (APIs) that enable external entities, like factory operators, to control and monitor network policies on an individual device basis. With the help of the above properties, it is possible to instantiate a set of network functions to form a complete network so that the requirements of a predefined use case can be fulfilled. With this technique, known as network slicing, different logical networks extending from device to data processing can be deployed on top of one physical infrastructure, with each slice optimized with respect to different performance criteria such as latency or bandwidth. Bridging the gap from use cases with real world requirements to a tailored connectivity service is achieved with the help of Service Level Agreements (SLAs) that describe the SERVICE LEVEL NETWORK LEVEL RESOURCE LEVEL USE CASES REAL WORLD REQUIREMENTS SERVICE LEVEL AGREEMENTS TOPOLOGY, QOS, RELIABILITY SLICE #1 ( e.g., URLLC) SLICE #2 ( e.g., e MBB) SLICE #N DEFINE VIRTUALIZED RESOURCES AND NETWORK FUNCTIONS REQUESTS VIRTUAL NETWORK FIGURE 4.1.: GENERATION OF NETWORK SLICES BASED ON USE CASE REQUIREMENTS requirements of the use case in a formalized way. These SLAs are passed to a service management entity that selects appropriate resources from the resource pool and deploys virtualized network functions. In this way, network slices optimized for the respective use case can be generated in an automated fashion (shown in Figure 4.1). INSTANTIATES SLICE OFFERS RESOURCES AND FUNCTIONS SERVICE MANAGEMENT

9 Data Driving the Smart Factory the shop floor. This data can then be fed into an engine that Artificial Intelligence and Machine Learning in the pert knowledge is incorporated by devising hybrid-driven AI/ not only allows fast, real-time, streaming-based processing Smart Factory ML solutions that optimally combine data and model-based Architecture for Smart Data but also stores relevant sensor data to consider the current Modern communication networks and massively deployed approaches. One major challenge is the implementation of an architec- and historical digital representation of the given artifact. This sensor networks, in particular, collect, generate and pro- ture for big and fast data that enables all the steps in the results in a data-driven production with learning capability, in cess a huge amount of data. Reliable and efficient access Building Blocks for Data Processing in Edge and MAPE loop (Monitoring, Analysis, Planning, and Execution) which observed behavior is used by prediction mechanisms. to this data in real-time will accelerate the advancement Cloud Computing and that addresses the required latency and volume of data of AI/ML technologies for use in the context of Industry As a result, generic functionalities bundled as components processing. Furthermore, this architecture must be able to Analytics in the Smart Factory 4.0. In addition to enabling new industrial applications and are needed for sensor data acquisition, data storage, data support the different steps, which are to be executed to a The future shop floor will contain a large range of sensors businesses, these technologies will help to cope with the analysis, data visualization, and industrial processing. Exe- certain degree in the edge, in order to allow preprocessing. ranging from temperature, humidity, audio, or light to video hugely increased complexity of communication networks, for cution components are used to close the loop and feed the In addition, the data from different edge environments in a and location data streams from moving vehicles or robots. instance, they will enhance their efficiency and robustness by results back to the shop floor. To connect the components, cloud architecture must be centrally aggregated. There are Such massive sensor networks act as enablers for a variety enabling new communication technologies and by making sensors, and shop floor artifacts, a reliable and fast connec- existing architectures for centralized big data implementa- of specific use cases or applications for controlling machines, the vision of self-organizing networks reality. ML technolo- tion framework is needed. Based on the design paradigm of tions, for instance, the Lambda and the Kappa architecture. monitoring, anomaly detection, visualization, or long-term gies are expected to provide robust predictions that are not edge computing, for fast communication between co-lo- However, in the case of Industry 4.0, the focus of interest is data analysis. Different scenarios pose different requirements only a basis for industrial applications, such as predictive cated devices and to support analyses of data streams with shifting from the notion of big data to the idea of distribut- for the building blocks of the system. Training machine maintenance, but are also a key ingredient in the design of low-latency requirements, the components can be deployed ed smart data. Cyber-physical systems typically use sensors learning models on large amounts of collected sensor data is ultra-reliable low-latency communication networks. directly on an edge node. In order to cover the entire to obtain the situation, condition, and movement data of a big data scenario, while video stream analysis from moving cyber-physical system or for long-term analysis, a cloud com- artifacts (processes, machines, equipment, and products) on robots poses low-latency requirements. Since the importance of wireless communication for indus- puting backend can fulfill the demands for higher disk space trial applications is constantly increasing, new AI/ML tech- and computing power. nologies will have to be developed for big data analytics in wireless networks. These technologies need to take into account the limitations of wireless networks (e.g., limited bandwidth, severe limitations on battery capacity and computing power, etc.) to fully exploit their inherent properties. The main challenges posed by wireless networks include the high mobility of mobile devices, which leads to changes in network topology. In addition, noisy, capacity-limited wireless links are generally exposed to interference, making them error-prone and unreliable. The limitations of wireless networks together with the fact that data is distributed at different geographical locations call for the development of distributed AI/ML methods of low-complexity for the efficient use of scarce wireless resources. While being amenable to real-time implementation, the methods envisioned will have to have good tracking capabilities and provide robust results based on relatively small data sets and under strict latency constraints. In order to achieve these goals, and also to meet the stringent requirements of many industrial applications, it is essential that the rich structure of the wireless channel and the propagating signals are exploited while the context information and ex-

10 Security Protecting the Smart Factory Furthermore, a significant challenge is that most of the Industrial IoT (IIoT) [5] infrastructure is designed for long Increased connectivity and in turn increased data processing life cycles. This means that the components responsible for leads to new mobile and modular production methods that system security must also be safe in the long term so that have new security requirements. With these new approaches, facilities have to update or upgrade security mechanisms, huge amounts of data will be transmitted over a wireless methods, and services in line with industry standards and connection and processed, for example, on the edge cloud. production processes. Traditional security approaches, such as network layering with firewalls, have to be adapted or completely replaced Cloud-based security services as well as applications on the with up-to-date security technologies like intrusion detection gateway provide reliable access management by setting up and end-to-end encryption. OT security needs to address a role-based connection with requirement-specific restric- requirements, such as real-time processing, long life cycles tions for remote maintenance or control. This bridges differ- and proprietary protocols. Security should no longer be seen ent wired and wireless network technologies and supports as an on-top option, but considered as soon as new systems different industrial application standards, such as OPC-UA are planned in order to protect data, prevent incidents, and or MQTT. It also creates, manages, and distributes digital improve the reliability of Industry 4.0 production processes. identities by utilizing a public key infrastructure (PKI). Thanks That being said, however, new security approaches will have to digital identities, trusted nodes can be used in a massive to be compatible with old industrial systems. Three general sensor network without any intrusion by malicious devices. topics have been identified and will be described in detail: Reliable Wireless Communication Protection Secure Connectivity end-to-end security in production for the new Medium ing device like a jammer, plant operators have to know the Near real-time industrial data analytics may also rely on new Reliable Wireless Communication protection for the In the future, wireless industrial communication could precise location of the device. The operator can then either processing methods, e.g., by leveraging machine learning. new medium increase, providing mobility and flexible ad-hoc commu- turn the malicious device off or inform the authorities about These new methods allow anomalies in production data to Monitoring Processes the use of edge cloud and nication between the machines themselves and between its existence and location. This kind of system can be also be detected and can indicate machine manipulation or ma- data analytics machines and the Industry 4.0 product. In order to ensure used to identify machines and processes that interfere with licious intrusion. Analysis of sensor data can also be used for reliable and secure wireless communication, additional data the radio channel, e.g., like frequency converters or welding predictive maintenance in order to detect a machine failure Secure Connectivity End-to-End Security in Production analysis and detection methods will be used. robots so that appropriate measures, such as EM shielding, before it happens so that preemptive action can be taken. Industry 4.0 production processes are becoming more and can be taken. Audio data, for example, can be used to listen to anomalies more complex. Production plants are made up of modular First and foremost, a comprehensive authentication scheme that indicate failure in engines, bearings or shafts. The more machines that can be rearranged individually and commu- for devices and encryption of data ensures that data can- Monitoring Processes and Data Analytics data is acquired, the more computing power will be needed. nicate with each other. Additionally, machines are able to not be altered or false data injected. However, a growing Massive sensor networks in Industry 4.0 production plants Depending on latency and power requirements, the pro- communicate with other Industry 4.0 assets. Due to these number of wireless-enabled devices and wireless transmis- constantly monitor the environment in order to detect anom- cessing units can be placed both on the edge of the network new communication possibilities, the new security require- sions will impact the stability and reliability of a wireless alies or to identify attrition to support for instance predictive (edge cloud) or centrally. The use of new detection methods ments mentioned earlier must be taken into consideration. connection. Simultaneous wireless transmissions especially maintenance. This, accordingly, generates huge amounts enables the detection of failures in hardware and software Secure end-to-end communication is needed for remote can cause interference and, accordingly, degrade transmis- of measured data for data analysis, i.e., making big data that may be caused by wear and tear or attack. With the access in order to load updates and read maintenance infor- sion rates or even disrupt connections. To identify the root analysis vital if the information is to be processed efficient- factory now connected to the enterprise network or even the mation. A gateway is hence introduced to ensure a secure cause of a wireless transmission disruption, classification can ly. This means that distributed data storage is essential for Internet, new threats must be addressed which are familiar connection between devices and remote operators. The be used to determine whether the interference was uninten- storing huge amounts of data. Furthermore, some informa- from the Internet. A hacker could launch an attack from security gateway, which will be placed as a hardware trust tional or malicious. Classifying the interference allows the tion needs to be processed as close as possible to the origin cloud-based services or could hijack parts of the production anchor, enables existing production facilities for Industry 4.0 appropriate measures to be selected, e.g., to either identify to reduce latency, e.g., when near real-time requirements environment. This cannot be prevented if the attacker uses applications. Devices will also require a mechanism so that a jamming device or perform radio resource management are paramount. Moreover, in order to protect the data, the zero-day or known exploits, but if a breach is detected, the they can authenticate each other in order to start trusted in order to prevent a disruption or massive loss of perfor- system needs the capabilities for inherent encryption and infected device can be excluded from communication in device-to-device communication. mance. In order to be able to switch off a malicious interfer- user management for access control. order to protect the other devices from infection.

11 REFERENCE ARCHITECTURE AND BUILDING BLOCK APPROACH REQUIREMENTS FROM USE CASES BUSINESS PROCESSES APPLICATION DOMAIN, DATA ANALYTICS DATA, DATA MODEL, SERVICES PLATFORM MANAGEMENT & CONTROL COMPUTING NETWORKING STORAGE EDGE CLOUD SECURITY ICT INFRASTRUCTURE ENTERPRISE CLOUD (PRIVATE/HYBRID) PRIVATE INTERFACES AT COMPANY BOUNDARIES PUBLIC CLOUD PUBLIC WIRED WIRELESS APPLICATION LAYER SECURITY COMMUNICATION & COMPUTING INFRASTRUCTURE ACCESS SUBSYSTEM This section describes our approach to the IC4F reference architecture. First of all, the layers of the architecture are described. This is followed by how the architecture can be used to realize real-world implementations (Section 5.1) using our building block approach (Section 5.2) The IC4F Reference Architecture Industry 4.0 is bringing new business opportunities while raising new challenges for the underlying ICT infrastructure in the context of the factory of the future. The IC4F project is examining the convergence of operational technology, information technology, and communication technologies in order to fulfill the requirements of the Industry 4.0 use cases. To this end, IC4F takes a holistic view of the industrial ICT infrastructure, applications, and data models. In particular, this approach goes beyond a pure physical view of the communication infrastructure (box view), as it considers higher layers and application frameworks. It also addresses scenarios like cloud computing on the shop floor, 4G/5G in the factory, and scalable fast data architectures for massive sensor networks. Consequently, the resulting IC4F reference architecture can be described on a high abstraction level by two layers: The ICT infrastructure layer provides wireless or wired connectivity to all objects on the shop floor and may connect them with cloud resources in different network domains The application and data layer includes factory applications; modeling, management, analytics, and visualization of data; as well as AI algorithms. Both layers are complemented by security as well as management and control functions that are frameworks rather than functions represented within a single layer. The placement of the physical systems is especially important with a view to security, availability and scalability. The placement may range from close to the production process on the shop floor, e.g., sensors that monitor the system state or wireless URLLC connections for closed loop machine control, up to external partners along the value chain who may be connected via public networks. Placement in this case stems from the requirements of the use case (cf., chapter 3), e.g., low latency requirements or specific security requirements in a certain network domain. Furthermore, use cases that cover different position ranges may require specific solutions, e.g., an edge cloud for low latency applications or a security gateway for remote access via the public internet. Figure 5.1 shows the different perspectives considered in Industry 4.0 use cases. Based on the use case requirements, the application and data as well as the underlying ICT infrastructure can be defined and implemented. One objective of the IC4F project is to capture architecture knowledge in the different domains in building blocks that can then be reused by enterprises to build their own architectures. SENSORS ACTUATORS MACHINE EQUIPMENT PRODUCTION CELL AND LINE FACTORY COMPANY VALUE CHAIN BUILDING BLOCKS FOR A SECURE REAL-TIME COMMUNICATION AND COMPUTING INFRASTRUCTURE IN INDUSTRY 4.0 FIGURE 5.1.: IC4F REFERENCE ARCHITECTURE FOR AN INDUSTRIAL ICT INFRASTRUCTURE, APPLICATION AND DATA The IC4F approach corresponds clearly with existing frameworks and reference architectures for communication and Internet technologies (OSI model), for software architectures (The Open Group Architectural Framework, TOGAF [6]), and for the industrial context (RAMI 4.0 [7] and Industrial Internet Reference Architecture (IIRA) [8]). In RAMI 4.0, communication is one of the horizontal layers, which is defined as the mechanism to exchange information and to form an integrated physical asset. Accordingly, the IC4F architecture may be viewed as one facet of the RAMI 4.0 cube (hierarchy levels IEC62264 / IEC61512) where the ICT infrastructure layer corresponds to the RAMI 4.0 communication layer while the application and data layer corresponds to the information layer. On the other hand, the Industrial Internet Consortium (IIC) goes one step further and extends its Industrial Internet Reference Architecture PLANT LEVEL (IIRA) with an industrial communication framework. In this framework, communication is further split into several layers. These layers are inspired by the OSI model. The IIC provides a framework that can be used to structure Industry 4.0 topics. The choice of Internet technology and the introduction of an OSI-like communication model are important steps towards practical implementations in all of the approaches. However, RAMI 4.0 and IIRA models still lack important steps before industrial use cases can be implemented: Each of the layers can be implemented using different technology choices. Thus, the best technology needs to be selected with regard to the use case requirements. The technologies selected need to be finally deployed on a physical infrastructure to enable efficient implementation of communication-driven factory applications.

12 22 23 Based on the analysis of industrial use cases and existing technologies, the IC4F project addresses these points and provides building blocks for solutions in a much finer granularity. This building block approach should help SMEs to implement their use cases. The overall IC4F approach to implement a specific use case is depicted in Figure 5.2. Predefined building blocks can be selected to create the architecture for different use cases. These describe the functionalities required to meet REFERENCE ARCHITECTURE DATA FLOW VIEW NETWORK VIEW OTHERS DESIGN PATTERN CONDITION MONITORING MOBILE ROBOTICS OTHERS TO ARCHITECT CONCEPTUAL AND ARCHITECTUAL PERSPECTIVE INFORMATION SYSTEM (APPLICATION & DATA) TECHNOLOGY (ICT INFRASTRUCTURE) ARCHITECTURE BUIL- DING BLOCKS (ABBs) ABB SECURITY ABB the business needs in a vendor and product-independent manner. Accordingly, these reusable architectural building blocks can be used to design the solution for a specific use case via the solution building blocks. The solution building blocks implement the functionalities described by the architectural building blocks. In the IC4F project, demonstrating specific use cases will be used to validate the IC4F reference architecture. TO DESIGN SOLUTION AND IMPLEMENTATION PERSPECTIVE IC4F DEMONSTRATOR 1 IC4F DEMONSTRATOR 2 IC4F DEMONSTRATOR IC4F DEMONSTRATOR N SOLUTION BUILDING BLOCKS TO BUILD REAL IMPLEMENTATION 5.2. The IC4F Building Block Approach The objective of the IC4F project is to define the reference architecture and to provide building blocks to implement Industry 4.0 use cases based on existing enterprise architecture standards. In conformity with the ISO/IEC/IEEE 42010:2011 standard, The Open Group Architecture Framework (TOGAF [9]) provides an Architecture Development Method (ADM) and concepts for defining architectures for different perspectives and for iteratively refining architecture building blocks to form solution building blocks in order to implement a specific enterprise architecture. It is based on an iterative process model supported by best practices and a reusable set of existing architecture building blocks [10,11]. The IC4F approach applies to TOGAF because it addresses the different architectures required not only for an enterprise architecture but also for the factory. It also provides a practical and intuitive building block approach while the ADM, as a generic framework, supports the development of a foundation architecture made up of architecture building blocks that can be reused in specific use cases. TOGAF therefore provides methods and concepts that help us to achieve the overall objective of a reference architecture with generic, reusable building blocks. The IC4F reference architecture based on TOGAF is described below. Figure 5.3 depicts different architectures addressed by TOGAF and how the domains mainly addressed by the IC4F project fit into these architectures. Based on the TOGAF ADM, the business is first developed followed by the data and application and finally the technology architecture. These phases of the architecture development method are used to define reusable architecture building blocks for the different architectures and serves as a basis for implementing specific use cases. Architecture Building Blocks (Conceptual View) Architecture building blocks (ABBs) define packages of functionalities to meet business needs. Furthermore, building blocks are described by templates which include a description of functionality and context, exposed public interfaces, interoperability, service parameters, and possible implementations. and possible implementations. Use cases can be built up from a collection of interoperating building blocks. Therefore, interfaces and relations to other building blocks need to be defined as well. Moreover, ABBs can be defined at different levels of detail. Accordingly, depending on the objective of the building block, both generic and refined ABBs can be defined to facilitate the support of generic as well as more specific functionalities. BEST PRACTICES SYSTEM DESIGN IMPROVEMENTS FIELD FEEDBACK BUSINESS ARCHITECTURE INFORMATION SYSTEM ARCHITECTURE COMMON TERMINOLOGY AND TAXONOMY FUTURE TRENDS BEST PRACTICE TEMPLATES OVERVIEW OF TECHNOLOGY BUILDING BLOCKS SERVICE PARAMETERS TECHNOLOGY ROADMAP AND MIGRATION STRATEGIES DESIGN AND INTEGRATION TESTING PLAN IMPLEMENTATION DOCUMENTATION APPLICATION COMMUNICATION AND COMPUTE TECHNOLOGY ARCHITECTURE DEPLOYMENT DATA INFRASTRUCTURE AND HARDWARE SECURITY FIGURE 5.2.: IC4F S OVERALL BUILDING BLOCK APPROACH FOR IMPLEMENTING USE CASES FIGURE 5.3.: USE OF TOGAF ARCHITECTURES TO SPECIFY THE IC4F REFERENCE ARCHITECTURE

13 24 25 The purpose of generic architecture building blocks is to provide an orientation within the framework and to under- from the edge to public nodes. The IC4F building blocks are continuously advanced throughout the project. In particular, OPEN WORLD APPLICATION AND DATA stand the related concepts for a certain use case. Since the the framework is extendable to consider future trends and placement of ICT and application components plays an important role in Industry 4.0 use cases, this placement consideration must also be taken into account for the generic architecture building blocks. Possible placement domains are the machine, factory, enterprise, or public (open world) level as shown in Figures 5.4. and 5.5. Each domain contains technologies. When it comes to flexibility and dynamics in a distributed end-to-end scenario/use case, two levels can be distinguished. On the communication infrastructure level, SDN and NFV technologies allow for different optimized deploy- ENTERPRISE (CENTRAL CLOUD) FACTORY (EDGE CLOUD) CONNECTED MACHINE EDGE DATA MANAGEMENT CENTRAL DATA MANAGEMENT DATA VISUALIZATION generic functions, such as compute, storage, networking and access. This references current operational domains such ments for multiple distribution schemes according to changing needs and topologies. On the service and application DATA ANALYTICS as public cloud/networks, IT cloud network and shop floor/ level, similar degrees of freedom and optimization potential OT networks. Today, these domains usually operate independently. The IC4F project plans to investigate the seamless use across domains, e.g., connectivity and QoS mechanisms can be achieved with micro-services, modularized applications, and orchestration frameworks like TOSCA. Building blocks are, for instance edge computing, Industrial wireless, DATA PRODUCER MESSAGING MIDDLEWARE (PLATFORM SERVICE, NOT PART OF APPLICATION SERVICE) from the shop floor to remote sites, cloud resource access and (big) data analytics. INDUSTRIAL SERVICES, E.G. MES BUSINESS SERVICES, E.G. ERP TECHNOLOGY LOGICAL COMMUNICATION PATH OPEN WORLD PUBLIC WIRELESS PUBLIC NETWORK PUBLIC COMPUTE PUBLIC WIRED ACCESS INTERNET PUBLIC STORAGE FIGURE 5.5.: HIGH-LEVEL CONCEPT SHOWING AN EXAMPLE OF BUILDING BLOCKS FOR THE INFORMATION SYSTEM ARCHITECTURE ENTERPRISE FACTORY WIRELESS CONNECTED MACHINE CONTROL UNIT FIELD NETWORK WIRELESS MODEM WIRED CONNECTED MACHINE CONTROL UNIT FIELD NETWORK ENTERPRISE WIRELESS INDUSTRIAL WIRELESS INDUSTRIAL NETWORK ENTERPRISE STORAGE ENTERPRISE COMPUTE ENTERPRISE NETWORK INDUSTRIAL COMPUTE LOGICAL COMMUNICATION PATH FIGURE 5.4.: HIGH-LEVEL CONCEPT SHOWING AN EXAMPLE OF BUILDING BLOCKS FOR THE TECHNOLOGY ARCHITECTURE ICT infrastructures have traditionally been separated in various physical areas like the field/machine, shop floor/factory, enterprise and public area. In the past, different technologies, ecosystems, and business models have evolved along these separation lines. In the IC4F project, we expect that that these boundaries are successively breaking down and that technologies from one area can be adapted and used in other areas. One example of this, is the virtualization of compute resources. In addition to making resources available for multiple purposes, these are also interconnected across the different areas. This means that there is a network of computer resources available, ranging from local, enterprise wide to public compute resources, that forms a seamless compute cloud. Figure 5.4. shows how the different areas with various computing, networking, storage, and wireless functions in the domains could be interconnected. It provides a view of the building blocks for more detailed solutions within the overarching ICT infrastructure. The application and data domain (see Figure 5.5.) contains generic blocks that depict the logical data flow from data producers, data distribution between the various user applications, data management, data processing up to its visualization. Unlike the ICT Infrastructure, where the focus is more on the physical and virtual infrastructure, the emphasis here is on the logical data flow. There is a generic flow, i.e., data is generated, transported, processed, analyzed and then visualized somewhere or further events are caused. Within this pattern, the data may cross various areas, it may be processed and used at any place, depending on the specific need. Furthermore, as data has a tendency to grow along that flow line (i.e., replicating and generating new data), a new need to manage data arises in the respective area. This covers functions to store data at the right place, transform it where needed, and make it available when permitted. There are also area-specific

14 26 27 services like Manufacturing Execution System (MES) for the factory area and Enterprise Resource Planning (ERP) in the enterprise area that utilize the data flow above. Along the areas above, there is a correlation between the type of data and services/applications running on top of a certain type of ICT infrastructure. In the past, these were hard boundaries. The IC4F project is investigating what needs to be done in order to establish communication across these boundaries in a controlled and defined way. Refined architecture building blocks can be defined to meet specific use case requirements, following the generic building blocks approach. The refinement is based on an iterative process of selecting appropriate building blocks for a specific use case. With refined architecture building blocks, technology choices, interworking, solution integration/interfacing, and migration strategies can be considered and visualized. Figure 5.6. shows examples of functionalities (building blocks) which are possible choices for the architecture of particular use cases. In this figure, blue building blocks represent generic building blocks and orange blocks represent the more specific refined building blocks. Independent of their refinement level, these building blocks are vendor and product independent. Solution Building Blocks (Solution/Instantiation View) The solution building blocks (SBBs) represent vendor-specific deployable/executable components related to the architecture building blocks. The SBBs provide the performance details required for the implementation of specific use cases. In the IC4F project, SBBs are generated within the scope of the selected IC4F demonstrators selected. However, these are vendor and use-case specific and consequently do not embody a general view. 6. APPLICATION OF BUILDING BLOCKS IN DEMO SCENARIOS In order to validate the reference architecture and building block approach outlined in the previous section and to show its practical relevance, it is essential that some real-world examples are considered along with how this approach can be used to implement concrete use cases and applications. To this end, four different use cases, which are outlined in Section 2, are briefly discussed. Specifically, the four different use case are a subset of the uses cases which will be shown through ten advanced demonstrators. INFORMATION SYSTEM ARCHITECTURE TECHNOLOGY ARCHITECTURE SECURITY DATA ANALYTICS DATA STORAGE & MGMT. WIRELESS CONNECTIVITY WIRED NETWORK CLOUD ORCHESTRATION SECURE GATEWAY BIG DATA ANALYTICS/BATCH PROCESSING RELATIONAL DATABASE MGMT. SYSTEM WIRELESSHART PROFINET TOSCA APPLICATION DEPLOYMENT & MGMT. ENGINE TCOS SMART CARD COMPLEX EVENT PROCESSING NOSQL DATABASE MGMT. SYSTEM WLAN SERCOS TOSCA APPLICATION & MODELING TOOL ROLE MANAGEMENT STREAM ANALYTICS NEWSQL DATABASE MGMT. SYSTEM MULTEFIRE ETHERNET TOSCA CLOUD SERVICE TEMPLATE CERTIFICATE/KEY MANAGEMENT TIME SERIES DATABASE MGMT. SYSTEM 4G - LTE MPLS PUBLIC KEY INFRASTRUCTURE DATA LAKE 5G-NEW RADIO TSN FIGURE 5.6.: EXAMPLES OF ARCHITECTURE BUILDING BLOCKS AT DIFFERENT LEVELS (MARKED IN BLUE AND ORANGE)

15 28 29 REMOTE SERVICE CENTER FACTORY SUPPORT SYSTEM WAREHOUSE DIGITAL TWIN DATA VISUALIZATION SECURITY GW ANALYTICS VISUALIZATION APPLICATIONS/ SERVICES ANOMALY ANALYTICS LOCATION ANALYTICS TOSCA DEVICE MODELING VISUAL ANALYTICS PUBLIC 4G NETWORK PLAT- FORM DATA STREAM PROCESSING ICT INFRASTRUCTURE MQTT BROKER TOSCA OR- CHESTRATION TRANSPORT PROTOCOLS DATA MANAGEMENT FACTORY MACHINE 4G MODEM SECURITY GW SENSOR / ACTUATOR / CONTROLLER LOCATION BEACONS QR CODE 4G SYSTEM URLLC SYSTEM NETORK MANAGEMENT ENTERPRISE NETWORK IT EDGE CLOUD INDUSTRIAL EDGE CLOUD GRAND MASTER CLOCK (PTP) IT DATA STORAGE FIGURE 6.1.: BASIC SETUP OF REMOTE MACHINE ACCESS USE CASE INCLUDING SELECTED BUILDING BLOCKS FOR IMPLEMENTING SUCH A SCENARIO AGV 1 AGV Remote Machine Access In some situations, it may be helpful to remotely connect to a certain machine or component, for example, in case of malfunctions or for remote maintenance. As the supplier of such a machine or component does not necessarily know in advance where his equipment will ultimately be used and what communication infrastructure will be available, the easiest and presumably most generic way to implement remote access is via a cellular 4G network. However, this generally poses security challenges, because this kind of bypass to a public network infrastructure may vitiate any local security mechanisms in place and hence lead to a potential security threat. One possible way to address this challenge is to carefully monitor, control, and log the traffic that goes from and to a remotely connected machine or component, for example, via a dedicated security gateway. The principle setup of such a system, including selected building blocks outlined in the previous section for implementing this kind of use case, are depicted in Figure Automated Guided Vehicles Automated guided vehicles (AGVs) that take care of the flow of goods and material in a factory in an autonomous manner are considered as another relevant example. Due to their mobility, wireless connectivity is a natural choice for such devices. In the simplest case, this connection can be used to transmit new tasks or to retrieve status information. However, as more and more reliable and powerful wireless technologies become available, advanced functionalities may be implemented. One example could be to offload a lot of the intelligence that is traditionally contained in the AGV SENSOR ACTUATOR LOCATION TAGS ONBOARD GW 4G-MODEM URLLC-MODEM DEVICE-TO-DEVICE COMMUNICATION 4G-MODEM URLLC-MODEM FIGURE 6.2.: BASIC SETUP OF THE AUTOMATED GUIDED VEHICLES USE CASE INCLUDING SELECTED BUILDING BLOCKS FOR IMPLEMENTING SUCH A SCENARIO itself (e.g., video processing for recognizing the environment or analytics functionality) to an edge cloud. Likewise, AGVs could communicate directly with each other, e.g., via direct device-to-device communication, in order to jointly collaborate in a swarm-like manner so that more complex or difficult tasks can be managed than by a single AGV, such as joint lifting of heavy goods. Moreover, localization technologies LOCATION TAGS ONBOARD GW SENSOR ACTUATOR integrated into the wireless infrastructure could be used to assist in positioning an AGV on the factory floor as well as to the current destination. A likely architecture, including certain building blocks that are required to build such a system, is shown in Figure 6.2.

16 30 31 FACTORY SUPPORT SYSTEM 6.4. Mobile Cooperation and Control with Ultra-Reliable Machine Communication with new wireless technologies, such as 5G with its ultra-reliable and low-latency communication, a wireless connection ANOMALY DETECTION LOCAL DATA AGGREGATION BACKEND DATA AGGREGATION As a last example, we are considering a mobile control panel becomes possible, for example, in combination with appropriate safety protocols such as PROFIsafe. To this end, the APPLICATION / SERVICES ICT INFRASTRUCTURE that can be used to configure or monitor a machine. Such control panels typically also have safety-critical functions, e.g., an emergency stop button. Most panels currently have mobile control panel must be connected to a 5G network via a 5G modem and a suitable gateway to communicate with the machine control unit. Figure 6.4 depicts a possible setup IT EDGE CLOUD wired connections due to the demanding reliability and latency constraints of the safety-critical functions. However, of such a system using ABBs. 4G SYSTEM ENTERPRISE NETWORK IT DATACENTER CLOUD INDUSTRIAL EDGE CLOUD FACTORY SUPPORT SYSTEM SAFETY CONTROLLER APPLICATION / SERVICES ICT INFRASTRUCTURE WIRELESS SENSOR WIRELESS SENSOR 5G - SYSTEM INDUSTRIAL GW 4G - MODEM SENSOR 4G - MODEM SENSOR FIGURE 6.3.: BASIC SETUP OF THE MASSIVE WIRELESS SENSOR NETWORK USE CASE INCLUDING SELECTED BUILDING BLOCKS FOR IMPLEMENTING SUCH A SCENARIO USER EQUIPMENT 5G - MODEM CONTROL PANEL 6.3. Massive Wireless Sensor Networks A wide variety of different sensors may be deployed in a factory to implement functions, such as condition monitoring, predictive maintenance or to detect anomalies. In many cases, it makes sense to connect these sensors wirelessly as this facilitates easy retrofit solutions, so that existing machines can also be easily upgraded simply by integrating additional sensors. Moreover, this can reduce maintenance and installation work and improve usability. In fact, we envision that in future hundreds or thousands of sensors may be deployed in a factory, leading to a potentially significant ac- cumulated data rate. However, it is not necessary to transmit every sensor value to the cloud since much of the data may be redundant or correlated and since adequate actions may only have to be carried out locally. Therefore, one promising approach is to have some local pre-processing/pre-aggregation, for example, in an edge cloud, and to forward only the pre-processed data to an actual backend cloud. One major challenge in this respect is how distributed processing with potential instances in the end devices, the edge cloud and the backend cloud can be properly orchestrated and deployed. A likely architecture of this use case, including selected building blocks, is shown in Figure 6.3. FIGURE 6.4.: BASIC SETUP OF USE CASE MOBILE COOPERATION & CONTROL WITH ULTRA-RELIABLE MACHINE COMMUNICATION INCLUDING SELECTED BUILDING BLOCKS FOR IMPLEMENTING SUCH A SCENARIO

17 ABOUT IC4F The flagship project Industrial Communication for Factories (IC4F) aims to develop secure, robust, and real-time communication solutions for the manufacturing industry. Throughout the project, the IC4F partners develop building blocks for a trusted industrial communication and computing infrastructure based on an open cross-domain architecture that allows modular expansion for new applications and communication technologies. Key technologies include 5G, multi-access edge computing, cloud computing, virtualization, and industrial monitoring and analytics. The building blocks are designed to enable users to select the appropriate ICT technologies, according to the new Industry 4.0 requirements and the specific migration approach. The IC4F reference architecture will provide a validated approach for defining Industry 4.0 communication systems in a variety of factory ecosystems. Accordingly, IC4F involves relevant stakeholders along the value chain and brings together the expertise from different specialist disciplines. The project is supported by the German Federal Ministry of Economic Affairs and Energy (BMWi).

18 34 35 REFERENCES, ABBREVIATIONS References Abbreviations [1] Cockburn, Alistair, Writing effective Use Cases, Addison-Wesley, 2001 [2] Anwendungsbeispiele der Plattform Indutrie 4.0, plattform-i40.de/i40/navigation/karte/siteglobals/forms/ Formulare/karte-anwendungsbeispiele-formular.html [3] EFFRA: Factories 4.0 and Beyond, Recom-mendations for the work programme of the FoF PPP under Horizon 2020, Version: v30 Date: 12/09/2016 [4] NGMN Alliance (2014), 5G White Paper -Executive Version [5] Jeschke, S., Brecher, C., Song, H., & Rawat, D. B. (2017), Industrial Internet of Things, Cham: Springer International Publishing, (Last retrieved on March 15, 2018) [6] Open Group, TOGAF standard, subjectareas/enterprise/togaf/ [7] Deutsches Institut für Normung (2016), Referenzarchitekturmodell Industrie 4.0 (RAMI4.0) [8] Industrial Internet Consortium (2015),Industrial Internet Reference Architecture, G1_V1.80_ pdf [9] chap32.html [10] chap37.html [11] chap37.html#tag_37_03 2G/3G/4G/5G 2nd/3rd/4th/5th Generation Mobile Network 3GPP 3rd Generation Partnership Project 5GC 5G Core ABB Architecture Building Block ADM Architecture Development Method AGV Automated Guided Vehicle AI Artificial Intelligence API Application Programming Interface AR Augmented Reality CT Communication Technology CU Central Unit embb Enhanced Mobile Broadband ERP Enterprise Resource Planning GW Gateway IC4F Industrial Communication for Factories IIC Industrial Internet Consortium IIoT Industrial Internet of Things IIRA Industrial Internet Reference Architecture IoT Internet of Things IT Information Technology KPI Key Performance Indicator LTE Long Term Evolution LTE-A Long Term Evolution-Advanced MAPE Monitoring, Analysis, Planning, and Execution MES Management Execution System ML Machine Learning MQTT Message Queue Telemetry Transport MTC Machine-Type Communication NFV Network Function Virtualization NR New Radio NRF Network Repository Function OSI Open Systems Interconnection OT Operational Technology PKI Public Key Infrastructure PaaS Platform as a Service QoS Quality of Service RAMI 4.0 Reference Architecture Model Industry 4.0 RU Remote Unit SDN Software-defined Network TOGAF The Open Group Architectural Framework TSN URLLC VM VNF VR WLAN Time Sensitive Network Ultra Reliable Low Latency Communication Virtual Machine Virtual Network Function Virtual Reality Wireless Local Area Network