JUNOS SPACE PLATFORM BROADBAND NETWORK SERVICES ORCHESTRATION AND MANAGEMENT SOLUTION DESIGN GUIDE

Design Guide JUNOS SPACE PLATFORM BROADBAND NETWORK SERVICES ORCHESTRATION AND MANAGEMENT SOLUTION DESIGN GUIDE Copyright 2014, Juniper Networks, Inc. 1

Table of Contents Introduction... 4 Scope... 4 Use Case Summary... 4 Design Considerations... 5 Junos Space Network Management Platform and Junos Space SDK Design Guidance...7 Junos Space Platform Features and Benefits... 8 Junos Space Management Applications... 8 Junos Space SDK and APIs... 9 Implementation... 9 Use Case: Using Junos Space for Services Automation and Management in Broadband Networks... 9 Juniper Dynamic Provisioning Solution Components...13 Junos Space Deployment...14 Operations Support and Readiness...14 High-Level Commissioning Process...15 Discovery...15 Installing Software and Scripts...15 Deploying Op Scripts...16 Configuring the BNG...16 Internal Housekeeping...16 RM Inventory Discovery Process...18 RM Reconciliation Steps...18 Up-to-Date BNG Inventory Information...18 Installing Software Upgrades...18 Up-to-Date Inventory of Business Subscriber Services...19 Fulfillment...19 Assurance...20 Security... 24 Summary...26 References... 27 Appendixes...28 About Juniper Networks...30 2 Copyright 2014, Juniper Networks, Inc.

List of Figures Figure 1: Network orchestration and dynamic service provisioning with Junos Space... 5 Figure 2: Junos Space Platform integration options...7 Figure 3: Junos Space Platform management applications... 8 Figure 4: BNG network diagram...10 Figure 5: Junos Space system integration architecture...11 Figure 6: Junos Space deployment diagram...14 Figure 7: BNG commissioning process with Junos Space...15 Figure 8: Inventory synchronization...17 Figure 9: RM configuring Line ID on the BNG...19 Figure 10: RPM architecture...21 Figure 11: OAM protocols used for Test and Diagnostics... 22 Figure 12: Test and Diagnostics overall integration... 23 Figure 13: Trouble Ticket Management... 24 Figure 14: User authorizations... 24 Fig. 15: Device segregation using persmission labels... 25 Figure 16: Inventory Navigation...28 Figure 17: Script execution on inventory components...29 Figure 18: Business service inventory provided by ESSM Insight...30 List of Tables Table 1: Services Delivered by the BNG...10 Table 2: NGSSM Processes, Key Functions, and Mapped Components...12 Table 3: Components of the Juniper Dynamic Service Provisioning Solution...13 Table 4: Sample Configlets Used for BNG Commissioning...17 Table 5: Audit Log Contents... 25 Table 6: Communication Requirements....26 Copyright 2014, Juniper Networks, Inc. 3

Introduction The new networks are versatile and enable a large variety of services including cloud, Software as a Service (SaaS), Infrastructure as a Service (IaaS), VoIP, video on demand (VOD), mobile applications, application delivery infrastructure, and many others. The variety of services, their dynamic nature, and the rapid pace of innovation associated with both networks and new services are making networking solutions more complex and creating demand for accelerated service delivery. The following are some of the key challenges that service providers are highlighting with the new networks: Ability to effectively manage end-to-end services, including provisioning, modifications of demand, monitoring, diagnostics, and troubleshooting Integration with legacy infrastructure, particularly business and operations support systems (BSS/OSS) components Keeping up with innovation, as well as the pace of new services and the integration of new software solutions Given the complexities of the new networks and associated services and the reality of the competitive service marketplace, efficient delivery of new services to customers and the management of the complete service lifecycle are completely dependent on the ability to rapidly and reliably deploy new services and operational efficiency in general. This is directly related to the level of automation and orchestration in the network. Service providers are looking for solutions to efficiently manage new services that also provide seamless integration with their existing systems. Juniper has been working on a number of programmable solutions and platforms that will help our customers solve those key issues with the introduction of programmable interfaces into the network. Juniper Networks Junos Space Network Management Platform has been designed as a centralized, highly scalable, programmable, and extendable network management and orchestration tool that can help service providers dynamically manage services, automate/ orchestrate the associated workflows, and efficiently integrate with existing OSS solutions. Junos Space Platform is a new generation of programmable network management solution and also represents the first step in the softwaredefined network (SDN) implementation. Scope This document describes one of the generic scenarios for dynamic service provisioning where Junos Space is used as a programmable network automation and orchestration platform. It has been created primarily for network and solution architects and designers to guide them in developing next-generation network and service architectures with a high level of automation and orchestration. Use Case Summary This guide includes a use case showing ways that service providers can use Junos Space Platform for management and orchestration of their next-generation broadband networks. In this generic example, the service provider enables fixed network/broadband, mobile communications, Internet and IPTV products and services for consumers, and information and communication technology (ICT) solutions for business and corporate customers. Like many other companies in the industry, this service provider has experienced a major increase in IP traffic volume over the network. This has resulted in rapidly increasing network CapEx and OpEx costs accompanied by increased complexity, making the required solution very difficult to manage. In order to make the network more efficient and manageable, this provider has decided to drastically simplify its production network and OSS/BSS integration. The most critical cornerstone of the new broadband network is the deployment of new broadband network gateway (BNG) routers capable of performing aggregation, Broadband Remote Access Server (BRAS), label edge router (LER), and dynamic source routing (DSR) functionality in a single network element, and integrating it into the next-generation service and management architecture. The deployed solution in this use case is based on Juniper Networks MX960 3D Universal Edge Router as the new integrated network element, and Junos Space Network Management Platform as the management and orchestration solution. Junos Space Platform is also used to integrate with higher level OSS solutions to enable significant reduction in CapEx and OpEx costs by providing a single pane of glass for managing the complete lifecycle of the BNG routers. The operations team is now able to perform complete fault, configuration, accounting, performance, and security (FCAPS) management of the BNG network using the Junos Space GUI with its high level of abstraction. This eliminates the need for training on Juniper CLI and enables operations to perform all day-to-day network management tasks more efficiently and without errors. The REST Web services APIs of Junos Space Platform enable rapid and efficient integration with OSS and IT systems resulting in significant cost savings. Junos Space acts as the Element Abstraction Layer that hides the complexities of the network elements from the higher level management components and provides well-defined abstract interfaces via its REST Web services APIs. Moreover, Juniper Networks Junos Space SDK is used to develop a custom app to provide inventory management and troubleshooting capabilities for business subscriber services. This application enables operators to efficiently manage, monitor, and troubleshoot business services that are governed by strict service-level agreement (SLA) policies. 4 Copyright 2014, Juniper Networks, Inc.

This solution design guide provides architecture level details about the components involved in this deployment and also system integration with OSS solutions. Junos Space and applications deployment, along with some configuration scenarios are described to provide relevant information for making network architecture/design decisions. The scope is limited to the features of Junos Space Platform and its applications relevant to this use case. For more detailed product information, please refer to individual product literature at www.juniper.net/techpubs. Design Considerations When architecting new networks to optimize service provisioning and management, network designers are increasingly focusing on orchestration, automation, reliability, and scale in order to minimize OpEx while enabling a highly reliable solution. Additionally, this approach helps with improving the customer experience and minimizing disruptions in the system. The OpEx savings related to automation and orchestration are not limited to service provisioning operations; they are also achieved in monitoring, diagnostics, and troubleshooting. The automation strategy is driven by the set of services and associated workflows that need to be supported, and they form the functional requirements for the new network. Another important factor to consider is the integration with existing OSS systems and solutions. Operational tasks are usually performed utilizing multiple OSS systems. It is important that new architecture integrates with the existing OSS systems and application seamlessly, while enabling the required level of customization and automation. In addition to providing a more operationally efficient solution, the new approach with automated service provisioning and management enables new services like service or bandwidth on demand and can provide important differentiation in a very competitive service provider market. The high-level architecture depicted in Figure 1 captures the key components of the solution. Junos Space Platform provides a real-time view of the network and associated resources and also provides a centralized interface to manage all devices and services. Based on real-time information from the network and external requests, the higher level applications make real-time adjustments using APIs. The programmable interfaces are Web services-based REST APIs and are dynamically extendable using a plug-and-play application framework. Higher level OSS Applications and Services Network, Service and Subscriber Information Service Activation Director Security Director Network Application Platform Service Insight Custom App Network Application Platform Real-Time Service Provisioning Figure 1: Network orchestration and dynamic service provisioning with Junos Space Copyright 2014, Juniper Networks, Inc. 5

The primary goal of Junos Space Platform as a network orchestration platform is to enable more dynamic and automated network provisioning and monitoring to help use and manage network resources more efficiently and reliably. In most cases, this requires modifying provisioning and monitoring workflows that are currently CLI-based using predominantly manual procedures. Hence, it is very important to properly design the workflows for automated dynamic provisioning prior to developing and implementing end-to-end solutions. The new workflows should include pre-validation and post-validation to ensure that automated service provisioning is highly reliable. Pre-validation and post-validation procedures are already implemented within Junos Space Platform and in applications at different levels, and these need to be integrated with the end-to-end workflows. Given that Junos Space APIs are Web servicess-based, they can be accessed either by the application deployed as a Junos Space native application or by an external application. This is one aspect of the solution architecture that needs to be carefully considered. For most of the greenfield deployments and new application development, it is better to implement the custom application within the Junos Space environment to extend the functionality and provide required customization. In this case, Junos Space SDK can be used to rapidly develop the application as it provides the ability to generate the application framework code and Rest APIs. Space SDK also includes useful development tools like Rest wizard, device simulators, GUI builder, and others. Native Junos Space Platform applications can automatically utilize built-in high availability, database, and messaging services. However, in some instances it is more practical to integrate the Junos Space Platform with external OSS or applications, e.g., for integration with legacy OSS solutions and applications, or for integration with existing specialized applications like customer portals. This is the case with the customer portal example described later in this document. Both options are presented in Figure 1, where a custom application is deployed within the Junos Space environment and at the same time, integration with higher level OSS and applications is implemented using Junos Space APIs. The choice of whether to develop a native or external application is driven by the architectural choices, available solution components, and long-term goals. Junos Space Network Management Platform provides flexibility to support both options, and the same set of APIs is used regardless of where the application is deployed. The other important aspect to consider when architecting custom deployment is related to required functionality. Junos Space Platform provides comprehensive element and network management for Juniper devices covering the complete FCAPS functionality. This include same day support for new devices and Juniper Networks Junos operating system releases, a task-specific user interface, and northbound APIs to easily integrate into existing network management systems (NMS) or OSS/BSS solutions and applications. This basic FCAPS element management and network management system (EMS/NMS) functionality is extended using plug-and-play applications which provide service-level abstractions. These three components Junos Space Platform, plug-and-play applications, and Junos Space APIs provide a flexible and extendable network orchestration platform designed for easy integration. Any subset of available applications can be combined to provide a customized solution best suited for a specific deployment scenario. Custom applications are running in the same environment and can use APIs available from the Junos Space Platform and any applications that are installed. Higher level OSS applications and native Space applications can access the APIs published by the custom applications. That is the primary mechanism to customize and extend APIs based on the deployment. Therefore, based on the requirements of a specific deployment, these three components need to be considered and combined to provide the complete solution. The general rule of thumb is to use the functionality that is currently available by combining the Space Platform and existing applications and then develop new functionality only for the features that need to be customized for that deployment. Junos Space Platform and APIs are described in more detail in the following section. 6 Copyright 2014, Juniper Networks, Inc.

Junos Space Network Management Platform and Junos Space SDK Design Guidance Junos Space has been designed as a centralized, highly scalable, reliable, and extensible network management and orchestration platform enabling single-pane-of-glass visibility into the network and a common management platform for managing and creating customized end-to-end network services. Given today s focus on automation, it has a complete infrastructure for automation of the common workflows, including inventory, configuration, fault, and performance management. Services enabled by Junos Space Platform and applications are exposed and accessible via a northbound REST-based API. The rich collection of open APIs provides core building blocks for customization and innovation, eliminating the need to build solutions from scratch. Junos Space Platform includes the following three building blocks: Junos Space Network Management Platform Provides comprehensive FCAPS and element management of Juniper devices to improve operator efficiencies with a programmable interface and exposable APIs that enable the development and integration of third-party applications Junos Space Management Applications Plug-and-play, domain-specific applications to help you provision new services and optimize workflow tasks across thousands of Juniper devices Junos Space SDK (software development kit) A programmable network solution that enables you to leverage the connections and intelligence imbedded in the network to create customized management solutions for your specific needs OSS Layer REST API MTOSI Custom SNMP XML/JSON HTTP(S) XML SOAP HTTP(S) ANY SNMP Trap Forward MTOSI Adapter Custom Adapter RESTful Web Services Network Application Platform Junos Space Figure 2: Junos Space Platform integration options Network operators can use these three components to create customized solutions specific to their needs. Different levels of customization are supported, ranging from the combination of available applications, the development of scripts using Junos Space APIs, the development of customized applications, and integration with other OSS applications. The integration with other OSS applications is facilitated via a set of supported northbound interfaces and adapters as depicted in Figure 2. The most efficient and recommended option for integration is to use native REST APIs. A Multi-Technology Operations System Interface (MTOSI) adapter has also been developed to expose MTOSI 2.1-compliant interfaces for integration supporting inventory and configuration procedures. The MTOSI adapter also represents an example of generic adapters that can be developed using Junos Space APIs. Similar adapters can be developed to support other protocols, and a standard SNMP-based trap forwarding interface is supported as well. Copyright 2014, Juniper Networks, Inc. 7

Junos Space Platform Features and Benefits Junos Space Network Management Platform provides the standard FCAPS functionality that is regularly provided by Element Management Systems, but it has been enhanced beyond the standard EMS features to enable orchestration capabilities as follows: Network-wide visibility and control, with a real-time view into the network and scaling capabilities Horizontal scaling of operations and services Rapid deployment of services with a high level of abstraction and built-in automation Complete management of Juniper devices Cross-vendor event and performance management Centralized network data source enabling management of end-to-end services and analytics Real-time problem identification and resolution SDK and APIs for customization, integration, and service differentiation Easy orchestration and automation to streamline operational procedures and reduce OpEx App App App App App Services Activation Director Platform DMI Network Director Service Insight Security Director Third Party Figure 3: Junos Space Platform management applications Junos Space Management Applications Junos Space management applications enable customization of the Junos Space Platform for various domains. The applications provide a high level of abstraction for service management, and an easy-to-use interface. Moreover, the applications enable the provisioning of new services across thousands of devices, and workflow optimization and customization for specific use cases within the core, edge, data center, campus, security, mobile network, and more. Junos Space applications developed internally within Juniper are described in more detail in the following section. Customerspecific applications developed for this deployment will be described later as a component of the use case description. Currently available Junos Space Platform applications are shown in Figure 3. The application environment and applications themselves have been designed to enable in-service application installation (plug-and-play) to simplify customization and functional upgrades to the system. Each deployment will require a specific subset of applications effectively creating a customized solution combined with Junos Space Platform. The subset of applications used for this deployment includes Junos Space Service Now and Service Insight applications, which are described in subsequent sections of this guide. 8 Copyright 2014, Juniper Networks, Inc.

Junos Space SDK and APIs The Junos Space SDK provides a complete rapid application development framework that includes a common infrastructure, a software development kit (SDK) with prebuilt core services and widgets to allow easy user interface prototyping, and standards-based APIs for third-party application integration. Using the Space SDK, users have the option of developing different classes of applications such as mashups, customized business process workflows, or native applications. Junos Space SDK enables developers to leverage the Junos Space Platform to abstract the capability and connections to Juniper routers, switches, and firewalls. It encompasses a rich set of tools including REST APIs, the Eclipse integrated developer environment (IDE), device simulators, reference applications, support tools, and documentation. The Junos Space Network Management Platform and it s open APIs provide REST access to all Junos OS-based devices, serving as a single entry point that abstracts your network to enable you to manage, monitor, control, and gather insight across your entire network infrastructure. Junos Space SDK includes the following components: 1. Development tools Junos Space Eclipse plug-in that allows wizard-based creation of different types of Junos Space applications, code generation, REST Explorer, automated build, deployment of applications for test and debug purposes, control of device simulations on device simulator, and other tools. 2. REST Web servicess interfaces Interfaces to the core capabilities of the Junos Space Platform, which are a part of the Junos Space network Management platform. 3. Device and environment simulators The development environment includes Junos Space Virtual Appliance that provides access to: -- A fully functional instance of the Junos Space network application platform for use in deploying and testing applications developed using the Junos Space SDK. -- Device and element simulators providing the ability to test applications against virtual Juniper devices. 4. Performance, analytics, security, and profiling tools While the Junos Space SDK does not ship performance, analytics, security, or profiling tools, it is compatible with the most popular tools available today, such as VisualVM, JBoss Tools, etc. Implementation Use Case: Using Junos Space for Services Automation and Management in Broadband Networks This use case is based on the actual implementation of a large-scale broadband network, and it describes ways that service providers can use Junos Space for management and orchestration of their next-generation broadband networks. In this generic example, the service provider enables fixed-network/broadband, mobile communications, Internet and IPTV products and services for consumers, and information and communication technology (ICT) solutions for business and corporate customers. Like many others in the industry, this provider had experienced a major increase in IP traffic volume on its network, resulting in rapidly increasing network CapEx and OpEx costs accompanied by complexity that was getting very difficult to manage. In order to make the network more efficient and manageable, this provider decided to drastically simplify its production network and OSS/BSS integration. The most critical cornerstone of the new broadband network is the deployment of new BNG routers capable of performing aggregation, BRAS, LER, and DSR functionality in a single network element, then integrating it into the next-generation service and management architecture. The recommended solution is based on the MX960 3D Universal Edge Router as the new integrated network element, and Junos Space Network Management Platform as the management and orchestration solution. Junos Space is also used to integrate with higher level OSS systems. Junos Space enables significant reduction in capital and operating costs by providing a single pane of glass for managing the complete lifecycle of the BNG routers. The operations team is now able to perform complete FCAPS management of the BNG network using Junos Space GUI, which provides a high level of abstraction. The REST Web services APIs of Junos Space enable rapid and efficient integration with OSS and IT systems resulting in significant cost savings. Junos Space acts as the Element Abstraction Layer that hides the complexities of the network elements from the higher level management components and provides well-defined abstract interfaces via its REST Web services APIs. Moreover, Junos Space SDK is used to develop a custom app to provide inventory management and troubleshooting capabilities for business subscriber services. This application enables operators to efficiently manage, monitor, and troubleshoot business services that are governed by strict SLA policies. Copyright 2014, Juniper Networks, Inc. 9

RADIUS DHCP Business Subscribers CPE Access Node MX960 Core Residential Subscribers CPE Figure 4: BNG network diagram Figure 4 shows a high-level network diagram for the broadband service deployment. MX960 routers are deployed as BNGs in all locations. More than one MX960 may be deployed in some locations to provide a flexible and scalable solution. In this example, each BNG serves around 15,000 residential subscribers and 1,000 business subscribers. These subscribers typically connect to the BNG via an access node (AN) such as a multiservice access node (MSAN). In the case of subscribers requiring high-bandwidth services, they may be directly connected to a port on the BNG via optical fiber (such subscribers are referred to as directly attached subscribers). Dynamic Host Configuration Protocol (DHCP) service is used for subscriber IP address management, and RADIUS is used for dynamic provisioning of subscriber services based on policies provisioned in the RADIUS servers. Table 1: Services Delivered by the BNG Residential Services Various flavors of single, dual, and triple play services with different bandwidth settings. Business Services Layer 3 high-speed Internet access services with QoS guarantees. E-LINE Services providing point-to-point Layer 2 connectivity between business locations with QoS guarantees. E-LAN Services providing multi-point L2 connectivity between business locations with QoS guarantees. Services delivered via the broadband network can be broadly classified into two categories: Residential Subscriber Services and Business Subscriber Services. Residential services include many flavors of multiplay services with differential bandwidth. Business services are pure data services offering high-speed Internet access for business locations as well as interconnecting different locations of a business via E-LINE or E-LAN (see Table 1). The key innovation and advantage of the Juniper solution is the ability to provision subscriber services dynamically without requiring the operator to manually provision each service. Without this dynamic subscriber service provisioning capability, the network administrator would need to manually provision each subscriber, each VLAN sub interface, each set of class-of-service bandwidth controls, and more. This complex manual process requires a significant amount of time and resources for preparation and configuration, not to mention time spent on the effort to debug and troubleshoot resulting errors. Hence, Juniper s approach for dynamic management and provisioning of services enables a service provider to: Manage a very complex deployment with a high level of automation and orchestration Use dynamic provisioning with almost instant service activation Deploy a complete solution without having to manually provision each subscriber Manage and monitor end-to-end services such as reporting, troubleshooting, diagnostics, etc. 10 Copyright 2014, Juniper Networks, Inc.

Details of this solution are described in the following sections. They include details of various interactions between the BNG, EMS, and OSS components, the complete service management lifecycle, dynamic provisioning, and how the Junos Space Platform enables this solution by providing element management and abstraction capabilities. Figure 5 depicts the high-level system integration architecture. Junos Space Platform is used as the element abstraction layer interfacing with various next-generation service and system management (NGSSM) components. Operators interact primarily with the GUI provided by the OSS components and Junos Space Platform to perform the required operations. NGSSM components are organized based on Level 1 processes defined by TMF etom [4]. Next Generation Service and System Management Fulfillment Operations Support and Readiness Assurance Billing RADIUS RM T & D TT RTM RPM Accounting RADIUS REST ESSM Insight REST Service Now Service Insight Junos Space Platform OSS/J JSR91 SNMP TRAPS SFTP Flat Files SFTP IPDR XML Files Element Management DMI SNMP ESSMD Subscriber Management MX960 BNG Network Element Figure 5: Junos Space system integration architecture Copyright 2014, Juniper Networks, Inc. 11

Table 2 provides a more detailed description of these processes, the key functions included within them, and the mapping to solution components. Table 2: NGSSM Processes, Key Functions, and Mapped Components Process Key Functions Components Operations support and readiness Fulfillment Discovery Discovers new resources, services, configurations, and topologies on the network Creates, configures, resumes, suspends, cancels, and removes discovery agents Periodically executes polls to locate modified resources and configurations offered by them Configuration Management Uses configuration management to perform device configuration and bring resources into operation Performs initial service-specific device configurations triggered by system integration and planning (SI&P) and keeps the configuration inventory up-todate Manages all changes to configurations, including software upgrades Handles resource and configuration changes detected by Discovery to keep the service and resource inventory up-to-date Provides a complete audit trail (i.e., when, by whom, and why configurations have been changed) Provisioning Creates a production plan for a given service that covers the activation sequence and timing consideration that has to be ensured Checks the availability of needed service and resource instances against the inventory Allocates and reserves resources for a given instance of a service Resource Manager (RM) Junos Space Junos Space RADIUS Resource Manager (RM) Junos Space Junos Space Assurance Billing Activation Activates services and resources Updates service and resource inventory with needed status changes Resource Trouble Management Receives, correlates, and classifies resource trouble events Implements active monitoring (polls key devices and components to determine their status and availability), and passive monitoring (to detect operational alerts or communications generated by devices and components) Testing and Diagnostics Performs various test and diagnostics actions on services and resources to identify root cause of problems Trouble Ticketing Hands major troubles over to Trouble Ticketing Management by initiating an incident ticket Resource Performance Management Involves collection and processing of performance data from the network Includes monitoring and management of thresholds and Key Performance Indicators (KPIs) Makes notifications to service quality management in case of potential resource degradations Accounting Collects usage data for all services to prepare accurate bills. RADIUS BNG Resource Trouble Management (RTM) Junos Space T&D SYSTEM Junos Space Trouble Ticketing (TT) Integrity Junos Space Resource Performance Management (RPM) Junos Space BNG Accounting Server 12 Copyright 2014, Juniper Networks, Inc.

Juniper Dynamic Provisioning Solution Components The Juniper Dynamic Provisioning solution has been designed using key features and components at different layers of the overall network architecture as listed below. These components are described in more detail in Table 3. Device-level components (MX960), Junos OS subscriber management features, and Extensible Subscriber Services Management (ESSM) Junos OS automation scripts managed by Junos Space and executed on devices Junos Space-based element management, automation, and orchestration solution Junos Space integration capabilities Table 3: Components of the Juniper Dynamic Service Provisioning Solution MX960 Billing MX960 running Junos OS 13.1 is used as the BNG router, and the following components of Junos OS play a significant role in the overall solution: Broadband Subscriber Management feature [1] of Junos OS. This capability in Junos OS takes care of dynamically provisioning and managing residential subscriber access. It uses authentication, authorization, and accounting (AAA) configuration in the RADIUS server in conjunction with dynamic profiles to provide dynamic, per-subscriber authentication, addressing, access, and configuration for all residential subscriber services. Extensible Subscriber Services Management (ESSM) Framework [2]. This component of Junos OS takes care of dynamically provisioning and managing business subscriber services. It intercepts authentication message exchanges between the customer premises equipment (CPE) and the RADIUS server and dynamically provisions services based on vendor-specific attributes (VSAs) returned from the RADIUS server. The set of VSAs that are relevant and the provisioning actions corresponding to each VSA are configured via a dictionary in XML format. Provisioning actions themselves are packaged as Op Scripts. This design of the ESSM Framework with its reliance on a configurable dictionary and a set of op scripts makes it a fully extensible framework that can be used by service providers to dynamically provision any kind of services for subscribers. Junos OS Automation Scripts [3]. Junos OS automation consists of a suite of tools used to automate operational and configuration tasks on network devices running Junos OS. These scripts are used by the ESSM Framework for provisioning business services and for a variety of operational and management actions on the BNG. Junos Space 13.1 is used as the EMS and provides the Element Abstraction Layer that facilitates integration between NGSSM components and the BNG network. Junos Space Network Management Platform Junos Space Network Management Platform provides complete FCAPS functionality at the element management layer that can be accessed using a simple Web 2.0 GUI as well as via REST Web servicess API. The GUI is used by operators to perform full lifecycle management of BNGs. REST APIs are used to integrate the BNG network with NGSSM OSS components for process automation. Junos Space Service Now Junos Space Service Now is an automated incident management application. It automatically detects problems on devices and collects troubleshooting data from the device at the same time. It can raise support cases with Juniper Networks Technical Assistance Center (JTAC) and speeds time-to-resolution by eliminating manual processes. It also implements an OSS/J JSR91 Trouble Ticketing API to allow OSS components to create support cases with Juniper s technical support team. Junos Space Service Insight Junos Space Service Insight helps reduce network downtime by delivering proactive bug notifications specific to the target network, and thorough automated end-of-life/support analysis. Junos Space Service Insight delivers targeted bug notifications, identifies which network devices could potentially be impacted by them, and performs impact analyses for End-of-Life/End-of-Support (EOL/EOS) notifications. Junos Space ESSM Insight Junos Space ESSM Insight is a custom application developed using the Space SDK. It provides a simple GUI for performing inventory management and troubleshooting actions on business subscriber services. It interacts with the ESSM daemon running on the BNG to collect data about business services. The app also provides a REST Web servicess API layer which is integrated with the T&D component to automate the process of running test and diagnostics actions on business services. Copyright 2014, Juniper Networks, Inc. 13

Junos Space Deployment For simplicity and based on the scaling requirements in this case, the chosen deployment is of a cluster of two Juniper Networks JA1500 Junos Space Appliances in the same data center. The two appliances are installed in two separate buildings and connected to the in-band Dynamic Circuit Network (DCN) via two separate switches as shown in Figure 6. This protects the cluster from complete outages that may occur in any one of the buildings. The gigabit Ethernet interface ETH0 on each appliance is connected to the switch. A floating virtual IP address is configured for the cluster, and this is used to access services on the Junos Space cluster by all GUI operators as well as OSS components. All MX960 routers, Space operator workstations, and OSS servers are connected to the same DCN. The Junos Space cluster can be easily extended to include more appliances if required in the future. Two appliances configured within the cluster provide full high availability (HA) capabilities. Space Operators OSS Floating VIP Building 1 ETHO TACACS+ JA1500 JA1500 DCN Backup Building 2 ETHO Figure 6: Junos Space deployment diagram In this example, a pair of TACACS+ servers is used for centralized authentication and authorization related to operator access to all network element, EMS, and OSS systems. The Junos Space cluster as a component of the overall NMS solution is also configured to perform remote authentication and authorization against this pair of TACACS+ servers that are accessible via the DCN. More details related to remote authentication and authorization are provided in the section dealing with Security. Finally, Junos Space is also configured to perform daily backups via SCP to a remote backup server. This is accomplished by scheduling a recurrent database backup job to happen shortly after midnight every 24 hours, ensuring that the backup job gets executed during an interval of least usage of the system and the DCN. The remote backup allows the customer to rebuild the Junos Space cluster and bring it up-to-date to the point in time when the most recent backup was taken. Operations Support and Readiness The Operations Support and Readiness process grouping encompasses all NGSSM functions related to deploying and maintaining the BNG network in support of activities in the Fulfillment, Assurance, and Billing process groups as defined in TMF etom [4]. This includes functions such as deployment of a new BNG, discovering and managing the configuration and inventory on the BNG, maintenance operations to be carried out on the BNG, ensuring that the BNG has up-to-date software and configuration required for activation of subscriber services, etc. In this section, we will take a look at how the Junos Space Platform assists network operators in carrying out these functions. 14 Copyright 2014, Juniper Networks, Inc.

RM Junos Space BNG Periodic duscovery and synchronization of physical and logical inventory of BNG routers Discover the device into Junos Space Install the VSA dictionary required by ESSMD, and provisioning scripts on the router Deploy Jusos OS automation scripts used for monitoring and managing chassis components Deploy necessary configuration to make a functioning BNG Perform internal house-keeping actions to manage the BNG Installs basic config on the router and brings it online Figure 7: BNG commissioning process with Junos Space High-Level Commissioning Process A high level overview of the process of commissioning a new BNG is depicted in Figure 7. When a new MX960 router is to be deployed as a BNG, the technician installs a basic configuration on the router that will assign a unique management IP address to it and enable it to join the in-band management DCN. This management IP address is configured with the master-only keyword [5] to ensure that the IP address is owned by the master routing engine of the router at all times. SSHv2 is also enabled on the router and a login account is created with superuser permissions. This is the login account that will be used by Junos Space for discovering and managing the router. Discovery Once the router boots up and is IP-reachable via the DCN, it is ready to be discovered into the Junos Space Platform. The technician hands over the management IP address and the SSH login credentials of the router to an operator who has permission to discover new devices into Junos Space. This operator now uses this information to discover the new router as a managed device in Space. During this process, Space will establish a dedicated SSHv2 connection with the router and import complete inventory and configuration information from it. This information is persistent in the Space database and is kept up-to-date with changes happening on the router by listening to system logging events sent by the router indicating such changes. This ensures that Junos Space is always in sync with inventory and the configuration of the network and can act as a reliable source of this information to other OSS components. It is also noteworthy that Space supports RSA key-based SSH authentication to managed devices in addition to the usual password-based authentication. This customer chose to use RSA key-based authentication for enhanced security. This means the public key of Space needs to be configured on the router, and this is done prior to discovery via a simple action initiated from the Space GUI. Installing Software and Scripts The next main step in the commissioning process is to set up the software infrastructure on the router to allow it to perform its tasks as a BNG. Support for dynamic provisioning of residential services is built into Junos OS. However, dynamic provisioning of business services is achieved using the ESSM Framework [2]. This framework intercepts authentication message exchanges between the CPE and the RADIUS server and dynamically provisions services based on VSAs returned from the RADIUS server. The set of VSAs that are relevant and the provisioning actions corresponding to each VSA are configured via a dictionary in XML format. Provisioning actions themselves are packaged Copyright 2014, Juniper Networks, Inc. 15

as Op Scripts [3]. This design of the ESSM Framework with its reliance on a configurable dictionary and a set of Op Scripts makes it a fully extensible framework that can be used for dynamically provisioning any kind of services for subscribers. This means that the following must be performed on each MX960 as part of commissioning it as a BNG: 1. Copy the VSA dictionary XML file to the BNG. This is performed using the ESSM Insight application on Junos Space. 2. Insert configuration into BNG to make Extensible Subscriber Services Management Daemon (ESSMD) refer to this dictionary. This is performed using an Op Script that can be executed from Junos Space GUI using its Script Management feature. 3. Deploy the bundle of Op Scripts to be used by ESSM Framework for business service provisioning. This is performed using the Script Management feature of the Junos Space Platform. In order to automate this step, a workflow (known as Operations in Space parlance) has been designed which performs the steps in sequence. Each step will be executed if the previous step completes successfully. The operator just has to select the router and execute the operation to automatically perform the three steps identified above. Deploying Op Scripts The next step in the commissioning process is to deploy Junos OS Op Scripts that will be used for monitoring and managing chassis components, physical ports, and logical interfaces on the BNG. Junos OS allows extensive automation via Op Scripts. Several Op Scripts have been developed to perform monitoring and administrative actions on various components of the BNG and the BNG as a whole. These scripts have been packaged into a bundle that can be deployed onto the BNG from Junos Space using its Script Management feature. Once deployed, these scripts can be executed by the operator from Junos Space GUI by selecting the appropriate component (equipment, port, logical interface, etc.) and choosing the script from the right-click action menu. The script gets executed on the BNG and results displayed immediately in the GUI screen. We will see some examples of this later on in this guide. Configuring the BNG The most important and complex step in the whole process is the deployment of necessary configuration on the router to make it function as a BNG in the network. This includes configuring core-facing and subscriber-facing interfaces, configuration required for making the BNG participate in various routing protocols used in the network, configuration required for residential subscriber management, quality-of-service (QoS) profiles for subscriber services, etc. This configuration on a BNG can run into several thousands of lines and is difficult to deploy and troubleshoot if done manually using the CLI. However, the Junos Space CLI Configlets feature allows complex configuration snippets to be encapsulated into simple, parameterized configlets. Each configlet internally contains the configuration required for a set of related Junos OS features and exposes necessary parameters 1 whose values need to be set by the operator when a configlet is deployed. These parameters can be simple text fields where the operator can type in values, or dropdown combo boxes in which the operator can choose a value from a list of options. In the latter case, the list of choices can be a static set programmed into the configlet or it can be dynamically computed from the existing inventory and configuration of the router (e.g., set of ports which are up and on which MPLS is enabled). In addition, the configlet can also contain Velocity Template Language (VTL) directives [6] for condition evaluations, iterations, etc. All these capabilities combined together make CLI configlets a powerful but simple tool for performing configuration deployment and changes on the router. Configlets are typically created by expert users who are proficient in Junos OS configuration. However, they provide a simple GUI that makes it easy for nonexpert operators who are not trained in Junos OS configuration to apply configuration changes on Juniper routers. The GUI also provides an option to first validate the configuration change on the router before applying it. This allows the operator to catch errors before they can cause any disruption in the network. In the case of this customer, Juniper Professional Services was engaged to create a set of configlets that could be used for commissioning a BNG. The operator who uses these configlets to bring up new BNGs in the network does not know any details about the contained configuration, but is able to validate and apply the configuration on the routers via the simple GUI. A list of sample configlets used in this deployment is listed in Table 4. Internal Housekeeping The final step is to perform some internal housekeeping actions within Junos Space for day-to-day management of the router in a robust and secure manner. These actions include: Attach appropriate permission labels to the new router. Permission labels allow you to segregate your network into different sets that are allowed to be accessed by different sets of operators. In this use case, the customer chose to do this segregation based on regions, and all BNG routers within a region are assigned the same permission label. This permission label is also assigned to a set of operators that are allowed to access BNGs within this region. 16 Copyright 2014, Juniper Networks, Inc.

Table 4: Sample Configlets Used for BNG Commissioning Configlet System settings Core-facing interface Subscriber-facing interface Protocol settings Dynamic profile settings RPM configuration RTM configuration Description Configuration for system-wide settings such as Network Time Protocol (NTP) servers, AAA servers, common user accounts, etc. Configuration required for provisioning a core-facing interface on the BNG. Configuration required for provisioning a subscriber-facing interface on the BNG. Configuration for various networking protocols used in the network. Configuration for dynamic profiles used for residential subscriber management. Configuration for enabling the functioning of RPM processes in NGSSM. Specifically, this includes configuration for the device to periodically generate comma-separated value (CSV) files with performance data and transfer them to the RPM server. Configuration for enabling the functioning of RTM processes in NGSSM. Specifically, this includes configuration for SNMPv3 access and sending traps to RTM systems. RM Get all ManagedElement objects GET /api/space/managed-domain/managed-elements Junos Space Get EquipmentHolder objects for each ManagedElement GET /api/space/managed-domain/managed-elements/{id}/equipment-holders Get Equipment hierarchy under each top-level holder GET /api/space/managed-domain/managed-elements/{id}/equipment-holders{id} Get PTP objects under all ManagedElement objects GET /api/space/managed-domain/ptps Get each PTP object and all CTP objects under it GET /api/space/managed-domain/ptps/{id} Figure 8: Inventory synchronization Attach required tag to the new router. Junos Space allows you to assign tags onto managed devices to identify and classify them. Tags provide a flexible way of annotating devices for locating them easily and for performing operations on matching devices. You can assign multiple tags to a device, and you can also assign a tag to multiple devices. An important use of tags is to create a dynamic set of devices that can be used as the target of certain operations. For example, in this customer deployment configuration backups are performed daily on all BNG routers using the Config File Management feature of Junos Space. This is done using a recurrent job created by the administrator using Space GUI. The target of this operation is specified as the tag Perform Backup. This means that each time the job is executed, it will seek out all devices that have this tag and perform a configuration backup on all those devices. When a new BNG is added, it needs to be assigned the tag Perform Backup to automatically ensure that Space will start performing daily configuration backups of the new router starting from the next scheduled run of the job. This means that Space database will have daily snapshots of the configuration of all BNGs. The GUI allows operators to view the contents of each version, compare between versions, as well as restore a selected version back onto the device providing a safeguard against disasters that can potentially cause complete configuration wipeouts on the BNG router. Copyright 2014, Juniper Networks, Inc. 17

RM Inventory Discovery Process As shown in Figure 5, Resource Manager (RM) is deployed in the OSS layer and is responsible for resource management in the NGSSM architecture of this customer. This system needs to have an accurate view of the complete physical and logical inventory of each BNG in the network. Junos Space maintains up-to-date inventory and configuration information on all BNGs in its database, as was discussed earlier. This information is modeled based on the object model defined in MTOSI 2.0 [7] and is exposed via a set of simple REST Web services APIs. An adapter was developed for RM to invoke these APIs over HTTP transport using SSLv3 encryption. The sequence diagram in Figure 8 depicts the main interactions between this adapter and Junos Space when RM performs a discovery and synchronization of BNG inventory information. The figure also shows the REST API URL for each step. As the first step, RM retrieves all ManagedElement objects from Space. Each ManagedElement represents a BNG in the network. Then it retrieves the top-level EquipmentHolder object representing the chassis of each BNG, followed by the complete equipment hierarchy (slots, sub-slots, SFPs) under each chassis. The next step is to retrieve all Physical Termination Point (PTP) objects across all BNGs in the network. This is followed by the retrieval of the details of each PTP object including all Connection Termination Point (CTP) objects under it. RM Reconciliation Steps After completion of the inventory discovery process, RM internally reconciles this information with what is already stored in its database. This process is scheduled to repeat on a daily basis. Please note that Junos Space API is capable of notifying clients when there are inventory or configuration changes on each managed device. However in this deployment, it was decided not to utilize this feature for two main reasons: (a) to simplify development of the RM adapter component, and (b) real-time update of the resource inventory was not a critical requirement. Hence, it was decided that RM would perform inventory discovery and reconciliation on a daily basis. Up-to-Date BNG Inventory Information Junos Space GUI provides ready access to up-to-date inventory information on each BNG. Hardware inventory is depicted in the GUI using a hierarchical tree view that allows you to explore containment relations between various hardware components. Equipment-to-port relationship is also modeled, allowing you to navigate from a selected equipment object to a list of physical ports contained by that equipment. Moreover, you can navigate from a selected physical port to all logical interfaces provisioned on that port. This navigation is depicted in the screenshots in Figure 16 in the appendix. The operator selects the equipment Xcvr 0 under PIC 1 under FPC 2 and chooses the right-click option, View Physical Interfaces. This brings up the view in the second screenshot that shows the port ge-2/1/0 contained by the selected equipment. The operator then clicks on the View link under the Logical Interfaces column and brings up the view shown in the third screenshot. It is also possible to directly navigate from a device to the list of all ports contained by it as well as to the list of all logical interfaces provisioned on the device. Common day-to-day management actions that need to be performed on BNG inventory components have been encapsulated into a set of automation scripts that can be executed easily from Junos Space GUI. Figure 17 in the appendix shows screenshots from the workflow of selecting a device (FPC 2) and executing a script to view its current status. The script runs on the device and its results are rendered in the GUI in the final step of the workflow. Installing Software Upgrades One of the complex and error-prone tasks in managing a large network is to install software upgrades on networking devices. Juniper Networks releases a new version of Junos OS every four months and these releases contain important new features and bug fixes. Junos Space Platform acts as a central repository for all device OS images and provides flexible workflows for downloading and installing these images on managed devices. In this customer deployment, tasks that need to be performed on each router prior to an upgrade and after the upgrade is complete are modeled as Op Scripts. Steps required for deployment and execution of these scripts and the installation of a Junos OS upgrade are modeled into an automated operation. This operation is scheduled for execution on a selected set of routers at a specific time chosen by the operator. When the operation executes, each step in the operation is executed in the designed sequence. These include execution of a pre-upgrade script, copying of the Junos OS upgrade package to the routers, installation of the actual upgrade, and execution of any post upgrade scripts. 18 Copyright 2014, Juniper Networks, Inc.

Up-to-Date Inventory of Business Subscriber Services The ESSM Insight application manages up-to-date inventory of all business subscriber services on all BNGs in the network, and it provides a simple GUI that allows operators to access and visualize this inventory on demand. The GUI allows the operator to view all business subscriber sessions served by a BNG. For each session, it displays the Line ID and Point-to-Point Protocol (PPP) username of the subscriber, location of the physical port on the BNG serving the session, and the name of the demux interface. For each session, it maintains inventory of all services riding on it. Please see the screenshots in Figure 18 in the appendix. The application also provides REST Web servicess APIs that can be used to enable OSS components to collect this inventory information. Fulfillment The Fulfillment process grouping is responsible for providing customers with their requested products in a timely and accurate manner. It translates the customer s business or personal need into a solution that can be delivered using specific product offerings from the service provider. Key functions in this group include: (a) provisioning and allocation of resources to planned service instances, and (b) actual activation of these services. In this deployment, resource provisioning is performed using RM, which allocates and configures resources on the access node or BNG. Service policies, subscriber identity, as well as the mapping from subscriber to corresponding services are all provisioned into databases accessed by the RADIUS server. Activation and deactivation of services happen dynamically based on message exchanges between the RADIUS server and the BNG. Each subscriber is allocated a unique Line ID. RM manages the complete inventory of the network and is responsible for assigning and tracking this Line ID on the network port to which the subscriber connects. In the typical case of a subscriber connecting to an access node port, the Line ID needs to be configured on the access node port. This is performed by RM by invoking the NorthBound -based Interface (NBI) provided by the vendor-provided EMS managing the access node. In the case of a directly attached subscriber connecting directly to a BNG port, the Line ID needs to be configured on the BNG port. This is performed by RM by invoking the REST Web services APIs provided by Junos Space as depicted in Figure 9. RM Junos Space BNG POST /api/space/configuration-management/cli-configlets/{id}/apply-configlet Request: <cli-configlet-management> <deviceid>{deviceid}</deviceid> <cli-configlet-param> <parameter>port</parameter> <param-value>ge-2/1/1</param-value> </cli-configlet-param> <cli-configlet-param> <parameter>line ID,/parameter> <param-value>abc123xyz</param-value </cli-configlet-param> <cli-configlet-management> Response: <task> <id>{jobid}</id> </task> Create a backend job to push config Push config to the BNG Get /api/space/job-management/jobs/{jobid} Get the completion status and results for the backend job Figure 9: RM configuring Line ID on the BNG The configuration required to provision the Line ID on a BNG port has been abstracted into a CLI configlet in Junos Space. RM invokes the apply-configlet API on this configlet as shown in the POST method invocation. The figure shows the syntax of the request body supplied by RM in this invocation as well as that of the response body coming back from Space. As you can see, all the complexity of the actual BNG configuration that needs to be pushed to the BNG is hidden from RM. As a client of this API, RM just needs to identify the BNG via its unique ID (shown as {deviceid}2 in the figure) and specify the values for the two parameters (PORT and LINE_ID). The example in this figure configures the Line ID ABC123XYZ on the port ge-2/1/1 of the BNG. Space creates a backend job to compute the actual configuration that needs to be applied and push it to the BNG. Copyright 2014, Juniper Networks, Inc. 19

This job is performed asynchronously and its unique ID (shown as {jobid} in the figure) is returned in the response body. The asynchronous semantics fits in well with the threading model used in RM and helps scale the solution by performing multiple configuration changes on multiple devices concurrently. RM has a separate process that monitors the status and results of these background jobs using the job management API as shown in the figure. Activation of subscriber services is performed dynamically by the BNG based on policy provisioned in RADIUS. The native broadband subscriber management capability of Junos OS is used for residential subscriber services, and the ESSM Framework is used for business subscriber services. Assurance The Assurance process grouping is responsible for the execution of proactive and reactive maintenance activities to ensure that services provided to customers are continuously available and are meeting SLA or QoS performance levels. Key functions in this area include Resource Performance Management (RPM), Resource Trouble Management (RTM), Test and Diagnostics, and Trouble Ticketing. Junos Space plays a key role in these functions as identified earlier in Table 2. In this section, we will take a look at how operators and OSS components use Junos Space for these functions. KPIs are defined for measuring the network performance and for the identification of performance trends. These KPIs allow for the visualization of the most important performance figures as well as the generation of warnings in case of upcoming problems. The KPIs are in turn mapped to a set of performance counters that are to be monitored on the BNG. Junos OS provides four different ways by which these counters can be measured: (a) SNMP polling; (b) CLI; (c) XML RPC; and (d) generate comma-separated value (CSV) files. Due to the volume and frequency of performance data that needs to be collected, the most efficient mechanism in this case turned out to be the fourth option, generating CSV files. This is achieved by configuring a feature known as accounting profiles in Junos OS. An accounting profile represents common characteristics of collected accounting data, including the following: Collection interval File to contain the accounting data Specific fields and counter names on which to collect statistics Archive option for the file transfer Once an accounting profile is configured on the router, it automatically collects the configured statistics at the configured intervals and writes them to a CSV file as per the configured filename. The file can also be automatically transferred to an external server. Three different accounting profiles are used in this deployment: Routing Engine (RE) profile to collect device-level counters Interface profile to collect physical and logical interface level counters MIB profile to collect values from some specific MIB object identifiers (OIDs). As shown in Figure 10, accounting profiles are configured on the BNG from Junos Space GUI using its CLI Configlets feature, as part of the BNG commissioning process described earlier. Two separate configlets have been designed for this, one to configure the RE profile and the other to configure the interface profile. This configuration remains static over the lifetime of the BNG. However, the MIB accounting profile is configured and maintained by an event script. An event script is a Junos OS automation script that is automatically triggered by certain events occurring on the router or based on a timer. In this case, the event script is configured to be triggered every midnight. When it runs, the script determines the set of interfaces that are core-facing and uses this information to configure the set of MIB OIDs whose values need to be collected. It is designed this way because a more generic collection of MIB counters by performing MIB walks can be detrimental to the performance of the BNG RE. Hence, the event script is used to determine specific OIDs that need to be read, and they are explicitly configured into the MIB profile. This event script is maintained on Junos Space and is deployed onto the BNG from Space during the BNG commissioning process. 20 Copyright 2014, Juniper Networks, Inc.

CLI Configlets for: Routing Engine Profile Interface Profile DMI Accounting profiles configured via configlets Event script to configure MIB profile BNG Event Script to auto configure MIB Profile Data Collection ESSM Framework Junos Space BNG monitoring Business service monitoring RPM Scripts SFTP csv file Router metrics csv file Interface metrics xml file Business Service Accounting for Performance metrics SQM File Server csv file MIB metrics Figure 10: RPM architecture Performance metrics configured in these profiles are collected periodically (at 15 minute intervals in this deployment) and written into CSV files. In addition, performance metrics for business services get written to XML files by the ESSM Framework. These files are pushed using Secure File Transfer Protocol (SFTP) to an external file server where they undergo some post processing using custom scripts. After processing, these files are consumed by the RPM and Service Quality Management (SQM) systems compute device-specific and service-specific KPIs to monitor SLA compliance. The RTM system deployed in the OSS layer performs network-wide fault management in this customer s NGSSM architecture. It receives SNMP v3 traps directly from all network elements, including the BNG routers in the network. This system maintains the current log of alarms and correlates it with other collected information to determine the probable cause of problems. It also performs SNMP v3 queries on the network elements to discover network topology and to monitor the health of various components. SNMP v3 access details as well as the trap destination are configured on the BNG routers from Space using a configlet that has been specifically designed for this purpose. This is done as part of the BNG commissioning process described earlier. An important function within the Assurance process grouping is to perform various testing and diagnostics operations on network elements. These operations may be in response to a reported or suspected problem as well as done proactively to prevent performance degradation or failures. In either case, Junos Space provides a rich set of capabilities to perform these tests using its GUI or via its REST Web servicess APIs. A typical test and diagnostic operation can be broken down into three main steps: Setup: This step inserts the required configuration on the network elements on which the test needs to be performed. A set of configlets has been designed for this step of various tests. The operator using Space GUI can apply these configlets on BNGs. In addition, the apply-configlet API has been used to integrate Space with the T&D system software, which is the designated OSS component for performing test and diagnostics operations in this deployment. This ensures that the entire operation can be performed as one workflow from the T&D system GUI. Execution: This step performs one or more commands on the network elements and collects the results of the test. Various Op Scripts have been designed to perform this step for the tests required in this deployment. These scripts perform the required commands on the network element for each test and generate test results that can be evaluated by the operator. An operator can use Space GUI to execute these scripts and in this case test results will be rendered in Space GUI. Alternatively, script execution can be triggered via an API call from the T&D system to Space to facilitate running of the Test and Diagnostics operation as a single workflow from the T&D systemgui. Teardown: This step removes the configuration on the network elements that have been inserted in the first step. A set of configlets has been designed for this step of various tests. As in the case of the previous steps, this step can also be performed from Space GUI or from the T&D system GUI. Copyright 2014, Juniper Networks, Inc. 21

802.1ag session per service 802.3ah session ANCP keepalive CPE AN BNG A) Subscriber connected via Access Node 802.1ag session per service 802.3ah session for the link CPE B) Directly Attached Subscriber Figure 11: OAM protocols used for Test and Diagnostics BNG Figure 11 shows various protocols used for Test and Diagnostics in this deployment. In the case of subscribers connecting via access node, an 802.3ah link fault management (LFM) session is configured between the CPE and the access node. The link between the access node and the BNG is monitored using Access Node Control Protocol (ANCP) keepalives. In the case of directly attached subscribers, an 802.3ah LFM session is configured between the CPE and the BNG. LFM configuration is inserted on the BNG port using a configlet in Space during the setup phase of the test and removed using another configlet during the teardown phase. Test readouts are performed using Op Scripts deployed and executed on the BNG by Space. Figure 11 also shows 802.1ag connectivity fault management (CFM) sessions between the CPE and the BNG for each service that is activated. The configuration for this is dynamically applied on the BNG as part of service activation via RADIUS message exchanges. Similarly, this configuration is dynamically removed on the BNG when the service is deactivated. At any time, CFM readouts can be performed using an Op Script either from Space GUI or from the T&D system GUI. Figure 12 shows the overall integration architecture for the Test and Diagnostics function listing the main components of this solution and highlighting the fact that test operations can be initiated either from Space GUI or from the T&D system GUI. The ESSM Insight application running on Junos Space also plays an important role in Assurance for business services by providing a GUI for performing troubleshooting actions on them. It allows the operator to examine the status of a service, look at detailed statistics on the associated logical interface, perform a readout of the CFM session parameters corresponding to this service, and examine the history of events related to the service. All of these capabilities are also exposed as REST Web services APIs by this app and are used for integrating with the T&D system to create Test and Diagnostics workflows for business services. 22 Copyright 2014, Juniper Networks, Inc.

Test and Diagnostics REST Configlets for setup and teardown Scripts for readout ESSM insight DMI Junos Space MX960 BNG Figure 12: Test and Diagnostics overall integration Another important function within the Assurance process grouping is Trouble Ticket Management. When network- or service-level problems are identified by RPM and RTM systems and cannot be rectified locally by the service provider, a trouble ticket needs to be submitted and tracked with the equipment vendor. In this deployment, the trouble ticketing (TT) system talks to the Junos Space Service Now application via the OSS/J JSR91 Trouble Ticket API [8] to automate the process of creating and managing support cases with Juniper. This API allows clients to accomplish the following: Query, create, close, or cancel trouble tickets Change the values of trouble tickets Be informed of trouble ticket changes via notifications When a trouble ticket related to Juniper equipment or software is created in the TT system, a support case is automatically created with Juniper Support System (JSS) using this API as shown in Figure 13. This API also allows the TT system to be notified whenever there are changes to the state of these trouble tickets, allowing it to keep track of the progress of each support case that it has submitted. In addition, the API is also used by the TT system to perform other management operations such as canceling and closing support cases. As shown in Figure 14, the Trouble Ticket API defined by JSR91 is implemented as a Web services adapter that internally invokes native REST APIs exposed by Service Now to create and manage support cases with JSS. The adapter implements change notification by periodically polling JSS for case status and sending notification messages to subscribed clients. This adapter is packaged along with the Service Now application for easy deployment. Copyright 2014, Juniper Networks, Inc. 23

SOAP Request TT SOAP Response Notify JSR 91 Web Service API Adapter REST Service Now Poll Status Create Case Juniper Support System (JSS) Junos Space Figure 13: Trouble Ticket Management Security Junos Space is designed as a secure network management platform. It is based on the CentOS operating system that is binary compatible with Red Hat Enterprise Linux and is further hardened for security by the disabling of all unnecessary operating system services and the securing of all networking connections. Junos Space uses Device Management Interface (DMI) to communicate with all managed devices. This interface runs on top of a secured SSHv2 connection, which provides authentication, confidentiality, and integrity for all communication with devices. Northbound interfaces including both Web browser-based clients as well as the NBI interfaces towards the higher level OSS systems utilize secured HTTPS connections. Roles Determines: Types of objects user can access Actions that can be performed on these objects User Can have: GUI-only access API access Both Permission Labels Determines: The actual sub set of objects that the user can access Figure 14: User authorizations 24 Copyright 2014, Juniper Networks, Inc.

All access to Junos Space from the GUI or from an NBI client is authenticated and authorized by a comprehensive role-based access control mechanism. Authentication can be based on a username/password combination or the PKI/X.509 certificate presented by the client. Authorization is based on the set of roles and permission labels assigned to the user account. To access and manage Junos Space, a user account must be assigned one or more roles, which are validated during authorization. These roles control the workspaces the user can access and the tasks that can be performed on the objects that are managed within a workspace. Hence, a role can be considered as defining the types of objects that a user can access and the actions that the user can perform on these object types. Junos Space ships with a set of predefined roles and allows the administrator to create fine-grained, customized user roles that match the type of access control that the administrator wants to enforce. The set of objects that a user is allowed to access is determined by the set of permission labels assigned to the user account. When an operator logs into Space, only those BNGs that have the permission label that has been assigned to the operator s user account in Space will be visible. This is illustrated in Figure 15 that shows four different permission labels for four different regions. Each permission label is assigned to all devices in that region and to two user accounts. To take an example, user1 would only be able to see the four devices that have been assigned with the permission label Region 1, and user5 would only be able to see the four devices assigned with the permission label Region 3. However, the superuser will be able to see all devices at all times. Region 1 user1 user2 Region 3 user5 user6 super Region 2 user3 user4 Region 4 user7 user8 Fig. 15: Device segregation using persmission labels Table 5: Audit Log Contents Field Username User IP Task Timestamp Result Job ID Description Description The login ID of the user who initiated the task The IP address of the client computer from which the user initiated the task The name of the task that triggered the audit log The UTC time in the database that is mapped to the local time zone of client computer The execution result of the task that triggered the audit log: Success job completed successfully Failure job failed and was terminated Job Scheduled job scheduled but has not yet started. Audit log including the job ID for each job-based task A description of the audit log Copyright 2014, Juniper Networks, Inc. 25

Earlier in this guide, we saw how this feature is used to segregate BNG routers into different regions and have separate sets of users whose access is limited to the BNGs within their region. Another use of permission labels is to control the set of configlets and scripts that a user is allowed to access. For example, using permission labels, it is possible to create a user who can execute configlets and scripts related to testing and diagnostics only, and another user who can execute configlets and scripts related to commissioning only. Figure 14 shows how user rights are controlled using roles and permission labels. As shown in the figure, it is also possible to limit the access of a specific user account to GUI-only access or API-only access. In this customer deployment, separate user accounts are created for each OSS component that is integrated with Junos Space. These user accounts are defined as API-only accounts, and custom roles are assigned to them to limit the set of actions they can perform. Operators have their own individual user accounts with access rights for only the GUI and not the API. Junos Space maintains audit log entries for all actions performed by GUI and API clients. Since Junos Space is used as the central EMS system for making any change on BNGs, this provides a reliable audit trail of all changes made on the BNGs in the network. Table 4 shows the details maintained by Junos Space for each audit log entry. Audit log entries are archived monthly onto a remote server via SCP. Junos Space uses a small number of communication ports for inbound and outbound communications. It has a builtin firewall based on iptables which is enabled by default. For additional security, the Junos Space cluster is deployed behind a firewall and only the minimum required ports are opened as shown in Table 5 below. Table 6: Communication Requirements. Source Destination Application Protocol Port Authentication/ Encryption Description Operator, Web GUI Space Cluster VIP HTTPS TCP 443 yes/yes Web GUI Administrator, CLI Space Node-1 IP SSH TCP 22 yes/yes CLI Administrator, CLI Space Node-2 IP SSH TCP 22 yes/yes CLI OSS Components Space Cluster VIP HTTPS TCP 443 yes/yes NBI (RESTful API) Space Node-1 IP Network elements IP SSH TCP 22 yes/yes DMI Space Node-2 IP Network elements IP SSH TCP 22 yes/yes DMI Space Node-1 IP Network elements IP Ping ICMP n/a no/no Discovery Space Node-2 IP Network elements IP Ping ICMP n/a no/no Discovery Space Node-1 IP Network elements IP SNMPv3 UDP 161 yes/yes Discovery Space Node-2 IP Network elements IP SNMPv3 UDP 161 yes/yes Discovery Network Elements IP Space Cluster VIP SNMPv3 UDP 162 yes/yes Traps Space Node-1 IP AAA server, TACACS+ TACACS+ TCP 49 yes/yes Authentication/ authorization Space Node-2 IP AAA server, TACACS+ TACACS+ TCP 49 yes/yes Authentication/ authorization Space Node-1 IP Space Node-2 IP Space Node-1 IP Space Node-2 IP Backup server, SCP target Backup server, SCP target Archiving server, SCP target Archiving server, SCP target SCP TCP 22 yes/yes Backup SCP TCP 22 yes/yes Backup SCP TCP 22 yes/yes Audit log archiving SCP TCP 22 yes/yes Audit log archiving Summary Broadband Network Services Orchestration and Management represents one of the important use cases for implementing dynamic service provisioning using Junos Space Platform and its REST APIs. The same generic solution architecture can be used for dynamic provisioning of other services and as a component of service and operations automation. The key goals of the solution are: To significantly reduce the CapEx and OpEx costs for the network operator by providing a single pane of glass for managing the complete lifecycle of BNG routers and associated services To make the service provisioning process highly reliable by eliminating manual procedures To accelerate the overall service provisioning process To enable the rapid and efficient integration with existing OSS/NMS solutions using Junos Space REST APIs 26 Copyright 2014, Juniper Networks, Inc.

This solution has been enabled with the programmable Junos Space Platform and represents a major shift in development of network orchestration solutions. Programmable interfaces into the network provide the ability to automate the process completely. Additionally, programmable interfaces enable custom applications to be developed to extend and customize functionality. In this particular use case, a customized provisioning application was developed to automate the provisioning while maintaining the same customized process. This prevented the need to replace the existing OSS, BSS, and IT systems, and to modify the methods and processes that had already been designed as part of the NGSSM architecture. Further automation for inventory management, and troubleshooting capabilities for business subscriber services, show the power of the programmable platform to orchestrate the end-to-end service management process. References 1. Junos OS Broadband Subscriber Management Solutions Guide 2. Junos OS Extensible Subscriber Services Management Framework www.juniper.net/techpubs/en_us/junos13.2/information-products/pathway-pages/product/13.2/index.html 3. This Week: Mastering Junos Automation Programming 4. TM Forum Enhanced Telecom Operations Map http://en.wikipedia.org/wiki/enhanced_telecom_operations_map 5. Junos OS Network Interfaces Configuration Guide www.juniper.net/techpubs/en_us/junos13.1/information-products/topic-collections/subscriber-mgmtsolutions/broadband-subscriber-mgmt-solutions.pdf www.juniper.net/in/en/community/junos/training-certification/day-one/automation-series/mastering-junosautomation www.juniper.net/techpubs/en_us/junos13.1/information-products/pathway-pages/config-guide-networkinterfaces/network-interfaces.html 6. Apache Velocity Template Language Reference Guide http://velocity.apache.org/engine/releases/velocity-1.6.2/user-guide.html 7. Multi Technology Operations Systems Interface (MTOSI) http://en.wikipedia.org/wiki/mtosi 8. OSS/J Trouble Ticket API http://jcp.org/en/jsr/detail?id=91 9. Representational State Transfer (REST) http://en.wikipedia.org/wiki/representational_state_transfer Copyright 2014, Juniper Networks, Inc. 27

Appendixes A) Select an equipment and view contained ports B) Ports under the selected equipment C) Logical interfaces under the selected port Figure 16: Inventory Navigation 28 Copyright 2014, Juniper Networks, Inc.

A) Select an equipment and choose Execute Scripts B) Select the script and click Execute C) Results are displayed immediately Figure 17: Script execution on inventory components Copyright 2014, Juniper Networks, Inc. 29

Figure 18: Business service inventory provided by ESSM Insight About Juniper Networks Juniper Networks is in the business of network innovation. From devices to data centers, from consumers to cloud providers, Juniper Networks delivers the software, silicon and systems that transform the experience and economics of networking. The company serves customers and partners worldwide. Additional information can be found at www.juniper.net. Corporate and Sales Headquarters Juniper Networks, Inc. 1194 North Mathilda Avenue Sunnyvale, CA 94089 USA Phone: 888.JUNIPER (888.586.4737) or +1.408.745.2000 Fax: +1.408.745.2100 www.juniper.net APAC and EMEA Headquarters Juniper Networks International B.V. Boeing Avenue 240 1119 PZ Schiphol-Rijk Amsterdam, The Netherlands Phone: +31.0.207.125.700 Fax: +31.0.207.125.701 To purchase Juniper Networks solutions, please contact your Juniper Networks representative at +1-866-298-6428 or authorized reseller. Copyright 2014 Juniper Networks, Inc. All rights reserved. Juniper Networks, the Juniper Networks logo, Junos and QFabric are registered trademarks of Juniper Networks, Inc. in the United States and other countries. All other trademarks, service marks, registered marks, or registered service marks are the property of their respective owners. Juniper Networks assumes no responsibility for any inaccuracies in this document. Juniper Networks reserves the right to change, modify, transfer, or otherwise revise this publication without notice. 8020017-001-EN Jan 2014 30 Copyright 2014, Juniper Networks, Inc.