Federation of trouble ticketing systems

Transcription

1 Federation of trouble ticketing systems Pavle Vuletić, AMRES, Jovana Vuleta-Radoičić, University of Belgrade, Dimitrios Kalogeras, GRNET/ NTUA 8th TF-NOC meeting 28 May 2013

2 Motivation! This work is one of several results of 4 years of research within GN3 Control and Management research task.! The aim of the task was to promote modern network management best practice and make recommendations towards their adoption for the multi-domain network services and tools that are being developed in Geant.! Multi-domain services are more complex to manage than single-domain multitude of operational procedures, device types, OSS components! Modern network management aims to make (multi-domain) services fully automated. There is a general trend towards autonomic networking. There are many papers, special journal issues about these topics, specifications, software artefacts...! Autonomic networking in multi-domain environment assumes that a tool from remote domain initiates configuration of resources in another domain! And... we heard many times: NOCs will never accept that! 2

3 Why are NOCs reluctant to adopt autonomic solutions?! Henry Fayol ( ) -14 principles of management:! Division of work! Authority! Discipline! Unity of command. Every employee should receive orders from only one superior.! Unity of direction.! Subordination of individual interests to the general interest! Remuneration! Centralisation! Scalar chain! Order! Equity! Stability of tenure of personnel. Initiative.! Esprit de corps. Is this principle also applicable to network elements? 3

4 GEANT-NREN environment! GEANT is a federated environment in which multi-domain services span at least the following domains operated by different entities:! Campus networks for end institutions! NRENs (in few cases composed of regional networks)! GÉANT network as the backbone! Tightly coupled vs. loosely coupled federations:! set of domains that are governed by a single central authority in which each domain has a set of limited powers regarding their own local interests! set of independent network domains that retain the responsibility over their internal management! GEANT-NREN is closer to a loosely coupled federated environment (domains are autonomous and have full control over the resources they own).! Such environment is not suited for automated multi-domain service operations (provisioning). 4

5 What if we virtualize network elements?!... and create multi-domain services out of virtual parts of the infrastructure in various domains which are used exclusively for those services! From the service operations viewpoint, exposing and controlling virtual instances of resources in distant domains is not always the case of service operations in the federated service environment.! A set of resources (physical or virtual) that are administered without any restriction by a single entity wherever these resources are physically placed is a case of single domain service operations! From the business level viewpoint such case is the case of federated service environment, as multiple domains create service instances and appropriate contracts and policies that regulate the use of virtual resource instances in remote domains have to be defined, 5

6 So... what could we do then?! Process flows are useful tools to detect main actors and inter-domain communication patterns in multidomain environment! Federated service problem resolution a potential process flow User Relationship Management Service Management & Operations Service user 1.a 8.d Manage Request User Interface Management 1.b Problem Handling Create User Problem Report 8.c 1.c 8.a 5.f Isolate User Problem 1.d Track & Manage User Problem 1.f Create Service Trouble Report 2 Track & Manage Service Problem 5.a 1.e 8.b Service Problem Management 3.a j4.a Correct & Recover User Problem Close User Problem Report Domain A Diagnose Service Problem Correct & Resolve Service Problem Close Service Trouble Report Enable Multidomain Service Problem Management Multi-domain Service Interaction 4.b 4.i Domain B Service Problem Management 4.c Enable Multidomain Service Problem Management Multi-domain Service Interaction Create Service Trouble Report 4.d Track & Manage Service Problem 4.h 4.e 4.f 4.g Diagnose Service Problem Correct & Resolve Service Problem Close Service Trouble Report Resource Management & Operations Create Resource Trouble Report 5.b Track & Manage Resource Trouble 5.c 5.d 5.e Localize Resource Trouble Correct & Resolve Resource Trouble Close Resource Trouble Report Resource Trouble Management 6

7 Federated trouble ticketing system! Key inter-domain processes are information exchanges which enable service configuration, problem resolution, service quality management and so on! Information can be exchanged using , phone calls but we prefer TTS! TTS is more efficient than simple /phone interaction because it has inherent capabilities for tracking, escalation, expiration,! Federation of TTS: a ticket created in one domain is sent to the other domains which are part of the service instance that has to be configured or checked for problems or performance issues.! Trouble ticket which is received by receiving domain is automatically installed in the TTS of the receiving domains 7

8 Federation of TTS! Not a new concept! Previous solutions/ideas:! Vodafone case [1] exchange of identity information between Vodafone owned and partner companies worldwide using federation of TTS and standard OSS/J interfaces! The possibility of exchanging trouble tickets across federation boundaries, together with the service orders is marked as the key control plane interaction between the federation members in federated defence management systems [2]! GRID community Federated TTS for GRID operations/problem management centralized FTTS architecture (one central NOC), [RFC 6137] defined ticket data model [1] M. Meyners, B. Driss, U. Feger, Case study TMF615 OSS Identity management, February 2008, [2] Federated Service Management for Defence (Catalyst) - Blueprint, TMF 866, TM Forum Approved, Version 1.2, April

9 Architecture of the proposed solution! Other components that can improve the system:! Federated Service Inventory (a database of service instances and contact points in domains)! Federated Service Design support tool 9

10 How should it work? Initiating domain! Problem reported! Ticket created in the home domain! FSI queried for the list of affected domains! TT sent to affected domains and TT set to Wait_for_others state >! Check end-to-end operations! If problem is resolved close the ticket, else, reopen it. Participating domains! Receive the ticket! Participating domains analyze the problem and return either problem_resolved or problem_not_detected message! <

11 Geant FTTS main design guidelines! 29 NRENs use 11 different TTS, some home built [3]! Domains are typically reluctant to abandon their own tools as this often implies additional transition and training costs and reduced reliability of operations in the transition period! Development and maintenance of custom made software tools or plugins for a variety of TTS systems used by NRENs has high development cost! We prefer lightweight solutions based on pure configuration and customization of the existing systems! We wanted to enhance the existing capabilities of the already installed TTS in NRENs in order to implement the proposed FTTS! FTTS could be used not only for problem management, but also for configuration, performance management and all other federated processes which require inter-domain information exchange.! Fully decentralized solution: no central NOC [3] Gandia Carriedo, M. I., Listrom, S., Limberg, S., Vuletić, P., Szegedi, P.: TF-NOC Software Tools Survey Results: Analysis and Dissemination, TERENA 2012, 11

12 Required TTS capabilities! Almost all TTSs (commercial and open source) available today have the required capabilities for the TTS federation:! The possibility to create several types of trouble tickets! Customizable trouble ticket data model (we want to add e.g. service instance ID as an identifier of the federated TT exchange)! Customizable trouble ticket lifecycle (workflow) with custom states and post-functions! External communication via as a trigger or as post function for a trouble ticket status change! The possibility to define user defined templates for communication 12

13 Implementation details! A prototype which consists of several RT and Jira instances! FSI stub: Java application! Inter-TTS communication: , can be web-services (OSSJ TTS API)! TTS-FSI communication: communication with the standalone Java application which is integrated with FSI using web-services (WS_integration_profile) TTS TTS Java app Java app WS FSI WS FSI 13

14 Some conclusions! In loosely coupled federations with the current lack of protocol that can automate configuration beyond the domain border according to the business level decisions/constraints, FTTS seems to be the highest possible level of integration between the domains! FTTS can be used as expected for problem resolution, but as well for the exchange of configuration requests and other inter-domain request exchanges! FTTS can be achieved using mainly configuration of TTS with as little as possible software development 14

15 Thank you!