Providing quality of service (QoS) guarantees

Transcription

1 INTERNATIONAL JOURNAL OF NETWORK MANAGEMENT Int. J. Network Mgmt 2000; 10:75 90 Monitoring QoS distribution in multimedia networks By Chen-Khong Tham, Yuming Jiang Ł and Chi-Chung Ko This paper presents two schemes, relevant monitor (RM)-based and improved relevant monitor (IRM)-based, for QoS distribution monitoring. With these schemes, when monitoring a real-time flow, a network manager can locate relevant monitors that are metering the flow. Copyright 2000 John Wiley & Sons, Ltd. Introduction Providing quality of service (QoS) guarantees is an important requirement for multimedia networks. To maintain agreed QoS, it is not sufficient to just commit resources because QoS degradation can be caused by many factors and is often unavoidable, e.g. any fault or weakening of the performance of a network element may result in the degradation of contracted QoS. Thus, performance management is required to ensure that the contracted QoS is sustained. 1 To date, there has been a considerable amount of research within the field of QoS management support for multimedia networks, including the service model, 2 QoS control 3 and QoS architecture, 4 approaches. However, one limitation still exists: the lack of distributed monitoring mechanisms to support QoS guarantees. 4 The intention of QoS monitoring research is to allow a network manager to track the ongoing QoS, compare monitored QoS against expected performance, detect possible QoS degradation, and then tune network resources accordingly to sustain the delivered QoS. 4 Since the real-time flow of a multimedia application may cross several network segments that provide different levels of QoS, in order to locate the segment(s) causing possible QoS degradation, QoS monitoring should be based on QoS distribution monitoring instead of end-to-end monitoring, i.e. a network manager should be capable of monitoring the distribution of QoS experienced by the flow in different network segments in order that the manager can locate the cause of possible QoS degradation. This paper presents two schemes for QoS distribution monitoring. They are the relevant monitor (RM)-based scheme and improved relevant monitor (IRM)-based scheme. With these schemes, when monitoring a real-time flow, a network manager can locate relevant monitors that are metering the flow, and retrieve traffic information from them simultaneously. By consolidating the information from these monitors that are embedded in different network segments, not only can end-to-end QoS be monitored, but the QoS distribution in different network segments for this flow can also be derived. Based on these schemes, the network manager can locate the cause of possible QoS degradation and thus control the network accordingly. The rest of this paper is organized as follows. The next section presents the challenges involved in QoS monitoring and gives an outline of previous work in the areas of application-level monitoring and QoS monitoring. The third section introduces two new schemes, RM-based and IRM-based, for QoS distribution monitoring. The fourth section describes CORBA-based implementations of the two schemes which show their feasibility. The fifth section provides the implementation details of our prototypes and some results. The methods for deriving QoS related parameters and for synchronizing traffic displays are also introduced in this section. After this, the sixth section presents a qualitative comparison among monitoring mechanisms surveyed in the second section and presented in the third section. The final section offers some concluding remarks. Ł Correspondence to: Yuming Jiang, Department of Electrical Engineering, National University of Singapore, 10 Kent Ridge Crescent, Singapore feletck, engp7450, elekoccg@nus.edu.sg Copyright 2000 John Wiley & Sons, Ltd.

2 76 CHEN-KHONG THAM ET AL. Motivation and Related Work Performance monitoring is an absolute prerequisite for the management of a communications network. A network manager cannot hope to manage and control a network unless he or she can monitor its performance. In multimedia networks, real-time multimedia applications such as Internet telephone and video conferencing form an important part of network traffic. These applications are time-critical and may have different QoS requirements. Thus, QoS monitoring is central to performance monitoring in multimedia networks. Compared with traffic monitoring in traditional data networks, to provide QoS monitoring in multimedia networks imposes the following challenges. Performance monitoring is an absolute prerequisite for the management of a communications network. First, QoS monitoring requires applicationlevel monitoring. Throughout this paper, any protocol above the network-layer is considered application-level as defined in the remote network monitoring specification version 2. 7 In traditional networks, the monitored objects of traffic monitoring are the total traffic into and out of the device that a monitoring agent (e.g. SNMP 5 agent) attaches to. Thus, traditional traffic monitoring is limited to the network layer. In contrast, in multimedia networks, real-time applications with different QoS requirements contribute an important part of network traffic and the monitored objects are real-time flows. Thus, application-level monitoring is required for QoS monitoring, i.e. QoS for each multimedia application is monitored. Second, it is QoS distribution that should be monitored instead of end-to-end QoS. Consider the example shown in Figure 1. Q inj denotes the QoS Figure 1. QoS distribution monitoring experienced by flow i in network segment n j ; Q i denotes the end-to-end QoS seen by flow i;andp i D n 1,...,n j,...,n k is the set of segments that flow i passes. Then, we have Q i D F Q in1,...,q ink.this equation means that the end-to-end QoS of a flow is a function of the QoS experienced by the flow in each network segment. Given that all Q inj, n j 2 P i, are known, it is possible to derive Q i. For example, given the delay and loss rate in each segment, the end-to-end delay and loss rate can be easily derived. However, if only the end-to-end delay or loss rate is known, it is not possible to determine the delay or loss rate in each segment. Therefore, in order to detect possible QoS degradation and locate the cause of the degradation, not only does a network manager need to know the Q i, but also the Q inj, n j 2 P i. Hence, end-to-end QoS monitoring is not sufficient and QoS distribution Q in1,...,q ink should be monitored. Third, since different real-time flows may cross different network segments, the monitors involved in QoS monitoring of different real-time flows can be different. Thus, a QoS monitoring scheme should be able to find out which monitors are monitoring a certain real-time flow so that traffic information can be collected from these monitors and QoS distribution of the flow can be further derived. Lastly, since a network manager can be mobile, for example Web-based, the QoS monitoring scheme is required to provide mechanism(s) for the mobile manager to locate relevant monitors andviceversa. Over the past few years, several mechanisms have been proposed in the context of applicationlevel monitoring and QoS monitoring. Waldbusser 7 proposed an extension to the remote network monitoring (RMON) specification, 6 which is referred to as RMON-2. With RMON-2, an RMON-2 probe is not limited to monitoring and decoding network-layer traffic. It can also view higher-layer protocols running on top of the network-layer protocol. In particular, an RMON-2 probe is capable of seeing above the IP layer by reading the enclosed higher-level headers such as TCP and viewing the headers at the application level. This allows a network manager to perform application-level monitoring that is required in QoS monitoring. Brownlee et al. 8 and Brownlee 9 proposed an architecture, referred to as the real-time flow

3 MONITORING QoS DISTRIBUTION 77 measurement (RTFM) architecture, for the measurement and reporting of network traffic flows. A traffic flow, which may be generated by a multimedia application, is defined as an artificial logical equivalent to a call or connection, belonging to an accountable entity. 8 In practical terms, a flow is a stream of packets passing across a network between two end points or being send from a single end point. A flow s accountable entity is specified by the values of its address attributes which may include one or more specific addresses for each layer of the OSI reference model. Thus, the RTFM mechanism can also provide applicationlevel monitoring. Mourelatou et al. 10 presented an agent-based approach to identifying QoS problems. The agents are responsible for monitoring the end-to-end QoS. It was shown that a management system is capable of identifying the cause of performance degradation by correlating the information from these QoS monitoring agents. One key assumption in this approach is that end-to-end QoS can be monitored and the corresponding information provided by dedicated agents. However, the mechanism through which the agent could monitor the end-to-end QoS was not discussed. Ehab Al-Shaer 11 proposed an event-driven dynamic monitoring approach for multimedia networks. The task of detecting primitive and composite events is distributed among dedicated monitoring agents as in reference 10. An example of an event is the QoS degradation of a multimedia application. Unlike the assumption in reference 10, this approach requires that prior to any monitoring operation, a system manager must describe the physical or geographical distribution of the multimedia application that the manager intends to monitor. However, it was not shown how the system manager can determine the distribution of a multimedia flow inside the multimedia network. Chen et al. 12 introduced a software approach to monitoring end-to-end QoS in ATM networks. In order to monitor the QoS of a selected virtual connection, test cells are sent along a parallel connection that has been established with the same route and QoS class as the selected virtual connection. By taking advantage of the fact that ATM switches will statistically multiplex the test cell stream and user cell stream together, the test cells will experience a QoS similar to that seen by the selected user connection. Moreover, because the QoS of the test connection can be derived from the information carried by the test cells, the QoS of the selected user connection can be deduced accordingly. A key requirement for this approach is that a parallel test connection with the same route and QoS class be set up to test the selected user connection. Therefore, permanent virtual connections (PVCs) pre-established by network managers for user connections are used, so that the network managers know the route of a user connection and hence the test connection can be set up accordingly. However, this may be a restrictive requirement since in a real network environment, user connections can be established dynamically and the corresponding routes can also be dynamic. Another possible scheme that may be used for QoS monitoring is the real-time control protocol (RTCP)-based scheme. 13 In the RTCP-based scheme, traffic information and QoS parameters can be directly retrieved from multimedia transmission control messages, i.e. RTCP messages. This retrieval can be done by RTCP monitors. 13 However, since the QoS parameters retrieved in this scheme are between sender(s) and receiver(s), only end-to-end QoS can be monitored. In summary, various researchers have proposed several QoS monitoring and related approaches from different perspectives and shown that QoS monitoring can be achieved if certain prerequisites or assumptions can be satisfied. Waldbusser, 7 Brownlee et al. 8 and Brownlee 9 show that application-level monitoring is possible and feasible; Mourelatou et al. 10 and Ehab Al-Shaer 11 provide further analysis or operations when QoS degradation has been detected and relevant information has been provided; Chen et al. 12 and the RTCP-based scheme show that end-to-end QoS monitoring can be achieved if the route of a realtime flow is given in Chen et al. 12 or a real-time flow is based on RTP. 13 However, none of these mechanisms addresses the problem of QoS distribution monitoring directly; there is no means for a network manager, which can be mobile, to find out the route of a real-time flow and where relevant monitors that meter the flow are located. QoS Distribution Monitoring This section presents two relevant monitorbased schemes for QoS distribution monitoring.

4 78 CHEN-KHONG THAM ET AL. They are the RM-based scheme and IRM-based scheme. In these schemes, when monitoring a real-time flow, traffic information is collected from relevant monitors that meter the flow in different network segments. By consolidating the information from these monitors, not only can endto-end QoS of the flow be monitored, but the QoS distribution in different network segments for this flow can also be derived. In addition, in the IRMbased scheme, such collection can be done from several network management domains, so that network managers from all network management domains can monitor the QoS distribution of the real-time flow. RM-based Scheme In the RM-based scheme shown in Figure 2, when the QoS distribution of a real-time flow is being monitored, only monitors that are monitoring the flow will be involved. This is achieved in the following way. Prior to the monitoring, a real-time application name server (RTANS) is set up and made known to all monitors and network managers. During the monitoring, each monitor registers the traffic attributes of monitored real-time flows and its reference with the RTANS. The attributes of a flow include addresses for the flow s source and destination, counts (e.g. packets and bytes) of the flow s traffic and other attributes. Then, in the RTANS, relevant monitors for each real-time flow are sorted according to the traffic attributes to generate a real-time application (RTA) list. Finally, a network manager selects a real-time flow from the RTA list and finds all relevant monitors to communicate with and retrieve traffic information from. Modules Figure 2 shows the model of the RM-based scheme. In this model, there are three Figure 2. RM-based model principal modules, i.e. the analysis application module, the monitor module and the RTANS module. An analysis application is a part of a network manager program that analyzes traffic information retrieved from different monitors and provides results such as QoS-related parameters and QoS distribution of certain real-time flows to users. The retrieval of traffic information is achieved through the following steps. First, a user uses the analysis application program to select a real-time flow to monitor. This selection is from the RTA list obtained from the RTANS. Second, from the list, the analysis application can find out which monitors are monitoring the flow. It then communicates with and retrieves traffic information from them. Finally, the analysis application provides analysis results (e.g. QoS distribution) to users according to the information collected from all relevant monitors. The monitors (e.g. RMON-2 or RTFM monitors) residing in different network segments provide real-time measurement of real-time flows. In this scheme, when a real-time flow is selected for monitoring, only those monitors that are monitoring this flow will be involved in its traffic information collection and reporting. A real-time flow, corresponding to a multimedia application, is identified by its source and destination addresses at various network layers 8 which may include its source and destination IP addresses, transmission port number (e.g. UDP port) and real-time transport protocol (e.g. RTP 13 ). Each monitor can provide traffic information (such as numbers of packets and bytes) of the real-time flow to analysis applications that have selected this real-time flow to monitor. In the RM-based scheme, an RTANS is set up prior to all monitoring operations. The RTANS provides a mechanism to bridge monitors and analysis applications. Such a mechanism enables the analysis applications to locate relevant monitors of a real-time flow and retrieve traffic information from them. The mechanism is as follows: (1) The RTANS is known to all analysis applications and monitors. Hence, even though an analysis application is mobile, it can still find relevant monitors of a realtime flow from the well-known RTANS. (2) The RTANS dynamically maintains a RTA list. In the list are current real-time flows that are monitored

5 MONITORING QoS DISTRIBUTION 79 in the network and references of relevant monitors that are monitoring these flows. The monitors involved in metering different flows may be different. The reference of a monitor can be used by an analysis application to locate and retrieve traffic information from the monitor. Figure 3 and Table 1 illustrate an example of the RTA list. In Figure 3, there are two real-time flows, Flow A from Snd A to Rcv A through S1, SW and S2, and Flow B from Snd B to Rcv B through S3, SW and S4, where S1, S2, S3, S4 and SW are five network devices with embedded monitors. Then, the RTA list generated in the RTANS will look like Table 1: S1, SW and S2, and, S3, SW and S4 are involved in monitoring Flow A and Flow B respectively. Interactions Figure 2 also shows the interactions between modules of the RM-based scheme. All monitors in the system are pre-configured to know where the RTANS is. During the monitoring, monitors register attributes of real-time flows and their references with the RTANS. The monitor reference refers to the communication objects that are generated to report traffic information to all analysis applications. The RTANS is used by analysis application(s) to find out which real-time flows are being monitored and which monitors are metering the flows. An analysis application can retrieve references of all relevant monitors from the RTA list in the RTANS. Once an analysis application locates the relevant monitors of a real-time flow, it will add its reference to a network manager list in each monitor. This list stores the references of all analysis applications that are collecting traffic information of the same real-time flow. Then, the monitor can use these references to locate corresponding analysis applications and report traffic information to them. Limitations There are two limitations in the RM-based scheme. One is that there is only one common RTANS for the whole network. Hence, this scheme cannot be used when there are more than one network management domains. Another limitation is that in this scheme, each monitor is required to be pre-configured to know where the RTANS is. This requirement is difficult to satisfy when monitors are distributed in a large-scale network. IRM-based Scheme The IRM-based scheme, shown in Figure 4, is an improvement over the RM-based scheme. The IRM-based scheme supports multiple network management domains. In this scheme, each network management domain has a RTANS that is well known only within its own domain. In addition, each RTANS should register its reference with all monitors so that each monitor knows where the RTANS is. Modules Figure 4 shows the model of the IRM-based scheme, which comprises of the same modules as the RM-based model. They are the analysis application module, the monitor module and the RTANS module. The functionality of the analysis application module in the IRM-based model is similar to that in RM-based model. The only difference is that in the Figure 3. A sample network Real-time Flow (sender, receiver) Flow A (Snd A, Rcv A) Flow B (Snd B, Rcv B) Relevant monitors S1, SW, S2 S3, SW, S4 Table 1. RTA list in RTANS Figure 4. IRM-based model

6 80 CHEN-KHONG THAM ET AL. IRM-based model, analysis application programs may be from different network management domains. The monitor module in the IRM-based model is slightly different from that in RM-based model. In the IRM-based model, each monitor maintains a RTANS list generated automatically by the registration of each new RTANS. The monitor uses the RTANS list to find with which RTANSs it should register metered real-time traffic attributes and its reference. However, in the RM-based model, each monitor is manually pre-configured to know where the only RTANS is. Unlike the RM-based model, there can be more than one RTANSs in the IRM-based model. Each RTANS belongs to a different network management domain and is made well known only to its corresponding domain. In addition, instead of manual pre-configuration, each new RTANS makes itself known to all monitors by registering with them. The references to all monitors are manually configured at each RTANS. Despite the difference between the RM- and IRMbased schemes mentioned above, each RTANS in the IRM-based model functions like the RTANS in the RM-based model: each RTANS dynamically maintains a RTA list from which an analysis application can find the references of relevant monitors that are monitoring certain flows and hence retrieve traffic information from them. Interactions Figure 4 also shows the interactions between modules of the IRM-based scheme. A new RTANS registers its reference to the RTANS list in each monitor. The monitor will use the RTANS references to locate corresponding RTANSs. The RTANS in each network management domain is used by analysis application(s) within its domain to find out which real-time flows are being monitored and which monitors are metering the same flows. The analysis application can retrieve references of all relevant monitors from the RTA list in the RTANS before monitoring a real-time flow. Once an analysis application locates relevant monitors of a real-time flow by using the monitor references generated and retrieved above, it adds its reference to a network manager list in each monitor. This list maintains the references of all analysis applications that are collecting traffic information of the same real-time flow from different network management domains. Then, the monitor can use these references to locate corresponding analysis applications and report traffic information to them. Improvement and tradeoff The interactions between modules in the IRM-based model show that it is similar to the RM-based scheme. But there are three principal differences: ž In the RM-based scheme, there is only one common RTANS for the whole network, while in the IRM-based scheme there are more than one RTANS, each for a different network management domain. ž In the RM-based scheme, prior to all QoS monitoring operations, each monitor is configured to know where the RTANS is. In contrast, in the IRM-based scheme, each RTANS registers its reference with all monitors during run-time. ž In addition, in the IRM-based scheme, each monitor can report traffic information of a certain real-time flow to multiple analysis applications belonging to different network management domains. These differences contribute to the improvement of the IRM-based scheme over the RM-based scheme. With the improved scheme, not only can relevant monitors be found and the traffic information retrieved, but these operations can also be performed by analysis applications from different network management domains. However, there is a tradeoff between the improvement and the system complexity. The RMbased scheme is less complex but only supports one network management domain. The IRM-based scheme is more flexible but more complex. Thus, when there is only one network management domain, the RM-based scheme is preferred. However, if more than one management domains exist in a network, the scheme to adopt should be the IRM-based scheme. CORBA-based Implementation To show the feasibility of the two proposed schemes, this section considers how these schemes

7 MONITORING QoS DISTRIBUTION 81 can be implemented in a heterogeneous and distributed network environment. The technique adopted is to use the Common Object Request Broker Architecture (CORBA). 14 CORBA Overview CORBA offers an environment for building distributed object-oriented applications. With CORBA, users gain access to distributed objects transparently without having to know where they are located or what software or hardware platform they reside on. The Interface Definition Language (IDL) and different programming language mappings to interfaces defined by the IDL enable client/server objects to interact among different Object Request Brokers (ORBs). To date, a lot of work in the context of network control and management has been done in the academic and industrial domains using the CORBA technique These achievements show that CORBA is a suitable technological framework for network management. RM-based Scheme In this section, the IDL interfaces and corresponding object interactions of the CORBAbased implementation of the RM-based scheme are described. IDL interfaces Tables 2, 3 and 4 define IDL interfaces of the three modules in the RM-based module RTANS f// Real-Time Application Name Server struct RtAttributes f...;g; struct RtItem f Monitor::ForManager monref; RtAttributes rtattributes;g; typedef sequence hrtitemi RtaList; interface ForMonitor f void register (in RtAttributes rtattr, in Monitor::ForManager monref);g; interface ForManager f void getrtalist (out RtaList rtalist);g; g; //End of RTANS module Table 2. Interfaces of RTANS module module Monitor f// Monitor interface ForManager f void setmanagerobj (in Manager:: ForMonitor mngref); void setupdateinterval (in long interval);g; g; //End of Monitor module Table 3. Interface of monitor module module Manager f// Manager struct Record f...;g; typedef sequence hrecordi Records; interface ForMonitor f void update TrafficInfo (in Records records);g; g; // End of Manager module Table 4. Interface of manager module (with analysis application) scheme, which are abstracted from the interactions described above. Note that the analysis application runs in the manager module. Interactions among CORBA objects Figure 5 shows interactions among CORBA objects of the RM-based scheme, which correspond to the client-side and server-side objects, i.e. the client object and implementation object respectively, of the interfaces defined in Tables 2, 3 and 4. In the following description, interface I in module M will be written as M::I. The interactions can be summarized in the following steps. (1) First, once a new realtime flow has been detected, a monitor initiates three CORBA objects: a client object of RTANS::ForMonitor, a client object of Manager::ForMonitor, and an implementation object of Monitor::ForManager. (2) Second, the monitor registers attributes of the real-time flow, i.e. RTANS::RtAttributes type, and the reference of the implementation object of Monitor::ForManager with the RTANS by invoking the register operation from the client object of RTANS::ForMonitor. At the same time, in the RTANS, the implementation object of RTANS::ForMonitor will add this new registration item to its relevant RTA list according to its traffic attributes. (3) Third, the manager gets the RTA list, RTANS::RtaList type, from the RTANS by invoking

8 82 CHEN-KHONG THAM ET AL. Figure 5. RM-based scheme: Interactions among CORBA objects getrtalist operation from the RTANS::ForManager. The manager selects a real-time application and its relevant monitors Monitor::ForManager references from the RTA list and starts to retrieve traffic information from these monitors. This causes the generation of several groups of CORBA objects. Each group, responsible for communicating with one monitor, consists of a client object of Monitor::ForManager and an implementation object of Manager::ForMonitor. (4) After these, the fourth step is that each object group of the manager sends its reference of implementation object of Manager::ForMonitor to its corresponding monitor by invoking setmanagerobj operation through the client of Monitor::ForManager. In addition, the manager sets the interval for the monitor to report traffic information through the operation setupdateinterval. (5) Lastly, each monitor, after receiving the Manager::ForMonitor implementation object reference, reports the current traffic information, of type Records, periodically in the required interval to the manager by calling the updatetrafficinfo operation through the client of the Manager::ForMonitor. IRM-based Scheme This section presents the IDL interfaces and corresponding object interactions of the CORBAbased implementation of the IRM-based scheme. IDL interfaces Tables 5, 6 and 7 define IDL interfaces of the three modules in the IRM-based scheme, which are abstracted from the interactions described above. Interactions among CORBA objects Figure 6 shows the interactions among CORBA objects of the IRM-based scheme. These objects correspond to the client-side and server-side objects of the interfaces defined in Tables 5, 6 and 7. The interactions can be summarized in the following six steps. (1) Setting up a new RTANS module RTANS f// Real-Time Application Name Server Struct RtAttributes f...;g; Struct RtItem f Monitor::ForManager monref; RtAttributes rtattr;g; Typedef sequence hrtitemi RtaList; Interface ForManager f void getrtalist (out RtaList rtalist);g; Interface ForMonitor f void register (in RtAttributes rtattr, in Monitor::ForManager monitorref);g; g; //End of RTANS module Table 5. Interfaces of RTANS module

9 MONITORING QoS DISTRIBUTION 83 module Monitor f// Monitor Interface ForRTANS f void register (in RTANS::ForMonitor ansref);g; Interface ForManager f void register (in Manager::For Monitor mngref); void setupdateinterval (in long interval);g; g; //End of Monitor module Table 6. Interfaces of monitor module module Manager f// Manager Struct Record f...;g; Typedef sequence hrecordi Records; Interface ForMonitor f void update TrafficInfo (in Records records);g; g; //End of Manager module Table 7. Interface of manager module (with analysis application) invokes the registration of its reference, of type RTANS::ForMonitor, to each monitor through the client object of Monitor::ForRTANS. Correspondingly, the implementation object of Monitor::For RTANS adds the reference to the RTANS list in the monitor. (2) Detecting a new real-time flow causes the registration of corresponding parameters from a monitor to all RTANSs that are in the monitor s RTANS list. The parameters include traffic attributes, of type RTANS::RtAttributes, and the reference of the implementation object of Monitor::ForManager. (3) This operation is done through the client object of RTANS::ForMonitor. Accordingly, the implementation object of the RTANS::ForMonitor adds the attributes and reference to its RTA list. (4) A network manager invokes the operation getrtalist to get the RTA list through the client and implementation objects of RTANS::ForManager. The manager selects one real-time flow from the RTA list to monitor. As in the RM-based scheme, the manager finds the references, of type Monitor::ForManager, of relevant monitors from the RTA list and starts to retrieve traffic information from them. This causes the generation of several groups of CORBA objects in the manager. Each group, responsible for communicating with one monitor, consists of a client object of Monitor::ForManager and an implementation object of Manager::ForMonitor. (5) After these steps, the manager registers the reference of the implementation object of Manager::ForMonitor with the corresponding monitor. This is achieved through the client object of Monitor::ForManager.Theimplementation object of Monitor::ForManager in the monitor adds the manager s reference to its manager list. In addition, the interval for the monitor to report traffic information is set by the operation setupdateinterval. (6) Lastly, each monitor uses the references, of type Manager::ForMonitor, in its manager list to locate all managers and update traffic Figure 6. IRM-based scheme: Interactions among CORBA objects

10 84 CHEN-KHONG THAM ET AL. information to them periodically in the required interval. Implementation Details and Results Through the steps described above a network manager can locate and communicate with relevant monitors simultaneously, and the monitors can report traffic information to the manager directly. Hence, the manager can get traffic information of a real-time flow from different network segments. In the following sub-sections, the CORBA-based prototype implementations of the two schemes will be presented, followed by an explanation of the sample displays that can be obtained. Implementation Details In the previous section, we have introduced the IDL interfaces defined for the CORBA-based implementations of the RM-based and IRM-based schemes. Corresponding CORBA objects and their interactions were also described. Based on this, we have implemented two prototypes, one for the RM-based scheme and one for the IRM-based scheme. Derivation of usage parameters In our design, each monitor comprises of a meter and several CORBA objects shown in Figures 5 and 6. The meter is based on RTFM architecture 8 which measures flows in real-time. The meter can provide various usage information of a flow. 8 In our prototype, the measurements of a real-time flow include start timestamp and byte count, from which the QoS related parameters of the flow, such as throughput and loss rate can be easily derived. The flow start timestamp records the start time of a flow, which is the time when the meter detected the start packet of the flow. The start packet is the first packet sent by the flow sender. In our prototype, we set the sequence number of the start packet to 1. The sequence number increments by one for each data packet of the flow. If the start packet has not been dropped during the transmission, the start time is exactly the time when the first packet of the flow was detected at the measurement point. If the start packet has been lost before the measurement point, the meter needs to estimate the arrival time of the start packet, the start time, as if it had not been lost. To achieve this, we need to estimate the arrival interval between the lost start packet and the first detected packet. We use the next arrival interval between the first and second detected packets to estimate it based on the assumption that the network status is unlikely to change during the two intervals. Denote the start time of flow i by t i s, the timestamps of the first and second detected packets at the measurement point by t i and 1 ti, 2 and the sequence numbers of the first and second detected packets of the flow by s i and 1 si.wehave: 2 t i D s ti 1 s i 1 1 Ł t i 2 t i 1 / si 2 s i 1. Note that in this method only the first and the second detected packets are timestamped. The meter uses rolling counters to record subsequent traffic quantities on the same flow. The quantities are counted in number of packets and number of bytes. The byte count stores the number of bytes of the flow counted since its start time. With the start time and byte count, the following parameters, mean throughput in a certain interval, average throughput, and maximum throughput of the flow can be determined. Denote the byte count of flow i at time t by N i t, the time interval for the monitor to report traffic information by 1, the mean throughput of the flow in the jth interval from the start time t s by r i j, the average throughput of the flow since its start time by r i, and the maximum mean throughput of the flow since its start time by r i.wehave, max 8Ł N i N i r i ts j D icjł1 ts i C j 1 Ł1 1 r i D j kd1 ri k j r i max D max[r i k, k D 1,...,j] By consolidating traffic information from two monitors residing in different network segments, other QoS related parameters of the flow, such as loss rate between the two network segments, can also be derived. Denote by r i j n K the mean throughput r i j measured by monitor n K. Denote the mean loss rate in the jth interval of the flow between monitors n A and n B by d i j n Ajn B, the average loss rate of the flow between the two monitors by

11 MONITORING QoS DISTRIBUTION 85 d i n A jn B. Assuming the flow passes the segment where monitor n A is attached first, we have d i j n Ajn B D r i j n B r i j n A j d i kd1 n A jn B D ri k n B r i k n A j 4 5 Synchronization of displays In this section, we consider the synchronization problem of displaying traffic graphs, each of which displays traffic information reported by one monitor. In order that a network administrator can monitor the QoS distribution of a real-time flow through his/her manager application, one direct method is to display all relevant traffic graphs of the flow in the manager at the same time. To achieve this, there are two challenges. One is to synchronize information reporting in all relevant monitors of the flow, since different monitors usually start reporting traffic information at different times. Another is to synchronize the displaying of the traffic information from each monitor. Due to various network delays, the information of a part of the flow reported by one monitor arrives at the manager at a different time from those by other monitors. In our implementations, we mainly consider the synchronization of information reporting while leaving the synchronization of display to later. To synchronize information reporting, we adopt the following approach. Denote by t i r n K the time a relevant monitor n K has received the manager s reference, described above, and is ready to report traffic information of the flow i. Clearly, there is an integer L such that t i s C L 1 Ł 1 < t i n r K < t i sclł1. The monitor starts reporting at time t i s C L C 1 Ł 1 and repeats it in the interval 1. The information reported is the byte count. Using this way, each monitor reports traffic information in the same interval starting from its own measured start time. Ideally, if the flow is sent constantly by the sender and there is no delay variation in the transmission, the information reporting is synchronized in the time scale. Each traffic graph displays traffic information reported by one relevant monitor, which has been selected to retrieve information from. The graph is updated once new information is reported from the corresponding monitor and processed by the manager. If the delay variation between two information reportings of the monitor and the variation of processing times for the two reportings are much smaller than the reporting interval 1, the graph is updated in the same interval. Besides, if the transit delays of the flow are also much smaller than the reporting interval, all traffic graphs of the flow are synchronized. The information displayed includes the mean throughput in one reporting interval, the average throughput since the reporting was started, and the maximum mean throughput, which are calculated from equations (1), (2) and (3) respectively. Figure 7. Testbed for RM-based scheme

12 86 CHEN-KHONG THAM ET AL. Sample Displays RM-based scheme The network topology of our testbed for the implementation of the RMbased scheme is shown in Figure 7. In this testbed, there are three segments: the ATM network, the Gateway and the Ethernet network. A test flow is sent from the Sender in the ATM network to the Receiver in the Ethernet network through the Gateway. In the testbed, there are three monitors, Monitor 1, 2 and 3, which are located in the Sender, the Gateway and the Receiver respectively. The monitored traffic graphs of the test flow are displayed in Figure 8. In our test, the interval for a monitor to report metered information was set to 2 seconds. Each graph in Figure 8 gives the following information of the flow: the mean throughput of each reporting interval, the average throughput since the starting of the monitoring at the corresponding measurement point, and the maximum mean throughput. These parameters are calculated according to equations (1), (2) and (3). The graph is updated in the reporting interval in accordance with the method described in the last sub-section. From the traffic graphs shown in Figure 8, we can immediately estimate the average loss rate between any two monitors. IRM-based scheme Figure 9 shows the testbed for the implementation of the IRM-based scheme, which is similar to that in Figure 7. There is a difference between the two testbeds: Figure 9 testbed has two management domains, Domain 1 and Domain 2, while Figure 7 testbed has one single domain. A test flow is sent using the same path of the test flow for the RM-based implementation. Figures 10 and 11 show traffic graphs of the flow displayed in the two managers: Manager 1 and Manager 2 that belong to Domain 1 and Domain 2 respectively. The two managers started monitoring the flow at different times. Figure 10 also shows that there is obvious QoS degradation between Monitor 1 (Sender) and Monitor 3 (Receiver). From Figure 11, the cause of such degradation can be further investigated and is found to occur in the network segment between Monitor 1 and Monitor 2. Although the degradation can also be detected using the monitoring mechanisms surveyed in the second section, they are not able to Figure 8. Sample displays (a) Monitor 1, (b) Monitor 2, (c) Monitor 3 further locate the source of the degradation since they only monitor end-to-end QoS. Discussion In our test, the initial packet sequence number of the test flow was set to 1 and we adopted the method introduced above to derive the start time. However, if the initial value of the packet sequence number is random as recommended by RTP, 13 that method can no longer be used. The

13 MONITORING QoS DISTRIBUTION 87 Figure 9. Testbed for IRM-based scheme Figure 10. Domain 1 sample displays (a) Monitor 1, (b) Monitor 3 randomness of the initial sequence number makes it impossible for a meter to determine whether the first detected packet is the start packet and how many packets are there between the first detected packet and the start packet. Thus, the start time of the flow cannot be derived. In order to synchronize the traffic information reporting and overcome the uncertainty of the start packet, a so called virtual start time is adopted instead of the start time. The following method can be adopted to calculate the virtual start time of the flow. Consider that real-time flow i is measured by several monitors n K, K 2 A, B, C,....Theleast sequence number of the first detected packets of the flow by these monitors is s i D min min[si n I K, K 2 A, B, C,... ], where s i n I K denotes the sequence number of the first detected packet of the flow by monitor n K. Then, the virtual start time of the flow in monitor n K, denoted by vt i n s K, is the time when the packet with the least sequence number arrived at the measurement point if it had not been lost and it can be determined as: vt i n s K D t i I s i I s i Ł min ti 2 t i 1 / si 2 s i 1. With the virtual start time, the reporting of traffic information from different monitors is synchronized from vt i n s K rather than from t i n s K. In our test, the reporting interval was selected to be 2 seconds. Although a smaller interval gives more detailed information, however, it comes at a certain cost. One reason is that the smaller the reporting interval, the larger the management traffic. Another reason is that if the reporting interval is smaller than the delay for reporting the metered information from a monitor to the

14 88 CHEN-KHONG THAM ET AL. Figure 11. Domain 2 sample displays (a) Monitor 1, (b) Monitor 2, (c) Monitor 3 manager, some metered information will be lost and thus the traffic graph from this monitor is wrong and is not synchronized with other graphs any more. To overcome this problem, the simplest way is to increase the reporting interval and restart the retrieval of traffic information from all relevant monitors. Otherwise, additional mechanisms, which require further study, should be provided. We have considered the synchronization problem of information reporting. However, even if the reporting of one monitor is synchronized with another one, the times when the information from the two monitors arrived at the manager are still different due to various network delays. Two principal delays contribute to the difference. One is the transit delay for the flow to transfer from one measurement point to another measurement point. The other is the transmission delay for each monitor to report its monitored information. Due to the uncertainty of these delays, it is difficult to synchronize the traffic graphs in the manager. One possible method is to include the number of reporting time intervals in the reported information, e.g. L C 1 for the first reported information described above. The manager uses a buffer to store reported information from all relevant monitors. It monitors the buffer until all jth information have been received. After this, the manager updates all traffic graphs, removes the jth information from the buffer and waits for the next j C 1 th information. The advantage of this method is that all traffic graphs are updated at the same time and the information displayed are synchronized. Its disadvantage is that if one piece of reported information has experienced a large network delay or is lost, all traffic graphs will not be displayed properly. In contrast, under this condition, only the traffic graph corresponding to the lost information is influenced when the method introduced above is adopted. Among the four critical QoS criteria, throughput, delay, delay variation and loss rate, we have proposed several methods for calculating the throughputs and loss rates of a flow. The transit delay and delay variation between two network segments of the flow can also be derived if all packets are timestamped. Denote the timestamp of the mth packet of flow i in monitor n K, K 2 A, B, C,... by t i m n K, the transit delay of this packet between monitors n A and n B by i m n Ajn B, the variation between i m n Ajn B and i mca n Ajn B by i mjmca n Ajn B where a 2 1, 2, 3,... is a pre-defined integer. We have i m n Ajn B D t i m n A t i m n B υ n A jn B 6 i mjmca n Ajn B D i mca n Ajn B i m n Ajn B D t i mca n A t i m n A t i mca n B 7 t i m n B a 2 1, 2, 3,... where υ n A jn B is the time difference between the two monitors n A and n B.

15 MONITORING QoS DISTRIBUTION 89 From equation (6), we can conclude that υ n A jn B must be provided to determine the transit delay i m n Ajn B. In contrast, equation (7) shows that the delay variation i mjmca n Ajn B can be derived without any knowledge of υ n A jn B. However, the requirement of timestamping every packet and the assumption that no packet is lost apply to both the calculation of the transit delay and the calculation of the delay variation. Note that in our methods for determining throughputs and loss rates of a flow, only the first two detected packets of the flow in each monitor are timestamped. Practically, timestamping every packet may cause a monitor to exhaust its processing ability and packet loss is often unavoidable. Hence, for monitoring delay and delay variation, further investigation is required. Comparison Table 8 presents a comparison of current monitoring mechanisms surveyed earlier and the two new QoS distribution monitoring schemes described in this paper. Except for SNMP 5 the others provide mechanisms for QoS monitoring from different perspectives; SNMP 5 can only provide network-layer monitoring and is thus not suitable for QoS monitoring. We can view other mechanisms at three levels: measurement level, QoS monitoring level, and higher analysis level. The measurement level, including RMON-2, 7 and RTFM, 8,9 is capable of performing application-level monitoring and providing monitored information to the QoS monitoring level. The QoS monitoring level comprises of the RTCP-based scheme, the RM-based and IRM-based schemes and the scheme proposed by Chen et al. 12 Except for Chen et al., schemes at this level are responsible for finding relevant monitors for each real-time flow usually arising from multimedia applications. They support end-to-end QoS monitoring as well as QoS distribution monitoring except for Chen et al. 12 and the RTCP-based scheme. In addition, they can also detect possible QoS degradation. The higher analysis level uses the results generated by the QoS monitoring level to perform further analysis or operations, such as identifying QoS problems in Mourelatou et al. 10 and dynamically adjusting the monitoring system in Ehab Al-Shaer. 11 Generally, although the measurement level does not provide QoS monitoring directly, it collects all necessary traffic information for the upper levels. The QoS monitoring level uses the information from the measurement level to find the route of each monitored real-time flow, locates relevant monitors metering the flow, and traces the ongoing QoS and its distribution. The higher analysis level provides further analysis or operations based on information from the QoS monitoring level. P roviding sustained QoS to multimedia applications requires QoS monitoring. Conclusion Providing sustained QoS to multimedia applications requires QoS monitoring. In this paper, we Monitoring NLM ALM Route finding EtE QM QoS-DM Multi-NMD mechanism SNMP [5] Supported Not Supported Supported RMON-2 [7] Supported Supported Supported RTFM [8, 9] Supported Supported Supported RTCP-based [13] Required Supported Not Supported Supported RM-based Required Supported Supported Supported Not Supported IRM-based Required Supported Supported Supported Supported Chen et al. [12] Required Supported Not Supported Mourelatou et al. [10] Required Required Ehab Al-Shaer [11] Required Required Required -: not applicable/not mentioned, NLM: network-layer monitoring, ALM: application-level monitoring EtE QM: end-to-end QoS monitoring, DM: distribution monitoring, NMD: network management domain Table 8. Comparison of monitoring mechanisms

16 90 CHEN-KHONG THAM ET AL. have argued that end-to-end QoS monitoring is not sufficient for the management of multimedia networks and QoS distribution monitoring is required. However, few current QoS monitoring mechanisms can provide QoS distribution monitoring. To meet this challenge, we have proposed two schemes, the RM-based scheme and the IRM-based scheme. With these schemes, when monitoring a real-time flow, a network manager, which can be mobile, is able to locate relevant monitors that are metering the flow and retrieve traffic information from them simultaneously. By consolidating the information from these monitors that are embedded in different network segments, not only can end-to-end QoS be monitored, but the QoS distribution in different network segments for this flow is also derived. Based on these schemes, the network manager is able to locate the cause of possible QoS degradation and thus control the network accordingly. In addition, with the IRM-based scheme, QoS distribution monitoring can be performed in multiple network management domains. Moreover, we have also presented CORBA-based implementations of these two schemes to show their feasibility. References 1. Pacific G, Stadler R. An architecture for performance management of multimedia networks. Proceedings of IFIP/IEEE International Symposium on Integrated Network Management, Ferrari D. Multimedia network protocols: where are we? Multimedia Systems 1996; 4: Aras CM, Kurose JF, Reeves DS, Schulzrinne H. Realtime communications in packet-switched networks. Proceedings of the IEEE 1994; 82: Aurrecoechea C, Campbell AT, Hauw L. A survey of QoS architectures. Multimedia Systems 1998; 6: Case J, Fedor M, Schoffstall M, Dåvin J. A simple network management protocol (SNMP). IETF RFC Waldbusser S. Remote network monitoring management information base. IETF RFC Waldbusser S. Remote network monitoring management information base version 2 using SMIv2. IETF RFC Brownlee N, Mills O, Ruth G. Traffic flow measurement: Architecture. IETF RFC Brownlee N. Traffic flow measurement: Experiences with NeTraMet. IETF RFC Mourelatou KE, Bouloutas AT, Anagnostou ME. An Approach to identifying QoS problems. Computer Communications 1994; 17: Al-Shaer E. Hierarchical filtering-based monitoring architecture for large-scale distributed systems, PhD Dissertation, Computer Science Department, Old Dominion University, July Chen TM, Liu SS, Procanik MJ, Wang DC, Casey DD. INQIRE: A software approach to monitoring QoS in ATM networks. IEEE Network 1998; Schulzrinne H, Casner S, Frederick R, Jacobson V. RTP: A transport protocol for real-time applications. IETF RFC Object Management Group. The Common Object Request Broker: Architecture and Specification v Lazar AA, Bhonsle SK, Lim KS. A binding architecture for multimedia networks. Journal of Parallel & Distributed Systems, 1995; Mazumdar S, Swanson K. Web based management CORBA/SNMP gateway approach. 7th IFIP/IEEE International Workshop on Distributed Systems: Operations & Management (DSOM) Deri L, Ban B. Static vs. dynamic CMIP/SNMP network management using CORBA. IBM Research Report, IBM Zurich Research Laboratory Chen G, Neville M, Kong G. Distributed network management using CORBA/TMN. 7th IFIP/IEEE DSOM Maffeio S, Schmidt DC. Constructing reliable distributed communication systems with CORBA. IEEE Communications Magazine, 1997; Pavon J, Tomas J, Bardout Y, Hauw H. CORBA for network and service management in the TINA framework. IEEE Communications Magazine, 1998; Jiang Y, Tham CK, Ko CC. A Web-based realtime traffic monitoring scheme using CORBA. 2nd IFIP/IEEE International Conference on Management of Multimedia Networks and Services, November If you wish to order reprints for this or any other articles in the International Journal of Network Management, please see the Special Reprint instructions inside the front cover.