Dimitrios P. Pezaros Department of Computer Science University of Glasgow Glasgow, UK David Hutchison

Transcription

1 On the Characterization of Network Traffic Dynamics Angelos K. Marnerides Infolab Computing Department Lancaster University Lancaster, UK ABSTRACT Characterizing backbone networks poses a significant challenge due to the unstable and fluctuated behavior exhibited by network traffic dynamics. Modeling techniques developed for volumebased traffic profiling rely on the statistical assumptions of stationarity, Gaussianity and linearity, whose applicability has not been thoroughly investigated throughout past and recent work. We argue that modeling assumptions should be rigorously validated since they determine the accuracy of any model applied to describe the traffic process. In this work we introduce and illustrate the suitability of Time-Frequency TF) representations and the Hinich algorithms for the validation of modeling assumptions on captured backbone and edge link network traces. Through the employment of these algorithms, we statistically show that link traffic is extremely dynamic and model characteristics change in small timescales. Furthermore, we highlight the benefits of a transport-layer traffic decomposition approach where protocols are modeled independently and protocol-specific characteristics are revealed, as opposed to analysis based solely on volume aggregates.. INTRODUCTION Understanding the underlying traffic behavior of backbone and edge networks is vital for traffic engineering tasks such as anomaly diagnosis, traffic classification and capacity planning []. In particular, traffic characterization is typically being tackled using statistical analysis of individual links) and network-wide volume-based characteristics i.e. counts of bytes and packets) [] [], as well as by analyzing the distributional behavior of particular packet header fields [3][4]. Throughout past and current literature, numerous statistical and signal processing techniques have been employed for constructing traffic models. Solutions as in [5][6][7] referring either to a network-wide, single PoP or link volume traffic characterization have employed mathematical models based on the de facto statistical assumptions of stationarity, gaussianity and linearity. The majority assume these properties with no validation, whereas others, as in [][5], use nd order statistics i.e. mean, variance, autocorrelation sequence) to validate these assumptions. However, nd order statistics are problematic in validating such timeseries properties, since they suppress up to a large scale phase Dimitrios P. Pezaros Department of Computer Science University of Glasgow Glasgow, UK dp@dcs.gla.ac.uk David Hutchison Infolab Computing Department Lancaster University Lancaster, UK dh@comp.lancs.ac.uk Hyun-chul Kim Department of Computer Science & Engineering Seoul National University Seoul, South Korea hkim@mmlab.snu.ac.kr characteristics such as magnitude and they stand unable to capture phase transition peaks [8][9]. Hence, any subsequent modeling leads to questionable levels of accuracy which in the case of backbone IP networks results in the inaccurate interpretation of traffic dynamics. In this paper, we focus on the rigorous validation of such statistical properties before they are employed by traffic models. We also show that methodologies derived by nd order statistics which address traffic profiling impose inadequacies in respect to the exact detection of traffic fluctuations throughout the observational time frame. In order to overcome these constraints, we propose the use of the bispectrum which as a tool for examining a timeseries' 3rd order statistical properties. The bispectrum is capable of capturing a signal's i.e. timeseries') phase information and can therefore enable the accurate representation of traffic fluctuations. In general, our work contributes towards the mathematical interpretation of volume-wise traffic dynamics using Time- Frequency TF) representations and 3rd order statistics estimated by the Hinich algorithms and the bispectrum[0][]. The combination of our theoretical and experimental evaluation of sample datasets from monitored backbone and edge links shows that: In contrast to schemes as in [][] where aggregate volume is considered, traffic decomposition enables a detailed characterization of the traffic dynamics shown by each protocol and enables the extraction of hidden patterns. The applicability of any traffic profiling model depends on the underlying statistical assumptions of stationarity, linearity and Gaussianity which cannot be assumed to hold true for the entirety of a traffic trace; hence such assumptions should be rigorously verified before applying any scheme. Failure to fully validate them results in modeling inaccuracies. Link traffic exhibits significant fluctuations in short timescales < 30 mins.) which may be described by diverse rather than holistic models, not adhering to identical statistical assumptions. Byte counts mostly expose different modeling characteristics than packet counts; hence, a volumebased approach should consider them independently. This paper has been submitted to the ACM X Conference and due to the double blind review the name of the conference is not stated. In addition results are partially and not fully presented.

2 Set Date Day Start Duration Link Type Packets Bytes Avg. Util. Flows/min WIDE Fri :45 55min Backbone 3M 4G 35Mbps 63K Keio-I Tue 9:43 30min Edge 7M 6G 75Mbps 3K Keio-II Thu 0:8 30min Edge 5M 6G 75Mbps 9K. DATA & FEATURES Our analysis is based on unidirectional traffic flows extracted from three anonymized payload packet traces collected in US and Japan, as shown above in Table. The WIDE trace was collected at a 00 Mb/s FastEthernet US-JAPAN Trans-Pacific backbone link which is a carrier of commodity traffic for WIDE member organisations[3]. The Keio traces were captured on a Gb/s Ethernet link from the Japanese academic network of the Keio University s Shonan-Fujisawa campus. Our analysis is based on decomposing transport traffic into bytes and packet counts for TCP, UDP and ICMP protocols although ICMP is considered as a network layer protocol, we treat it as transport here, since it carries traffic which is distinct from TCP and UDP). Table : Traces pre-processing. Set Duration TCP UDP WIDE-I 3.75min 4K 30K 4K WIDE-II 3.75min 8K 3K 4K WIDE-III 3.75min 4K 9K 3K WIDE-IV 3.75min 3K 30K 4K Keio-I 30min K 0K 6K Keio-II 30min 8K 9K 4K ICMP and F a v) the Fourier transform of t) frequency is defined as : f t) = d arg G a t) π dt and its group delay is : t G d arg Fa ν ) ν ) = π dν G a the instantaneous In practice ft) denoted the amplitude of frequency we observe in a single count of a packet/byte arrival in time t and t ) is the G ν time distortion occurred due to the signal s instantaneous frequency. However, our experimentation shown Fig. ) that transport layer traffic within all our traces is extremely highly non-stationary since there was never the case of constant behaviour of the two concepts we introduce in formulas ) and ). ) ) 3. METHODOLOGY & RESULTS One of our targets is to illustrate the importance of validating the de-facto assumptions of stationarity, Gaussianity and linearity for volume-based network traffic modeling and further introduce applicable methodologies that accurately achieve this. The theoretical principles we apply on our packets and bytes timeseries originate from statistical signal processing, and following sections highlight their basic concepts. We first introduce the main metrics employed in order to identify stationary characteristics. Then, we discuss the basic principles of bispectrum and bicoherence as the core elements of the Hinich algorithms which enable statistical tests for validating Gaussianity and linearity[0]. 3. Stationarity Test A signal is considered to be stationary if the elements in its analytical form keep a constant instantaneous frequency and group delay respectively. If we map the counts of bytes and packets for each unidirectional flow as the process gt), t) its analytical form after applying a Hilbert transformation G a Figure : Stationarity Analysis on Keio-I 3. Hinich Algorithms Firstly introduced in [0], Hinich algorithms are a set of statistical hypothesis tests to detect non-linear and further Gaussian or non- Gaussian characteristics on a given random stochastic process g t). As mentioned above, the process gt) in our case is

3 Table :Estimated Statistics for the KEIO trace. ZSP denotes the Zero Skewness Probability and Conclusions defined as: L=Linear, NL = Non-Linear, G = Gaussian, NG = Non-Gaussian. Protocol Feature Estimated Ψ ω, ω) Theoretical Υ ω, ω ) Gaussianity metric κ ) Noncentrality parameter η ) ZSP θ ) Conclusion Keio-I Keio- II Bytes L & NG TCP Packets L & NG Bytes NL & NG UDP Packets NL & NG Bytes NL & NG ICMP Packets NL & NG Overall Protocol Bytes L & NG Packets L & NG Feature Estimated Ψ ω, ω) Theoretical Υ ω, ω ) Gaussianity metric κ ) Noncentrality parameter η ) ZSP θ ) Conclusion Bytes NL & NG TCP Packets L & NG Bytes NL & NG UDP Packets NL & NG Bytes L & NG ICMP Packets L & NG Overall Bytes NL & NG Packets L & NG the count of packets and bytes monitored in discrete time bins of length n, denoted as T = {,,... n }. Since the core elements for the Hinich algorithms are those of the bispectrum and bicoherence we following provide their mathematical definitions. The bispectrum provides a dual-frequency representation on the time-frequency plane in contrast to the one-dimensional interpretation of the power spectrum, and considers the 3 rd order cumulant sequence c 3, ) as its basic tuning function: + + j ω + ω ) B ω, ω ) = c3, ) e 3) = t = Bicoherence is used to detect quadratic non-linearities on the time-frequency TF) plane as well as quadratic phase coupling. Nevertheless, this document does not intend to present the properties of bicoherence in detail and therefore we only provide its definition, which in practice is denoted as the squared normalized version of the bispectrum. Hence, if we use formula 3, its squared normalized version looks like: B ω, ω ) b k B = 4) ω, ω ) S ω + ω ) S ω) S ω ) 3.. Gaussianity & Linearity Tests The Gaussianity test focuses on the bicoherence value as the result of the normalized bispectrum estimate. The Gaussian also known as zero-skewness) assumption according to Hinich is considered E b k B ω, ω ) the case where the estimated bicoherence value { } as well as the skewness equal to zero. Since in reality we can never have a flat, zero-bicoherence due to noise), we take the mean bicoherence value which in practice represents a quantitative Gaussianity metric. By using the definition of formula 4) we can calculate the mean bicoherence power estimate κ with: κ = b k 3) B ω, ω ) [4][0] and [5] employ some comparisons of the κ value in order to establish a concrete conclusion on whether the dataset under test actually follows a Gaussian distribution or not. As they suggest the computed value of κ is χ distributed chi-squared distributed) with a Fast Fourier Transform FFT) length function for approximating the number of freedom degrees to the closest Gaussian fit. Therefore, in case the estimated κ indicates that our timeseries has less than or equal to 0 freedom degrees, then we conclude that the dataset was following a Gaussian distribution. In parallel with κ, the Zero Skewness Probability ZSP) of false alarm θ is computed. θ illustrates the probability of the newly approximated value being much larger than the initial κ

4 estimate. Whenθ is small, we can reject the zero-skewness i.e. Gaussian) hypothesis. In our tests we use 0. θ as a valid range for approving Gaussianity, based on findings from [0]and [5]. The linearity assumption holds in case where the bispectrum: B ω, ω) 0 = C, ω,ω 4) where C is a constant value for all ω and ω. As suggested by Swami [], it is essential to approximate values of a sample interquartile range under a new bispectral estimate Ψ ω, ω) derived by B ω, ω) : Ψ ω, ω ) = B ω, ) 5) ω ρ M where Μ is the resulting boxcar window length, after rounding the FFT length with a resolution parameter ρ. The intuition for calculating the interquartile range is to compare it with a theoretical interquartile range Υ ω, ω) which similarly with ω, ω) defined as: Ψ is chi-squared with a non-centrality parameter η ρ η = M ) c, ) 6) 3 In our experimentation we keep the values of an FFT length of 8 and a resolution parameter of 0.5. The main step is to compare Ψ ω, ω) with Υ ω, ω) and if the difference is quite high based on the limits provided by [0] and [], then we reject the linearity hypothesis. The Keio characterization as presented in Table 3 has produced some significant outcomes related to the statistical behavior of each protocol on byte and packet representations. It is observed that during the same day and in such a small temporal interval i.e. 30mins), TCP as well as ICMP may not be modeled under the same assumptions. In general, it is also obvious that not all protocols may be modeled in the same way therefore their independent analysis would be beneficial as we also presented previously in section 4... Throughout the Keio-I, only ICMP and UDP can be modeled in the same way, whereas TCP should be treated under linear but non-gaussian assumptions. All the three protocols for both packets and bytes analyses have a zero probability for exhibiting zero skewness. This zero probability is the main strong evidence for concluding that the data do not exhibit a Gaussian behavior. In addition, the linearity exposed by TCP is attributed to the small numerical difference <0) between the theoretical and the estimated interquartile range. Similarly, we conclude that nonlinearity exists in both UDP and ICMP since the difference between the interquartile ranges gets greater than 0. Similarly, evaluation on the Keio-II compliments the results from the Keio-I indicating again the inability of modeling all protocols together. In addition, it is evident that even a volume-based approach for modeling a single protocol should consider the counts of bytes and packets as independent and analyze them separately. 4. CONCLUSIONS Current trends in network traffic profiling include the examination of aggregate volume characteristics, and the employment of models that inherently assume stationarity, linearity and Gaussianity of the timeseries. In this paper, we have argued that such statistical assumptions do not hold universally, and that they should therefore be rigorously validated. We have proposed the use of Time-Frequency TF) representations for determining stationarity of a timeseries, and the Hinich algorithms for validating its linearity and Gaussianity. Our quantitative evaluation on real network traces challenges the existence of stationarity and Gaussianity in small timescales, while linearity is evident in some traces but not in others. REFERENCES [] Papagiannaki, K., Taft, N., Zhang, Z., Diot, C., 003 Long- Term Forecasting of Internet Back-bone Traffic: Observations and Initial Models. In IEEE INFOCOM, San Francisco, U.S.A., March 003 [] Soule, A., Salamatian, K., Taft, N., Combining filtering and statistical methods for anomaly detection, Proc. of the 5th ACM SIGCOMM on Internet Measurement. Berkeley, CA, USA: USENIX Association, 005, pp. 3 3 [3] Lakhina, A., Crovella, M., Diot, C., Mining Anomalies Using Traffic Feature Distributions, 005, in ACM SIGCOMM Computer Communication Review CCR), vol. 35, no. 4, 005 pp [4] Silveira, F., Diot, C., Taft, N., Govindan, R., Astute: detecting a different class of traffic anomalies, in Proc. of the ACM SIGCOMM, New Delhi, India, Aug 9 Sep., 00. [5] Medina, A., Taft, N., Salamarian, K., Bhattacharrya, S., Diot, C., Traffic Matrix Estimation: Existing Techniques and New Directions, in ACM SIGCOMM Conference, Pittsburgh, PA, USA, August 00. [6] Soule, A., Lakhina, A., Taft, N., Papagiannaki, K., Salamatian, K., Nucci, A., Crovella, M., Diot, C., Traffic Matrices: Balancing Measurements, Inference and Modeling, 005, In Proc. of ACM SIGMETRICS 005, Banff, Alberta, Canada [7] Taft, N., Bhattacharrya, S., Jetcheva, J., Diot, C., Understanding Traffic Dynamics at a Backbone PoP, In Proceedings of SPIE, Vol. 456, 50, 00 [8] Nikias, C.L. and J.M. Mendel, Signal processing with higher-order spectra, IEEE Signal Processing Magazine, Vol. 0, No 3, pp. 0-37, July 993Bowman, M., Debray, S. K., and Peterson, L. L Reasoning about naming systems. ACM Trans. Program. Lang. Syst. 5, 5 Nov. 993), DOI= [9] Nikias, C.L. and M.R. Raghuveer, Bispectrum estimation: A digital signal processing framework, Proc. IEEE, Vol. 75, pp , July 987.

5 [0] Hinich, M.J., Testing for Gaussianity and linearity of a stationary time series, J. Time Series Analysis, Vol. 3, pp , 98. [] Swami, A., Mendel, J., M., Nikias, C., L., Higher Order Spectral Analysis Toolbox User Guide, MathWorks Inc., January 998 [] Groschwitz, N., K., Polyzos, G., C., A Time Series Model of Long-Term NSFNET Backbone Traffic, in IEEE ICC 94, 994 [3] The WIDE MAWI working group: [4] Hasselman, K., Munk, W., MacDonald, G., Bispectra of Ocean Waves, in Time Series, Ed. New York, Wiley, 963 [5] Nikias, C, L., Petropulu, A, P., Higher Order Spectral Analysis: A Non-linear Signal Processing Framework, New Jersey, Prentice Hall, 993