Fortschritt-Berichte VDI Reihe 10 Informatik/ Kommunikation Dipl.-lnform. Helmut Gogl, Munchen Nr. 670 Measurement and Characterization of Traffic Streams in High-Speed Wide Area Networks VDI Verlag
V Contents 1 Introduction 1 1.1 The Traffic Engineering Tasks 1 1.2 Previous Work in Measurement and Characterization of Network Traffic.. 2 1.3 This Work in Comparison to Previous Research 8 1.4 Thesis Outline 9 2 The Asynchronous Transfer Mode 12 2.1 Basics of the Asynchronous Transfer Mode (ATM) 12 2.1.1 Cell Transport and Statistical Multiplexing 12 2.1.2 ATM Cell Format 13 2.1.3 ATM Virtual Connections 15 2.2 The ATM Protocol Layers 16 2.2.1 Protocol Reference Model 16 2.2.2 Physical Layer (PL) 17 2.2.3 ATM Layer 18 2.2.4 ATM Adaption Layer (AAL) 19 2.3 IP over ATM 21 i Measurement Techniques for High-Speed Networks 23 3.1 Network Performance Aspects.-. 23 3.2 Performance Analysis vs. Traffic Characterization 24 3.3 Network Traffic Measurement Techniques 25 3.3.1 Classification of Network Monitors 25 3.3.2 Software Monitor 26 3.3.3 Hardware Monitor 27 3.3.4 Software vs. Hardware Monitors 27 3.4 Evaluation of Hardware Monitoring Approaches 30 3.4.1 Basic Architecture of a Hardware Monitor 31 3.4.2 Classification and Evaluation of Hardware Monitor Architectures. 32 3.4.2.1 Simple PC-based Approach 32 3.4.2.2 PC-based Approach with Supporting Hardware 34
VI Contents 3.4.2.3 Dedicated Hardware Approach 36 3.4.3 Concluding Remarks 38 3.5 Concept(s) for Future High-Speed Network Monitoring 39 3.5.1 Motivating Parallel Processing.. 39 3.5.2 Parallel Processing Strategies 41 3.5.3 Motivating Data Reduction 43 3.5.4 Data Reduction Strategies 44 3.5.4.1 Simple Slicing Techniques 45 3.5.4.2 Improved Slicing Techniques 46 3.5.4.3 Filter Techniques 46 3.5.4.4 Online Traffic Analysis 49 3.5.5 Supporting Long-Term Measurements 51 3.5.6 Interconnection Alternative 52 3.5.7 Concept Summary 54 3.6 Network Monitoring Alternative 56 3.6.1 Distributed Network Monitoring Principle 56 3.6.2 Limitations/Problems of Distributed Network Monitoring 57 3.6.2.1 Type of Data Collection 57 3.6.2.2 Flow-Based Traffic View 57 3.6.2.3 Traffic Data Reporting 58 3.6.3 Distributed Monitoring vs. Hardware Monitoring 59 4 Environment for Traffic Analysis (ETA) 61 4.1 Network Traffic Analysis Principle 61 4.2 Online vs. Offline Processing 62 4.3 Problems of Data Post-Processing and Data Analysis '. 63 4.4 Purpose and Intention of ETA 64 4.5 Basic Concept of ETA 65 4.6 Architecture and Realization of ETA 67 4.6.1 The Data Collection Layer 67 4.6.2 The Data Post-Processing Layer 69 4.6.3 The ETA Post-Processing Core 71 4.6.3.1 The Database Management System 72 4.6.3.2 The Structure of the Database 73 4.6.3.3 The Import Filter 75 4.6.3.4 The User Interface 75
Contents VII 4.6.4 The Data Analysis/Visualization Layer 81 4.7 ETA compared to related Systems 82 4.7.1 MonETEN 83 4.7.2 JAM 83 4.7.3 NMAS 84 4.7.4 SNUFFLE 84 4.7.5 Comparison with ETA 85 4.8 Future Extensions of ETA 87 5 Mathematical Background for Modern Network Traffic Analysis 89 5.1 Stochastic Processes and Network Traffic 89 5.2 Definition of Self-Similar Processes 91 5.3 Properties of Self-Similar Processes 92 5.4 Methods for Detection of Self-Similarity 97 5.4.1 Aggregated Variance Plot 97 5.4.2 Index of Dispersion for Counts (IDC) 99 5.4.3 Periodogram Plot 100 5.4.4 Local Whittle Estimator 101 5.4.5 R/S Analysis 102 5.4.6 Wavelet Based Hurst Estimation 103 5.5 Network Traffic, (Heavy-tailed) Distributions, and Self-Similarity 106 5.6 Methods for Estimation of Distributions 107 5.6.1 Comparing Distributions using QQ-Plots 107 5.6.2 Assessing Distributions Tail Behavior 109 5.7 Remarks on (Non-)Stationarity and Time Series Analysis 114 5.7.1 The Problem of (Non-)Stationarity 114 5.7.2 Time Series Analysis (TSA) 114 6 Measurement of WAN Traffic 117 6.1 Network Configuration and Hardware Equipment 117 6.1.1 The German Broadband Research Network 117 6.1.2 Network Access Point at the B-WiN Entrance 118 6.1.3 Traffic Types and Protocol Stacks at the Network Access Point.. 119 6.1.4 Hardware used for Measurements 121 6.1.5 Connecting the Measuring Device 123 6.2 Description of the Measurements 125
[II Contents 6.2.1 Overview of the Measurement Types 125 6.2.2 Data Analysis Potentials 128 Characterization of WAN Data Traffic 131 7.1 Critical Review of the "Bellcore Data" 131 7.2 Mixing Ratio of Applications/Protocols 135 7.3 Medium Time Scale Characterization at the ATM Layer 138 7.3.1 Overall Traffic Profiles 138 7.3.1.1 Profiles of Aggregate Traffic 138 7.3.1.2 Profiles of Selected Applications 144 7.3.2 Characteristics of the Counting/Workload Process 147 7.3.2.1 Distribution of Cell Counts 147 7.3.2.2 Testing for Stationarity 152 7.3.2.3 Correlation in Cell Counts 157 7.3.2.4 Degree of LRD/Self-Similarity 159 7.3.2.5 Interpretation of H in Practice 162 7.4 Small Time Scale Characterization at ATM and Packet Layer 166 7.4.1 Measurements and General Traffic Characteristics 166 7.4.2 Characteristics of the Inter-Arrival Process 169 7.4.2.1 Distribution of Cell IATs 169 7.4.2.2 Distribution of IP Packet IATs 173 7.4.2.3 Distribution of IP Packet Sizes 175 7.4.2.4 LRD in Cell/Packet IATs 177 7.4.3 Characteristics of the Counting/Workload Process 180 7.4.3.1 Distribution of Cell/Byte Counts 181 7.4.3.2 Distribution of IP Packet Counts 183 7.4.3.3 LRD in Cell/Packet Counts 185 7.4.4 Analysis of Cell/Packet Bursts 188 7.4.4.1 Principles of the Applied Burst Analysis 188 7.4.4.2 Investigation of On- and Off-Periods 190 7.5 Small Time Scale Characterization at the TCP Layer 192 7.5.1 The Connection Stream Measurements 192 7.5.2 Characteristics of the Connection Inter-Arrival Process 193 7.5.2.1 Distribution of Connection IATs 193 7.5.2.2 Distribution of Connection Counts 195 7.5.2.3 Distribution of Connection Sizes 197
Contents IX 7.5.2.4 LRD in Connection IATs/Counts 200 7.6 Summary 204 8 Characterization of WAN Multimedia Traffic 208 8.1 Related Work 208 8.2 (Video-)Telephony over IP 210 8.2.1 Principles behind IP Telephony 210 8.2.2 Technical Realization of IP Telephony 211 8.2.3 The Connection Establishment Problem 212 8.2.4 Extended Measurement Environment 213 8.2.5 The considered Audio/Video Codecs 214 8.2.6 Analysis of IP Video-Phone Traffic 215 8.2.6.1 The IPHONE Measurements 215 8.2.6.2 General IPHONE Traffic Characteristics 217 8.2.6.3 Correlation Structure of IPHONE Traffic 221 8.2.6.4 Cell Count/IAT Distribution of IPHONE Traffic 225 8.3 Television Broadcast over IP using the MBONE 229 8.3.1 Purpose and Concept of MBONE 229 8.3.2 The Experimental Setup 229 8.3.3 Analysis of MBONE Television Broadcast Traffic 230 8.3.3.1 The MBONE Measurements 230 8.3.3.2 General MBONE Traffic Characteristics 231 8.3.3.3 Correlation Structure of MBONE Traffic 234 8.3.3.4 Distribution of Cell Counts 237 8.4 Audio/Video Transmissions over ATM 239 8.4.1 The IKON Teleteaching Application 239 8.4.2 The Experimental Setup 240 8.4.3 Analysis of Native ATM Audio/Video Traffic 241 8.4.3.1 The IKON Measurements 241 8.4.3.2 General IKON Traffic Characteristics 242 8.4.3.3 Correlation Structure of IKON Traffic 245 8.4.3.4 Burst Length Distribution 246 8.5 Summary 248 9 Conclusion 250