Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic

Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic Min Dai and Dmitri Loguinov Department of Computer Science Texas A&M University College Station, TX 77843 23rd March 2005 M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 1/35

Outline 1 Motivation Preliminaries Challenges & Current Status 2 M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 2/35

Outline Motivation Preliminaries Challenges & Current Status 1 Motivation Preliminaries Challenges & Current Status 2 M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 3/35

Importance of Traffic Modeling Motivation Preliminaries Challenges & Current Status Importance Properly allocate network resources Evaluate protocols and effectively design networks Use as traffic descriptor to achieve certain Quality of Service (QoS) requirements Analyze and characterize a queue or a network M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 4/35

Goals of Traffic Modeling Motivation Preliminaries Challenges & Current Status Goals Capture the characteristics of video frame size sequences The marginal probability density function (PDF) of frame sizes The autocorrelation function (ACF) of video traffic Accurately predict network performance Buffer overflow probabilities Temporal burstiness M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 5/35

Group of Pictures Motivation Preliminaries Challenges & Current Status Intra-GOP Correlation Inter-GOP Correlation I 0 B 1 B 2 P 3 B 4 B 5 P 6 I 7 M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 7/35

Inter- and Intra-GOP Correlation Motivation Preliminaries Challenges & Current Status Definition 1 Inter-GOP correlation is the correlation among various GOPs, which is well characterized by the ACF of the I-frames. 2 Intra-GOP correlation is the correlation between P/B-frames and the I-frame in the same GOP. M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 8/35

Challenges & Current Status Motivation Preliminaries Challenges & Current Status Challenges Coexistence of long range dependency (LRD) and short range dependency (SRD) Coexistence of inter- and intra-gop correlation Strong cross-layer correlation in multi-layer video traffic Various PDF among I, P, and B-frame sizes distributions Current Status Difficult to capture both LRD and SRD properties Little work has considered both inter- and intra-gop correlation Most existing models only apply to single-layer video traffic Current multi-layer traffic models do not capture the cross-layer correlation M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 10/35

Wavelet Domain I-frame Size Modeling Single-layer & Base layer Intra-GOP Correlation Time Domain P and B-frame Size Modeling Inter-GOP Correlation Cross-layer Correlation Enhancement Layer Frame Size Modeling M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 11/35

Wavelet Decomposition A3 Scaling Function A 2 A 1 D 3 D 2 Wavelet Function D 1 Figure: A typical wavelet decomposition Wavelet Decomposition Wavelet decomposition can be simply considered as passing the original signal to a high-pass filter (wavelet function) and a low-pass filter (scaling function). Wavelet function generates the detailed coefficients {D j } and scaling function generates the approximation coefficients {A j }, where j is the decomposition level. M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 13/35

Definition of Frame Sizes Definition Assuming that n is the GOP number, 1 φ I (n) is the I-frame size of the n-th GOP. 2 φ P i 3 φ B i (n) is the size of the i-th P-frame in the n-th GOP. (n) is the size of the i-th B-frame in the n-th GOP. For example, φ P 3 (10) represents the size of the third P-frame in the 10-th GOP. M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 14/35

Modeling I-Frame Sizes Algorithm 1 Perform wavelet decomposition to I-frame sizes φ I (n) till decomposition level J 2 Estimate the coarsest approximation coefficients {A J } 3 Estimate the detailed coefficients at each level 4 Perform inverse wavelet transform to obtain the synthetic I-frame sizes M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 15/35

Modeling I-Frame Sizes Analysis of Wavelet Coefficients 1 ACF approx. 0.05 I-frame size 0.8 ACF detailed 0.04 approx. coefficients autocorrelation 0.6 0.4 0.2 probability 0.03 0.02 0 0.01-0.2 0 10 20 30 40 50 lag (a) 0 0 2000 4000 6000 8000 10000 bytes (b) Figure: (a) The ACF structure of coefficients {A 3 } and {D 3 } in single-layer Star Wars IV. (b) The histogram of I-frame sizes and that of approximation coefficients {A 3 } in the same sequence. M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 16/35

Modeling I-Frame Sizes Estimation of Wavelet Coefficients Estimate the coarsest approximation coefficients {A J } : Prior work uses independent random Gaussian or Beta variables Our model uses dependent random variables with marginal Gamma distribution Estimate detailed coefficients {D j } at each level: Prior work uses i.i.d. Gaussian random variables Our model uses i.i.d. mixture Laplacian random variables M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 17/35

Modeling I-Frame Sizes Detailed Coefficients Estimates 10000 10000 1000 1000 100 100 10 10 10000 1-700 -500-300 -100 100 300 500 700 coefficients (bytes) (a) Actual 10000 1-700 -500-300 -100 100 300 500 700 coefficients (bytes) (b) Gaussian 1000 1000 100 100 10 10 1-700 -500-300 -100 100 300 500 700 coefficients (bytes) (c) GGD 1-700 -500-300 -100 100 300 500 700 coefficients (bytes) (d) Mix-Laplacian M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 18/35

Modeling I-Frame Sizes Performance Comparison 1 1 actual autocorrelation 0.8 0.6 0.4 0.2 actual our model nested AR autocorrelation 0.8 0.6 0.4 0.2 our model GBAR Gamma_A 0 0-0.2 0 100 200 300 lag (a) LRD -0.2 0 5 10 15 20 25 30 35 lag (b) SRD Figure: The ACF of the actual I-frame sizes and that of the synthetic traffic in (a) long range and (b) short range. M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 19/35

Wavelet Domain I-frame Size Modeling Single-layer & Base layer Intra-GOP Correlation Time Domain P and B-frame Size Modeling Inter-GOP Correlation Cross-layer Correlation Enhancement Layer Frame Size Modeling M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 20/35

Modeling P/B-Frame Sizes Intra-GOP Correlation 0.3 0.2 cov(p1,i) cov(p2,i) cov(p3,i) 0.3 0.2 cov(b1,i) cov(b2,i) cov(b7,i) correlation 0.1 correlation 0.1 0 0-0.1 0 50 100 150 200 250 300 350-0.1 0 50 100 150 200 250 300 350 lag lag (a) (b) Figure: (a) The correlation between {φ I (n)} and {φ P i (n)}, i = 1, 2, 3, and (b) that between {φ I (n)} and {φ B i (n)}, i = 1, 2, 7 in Star Wars. M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 21/35

Modeling P/B-Frame Sizes Model Comparison Previous Work Previous work does not consider the intra-gop correlation and estimates P/B-frame sizes as i.i.d. random variables. Our Model The size of the i-th P-frame in the n-th GOP is: φ P i (n) = a φ I (n) + ṽ(n), where a = r(0)σ P σ I. Process φ I (n) = φ I (n) E[φ I (n)] and process ṽ(n) is independent of φ I (n). Parameter σ P is the standard deviation of {φ P i (n)}, σ I is the standard deviation of {φ I (n)}. M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 22/35

Modeling P/B-Frame Sizes Performance Comparison correlation 0.5 0.3 0.1 actual our model i.i.d methods correlation 0.8 0.7 0.6 0.5 0.4 0.3 0.2 actual our model i.i.d methods 0.1 0-0.1 0 70 140 210 280 350 lag (a) -0.1 0 50 100 150 200 250 300 350 lag (b) Figure: (a) The correlation between {φ P 1 (n)} and {φi (n)} in Star Wars and (b) that between {φ B 1 (n)} and {φi (n)} in Jurassic Park. M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 23/35

Brief Description Layered Video Coding Generates one base layer (BL) and one or more enhancement layers(el) BL provides a low but guaranteed level of quality and EL provides quality improvement The input to an EL is the residual between the original image and the reconstructed image from the BL. Cross-Layer Correlation The enhancement layer has a strong dependency on the base layer, which is referred to as cross-layer correlation. M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 25/35

Brief Description (cont.) Definition Assuming that n 1 represents the GOP number in an enhancement layer, 1 ε I (n) is the I-frame size in this GOP. 2 ε P i 3 ε B i (n) is the size of the i-th P-frame in this GOP. (n) is the size of the i-th B-frame in this GOP. M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 26/35

Modeling Enhancement Layer Algorithm 1 We estimate I-frame sizes in wavelet domain 2 Estimate P and B-frame sizes using the cross-layer correlation: ε P i (n) = aφ P i (n) + w 1 (n), ε B i (n) = aφ B i (n) + w 2 (n), where a = r(0)σ ε /σ φ. Parameter r(0) is the lag-0 cross-correlation coefficient, σ ε and σ φ are the standard deviation of the EL and the corresponding BL. Processes { w 1 (n)}, { w 2 (n)} are independent of {φ P i (n)} and {φ B i (n)}. M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 27/35

Modeling Enhancement Layer Cross-correlation Comparison 0.9 0.7 actual our model 0.9 0.7 actual Zhao et al. cross correlation 0.5 0.3 0.1 cross correlation 0.5 0.3 0.1-0.1-0.1-0.3-0.3 0 100 200 300 400 lag 0 100 200 300 400 lag (a) (b) Figure: The cross-correlation between {ε I (n)} and {φ I (n)} in The Silence of the Lambs and that in the synthetic traffic generated from (a) our model and (b) a popular model in related work. M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 28/35

Performance Evaluation Methods QQ Plots To verify the distribution similarity between the original traffic and the synthetic one. 3000 10000 synthetic frame size 2500 2000 1500 1000 500 synthetic frame size 8000 6000 4000 2000 0 0 1000 2000 3000 original frame size (a) Base-layer 0 0 2000 4000 6000 8000 10000 original frame size (b) Enhancement layer M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 30/35

Performance Evaluation Methods (cont.) Variance of traffic during various time intervals To check whether the second-order moment of the synthetic traffic fits that of the original one. 1.E+10 1.E+09 7.E+08 6.E+08 5.E+08 actual our model Zhao et al. bytes 1.E+08 actual 1.E+07 our model GBAR Gamma_B Nested_AR 1.E+06 0 1 2 3 4 5 time interval (s) (a) Star Wars BL bytes 4.E+08 3.E+08 2.E+08 1.E+08 0.E+00 0 1 2 3 4 time interval (s) (b) Star Wars EL M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 31/35

Performance Evaluation Methods (cont.) Leaky-bucket simulation To examine how well the traffic model preserves the temporal information of the original traffic Implementation: Pass the original and synthetic traffic through a generic buffer with capacity c and drain rate d. Evaluation metric: e = p p model, p where p is the actual data loss ratio and p model is the synthetic one. M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 32/35

Performance Evaluation Methods (cont.) 4% 3% our model Nested AR Gamma_A Gamma_B 1 0.8 actual synthetic relative error 2% 1% data loss ratio 0.6 0.4 0.2 0% 10 20 30 40 buffer capacity (ms) (a) 0 0 1 2 3 4 5 drain rate (b) Figure: (a) Comparison of several models in H.264 coded Starship Troopers. (b) The loss ratio p of the original and synthetic enhancement layer traffic for The Silence of the Lambs. M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 33/35

This paper proposed a traffic model applicable to both single-layer and multi-layer VBR video traffic. The presented traffic modeling framework captures both LRD and SRD properties of video traffic. This framework accurately describes both inter-/intra-gop correlation and the cross-layer correlation. Future Work Develop a unified model for multi-layer video traffic More applications in overlay networks M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 34/35

Thank you! Any questions? M. Dai and D. Loguinov Analysis and Modeling of MPEG-4 and H.264 Multi-Layer Video Traffic 35/35