QUADRATIC ρ-domain BASED RATE CONTROL ALGORITHM FOR HEVC Shanshe Wang 1, Siwei Ma, Shiqi Wang, Debin Zhao 1, Wen Gao 1 Department of Computer Science and Technology, Harbin Institute of Technology, Harbin, China Institute of Digital Media, Peking University, Beijing, China ABSTRACT In this paper, a novel quadratic ρ-domain frame layer rate control algorithm is proposed for Low Delay (LD) configuration in High Efficiency Video Coding (HEVC). Firstly, we propose a quadratic ρ-domain rate quantization (R-Q) model to establish the relationship between bit rate and quantization parameter. Subsequently, inspired by the specialized QP determination and reference picture set (RPS) mechanism, we design a RPS based hierarchical partition structure for rate control. Based on the hierarchical partition, an efficient hierarchical bit allocation scheme is provided. Finally, the quantization parameter (QP) is calculated via the proposed R-Q model after the bit allocation to meet the target bits. Experimental results demonstrate the proposed algorithm can significantly improve the R-D performance compared to the state-of-the-art rate control scheme for HEVC. Moreover, our proposed algorithm also provides smaller mismatch between the target bit rate and the actual bit rate. Index Terms HEVC, Rate Control, Rate-GOP 1. INTRODUCTION High Efficiency Video Coding (HEVC) is the latest video coding standard developed by JCT-VC (Joint Collaborative Team on Video Coding) [1] which significantly improves the coding efficiency than the previous video coding standards, such as H.64/AVC []. In HEVC, many new technologies are adopted to improve the coding performance, especially the quad-tree coding structure in which three block concepts named coding unit (CU), prediction unit (PU) and transform unit (TU) [3] are introduced. The specialized structure is introduced to specify the coding, prediction and transform information. This highly well-suited coding structure achieves a leap in coding performance and provides the encoder great flexibility to improve the coding efficiency. This work was supported in part by Major State Basic Research Development Program of China (973 Program, 9CB393), National Science Foundation (6111, 611388), National Hightech R&D Program of China (863 Program, SS1AA185) and National Sci-Tech Support Plan (11BAH8B1). Besides the quad-tree coding structure, the specialized reference frame set (RFS) [4] and QP determination mechanism are other two technologies which bring much coding gains. Rate control plays a crucial role in any coding standard. In the previous video coding standard, many rate control algorithms and rate quantization (R-Q) models have been investigated for rate control, such as TM5 for MPEG-[5], VM8 for MPEG-4[6] and TMN8 for H.63[7] etc.. According to the video applications, rate control can be classified to CBR (constant bit rate) rate control [][3][4] and VBR (Variable Bit Rate) rate control[8]. According to the rate control implementation method, it can be classified into one pass [6], two pass or even multi-pass rate control [9][1]. In the previous coding standard H.64/AVC, the quadratic R-Q model in [6] shows great efficiency in rate control which is proposed under the assumption that the predicted residues follow a single Laplacian Distribution. Many modified algorithms based on quadratic R-Q model are proposed to improve the rate control performance [11][1][13][14]. A novel rate control algorithm with an accurate linear model in ρ-domain in proposed in [15], which is proved to be very effective in all the previous coding standards including wavelet based video coding. At present, rate control for HEVC has not been thoroughly studied yet. Based on the traditional quadratic R-Q model, [16] proposed a unified R-Q model called quadratic pixelbased unified rate-quantization (URQ) model for rate control of HEVC. The algorithm considered the new features that the size of PU varies so the bit allocation must be accordance with the number of pixels. However, the performance is much worse than HM anchor. In [17], based on the linear correlation between the QP and the parameter Lambda, a R model based rate control algorithm is proposed for HEVC which provides much gain over the URQ model. Owing to the specialized coding structure and new coding technologies of HEVC, in this paper, we propose a quadratic ρ-domain frame layer rate control algorithm for HEVC. Firstly, based on the distribution of residual information, an R-Q model in ρ-domain is proposed. Then we design a hierarchical rate control structure based on the specialized reference frame set (RFS) of HEVC. Besides, an efficient hierarchical bit allocation method is proposed to
keep the frames in RFS be of high video quality. Finally the proposed quadratic ρ-domain R-Q model is utilized to calculate an accurate quantization parameter (QP) to meet the target bits. The rest of the paper is organized as follows. In section, a quadratic ρ-domain R-Q model is proposed based on the distribution of the residual information. Section 3 proposes a Rate-GOP based multi layer partition structure and an efficient hierarchical bit allocation scheme for rate control. Section 4 presents the experimental results. Finally, we conclude the paper in section 5.. PROPOSED QUADRATIC ρ-domain BASED RATE MODEL For rate control, the most crucial step is to compute a suitable QP which heavily depends on the R-Q model to meet the target bits of the frame. In the previous work [4], under the assumption that the residual information belongs to a single Laplacian distribution, the source rate is modeled as a quadratic function of the quantization step [6]. The R-Q model shows great efficiency in H.64/AVC. But it has been stated in [18] that a single Laplacian distribution is not appropriate to capture the distribution of transformed coefficients in HEVC due to the quad-tree structure, and thus a more accurate mixed Laplacian distribution is proposed to represent the distribution as follows. 1 i x f( x) f ( x) e (1) i i i where f i (x) denotes the distribution function of transform coefficients of CUs in i-th depth. In [15], it has been proved in ρ-domain that a linear relation between the texture bits and the number of non-zero transformed coefficients exists for a Laplacian distribution as follows. R () (1 ) Nnon _ zero where ρ and R denote the percentage of zero transformed coefficients and the bit rate of the coding frame. N non_zero indicates the total number of non-zero transformed coefficients of the frame. According to our experiments, the linear relation still holds firmly in HEVC. Fig. 3 shows the linear relation in ρ- domain for the sequence BascketballPass and BQSquare. It can be seen that the accurate linear relation still holds firmly with zero intercept. Similar observations can be drawn for other benchmark sequences. Based on the RDOQ of HEVC, for a certain i-th level in (1), the distortion can be calculated via (3) and the percentage of zero coefficients, ρ i, can be computed by (4). q ( j) q i i i ( j ) q j1 (3) D ( q) f ( x) dx f ( x) x jq dx q q i i x i fi( x) dx e dx 1 e q q i q (4) Bit rate(kbps) 16 14 1 1 8 6 4 y = 5.164x R² =.994 1 3 4 Nnon_zero (a) BascketballPass Bit rate(kbps) 15 9 75 6 45 3 15 y = 4.834x R² =.985 5 1 15 5 Nnon_zero (b) BQSquare Fig.1. The linear relation between the number of non-zero transformed coefficients and the texture bits in ρ-domain As illustrated in [15], for the i-th level with a Laplacian distribution, we still can obtain the linear relation as follows Ri i (1 i) (5) where ρ i and R i denote the percentage of zero transformed coefficients and the bit rate of the i-th level. Then bit rate of texture information for the frame can be represented as i R (1 ) e q (6) i i i Under a single Laplacian distribution, a mapping scheme between ρ and QP is provided in [15] to determine a suitable QP to meet the target bits. It may be not suitable to utilize the scheme in HEVC due to the complicated distribution as (1). Consequently, it is necessary to find a suitable way to figure out the quantization parameter. Assume that N i denotes the number of pixels of i-th level and combine with (4), we can get i N, _ = (1 ) q i non zero Ni i Ni e (7) where N i,non_zero denotes the number of non-zero transformed coefficients of i-th level in a coding frame. Thus the total number of non-zero transformed coefficients of the frame can be obtained as iq iq N N e N e (8) non _ zero i i A Taylor expansion of the above equation yields that the non-zero number of transformed coefficients can be represented by the quadratic function of quantization step. Nnon _ zero N a q b q (9) where N denotes the total number of pixel of the frame and the two parameters can be represented as a ( ), b i i (1) 3. RATE-GOP BASED MULTI LAYER PARTITION AND HIERARCHICAL BIT ALLOCATION 3.1 Rate-GOP Based Multi Layer Partition The specialized QP determination and RPS mechanism under Low_Delay configuration significantly improves the encoding efficiency. In HEVC Low_Delay configuration, four successive frames except I frame are considered as a coding group called as Rate-GOP in which the QP of each
frame has fixed difference with QP of I frame, QP I, as illustrated in (11). QPI 1 if ( POC%4) QP QPI if ( POC%4 ) (11) QPI 3 else Different with RPS of H.64/AVC, RPS of HEVC is composed of one frame with the nearest temporal distance and three frames with lowest QP in Decoded Picture Buffer (DPB). For each frame, the GopId is firstly calculated as follows GopId ( POC 1)%4 (1) Then the RPS for the current frame is determined by the corresponding Delta POC depending on GopId. Delta POC is difference between POC of the current frame and POC of previous coded frame. Table 1 presents the Delta POC of the reference frame for different GopId. Such specialized RPS takes more consideration of inter dependency into account and significantly improves the coding performance. Table 1. Reference Picture Set (RPS) in HEVC GopId Delta_POC -1-5 -9-13 1-1 - -6-1 -1-3 -7-11 3-1 -4-8 -1 From the Table 1, we can conclude different frame in a Rate-GOP has different influence on the following frames. Generally, the frame with QP as QP I +1 has relatively high reconstructed video quality and will be referred more than once which plays a crucial role for the final performance. Other frame in the same Rate-GOP is only referred once, so they are less important than the frame with QP I +1 in terms of the reference by the following frames. The frame with QP as QP I +1 is considered as a key frame in our paper. Fig.. Hierarchical Structure of Rate Control RateGop Based on the specialized RPS and different influence on the following frames, we define a Rate Control RateGop (RCRG) which is composed of four frames including one key frame and its following three frames and then divide the frames into three layers. Since the first frame plays a crucial role which will be referred by the following frame more times and has great influence on the final coding performance, so the first layer includes the first frame in a RCRG. The second layer includes the second frame of the Rate-GOP, whose video quality mainly depends on the first layer because its nearest reference frame usually has high video quality and the other reference frames is much further. The remaining two frames whose two nearest reference frames locate in the same RCRG belongs to the third layer. Fig. shows our proposed hierarchical structure of RCRG. 3. Hierarchical Bit Allocation Bit allocation is a most crucial part during the rate control utilized in the calculation of QP and determines the smoothness of bit rate. Based on the multi layer partition as illustrated in section 3, we propose a hierarchical bit allocation scheme. Firstly, the bits for a RCRG are allocated as following. TRateGop NRateGop Rremaining / Fnum (13) where T RateGop denotes the target bits for a RCRG, R remaining is the remaining bits after encoding the previous frames, N RateGop indicates the number of the frame in a RCRG and F num denotes the remaining frames to be coded. Due to the temporal correlation between the adjacent RCRGs and in order to get a smooth bit rate, the allocated bits for a RCRG is further modified as, T RateGop TRateGop +(1- ) TRateGop_ pre (14) where T RateGop-pre denotes the actual bits of the previous RCRG and is set as.5 empirically. Secondly is to allocate suitable target bits for each frame in a RCRG. The frame in upper layer should be allocated more bits due to its crucial role and fewer bits are allocated for the frames in lower layers. So based on the proposed hierarchical partition, the target bits allocated for the frames in different layers is implemented as follow, Ti, j i, jt RateGop j 1,, 3 (15) where T i,j and i, jdenotes the target allocated bits and bits ratio of the frame in j-th layer of i-th RCRG respectively. The initial value of the parameters in (5) is set as.5,.3 and.. Then the parameters will be updated adaptively as 1 = ( + + ) =1,,3 (16) ij, i-3, j i-, j i-1, j j 3 The proposed bit allocation scheme ensures that the frames in layer 1 can be allocated more bits, so frames in the current RFS can be of high video quality and improved coding performance can be expected. After the determination of bits allocation, the proposed rate model () and (9) is utilized to calculate the QP to meet the target bits. Summary of the proposed algorithm Allocate target bits for i-th RCRG based on (13) For each frame in the i-th RCRG 1) Allocate target bits for j-th frame as in (14) 1 n i ) Predict the value of θ as n i1 3) Calculate the N non based on () 4) Determine QP based on quadratic ρ-domain model (9) 5) Encoding with the QP end Updating the parameters a, b and θ end End
55 4. EXPERIMENTAL RESULTS In order to verify the efficiency of our proposed algorithm, we first integrated the proposed algorithm scheme into the HM8. and compared with the HM anchor and state-of-art rate control algorithm [17] which is the recommended rate control algorithm in HM at present. The R-D comparisons is tested by the scheme in [19] and the rate estimation accuracy is measured by the frame layer mismatch ratio by Rtarget -Ractual M %= 1% R target where R actual and R target denote the actual bits and the target allocated bits of the video sequence respectively. Standard test sequences in different classes provided by HEVC are adopted to test the proposed algorithm. PSNR(dB) 4 4 38 36 34 3 3 5 1 15 5 3 35 4 45 5 (a) FourPeople ( 18x7) Rate(kbps) 5 45 4 35 3 5 1 11 1 31 41 51 (b) FourPeople (18X7) Target bit rate = 435kbps Fig. 4. Bit rate fluctuation curves of the proposed algorithm Table. Performance comparison for the proposed algorithm with HM anchor and R-λ[17] for LB-main and LP-Main coding configuration testing LB-main Sequences Gain over HM8. Gain over R-λ[17] BD-Rate BD-PSNR(dB) BD-Rate BD-PSNR(dB) ClassB 3.5% -.7-4.%.7 ClassC -1.4%.9-3.9%. ClassD -.%.8-7.5%.3 ClassE -3.5%.11-5.%.16 Avg -.9%.5-5.%.19 LP-main Sequences Gain over HM8. Gain over R-λ[17] BD-Rate BD-PSNR(dB) BD-Rate BD-PSNR(dB) ClassB.% -.5-4.6%.1 ClassC -.4%.9-4.8%.18 ClassD -1.6%.7-6.7%.5 ClassE -3.1%.11-4.6%.14 Avg -.3%.6-5.%.17 PSNR(dB) 4 38 36 34 3 3 8 6 4 5 1 15 5 3 35 4 45 (b) BachetballPass (416x4) Fig. 3. Quality fluctuation curves of the proposed algorithm Table states the performance comparisons for the proposed algorithm with HM anchor and the state-of-art rate control R-λ algorithm. It can be observed that the proposed algorithm works well for both high resolution and low resoution sequences. From the tables, we can conclude the proposed algorithm presents much better encoding performance. Compared with R-λ algorithm [17], the BD_Rate and BD_PSNR gain can be over 5% and.18db on average. Table 3 shows the bit rate mismatch ratio comparisons. It is shown that the proposed algorithm generates smaller mismatch between target bit rate and actual bit rate. Fig. 3 and Fig. 4 shows the flunctuation curves of quality and bit rate of FourPeople and BascketballPass. It can be seen the proposed shceme also provides smooth video quality and bit rate. Rate(kbps) 3 5 15 1 Table 3. the rate mismatch of the proposed algorithm for LB-main and LPmain configuration Sequences Rate Mismatch LB-main LP-main ClassB.31%.% ClassC.15%.68% ClassD.3%.8% ClassE.1%.9% Avg.%.3% 5. CONCLUSIONS AND FUTURE WORK Based on the new coding technologies of HEVC, a novel quadratic ρ-domain rate model is proposed and a frame layer rate control algorithm is studied in our paper. The novelty of this paper lies in that we employ the ρ- domain rate model and hierarchical bits allocation scheme to get a better rate control performance. The efficiency of the proposed algorithm can be verified by the experimental results. Moreover, due to the simple rate model and parameters updating scheme, almost no extra computational complexity is added for the HM platform. We focused on the Low Delay configuration in this work and will continue to investigate rate control algorithms in Random Access setting in future. 5 1 11 1 31 41 (a) BascketballPass (416x4) Target bit rate = 16kbps
6. REFERENCES [1] B. Bross, W.-J. Han, G. J. Sullivan, J.-R. Ohm, T. Wiegand (Editors), High Efficiency Video Coding (HEVC) text specification draft 6, Document: JCTVC-H13, Joint Collaborative Team on Video Coding (JCT-VC)of ITU-T SG16 WP3 and ISO/IEC JTC1/SC9/WG118th Meeting: San José, CA, USA, 1 1 February, 1. [] Draft ITU-T Recommendation and Final Draft International Standard of Joint Video Specification, ITU-T Rec. H.64 and ISO/IEC 14 496-1 AVC, Joint Video Team, Mar. 3. [3] I. K. Kim, J. Min, T. Lee, W. J. Han, and J. Park, Block Partitioning Structure in the HEVC Standard, IEEE Trans. Circuits Syst. Video Technol., in publication [4] R. Sjöberg, J. Samuelsson, Absolute signaling of reference pictures, Joint Collaborative Team on Video Coding (JCT- VC)of ITU-T of ITU-T SG16 WP3 and ISO/IEC JTC1/SC9/WG116th Meeting: Torino, 11. [5] Test Model 5. [Online]. Available: http://www.mpeg.org /MPEG / MSSG / tm5. [6] T. Chiang and Y. Zhang, A new rate control scheme using quadraticrate distortion model, IEEE Trans. Circuits Syst. Video Technol., vol. 7,no., pp. 87 311, Apr. 1997. [14] D. Kwon, M. Shen and C. C. Jay Kuo, Rate control for H.64 video with enhanced rate and distortion models, IEEE Trans. Circuits Syst. Video Techn.,17(5): 517-59, 7. [15] Z. He and S. Mitra, Optimum bit allocation and accurate rate control for video coding via -domain source modeling, IEEE Trans. Circuits Syst. Video Technol., vol. 1, no. 1, pp. 84 849, Oct.. [16] H. Choi, J. Nam, J. Yoo, D. Sim, I.V. Bajić, Rate control based on unified RQ model for HEVC, Joint Collaborative Team on Video Coding (JCT-VC)of ITU-T SG16 WP3 and ISO/IEC JTC1/SC9/WG118th Meeting: San José, CA, USA, 1 1 February, 1. [17] B. Li, Rate control by R-lambda model for HEVC, Joint Collaborative Team on Video Coding (JCT-VC)of ITU-T 1/SC 9/WG 11, 11th Meeting: Shanghai, CN, 1 19 Oct. 1. [18] Bumshik, M. Kim, Modeling Rates and Distortion Based on a Mixture of Laplacian Distribution for Inter-Predicted Residuals in Quadtree Coding of HEVC, IEEE signal processing letters, vol. 18, no. 1, October 11. [19] G. Bjontegaard, Calculation of average PSNR difference between RDcurves, in Proc. ITU-T Q.6/SG16 VCEG 13th Meeting, Austin, TX,Apr. 1, document VCEG-M33. [7] J. Corbera and S. Lei, Rate Control for Low-Delay Video Communications, ITU Study Group 16, Video Coding Experts Group, Portland, Documents Q15-A-, 1997. [8] M. Hamdi, J. W. Roberts and P. Rolin, Rate control for VBR video coders in broad-band networks, IEEE Journal on Selected Areas in Communications, Vol. 15, No. 6 Aug. 1997. [9] P. H. Westerink, R. Rajagopalan and C. A. Gonzales, Twopass MPEG- variable-bit-rate encoding, IBM J.RES. DEVELOP, Vol. 43 No.4 Jul. 1999. [1] D. Zhang, K. N. Ngan, and Z. Chen, "A two-pass rate control algorithm for H.64/AVC high definition video coding," Signal Processing: Image Communication, vol. 4, no. 5, pp. 357--367, May. 9. [11] S. Ma, W. Gao and Y. Lu, Rate-distortion analysis for H.64/AVC video coding and its application to rate control, IEEE Trans. Circuits Syst. Video Technol., vol. 15, no. 1, pp. 1533-1544, Dec. 5. [1] JVT-G1 Z. G. LI, F. Pan, K. P. Lim and G. N. Feng. Adaptive Basic Unit Layer Rate Control for JVT. Joint Video Team of ISO/IEC MPEG and ITU-T VCEG, JVT-G1, 7th meeting, Pattaya, Thailand, 7-14(3). [13] M. Jiang and N. Ling, On enhancing H.64/AVC video rate control by PSNR-based frame complexity estimation, IEEE Trans. Consum. Electron., vol. 51, no. 1, pp. 81-86, Feb. 5.