A Control Theory Approach to Video Stream Adaptation for Restricted Bandwidth Networks

Transcription

1 A Control Theory Approach to Video Stream Adaptation for Restricted Bandwidth Networks Rodrigo F. Coelho and Anand Kotra and Gerhard Fohler Technische Universität Kaiserslautern, Germany Abstract In this paper we present a lean method for real-time adaptation of video streaming to limited and varying network bandwidth. It does not perform or modify an actual stream adaptation algorithm, rather uses a closed control loop to monitor resources and determine the operation of the adaptation algorithm. Our method prevents oscillation between over- and underutilization of resources, which can bring the system to an unstable state, with low effective resource usage. Further, its low computation requirements qualifies the application of this solution for adaptation of live streams. As a case study, we use the adaptation of a live MPEG- transport stream for transmission over network with constrained bandwidth. The method can further be applied for other constrained resources as well. Results from our experiments demonstrate the increase of effective resource usage. 1 Introduction Embedded computing systems face strict resource constraints e.g., limited processing power, energy, or network bandwidth availability. In video streaming applications both the resource demands of video streaming and the resource availability vary continuously and are not known in advance. Thus, real-time stream adaptation in embedded systems has to provide both for acceptable Quality of Service (QoS) in presence of insufficient resources, as well as utilization of (constantly changing) available resources. As in a control system, crude adaptation to varying resources can lead to oscillation between over- and underutilization of resources, which can bring the system to an unstable state and with low effective resource usage. In this paper, we present a method based on control theory to increase the effective resource usage for real-time video stream adaptation. It does not interfere with the actual stream adaptation algorithm, as it uses a closed control loop to monitor resources and determine the operation for the adaptation. Thus, its principles can be used with a range of adaptation algorithms and independently of the stream content. Runtime overhead of the method is very small as the actual controller has quite low complexity. As a case study, we use the adaptation of MPEG- transport streams transmitted over a 8.11g wireless network. Throughout this paper, we will show that the implementation of the closed control loop improves the resource utilization, thus improving the perceived QoS. 1.1 Related Work Existing work addresses the video adaptation issues under constrained resources with diverse strategies. For instance [8] proposes an adaptation tool to construct scalable priorityaware video streams. Despite the high quality achieved, adaptation based on scalable video demands dedicated codecs. In [6], a method for real-time stream adaptation based on frame skipping is presented. The proposed method does not require any specific codec: it can be applied to any MPEG- compliant video stream. However this adaptation algorithm wastes resources due to the granularity of frame skipping. Further [] focuses on adaptive streaming of MPEG transport streams by frame dropping, however the approach does not consider maximization of network bandwidth utilization. On a different level, [1] propose mechanisms to use feedback control approaches to dynamically schedule resource allocation for soft-real time multimedia applications. In this work we propose the use of a closed control loop to improve the resource utilization of a real-time video stream adaptation algorithm. 1. Motivation Current video player systems are typically not implemented on dedicated hardware, rather as part of a programable platform [5]. The available resources in these systems are strictly constrained, and must be shared among all the applications running on the same platform. For instance, the end-to-end available bandwidth between a video server and a mobile client can be dramatically affected if this client starts a VoIP call. Besides the conditions intrinsic to the client system, such scenario severely suffers from external interferences, e.g., radio interferences on wireless network.

2 Not only the resource availability changes continuously, resource demands also vary dramatically. For instance, resource demands for video stream transmission are closely related to the video contents: scenes with detailed images or fast motion as in action movies demand much more network bandwidth than a news program where the news anchor is shown for a long time on a mostly unchanged background. Digital video decoding is a real-time task: frames must be decoded and displayed on the screen at a defined frame rate. Real-time issues for MPEG- playout are discussed in detail in [7]. This concept can be extended to video stream transmission: frames must arrive at the client at the same frame rate they arrive at the server, otherwise severe drop on QoS will be perceived on the displayed video. In order to avoid over-utilization of constrained resources and thus deadline misses, incoming video streams must be adapted prior to being transmitted (or decoded). 1.3 Stream Adaptation Figure 1 presents the system scenario in which stream adaptation occurs prior the transmission (decoding) task. For the sake of simplicity, stream adaptation for a transmission task will be considered. The same concept can also be applied to decoding tasks though. Based on the available network bandwidth and the stream content, the video stream adaptation algorithm is applied such that the adapted stream will not demand more network bandwidth than is available. In the case study presented in this work two adaptation algorithms will be used: Quality Aware Frame Skipping (QAFS) [6] and QAFS and DCT (QAD). QAFS first assigns importance to the frames within a Group of Pictures (GOP) and then discards less important ones until the adapted stream requires less resources than is available (or at maximum the same amount of resources). The use of resources in this method is then bound by the instantaneous resource availability and its granularity is imposed by frame sizes. Thus, the resource is not fully utilized for practically all GOPs. QAD is an extension of QAFS and provides much finer granularity. In QAD frames are skipped as in QAFS until the point where skipping the next frame brings the resource demand below the resource availability. Skipping a complete frame is avoided from this point on. The remaining frames are sorted according to the priorities assigned in QAFS and considered for DCT coefficient skip. The selection of the DCT coefficient to be skipped is done as follows: slices are selected alternatively from the top vertical position and the bottom vertical position in the frame. Inside these slices all AC coefficients, except for the first DC coefficient in every block, are skipped until the resource demand is less than the availability. If after skipping all AC coefficients within a frame this condition is not fulfilled, the next frame is considered for DCT coefficient skip. Once the resource demand become less than or equal to the resource availability, the process of skipping coefficients finishes and the result GOP is passed for transmission. Figure 1. Video stream adaptation prior decoding / transmission. 1.4 Goal Traditional methods for stream adaptation impose the instantaneous value of resource availability (or filtered value of it) to be the upper bound of resource utilization of the adaptation algorithms. These approaches, however, lead to underutilization of available resources. The proposed approach overcomes this drawback by applying a closed control loop to alter the value of available resource passed to the adaptation algorithm. The goal of this paper is to improve the utilization of constrained resources required for MPEG video stream adaptation. Further the algorithm implemented in this approach should not be complex and thus keep the runtime overhead very small. Control Approach This section presents how the system presented in Figure 1 must be modified to include an efficient closed control loop and thus improve resource utilization..1 Mapping of the Video Processing System into a Closed Control Loop Block Diagram In the scenario presented in Figure 1, both the resource availability and MPEG video stream are passed to the adaptation algorithm. Based on these two information, the video stream adaptation algorithm is applied and the adapted stream is passed to the transmission block. The adapted stream is then transmitted over the network. This cycle takes place repeatedly at a pace given by the adaptation algorithm. In this work it is assumed that the adaptation algorithm is optimally tuned to deliver the best QoS. Therefore no change in the adaptation algorithm itself will be applied. However, available resources are not fully utilized by the adaptation algorithm. At the system level, one iteration of the adaptation algorithm does not depend on the previous one, e.g., the adaptation of GOP (k) does not depend on the adapted GOP (k 1).

3 In this scenario, the stream adaptation system presented in Figure 1 contains no dynamics. Therefore, in order to insert a dynamic state and thus control the resource utilization of the video stream adaptation system, the network transmission buffer is analyzed as part of the video adaptation system. Figure depicts the closed control loop block diagram with the transmission buffer (in contrast to Figure 1, Figure will be called closed loop system). The description of each block and signal is presented next. Transmission: represents the amount of resource used on the transmission task. It is treated as disturbance to the adapted stream bandwidth. Plant: is the block to be controlled. For video stream transmission, this block represents the network transmission buffer. The plant is modeled as an integrator and to ensure only positive values, its equation is defined as follows: B L(k) = max{,(b L(k 1) +(NB data(k) ANB data(k)))} (1) Sensor: converts the actual buffer level (bytes) into the corresponding value of bandwidth (bytes per second). B BW(k) = B L(k) T s w (k) = y (k) T s ().1.3 Signals Figure. Closed control loop for video stream adaptation. * In this block diagram, except fory which is the actual buffer level in bytes, all signals represent bandwidth (bytes per second). set point (r): defines the buffer bandwidth that must be present in the buffer before the transmission task requires the data to be transmitted. The set point value is updated for every sampling time with the value of the resource availability (section.1.4 explains the choice of this value). r (k) = ANB (k) (3).1.1 Definitions k Sampling time reference k + Sampling time reference for the instant after the transmission block removes the corresponding data from the buffer T s Sampling period B L Buffer level B BW Buffer bandwidth AN B Available Network Bandwidth ANB data Amount of data that can be transmitted duringt s ANB Available Network Bandwidth passed to the Adaptation Task NB Network Bandwidth required by the adapted stream NBdata Amount of data required by the adapted stream.1. Blocks Actuator: the adaptation algorithm is mapped as the system actuator. It receives the signal from the controller and is responsible to forward it to the plant. MPEG- source stream: represents the disturbance imposed to the actuator (see.1.3). controller output(v): signal computed by the controller representing how the resource availability perceived by the adaptation task should deviate from the actual one. actuator input(h): represents the amount of resource that can be used by the adaptation task. h (k) = ANB (k) (4) disturbance (d): represents how the incoming video stream interferes on the actual amount of resource required by the adapted stream, i.e., disturbances will influence how the actuator output deviates from its input. actuator output (u + ): signal containing the amount of resource required by the adapted stream. u +(k) = NB (k) (5) resource availability (u ): amount of resource available to the transmission task. The transmission occurs after the sensor computes the buffer bandwidth (section.1.4 explains this procedure). u (k) = ANB (k) (6) plant input (u): bandwidth given to the buffer. For each sampling period, the signaluassumes two distinct values. before the transmission task is executed This is the value used to compute the increase on buffer bandwidth. u (k) = NB (k) (7)

4 after the transmission task is executed It represents the difference on the buffer bandwidth after the transmission task removes the corresponding data from the buffer. u (k+ ) = NB (k) ANB (k) (8) data buffer level(y): amount of data present in the buffer after the transmission task is executed. y (k) = B L(k) (9) bandwidth buffer level(w): bandwidth corresponding to the amount of data present on the buffer prior the transmission. This value is passed to the controller by the sensor according to equation. The closed control loop presented in Figure differs to the description presented in section.1 in the manner how the available bandwidth is passed to the adaptation algorithm. Figure presents the bandwidth flow through the system (except for y, all other signals represent bandwidth). Based on the network transmission buffer level, the controller adjusts the amount of resources passed to the adaptation algorithm. Then adapted stream is then passed to the buffer and the amount of data respective to the available network bandwidth is removed from it (data to be transmitted). In the next section, more details on the system set point is given..1.4 Sensor and Buffer Bandwidth The controller s goal is to guarantee that the stream adaptation algorith produces a video stream that improves the use of the available network bandwidth. A natural approach would set the buffer set point to zero and design a controller to ensure that the buffer is empty after the sampling cycle is completed. In this case however imprecise values for the buffer level would be fed back. The following example depicts the flaw in this approach and presents an alternative solution. Example: Assume the buffer has a data level corresponding to B BW(k 1) = 1 K and it receives u +(k) = 9 K from the adaptation task. Moreover, the network bandwidth available to the transmission task is ANB (k) = u (k) = 1 K. The value for the computed buffer levelb c BW(k + ) is: B c BW(k + ) = B BW(k 1) + +(u +(k) u (k) ) B c BW(k + ) = 9 However, the buffer will be empty after the transmission task removes the data, since no negative amount of data can be present in the buffer. In this case, the measured bandwidth buffer level is zero B BW(k + ) = (in fact, the measured value would be zero for all values of ANB 91K). B BW(k + ) = { ifb c BW(k + ) < B c BW(k + ) otherwise In this example, the information expected to be fed back from the buffer is: the transmission task has used 9 K less than it could have used. However, this information is lost. To overcome this problem, the signal containing the buffer level is stored before the transmission task removes the data from it. The value of the buffer bandwidth in the previous example becomes: B BW(k) B BW(k) = 91 = B BW(k 1) +u +(k) Therefore, the value fed back to the controller is computed by subtracting B BW(k) from ANB (k). In this scenario ANB (k) is the system set point(r) and the buffer level is measured before the transmission.. The Controller In this work we use a PID controller with anti-windup as the algorithm for set point tracking. The PID controller can be seen as the sum of three controllers that will act on the error signal. Their multiplicative (K p ), integral (K i ) and differential (K d ) coefficients indicate how the algorithm reacts to the instantaneous error, to the sum of the recent errors and to the rate at which the error signal has been changed respectively. Assuming the signal v as the controller output and the error signal e = r w (both presented in Figure ), the continuous controller equation, its respective transfer function and difference equation are described in Equations 1 to 1. v(t) = K p e(t)+k i t V(s) E(s) v (k) = K d(s + Kp K d s+ Ki K d ) s e(τ)dτ +K d de(t) dt (1) (11) = v (k 1) +( K d T s +K p +K i T s )e (k) (1) ( K d T S +K p )e (k 1) + K d T S e (k ) The controller parameters are selected according to the Ziegler-Nichols rules and fine tuned afterwards. The Ziegler- Nichols method [3] states that the controller s parametersk i and K d should be initialized to zero while the proportional gaink p is increased until the system oscillates. At this point, both the gain and the oscillation period are measured and respectively defined as critical gain (K u ) and critical period (P u ). Based on these valuesk i,k d andk p are defined. In order to not violate the Nyquist sampling theorem[4], and without loss of generality, it is assumed that the system

5 sampling period is much shorter than the time required for significant set point changes. The strong influence of disturbances on the actuator causes a continuous discrepancy between the values computed by the controller and those produced by the adaptation algorithm. It leads to a system very prone to the windup effect. This undesired effect takes place in the controller integrator. The controller performs the integration based on its output that is passed to the actuator. However, the plant receives the adulterated signal passed by the actuator output and thus produces a larger error. To correct this error the controller generates higher output that again is not correctly given to the plant. This cycle repeats until the moment when the actuator can respond properly. At this moment the large value generated by the controller is passed to the plant and produces a high output signal (as depicted in Figure 3(a) att = 3). To overcome this problem an anti-windup algorithm is included in this design. Buffer Bandwidth 4 x 15 Set Point 3.5 Level Time (a) Presence of Windup Buffer Bandwidth 4 x 15 Set Point 3.5 Level Time (b) Anti-windup effect Figure 3. Antiwindup drastically reduces overshoot. The anti-windup mechanism implemented here is a modified version of a commonly used anti-windup algorithm. It changes the controller internal state according to the three distinct operation modes: 1. when the resources available are not enough for the adaptation task to generate a valid video stream. when the incoming stream requires less resources than are available to the system 3. for the most frequent operation mode when the incoming stream requires more resources than are available and the adapted stream less than available. For the first mode the controller internal state is set to zero, i.e., all previously integrated values are discarded. For the second and third mode, a value proportional to the difference between the actuator input and outputξ(h u + ) is subtracted from the controller internal state value. This constant is smaller for mode 3 than for mode (ξ 3 ξ ). Thus the Percentual Improvent wrt. Open Loop Variety: QAD News: QAD Sport: QAD Cartoon: QAD City: QAD Animation: QAD Variety: QAFS News: QAFS Sport: QAFS Cartoon: QAFS City: QAFS Animation: QAFS BW Frames Figure 4. Resource utilization improvement for closed control loop: distinct video contents. antiwindup slightly alters the controller internal state for the most frequent operation mode and enforces a larger change for the third operation mode. In Figure 3 we show how the anti-windup mechanism avoids larger overshoot on the buffer level. In this experiment the demanded resource becomes less than the available one (mode ) at t = 85s. 3 Results 3.1 Experiments We applied the proposed closed control loop on an MPEG- video stream transmission test bed with both QAFS and QAD adaptation algorithms. Both permit on-the-fly transport stream adaptation. Video streams received by a digital satellite receiver have been used as source streams. In order to select the controller parameters, a simplified model of the system has been simulated with Matlab/Simulink. The controller gainsk p,k i andk d have been chosen, as mentioned in section., according to the Ziegler-Nichols method. Moreover, the antiwindup parameters have been empirically selected based on the simulation results. Six distinct MPEG- video streams have been used to generate the measurements presented in the next section. They represent a wide range of possible contents, e.g., sports, news, variety, urban landscapes, cartoon and animation in different resolutions (19 by 18 and 64 by 48 pixels). The network bandwidth availability used in the experiments is imposed as a series of five steps of same length. The amplitude of each step is given by: P = P1 m a, where P 1 = (.3,.7,.,.5,.9) represents the multiplication factor of each step. m a is the average network bandwidth required by the original video stream (without adaptation). 3. Evaluation We run experiments with open loop and closed control loop. To evaluate our approach we compared the network bandwidth utilization and two QoS parameters (number of transmitted frames and transmission delay) in both scenarios. A summary of the improvements obtained in our experiments is depicted in the Figure 4. For all types of video content used in our experiments, the

6 closed control loop has improved the efficiency of the adaptation algorithm. Figure 4 shows that the QAFS with closed control loop utilizes close to 3% more network bandwidth (BW) than the the same adaptation algorithm with open loop. The higher bandwidth utilization leads to a similarly large improvement on the number of frames present on the adapted stream: 5% more frames are present in the stream generated by the closed control loop. This large improvement in the cartoon stream is explained by the fact that it contain large I-frames, if compared to the other frame types. For the open loop system, the adaptation algorithm has less chances to include this frame when the network bandwidth is low. On the other hand, the closed control loop can increase the network bandwidth passed to the next iteration of the adaptation algorithm whenever no frame is transmitted. The drawback of using the closed control loop is the possible occurrence of jitter. For all experiments we ran, the maximum jitter experienced was two sampling periods in less than 1% of all analyzed GOPs. Despite the finer granularity of QAD, the closed control loop improves nearly 8% of the used network bandwidth comparing to the open loop system. A time response to the experiment with the variety program video stream mentioned in the previous section is presented in Figure 5. The plotted values are computed for each sampling time, which in the case of our environment is done once per GOP. Figure 5.1 displays the time-variant network bandwidth availability and network bandwidth demand for the incoming video stream. Figure 5. shows that the increase on the resource usage is present during the major part of the video, as well as the improvement on the number of transmitted frames presented in Figure 5.3. The buffer bandwidth level is controlled with no stationary error and it quickly responds to the changes imposed by the adaptation task (Figure 5.4). Larger variations on the buffer level with respect to the set point are seldom and they mostly occur when the availability is low. 4 Conclusion and Future Work In this paper we presented the implementation of a closed control loop which increases the resource efficiency in realtime stream adaptation for environments with limited and varying resource availability. Results from our experiments, showed that the increases on resource utilization lead to improvements on the Quality of Service delivered by the application. Moreover, although only two video stream adaptation methods were used as case study, other algorithms can make use of the method presented in this paper. Future work will address adaptive mechanisms for the controller parameters as well as the further analysis to control other types of resources. Furthermore, the inclusion of multiple resources constraints (e.g., energy and memory) as parameters to adjust the adaptation task is under investigation. % of Available Net. Bw # Frames x 1 5 x Figure 5.1: Network Bandwidth Figure 5.: Improvement on the Network Bw. Utilization Figure5.3: Improvement on the Number of Transmitted Frames Figure 5.4: Buffer Bandwidth Time (s) Figure 5. Variety video stream: adaptation improvements References Available Demand Set Point Level [1] L. Abeni, T. Cucinotta, G. Lipari, L. Marzario, and L. Palopoli. Qos management through adaptive reservations. Real-Time Systems, 9:5. [] M. Burza, J. Kang, and P. Stok. Adaptive streaming of mpeg-based audio/video content over wireless networks. Journal of Multimedia, (), 7. [3] G. F. Franklin, J. D. Powell, and A. Emami-Naeini. Feedback control of dynamic systems. Pearson Pretince Hall, fifth edition, 6. [4] G. F. Franklin, J. D. Powell, and M. L. Workman. Digital Control of Dynamic Systems. Pretince Hall, third edition, [5] C. Hentschel, R. Bril, Y. Chen, R. Braspenning, and T.- H. Lan. Video quality-of-service for consumer terminals - a novel system for programmable components. Consumer Electronics, IEEE Transactions on, 49(4): , Nov. 3. [6] D. Isovic and G. Fohler. Quality aware MPEG- stream adaptation in resource constrained systems. In 16th Euromicro Conference on Real-time Systems (ECRTS 4), Catania, Sicily, Italy, July 4. [7] D. Isovic, G. Fohler, and L. F. Steffens. Real-time issues of MPEG- playout in resourse constrained systems. International Journal on Embedded Systems, 4. [8] C. Krasic and J. Walpole. Quality-adaptive media streaming by priority drop. In In NOSSDAV 3: Proceedings of the 13th international workshop on Network and, pages ACM Press,.