Recent Advances in Circuits, Communications and Signal Processing

Transcription

1 Tarek FRIKHA, Nader BEN AMOR, Mohamed Ramzi Ben Yemna, Jean-Philippe DIGUET*, Mohamed ABID CES-Laboratory,Lab-STICC* Sfax University, National Engineering School of Sfax, Sfax, TUNISIE Université de Bretagne SUD, Lorient, France Sfax TUNISIA Abstract: Embedded multimedia applications grow up nowadays. These applications, especially video one are not only developed for the PCs but also for embedded systems such as game console, smartphone, touchpad The embedded system limited resources and particularly in network communication between emitter and receiver platforms need an adaptation to different parameters and specially environment noises. To solve this type of problem, we ll test a video application adaptation on FPGA platforms. This application consists on sending images by network from a platform ML507 to a PC. In this paper, we describe the implementation and the adaptation of the application data transfer to the network transfer. Keywords: Network adaptation, embedded system, FPGA, video adaptation, systems communication. : The multimedia embedded applications inflate the computer sciences domain. Watching a HD video or a 3D movie is now possible not only with a 3D TV but also possible on small portable systems such as smartphone and tablets. The design of such systems faces new challenges due to the limited available resources and the external environment fluctuations such as noise, bandwidth fluctuations, available energy ) Nowadays, network video diffusion becomes more popular. The connectivity and bandwidth amelioration makes possible to have a fast access to video data with various quality from mobile format to full HD one. Despite the significant increase of the rate of flow, the shared network has a limited capacity for transferring high definition and high resolution video. To remedy to this limitation, we need to concept an adaptive system for multimedia treatment. [1] In this paper, we present the conception, the implementation and the optimization of an adaptive MJPEG compressed video transmission system on an IP network for connected mobile terminal. The related work permits to validate four different axes: MJPEG streaming video coder realization. Bandwidth coder adaptation. Moving detection application realization. Application layer adaptation for respecting time execution constraints. The paper is organized as follow. Section II describes the related works. Section III presents the embedded multimedia system. We present also the application design and the adaptive video diffusion application. In Section IV, we talk about the obtained results. Finally, in section V, we conclude the paper with a brief outlook on future works. In this section, we detail the related work in the adaptation techniques. In this context, multiple works have touched the issue of adaptation in application layer and the layer of the OS. There is an algorithm for selecting a configuration where the criterion is the minimization of the use of resources (CPU and bandwidth). The use of such technique for specific critical systems requires implementing formal verification techniques which ensure that the chosen configuration check specific constraints for this type of system. The use of formal ISBN:

2 methods is a difficult task of process development, which often faces the problem of the combinatorial explosion of the number states to take into account. Proposed solution in [3] was to find the best setting as the number of images of reference, the size of the search window, etc. of the encoder to achieve the proportional balance between the video quality and power consumption. The author in [4] presents an approach of adaptation in the application layer based on the modification of certain parameters of the application to manage the desired quality of service. This approach has been validated by a simulation using high-level synthesis of 3D as a case study but it was not validated on an embedded system. An approach in [5] presents an adaptation layer between the hardware, OS, and application layers to multimedia reconfigurable embedded systems. The proposed approach improves the quality of service while respecting system constraints (execution time, duration of life, QoS) and user preferences (minimum level of quality of service, duration of life). Although the author has been able to improve the adaptation of multi-level approach but it is very limited in application level. In [6], the author has added the flow as a constraint and he was able to auto-generate settings reconfiguration of the H264 application. However, this adaptation technique has some disadvantages because it supports only two constraints, a limited number of configuration parameters without taking into account the stability of the system. In [7], the author proposes an adaptive technique based on the coordination between resource allocation and dynamic adaptation in response to changes in environmental resources of the multitasking mobile. Variation can be caused by the variations in the resource availability due to the environment changing as well as to the mobility of the system, or by varying the number of processed applications. The proposed adjustment supports three types of variation, the change in the number of tasks, changes in the availability of network bandwidth and finally the change in the level of energy which is available in the battery of the system. However, this adaptation technique has been tested using a database of configurations which is specific and generated for the application H.264/AVC. The above techniques are based on adjustments either of the application layer or the hardware one. These adjustments are local to the system board. In this paper, we present an approach based on a network adaptation based on implemented Linux OS. The OS permits the management of the hardware application layer. To validate the described approach a mixed demonstrator based on an embedded platform as emitter and a PC as receiver is established. In our work, we propose to design an adaptive strategy including adaptive data transfer via network. The adopted system permits to perform the MJPEG codec which is a complex application. To stream video on the Internet, two solutions are possible. As the capacity of the Internet is limited and it does not broadcast with sufficient quality, you must: reduce the speed (and therefore the quality) of video sequences while maintaining the real time get rid of the real-time broadcasting In our work, we maintain the second solution as it allows us to achieve good results without overloading the system and it is suitable for Video cast application as the display of images in the first solution is done at the time of reception of the image and requires a good connection to ensure good reception. Our system includes three layers: software, hardware and embedded operating system layer which links between the hardware and applications while performing a variety of tasks. 3.1.Bandwidth adaptation: We start by explaining the operation of the server and video casts mechanisms on the network sends TCP adjusts its transfer speed bandwidth, MJPG-streamer cannot send the next frame if the current frame has not been completely transferred. A new image is captured from the sending of the current image. The image to be sent is stored in RAM. The exchange between the input module and output module is synchronized by using a mutual exclusion. In this way, all modules have quick access to the current image stored in RAM without the risk of being overwritten by a new plugin when the output is still the copy. The plugin expects the HTTP output connection from a client (the web browser such as Firefox, Safari, Opera, etc.) to serve the MJPEG stream. The plugin sends the headers that tell the browser what type of data it will receive (HTTP headers). Then it receives the current frame from its location in the memory system videocasts. TCP socket does not report anything until the data is transferred (or to indicate that an error has occurred. Once the image transfer is done, the next step is to send the image to the ISBN:

3 next customer. If a client is slow, it will miss a large number of images. 3.2.Design of the hardware architecture of the system: For the establishment of a broadcasting system, we adopted the following architecture as it is shown in Figure 1. Figure 2. Hardware architecture adopted for videocast system 3.3.The proposed demonstrator: A demonstrator based on an emitter and a receiver part was implemented and described in this part. Inspired by the different parts described in figures 2 & 5, the realized demonstrator is a mixed one. The figure 6 illustrates it. Figure 1. System architecture of an embedded video broadcast The proposed adaptive system of Videocast comprises a device Video capture (webcam or other video source) connected to a core video coding. This latter creates a MJPEG stream from the video source, including compression of JPEG images where the source provides not compressed images. Once the MJPEG stream is created, it should be redirected to the terminals customers through a Web server which is installed on a PC. We used a Xilinx ML507 board, This choice is due to the hardware core: PowerPC. Emitter MJPEG display Network transfer As it is illustrated in Figure 2, our adopted system consists of a PowerPC processor, DDR2SDRAM memory, an Ethernet port, a USB port and a JTAG port. The Ethernet port provides connectivity of the board with the clients in LAN. The USB port plays the role of a video source connected to a USB Webcam. To access to the USB port of the ML507 board, we need a bridge to convert the data from PLB to OPB bus and vice versa. Transmitter Figure 3. Proposed demonstrator for the application validation. The emitter part contains a webcam as the input of the bloc. The webcam project the videoflow by the USB port into the Xilinx ML 507. The projected frames will ISBN:

4 be compressed using the MJPEG video compression algorithm. According to the bandwidth characteristics, the frames sizes will be adapted to the network characteristics. If the bandwidth size decrease, the frame size decreases from 640*482 to 320*240. The video flow also starts at 25 frames by second but grow down into 5 frames by second if the bandwidth is saturated. In the next part we describe the implementation obtained results Design of the software part of the system videocast: The software part of our system configures and maps the linux kernel. It presents many economic and technical advantages on the development kit and the design and implementation of the application using the Web server and the MJPEG codec. We present the compilation and portage of the embedded Linux: we start with the hardware design with Xilinx Platform Studio. Once the design is completed, we generate the bitstream and download it on our FPGA with impact tool. Then, we generate a hardware description in a DTS file (device tree file). This file will allow the kernel to recognize the hardware that is implemented on the board. We add to our kernel, a system file in the root RamDisk. Once everything is assembled, we cross-compile our operating system. After compilation, we get an Elf file. The software is downloaded on the board via the utility XMD as it is shown in Figure 3. We choose the MJPG-streamer because it has been integrated successfully in previous video projects on the internet. Furthermore, it is an open-source application which is developed for the x86 architecture. It includes a Web server based on httpd and supports UVC webcams which are compatible with Linux. The diffusion of M-JPEG video via HTTP is very simple to implement, it is primarily to inform the client (computer in this case) using a particular content type mime-type multipart / x-mixedreplace; boundary = xxxxx that informs the customer that he will receive a series of JPEG files separated by the string limit ('xxxx') as following: As long as the connection is opened, video cast system continues to send images. Each new image replaces the previous sent images as soon as it is fully received. Customers must treat different images upon arrival before being crushed by the following images. Figure 4 shows the display of a MJPEG stream on a computer connected to our videocast system. The display is in Firefox and Google chrome browser in the client computer. The MJPEG stream is created from a sequence of images saved in a header file. Figure 4. Preparation of the kernel and the integration of the application in the Ramdisk Figure 5. Demonstration of running MJPG-streamer on Ml507 board from a stored image ISBN:

5 The Figure 5 illustrates the role of the web server application in broadcast video: 4.1.Frames per second variation: Using a rate greater than 15 frames per second is a challenge for our server with a variable number of clients. Setting the compression rate and image resolution can easily detect the number of clients: our application is not broadcast but a multicast one. Figure 7 shows an fps variation using our shell script to detect the rate of flow changing and saturation and to stop saturation. We use a wired connection inferior than 1100 Kb/s upload. We denote "saturated connection" the state where the emission s peak compared to the limited capacity of our cable connection FPS adaptation 1000 Figure 6. The role of the web server application in broadcast video To validate our adaptation technique designed for multimedia embedded systems we ll adopt M-JPEG video compression as a case study for adaptive mechanism. We start by changing the number of frames per second. Then, we present the compression ratio steps before proceeding to images sizes variations. Finally, we present the proposed approach to outsourcing movement detection algorithm and the obtained results of these tests. First, our tests have relied on the MJPEG streamer configuration server settings for varying the frames number per second fps, JPEG compression rate and the variation of image resolution. Then we outsource the algorithm of motion detection from the embedded platform to the terminal browser. The information collection is done using the pmap linux command. This command provides different information about the memory use of a specific PID (Process Identification) process as well as information about the process shared libraries. 0 Figure 7. FPS variation adaptation Once the server start up at 25 fps (Figure 7), the emission rise with the connected clients increase until being at a state of saturation. Then, we reconfigure our application by reducing the number of frames per second up to 5 fps which allows serving the large number of customers connected avoiding saturation problems. The server restart permits to an important number of clients to be connected. 4.2.Variation of Compression: As materiel, we used a Logitech webcam integrating a hardware JPEG compression. Thus, the created flow by combining instantly captured images in the buffer to yield a broadcast. The variation in compression ratio requires the use of the function jpeg_ create_ decompress then jpeg_create_compress then the captured image to decompress and then recompress. ISBN:

6 These two functions are part of the library management Libjpeg with JPEG images. This function must be repeated for every image buffer that makes a huge load on the CPU and RAM s card without network saturation. The processor cannot perform the same algorithm for all service images. For any compression decompression operation, running time is ranging between 3.4 s and 4.1 s on a 1.8GHz processor with 512MB Ram. Thus it appears that lower occupancy rate of the network is not due to the new size of the images: our JPEG encoder is purely software. Change image resolution. 4.3.Change image resolution: Resizing the media streams is achieved by using the-r parameter or input resolution. Once the bandwidth is saturated, we change the resolution to low resolution "QVGA" 320x240. Figure 8 shows the dynamic change server settings. Figure 8. Resolution variation adaptation The release of network bandwidth due to the change of the resolution providing more bandwidth for new users was represented by Figure 7. hardware components and their diversity of use. Furthermore, maximizing QoS, presents an inherent conflict in the design of embedded systems to satisfy the customers. In this paper, we designed the architecture of a Video cast system. Then, we focused on embedded Linux kernel architecture. We have also integrated the application of M-JPEG video broadcast on embedded Linux and we have added rules to adapt bandwidth and execution time for motion detection code. As perspectives, we would treat video streams out of our system in a cloud environment and retrieve the results on the FPGA platform. We can also integrate hardware accelerators compression / decompression JPEG system architecture. Finally, we can adapt the sending data to RTP. REFERENCES [1] Tarek Frikha, Nader Ben Amor, Kais Loukil, Agnes Ghorbel, Mohamed Abid, Jean-Philippe Diguet Hardware accelerator for self adaptive augmented reality systems (HPCS 2012) [2] P. Tao Zhang, Santosh Pande, and Antonio Valverde.Tamper-resistant whole program partitioning. Conf. on Languages, Compilers, and Tools for Embedded Systems. [3] S. Lin, P. C. Tseng, P. Lin, and L., G. Chen, "Multimode content-aware motion estimation algorithm for power-aware vidéo coding systems," in Proc. IEEE Workshop on Signal Processing Systems (SIPS 2004) pp , [4] N. BEN AMOR, "Approche de conception de processeurs de vision embarqué," PhD thesis ENIS, Tunisia Dec. 12, [5] K. Loukil, N. Ben Amor, M. Abid, Self adaptive reconfigurable system based on middleware cross layer adaptation model, SSD, Djerba, Tunisia, Apr [6] M. Ben Saïd A multi-constraints adaptation technique for embedded multimedia systems master, ENIS june2010. [7] L.F.Ye Multiprocessor self-adaptive reconfigurable architectures PHD may 2011 The growing number of complex multimedia applications has raised new challenges in the design of embedded mobile multimedia systems. Or, the field of embedded systems is one of the areas that are in vogue and its development is related to the evolution of ISBN: