Research and Improvement on A Lightweight VOD Cloud Storage Engine

Transcription

1 Journal of Computational Information Systems 8: 2 (2012) Available at Research and Improvement on A Lightweight VOD Cloud Storage Engine Chang SHU, Xingming ZHANG School of Computer Science and Engineering, South China University of Technology, Guangzhou, China Abstract Cloud storage is a new concept which has expanded and developed from cloud computing. Despite being a relatively new concept, it already has a wide range of applications. In recent years, researchers in the field of cloud storage have focused on cloud computing s storage capabilities and the structure of its levels of improvement. However, comparatively less attention has been given to the specific application environment of cloud storage needs, especially in VOD (Video-On-Demand), resource storage, and business applications; the cloud computing needs of cloud storage solutions have not been addressed effectively as well. Therefore, the present study provides an in-depth and detailed analysis of the current application of common cloud storage design ideas and the implementation method, including its performance advantages and disadvantages, focusing on its resource sharing capability and rapid lowcost building capacity. A lightweight cloud storage engine design suitable particularly for the storage of multimedia resources is proposed. This proposed design is based on the current mainstream open source cloud storage system, focusing on cloud storage engine architecture, communication protocols, synchronization mechanisms, and the positioning mechanism of block storage files. Finally, a simulation environment for the comparison test shows that the system is stable and highly available. Keywords: Cloud Computing; Lightweight Engine; Cloud Storage; Vod; Load Balancing 1 Introduction In recent years, major companies have introduced a variety of cloud storage techniques [1-4], and related products online offer cloud storage services; however, hardware costs and the complexity of large software architectures bring a certain degree of difficulty in the field of cloud storage application. Given current technical conditions, the spread of cloud storage application must rely on clustering technology [5-6], storage virtualization technology [7-8], distributed file systems [9-10], and other key technology solutions. Researchers on storage cloud computing are often more concerned with cloud computing storage capabilities and the structure of the level of improvements. Hao and Chuohao made a film based on the automatic detection of modified data to trigger the sync, a strategy to improve the efficiency of synchronization [11]. Nepal, Chan, and Moreland Corresponding author. address: shuchangok@yahoo.com.cn (Chang SHU) / Copyright 2012 Binary Information Press January 2012

2 684 C. Shu et al. /Journal of Computational Information Systems 8: 2 (2012) proposed the adoption of virtual storage architecture in the design of a simple XML-based allocation of storage interface [12] to achieve scalable distributed storage service purposes. Likun, Yongwei, and Guangwen designed a storage architecture based on Zetta Data, proven by experiments to increase the throughput and reduce the resource consumption of the effective storage cluster [13]. Talattinis and Hadoop conducted commercial data processing of a large number of experiments [14], which was followed by An and Dandan who also focused on the cloud-based storage solutions Hadoop studied [15]. However, these researchers are less concerned with the specific application environment of cloud storage needs, especially in VOD, resource storage, and business applications, and particularly the cloud computing needs of cloud storage solutions which have not been addressed effectively to date. Compared with general public storage requirements on the Internet, these applications have a greater need for resource sharing capability and rapid low-cost building capacity. Therefore, the present work provides an in-depth and detailed analysis of the current application of common cloud storage design ideas and the implementation method, including its performance advantages and disadvantages, focusing on its resource sharing capability and rapid low-cost building capacity. A lightweight cloud storage engine design (LiDFS) particularly suitable for the storage of multimedia resources is proposed. The proposed design is based on the current mainstream open source cloud storage system, focusing on cloud storage engine architecture, communication protocols, synchronization mechanisms, and positioning mechanism of block storage files. Finally, a simulation environment for the comparison test shows that the system is stable and highly available. 2 A Lightweight Cloud Storage Engine 2.1 Design and system architecture The cloud storage system abstracts business computing and stores resource physical mapping. It can allocate storage based on business resources dynamically. It is highly scalable, highly reliable, low-level transparent, and secure; it has a common underlying architecture of monitoring and service interface. Typically, a standard cloud storage system at least includes the following levels: access layer, application interface layer, basic management and storage layer, and different systems with different complexity levels for different cloud storage service targets contained in all levels. The present article draws on common cloud storage architecture, and basing on the relaxation coupled asymmetric structure framework, it proposes a special class of multimedia resources for the video store, which is a lightweight cloud storage engine framework called LiDFS. The system logical framework is shown in Figure 1. In the above figure, the controller (clusters) is responsible for scheduling work and has a loadbalancing effect for access connections. The storage node in the cluster group and status information will be recorded in the memory, which is a hub connecting the client to the storage node. All servers are equal; no Master-Slave relationship is required to avoid a single point of failure effectively. The storage cluster is assembled as group based; the same groups of files on the storage nodes are identical (RAID 1). All storage nodes files and file attributes (meta data) are saved to their own nodes. Under the policy of storage, the active node status information is reported regularly to the controller; different groups of storage servers do not communicate with one another. The simple working mechanism of the proposed lightweight system reduces the load

3 C. Shu et al. /Journal of Computational Information Systems 8: 2 (2012) Fig. 1: System logical framework substantially, thus improving access efficiency effectively. 2.2 Design of communication protocols and access procedures For any storage system, reading the data storage device of the system bandwidth is an important factor in effective throughput. Developing a large-scale video network service system design and improving the system performance and memory bandwidth remain a bottleneck in terms of resources; appropriate optimization strategies must be adopted to improve the rate of data retrieval and transmission and thereby achieve the objective of improving the overall system throughput. Therefore, GoogleFS [16], MongoDB [17], and other general designs of a distributed storage system for the engine are utilized, focusing on inter-module communication protocol from the file access protocol and design improvements. (1) Protocol Format The communication protocol is based on TCP/IP, with the protocol package consisting of two parts: the header and the body. The header is 10 bytes in size, and the format is as follows: Table 1: Communication protocol format Header(10 bytes) Body(N) body length (8 bytes) command(1 byte) status(1 byte) The body in the packet format depends on the command, and it can be empty. (2) File Storage and Reading Process The file storage and reading process is shown in Figure 2/3.

4 686 C. Shu et al. /Journal of Computational Information Systems 8: 2 (2012) Fig 2: File storage process Fig 3: File storage process The client asks the controller to u- pload to the storage node. The controller returns an available storage node. The client and the storage node communicate directly and complete the file upload; the storage node returns the file ID. The client has the priority to obtain the download path directly from the file ID (priority to download from the initial group, the source n- ode; if the node fails or refuses, the controller has to be asked for other effective nodes. The client asks the controller for a storage node which can download the specified file; the argument is the file ID (group name and file name). The controller returns an available storage node. The client and storage node communicate directly to complete the file download. 2.3 Improved storage mechanism based on peer grouping Typically, for a large-scale VOD application, resource files are often stored on an NAS/SAN-based multi-level storage system. In recent years, with the development of video-on-demand application technology, some researchers made improvements on the storage structure and scheduling algorithm. For instance, Sanchez developed the MAPFS-Grid architecture [18]. Yang et al. proposed a P2P-based metadata storage cluster [19]. However, although these studies improved effectively the efficiency of resource access, their contributions are too hardware dependent, thus reducing business expansion significantly. For example, when faced with a large-scale video-on-demand which reads requests rather than writes them, and a single file size is very large (a 100-minute 720 p standard definition video is about 1.5G), the storage requirement would be unimaginable. Although HadoopDFS [20], GoogleFS [16] and other such distributed file systems can provide good scalability and compatibility, their complex structure and file block storage overhead caused by additional indexing are not suitable for video resource storage. Therefore, for such application-

5 C. Shu et al. /Journal of Computational Information Systems 8: 2 (2012) s, the expansion capacity of storage systems, the cost, and the compatibility of factors involved need to be considered. (1) Storage Strategy According to the design of the engine, the destination node of the uploaded file needs to be selected by the controller. Uploaded files generate a file ID, which makes downloading the same file easier for the client; the client can have priority access to the download path based directly on the file ID (priority to download from the initial group, the source node). If the node fails or refuses, the controller has to be asked for another effective node. To improve the hit rate in reading and spreading the connection of each node request as much as possible (because the time it takes to read a file is often too long; scattering reading services to different nodes can improve substantially the storage quality of the service), requesting files many times should be avoided in the same disk storage. Video files tend to use the traditional block storage strategy. Although it offers increased flexibility from the perspective of file scheduling and content distribution, the cost of the system resources of the server is indeed improved substantially. Further, the complexity of the algorithm also makes the storage system difficult to extend and construct rapidly. Therefore, the lightweight storage engine does not offer file block storage compared with simpler and more efficient engines. It can also meet the actual needs of the vast majority of Internet applications. Taking all these factors, the various nodes are defined within the same group as the parameter index (because the upload Group ID is designated generally by business applications; different sets of parameters are not compared here): Node disk I/O maximum bandwidth W. The node has access to the file transfer bandwidth Node CPU utilization P c. Node Memory utilization P m. n i=1 A i Then the current node real-time performance indicators σ = (1) are calculated, where W P c P m describes the value of the greater and better performance of the node. Therefore, the storage engine will be stored in two different business strategies to meet the types of services: Strategy No. 1: Always upload file to a specific group of nodes with the best performance, according to indicators, that is,t. The node s CPU, memory, disk I/O, and other parameters are the best among all nodes in this group. Strategy No. 2: Always upload file to a specific group of nodes whose performance indicators are the worst, that is. The node s CPU, memory, disk I/O, and other parameters may not be the best among all nodes in the group, but they are the least busy. Next, the file read service hit rate is calculated for these two strategies: Assume the storage node to support I/O connections is the number N, and the file storage transfer time is T. Users request service randomly. Suppose the requested time of user i is ; the request before N allows users to obtain the service immediately. If before the N + 1 user request, the first user s service is completed, that is, then the N + 1 requests can obtain a response immediately; otherwise, time interval of the request will be denied. After, the first user connection is released, and the system can now respond to new requests.

6 688 C. Shu et al. /Journal of Computational Information Systems 8: 2 (2012) This way, the file read request can be considered approximately as many consecutive cycles; the length of each cycle t 1 + T, can be divided into the acceptable and rejection stages. The length of the acceptable stage is t N, and the length of the acceptable stage is t 1 + T t N. For the length of time in t, the average request is λ t, and the average is. Therefore, for reading services, the rejection rate can be expressed as follows: ρ 1 = Number Rejected/T otal Number of Requests = λ Rejected T ime/λ T otal Request T ime = (t 1 + T N λ )/( 1 λ + T ) = (1 + λ T N)/(1 + λ T ) = 1 N)/(1 + λ T ) (2) When T increases, ρ 1 increases, and when T decreases, ρ 1 decreases. Therefore, to reduce the rejection rate of reading services, T must be decreased or N increased. The main factors affecting T are the performance parameters of the node itself, the client connection times, file size, and bandwidth, among others. Figures 2 and 3 show the engine s file access process. When the client uploads a file, the file ID is not specified by the client, but the storage nodes help generate and return it to the client. The file ID contains a group name, the file relative path, and the filename, as shown in Table 2. When the client needs to read the file, it can be downloaded directly from the file ID to obtain the path which navigates to the file (priority to download from the initial group, the source node). If the node fails or refuses, the client can try to ask for another effective node from the controller. Table 2: Uploaded file name definition File ID group name disk name folder name file name Therefore, pictures and documents for the class of small files usually take storage Strategy No. 1. This method can guarantee effectively the quality of service by high-performance nodes and can reduce the rejection rate of the file read service. For the storage file of video resources of large files, reading takes a long time because of the connection, and the connected state should be maintained; further, reducing effectively the value of T is difficult. Therefore, storage Strategy No. 2 should be adopted for such documents. According to the policy of increasing duplicate numbers, the N value should be increased as much as possible so that when a large number of file reads are requested, the service rejection rate will be reduced effectively. (2) File Synchronization and Storage Expansion The engine is used in grouping identical files on the storage server (RAID 1), so different groups of storage node do not communicate with one another. Based on a lightweight design, the controller s configuration file does not appear in the storage node, and the storage node configuration file will list all of the controllers. This rule indicates that the connection between the storage node and the controller will be initiated by the former. The storage node will start a thread connection and communicate with each controller. The controller will then store the classification in the memory and storage nodes under various groups; the storage connects nodes through their own groups to a file. Therefore, the service can

7 C. Shu et al. /Journal of Computational Information Systems 8: 2 (2012) be restarted directly by obtaining information from local disk stores. The storage node will be recorded in the memory of all servers in this group, and the server s log information will be saved to the file. The controller and the storage node will be synchronized with each storage node list. When adding a new controller, the storage node connected to the controller finds that the latter returns a list of storage nodes fewer than the local records. Then the storage node will be synchronized to the controller. The same groups of storage nodes are equivalent; file upload, deletion, and other operations can be carried out in any storage node. The file synchronization within the same group is only between storage nodes; using the push mode, the source server is synchronized to the target server. In a file upload, a group may have three storage nodes A, B, and C. File F is uploaded to server B, and the file F from the remaining two servers synchronize A and C. The operation that uploads file F to server B may be called source operation. File F in server B is the source data file. It is synchronized to the operation of servers A and C as a backup operation; file F in A and C is backup data. The synchronization rules are summarized as follows: Synchronization occurs only in the group between storage nodes. Only the source data require synchronization; backup data do not require re-synchronization. The second rule has an exception: when adding a new storage node, one of the existing storage nodes will have all the data (including the source and backup data) synchronized to the new server. The storage node is set in seven states: FDFS STORAGE STATUS INIT: It has not yet been synchronized with the data source server. FDFS STORAGE STATUS WAIT SYNC: The source of the existing data synchronization server is awaiting synchronization. FDFS STORAGE STATUS SYNCING: In synchronization FDFS STORAGE STATUS DELETED: The server is removed from the group. FDFS STORAGE STATUS OFFLINE: Offline. FDFS STORAGE STATUS ONLINE: Online; cannot provide services. FDFS STORAGE STATUS ACTIVE: Online; can provide services. When the storage node status is FDFS STORAGE STATUS ONLINE, and the storage node initiates a heartbeat to the controller, the controller changes the status to FDFS STORAGE STATUS ACTIVE. When adding a new storage node, the system will complete automatically the existing data synchronization processing logic as follows:

8 690 C. Shu et al. /Journal of Computational Information Systems 8: 2 (2012) Fig. 2: Process of node add-ins Storage node A connects to the controller, and the controller sets storage node A s status to FDFS STORAGE STATUS INIT. Storage node A asks the source server for additional synchronization time. If the group only has storage node A or the group has uploaded 0 files successfully, the data do not need to be synchronized, and storage node A can provide online services. At this time, the controller sets its status to FDFS STORAGE STATUS ONLINE; otherwise the controller will set its state FDFS STORAGE STATUS WAIT SYNC into the second step of the treatment. If the controller assigned to storage node A synchronizes existing data from source storage node B, other storage nodes in the same group will communicate with the controller; they will know that storage node A is added and will start the synchronization thread. They will ask the controller for the source data server and the synchronized deadline of storage node A. Storage node B will synchronize all data to storage node A before the synchronization deadline. The rest of the storage node will start normal synchronization from the synchronized deadline; only the source of the data synchronization will be synchronized with storage node A. After the deadline for storage node B to synchronize data to storage node A, additional synchronization will be switched to normal, and only the source data will be synchronized. After storage node B synchronized all of the data to storage node A, no more data need to be synchronized. Storage node B can request the controller to set the state of storage node A to FDFS STORAGE STATUS ONLINE; When storage node A launches a heartbeat to the controller, the controller will set the status to FDFS STORAGE STATUS ACTIVE;

9 C. Shu et al. /Journal of Computational Information Systems 8: 2 (2012) Based on the above design of synchronization, the expansion mechanism can achieve flexibly the storage engine capacity increase. When a group s storage server access pressure is large, the storage server in this group can expand service capabilities (vertical expansion). When the system capacity is insufficient, the group can expand storage capacity (horizontal expansion). This condition will not only ensure the stability of the storage service but will also improve substantially the system scalability. File synchronization is asynchronous, so some cases can cause synchronized delays. To avoid the situation wherein a newly uploaded file cannot be read in a storage node that has not yet been synchronized, the controller turns need to be set to storage node selection (round-robin). When the client asks the controller storage node for the file to be downloaded, the controller returns the following four conditions to satisfy one of the storage nodes: The file is uploaded to the source storage node; files are directly uploaded to the server. File creation time stamp < the storage node is synchronized to the file timestamp. This means that the current file has been synchronized. File creation time stamp = The storage node is synchronized to the file timestamp, (Current timefile creation time stamp) > A file synchronization requires the maximum time (e.g., 5 minutes); (Current time File creation time stamp) > File synchronization delay threshold such as the set threshold value of 1 day; the file synchronization must be completed in one day. This approach avoids not only file synchronization delay but also timing problems in updating the storage node in more than one file. 3 Experiment and Analysis 3.1 Test model In testing, different storage engines were used based on Gigabit Ethernet (default configuration includes three storage nodes). Using Load Runner as the load generator, a large number of prewritten scripts were employed to test the performance of the client. Access to a large number of connections is needed to test streaming server performance. These connections to access the file content according to distribution and the distribution of service request time intervals should be defined in different spatial models (access content model) and time models (access time of interval model). Content access model To verify the generality of the storage system and to simulate the client s access behavior, the test access to the target file in the distribution followed uniform distribution, that is, all simulated client visits were distributed evenly in all of the documents. Meanwhile, to explain better the storage engine in terms of the effect of different application types of support, the document test was classified as follows:

10 692 C. Shu et al. /Journal of Computational Information Systems 8: 2 (2012) Table 3: Classification of test file File Size Number of files Group KB Group2 200KB-10MB 2000 Group3 10MB-500MB 100 The testing and storage engine was connected in a short manner, and the file upload was repeated for a certain type of file (such as type 0C200 KB). Uploaded files that reached a specific number (e.g. 100,000) were no longer uploaded. Time access model What time of the terminal server access, will affect the server performance. The start of the session in the timeline according to the distribution of the interval was designed to be uniform, following the Poisson distribution model. In uniform distribution, the number of clients is at a fixed time interval, request to the server one by one. This is an ideal model; a relatively realistic model is the Poisson distribution, which has statistics and probability similar to the discrete probability distribution. It is expressed as follows: P {ξ = k} = λk k! e λ, k = 0, 1, (3) Where λ is the request rate, and K is the number of simulated clients. 3.2 Aggregate throughputs To test the performance of file reading and writing based on different protocol architectures, the present study conducted simulation experiments comparing GoogleFS, Hadoop, mongodb, and LiDFS classification of the various data files. The results are shown in Table 4. The analysis is shown in the image above. In the design requirements of GoogleFS, all data blocks are stored in 64 MB. Although it is effective in reducing the size of the metadata, the improvement in file access speed was relatively small. Group1CGroup3 files access speed was ideal. However, the file was still used due to the block storage mode, and the Master node was used to control and manage the various data chunk nodes and to read obvious bottlenecks. Therefore, in reading and writing large files, especially in high concurrency, the read and write rates decreased obviously. The MongoDB data used a lightweight TCP/IP writing protocol, which facilitates a simple packaging above the BSON data layer. This makes the written data command include a 20-byte message header; moreover, it used a larger system level virtual memory mechanism, improving substantially the read and write performance for small files. However, it also increased severely the memory overhead of the operating system, particularly in large file access performance, resulting in a serious decline. The proposed LiDFS not only further simplified the access protocol (10 bytes) and allowed the client to read directly the file from the source storage node to some extent. It also offered effective access pressure sharing of large files to the whole system. Therefore, its large file reading performance was the best.

11 C. Shu et al. /Journal of Computational Information Systems 8: 2 (2012) Table 4: File read and write performance in different protocol architecture Write Rate Test (Gigabit LAN) Read Rate Test (Gigabit LAN) Group1 Group2 Group3 4 Conclusions The present study made an in-depth and detailed analysis of the current application of common cloud storage design ideas, including the implementation method and its performance advantages and disadvantages. It also focused on resource sharing capability and rapid low-cost building capacity, and correspondingly proposed a lightweight cloud storage engine design (LiDFS) which is particularly suitable for the storage of multimedia resources. The design was based on the current mainstream open source cloud storage system, particularly on cloud storage engine architecture, communication protocols, synchronization mechanism, and positioning mechanism of block storage files which enhance the design. Finally, a simulation environment for the comparison test showed the characteristics of the engine and its suitable application scenarios. However, the experiment on the storage system was based mainly on test results under normal circumstances. The following issues also need to be considered: engine performance in a production environment, reliability and exceptional cases in the storage node, and the performance of the storage engine. The proposed storage mechanism and other groups in the present two strategies have their advantages and disadvantages. The application is selected automatically based on file size and the rules of the second strategy. Our future work will focus on the adaptive strategy of

12 694 C. Shu et al. /Journal of Computational Information Systems 8: 2 (2012) the present mechanism. References [1] [2] EMC Corporation. The VMWare software paekage [EB/OL]. vmware.com [3] Cambridge Univ.Xen [EB/OL]. SRG/netos/xen [4] RedHat Cloud Computing [EB/OL]. solutions/cloud/ [5] Diane Barrett, Gregory Kipper. Visions of the Future: Virtualization and Cloud Computing. Virtualization and Forensics, 2010, pages [6] P. Padala, X. Zhu, Z. Wang, S. Singhal, and K. G. Shin, Performance evaluation of virtualization technologies for server consolidation, HP Techinal Report HPL R1, [7] Peter Zornig. Statistical simulation and the distribution of distances between identical elements in a random sequence. Computational Statistics & Data Analysis, Volume 54, Issue 10, 1 October 2010, pages [8] Chen Xiaojun, Zhang Jing, Li Junhuai, Li Xiang. Resource reconstruction for optimal utilization in virtual computing systems. Journal of Computational Information Systems, Vol. 7, p , [9] Virtualization Technology 2010, (accessed on ). [10] Thomas Olzak, Jason Boomer, Robert M. Keefer, James Sabovik. Integrating application and presentation virtualization (Terminal Services). Microsoft Virtualization, 2010, pages [11] Hao Zhang, Chuohao Yeo, Kannan Ramchandran. Rate efficient remote video file synchronization, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, 19 April 2009 to 24 April [12] Shiping Chen, Surya Nepal, Jonathan Chan, David Moreland, John Zic. Virtual Storage Services for Dynamic Collaborations. 16th IEEE International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE 2007), 18 June 2007 to 20 June [13] Likun Liu, Yongwei Wu, Guangwen Yang, Weimin Zheng. ZettaDS: A Light-weight Distributed Storage System for Cluster, The Third ChinaGrid Annual Conference (chinagrid 2008), 20 August 2008 to 22 August [14] Kyriacos Talattinis, Aikaterini Sidiropoulou, Konstantinos Chalkias, George Stephanides. Parallel Collection of Live Data Using Hadoop, th Panhellenic Conference on Informatics, 10 September 2010 to 12 September [15] An Qin, Dandan Tu, Chengchun Shu, Chang Gao. XConveryer: Guarantee Hadoop Throughput via Lightweight OS-Level Virtualization, 2009 Eighth International Conference on Grid and Cooperative Computing, 27 August 2009 to 29 August [16] The Google File System. [17] MongoDB. [18] Alberto Sanchez, Maria S. Perez. A high performance suite of data services for grids. Future Generation Computer Systems, 26 (2010), [19] YANG Jun, ZHAO Guo-hua, WANG Ke-jia, SONG Jun-de. A modern service-oriented distributed storage solution. The Journal of China Universities of Posts and Telecommunications [20] Hadoop.