Cloud-based Resource Scheduling Management and Its Application - With Agricultural Resource Scheduling Management for Example

Transcription

1 Cloud-based Resource Scheduling Management and Its Application - With Agricultural Resource Scheduling Management for Example Cloud-based Resource Scheduling Management and Its Application - With Agricultural Resource Scheduling Management for Example 1 CHEN Ying, *2 HUANG Xiao-Ying 1 ZheJiang A & F University, @qq.com 2 ZheJiang A & F University, [email protected] Abstract Cloud Computing has become one of the most popular technologies. However, to truly exert its advantages, resource scheduling technology, the core of Cloud Computing application, is the key technology for the wide application, system performance improvement and resource integration of Cloud Computing. Based on the introduction to the classification of cloud-based resource scheduling and its program analysis, this paper is to take agricultural resource scheduling management for example and build the system architecture of cloud-based agricultural resource scheduling management, which provides certain reference value for cloud-based resource scheduling management, especially for the further analysis and study of the application of Cloud Computing to the agricultural field. Keywords: Cloud Computing, Resource Scheduling, Agricultural Resource Scheduling Management, System Architecture of Agricultural Resource Scheduling Management Introduction Since Google Inc. put forward the concept of Cloud Computing, it has attracted widespread attention and developed rapidly[1]. The pioneers of Cloud Computing-- Google, Microsoft, IBM, Amazon, etc, have already launched their own Cloud Computing platforms and solutions in Cloud Computing fields[1-3]. Sun Microsystems, Apple, Intel, HP, Dell, Yahoo, etc, also entered the Cloud Computing market in succession, and have made significant achievements in Cloud Computing field. However, the development potential of domestic Cloud Computing market has drew even more attention. China Mobile, China Telecom and some other companies have made breakthroughs in Cloud Computing applications; in the meanwhile, Rising, Trends, Kaspersky, McAfee, Symantec, Jiangmin, Panda, Kingsoft, 360 Security Guards, etc, also have achieved considerable progress in Cloud Security solutions[1-3]. Cloud Computing resource scheduling[4] is the key technology of large scale application, system performance improvement and energy conservation and emission reduction. The question how to manage dynamically and distribute effectively the virtual shared resources in Cloud Computing data center according to users demand, and how to improve the efficiency in the use of resources to provide convenience for wide application of Cloud Computing has now become the key of the study.[4-5] 1. Cloud-based Resource Scheduling Management Cloud Computing resources scheduling indicates that N isomerism available resources are allotted to M independent application tasks, in order to make it fully attain effective resources in the shortest possible time[6]. The aims are fast searching, mass calculation and mass storage. While the important resource scheduling problems to be solved include resource monitoring, dynamic scheduling, deployment and maintenance, etc. 1.1 The Classifications of Cloud Computing Resources Scheduling At present, Cloud Computing resources scheduling has the following several main classifications: According to scheduling method, it can be classified into resource layer scheduling and application layer scheduling[7]. The resources layer scheduling is the unified management of resources through Advances in information Sciences and Service Sciences(AISS) Volume5, Number4, Feb 2013 doi: /AISS.vol5.issue

2 virtualization technology, and the wide resources scheduling method of mapping based on the principle of optimization distribution of tasks and resources; Application layer scheduling is making tasks and resources decompose into small node, and conducting unified scheduling and management through the scheduling control center. The most typical application layer scheduling method is the Map/Reduce algorithm put forward by Google. [7-9] From the angle of data center application, it can be classified into the following five categories: on-demand scheduling, online rent scheduling, optimization target scheduling, load balance scheduling and energy scheduling. [7-9] 1.2 The Program Analyses of Cloud Computing Resources Scheduling At present, Google[10], Microsoft[11], Amazon[12], VMware[13], IBM [14]and other companies have launched their own Cloud Computing program, and have done certain explorations and applications in Cloud Computing resources scheduling[15]. Google[2,10], as one of the initiators of Cloud Computing, based on the MapReduce, they designed the GFS file system, distributed storage system Megastore, distributed structured data table Bigtable, distributed lock service Chubby, distributed computing programming model and distributed monitoring system MapReduce Dapper, etc. However, due to the excessively closely combination with their own product development, there are many restrictions in use, such as only supporting Python and Java language, the Web application based on Django architecture, and so on. Amazon[2,11] is currently believed to be one of the most successful manufacturers in promoting Cloud Computing application. Their platform is safe-distributed and decentralized, and the bottom architecture Dynamo stores a lot of customer service data in the way of key/value. Moreover, on the basis of it, Amazon has constantly done technical innovations so that it has developed a series of services, such as elastic calculation cloud EC2, Simple storage service S3, Simple database service Simple DB, Simple queue service SQS, elastic MapReduce services, content delivery service CloudFront, electronic business service DevPay and FPS, etc. Microsoft's Azure platform is mainly for software developers[2,12]. Windows Azure clouds operating system offers a variety of computing and storage services and, on that basis, AppFabric and SQL Azure respectively provide cloud infrastructure services and database service. Different from the other programs, Azure considers the function of the local circumstances in Cloud Computing program, and the program of Azure can still work in local circumstances when off-line. VMware is the main supplier of server virtualization[2,13], it works through the use of distributed virtual machine and centralized virtual machine for data center dynamic allocation management, whose main work is to promote the resource utilization efficiency through virtualization, dynamically migrate virtual machine and Disaster Recovery, etc, but care less about the resources dynamic scheduling management. The core scheduling of IBM Cloud Computing is based on Hadoop MapReduce framework[2,14], whose basic platform is to open source Xen virtual machine Linux platform and Hadoop cluster platform. It adopts IBM Tivoli network resources monitoring and WebSphere network services, and mainly relies on the virtual computing technique. The Cloud Computing solutions of the companies mentioned above are all based on Private Cloud. The current Hadoop MapReduce applicable for sea quantity information processing and small computing platform Eucalyptus are both open source Cloud Computing solutions. In fact, many other companies, including Google, IBM, have adopted the design thought of MapReduce on the basic architecture. [2,16] 2. Urgency of agricultural resource management The balance of supply and demand of agricultural products and price stability concerns the development of agricultural economy[17], and even the people s daily life and social stability, which really makes the difference. However, the ups and downs of agricultural products prices have appeared in recent years at home and abroad. Phenomena[18], such as low price of vegetables injuring farmers, high price hurts people, hard selling and hard buying, often occurred. In 2009, Banana Event in Guangxi and Garlic Event in 192

3 Shandong happened, and also Hainan chili and the north Chinese cabbage in 2010; Vegetable was sold at cut-throat price in many areas in 2011; At the beginning of the year 2012,the price of eggs from 2 yuan per jin, rising into rocket egg now. Agricultural products prices look like roller coaster, ups and downs of violent fluctuation alternate, causing serious waste of the limited agricultural resources, bringing a great loss to agricultural development and farmers, which has a serious impact on people s daily life and social stability, highlighting the real severity of agricultural macroscopic management with China as a big country for its large population and agriculture. The long agricultural production cycle, the big market, the wide links, and the dynamic change of supply and demand relationship make it difficult to obtain the accurate data[19]. In addition, our country agricultural informatization system is not complete, leading to the serious unbalance of each link in the agricultural production, circulation and consumption information, especially producers information mostly depends on the neighbor's hearsay and local TV program, so the information supply channel is insufficient, and the regional information limitation is obvious. But cloud computing has advantages in solving the problems of agricultural products dispersion, timely and transparent information of agricultural resources and rationality of agricultural prices, which accelerates the diversion from agriculture information technology to cloud computing. The application of cloud computing can not only solve the problem of dispersion of agricultural production and information limitation of the producers, but also can save a lot of cost of hardware, software and maintenance personnel[20]. Meanwhile, it can also timely collect and release the information of the demand and supply of agricultural products, which makes up the serious dispersion of agriculture, small production scale, space and time variation, quantitative and scale difference, and low stability and controllability. Cloud-based agricultural resource scheduling management is supposed to include the system architecture, key technology and scheduling algorithm, dynamic scheduling management and its implementation, as well as resource monitoring, deployment and maintenance. This paper is to study the system architecture alone in order to shed some light and provide a basic framework and significant research basis for the further study of agricultural resource scheduling management, such as key technology and scheduling algorithm, dynamic scheduling management and its implementation, and resource monitoring. 3. Cloud-based agricultural resource scheduling management 3.1 Technical architecture of cloud-based agricultural resource management The technical architecture of cloud-based agricultural resource management is divided into three layers application layer, service layer, and physical layer[21]. For the concrete architecture diagram, see Figure 1. Application layer: namely the cloud, in order to ensure a relatively stable price, farmers, producers, operators, consumers, and relevant government departments can be both demanders and suppliers to the cloud. In other words, they can put what they demand or what they own in the cloud for others to take and meanwhile obtain what they need from other cloud users. In the cloud, farmers and producers can appropriately adjust their agricultural planting in accordance with the market demand; consumers can reveal their needs in time to the market and producers; and based on the information provided by consumers and producers, the government can improve agricultural policies and adjust the structure of agricultural products, giving guidance to the farmers, producers and operators for more scientific planting and management. Service layer: it mainly refers to the resource scheduling center, including cloud service management, resource management, resource scheduling, data and repository. Cloud service management provides the corresponding service interface for more requests and at the same time ensures data preparation and security via cloud security management. Resource management includes lifecycle management, image deployment and management, task management, and fault detection and recovery, while resource scheduling includes scheduling request, scheduling algorithm and scheduling management. Physical layer: it mainly refers to the physical resources, including such things as computer, server, network facility, database, and software. 193

4 Figure 1. Architecture Diagram of Cloud-based Agricultural Resource Scheduling Management 3.2 Strategy analysis of cloud-based agricultural resource scheduling management When users in the cloud have a request, they can issue it via the cloud server. And through the user authentication (such as the geographical location) in the scheduling center and the requested service features (such as the quantity and quality requirements), the cloud server separates the request into small tasks and submits them to the appropriate data centers by virtue of such ways as virtual management and data center management. Then the data centers submit those small tasks to a given scheduling domain which performs a certain scheduling algorithm and requests resource allocation. Meanwhile, as soon as the scheduling algorithm feeds back the information of available resources, the user can begin to use the resources. For the scheduling strategy, see Figure 2. Figure 2. Strategy Diagram of Cloud-based Agricultural Resource Scheduling Management 194

5 Suppose there is a set of servers in the architecture S= {S 0, S 1, S 2 S n-1 }, P (S 1 ) refers to the weight of Server S 1, T (S 1 ) refers to the current connection count of Server S 1, and ServerTable[] is a hash table with 256 Buckets. For the data consistency, the algorithm can be done according to the following procedures: n= ServerTable[hashkey(dest_ip)]; if (n==0)or (P(n)==0) or (T(n)>2*P(n)) return NULL; for(m=0;m<n;m++) {If (P(n)>0 ) {For (t=m+1;t<n;t++) {if(p(s m )*T(S m )>P(S t )*T(S t )) m =t; Return(dest_ip*S m )&HASH_TAB_MASK; }}} 4. Conclusion Cloud computing has started its application and exploration in such areas as web searching, scientific computing, virtual environment, energy and biological information. Its core cloud resource integration and scheduling, is the key technology for its wide-scale application, system performance improvement as well as energy conservation and emission reduction. Currently, progress has been made in the study of the architecture and algorithm, but its application to a certain practical field is open to discussion. On the other hand, the effective utilization of the agricultural products is the key issue that China s agriculture has been focusing on. Therefore, reasonable resource scheduling can be a good solution to the existing problems in the agricultural resources; moreover, it is of great significance to utilization efficiency, energy conservation, resource sharing and operating cost reduction, thus deserving further study. 5. Acknowledgement Fund Project: Science Foundation by Ministry of Education of China ( 12YJA870008), Foundation by Education Department of Zhejiang Province of China (Y ) 6. References [1] Luis M. Vaquero, Luis Rodero-Merino, Juan Caceres, Maik Lindner, "A break in the clouds: towards a cloud definition," ACM SIGCOMM Computer Communication Review, Vol. 39, No. 1, pp. 50~55, [2] LIU Peng,Cloud Computing(Second Editon), Publishing House of Electronics Industry,China,2011. [3] LIU Wan-jun,ZHANG Meng-hua,GUO Wen-yue, " Cloud Computing Resource Shedule Strategy Based on MPSO Algorithm",Computer Engineering,Vol.37,No.11,pp.43~48,2011. [4] TIAN Wen-hong, HONG Yong, Cloud Computing Resource Scheduling Management, National Defense Industry Press,China,2011. [5] FANG Jin-ming, "Decision System of Virtual Resources Scheduling in Cloud Computing Environment", Computer Measurement & Control, Vol. 19, No. 12, pp.3145~3148, [6] ZHAO Jian-feng,ZENG Wen-hen,LIU Miu,LI Guang-ming,"A model of Virtual Resource Scheduling in Cloud Computing and Its Solution usin g EDAs", JDCTA: International Journal of Digital Content Technology and its Applications, Vol. 6, No. 4, pp. 102 ~ 113, [7] SUN Rui-feng,ZHAO Zheng-wen, "Resource Scheduling Strategy Based on Cloud Computing", Aeronautical Computing Technique, Vol.40,No.3,pp.103~105,2010. [8] CHEN Kang, ZHENG Wei-Min, "Cloud Computing: System Instances and Current Research", Journal of Software,Vol.20,No.5,pp.1337~1348,

6 [9] Wickremasinghe B,et al, "CloudAnalyst: A CloudSim-based Tool for Modelling and Analysis of Large Scale Cloud Computing Environments",Proceedings of the 24th IEEE International Conference on Advanced Information Networking and Applications(AINA), pp.20~23,2010. [10] Armbrust Michael, Fox Armando, Griffith Rean, Joseph Anthony D., Katz Randy H., Konwinski Andrew, Lee Gunho, Patterson David A., Rabkin Ariel, Stoica Ion, Zaharia Matei, "Above the clouds: A berkeley view of cloud computing," EECS Department, University of California, Berkeley, Tech. Rep. UCB/EECS , [11] S.E.Dowd,J.Zaragoza,J.R. Rodriguez,M.J.Oliver,and P.R.Payton, "Windows.net network distriuted basic local alignment search toolkit",bmc Bioinformatics,Vol.6,No.93,2005. [12] Erik Brynjolfsson, Paul Hofmann, John Jordan, "Cloud Computing and Electricity: Beyond the Utility Model," Communications of the Acm, Vol.53,No.5,pp.32-34,2010. [13] WANG Chun-hai. The Typical Application Guideline of VMware Workstation and ESX Server, China Railway Publishing House, China,2011. [14] LIU Hai, HE Chao-bo,TANG Yong, HUANG Shi-ping, "Research and Applicatoin of Service-Or iented Scholar Cloud Platform", JCIT: Journal of Convergence Information Technology, Vol. 7, No. 5, pp. 333 ~339, [15] Rajkumar Buyya, Chee Shin Yeo, Srikumar Venugopal, James Broberg, Ivona Brandic, "Cloud computing and emerging IT platforms: Vision, hype, and reality for delivering computing as the 5th utility," Future Generation Computer Systems-the International Journal of Grid Computing-Theory Methods and Applications, Vol.25, No.6,pp ,2009. [16] WANG Geng-jia, ZHANG YUAN-Biao, CHEN Jia-Wei, "A novel algorithm to solve the vehicle routing problem with time windows: Imperialist competitive algorithm," Advances in Information Sciences and Service Sciences, Vol.3,No.5, pp.108~116,2011. [17] DU Liang-sheng,ZHOU Bin,DUAN Peng-fei, "ON Interaction Mechanism of Agricultural Product Price and Inflation ",Economic Theory and Business Management,No.6,pp.23~33,2012. [18] Nicholas Apergis,Anthony Rezitis, "Mean Spillover Effects in Agricultural Prices:The Case of Greece",Agribusiness,Vol.19,No.4,pp.425~437,2003. [19] ZHANG Shi-rui,ZHENG Wen-gang,SHEN Chang-jun,XING Zhen."National Engineering Research Center for Information Technology in Agriculture",Computer engineering and design,vol.33,no.5,pp.1816~1820,2012. [20] PENG Xiu-yuan, WANG Xin, LU Chuang, XUAN Kai, "Application of Cloud Computation in the Agriculture", Agriculture Network Information,No.2,pp.8~10,2011. [21] LEI Lei, "Towards a High Performance Virtual Hadoop Cluster", JCIT: Journal of Convergence I nformation Technology, Vol. 7, No. 6, pp. 292 ~ 303,