Storage and Data processing for Wireless Sensor Network integrated with Cloud Computing

Transcription

1 International Journal of Research In Science & Engineering e-issn: Special Issue: Techno-Xtreme 16 p-issn: Storage and Data processing for Wireless Sensor Network integrated with Cloud Computing Mr.Milind Tote 1,Dr. M.B. Chandak 2 1 Research Scholar, Computer Technology, RGCER,Nagpur, milind_tote@yahoo.com 2 Professor, Computer Science &Engineering, RCOEM, Nagpur ABSTRACT WSNs are being utilized in several areas like healthcare, military target tracking and surveillance, natural disaster relief and so forth. These sensors provide various useful data which can be used to monitor and control the environment in which they are deployed. The amount of data in a sensor network is huge, heterogeneous and multidimensional in nature. To store and process these data, high amount of storage and computation power is required. Unlike traditional networks, a WSN has its own design and resource constraints. This paper proposes the idea of integrating wireless sensor network with cloud base storage and computing. The distributed databases to store sensor data and MapReduce programming model for largescale sensor data parallel processing. Hadoop Distributed File System (HDFS) and HBase are used for sensor data storage and Hadoop MapReduce is used for data processing application execution framework. Keywords: Wireless Sensor Network, cloud computing, parallel data processing INTRODUCTION The important features of large-scale WSN include: Involving large number of sensor nodes, typically more than thousands, Generating large volume of sensor data and Long-term surveillance for over one year. Largescale WSN demands high requirements of data storage, data analysis and processing. Due to the limitation of hardware design, each sensor node in a WSN is generally a simple resource-limited independent system. It means that lower communications, limited computing power, limited storage capacity, and limited battery energy. There is a high performance network aggregation node or base station, and data collected by ordinary nodes is passed to the base station in the way of self-organizing and multi-hopping. Then, the sensor data is analyze and processed by the base station, or even simply processed by ordinary nodes. In current large-scale WSN, how to store and process sensor data becomes the key factors which constrain and affect the application of WSN. Complex analysis of huge amount of sensor data demand high processing power and large storage memory space. These constraints impose forwarding the sensor data from the WSN to powerful backend machines, where they are stored and processed for different analysis. Two varieties of data processing are possible: The sequential and the parallel. When information is passed through several consecutive stages of processing, it is said that it goes through a sequential data processing chain. The parallel processing consists then in dividing the application into smaller processing stages, so called tasks, and execute them concurrently on several machines. As a result, the application is completed more quickly. The integration of wireless sensor network with cloud based approach by building an external Cloud Data Center to provide computing service and storage service for the large amount of sensor data. The objective of the system is to compensate the disadvantages of sensor nodes limited computing and storage capability. In this architecture, sensor data is

2 transferred to cloud back-end for further storing and processing through the Sink node and data gateway. 2. LITERATURE REVIEW In traditional data management in WSN, query-based access method is widely used. Data management technologies consist of data aggregation, data storing, data querying, and data accessing, which are also the core of WSN. In particular, there are mainly three data storing strategies. 1) Centralized storing: In which data collected by nodes is transmitted to base station for storing, accessing and processing, while there is a large node communication overhead. 2) Distributed storing and indexing: In which data is distributed in network and a data index is built for high efficient query. 3) Locally storing: In which data is stored in sensor node, so there is a lower communication overhead, also lower query efficiency. The MapReduce model has been proposed by Google in 2004 for parallel processing of huge data. The parallelization is done by splitting an application into smaller parts that can be executed simultaneously on different machines of a cluster. The task is the smallest unit of code and can be either of two types: Map or Reduce. A job is a group of tasks sharing common characteristics, e.g., maximum number of machines a job can be executed on. Each machine in the cluster is assigned one or more Map and Reduce tasks constituting a predefined job. The input data are sets of <key, value> pairs that are split into equally sized units to be processed by the Map tasks. Once all Map tasks are completed, their output <key, value> pairs are split into units for Reduce tasks ensuring that all the values of the same key go to the same unit. The Reduce tasks are then assigned dynamically to machines such that each Reduce task must process all values for the same key. If a machine does not complete a task within a reasonable time, a failure is assumed and a second attempt is launched to re-execute it. The Oracle real application cluster is an option in the Oracle database 11:g standard edition. The parallel processing is realized by decoupling the Oracle applications from the Oracle database. The servers hosting application processes are physically decoupled from the databases that store the data. The servers are connected via a local area network to the external world from which they receive user applications. An additional private network is required for interconnecting the sever machines and allowing inter-messaging. The clustered database is a single database that can be accessed by multiple applications, each one running on a separate server in the cluster. This physical decoupling between the machines gives the cluster the advantage of expanding and adding new resources with no down time, which is not the case for Hadoop. HBase implemented in JAVA, and Hypertable implemented in C++ are two Bigtable-alike systems build on top of Hadoop MapReduce programming model. The stored data is organized into tables, rows and cells. Each cell in the table is indexed by a row key, column key and a timestamp. The timestamp allows a cell to contain multiple versions of the same data. An iterator-like interface is available for scanning through columns. While Hypertable allows different logical column families to be together physically, HBase allows it for the columns of a single family. IJRISE editor@ijrise.org

3 3. SYSTEM ARCHITECTURE 3.1. General Overview The major advantages of using Cloud Computing include (1) storing large amount of data from anywhere without worrying about maintenance of the data, (2) transfer the data from one server to another server with no need to worry about server types and (3) there is no change in the performance if the amount of traffic rises. Form the viewpoint of service computing, there are also some advantages in that cloud service could be hired by WSN system, as a remote and back-end assistant to WSN. Fig.1.Interconnection between WSN and cloud data center 3.2. SYSTEM ARCHITECTURE The objective of the integration of WSN with Cloud is also to realize remote management platform for data storage that leverages powerful cloud computing technologies to provide excellent data scalability, rapid visualization, and user programmable analysis. It is designed to support long-term deployments of wireless sensors network through a simple Data Management API. The detailed system architecture is shown in Fig. 1. In this figure, there are three entities: WSNs, Cloud Data Center and Users. It is a general architecture of interconnection of several WSNs through Internet. The server node located in the edges is considered to be a data gateway, which receives data from the Sink node. Cloud-based data storing and processing center is deployed. The huge amount of data collected by the sensors can be processed, analyzed, and stored using the computational and data storage service of the cloud. In this architecture, the sensor data can be efficiently shared by different users and applications under flexible usage scenarios. Each user can access a subset of the sensors, and run a specific application, and search the desired sensor data, for example, through a web-based interface

4 Fig.2. Hadoop-based Cloud Provider 3.3. Data storage model-hdfs and HBase: The Apache Hadoop project develops open-source software for reliable, scalable, distributed computing. The Hadoop software offers Hadoop Distributed File System (HDFS), a distributed file system that provides high-throughput access to application data. HBase is an open-source, scalable, distributed database that supports structured data storage for column-oriented large tables. It provides Bigtable-like capability on top of Hadoop and HDFS, and easy to use Java API for client access (reads and writes). It is used for hosting of very large tables atop clusters of commodity hardware and to facilitate random, real-time read/write access to your Big Data. Figure 2 shows the detail components of Hadoop-based Cloud provider, from which we know that HBase is a distributed database, and each node is managed by the master node Data processing framework-hadoop MapReduce: MapReduce is a parallel programming paradigm successfully used by large Internet service providers to perform computations on massive amounts of data. After being strongly promoted by Google, it has also been implemented by the open source community through the Hadoop project. The key strength of the MapReduce model is its inherently high degree of potential parallelism. In Hadoop MapReduce framework, the computing is divided into two stages: Map and Reduce. Map processes a key/value pair to generate a set of intermediate key/value pairs, and Reduce merges all intermediate values to form the final output. HBase also uses Hadoop MapReduce to process the large-sale data stored in HBase, and provides native Java API for database operation for MapReduce Job. 3.5 Interconnection of cloud and WSN- The interconnection is implemented by data gateway. Gateway is a client which accesses remote cloud service through Java API interface for data writes. The gateway receives data from the Sink node and then writes data into local storage as a backup, and a daemon thread is in charge of periodically writing data to Cloud Data Center. As it is shown by Fig. 2, the I/O Controller module is designed for interaction between gateway and cloud. So as to the Web Server module, it is used for provide friendly web interface for users to access sensory data, and submit query and processing request job

5 4. CONCLUSION Based on a survey of current research status on large-scale WSN data processing and management, this paper proposes a WSN sensor data processing system using emerging cloud computing approach. High performance data processing centre is adopted as a solution to assistant the lack of sensor nodes data storage and data processing capability. This novel hybrid architecture enables the collection, processing, sharing, accessing and searching of large amounts of sensor data. 5. REFERENCES [1] Liu, Y., G. Zhou, J. Zhao, G. Dai, X Li, M. Gu, H. Ma,L. Mo, Y. He, J. Wang, M. Li, K. Liu, W. Dong and W. Xi, Long-term large-scale sensing in the forest: Recent advances and future directions of GreenOrbs. Front. Comput. Sci. China, 4(3): [2] Dean, J. and S. Ghemawat, MapReduce: Simplified data processing on large clusters. Commun. ACM,51(1): [3] Gaynor, M., S. Moulton, M. Welsh, E. LaCombe, A. Rowan and J. Wynne, Integrating wireless sensor networks with the grid. IEEE Internet Comput., 8(4): [4] Li, J.Z., J.B. Li and S.F. Shi, Concepts, issues and advance of sensor networks and data management of sensor networks. J. Software, 14(10): [5] Dash, S, K., Sahoo, J, P., Mohapatra, S., and Pati, S, P., Sensor-cloud: assimilation of wireless sensor network and the cloud, Advances in Computer Science and Information Technology. Networks and Communications- Second International Conference, CCSIT 2012, Bangalore, India, January 2-4, 2012, Vol. 84, pp , [6] Simitci, A., Storing and Processing Sensor Networks Data in Public Clouds, UWB CSS 600, 2012 [7] Rajesh, V., Pandithurai, O., and Mageshkumar, S., Wireless sensor node data on cloud, IEEE International Conference on Communication Control and Computing Technologies (ICCCCT), pp: , 2010.