Indian Journal of Science The International Journal for Science ISSN EISSN Discovery Publication. All Rights Reserved

Transcription

1 Indian Journal of Science The International Journal for Science ISSN EISSN Discovery Publication. All Rights Reserved Perspective Big Data Framework for Healthcare using Hadoop Publication History Received: 25 December 2015 Accepted: 12 January 2016 Published: 1 February 2016 Citation Bhuvaneswari Ragothaman, Elsa Jose, Surya Prabha M, Sarojini Ilango B. Big Data Framework for Healthcare using Hadoop. Indian Journal of Science, 2016, 23(78), Page121

2 Big Data Framework for Healthcare using Hadoop Bhuvaneswari Ragothaman 1, Elsa Jose 2, Surya Prabha M 3, Dr. B.Sarojini Ilango 4 1,2 MPhil Scholar, Avinashilingam University, Coimbatore 3 Ph.D Scholar, Avinashilingam University, Coimbatore 4 Assistant Professor, Avinashilingam University, Coimbatore Abstract The technological advancements prevailing today make it possible to sense, collect, transmit, store and analyze the voluminous, heterogeneous medical data. This data could be mined to provide valuable and interesting patterns. However, extracting knowledge from big data poses a number of challenges that need to be addressed. In this paper we propose big data architecture for processing medical data. To handle this large volume of data, Big Data s Hadoop tool can be used. The Apache Hadoop Framework for processing distributed healthcare database is also discussed. Keywords: Big Data, knowledge extraction, Hadoop Frame work. Introduction Thanks to the advancements in technology the hospitals are producing voluminous, heterogeneous medical data. The data from healthcare is predicted to increase to 25,000 petabytes in 2020 from current estimation of 500 petabytes. This data can be computationally analyzed to provide useful knowledge and information. Imagine a scenario where it is possible to track an individual s health by monitoring the vital health parameters like E.C.G, blood pressure, pulse rate etc., which could be stored and analyzed in real-time. It means that every one of us is having an up- to- date health document that contains all health-related information and which is getting updated on a realtime basis. This voluminous (volume), heterogeneous (variety) data that changes from time to time (velocity) are the Big Data. The analysis of the large amount of data termed as Big Data, reveals useful patterns, trends and associations between the features that can be used in the identification of disease, for diagnosis and for treatment. Big Data analytics in health care improves clinical decision making and in predicting future with the current data. Research shows that the analysis of data improves the quality of healthcare by providing high knowledge of information [1].Analysis of health care data analytics play a vital role in the early identification of diseases. The early detection of fatal diseases reduces health care cost and improves quality of treatment. The analysis of medical data is always a challenging process considering the high volume, complexity and heterogeneity. The analysis of real time data poses an additional challenge on speed of processing. These problems could be sorted out by using few tools which are available in Big Data and Big Data Analytics. Characteristics of Big Data Big Data is characterized by 4V s as Volume, Variety, Velocity and Veracity [3]. 1. Volume consists of large amount of data obtained from various sources with different in size and structure like flat files, tables, graph structures, image data. Large amount of data obtained from the source overcome the problem of data hidden and missing values. 2. Variety consists of different data types of data obtained from source via smart cards, cell phones, files, internet, etc. Here cell phones data and file data can be images, tables and flat files which are semi structured and structured. Data received from internet are unstructured. 3. Velocity consists of frequency of incoming data needs to be processed. Machine generated mails or messages sent to the mailboxes or mobiles phones while the usage of credit cards, which are high in velocity. 4. Veracity denotes the trustworthy of data. Data must remain consolidated, cleaned, and consistent and updated which helps to make the right decisions [4]. Page122

3 Volume Velocity Big Data Variety Veracity Fig1: 4V s of Big Data Role of Big Data in Healthcare In healthcare there is a huge amount of data generated from various departments like research data, clinical data, medical insurance claim, drug details, electronic medical record (EMR) and etc. This leads to enormous amount of data which is said to be Big Data, which helps in overcoming the problems like storage, processing and management. There is various advancement in healthcare due to Big Data analytics. With the help of analysis, disease could be identified easily and drugs and treatment can be given efficiently. And side effects of the treatment could be avoided with this process. This also minimizes the cost for the treatment. The quality of the treatment and the quality of hospital could be improved with the examination which is done by the Indian Medical Council HEALTHCARE DATA SET DRUG RESEARCH SOCIAL MEDIA TEST RESULT MOBILE APPS PATIENT RECORD EMR Fig 2: Healthcare Data Set Page123

4 Literature Survey Big Data and Hadoop are used in the enhancement of healthcare. Here are few techniques used in Hadoop for development of healthcare. In paper [1], Big Data Analytics and Hadoop s Map Reduce is used to detect Diabetics Mellitus from patient s blood serum and give effective treatment in the low cost. HDFS framework is used to do pattern matching in identifying the chances for the diabetics for the patient. In paper [6], proposes a smart health system assisted cloud and Big Data, along with various layers of architecture in collecting data, managing data and in application services of those data. Application Program Interface (API) is developed for developers and users friendliness. Big Data Architecture for Healthcare Big Data Architecture for healthcare consists of three layers as Data Collection Layer, Data Management Layer and Application Service Layer [6]. 1. Data Collection Layer: This layer performs data collection and preprocessing. Preprocessing is done by the adapter which is encapsulated in this layer. Data are obtained from various sources like Hospital, Internet and User Generated Data. Those data includes research data, clinical data, medical expense data and emotional data. Here adapter is used as the middleware which acts as raw data preprocessor. 2. Data Management Layer: This layer deals with the data management and it consists of Distributed File Storage (DFS) module and Distributed Parallel Computing (DPC) module. DFS deals with the massive storage of data along with those it deals with Data Description, Data Entity and Security check for data. DPC deals with real time analysis and off line analysis of data. 3. Application Service Layer: This layer consists of Application Program Interface (API), User Interface and Data Access. API is used for the developer and the user interface is used for end user in an efficient way to access the data. Figure: Big Data Architecture for Healthcare Apache Hadoop Framework An open source framework used to store and process Big Data in distributed systems with cluster of computers using simple programming models. There are more than 150 eco systems available in Hadoop like Hive, Pig, HCatalog, Map Reduce, HDFS, Zoo Keeper and etc. Here most of the eco system are used to store, analysis and process the large amount of data. Map Reduce and HDFS are Page124

5 the two main eco systems of Hadoop. Hadoop was developed by Google and facebook to handle a large amount of data generated and later it was taken by apache to handle the large amount of data in various fields. Hadoop framework consists of four modules as Hive 1. Hadoop Common consists of common utilities that support the other Hadoop modules. 2. Hadoop Distributed File System (HDFS) is a distributed file system that provides high-throughput access to application data. 3. Hadoop YARN is a framework for job scheduling and cluster resource management. 4. Hadoop Map Reduce is a YARN based system for parallel processing of large amount of data sets. Hive provides the data warehouse for Hadoop. Using SQL like language data sets could be summarized and queries could be used to analyses of those data set. Hive uses table structures to manipulate those data. Pig Pig is a language used for analyzing and processing of large data sets. It is done in two modes like Local mode and Hadoop mode. To run the script in Local mode no HDFS or Hadoop is required. It is run in local file systems. To run the script in Hadoop mode Hadoop installation is required. The code is run in the Hadoop virtual machine. Hadoop Distributed File System (HDFS) HDFS is a distributed file system designed to hold very large amount of data and this is highly fault tolerant. In the case of any system failure the loss of data is being minimized. HDFS replicates the data nodes to all the computers in the cluster and manages the data transfer. HDFS Architecture HDFS follows master slave architecture. HDFS consists of Name node, Data node and Blocks. Name Node: This acts as the master server and regulates client to access the files and manages the file system. It also executes file system operations such as renaming, closing and opening files and directories. Figure: HDFS Architecture Data Node: This node manages the data storage and perform read write operations of the file system as per the client request. According to the instructions given by the name node it also perform operations like block creation, deletion and replication of the blocks. Blocks: Data are stored in HDFS are divided into segments and stored in individual data nodes know as blocks. And the minimum amount of data that HDFS can read and write is called as Block Page125

6 Map Reduce Map Reduce works as job scheduler and resource manager. This divides the datasets into small blocks of tasks and uses distributed algorithms to group the computers in a cluster to process the task. This consists of two functions as Map() and Reduce(). Map() function acts as master node and divides the larger task into smaller sub tasks and distributes it to worker nodes and process result for the task. Reduce() function collects the results of all worker node task and produce a final result to the large task as the answer to the big query. HCatalog HCatalog presents a relational view of data. Data is stored in tables and these tables can be placed into databases. Conclusion The processing of accumulated healthcare data will improve the quality of healthcare. Identifying necessary patterns for diagnosis helps in improving future prediction and decision making. The Hadoop framework is used as the tool for analyzing the data. Hadoop framework and its eco system help in processing and producing the results effectively for the large amount of data or a big query. The application of technology in health care helps in early identification of diseases and helps to achieve treatment at low cost by storing and processing data. References: 1. Dr Saravana kumar N M, Eswari T, Sampath P & Lavanya S, Predictive Methodology for Diabetic Data Analysis in Big Data, Elsevier - 2nd International Symposium on Big Data and Cloud Computing (ISBCC 15) J.Archenaa and E.A.Mary Anita, A Survey Of Big Data Analytics in Healthcare and Government, Elsevier - 2nd International Symposium on Big Data and Cloud Computing (ISBCC 15) Aisling O Driscoll, Jurate Daugelaite, Roy D. Sleator, Big data, Hadoop and cloud computing in genomics, Elsevier - Journal of Biomedical Informatics Marco Viceconti, Peter Hunter and Rod Hose, Big Data, Big Knowledge: Big Data for Personalized Healthcare, IEEE Journal of Biomedical and Health Informatics, Vol 19, No 4 July IbrahimAbakerTargioHashem, IbrarYaqoob, NorBadrulAnuar, Salimah Mokhtar, AbdullahGani, SameeUllahKhan, The rise of big data on cloud computing: Review and open research issues, Elsevier Information Systems47(2015) Yin Zhang, Meikang Qiu, Chun-Wei Tsai Mohammad Mehedi Hassan and Atif Alamri, Health CPS: Healthcare Cyber-Physical System Assisted by Cloud and Big Data.IEEE Systems Journal Page126