Development of CEP System based on Big Data Analysis Techniques and Its Application



Similar documents
Development of Real-time Big Data Analysis System and a Case Study on the Application of Information in a Medical Institution

Integration of Hadoop Cluster Prototype and Analysis Software for SMB

On a Hadoop-based Analytics Service System

Cloud-based Distribute Processing of User-Customized Mobile Interface in U-Sensor Network Environment

UPS battery remote monitoring system in cloud computing

From Data to Insight: Big Data and Analytics for Smart Manufacturing Systems

Develop Total IT Service Monitoring System of Agentless Method for Total Management of based on Cloud Service Demand

A Study on Data Analysis Process Management System in MapReduce using BPM

A Divided Regression Analysis for Big Data

On site big data analysis system model to promote the competiveness of manufacturing enterprises

Home Appliance Control and Monitoring System Model Based on Cloud Computing Technology

The Development of an Intellectual Tracking App System based on IoT and RTLS

Extraction of Risk Factors Through VOC Data Analysis for Travel Agencies

Managing Cloud Server with Big Data for Small, Medium Enterprises: Issues and Challenges

Cloud Computing based Livestock Monitoring and Disease Forecasting System

Customized Efficient Collection of Big Data for Advertising Services

Chapter 7. Using Hadoop Cluster and MapReduce

An Approach to Implement Map Reduce with NoSQL Databases

BIG DATA IN BUSINESS ENVIRONMENT

Volume 3, Issue 6, June 2015 International Journal of Advance Research in Computer Science and Management Studies

Comprehensive Analytics on the Hortonworks Data Platform

Secret Sharing based on XOR for Efficient Data Recovery in Cloud

The emergence of big data technology and analytics

How to use Big Data in Industry 4.0 implementations. LAURI ILISON, PhD Head of Big Data and Machine Learning

INTERNATIONAL JOURNAL OF PURE AND APPLIED RESEARCH IN ENGINEERING AND TECHNOLOGY

Converged, Real-time Analytics Enabling Faster Decision Making and New Business Opportunities

TECHNOLOGY ANALYSIS FOR INTERNET OF THINGS USING BIG DATA LEARNING

Smart Manufacturing Systems Design and Analysis

Mobile Storage and Search Engine of Information Oriented to Food Cloud

Cyber Forensic for Hadoop based Cloud System

System Framework of Livestock Disease Forecasting based on Cloud

BIG DATA ANALYSIS USING RHADOOP

Hortonworks & SAS. Analytics everywhere. Page 1. Hortonworks Inc All Rights Reserved

WELCOME TO THE WORLD OF BIG DATA. NEW WORLD PROBLEMS, NEW WORLD SOLUTIONS

Designing and Embodiment of Software that Creates Middle Ware for Resource Management in Embedded System

Design of Big Data-based Greenhouse Environment Data Consulting System for Improving Crop Quality

Hadoop. MPDL-Frühstück 9. Dezember 2013 MPDL INTERN

Success factors for the implementation of ERP to the Agricultural Products Processing Center

Manifest for Big Data Pig, Hive & Jaql

International Journal of Advanced Engineering Research and Applications (IJAERA) ISSN: Vol. 1, Issue 6, October Big Data and Hadoop

GAIN BETTER INSIGHT FROM BIG DATA USING JBOSS DATA VIRTUALIZATION

Detection of Distributed Denial of Service Attack with Hadoop on Live Network

Component Based Development for Mobile Enterprise Application

Virtualization of Desktop for Improvement of Maintenance Cost Saving

How To Handle Big Data With A Data Scientist

Lecture 32 Big Data. 1. Big Data problem 2. Why the excitement about big data 3. What is MapReduce 4. What is Hadoop 5. Get started with Hadoop

Performance Comparison Analysis of Linux Container and Virtual Machine for Building Cloud

Design of a Web-Services Based Spam Filtering

The 4 Pillars of Technosoft s Big Data Practice

BIG DATA IN SUPPLY CHAIN MANAGEMENT: AN EXPLORATORY STUDY

Hadoop and its Usage at Facebook. Dhruba Borthakur June 22 rd, 2009

Big Data Framework for u-healthcare System. Tae-Woong Kim 1, Jai-Hyun Seu 2.

Beyond Web Application Log Analysis using Apache TM Hadoop. A Whitepaper by Orzota, Inc.

Hadoop Evolution In Organizations. Mark Vervuurt Cluster Data Science & Analytics

BIG DATA CHALLENGES AND PERSPECTIVES

Development of Framework System for Managing the Big Data from Scientific and Technological Text Archives

Design of Electric Energy Acquisition System on Hadoop

RESEARCH ARTICLE Intelligent Forecast of Product Purchase Based on User Behaviour and Purchase Strategies using big data

A Database Hadoop Hybrid Approach of Big Data

A High-availability and Fault-tolerant Distributed Data Management Platform for Smart Grid Applications

Data Refinery with Big Data Aspects

How Companies are! Using Spark

A Conceptual Approach to Data Visualization for User Interface Design of Smart Grid Operation Tools

Research into a Visualization Analysis of Bigdata for the Decision Making of a Tourism Policy

The Big Data Ecosystem at LinkedIn Roshan Sumbaly, Jay Kreps, and Sam Shah LinkedIn

Generic Log Analyzer Using Hadoop Mapreduce Framework

Big Data Collection Study for Providing Efficient Information

Intelligent Decision Support System (DSS) Software for System Operation and Multiple Water Resources Blending in Water Treatment Facilities

A Medical Decision Support System (DSS) for Ubiquitous Healthcare Diagnosis System

R.K.Uskenbayeva 1, А.А. Kuandykov 2, Zh.B.Kalpeyeva 3, D.K.Kozhamzharova 4, N.K.Mukhazhanov 5

Concept Design of Testbed based on Cloud Computing for Security Research

A STUDY ON HADOOP ARCHITECTURE FOR BIG DATA ANALYTICS

Managing Healthcare Data for Improved Patient Care Using Crowdsourcing and Managed Services

Scalable Multiple NameNodes Hadoop Cloud Storage System

Social Big Data Analysis on Perception Level of Electromagnetic Field

Big data platform for IoT Cloud Analytics. Chen Admati, Advanced Analytics, Intel

A Study on the Collection Site Profiling and Issue-detection Methodology for Analysis of Customer Feedback on Social Big Data

QLIKVIEW DEPLOYMENT FOR BIG DATA ANALYTICS AT KING.COM

Big Data and Analytics in Government

A Framework for Data Warehouse Using Data Mining and Knowledge Discovery for a Network of Hospitals in Pakistan

Data Management in SAP Environments

Hadoop IST 734 SS CHUNG

Big Data Are You Ready? Jorge Plascencia Solution Architect Manager

How To Use Big Data In Education

Introduction to Hadoop HDFS and Ecosystems. Slides credits: Cloudera Academic Partners Program & Prof. De Liu, MSBA 6330 Harvesting Big Data

NTT DOCOMO Technical Journal. Large-Scale Data Processing Infrastructure for Mobile Spatial Statistics

Keywords: Big Data, Hadoop, cluster, heterogeneous, HDFS, MapReduce

The Application Method of CRM as Big Data: Focused on the Car Maintenance Industry

Fujitsu Big Data Software Use Cases

University Dedicated Next Generation ERP System

International Journal of Advance Research in Computer Science and Management Studies

Research Article Hadoop-Based Distributed Sensor Node Management System

Big Data Using Cloud Computing

How To Use Spagobi Suite

Distributed Framework for Data Mining As a Service on Private Cloud

Problem Solving Hands-on Labware for Teaching Big Data Cybersecurity Analysis

Smart Integrated Multiple Tracking System Development for IOT based Target-oriented Logistics Location and Resource Service

Keywords Big Data, NoSQL, Relational Databases, Decision Making using Big Data, Hadoop

Managing Data in Motion

Transcription:

, pp.26-30 http://dx.doi.org/10.14257/astl.2015.98.07 Development of CEP System based on Big Data Analysis Techniques and Its Application Mi-Jin Kim 1, Yun-Sik Yu 1 1 Convergence of IT Devices Institute Busan(CIDI), Dong-eui University, 176 Eomgwang-ro, Busanjin-gu, Busan, Korea {agicap, ysyu}@deu.ac.kr Abstract. In the rapidly business and managerial environment and marketplace, there are increasing needs for massive amount Big Data on a various corporate entity levels. In reality, however, access to Big Data use has been limited and not easy due to the high cost of analytics system. In addition, as real time analysis being emerged as an alternative of conventional data analytics, it requires to develop CEP-based system that analyzes data from the perspectives of real time processing event, rather than Hadoop-based one from the perspectives of batch processing and batch analytics. In this paper, CEP-based real time analytics system was developed to establish low-cost Big Data system for small-medium size hospitals. Another purpose is to analyze patient-oriented information and equipment data at hospitals in combination of hospital ERP system, thereby implementing the system that can provide systemic and efficient patient management and administrative management at hospitals and suggesting such verified effect. Keywords: Big data, CEP(Complex Event Processing), Real-time Analysis 1 Introduction With the development of IT technologies and the recent increase in data amount generated by individuals, governments and corporate entities; more interest in Big Data is growing. Big Data is not just any new concept, it refers to huge and massive amounts of data accumulated in variety of formats, in excess of the scope of data saving, management and analysis allowed by the previous database such as a Relational Database [1]. Big Data is currently generated by and used in many fields, and highlighted especially in health and medical fields. The increase in chronic and degenerative disease cases caused by the population aging, there have been much effort by health and medical fields in many studies, in order to use Big Data to reduce medical expenses, prevent contagious diseases, and raise the bar in quality of medical services. In the results, Big Data is proposed as effective alternatives to explore more efficient diagnosis and treatment, as well as to predict prognosis. It is estimated that there will be cost-saving effects of $300 billion annually, if Big Data is well applied in the US medical fields[2]. A McKinsey report also said about the added-value in medical ISSN: 2287-1233 ASTL Copyright 2015 SERSC

fields which is connected to allowing to save nation-wide medical expenses and to do innovative clinical trials. The purpose of this study is to verify the effectiveness of Big Data as a useful method for information-oriented health and medical fields, by developing Big Databased real time monitoring system in manners of real time analysis rather than previous batch-type analytics method, and then by developing the system where massive amounts of data generated by health and medical fields are used and analyzed on a real time basis and systemic service can be provided by medical institutions in more efficient manners in patient and hospital management. 2 Related Research 2.1 Hadoop Hadoop is a solution with its key focused on distributed processing technology, which is currently most favorable for Big Data processing. It is Java-based framework with Apache open source process to process massive data, using a relatively simple program model[3]. Hadoop is used as a core technology by Yahoo and Facebook, while being applied to many other companies own solution. Hadoop is composed of the distributed file system called HDFS (Hadoop Distributed File System) and the distributed processing system called MapReduce. The operation method is HDFS, but MapReduce is composed of the masters called Namenode and Datanode as well as multiple slaves in its structure. 2.2 CEP(Complex Event Processing) CEP is complex event processing technology to extract meaningful data in real time basis from events from various event sources, thereby performing the corresponding actions[4]. Event data herein refers to stream data, which are data of continuous massive inputs, with important time sequences and endless data. It is impossible to process and analyze such stream data in real time basis into a conventional Relational Database. CEP is an event data processing solution that can provide real time analysis of such stream data. That is, it is possible to do real time processing of hundreds/millions of various high speed event stream based on In-Memory without saving it to database, file or Hadoop. 3 System Compositions and Implementation CEP technology is currently emerging across business fields, as the simplest and strongest method to implement the real time business intelligence based on timely Copyright 2015 SERSC 27

analysis, providing new values including real time monitoring, with an early alarm and production field management by processing and analyzing various events. In this study, we aimed at providing a systematic and organized business environment for efficient patient management and administrative management at hospitals, by using CEP-based advantages in consideration of the low cost of Big Data and then establishing the Big Data-applicable real time analytics system in combination with hospital ERP systems with yet a insufficient number of cases. In addition, the main Adaptor and data publisher/customizing functions were implemented, to allow to identify and develop UI screens - depending on the needs of each hospital. Fig. 1. CEP-based real-time analysis system architecture. To be brief on the processing of the entire system, Legacy saves log details generated in its own process as in Fig. 2, while performing real time analysis processing as far as major tasks among such processes. Batch layer collects data through data aggregation and extracts data through analysis works. Depending on such extracted results, there may be either storage to middle repository or request for real time analysis. Real time processing layer processes the requests from Legacy and from batch layers, requesting for data as necessary for processing and saving the results to the middle repository. For such processed results, there will be monitoring system API calls. Dashboard visualizes the data depending on the user s request, and then saved the Dashboard-processed data to the middle repository. Fig. 2. The entire system process definition. 28 Copyright 2015 SERSC

Fig. 3 is the screenshot of monitoring analyzed of all data on Output Adaptor settings on Dashboard. Through filtering functions, Server/Adaptor/Event/Hour can also be set up in order to analyze necessary parts alone. Fig. 4 is the screenshot of analysis, when filtering Server selected as 210.109.9.177, Output Adaptor selected as Event, Event selected as event_10 and Hour selected as 07:00 ~ 24:00 Fig. 3. Result screen of full data. Fig. 4. Result screen of the selected filter. 4 Conclusion It can be said that the fundamentals for Big Data analytics, using the rapidly developing information systems in medical institutions. Nevertheless, the potentials of such systems are not sufficiently utilized. If the system developed by this study can be used not only by government bodies but also by SMEs in such environment established to enable clinical trials, patient management and managerial management, it will lead to amazing developments in these fields in the future. Although such developed systems can be customized as necessary, continuous research activities are needed so that the system can be also applicable to many other fields including shipping and logistics. Acknowledgments. This article was written and the study was conducted as a part of 2014~2015 Nurimaru R&BD Project (SW convergence R & BD project) supported by the National IT Industry Promotion Agency and supervised by the NIPA Busan. References 1. Lee Sung-hoon and Lee Dong-woo. Current Status of Big Data Utilization The Journal of Digital Policy, The Society of Digital Policy & Management, 11(2), pp.229~233, 2013. 2. Manyika J, Chui M, Brown B, Bughin J, Dobbs R, Roxburgh C, Byers A. H. Big data: The next frontier for innovation, competition, and productivity, McKinsey Global Institute, 2011. 3. K.-W. Park, K.-J. Ban, S.-H. Song, and E.-K. Kim, Cloud-based Intelligent Management System for Photovoltaic Power Plants, J. of the Korea Institute of Electronic Communication Sciences, vol. 7, no. 3, June 2012, pp. 591-596. Copyright 2015 SERSC 29

4. Luckham, David C., and Brian Frasca. Complex event processing in distributed systems. Computer Systems Laboratory Technical Report CSL-TR-98-754. Stanford University, Stanford 28, 1998. 30 Copyright 2015 SERSC