HADOOP-BASED DISTRIBUTED SYSTEM FOR ONLINE PREDICTION OF AIR POLLUTION BASED ON SUPPORT VECTOR MACHINE

Transcription

1 HADOOP-BASED DISTRIBUTED SYSTEM FOR ONLINE PREDICTION OF AIR POLLUTION BASED ON SUPPORT VECTOR MACHINE Z Ghaemi a*, M Farnaghi b, A Aimohammadi b a Dep of Geodesy and Geomatics, KNToosi University of Technoogy - zghaemi@maikntuacir b Dep of Geodesy and Geomatics, KNToosi University of Technoogy - (Farnaghi, Aimoh_abb)@Kntuacir KEY WORDS: Urban Air Poution, Onine Prediction, Big Data, Spatia Anaysis, Distributed Computing, Support Vector Machine ABSTRACT: The critica impact of air poution on human heath and environment in one hand and the compexity of poutant concentration behavior in the other hand ead the scientists to ook for advance techniques for monitoring and predicting the urban air quaity Additionay, recent deveopments in data measurement techniques have ed to coection of various types of data about air quaity Such data is extremey vouminous and to be usefu it must be processed at high veocity Due to the compexity of big data anaysis especiay for dynamic appications, onine forecasting of poutant concentration trends within a reasonabe processing time is sti an open probem The purpose of this paper is to present an onine forecasting approach based on Support Vector Machine (SVM) to predict the air quaity one day in advance In order to overcome the computationa requirements for arge-scae data anaysis, distributed computing based on the Hadoop patform has been empoyed to everage the processing power of mutipe processing units The MapReduce programming mode is adopted for massive parae processing in this study Based on the onine agorithm and Hadoop framework, an onine forecasting system is designed to predict the air poution of Tehran for the next 24 hours The resuts have been assessed on the basis of Processing Time and Efficiency Quite accurate predictions of air poutant indicator eves within an acceptabe processing time prove that the presented approach is very suitabe to tacke arge scae air poution prediction probems 1 INTRODUCTION By deveoping data measurement techniques, the word has witnessed exposive, exponentia growth of data generation in recent years which eads the data coection outpace data processing capabiities Anayzing and processing vast amounts of information in a reasonabe time is one of the biggest scientist s chaenges in facing with such big amount of data (Zhai, Ong et a 2014) One of the crucia phenomenon requiring big data processing is the air poution Affecting by various factors and dynamic behavior of air poution ead the scientists to encounter big amount of data for prediction of the air quaity (Zheng, Liu et a 2013) In the other hand, urban air poution poses a significant threat to human heath (García Nieto, Combarro et a 2013) which turns it to a vita probem that requires immediate actions So, there is a strong need to find efficient soutions to dea with different aspects of such compex big data probems especiay data storage and rea-time processing In the ast years, numerous studies on the air-poution probems using statistica methods have been pubished Among them, machine earning agorithms aow scientists to sove hugey compex probems (García Nieto, Combarro et a 2013) SVM, as a newy presented machine earning agorithm, has been proven as a powerfu method to dea with compex and noninear phenomenon especiay air poution prediction (Ip, Vong et a 2010, García Nieto, Combarro et a 2013) But in rea-ife, machine earning agorithms incuding SVM, face with some imitations such as computationa compexity and computation time (Çatak and Baaban 2013) SVM cannot dea with big streaming data as it requires to be retrained by adding each newy gathered training sampe (Wang, Men et a 2008)The computation time and storage space of SVM agorithm are very argey because of arge scae kerne matrices which need to be recomputed severa times (Bottou, Weston et a 2004) In order to overcome the computationa probems of conventiona methods when facing with big and streaming data, some new techniques have been deveoped by researches One usefu soution is presenting onine agorithms based on the conventiona ones to speed up the process in dynamic appications (Wang, Men et a 2008) Athough the onine approaches have overcome the deficiencies of conventiona methods in deaing with streaming data and enhance the processing time, ack of required memory and demands of fast processing for big data probems sti poses chaenges Storing ever-increasing data of air poution on a singe machine and processing such big amount of data using singe processor is a imiting factor in onine appications Distributed computing is a recent soution to overcome the arge amount of memory and computation power requirements for training arge scae dataset such as air poution (Aham, Li et a 2013) Dividing the workoad between more than one processor can optimize the processing time to have higher performance especiay for onine appications One of the distributed frameworks which provides scaabe patform for * Corresponding author This contribution has been peer-reviewed doi:105194/isprsarchives-xl-1-w

2 storage and anaysis big data is Hadoop (Bhandarkar 2010) Hadoop consists of a processing part which is benefited MapReduce (Gao, Li et a) MapReduce can process exceedingy arge amounts of data by breaking down the overa probem into some parae sub-probems (Krämer and Senner 2015) The integration of MapReduce and SVM has been examined in some studies to speed up the processing time In a recent work, a MapReduce-based distributed SVM ensembe agorithm has been utiized for image cassification Each SVM is trained in parae using a custer of computers According to the resuts, the combination of ensembe SVMs with MapReduce achieved high accuracy and reduced the processing time significanty (Zhai, Ong et a 2014) In order to overcome the high computationa time required for arge datasets processing, Coobert, et a spit the dataset into severa parts and have utiized a mixture of severa SVMs Each SVM trained one part of data Finay, the resuts of a cassifiers were integrated to fine fina soution The resuts proved that using parae SVMs was much faster than training a singe SVM (Coobert, Bengio et a 2002) Based on the idea of exchanging support vectors among the connected networks, Lu, et a proposed a distributed SVM agorithm Severa sites had considered in each one partition of data was ocated Each subset of data was trained ocay via SVM on each site and the SVs were exchanged with the other sites (Lu, Roychowdhury et a 2008) Zanghirati and Zanni have decomposed the main SVM probem into smaer sub-probems The outcome of sub-probems were then combined The resuts showed that the proposed technique coud be usefu for training SVMs on mutiprocessor systems (Zanghirati and Zanni 2003) In addition to big data probems such as memory usage and computation time, there exists another probem in air poution prediction which is insufficient number of air quaity monitoring stations (Zheng, Liu et a 2013) Lack of enough stations prevent scientists from monitoring spatia distribution of poutant concentrations thoroughy In order to overcome this shortcoming, geographica parameters and spatia anaysis using Geographica Information System (GIS) can be empoyed to monitor the spatia distribution of air poutants more accuratey In this study, aong with poutant concentrations and meteoroogica data some spatia parameters incuding oca height, surface curvature and distance to the roads have been utiized The purpose of this study is to propose an onine system to predict the air quaity of Tehran one day in advance First, an onine agorithm based on SVM is deveoped to accuratey predict the air quaity Foowing that, the parae version of the onine agorithm using MapReduce is provided, which can sove both the arge-scae air poution probem and the onine prediction at the same time The obtained resuts have shown that distributing the process can effectivey enhance the processing time and provide much faster processing to anayze big air poution data The rest of this paper is organized as foows Section 2 briefy introduces SVM and onine SVM techniques It is foowed by introducing MapReduce and Hadoop Section 3 describes the study area and the design of the distributed onine SVM agorithm in more detai Section 4 evauates the performance of MapReduce onine SVM Section 5 concudes the paper and points out some future work 2 MATERIAL AND METODOLOGY 21 Support Vector Machine A Support Vector Machine (SVM) is a binary cassifier which is deveoped by Vapnik (Vapnik 1998) The goa of SVM is to perform cassification by constructing the optima hyperpanes that maximize the distance between the two casses (Wang, Men et a 2008) The input sampes are caed vectors and the vectors near the hyperpane are the support vectors (SV) In noninear cases, kerne functions are empoyed to map the origina data into a higher dimensiona feature space through a inear or non-inear mapping (Haifeng, Jun et a 2009) Given a training dataset of {xi ; yi}, i = 1, 2,, N with input data x i R n and corresponding abes y i {+1, 1} The cassification decision function is represented in equation (1) (Burges 1998): f(x) = sign{ i=1 α i y i k(x i, x j ) + b} (1) Subject to the constraints: i=1 α i y i = 0 and 0 α i C i C is the reguarization constant which highy infuence the SVM resuts by controing the tradeoff between the errors and maximizing the distance between casses (Yeganeh, Motagh et a 2012) The coefficients α i are obtained by soving the foowing probem: ; i=1 (2) Maximize i=1 α i 1 α 2 j=1 iα j y i y j K(x i x j ) Sampes with non-zero coefficients are known as support vectors (Wang 2005) Athough SVM is proven as an appropriate agorithm to address compex and noninear probems (Lu and Wang 2005, Juhos, Makra et a 2008), it is unabe to overcome big streaming data For some new coming training data, SVM needs to aways retrain the agorithm using a training data (incuding od and new coming data) (Wang, Men et a 2008) In order to dea with this probem, an onine earning agorithm based on SVM (LaSVM) is utiized in this study, which can meet the requirement of onine earning to update the existing agorithm (Bordes, Ertekin et a 2005) LaSVM incudes two more steps than typica SVM named PROCESS and REPROCESS By adding each new training sampe, PROCESS checks if it can be a new support vector REPROCESS phase removes the non-support vectors from the current dataset Then the coefficient and consequenty the separating hyperpanes wi be updated (Bordes, Ertekin et a 2005) In this case, LaSVM utiizes ony the extracted support vectors and the newy inserted sampe for training rather than using a existing training data This process speeds up the training step and reduces the required memory 22 MapReduce Mode MapReduce, introduced by Googe (Lämme 2008), provides parae computing power to process paraeizabe probems across huge datasets Each MapReduce process consists of two phases: the Map phase and the Reduce phase which their functions are defined by user (Aham, Li et a 2013) This contribution has been peer-reviewed doi:105194/isprsarchives-xl-1-w

3 The Mapper receives input data and transforms each eement to an output in a parae manner The output from the Mapper is a ist of (key, vaue) pairs Reduce phase reads key-vaue generated by Mappers and process the vaues using the reduce function In other words, the reduce function accepts an intermediate key and a set of vaues for that key It merges together these vaues to form a possiby smaer set of vaues Finay, the reducer send the obtained resuts to the output (Cary, Sun et a 2009) faster when the input data set is very arge, a paraeization method for the LaSVM cassifier is proposed and impement based on Hadoop s MapReduce framework The training phase is performed on the server on which Hadoop is running A MapReduce program is designed to acceerate soving A map/reduce job consists of some independent map tasks and some independent reduce tasks The number of Mappers determines the eve of paraeism The input data is divided into some independent chunks which are processed by the map tasks in a parae manner The outputs of the Mappers are then sorted by the MapReduce framework and sent to the reduce tasks as inputs Both the input and the output of the job are stored in a fie system (Dean and Ghemawat 2008) 23 Hadoop Hadoop is an open source software patform which can provide fast and reiabe anaysis to hande arge data sets (Loebman, Nuney et a 2009) Hadoop incudes many subproject but the two main ones are Hadoop Distributed Fie System (HDFS) and MapReduce MapReduce is appied to process data and HDFS is utiized for storing data (Jorgensen, Rowand-Jones et a 2014) MapReduce integrates tighty with HDFS MapReduce tasks run directy on the HDFS nodes that hod the required data When running a MapReduce job, some transformations from the source of data to the resut data set are provided The input data is fed to the map function and the resut of map is sent to a reduce function Hadoop's MapReduce jobs manage the process of how to appy these transformations to the data across the different nodes in parae (Turkington 2013) 31 Case Study and Dataset 3 DEVELOPMENT Tehran, the capita of Iran, has been chosen as case study as it suffers from severe air poution especiay in winters The prediction is performed based on the observed data of NO2, CO, SO2, PM10 and O3, for the past years and meteoroogica parameters incuding wind speed, temperature, reative humidity, pressure and coud cover Athough this data is gathered dynamicay, its avaiabiity is imited ony at monitoring stations Whereas the spatia distribution beyond these ocations sti remains uncertain as it is infuenced by geographica factors To address the effect of spatia autocorreation, aong with poution concentrations and meteoroogica factors some geographica parameters such as distance to the roads, oca height and topography are empoyed to mode the effects of spatia distribution of air poution Figure 1 demonstrates the study area and distribution of air poution stations 32 Paraeization of LaSVM based on MapReduce This work invoves two steps: first, training a data set using LaSVM to obtain a mode and second, using the mode to predict the air poution for the next 24 hours In order to share the workoad, training and prediction phases are performed on two separated servers To make LaSVM agorithm has a good scaabiity and run Figure 1 The study area and distribution of air poution monitoring stations LaSVM In order to impement parae LaSVM, the training data are retrieved from database in a form of primary key-vaue pairs In this study, the keys are date and ID of poution monitoring station Vaues are array which incudes abe of air quaity aong with meteoroogy, poution and geographica data The vaue arrays are empoyed for training LaSVM After training, the SVs are extracted and arranged as second key-vaue pairs In next step, the extracted SVs in the form of key-vaue from Mapper are sent to the reduce phase The reduce task aggregates a SVs The fina SVs from reduce phase are saved in a fie When new training sampe is avaiabe, the saved SVs aong with the newy entered training sampes are returned to the map phase to be used in training The saved SVs are aso sent to the second server as a request for air quaity prediction is made An overview of the idea is presented in Figure 2 4 RESULTS The agorithm for onine prediction of air poution is buit on LaSVM and impemented using the Hadoop impementation of MapReduce and the Java programming anguage Gaussian kerne was used for LaSVM Optima parameters (C, kerne specific parameter (γ)) were estimated by cross-vaidation method and set to 2 and 0001 respectivey The designed forecasting system can receive data in sequence, train LaSVM dynamicay and predict the air quaity one day in advance Transferring data from one step to another is demonstrated in Figure 3 In order to test the onine agorithm and assess performance of the system, one year data was set as testing data The efficiency of the onine agorithm is evauated based on Accuracy, RMSE and RSquered estimators The Accuracy of 07, RMSE of 06 and RSquered 0f 08 proved the feasibiity of the onine agorithm Aso, using the MapReduce mode for distributing the process sped up the processing time Therefore, the system coud not ony achieve promising resuts, but aso possess a good prediction performance as we This contribution has been peer-reviewed doi:105194/isprsarchives-xl-1-w

4 Database Server Database <K1,V1> Prediction Request Hadoop MapReduce Mapper Train LaSVM Extract SVs <K2,V2> Saved SVs + New Sampe Prediction Air Quaity Labe Reducer Fina SVs Save SVs in a fie Figure 2 An overview of MapReduce based system for onine air poution prediction 5 CONCLUSION To overcome the most chaenging probems of big air poution data, memory usage and computation time, a parae LaSVM agorithm was deveoped based on Hadoop-MapReduce in this study The proposed system had been utiized for onine air poution prediction of Tehran one day in advance Geographica data was aso appied to dea with coverage imitation of air poution monitoring stations The obtained resuts seemed extremey encouraging and suggested that the proposed system coud aow training SVM-ike modes for very arge scae data set in a reasonabe time Spitting the whoe dataset over data nodes and training each subset of data in parae wi be the future research work It is aso panned to compare the obtained resuts by empoying Hadoop and MapReduce programming with the onine agorithm that had been executed on a singe machine to evauate the feasibiity of the onine distributed system Output Reducer Training Mapper Input ,13,17,4,66, ,41,39,7,51, ,1,18,7,46, ( , [2,13,17,4,66, ]) ( ,[1,41,39,7,51, ]) ( , [2,13,17,4,66, ]) ( ,[1,41,39,7,51, ]) Remove non-svs SVs ( , [2,13,17,4,66, ]) ( ,[1,41,39,7,51, ]) Trained Onine Agorithm Figure 3 Transfer of data within map and reduce phases REFERENCES Aham, N K, M Li, Y Liu and M Qi (2013) "A MapReduce-based distributed SVM ensembe for scaabe image cassification and annotation" Computers & Mathematics with Appications 66(10): Bhandarkar, M (2010) MapReduce programming with apache Hadoop Parae & Distributed Processing (IPDPS), 2010 IEEE Internationa Symposium on, IEEE Bordes, A, S Ertekin, J Weston and L Bottou (2005) "Fast kerne cassifiers with onine and active earning" The Journa of Machine Learning Research 6: Bottou, L, J Weston and G H Bakir (2004) Breaking SVM compexity with cross-training Advances in neura information processing systems Burges, C J (1998) "A tutoria on support vector machines for pattern recognition" Data mining and knowedge discovery 2(2): This contribution has been peer-reviewed doi:105194/isprsarchives-xl-1-w

5 Cary, A, Z Sun, V Hristidis and N Rishe (2009) Experiences on processing spatia data with mapreduce Scientific and statistica database management, Springer Çatak, F Ö and M E Baaban (2013) "A MapReduce based distributed SVM agorithm for binary cassification" Turkish Journa of Eectrica Engineering & Computer Science Coobert, R, S Bengio and Y Bengio (2002) "A parae mixture of SVMs for very arge scae probems" Neura computation 14(5): Dean, J and S Ghemawat (2008) "MapReduce: simpified data processing on arge custers" Communications of the ACM 51(1): Gao, S, L Li, W Li, K Janowicz and Y Zhang "Constructing gazetteers from vounteered Big Geo-Data based on Hadoop" Computers, Environment and Urban Systems García Nieto, P J, E F Combarro, J J de Coz Díaz and E Montañés (2013) "A SVM-based regression mode to study the air quaity at oca scae in Oviedo urban area (Northern Spain): A case study" Appied Mathematics and Computation 219(17): Haifeng, W, F Jun and G Chong (2009) Research on the assessment for air environment quaity based on Support Vector Machine Contro and Decision Conference, 2009 CCDC'09 Chinese, IEEE Turkington, G (2013) Hadoop Beginner's Guide, Packt Pubishing Ltd Vapnik, V N (1998) Statistica earning theory, Wiey New York Wang, L (2005) Support Vector Machines: theory and appications, Springer Wang, W, C Men and W Lu (2008) "Onine prediction mode based on support vector machine" Neurocomputing 71(4): Yeganeh, B, M Motagh, Y Rashidi and H Kamaan (2012) "Prediction of CO concentrations based on a hybrid Partia Least Square and Support Vector Machine mode" Atmospheric Environment 55: Zanghirati, G and L Zanni (2003) "A parae sover for arge quadratic programs in training support vector machines" Parae computing 29(4): Zhai, Y, Y-S Ong and I W Tsang (2014) "The Emerging" Big Dimensionaity"" Computationa Inteigence Magazine, IEEE 9(3): Zheng, Y, F Liu and H-P Hsieh (2013) U-Air: When urban air quaity inference meets big data Proceedings of the 19th ACM SIGKDD internationa conference on Knowedge discovery and data mining, ACM Ip, W, C Vong, J Yang and P Wong (2010) Forecasting daiy ambient air poution based on east squares support vector machines Information and Automation (ICIA), 2010 IEEE Internationa Conference on, IEEE Jorgensen, A, J Rowand-Jones, J Wech, D Cark, C Price and B Mitche (2014) Microsoft Big Data Soutions, John Wiey & Sons Juhos, I, L Makra and B Tóth (2008) "Forecasting of traffic origin NO and NO 2 concentrations by support vector machines and neura networks using principa component anaysis" Simuation Modeing Practice and Theory 16(9): Krämer, M and I Senner (2015) "A moduar software architecture for processing of big geospatia data in the coud" Computers & Graphics 49: Lämme, R (2008) "Googe s MapReduce programming mode Revisited" Science of computer programming 70(1): 1-30 Loebman, S, D Nuney, Y Kwon, B Howe, M Baazinska and J P Gardner (2009) Anayzing massive astrophysica datasets: Can Pig/Hadoop or a reationa DBMS hep? Custer Computing and Workshops, 2009 CLUSTER'09 IEEE Internationa Conference on, IEEE Lu, W-Z and W-J Wang (2005) "Potentia assessment of the support vector machine method in forecasting ambient air poutant trends" Chemosphere 59(5): Lu, Y, V Roychowdhury and L Vandenberghe (2008) "Distributed parae support vector machines in strongy connected networks" Neura Networks, IEEE Transactions on 19(7): This contribution has been peer-reviewed doi:105194/isprsarchives-xl-1-w