ECE544NA Final Project: Robust Machine Learning Hardware via Classifier Ensemble


 Griffin Richards
 2 years ago
 Views:
Transcription
1 1 ECE544NA Fnal Project: Robust Machne Learnng Hardware va Classfer Ensemble Sa Zhang, Dept. of Electr. & Comput. Eng., Unv. of Illnos at UrbanaChampagn, Urbana, IL, USA Abstract In ths paper, we propose to use classfer ensemble (CE) as a method to enhance the robustness of machne learnng (ML) kernels n presence of hardware error. Dfferent ensemble methods (Baggng and Adaboost) are explored wth decson tree (C4.5) and artfcal neural network (ANN) as base classfers. Smulaton results show that ANN s nherently tolerant to hardware errors wth up to 1% hardware error rate. Wth smple majorty votng scheme, CE s able to effectvely reduce the classfcaton error rate for almost all tested data sets, wth maxmum test error reducton of 48%. For tree ensemble, Adaboost wth decson stump as weak learner gves best results; whle for ANN, baggng and boostng outperform each other dependng on data set. I. INTRODUCTION Ths project s motvated by recently works appeared n machne learnng (ML) on slcon [1]. There s a growng nterest n VLSI and crcut area to effcently brng ML kernels onto slcon, largely due to the performance and power lmtaton of software only approaches. On the other hand, n deep submcro era, devces exhbt statstcal behavors whch degrade the system relablty. To reduce the energy consumpton of crcuts, sub/near threshold computng has been proposed to offer 1X power reducton at the expense of large delay varaton as shown n Fg. 1. To enhance the relablty and robustness of ML kernels on slcon, Verma et al. propose to utlze ML algorthm to effcently learn the error behavor and make correct predcton/classfcaton n presence of hardware errors. Km [2] et al. shows that some ML algorthm nherently has tolerance to hardware errors. These work motvate us to look nto methods to effectvely enhance the robustness of ML kernels n presence of hardware errors. Classfer ensemble (CE, also referred to as Multple Classfer System) have been employed to enhance the performance of sngle classfer system [3], [4]. As summarzed n [5] [8], popular ensemble methods nclude: Baggng, Adaboost, Bayesan votng, Random forest, Rotaton forest, Error correctng output codng, etc. In terms of decson combnng, [9], [1] summarzes typcal classfer fuson methods. Therefore, n ths paper, we explore the dea of usng CE to enhance the robustness of ML kernels. Dfferent ensemble methods (Baggng and Adaboost) are explored wth decson tree (C4.5) and artfcal neural network (ANN) as base classfers. Smulaton results show that ANN s nherently tolerant to hardware errors wth up to 1% hardware error rate. Wth smple majorty votng scheme, CE s able to effectvely reduce the classfcaton error rate for almost all tested data sets, wth maxmum test error reducton of 48%. For tree ensemble, Adaboost wth decson stump as weak learner gves best results; whle for ANN, baggng and boostng outperform each other dependng on data set. The rest of the report s organzed as follows: secton 2 gves background of the CE methods we use n the project, secton 3 presents error model, and smulaton setup; secton 4 presents smulaton results wth concluson provded n secton 5. A. Ensemble Methods II. BACKGROUND We gve a bref overvew of methods to construct classfer ensembles. 1) Average Bayes Classfer: In the average Bayes settng, every classfer output s nterpreted as a posteror probablty of class membershp, condton on the nput and the hypothess. h(x) = P (f(x) = y x, h) (1) Here we assume the hypothess follow a dstrbuton n the hypothess space H. If the hypothess space s small, t s possble to enumerate all possble classfers (h(x)). The fnal output s the average of these hypothess, weghted by the posteror probablty of h(x), as shown n the followng equaton: P (f(x) = y S, x) = h H h(x)p (h S) (2) Fg. 1. Wth reduced supply voltage, energy effcency s mproved but delay varaton s larger. Usng Bayes rule, we can wrte the posteror probablty P (h S) as:
2 2 P (h S) P (S h)p (h) (3) The practcal dffculty n mplementng ths method les n that when the hypothess space s large, t s dffcult to enumerate all h(x), also t s not always possble to know the pror of each of the hypothess. Fg. 2 provdes a llustraton of the algorthm. The true mappng whch the algorthm tres to learn s denoted as f(x). To construct the ensemble, all hypothess h(x) n the space H s learned, and the fnal mappng s a weghted average based on (2). Fg. 2. Average Bayes Classfer 2) Baggng: Bootstrap aggregatng s a popular method to construct ensembles. The basc dea s to frst generate many tranng sets from orgnal tranng samples by random samplng wth replacement, then tran multple classfers, each based on one of the tranng sets, as shown n Fg. 3. By random samplng wth replacement, each tranng set contans on average 63.3% of the orgnal tranng set. The fnal decson s made by takng the majorty votng of ndvdual weak classfers. Baggng has been shown to mprove the performance of unstable classfers, such as neural networks, decson trees etc. 3) Adaptve Boostng: Adaboost s another popular method for ensemble generaton. The basc dea can be depcted n the Fg. 4. The tranng has T teratons; each teraton wll produce a new weak classfer. The basc dea s that after each teraton step, the examples are reweghted so that mssclassfed examples get hgher weght. In the subsequent teraton step, the tranng process wll focus more on classfyng the mssclassfed samples from prevous teraton. Fg. 5 gves the algorthm flow of Adaboost. Note that the basc dea of Adaboost s very smlar wth Baggng, wth two notable dfferences: 1) n generatng dfferent tranng sample, Baggng uses random sample wth replacement whle Adaboost uses the accuracy of prevous classfers to gude the selecton of subsequent tranng samples; and 2) Baggng uses smple majorty votng to generate decson whle Adaboost uses weghted average. It can be shown that the selecton of tranng sample dstrbuton and weghts n Adaboost wll mnmze an exponental loss functon defned as Fg. 3. Baggng Fg. 5. Adaboost algorthm L(f(x), y) = exp( y f(x )) (4) Proof: We are tryng to fnd a classfer of the form: H(x) = M α f m 1 (x) m=1 to mnmze loss n (4). At step m, we have H m (x) = H m 1 (x) + α m f m (x) The mnmzaton problem can be formulated as:
3 3 Fg. 4. Adaboost [α m, f m ] = arg mn [α m, f m ] = exp( y (H m 1 (x) + αf(x))) arg mn { N exp( y H m 1 (x)) exp(α)1[y f(x)] + exp( y H m 1 (x)) exp( α)1[y = f(x)]} [α m, f m ] = arg mn {exp(α) N exp( y H m 1 (x)) 1[y f(x)] + exp( α)( exp( y H m 1 (x)) exp( y H m 1 (x)) exp( α)1[y = f(x)])} Ths s equvalent to [α m, f m ] = arg mn {exp(α) N w m 1 () 1[y f(x)] + exp( α)(1 w m 1 ()) exp( α)1[y = f(x)])} where w m 1 () = smplfy the optmzaton nto: exp( y H m 1(x) exp( y H m 1(x)) [α m, f m ] = arg mn {exp( α) + [exp(α) exp( α)], we can further w m 1 () 1[y f(x)]} The optmal α and f can be solved by settng the dervatve wth respect to them to, we can thus obtan the optmal α m, f m as: f m = arg mn f α m = 1 2 log(1 ε m ε m ) w m 1 () 1[y f(x )] where ε m = N w m 1 () 1[y f(x)]. Therefore, we can see that n Adaboost, the dstrbuton and weght are selected such that the loss functon n (4) s mnmzed. 4) Injecton Randomness: The fnal method to construct ensemble s by njectng randomness nto the classfer. For good ensemble performance, the weak classfer used to construct the ensemble need to be unstable. Good unstable classfers are trees or neural networks. In the random njectng method, the ensemble s constructed by smply changng the ntal the weghts of NN, or choosng randomly the features used to splt the trees. B. Decson Combnaton The second mportant aspect of CE s to decde how to combne the output of each classfer to obtan the fnal decson. Many decson combnaton methods have been proposed, as shown n Fg. 6. If the output of each classfer s not a hard decson but a contnuous measure. We can use averagng method to get the ensemble output, popular averagng method ncludes mean, medan, weghted mean etc. If the output of classfer s a dscrete number, votng methods can be used. The smplest votng methods s majorty votng, whch takes the fnal output as the class most of the classfers agree on. Weghted average method can also be used to weght the decson of each classfer based on ther accuracy measure as n the case of Adaboost. A. Hardware error smulaton III. METHODOLOGY Error njecton s performed n smulaton level snce we do not have realstc hardware realzaton of the classfers. The error njecton presents dffculty due to at least two
4 4 TABLE I EXPERIMENTAL SETUP Base Classfer Ensemble Methods Combnaton ANN Baggng Majorty votng C4.5 Boostng Weghed votng Dec. stump Intal weghts Baggng, smple majorty votng scheme s employed. We use 4 data sets from UCI machne learnng repostory. For each data set, 1 fold crossvaldaton methods are used to evaluate ther performances. Fg. 6. Decson combnaton methods requrements: 1) the mappng from nput space to output error space need to have suffcent randomness, other wse t wll be trval to remove the error by addng some bas to the network output. 2) for same nput pattern, t wll always generate the same random error pattern. To solve ths ssue, we use a nput pattern dependent random number generator for error njecton, as shown n Fg. 7. When presented wth a feature vector, the random generator uses each dmenson as a seed to generate a Gaussan random number; the random number from dfferent dmenson are then added and scaled to provde an output err N(, 1). Ths method ensures that for dfferent feature vector, the generated random errors are dfferent, whle at the same tme same feature vector always results n same error value. The err s then used to nject hardware errors nto trees and ANNs. For decsons tree, we generate two threshold T hreshold 1 and T hreshold 2 for each node, when err falls nto the nterval [T hreshold 1, T hreshold 2 ],the decson of the node s flpped. For ANNs, smlar thresholds are generated for each weght, and f the err falls nto [T hreshold 1, T hreshold 2 ], the weght matrx s perturbed to smulate hardware defects. We can control the error rate by controllng the thresholds,.e. hgher error rate can be acheved f T hreshold 2 T hreshold 1 s ncreased. IV. RESULTS Fg. 8 shows the test error rate vs. ensemble sze for all evaluated ensemble methods. For Baggng wth decson trees, test error rate decreases as ensemble sze ncreases for both hardware error free (red) and hardware error njected (blue) case. However, hardware error degrade the performance of the ensemble and ths degradaton cannot be effectvely compensated for wth ncreased ensemble sze. For Adaboost wth decson tree, Fg. 8 ndcates that the ensemble suffer from overfttng. As ensemble sze grows, the error rate frst decreases, then starts to ncrease at ensemble sze of 3 and 2 for error free and erroneous case. Adaboost wth decson stump gves best performance n the famly of tree ensembles, the error njected ensemble almost acheves the same error reducton as the error free ensemble. For ANN wth baggng, both error njected and error free ensemble acheve effectve reducton of test error rate wth ncreasng ensemble sze. However, for ANN wth Adaboost, overfttng starts to occur at ensemble sze of 4 and 3 for error free and erroneous ensemble, respectvely. Fg. 7. Error njecton methodology B. Experment Setup Table I shows the experment setup. In ths project, we explore Baggng and Adaboost wth ANN and C4.5 decson tree and decson stump (a decson tree wth depth 1) as base classfers. When tranng ANN, random ntal weghts are used to further ntroduce dversty nto the ensemble. For Adaboost, we use weghted votng as shown n Fg. 5, and for Fg. 8. Results: test error rate vs. ensemble sze Fg. 9 shows the result for 4 data set from UCI machne learnng repostory at dfferent njected hardware error rate. From the fgure several observaton can be made: 1) In terms of nherent error tolerance, ANN s better than decson trees. Ths can be seen from Fg. 9 by notng that at 3% error rate, erroneous C4.5 already have severely degraded performance at data set 1 and 2, but ANN s able to mantan smlar performance regardless of error njecton
5 5 Error rate of tree ensemble Error rate of NN ensemble 3%.4.2 C4.5 error free C4.5 error njected C4.5 baggng C4.5 boostng Dec stump boostng.4.2 NN error free NN error njected NN baggng NN boostng 1% % Fg. 9. Results: test error rate for dfferent data set at error rate of 3%, 1% and 2% tll 1% percent error rate. The dfference can be partally explaned by notng that for decson trees, at each node a hard decson s made, so error have hgh chance to propagate to output, whle for ANN, each layer does not make a hard decson but use a sgmod functon to suppress the output, whch mght help reduce the effect of node errors. 2) For tree ensemble, Adaboost wth decson stump always gve best performance, consstent wth Fg. 8. Boostng decson tree tends to suffer from overfttng at low error rate. Ths problem s mtgated at hgh error rate due to the fact that at hgh error rate, the ensemble sze tends to be reduced. 3) For ANN ensemble, Adaboost and Baggng can out perform each other dependng on data set, and there s no consstent behavor across dfferent error rate. The overfttng problem of ANN s less severe compared wth trees, and smlarly the overfttng gets mtgated at hgh error rate due to the reducton of ensemble sze. V. CONCLUSION AND FUTURE WORK In ths paper, we show that CE s an effectve method to enhance the robustness of ML hardware. Baggng and Adaboost are explored wth decson tree and ANN as base classfers. Smulaton results show that ANN s nherently tolerant to hardware errors wth up to 1% hardware error rate. Wth smple majorty votng scheme, CE s able to effectvely reduce the classfcaton error rate for almost all tested data sets, wth maxmum test error reducton of 48%. For tree ensemble, Adaboost wth decson stump as weak learner gves best results; whle for ANN, baggng and boostng outperform each other dependng on data set. In the future, we can extend the framework to nclude more ensemble methods, to handle multclass problems, to mtgate the overfttng problem by usng regularzaton. Also t wll be helpful to mplement part of the algorthm n hardware to evaluate the performance of CE on ML kernels. ACKNOWLEDGMENT The author would lke to thank Prof. HasegawaJohnson and Sujeeth Subramanya Bharadwaj for ther help and gudance. REFERENCES [1] N. Verma, K. H. Lee, K. J. Jang, and A. Shoeb, Enablng systemlevel platform reslence through embedded datadrven nference capabltes n electronc devces, n Acoustcs, Speech and Sgnal Processng (ICASSP), 212 IEEE Internatonal Conference on, 212, pp [2] J. Cho, E. P. Km, R. A. Rutenbar, and N. R. Shanbhag, Error reslent mrf message passng archtecture for stereo matchng, n Sgnal Processng Systems (SPS), 213 IEEE Workshop on, 213, pp [3] L. yng Yang, Z. Qn, and R. Huang, Desgn of a multple classfer system, n Machne Learnng and Cybernetcs, 24. Proceedngs of 24 Internatonal Conference on, vol. 5, 24, pp vol.5. [4] G. Gacnto, F. Rol, and G. Fumera, Desgn of effectve multple classfer systems by clusterng of classfers, n Pattern Recognton, 2. Proceedngs. 15th Internatonal Conference on, vol. 2, 2, pp vol.2. [5] T. G. Detterch, Ensemble methods n machne learnng, n MULTI PLE CLASSIFIER SYSTEMS, LBCS Sprnger, 2, pp [6] J. Rodrguez, L. Kuncheva, and C. Alonso, Rotaton forest: A new classfer ensemble method, Pattern Analyss and Machne Intellgence, IEEE Transactons on, vol. 28, no. 1, pp , 26. [7] D. Optz and R. Macln, Popular ensemble methods: An emprcal study, Journal of Artfcal Intellgence Research, vol. 11, pp , [8] L. B. Statstcs and L. Breman, Random forests, n Machne Learnng, 21, pp
6 [9] L. Xu, A. Krzyzak, and C. Suen, Methods of combnng multple classfers and ther applcatons to handwrtng recognton, Systems, Man and Cybernetcs, IEEE Transactons on, vol. 22, no. 3, pp , [1] T. K. Ho, J. Hull, and S. Srhar, Decson combnaton n multple classfer systems, Pattern Analyss and Machne Intellgence, IEEE Transactons on, vol. 16, no. 1, pp ,
Logistic Regression. Lecture 4: More classifiers and classes. Logistic regression. Adaboost. Optimization. Multiple class classification
Lecture 4: More classfers and classes C4B Machne Learnng Hlary 20 A. Zsserman Logstc regresson Loss functons revsted Adaboost Loss functons revsted Optmzaton Multple class classfcaton Logstc Regresson
More informationWhat is Candidate Sampling
What s Canddate Samplng Say we have a multclass or mult label problem where each tranng example ( x, T ) conssts of a context x a small (mult)set of target classes T out of a large unverse L of possble
More informationForecasting the Direction and Strength of Stock Market Movement
Forecastng the Drecton and Strength of Stock Market Movement Jngwe Chen Mng Chen Nan Ye cjngwe@stanford.edu mchen5@stanford.edu nanye@stanford.edu Abstract  Stock market s one of the most complcated systems
More informationThe Development of Web Log Mining Based on ImproveKMeans Clustering Analysis
The Development of Web Log Mnng Based on ImproveKMeans Clusterng Analyss TngZhong Wang * College of Informaton Technology, Luoyang Normal Unversty, Luoyang, 471022, Chna wangtngzhong2@sna.cn Abstract.
More informationAn InterestOriented Network Evolution Mechanism for Online Communities
An InterestOrented Network Evoluton Mechansm for Onlne Communtes Cahong Sun and Xaopng Yang School of Informaton, Renmn Unversty of Chna, Bejng 100872, P.R. Chna {chsun,yang}@ruc.edu.cn Abstract. Onlne
More informationOnLine Fault Detection in Wind Turbine Transmission System using Adaptive Filter and Robust Statistical Features
OnLne Fault Detecton n Wnd Turbne Transmsson System usng Adaptve Flter and Robust Statstcal Features Ruoyu L Remote Dagnostcs Center SKF USA Inc. 3443 N. Sam Houston Pkwy., Houston TX 77086 Emal: ruoyu.l@skf.com
More informationTHE APPLICATION OF DATA MINING TECHNIQUES AND MULTIPLE CLASSIFIERS TO MARKETING DECISION
Internatonal Journal of Electronc Busness Management, Vol. 3, No. 4, pp. 3030 (2005) 30 THE APPLICATION OF DATA MINING TECHNIQUES AND MULTIPLE CLASSIFIERS TO MARKETING DECISION YuMn Chang *, YuCheh
More informationLecture 2: Single Layer Perceptrons Kevin Swingler
Lecture 2: Sngle Layer Perceptrons Kevn Sngler kms@cs.str.ac.uk Recap: McCullochPtts Neuron Ths vastly smplfed model of real neurons s also knon as a Threshold Logc Unt: W 2 A Y 3 n W n. A set of synapses
More informationVision Mouse. Saurabh Sarkar a* University of Cincinnati, Cincinnati, USA ABSTRACT 1. INTRODUCTION
Vson Mouse Saurabh Sarkar a* a Unversty of Cncnnat, Cncnnat, USA ABSTRACT The report dscusses a vson based approach towards trackng of eyes and fngers. The report descrbes the process of locatng the possble
More informationLogistic Regression. Steve Kroon
Logstc Regresson Steve Kroon Course notes sectons: 24.324.4 Dsclamer: these notes do not explctly ndcate whether values are vectors or scalars, but expects the reader to dscern ths from the context. Scenaro
More informationFace Verification Problem. Face Recognition Problem. Application: Access Control. Biometric Authentication. Face Verification (1:1 matching)
Face Recognton Problem Face Verfcaton Problem Face Verfcaton (1:1 matchng) Querymage face query Face Recognton (1:N matchng) database Applcaton: Access Control www.vsage.com www.vsoncs.com Bometrc Authentcaton
More informationNonlinear data mapping by neural networks
Nonlnear data mappng by neural networks R.P.W. Dun Delft Unversty of Technology, Netherlands Abstract A revew s gven of the use of neural networks for nonlnear mappng of hgh dmensonal data on lower dmensonal
More informationGender Classification for RealTime Audience Analysis System
Gender Classfcaton for RealTme Audence Analyss System Vladmr Khryashchev, Lev Shmaglt, Andrey Shemyakov, Anton Lebedev Yaroslavl State Unversty Yaroslavl, Russa vhr@yandex.ru, shmaglt_lev@yahoo.com, andrey.shemakov@gmal.com,
More informationA novel Method for Data Mining and Classification based on
A novel Method for Data Mnng and Classfcaton based on Ensemble Learnng 1 1, Frst Author Nejang Normal Unversty;Schuan Nejang 641112,Chna, Emal: lhangege@126.com Abstract Data mnng has been attached great
More informationLuby s Alg. for Maximal Independent Sets using Pairwise Independence
Lecture Notes for Randomzed Algorthms Luby s Alg. for Maxmal Independent Sets usng Parwse Independence Last Updated by Erc Vgoda on February, 006 8. Maxmal Independent Sets For a graph G = (V, E), an ndependent
More informationA hybrid global optimization algorithm based on parallel chaos optimization and outlook algorithm
Avalable onlne www.ocpr.com Journal of Chemcal and Pharmaceutcal Research, 2014, 6(7):18841889 Research Artcle ISSN : 09757384 CODEN(USA) : JCPRC5 A hybrd global optmzaton algorthm based on parallel
More informationCS 2750 Machine Learning. Lecture 3. Density estimation. CS 2750 Machine Learning. Announcements
Lecture 3 Densty estmaton Mlos Hauskrecht mlos@cs.ptt.edu 5329 Sennott Square Next lecture: Matlab tutoral Announcements Rules for attendng the class: Regstered for credt Regstered for audt (only f there
More informationForecasting the Demand of Emergency Supplies: Based on the CBR Theory and BP Neural Network
700 Proceedngs of the 8th Internatonal Conference on Innovaton & Management Forecastng the Demand of Emergency Supples: Based on the CBR Theory and BP Neural Network Fu Deqang, Lu Yun, L Changbng School
More informationSupport Vector Machines
Support Vector Machnes Max Wellng Department of Computer Scence Unversty of Toronto 10 Kng s College Road Toronto, M5S 3G5 Canada wellng@cs.toronto.edu Abstract Ths s a note to explan support vector machnes.
More informationImproved SVM in Cloud Computing Information Mining
Internatonal Journal of Grd Dstrbuton Computng Vol.8, No.1 (015), pp.3340 http://dx.do.org/10.1457/jgdc.015.8.1.04 Improved n Cloud Computng Informaton Mnng Lvshuhong (ZhengDe polytechnc college JangSu
More informationDocument Clustering Analysis Based on Hybrid PSO+Kmeans Algorithm
Document Clusterng Analyss Based on Hybrd PSO+Kmeans Algorthm Xaohu Cu, Thomas E. Potok Appled Software Engneerng Research Group, Computatonal Scences and Engneerng Dvson, Oak Rdge Natonal Laboratory,
More information8 Algorithm for Binary Searching in Trees
8 Algorthm for Bnary Searchng n Trees In ths secton we present our algorthm for bnary searchng n trees. A crucal observaton employed by the algorthm s that ths problem can be effcently solved when the
More informationSingle and multiple stage classifiers implementing logistic discrimination
Sngle and multple stage classfers mplementng logstc dscrmnaton Hélo Radke Bttencourt 1 Dens Alter de Olvera Moraes 2 Vctor Haertel 2 1 Pontfíca Unversdade Católca do Ro Grande do Sul  PUCRS Av. Ipranga,
More informationMinimal Coding Network With Combinatorial Structure For Instantaneous Recovery From Edge Failures
Mnmal Codng Network Wth Combnatoral Structure For Instantaneous Recovery From Edge Falures Ashly Joseph 1, Mr.M.Sadsh Sendl 2, Dr.S.Karthk 3 1 Fnal Year ME CSE Student Department of Computer Scence Engneerng
More informationFeature selection for intrusion detection. Slobodan Petrović NISlab, Gjøvik University College
Feature selecton for ntruson detecton Slobodan Petrovć NISlab, Gjøvk Unversty College Contents The feature selecton problem Intruson detecton Traffc features relevant for IDS The CFS measure The mrmr measure
More informationAn Alternative Way to Measure Private Equity Performance
An Alternatve Way to Measure Prvate Equty Performance Peter Todd Parlux Investment Technology LLC Summary Internal Rate of Return (IRR) s probably the most common way to measure the performance of prvate
More informationCan Auto Liability Insurance Purchases Signal Risk Attitude?
Internatonal Journal of Busness and Economcs, 2011, Vol. 10, No. 2, 159164 Can Auto Lablty Insurance Purchases Sgnal Rsk Atttude? ChuShu L Department of Internatonal Busness, Asa Unversty, Tawan ShengChang
More informationbenefit is 2, paid if the policyholder dies within the year, and probability of death within the year is ).
REVIEW OF RISK MANAGEMENT CONCEPTS LOSS DISTRIBUTIONS AND INSURANCE Loss and nsurance: When someone s subject to the rsk of ncurrng a fnancal loss, the loss s generally modeled usng a random varable or
More informationLearning from Multiple Outlooks
Learnng from Multple Outlooks Maayan Harel Department of Electrcal Engneerng, Technon, Hafa, Israel She Mannor Department of Electrcal Engneerng, Technon, Hafa, Israel maayanga@tx.technon.ac.l she@ee.technon.ac.l
More information1 Example 1: Axisaligned rectangles
COS 511: Theoretcal Machne Learnng Lecturer: Rob Schapre Lecture # 6 Scrbe: Aaron Schld February 21, 2013 Last class, we dscussed an analogue for Occam s Razor for nfnte hypothess spaces that, n conjuncton
More informationAbstract. Clustering ensembles have emerged as a powerful method for improving both the
Clusterng Ensembles: {topchyal, Models jan, of punch}@cse.msu.edu Consensus and Weak Parttons * Alexander Topchy, Anl K. Jan, and Wllam Punch Department of Computer Scence and Engneerng, Mchgan State Unversty
More informationFault tolerance in cloud technologies presented as a service
Internatonal Scentfc Conference Computer Scence 2015 Pavel Dzhunev, PhD student Fault tolerance n cloud technologes presented as a servce INTRODUCTION Improvements n technques for vrtualzaton and performance
More informationModule 2 LOSSLESS IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur
Module LOSSLESS IMAGE COMPRESSION SYSTEMS Lesson 3 Lossless Compresson: Huffman Codng Instructonal Objectves At the end of ths lesson, the students should be able to:. Defne and measure source entropy..
More informationBayesian Cluster Ensembles
Bayesan Cluster Ensembles Hongjun Wang 1, Hanhua Shan 2 and Arndam Banerjee 2 1 Informaton Research Insttute, Southwest Jaotong Unversty, Chengdu, Schuan, 610031, Chna 2 Department of Computer Scence &
More informationPSYCHOLOGICAL RESEARCH (PYC 304C) Lecture 12
14 The Chsquared dstrbuton PSYCHOLOGICAL RESEARCH (PYC 304C) Lecture 1 If a normal varable X, havng mean µ and varance σ, s standardsed, the new varable Z has a mean 0 and varance 1. When ths standardsed
More informationThe Greedy Method. Introduction. 0/1 Knapsack Problem
The Greedy Method Introducton We have completed data structures. We now are gong to look at algorthm desgn methods. Often we are lookng at optmzaton problems whose performance s exponental. For an optmzaton
More informationOn the Optimal Control of a Cascade of HydroElectric Power Stations
On the Optmal Control of a Cascade of HydroElectrc Power Statons M.C.M. Guedes a, A.F. Rbero a, G.V. Smrnov b and S. Vlela c a Department of Mathematcs, School of Scences, Unversty of Porto, Portugal;
More informationOn Mean Squared Error of Hierarchical Estimator
S C H E D A E I N F O R M A T I C A E VOLUME 0 0 On Mean Squared Error of Herarchcal Estmator Stans law Brodowsk Faculty of Physcs, Astronomy, and Appled Computer Scence, Jagellonan Unversty, Reymonta
More informationHARDWARE SPECIALIZATION OF MACHINELEARNING KERNELS: POSSIBILITIES FOR APPLICATIONS AND POSSIBILITIES FOR THE PLATFORM DESIGN SPACE
HARDWARE SPECIALIZATIO OF MACHIELEARIG KERELS: POSSIBILITIES FOR APPLICATIOS AD POSSIBILITIES FOR THE PLATFORM DESIG SPACE (Invted) Kyong Ho Lee, Zhuo Wang, and aveen Verma Prnceton Unversty ABSTRACT
More informationLinear Circuits Analysis. Superposition, Thevenin /Norton Equivalent circuits
Lnear Crcuts Analyss. Superposton, Theenn /Norton Equalent crcuts So far we hae explored tmendependent (resste) elements that are also lnear. A tmendependent elements s one for whch we can plot an / cure.
More informationMining Feature Importance: Applying Evolutionary Algorithms within a Webbased Educational System
Mnng Feature Importance: Applyng Evolutonary Algorthms wthn a Webbased Educatonal System Behrouz MINAEIBIDGOLI 1, and Gerd KORTEMEYER 2, and Wllam F. PUNCH 1 1 Genetc Algorthms Research and Applcatons
More informationCommunication Networks II Contents
8 / 1  Communcaton Networs II (Görg)  www.comnets.unbremen.de Communcaton Networs II Contents 1 Fundamentals of probablty theory 2 Traffc n communcaton networs 3 Stochastc & Marovan Processes (SP
More informationA Secure PasswordAuthenticated Key Agreement Using Smart Cards
A Secure PasswordAuthentcated Key Agreement Usng Smart Cards Ka Chan 1, WenChung Kuo 2 and JnChou Cheng 3 1 Department of Computer and Informaton Scence, R.O.C. Mltary Academy, Kaohsung 83059, Tawan,
More informationCS 2750 Machine Learning. Lecture 17a. Clustering. CS 2750 Machine Learning. Clustering
Lecture 7a Clusterng Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square Clusterng Groups together smlar nstances n the data sample Basc clusterng problem: dstrbute data nto k dfferent groups such that
More informationLatent Class Regression. Statistics for Psychosocial Research II: Structural Models December 4 and 6, 2006
Latent Class Regresson Statstcs for Psychosocal Research II: Structural Models December 4 and 6, 2006 Latent Class Regresson (LCR) What s t and when do we use t? Recall the standard latent class model
More informationData Broadcast on a MultiSystem Heterogeneous Overlayed Wireless Network *
JOURNAL OF INFORMATION SCIENCE AND ENGINEERING 24, 819840 (2008) Data Broadcast on a MultSystem Heterogeneous Overlayed Wreless Network * Department of Computer Scence Natonal Chao Tung Unversty Hsnchu,
More informationL10: Linear discriminants analysis
L0: Lnear dscrmnants analyss Lnear dscrmnant analyss, two classes Lnear dscrmnant analyss, C classes LDA vs. PCA Lmtatons of LDA Varants of LDA Other dmensonalty reducton methods CSCE 666 Pattern Analyss
More informationPerformance Analysis and Coding Strategy of ECOC SVMs
Internatonal Journal of Grd and Dstrbuted Computng Vol.7, No. (04), pp.6776 http://dx.do.org/0.457/jgdc.04.7..07 Performance Analyss and Codng Strategy of ECOC SVMs Zhgang Yan, and Yuanxuan Yang, School
More informationInstitute of Informatics, Faculty of Business and Management, Brno University of Technology,Czech Republic
Lagrange Multplers as Quanttatve Indcators n Economcs Ivan Mezník Insttute of Informatcs, Faculty of Busness and Management, Brno Unversty of TechnologCzech Republc Abstract The quanttatve role of Lagrange
More informationA GENERAL APPROACH FOR SECURITY MONITORING AND PREVENTIVE CONTROL OF NETWORKS WITH LARGE WIND POWER PRODUCTION
A GENERAL APPROACH FOR SECURITY MONITORING AND PREVENTIVE CONTROL OF NETWORKS WITH LARGE WIND POWER PRODUCTION Helena Vasconcelos INESC Porto hvasconcelos@nescportopt J N Fdalgo INESC Porto and FEUP jfdalgo@nescportopt
More informationINVESTIGATION OF VEHICULAR USERS FAIRNESS IN CDMAHDR NETWORKS
21 22 September 2007, BULGARIA 119 Proceedngs of the Internatonal Conference on Informaton Technologes (InfoTech2007) 21 st 22 nd September 2007, Bulgara vol. 2 INVESTIGATION OF VEHICULAR USERS FAIRNESS
More informationA study on the ability of Support Vector Regression and Neural Networks to Forecast Basic Time Series Patterns
A study on the ablty of Support Vector Regresson and Neural Networks to Forecast Basc Tme Seres Patterns Sven F. Crone, Jose Guajardo 2, and Rchard Weber 2 Lancaster Unversty, Department of Management
More informationRiskbased Fatigue Estimate of Deep Water Risers  Course Project for EM388F: Fracture Mechanics, Spring 2008
Rskbased Fatgue Estmate of Deep Water Rsers  Course Project for EM388F: Fracture Mechancs, Sprng 2008 Chen Sh Department of Cvl, Archtectural, and Envronmental Engneerng The Unversty of Texas at Austn
More information) of the Cell class is created containing information about events associated with the cell. Events are added to the Cell instance
Calbraton Method Instances of the Cell class (one nstance for each FMS cell) contan ADC raw data and methods assocated wth each partcular FMS cell. The calbraton method ncludes event selecton (Class Cell
More informationFinancial market forecasting using a twostep kernel learning method for the support vector regression
Ann Oper Res (2010) 174: 103 120 DOI 10.1007/s1047900803577 Fnancal market forecastng usng a twostep kernel learnng method for the support vector regresson L Wang J Zhu Publshed onlne: 28 May 2008
More informationMarginal Returns to Education For Teachers
The Onlne Journal of New Horzons n Educaton Volume 4, Issue 3 MargnalReturnstoEducatonForTeachers RamleeIsmal,MarnahAwang ABSTRACT FacultyofManagementand Economcs UnverstPenddkanSultan Idrs ramlee@fpe.ups.edu.my
More informationChapter 4 ECONOMIC DISPATCH AND UNIT COMMITMENT
Chapter 4 ECOOMIC DISATCH AD UIT COMMITMET ITRODUCTIO A power system has several power plants. Each power plant has several generatng unts. At any pont of tme, the total load n the system s met by the
More informationA LoadBalancing Algorithm for Clusterbased Multicore Web Servers
Journal of Computatonal Informaton Systems 7: 13 (2011) 47404747 Avalable at http://www.jofcs.com A LoadBalancng Algorthm for Clusterbased Multcore Web Servers Guohua YOU, Yng ZHAO College of Informaton
More informationA Dynamic Load Balancing for Massive Multiplayer Online Game Server
A Dynamc Load Balancng for Massve Multplayer Onlne Game Server Jungyoul Lm, Jaeyong Chung, Jnryong Km and Kwanghyun Shm Dgtal Content Research Dvson Electroncs and Telecommuncatons Research Insttute Daejeon,
More informationInterleaved Power Factor Correction (IPFC)
Interleaved Power Factor Correcton (IPFC) 2009 Mcrochp Technology Incorporated. All Rghts Reserved. Interleaved Power Factor Correcton Slde 1 Welcome to the Interleaved Power Factor Correcton Reference
More informationHow Sets of Coherent Probabilities May Serve as Models for Degrees of Incoherence
1 st Internatonal Symposum on Imprecse Probabltes and Ther Applcatons, Ghent, Belgum, 29 June 2 July 1999 How Sets of Coherent Probabltes May Serve as Models for Degrees of Incoherence Mar J. Schervsh
More informationAn ILP Formulation for Task Mapping and Scheduling on Multicore Architectures
An ILP Formulaton for Task Mappng and Schedulng on Multcore Archtectures Yng Y, We Han, Xn Zhao, Ahmet T. Erdogan and Tughrul Arslan Unversty of Ednburgh, The Kng's Buldngs, Mayfeld Road, Ednburgh, EH9
More informationFuzzy Set Approach To Asymmetrical Load Balancing In Distribution Networks
Fuzzy Set Approach To Asymmetrcal Load Balancng n Dstrbuton Networks Goran Majstrovc Energy nsttute Hrvoje Por Zagreb, Croata goran.majstrovc@ehp.hr Slavko Krajcar Faculty of electrcal engneerng and computng
More informationDesign of Output Codes for Fast Covering Learning using Basic Decomposition Techniques
Journal of Computer Scence (7): 56557, 6 ISSN 5966 6 Scence Publcatons Desgn of Output Codes for Fast Coverng Learnng usng Basc Decomposton Technques Aruna Twar and Narendra S. Chaudhar, Faculty of Computer
More informationMATHEMATICAL ENGINEERING TECHNICAL REPORTS. Sequential Optimizing Investing Strategy with Neural Networks
MATHEMATICAL ENGINEERING TECHNICAL REPORTS Sequental Optmzng Investng Strategy wth Neural Networks Ryo ADACHI and Akmch TAKEMURA METR 2010 03 February 2010 DEPARTMENT OF MATHEMATICAL INFORMATICS GRADUATE
More information1 Approximation Algorithms
CME 305: Dscrete Mathematcs and Algorthms 1 Approxmaton Algorthms In lght of the apparent ntractablty of the problems we beleve not to le n P, t makes sense to pursue deas other than complete solutons
More informationMining Multiple Large Data Sources
The Internatonal Arab Journal of Informaton Technology, Vol. 7, No. 3, July 2 24 Mnng Multple Large Data Sources Anmesh Adhkar, Pralhad Ramachandrarao 2, Bhanu Prasad 3, and Jhml Adhkar 4 Department of
More informationLecture 18: Clustering & classification
O CPS260/BGT204. Algorthms n Computatonal Bology October 30, 2003 Lecturer: Pana K. Agarwal Lecture 8: Clusterng & classfcaton Scrbe: Daun Hou Open Problem In HomeWor 2, problem 5 has an open problem whch
More informationLearning from Large Distributed Data: A Scaling Down Sampling Scheme for Efficient Data Processing
Internatonal Journal of Machne Learnng and Computng, Vol. 4, No. 3, June 04 Learnng from Large Dstrbuted Data: A Scalng Down Samplng Scheme for Effcent Data Processng Che Ngufor and Janusz Wojtusak part
More informationPolitecnico di Torino. Porto Institutional Repository
Poltecnco d Torno Porto Insttutonal Repostory [Artcle] A costeffectve cloud computng framework for acceleratng multmeda communcaton smulatons Orgnal Ctaton: D. Angel, E. Masala (2012). A costeffectve
More informationBayesian Network Based Causal Relationship Identification and Funding Success Prediction in P2P Lending
Proceedngs of 2012 4th Internatonal Conference on Machne Learnng and Computng IPCSIT vol. 25 (2012) (2012) IACSIT Press, Sngapore Bayesan Network Based Causal Relatonshp Identfcaton and Fundng Success
More informationOptimal Choice of Random Variables in DITG Traffic Generating Tool using Evolutionary Algorithms
Optmal Choce of Random Varables n DITG Traffc Generatng Tool usng Evolutonary Algorthms M. R. Mosav* (C.A.), F. Farab* and S. Karam* Abstract: Impressve development of computer networks has been requred
More informationA Hierarchical Anomaly Network Intrusion Detection System using Neural Network Classification
IDC IDC A Herarchcal Anomaly Network Intruson Detecton System usng Neural Network Classfcaton ZHENG ZHANG, JUN LI, C. N. MANIKOPOULOS, JAY JORGENSON and JOSE UCLES ECE Department, New Jersey Inst. of Tech.,
More informationHallucinating Multiple Occluded CCTV Face Images of Different Resolutions
In Proc. IEEE Internatonal Conference on Advanced Vdeo and Sgnal based Survellance (AVSS 05), September 2005 Hallucnatng Multple Occluded CCTV Face Images of Dfferent Resolutons Ku Ja Shaogang Gong Computer
More informationPOLYSA: A Polynomial Algorithm for Nonbinary Constraint Satisfaction Problems with and
POLYSA: A Polynomal Algorthm for Nonbnary Constrant Satsfacton Problems wth and Mguel A. Saldo, Federco Barber Dpto. Sstemas Informátcos y Computacón Unversdad Poltécnca de Valenca, Camno de Vera s/n
More informationA Design Method of Highavailability and Lowopticalloss Optical Aggregation Network Architecture
A Desgn Method of Hghavalablty and Lowoptcalloss Optcal Aggregaton Network Archtecture Takehro Sato, Kuntaka Ashzawa, Kazumasa Tokuhash, Dasuke Ish, Satoru Okamoto and Naoak Yamanaka Dept. of Informaton
More informationVoIP Playout Buffer Adjustment using Adaptive Estimation of Network Delays
VoIP Playout Buffer Adjustment usng Adaptve Estmaton of Network Delays Mroslaw Narbutt and Lam Murphy* Department of Computer Scence Unversty College Dubln, Belfeld, Dubln, IRELAND Abstract The poor qualty
More informationImproved Mining of Software Complexity Data on Evolutionary Filtered Training Sets
Improved Mnng of Software Complexty Data on Evolutonary Fltered Tranng Sets VILI PODGORELEC Insttute of Informatcs, FERI Unversty of Marbor Smetanova ulca 17, SI2000 Marbor SLOVENIA vl.podgorelec@unmb.s
More informationFrequency Selective IQ Phase and IQ Amplitude Imbalance Adjustments for OFDM Direct Conversion Transmitters
Frequency Selectve IQ Phase and IQ Ampltude Imbalance Adjustments for OFDM Drect Converson ransmtters Edmund Coersmeer, Ernst Zelnsk Noka, Meesmannstrasse 103, 44807 Bochum, Germany edmund.coersmeer@noka.com,
More informationUsing Mixture Covariance Matrices to Improve Face and Facial Expression Recognitions
Usng Mxture Covarance Matrces to Improve Face and Facal Expresson Recogntons Carlos E. homaz, Duncan F. Glles and Raul Q. Fetosa 2 Imperal College of Scence echnology and Medcne, Department of Computng,
More informationA Multimode Image Tracking System Based on Distributed Fusion
A Multmode Image Tracng System Based on Dstrbuted Fuson Ln zheng Chongzhao Han Dongguang Zuo Hongsen Yan School of Electroncs & nformaton engneerng, X an Jaotong Unversty X an, Shaanx, Chna Lnzheng@malst.xjtu.edu.cn
More informationTime Series Analysis in Studies of AGN Variability. Bradley M. Peterson The Ohio State University
Tme Seres Analyss n Studes of AGN Varablty Bradley M. Peterson The Oho State Unversty 1 Lnear Correlaton Degree to whch two parameters are lnearly correlated can be expressed n terms of the lnear correlaton
More informationDynamic Resource Allocation for MapReduce with Partitioning Skew
Ths artcle has been accepted for publcaton n a future ssue of ths journal, but has not been fully edted. Content may change pror to fnal publcaton. Ctaton nformaton: DOI 1.119/TC.216.253286, IEEE Transactons
More informationA DATA MINING APPLICATION IN A STUDENT DATABASE
JOURNAL OF AERONAUTICS AND SPACE TECHNOLOGIES JULY 005 VOLUME NUMBER (5357) A DATA MINING APPLICATION IN A STUDENT DATABASE Şenol Zafer ERDOĞAN Maltepe Ünversty Faculty of Engneerng BüyükbakkalköyIstanbul
More informationRealistic Image Synthesis
Realstc Image Synthess  Combned Samplng and Path Tracng  Phlpp Slusallek Karol Myszkowsk Vncent Pegoraro Overvew: Today Combned Samplng (Multple Importance Samplng) Renderng and Measurng Equaton Random
More informationAn Enhanced SuperResolution System with Improved Image Registration, Automatic Image Selection, and Image Enhancement
An Enhanced SuperResoluton System wth Improved Image Regstraton, Automatc Image Selecton, and Image Enhancement YuChuan Kuo ( ), ChenYu Chen ( ), and ChouShann Fuh ( ) Department of Computer Scence
More informationModeling and Analysis of 2D Service Differentiation on ecommerce Servers
Modelng and Analyss of D Servce Dfferentaton on ecommerce Servers Xaobo Zhou, Unversty of Colorado, Colorado Sprng, CO zbo@cs.uccs.edu Janbn We and ChengZhong Xu Wayne State Unversty, Detrot, Mchgan
More informationPAS: A Packet Accounting System to Limit the Effects of DoS & DDoS. Debish Fesehaye & Klara Naherstedt University of IllinoisUrbana Champaign
PAS: A Packet Accountng System to Lmt the Effects of DoS & DDoS Debsh Fesehaye & Klara Naherstedt Unversty of IllnosUrbana Champagn DoS and DDoS DDoS attacks are ncreasng threats to our dgtal world. Exstng
More informationDynamic Pricing for Smart Grid with Reinforcement Learning
Dynamc Prcng for Smart Grd wth Renforcement Learnng ByungGook Km, Yu Zhang, Mhaela van der Schaar, and JangWon Lee Samsung Electroncs, Suwon, Korea Department of Electrcal Engneerng, UCLA, Los Angeles,
More informationPassive Filters. References: Barbow (pp 265275), Hayes & Horowitz (pp 3260), Rizzoni (Chap. 6)
Passve Flters eferences: Barbow (pp 6575), Hayes & Horowtz (pp 360), zzon (Chap. 6) Frequencyselectve or flter crcuts pass to the output only those nput sgnals that are n a desred range of frequences (called
More informationStudy on CET4 Marks in China s Graded English Teaching
Study on CET4 Marks n Chna s Graded Englsh Teachng CHE We College of Foregn Studes, Shandong Insttute of Busness and Technology, P.R.Chna, 264005 Abstract: Ths paper deploys Logt model, and decomposes
More informationDevelopment of an intelligent system for tool wear monitoring applying neural networks
of Achevements n Materals and Manufacturng Engneerng VOLUME 14 ISSUE 12 JanuaryFebruary 2006 Development of an ntellgent system for tool wear montorng applyng neural networks A. Antć a, J. Hodolč a,
More informationSketching Sampled Data Streams
Sketchng Sampled Data Streams Florn Rusu, Aln Dobra CISE Department Unversty of Florda Ganesvlle, FL, USA frusu@cse.ufl.edu adobra@cse.ufl.edu Abstract Samplng s used as a unversal method to reduce the
More informationTime Domain simulation of PD Propagation in XLPE Cables Considering Frequency Dependent Parameters
Internatonal Journal of Smart Grd and Clean Energy Tme Doman smulaton of PD Propagaton n XLPE Cables Consderng Frequency Dependent Parameters We Zhang a, Jan He b, Ln Tan b, Xuejun Lv b, HongJe L a *
More informationA Genetic Programming Based Stock Price Predictor together with MeanVariance Based Sell/Buy Actions
Proceedngs of the World Congress on Engneerng 28 Vol II WCE 28, July 24, 28, London, U.K. A Genetc Programmng Based Stock Prce Predctor together wth MeanVarance Based Sell/Buy Actons Ramn Rajaboun and
More informationResearch Article QoS and Energy Aware Cooperative Routing Protocol for Wildfire Monitoring Wireless Sensor Networks
The Scentfc World Journal Volume 3, Artcle ID 43796, pages http://dx.do.org/.55/3/43796 Research Artcle QoS and Energy Aware Cooperatve Routng Protocol for Wldfre Montorng Wreless Sensor Networks Mohamed
More informationTable of Contents EQ.10...46 EQ.6...46 EQ.8...46
Table of Contents CHAPTER II  PATTERN RECOGNITION.... THE PATTERN RECOGNITION PROBLEM.... STATISTICAL FORMULATION OF CLASSIFIERS...6 3. CONCLUSIONS...30 UNDERSTANDING BAYES RULE...3 BAYESIAN THRESHOLD...33
More informationA Fast Incremental Spectral Clustering for Large Data Sets
2011 12th Internatonal Conference on Parallel and Dstrbuted Computng, Applcatons and Technologes A Fast Incremental Spectral Clusterng for Large Data Sets Tengteng Kong 1,YeTan 1, Hong Shen 1,2 1 School
More informationRESEARCH ON DUALSHAKER SINE VIBRATION CONTROL. Yaoqi FENG 1, Hanping QIU 1. China Academy of Space Technology (CAST) yaoqi.feng@yahoo.
ICSV4 Carns Australa 9 July, 007 RESEARCH ON DUALSHAKER SINE VIBRATION CONTROL Yaoq FENG, Hanpng QIU Dynamc Test Laboratory, BISEE Chna Academy of Space Technology (CAST) yaoq.feng@yahoo.com Abstract
More informationEnsembling Neural Networks: Many Could Be Better Than All
Artfcal Intellgence, 22, vol.37, no.2, pp.239263. @Elsever Ensemblng eural etworks: Many Could Be Better Than All ZhHua Zhou*, Janxn Wu, We Tang atonal Laboratory for ovel Software Technology, anng
More information