Artificial neural networks and deep learning
|
|
- Mervyn Gibbs
- 7 years ago
- Views:
Transcription
1 February 20, 2015
2 1 Introduction Artificia Neura Networks (ANNs) are a set of statistica modeing toos originay inspired by studies of bioogica neura networks in animas, for exampe the brain and the centra nervous system. Concentrated research on artificia neura networks began in 1943, foowing work by McCuoh and Pitts, who showed the abiity of neura networks to compute arithmetic and ogica functions (Basheer and Hameer, 2000). ANNs have a wide variety of appications spanning a range of fieds. Part of the reason for this is due to their fexibiity, ANNs can be used to accompish a wide variety of statistica tasks incuding cassification, custering, regression and forecasting. ANNs are universa function approximators, meaning that they can approximate any function to a desired degree of accuracy. The use of ANNs in function approximation has been considered extensivey, particuary in microbioogy (Basheer and Hameer, 2000). A particuary important deveopment in the study of ANNs was buiding invariance to various transformations of the observed data into the structure of an ANN itsef (LeCun et a., 1998). This technique has been used extensivey for pattern recognition tasks in machine earning, such as in image anaysis and speech recognition. More recenty, with increases in computing power, ANNs of this type set a new benchmark in image cassification, as they became the first agorithms to achieve human-competitive performance. For a ong time ANNs had to be imited in size, due to difficuties in training. However Hinton et a. (2006) proposed a procedure which coud be used with much arger networks. This deveopment caused an increased interest in ANN research, as modeing with arger networks aowed more sophisticated modes to be buit. This report outines key definitions in the study of ANNs, aong with basic training procedures for the modes. Particuary important techniques that have been introduced more recenty, especiay among the machine earning community, are then discussed. 2 Definition We focus on a particuar cass of ANNs which have been most appied in the iterature. They are known as feedforward artificia neura networks. A feedforward ANN consists of ayers of units. There are generay one or more hidden ayers and one output ayer. Let us pick a particuar unit ocated in the k th hidden ayer with input variabes z (k 1) 1,..., z (k 1). Then the input to this unit wi consist of a inear combination of the input variabes D a (k) = w (k) z (k 1) + w (k) 0, (2.1) where w is the weight vector for this unit, and w 0 is a constant known as the bias for this unit. The input a is then transformed by a function h(.) to give the activation z of the unit z (k+1) = h(a (k) ). (2.2) This activation is then passed as an input to the next ayer of units. We define z (0) := x, where x is an observed vector. Generay we choose h(.) to be sigmoida. y W (2) z (1) W (1) x Output Hidden Input Figure 1: Diagram of a singe-hidden ayer feedforward neura network. 1
3 The units in the fina hidden ayer have the same structure as the hidden units defined in (2.1), (2.2), however their activations are passed as outputs y of the network. Feedforward simpy indicates that the activations of each unit is sent ony to the next ayer, not to a ayer behind it. Figure 1 depicts a singe-hidden ayer feedforward network. 3 Parameter optimization Due to the compexity of ANNs, the chosen error function E(W) (often found using the ikeihood function), can rarey be minimised anayticay. This function is aso rarey convex, so generay a good oca minima is found instead of the goba minima. For shaow networks, these can generay be found by running a numerica agorithm such as gradient descent mutipe times at different starting points. For arge datasets however, these methods take a ong time to perform each iteration. An aternative procedure, which performs better for arge datasets, is known as stochastic gradient descent. This method does not use a the data at each iteration, instead using a sampe of the data. In order to use these gradient based methods however, we require an efficient way to cacuate E(W) for a feedforward neura network. Two methods are outined in the next two sections. 3.1 Error backpropagation Error backpropagation provides us with a computationay efficient method for evauating derivatives for a broad cass of error functions of an ANN with respect to the weights. Generay an error function E(W), such as east squares error, comprises of a sum of terms for each data point. Ca each term E n (W). Then WLOG, we derive the evauation of one term E n (W), as in (Bishop, 2006), and drop the subscript n to avoid cuttered notation. Consider the derivative of E with respect to a weight w (k) i. Generay E depends on w(k) i ony via a (k). Assuming this is true then the chain rue aows us to write E w (k) i = E w (k) i = δ (k) z (k 1) i, (3.1) where z (k 1) i is obtained using (2.1) and we have defined δ (k) := E. (3.2) By using the chain rue again we obtain δ (k 1) = E a (k 1) = E a (k 1), where the index runs over a units to which unit sends connections to. Finay by making use of equations (2.2), (2.1) and the definition of δ we obtain the backpropagation formua ( ) δ (k 1) = h a (k 1) w (k) δ (k). (3.3) This aows us to write down the error backpropagation procedure given L 1 hidden ayers as foows: 1. Find the activations of a units for a given input vector x n using equations (2.1) and (2.2). 2. Evauate δ (L) for each output unit using (3.2). 3. Obtain δ (k) for each hidden unit using (3.3). 2
4 4. Use (3.1) to evauate the required derivatives. Error propagation can be extended to find the Hessian of the error and associated approximations. This aows more efficient agorithms such as quasi-newton methods to be used to minimise the error function. 3.2 Extreme earning machines Whie error backpropagation was a maor breakthrough in the study of neura networks, the agorithm was often quite sow, and woud sometimes get stuck at poor oca minima. However for singe-hidden ayer feedforward ANNs a new agorithm was deveoped, as described in Huang et a. (2006). This is significanty faster than traditiona error backpropagation agorithms and anayticay determines the output weights. Suppose we have a singe output function whose activation is inear, giving ( where h(a (1) ) = ) h(a (1) 1 ),..., h(a(1) L ). It was shown by Huang et a. (2006) that the input weights w (1) y(x; W) = w (2) h(a (1) ), (3.4) (incuding biases) can be randomy assigned and do not require tuning provided the activation function is infinitey differentiabe. This aows us to write down the foowing agorithm: 1. Randomy assign input weights w (1). 2. Using these weights and the avaiabe training data, write down the hidden ayer output matrix H = h(w (1) 1 x 1)... h(w (1) L x 1).... h(w (1) 1 x D)... h(w (1) L x D) 3. An estimate ˆβ of the vector of output weights w (2) can then be found by soving Hˆβ T where T is the matrix of target vectors. This has the soution where H is the Moore-Penrose generaised inverse of the matrix H. ˆβ = H T, (3.5) The agorithm has been used for a variety of appications, such as image processing, faut detection and forecasting. Despite its advantages, its use is imited as it can ony be appied to singe-ayer ANNs. Whie the agorithm is faster than backpropagation, generay the probem of finding good oca minima with error backpropagation is hard ony when there are a ot of ayers in the network. Therefore many procedures which require more than one hidden ayer, such as convoutiona neura networks discussed in the next section, sti use error backpropagation. 4 Deep earning 4.1 Convoutiona neura networks A maor appication for neura networks is image anaysis for machine earning. However in order for the network to pick up subte differences, for instance in handwriting, the network needs to be consideraby resistant to transformations in the input data. Technicay a neura network shoud eventuay become invariant to certain types of transformation by simpy being exposed to it in the training data. Therefore one technique is to augment the existing training data by appying random transformations to the data. Whie this is a beneficia technique, in practice it requires a arge neura network to be trained on a vast amounts of data which is computationay expensive. 3
5 A technique that has had a huge impact on machine earning is known as convoutiona neura networks (LeCun et a., 1998). The procedure takes advantage of the high correation that generay occurs between neighbouring pixes in images. This is performed by arranging the first hidden ayer units into a number of different grids, each grid is referred to as a feature map. This ayer of the neura network is known as the convoutiona ayer. Each unit wi take an input of a sma part of the image (for instance a 5 5 pixe grid). However the units within each feature map are constrained to have the same weight parameters. This aows each feature map to pick up different patterns in the images, but since each unit in the feature map is constrained to have the same weights, the ocation of the pattern is ess important. In order to make the neura network further invariant to transformations of the data, the convoutiona ayer passes its outputs to a second hidden ayer, known as the subsamping ayer. This ayer is again arranged into grids of units, however each unit in the subsamping ayer takes an input of severa units in the convoutiona ayer. This has the effect of reducing the resoution of the data. The fina ayer in a neura network wi typicay be a fuy connected ayer, with no weight sharing constraints. In practice a Convoutiona Neura Network wi have mutipe ayers of convoutiona and subsamping ayers. Figure 2 iustrates the LeNet-5 described in detai by LeCun et a. (1998). The weight sharing among units in the feature maps means there is a significant reduction in the parameters that need to be trained. This aows reativey arge convoutiona neura networks to be trained using a simpe modification of error backpropagation. However the depth of convoutiona ANNs, which tends to ead to better earning, is sti consideraby imited by computationa power. Figure 2: Iustration of the convoutiona neura network deveoped by (LeCun et a., 1998), [Source: (LeCun et a., 1998)]. 4.2 Energy based modes and deep beief networks As understanding of bioogica neura networks improved it became apparent that an important step for AI woud be the abiity to train ANNs with many hidden ayers. Few deep networks coud be trained efficienty with traditiona gradient based methods, though a notabe exception was convoutiona neura networks. In most deep networks the agorithm woud often get stuck at sadde points or poor oca minima, and the trained network woud often perform worse than a network with fewer ayers. A turning point in the study of ANNs was when Hinton et a. (2006) introduced deep beief networks (DBNs), which is a type of generative neura network where each unit is treated as a random variabe, rather than being deterministic. They showed that this network can be initiay trained efficienty ayer by ayer in an unsupervised way to find good vaues for the network parameters. These parameters can then be used to initiaise a sower agorithm for supervised earning. In order to introduce deep beief networks we first briefy outine energy based modes which can be used to define DBNs. Foowing the definition of Bengio (2009), energy based modes associate an energy vaue E(.) to each configuration of the variabes of interest x. The basic idea is that we woud ike pausibe configurations of x to have ow energy. In this sense we can define a probabiity distribution as P (x) = e E(x) Z, (4.1) 4
6 where Z = x e E(x) is a normaizing constant known as the partition function. We can therefore see that configurations with a ower energy function have a higher probabiity of being reaised. An extension of this is through the introduction of hidden, or unobserved variabes h. In this case the oint probabiity distribution can be written as in (4.1), but the energy now depends on x and h. The margina distribution of the observed variabes x can then be written as P (x) = h e E(x,h) Z = e F (x) Z, (4.2) where F (x) = og h e E(x,h), is known as the free energy. Given mode parameters θ, (4.2) aows us to write down the gradient of the og ikeihood as θ og P (x; θ) = θ F (x; θ) + 1 e F ( x;θ) θ F ( x; θ) (4.3) Z x = θ F (x; θ) + E P ( θ F (x; θ)), (4.4) where E P is the expectation under the margina distribution P. Therefore an estimate of the gradient of the og ikeihood can be found if we are abe to sampe from P and can compute the free energy tractaby. Restricted Botzmann Machines are a generative neura network that satisfy these properties, and form a key component of a DBN. A Restricted Botzmann Machine (RBM) is a type of Markov random fied that has two ayers of units, one visibe x, one hidden h. A graphica mode of an RBM is given in Figure 3a. As shown by Figure 3a, a key feature of RBMs is that the visibe units are independent of each other given the vaues of the hidden units and vice versa. h (2) h W x (a) Graphica mode of a Restricted Botzmann Machine. W (2) h (1) W (1) x (b) Graphica mode of a deep beief network. RBM Figure 3: Graphica modes for a Restricted Botzmann Machine and a deep beief network. The energy of an RBM is given by E(x, h) = b x c h h Wx, (4.5) where θ = {b, c, W} are the mode parameters. W are the interaction terms between the observed and hidden units and b and c are the bias vaues for the observed and hidden ayers respectivey. The conditiona independence of the units in the same ayer aows us to factorize the conditiona distributions as P (h x) = i P (h i x), so that we are abe to obtain a tractabe expression for P (h i x). Simiary we can obtain expressions for P (x i h). The form of the energy function aows the free energy to be computed efficienty as F (x) = b x i og hi e h i(c i +w i x). (4.6) 5
7 The tractabe expressions for P (h i x) and P (x i h) aow us to sampe from the distribution P using a Gibbs samper, where we initiaise using a sampe of the training data. Using this sampe, the observed data and (4.6); we are abe to obtain an approximation of the og ikeihood gradient. This used in conunction with a gradient descent agorithm aows us to approximatey minimize the og ikeihood to find maximum ikeihood estimates ˆθ. This procedure is known as contrastive divergence, a more detaied description of the agorithm can be found in Bengio (2009). A graphica mode of a deep beief network (DBN) is given in Figure 3b. As can be seen from the graphica mode, given a DBN with ayers, the units in the first 2 ayers of a deep beief network are independent given the vaues of the units in the ayers above. The oint distribution of the top two ayers in a DBN is the same as for a RBM. This aows the oint distribution to be written as ( 2 ) P (x, h (1),..., h () ) = P (h ( 1), h () ) P (h k h k+1 ) P (h (1) x), (4.7) where P (h ( 1), h () ) has the same form as a Restricted Botzmann Machine. Contrastive divergence for RBMs aows us to write down an agorithm for unsupervised training of deep beief networks by training each ayer as a Restricted Botzmann Machine: 1. Using contrastive divergence, find estimates for the first ayer parameters θ (1). 2. Fix the first ayer parameters θ (1) and using sampes from h (1) find estimates for the second ayer parameters θ (2) using contrastive divergence. 3. Recursivey repeat step 2 for the rest of the ayers in the network. After training the weights of the DBN in an unsupervised way as in the agorithm above, the weights can be used to initiaise a supervised agorithm which uses abeed training data. Since DBNs deveopment by Hinton et a. (2006), a variety of other deep architectures have been introduced, incuding convoutiona DBNs, and the fied of deep earning has gained significant industria interest. However more recenty, due to a significant increase in processing power, agorithms using traditiona convoutiona ANNs with many ayers and error backpropagation have become more feasibe. GPU-based impementations of traditiona convoutiona ANNs have been found to sti produce the best resuts for certain pattern recognition tasks such as image anaysis and speech recognition. A key criticism of DBNs is that the theoretica properties of the methods introduced, for exampe contrastive divergence, are not we understood. References Basheer, I. and Hameer, M. (2000). Artificia neura networks: fundamentas, computing, design, and appication. Journa of microbioogica methods, 43(1):3 31. Bengio, Y. (2009). Learning deep architectures for ai. Foundations and trends R in Machine Learning, 2(1): Bishop, C. M. (2006). Pattern recognition and machine earning, voume 4. Springer. Hinton, G., Osindero, S., and Teh, Y.-W. (2006). A fast earning agorithm for deep beief nets. Neura computation, 18(7): Huang, G.-B., Zhu, Q.-Y., and Siew, C.-K. (2006). Extreme earning machine: theory and appications. Neurocomputing, 70(1): LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. (1998). Gradient-based earning appied to document recognition. Proceedings of the IEEE, 86(11): k=1 6
Secure Network Coding with a Cost Criterion
Secure Network Coding with a Cost Criterion Jianong Tan, Murie Médard Laboratory for Information and Decision Systems Massachusetts Institute of Technoogy Cambridge, MA 0239, USA E-mai: {jianong, medard}@mit.edu
More informationAdvanced ColdFusion 4.0 Application Development - 3 - Server Clustering Using Bright Tiger
Advanced CodFusion 4.0 Appication Deveopment - CH 3 - Server Custering Using Bri.. Page 1 of 7 [Figures are not incuded in this sampe chapter] Advanced CodFusion 4.0 Appication Deveopment - 3 - Server
More informationFRAME BASED TEXTURE CLASSIFICATION BY CONSIDERING VARIOUS SPATIAL NEIGHBORHOODS. Karl Skretting and John Håkon Husøy
FRAME BASED TEXTURE CLASSIFICATION BY CONSIDERING VARIOUS SPATIAL NEIGHBORHOODS Kar Skretting and John Håkon Husøy University of Stavanger, Department of Eectrica and Computer Engineering N-4036 Stavanger,
More informationSimultaneous Routing and Power Allocation in CDMA Wireless Data Networks
Simutaneous Routing and Power Aocation in CDMA Wireess Data Networks Mikae Johansson *,LinXiao and Stephen Boyd * Department of Signas, Sensors and Systems Roya Institute of Technoogy, SE 00 Stockhom,
More informationChapter 3: e-business Integration Patterns
Chapter 3: e-business Integration Patterns Page 1 of 9 Chapter 3: e-business Integration Patterns "Consistency is the ast refuge of the unimaginative." Oscar Wide In This Chapter What Are Integration Patterns?
More informationCOMPARISON OF DIFFUSION MODELS IN ASTRONOMICAL OBJECT LOCALIZATION
COMPARISON OF DIFFUSION MODELS IN ASTRONOMICAL OBJECT LOCALIZATION Františe Mojžíš Department of Computing and Contro Engineering, ICT Prague, Technicá, 8 Prague frantise.mojzis@vscht.cz Abstract This
More informationFast Robust Hashing. ) [7] will be re-mapped (and therefore discarded), due to the load-balancing property of hashing.
Fast Robust Hashing Manue Urueña, David Larrabeiti and Pabo Serrano Universidad Caros III de Madrid E-89 Leganés (Madrid), Spain Emai: {muruenya,darra,pabo}@it.uc3m.es Abstract As statefu fow-aware services
More informationFixed income managers: evolution or revolution
Fixed income managers: evoution or revoution Traditiona approaches to managing fixed interest funds rey on benchmarks that may not represent optima risk and return outcomes. New techniques based on separate
More informationA Latent Variable Pairwise Classification Model of a Clustering Ensemble
A atent Variabe Pairwise Cassification Mode of a Custering Ensembe Vadimir Berikov Soboev Institute of mathematics, Novosibirsk State University, Russia berikov@math.nsc.ru http://www.math.nsc.ru Abstract.
More informationAn Idiot s guide to Support vector machines (SVMs)
An Idiot s guide to Support vector machines (SVMs) R. Berwick, Viage Idiot SVMs: A New Generation of Learning Agorithms Pre 1980: Amost a earning methods earned inear decision surfaces. Linear earning
More informationFinance 360 Problem Set #6 Solutions
Finance 360 Probem Set #6 Soutions 1) Suppose that you are the manager of an opera house. You have a constant margina cost of production equa to $50 (i.e. each additiona person in the theatre raises your
More informationFace Hallucination and Recognition
Face Haucination and Recognition Xiaogang Wang and Xiaoou Tang Department of Information Engineering, The Chinese University of Hong Kong {xgwang1, xtang}@ie.cuhk.edu.hk http://mmab.ie.cuhk.edu.hk Abstract.
More informationPricing Internet Services With Multiple Providers
Pricing Internet Services With Mutipe Providers Linhai He and Jean Warand Dept. of Eectrica Engineering and Computer Science University of Caifornia at Berkeey Berkeey, CA 94709 inhai, wr@eecs.berkeey.edu
More informationArt of Java Web Development By Neal Ford 624 pages US$44.95 Manning Publications, 2004 ISBN: 1-932394-06-0
IEEE DISTRIBUTED SYSTEMS ONLINE 1541-4922 2005 Pubished by the IEEE Computer Society Vo. 6, No. 5; May 2005 Editor: Marcin Paprzycki, http://www.cs.okstate.edu/%7emarcin/ Book Reviews: Java Toos and Frameworks
More informationSELECTING THE SUITABLE ERP SYSTEM: A FUZZY AHP APPROACH. Ufuk Cebeci
SELECTING THE SUITABLE ERP SYSTEM: A FUZZY AHP APPROACH Ufuk Cebeci Department of Industria Engineering, Istanbu Technica University, Macka, Istanbu, Turkey - ufuk_cebeci@yahoo.com Abstract An Enterprise
More informationBig Data projects and use cases. Claus Samuelsen IBM Analytics, Europe csa@dk.ibm.com
Big projects and use cases Caus Samuesen IBM Anaytics, Europe csa@dk.ibm.com IBM Sofware Overview of BigInsights IBM BigInsights Scientist Free Quick Start (non production): IBM Open Patform BigInsights
More informationFast b-matching via Sufficient Selection Belief Propagation
Fast b-matching via Sufficient Seection Beief Propagation Bert Huang Computer Science Department Coumbia University New York, NY 127 bert@cs.coumbia.edu Tony Jebara Computer Science Department Coumbia
More informationMulti-Robot Task Scheduling
Proc of IEEE Internationa Conference on Robotics and Automation, Karsruhe, Germany, 013 Muti-Robot Tas Scheduing Yu Zhang and Lynne E Parer Abstract The scheduing probem has been studied extensivey in
More informationAustralian Bureau of Statistics Management of Business Providers
Purpose Austraian Bureau of Statistics Management of Business Providers 1 The principa objective of the Austraian Bureau of Statistics (ABS) in respect of business providers is to impose the owest oad
More informationDiscounted Cash Flow Analysis (aka Engineering Economy)
Discounted Cash Fow Anaysis (aka Engineering Economy) Objective: To provide economic comparison of benefits and costs that occur over time Assumptions: Future benefits and costs can be predicted A Benefits,
More informationAssignment of Multiplicative Mixtures in Natural Images
Assignment of Mutipicative Mixtures in Natura Images Odeia Schwartz HHMI and Sak Institute La Joa, CA 94 odeia@sak.edu Terrence J. Sejnowski HHMI and Sak Institute La Joa, CA 94 terry@sak.edu Abstract
More informationA New Statistical Approach to Network Anomaly Detection
A New Statistica Approach to Network Anomay Detection Christian Caegari, Sandrine Vaton 2, and Michee Pagano Dept of Information Engineering, University of Pisa, ITALY E-mai: {christiancaegari,mpagano}@ietunipiit
More information3.5 Pendulum period. 2009-02-10 19:40:05 UTC / rev 4d4a39156f1e. g = 4π2 l T 2. g = 4π2 x1 m 4 s 2 = π 2 m s 2. 3.5 Pendulum period 68
68 68 3.5 Penduum period 68 3.5 Penduum period Is it coincidence that g, in units of meters per second squared, is 9.8, very cose to 2 9.87? Their proximity suggests a connection. Indeed, they are connected
More informationEarly access to FAS payments for members in poor health
Financia Assistance Scheme Eary access to FAS payments for members in poor heath Pension Protection Fund Protecting Peope s Futures The Financia Assistance Scheme is administered by the Pension Protection
More informationInterpreting Individual Classifications of Hierarchical Networks
Interpreting Individua Cassifications of Hierarchica Networks Wi Landecker Michae D. Thomure Luís M. A. Bettencourt Meanie Mitche Garrett T. Kenyon SFI WORKING PAPER: 2013-02-007 SFI Working Papers contain
More informationBetting Strategies, Market Selection, and the Wisdom of Crowds
Betting Strategies, Market Seection, and the Wisdom of Crowds Wiemien Kets Northwestern University w-kets@keogg.northwestern.edu David M. Pennock Microsoft Research New York City dpennock@microsoft.com
More informationMaintenance activities planning and grouping for complex structure systems
Maintenance activities panning and grouping for compex structure systems Hai Canh u, Phuc Do an, Anne Barros, Christophe Berenguer To cite this version: Hai Canh u, Phuc Do an, Anne Barros, Christophe
More informationWith the arrival of Java 2 Micro Edition (J2ME) and its industry
Knowedge-based Autonomous Agents for Pervasive Computing Using AgentLight Fernando L. Koch and John-Jues C. Meyer Utrecht University Project AgentLight is a mutiagent system-buiding framework targeting
More informationThe Use of Cooling-Factor Curves for Coordinating Fuses and Reclosers
he Use of ooing-factor urves for oordinating Fuses and Recosers arey J. ook Senior Member, IEEE S& Eectric ompany hicago, Iinois bstract his paper describes how to precisey coordinate distribution feeder
More informationBite-Size Steps to ITIL Success
7 Bite-Size Steps to ITIL Success Pus making a Business Case for ITIL! Do you want to impement ITIL but don t know where to start? 7 Bite-Size Steps to ITIL Success can hep you to decide whether ITIL can
More informationONE of the most challenging problems addressed by the
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 44, NO. 9, SEPTEMBER 2006 2587 A Mutieve Context-Based System for Cassification of Very High Spatia Resoution Images Lorenzo Bruzzone, Senior Member,
More informationLADDER SAFETY Table of Contents
Tabe of Contents SECTION 1. TRAINING PROGRAM INTRODUCTION..................3 Training Objectives...........................................3 Rationae for Training.........................................3
More informationAn Integrated Data Management Framework of Wireless Sensor Network
An Integrated Data Management Framework of Wireess Sensor Network for Agricutura Appications 1,2 Zhao Liang, 2 He Liyuan, 1 Zheng Fang, 1 Jin Xing 1 Coege of Science, Huazhong Agricutura University, Wuhan
More informationLarge-Scale Optimization of Hierarchical Features for Saliency Prediction in Natural Images
Large-Scae Optimization of Hierarchica Features for Saiency Prediction in Natura Images Eeonora Vig Harvard University vig@fas.harvard.edu Michae Dorr Harvard Medica Schoo michae.dorr@schepens.harvard.edu
More informationIntroduction to Machine Learning CMU-10701
Introduction to Machine Learning CMU-10701 Deep Learning Barnabás Póczos & Aarti Singh Credits Many of the pictures, results, and other materials are taken from: Ruslan Salakhutdinov Joshua Bengio Geoffrey
More informationMeasuring operational risk in financial institutions
Measuring operationa risk in financia institutions Operationa risk is now seen as a major risk for financia institutions. This paper considers the various methods avaiabe to measure operationa risk, and
More informationNCH Software Warp Speed PC Tune-up Software
NCH Software Warp Speed PC Tune-up Software This user guide has been created for use with Warp Speed PC Tune-up Software Version 1.xx NCH Software Technica Support If you have difficuties using Warp Speed
More informationNormalization of Database Tables. Functional Dependency. Examples of Functional Dependencies: So Now what is Normalization? Transitive Dependencies
ISM 602 Dr. Hamid Nemati Objectives The idea Dependencies Attributes and Design Understand concepts normaization (Higher-Leve Norma Forms) Learn how to normaize tabes Understand normaization and database
More informationBetting on the Real Line
Betting on the Rea Line Xi Gao 1, Yiing Chen 1,, and David M. Pennock 2 1 Harvard University, {xagao,yiing}@eecs.harvard.edu 2 Yahoo! Research, pennockd@yahoo-inc.com Abstract. We study the probem of designing
More informationEnhanced continuous, real-time detection, alarming and analysis of partial discharge events
DMS PDMG-RH DMS PDMG-RH Partia discharge monitor for GIS Partia discharge monitor for GIS Enhanced continuous, rea-time detection, aarming and anaysis of partia discharge events Unrivaed PDM feature set
More informationeg Enterprise vs. a Big 4 Monitoring Soution: Comparing Tota Cost of Ownership Restricted Rights Legend The information contained in this document is confidentia and subject to change without notice. No
More informationTeamwork. Abstract. 2.1 Overview
2 Teamwork Abstract This chapter presents one of the basic eements of software projects teamwork. It addresses how to buid teams in a way that promotes team members accountabiity and responsibiity, and
More informationA Supplier Evaluation System for Automotive Industry According To Iso/Ts 16949 Requirements
A Suppier Evauation System for Automotive Industry According To Iso/Ts 16949 Requirements DILEK PINAR ÖZTOP 1, ASLI AKSOY 2,*, NURSEL ÖZTÜRK 2 1 HONDA TR Purchasing Department, 41480, Çayırova - Gebze,
More informationMarket Design & Analysis for a P2P Backup System
Market Design & Anaysis for a P2P Backup System Sven Seuken Schoo of Engineering & Appied Sciences Harvard University, Cambridge, MA seuken@eecs.harvard.edu Denis Chares, Max Chickering, Sidd Puri Microsoft
More informationCI/SfB Ro8. (Aq) September 2012. The new advanced toughened glass. Pilkington Pyroclear Fire-resistant Glass
CI/SfB Ro8 (Aq) September 2012 The new advanced toughened gass Pikington Pyrocear Fire-resistant Gass Pikington Pyrocear, fire-resistant screens in the façade: a typica containment appication for integrity
More informationSpherical Correlation of Visual Representations for 3D Model Retrieval
Noname manuscript No. (wi be inserted by the editor) Spherica Correation of Visua Representations for 3D Mode Retrieva Ameesh Makadia Kostas Daniiidis the date of receipt and acceptance shoud be inserted
More informationSubject: Corns of En gineers and Bureau of Reclamation: Information on Potential Budgetarv Reductions for Fiscal Year 1998
GAO United States Genera Accounting Office Washington, D.C. 20548 Resources, Community, and Economic Deveopment Division B-276660 Apri 25, 1997 The Honorabe Pete V. Domenici Chairman The Honorabe Harry
More informationVendor Performance Measurement Using Fuzzy Logic Controller
The Journa of Mathematics and Computer Science Avaiabe onine at http://www.tjmcs.com The Journa of Mathematics and Computer Science Vo.2 No.2 (2011) 311-318 Performance Measurement Using Fuzzy Logic Controer
More informationMinimizing the Total Weighted Completion Time of Coflows in Datacenter Networks
Minimizing the Tota Weighted Competion Time of Cofows in Datacenter Networks Zhen Qiu Ciff Stein and Yuan Zhong ABSTRACT Communications in datacenter jobs (such as the shuffe operations in MapReduce appications
More informationAutomatic Structure Discovery for Large Source Code
Automatic Structure Discovery for Large Source Code By Sarge Rogatch Master Thesis Universiteit van Amsterdam, Artificia Inteigence, 2010 Automatic Structure Discovery for Large Source Code Page 1 of 130
More informationLeakage detection in water pipe networks using a Bayesian probabilistic framework
Probabiistic Engineering Mechanics 18 (2003) 315 327 www.esevier.com/ocate/probengmech Leakage detection in water pipe networks using a Bayesian probabiistic framework Z. Pouakis, D. Vaougeorgis, C. Papadimitriou*
More informationInsertion and deletion correcting DNA barcodes based on watermarks
Kracht and Schober BMC Bioinformatics (2015) 16:50 DOI 10.1186/s12859-015-0482-7 METHODOLOGY ARTICLE Open Access Insertion and deetion correcting DNA barcodes based on watermarks David Kracht * and Steffen
More informationThe Basel II Risk Parameters. Second edition
The Base II Risk Parameters Second edition . Bernd Engemann Editors Robert Rauhmeier The Base II Risk Parameters Estimation, Vaidation, Stress Testing with Appications to Loan Risk Management Editors Dr.
More informationElsevier Editorial System(tm) for Computers in Biology and Medicine Manuscript Draft
Esevier Editoria System(tm) for Computers in Bioogy and Medicine Manuscript Draft Manuscript Number: CBM-D--00R1 Tite: CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES Artice Type: Fu
More informationScience Unit 3 Light and Optical Systems Review Booklet In Action 8
Science Unit 3 Light and Optica Systems Review Booket n Action 8 1.0 Expanations, nventions & nvestigations about Light and Vision Key Concepts Scientific experiments to expain how ight and vision work
More informationSPOTLIGHT. A year of transformation
WINTER ISSUE 2014 2015 SPOTLIGHT Wecome to the winter issue of Oasis Spotight. These newsetters are designed to keep you upto-date with news about the Oasis community. This quartery issue features an artice
More informationWHITE PAPER BEsT PRAcTIcEs: PusHIng ExcEl BEyond ITs limits WITH InfoRmATIon optimization
Best Practices: Pushing Exce Beyond Its Limits with Information Optimization WHITE Best Practices: Pushing Exce Beyond Its Limits with Information Optimization Executive Overview Microsoft Exce is the
More informationVirtual trunk simulation
Virtua trunk simuation Samui Aato * Laboratory of Teecommunications Technoogy Hesinki University of Technoogy Sivia Giordano Laboratoire de Reseaux de Communication Ecoe Poytechnique Federae de Lausanne
More informationIn some states, however, the couple must live apart for a period of months or years before they can obtain a no fault divorce.
What is a "no faut" divorce? "No faut" divorce describes any divorce where the spouse asking for a divorce does not have to prove that the other spouse did something wrong. A states aow no faut divorces.
More informationWEBSITE ACCOUNT USER GUIDE SECURITY, PASSWORD & CONTACTS
WEBSITE ACCOUNT USER GUIDE SECURITY, PASSWORD & CONTACTS Password Reset Process Navigate to the og in screen Seect the Forgot Password ink You wi be asked to enter the emai address you registered with
More informationBooks on Reference and the Problem of Library Science
Practicing Reference... Learning from Library Science * Mary Whisner ** Ms. Whisner describes the method and some of the resuts reported in a recenty pubished book about the reference interview written
More informationWeek 3: Consumer and Firm Behaviour: The Work-Leisure Decision and Profit Maximization
AROEOOIS 2006 Week 3: onsumer and Firm Behaviour: The Work-Leisure Decision and Profit aximization Questions for Review 1. How are a consumer s preferences over goods represented? By utiity functions:
More informationAA Fixed Rate ISA Savings
AA Fixed Rate ISA Savings For the road ahead The Financia Services Authority is the independent financia services reguator. It requires us to give you this important information to hep you to decide whether
More informationLecture 7 Datalink Ethernet, Home. Datalink Layer Architectures
Lecture 7 Dataink Ethernet, Home Peter Steenkiste Schoo of Computer Science Department of Eectrica and Computer Engineering Carnegie Meon University 15-441 Networking, Spring 2004 http://www.cs.cmu.edu/~prs/15-441
More informationPresented at the 107th Convention 1999 September 24-27 New York
Room Simuation for Mutichanne Fim and Music 4993 (B-2) Knud Bank Christensen and Thomas Lund TC Eectronic A/S DK-8240 Risskov, Denmark Presented at the 107th Convention 1999 September 24-27 New York This
More informationWHITE PAPER UndERsTAndIng THE VAlUE of VIsUAl data discovery A guide To VIsUAlIzATIons
Understanding the Vaue of Visua Data Discovery A Guide to Visuaizations WHITE Tabe of Contents Executive Summary... 3 Chapter 1 - Datawatch Visuaizations... 4 Chapter 2 - Snapshot Visuaizations... 5 Bar
More informationEducation sector: Working conditions and job quality
European Foundation for the Improvement of Living and Working Conditions sector: Working conditions and job quaity Work pays a significant roe in peope s ives, in the functioning of companies and in society
More informationInfrastructure for Business
Infrastructure for Business The IoD Member Broadband Survey Infrastructure for Business 2013 #5 The IoD Member Broadband Survey The IoD Member Broadband Survey Written by: Corin Tayor, Senior Economic
More informationBreakeven analysis and short-term decision making
Chapter 20 Breakeven anaysis and short-term decision making REAL WORLD CASE This case study shows a typica situation in which management accounting can be hepfu. Read the case study now but ony attempt
More informationPacket Classification with Network Traffic Statistics
Packet Cassification with Network Traffic Statistics Yaxuan Qi and Jun Li Research Institute of Information Technoogy (RIIT), Tsinghua Uniersity Beijing, China, 100084 Abstract-- Packet cassification on
More informationWINMAG Graphics Management System
SECTION 10: page 1 Section 10: by Honeywe WINMAG Graphics Management System Contents What is WINMAG? WINMAG Text and Graphics WINMAG Text Ony Scenarios Fire/Emergency Management of Fauts & Disabement Historic
More informationBusiness schools are the academic setting where. The current crisis has highlighted the need to redefine the role of senior managers in organizations.
c r o s os r oi a d s REDISCOVERING THE ROLE OF BUSINESS SCHOOLS The current crisis has highighted the need to redefine the roe of senior managers in organizations. JORDI CANALS Professor and Dean, IESE
More informationACO and SVM Selection Feature Weighting of Network Intrusion Detection Method
, pp. 129-270 http://dx.doi.org/10.14257/ijsia.2015.9.4.24 ACO and SVM Seection Feature Weighting of Network Intrusion Detection Method Wang Xingzhu Furong Coege Hunan, University of Arts and Science,
More informationCLOUD service providers manage an enterprise-class
IEEE TRANSACTIONS ON XXXXXX, VOL X, NO X, XXXX 201X 1 Oruta: Privacy-Preserving Pubic Auditing for Shared Data in the Coud Boyang Wang, Baochun Li, Member, IEEE, and Hui Li, Member, IEEE Abstract With
More information3.3 SOFTWARE RISK MANAGEMENT (SRM)
93 3.3 SOFTWARE RISK MANAGEMENT (SRM) Fig. 3.2 SRM is a process buit in five steps. The steps are: Identify Anayse Pan Track Resove The process is continuous in nature and handed dynamicay throughout ifecyce
More informationChapter 3: JavaScript in Action Page 1 of 10. How to practice reading and writing JavaScript on a Web page
Chapter 3: JavaScript in Action Page 1 of 10 Chapter 3: JavaScript in Action In this chapter, you get your first opportunity to write JavaScript! This chapter introduces you to JavaScript propery. In addition,
More informationThe eg Suite Enabing Rea-Time Monitoring and Proactive Infrastructure Triage White Paper Restricted Rights Legend The information contained in this document is confidentia and subject to change without
More informationSemantics-based design for Secure Web Services
1 Semantics-based design for Secure Web Services Massimo Bartoetti Pierpaoo Degano Gian Luigi Ferrari Roberto Zunino bart@di.unipi.it degano@di.unipi.it giangi@di.unipi.it zunino@di.unipi.it Dipartimento
More informationOlder people s assets: using housing equity to pay for health and aged care
Key words: aged care; retirement savings; reverse mortgage; financia innovation; financia panning Oder peope s assets: using housing equity to pay for heath and aged care The research agenda on the ageing
More informationCERTIFICATE COURSE ON CLIMATE CHANGE AND SUSTAINABILITY. Course Offered By: Indian Environmental Society
CERTIFICATE COURSE ON CLIMATE CHANGE AND SUSTAINABILITY Course Offered By: Indian Environmenta Society INTRODUCTION The Indian Environmenta Society (IES) a dynamic and fexibe organization with a goba vision
More informationDynamic Pricing Trade Market for Shared Resources in IIU Federated Cloud
Dynamic Pricing Trade Market or Shared Resources in IIU Federated Coud Tongrang Fan 1, Jian Liu 1, Feng Gao 1 1Schoo o Inormation Science and Technoogy, Shiiazhuang Tiedao University, Shiiazhuang, 543,
More informationThis paper considers an inventory system with an assembly structure. In addition to uncertain customer
MANAGEMENT SCIENCE Vo. 51, No. 8, August 2005, pp. 1250 1265 issn 0025-1909 eissn 1526-5501 05 5108 1250 informs doi 10.1287/mnsc.1050.0394 2005 INFORMS Inventory Management for an Assemby System wh Product
More informationPROPAGATION OF SURGE WAVES ON NON-HOMOGENEOUS TRANSMISSION LINES INDUCED BY LIGHTNING STROKE
Advances in Eectrica and Eectronic Engineering 98 PROPAGATION OF SRGE WAVES ON NON-HOMOGENEOS TRANSMISSION LINES INDCED BY LIGHTNING STROKE Z. Benešová, V. Kotan WB Pisen, Facuty of Eectrica Engineering,
More informationEnergy Density / Energy Flux / Total Energy in 3D
Lecture 5 Phys 75 Energy Density / Energy Fux / Tota Energy in D Overview and Motivation: In this ecture we extend the discussion of the energy associated with wave otion to waves described by the D wave
More informationDesign and Analysis of a Hidden Peer-to-peer Backup Market
Design and Anaysis of a Hidden Peer-to-peer Backup Market Sven Seuken, Denis Chares, Max Chickering, Mary Czerwinski Kama Jain, David C. Parkes, Sidd Puri, and Desney Tan December, 2015 Abstract We present
More informationA quantum model for the stock market
A quantum mode for the stock market Authors: Chao Zhang a,, Lu Huang b Affiiations: a Schoo of Physics and Engineering, Sun Yat-sen University, Guangzhou 5175, China b Schoo of Economics and Business Administration,
More informationSensing Meets Mobile Social Networks: The Design, Implementation and Evaluation of the CenceMe Application
Sensing Meets Mobie Socia Networks: The Design, Impementation and Evauation of the CenceMe Appication Emiiano Miuzzo, Nichoas D. Lane, Kristóf Fodor, Ronad Peterson, Hong Lu, Mirco Musoesi, Shane B. Eisenman,
More informationOverview of Health and Safety in China
Overview of Heath and Safety in China Hongyuan Wei 1, Leping Dang 1, and Mark Hoye 2 1 Schoo of Chemica Engineering, Tianjin University, Tianjin 300072, P R China, E-mai: david.wei@tju.edu.cn 2 AstraZeneca
More informationThe growth of online Internet services during the past decade has
IEEE DS Onine, Voume 2, Number 4 Designing an Adaptive CORBA Load Baancing Service Using TAO Ossama Othman, Caros O'Ryan, and Dougas C. Schmidt University of Caifornia, Irvine The growth of onine Internet
More informationLoad Balancing in Distributed Web Server Systems with Partial Document Replication *
Load Baancing in Distributed Web Server Systems with Partia Document Repication * Ling Zhuo Cho-Li Wang Francis C. M. Lau Department of Computer Science and Information Systems The University of Hong Kong
More informationSentiment Analysis with Global Topics and Local Dependency
Proceedings of the Tenty-Fourth AAAI Conference on Artificia Inteigence (AAAI-10) Sentiment Anaysis ith Goba Topics and Loca Dependency Fangtao Li, Minie Huang, Xiaoyan Zhu State Key Laboratory of Inteigent
More informationInternational classification of financial reporting
3 Internationa cassification of financia reporting Christopher Nobes CONTENTS 3.1 Introduction 3.2 The nature of cassification 3.3 Cassifications by socia scientists 3.4 Cassifications in accounting 3.5
More informationCooperative Content Distribution and Traffic Engineering in an ISP Network
Cooperative Content Distribution and Traffic Engineering in an ISP Network Wenjie Jiang, Rui Zhang-Shen, Jennifer Rexford, Mung Chiang Department of Computer Science, and Department of Eectrica Engineering
More informationApplication-Aware Data Collection in Wireless Sensor Networks
Appication-Aware Data Coection in Wireess Sensor Networks Xiaoin Fang *, Hong Gao *, Jianzhong Li *, and Yingshu Li +* * Schoo of Computer Science and Technoogy, Harbin Institute of Technoogy, Harbin,
More informationGREEN: An Active Queue Management Algorithm for a Self Managed Internet
: An Active Queue Management Agorithm for a Sef Managed Internet Bartek Wydrowski and Moshe Zukerman ARC Specia Research Centre for Utra-Broadband Information Networks, EEE Department, The University of
More informationWe are XMA and Viglen.
alearn with Microsoft 16pp 21.07_Layout 1 22/12/2014 10:49 Page 1 FRONT COVER alearn with Microsoft We are XMA and Vigen. Ca us now on 0115 846 4900 Visit www.xma.co.uk/aearn Emai alearn@xma.co.uk Foow
More informationComparison of Traditional and Open-Access Appointment Scheduling for Exponentially Distributed Service Time
Journa of Heathcare Engineering Vo. 6 No. 3 Page 34 376 34 Comparison of Traditiona and Open-Access Appointment Scheduing for Exponentiay Distributed Service Chongjun Yan, PhD; Jiafu Tang *, PhD; Bowen
More informationSNMP Reference Guide for Avaya Communication Manager
SNMP Reference Guide for Avaya Communication Manager 03-602013 Issue 1.0 Feburary 2007 2006 Avaya Inc. A Rights Reserved. Notice Whie reasonabe efforts were made to ensure that the information in this
More informationADVANCED ACCOUNTING SOFTWARE FOR GROWING BUSINESSES
ADVANCED ACCOUNTING SOFTWARE FOR GROWING BUSINESSES Product Features 1. System 2. Saes Ledger Unimited companies with password protection User security Muti-user system: 1 user comes as standard, up to
More informationIEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 31, NO. 12, DECEMBER 2013 1
IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 31, NO. 12, DECEMBER 2013 1 Scaabe Muti-Cass Traffic Management in Data Center Backbone Networks Amitabha Ghosh, Sangtae Ha, Edward Crabbe, and Jennifer
More information