Design of FollowUp Experiments for Improving Model Discrimination and Parameter Estimation


 Melinda Greene
 3 years ago
 Views:
Transcription
1 Design of FoowUp Experiments for Improving Mode Discrimination and Parameter Estimation Szu Hui Ng 1 Stephen E. Chick 2 Nationa University of Singapore, 10 Kent Ridge Crescent, Singapore Technoogy Management Area, INSEAD, Fontainebeau CEDEX, France. One goa of experimentation is to identify which design parameters most significanty infuence the mean performance of a system. Another goa is to obtain good parameter estimates for a response mode that quantifies how the mean performance depends on infuentia parameters. Most experimenta design techniques focus on one goa at a time. This paper proposes a new entropybased design criterion for foowup experiments that jointy identifies the important parameters and reduces the variance of parameter estimates. We simpify computations for the norma inear mode by identifying an approximation that eads to a cosed form soution. The criterion is appied to an exampe from the experimenta design iterature, to a known mode and to a critica care faciity simuation experiment. (Design of Experiments; Mode discrimination; Parameter estimation; Entropy; Simuation) 1. Introduction A common purpose of many experiments is to obtain an adequate mathematica mode of the underying system, incuding the functiona form, and precise estimates of the mode s parameters. Response modes that describe the reationship between inputs to a system and the output can be usefu for design decisions, and much focus has gone into seecting inputs in a way that improves the estimate of the response mode [9, 29, 30, 40]. Response modes can be used in iterative processes to identify design parameters (e.g., number of servers, production ine speeds) that optimize some expected reward criterion (e.g., mean monthy revenue, average output), or to provide intuition about how input factors infuence aggregate system behavior. In simuation, response modes can reate the parameters of stochastic modes (e.g., demand arriva rates, infection transmission parameters) to system performance [2, 13, 24, 25, 32, 35]. 1 Corresponding author. 2 The authors acknowedge the financia support of the Nationa Institutes of Heath (grant R01 AI A1). 1
2 A number of design criteria are avaiabe to seect design factors (or inputs) for experiments. Severa authors [6, 16, 26] use the expected gain in Shannon information (or decrease in entropy) as an optima design criterion to seect vaues for the experiment s design factors. Bernardo [6] and Smith and Verdinei [38] adopted this approach and ooked at how to pan experiments to ensure precise estimates of the mode s parameters. However, many experiments aim to identify which factors most infuence the system response. Identifying the subset of most important parameters can be phrased as a mode seection probem [34]. Box and Hi [10] used Shannon information to deveop the MD design criterion for discriminating among mutipe candidate modes. Hi [21] stressed the importance of experimenta design for the joint objective of mode discrimination and parameter inference in his review of design procedures. But most design criteria focus either on identifying important parameters or improving estimates of response parameters, but not both. Exceptions are Hi et a. [22], whose joint criterion requires certain mode parameters to be estimated or known, and Borth [8], whose entropybased criterion can be chaenging to compute. This paper describes a new joint criterion for experimenta design that seects designs to simutaneousy identify important factors and to reduce the variance of the response mode parameter estimates. The new criterion is shown to simpify to a cosed form for the standard inear regression mode with norma observation errors, and is computationay more efficient than Borth s criterion. Our criterion does not require initia estimates of the mode parameters and incorporates prior information and data from preiminary experiments. It is fexibe for use in either starting or foowup experiments, particuary if resuts remain inconcusive about which factors most infuence the system response, and when the parameters are sti poory understood after an initia response surface experiment has been competed. We consider designs with a given number n of observations, and do not describe how to baance initia runs with foowup runs. Section 2 describes the mathematica formuation for the design space and response modes. It aso describes a Bayesian formuation to quantify input mode and parameter uncertainty, as we as the new entropy based design criterion. Three numerica experiments in Section 3 show that optima designs depend heaviy on the criterion seected, and highight the benefits and tradeoffs of the new joint criterion over individua mode discrimination and parameter estimation criteria, as we as existing joint criteria. The exampes stress the need for baancing the two types of entropy measures of the joint criterion. For the exampes considered, we aso find that the weights in our criterion are robust to misspecification. The new criterion does we at both identifying important factors and reducing parameter uncertainty, and is computationay more efficient than 2
3 Borth s joint criterion. 2. Formaism The design criterion is appied to a finite, but perhaps arge, set of potentia factors in a finite number n of runs, where n is seected by the experimenter. The design space and cass of regression modes is described before the new entropybased design criterion and computationa issues. 2.1 Design Space and Regression Modes Experiments often invove severa factors. Here we consider representing the performance of the systems by the usua inear mode. We consider a finite number q of reavaued factor inputs, x 1,..., x q, each of which may be chosen to take on a finite set of different vaues. These factors can be combined agebraicay to generate a finite number, p, of predictors, y 1,..., y p, each of which is some function of the input factors. We foow the formuation of Raftery et a. [34] to identify the most important of the p predictors. That is, we presume the existence of s = 2 p candidate response modes in a mode space M that are inear in some subset of the predictors. Assuming the th candidate response mode, the output z i of the ith run is presumed to be of the form z i = β 0 + β 1 y i,(1) + β 2 y i,(2) + + β t y i,(t) + ζ i, (1) where y i,(1),..., y i,(t) are the t predictors present in the th mode, the vaues of the β j may depend upon, and ζ i is a zero mean noise term. The seection of a candidate response mode identifies the important predictors, reative to the size of the noise in the response. See aso George [20]. Let D be the (finite) design space of a possibe ega combinations of the inputs for each of the n runs. A design x D can be represented as an n q matrix whose ith row contains the vaues of the factors for the ith run. If mode M has t predictors, the design matrix can be converted to an n (t + 1) predictor matrix y = y (x) whose rows contain the vaues of predictors for each run, and the first coumn corresponds to the intercept. Let z be the coumn vector of n outputs. 2.2 EntropyBased Formuation The probem is to choose a design x that in some sense is effective at identifying the most important predictors (i.e., seects the most appropriate mode in M), and estimate regression parameters. 3
4 We assess uncertainty about response mode seection and parameter estimation with probabiity distributions. The design that most improves an entropybased criterion is then seected Uncertainty Assessment One Bayesian approach to quantify the joint uncertainty about mode form and parameter vaues is to assign a prior distribution to each of the modes M M, then assign a conditiona probabiity distribution for the parameter vector β, given M. The identity of the best response mode and parameter is then inferred by Bayes rue, using the prior distributions and the probabiity distribution of the output, given the mode and input parameters. This is the approach taken by [14, 28, 34]. We make a standard assumption of jointy independent, normay distributed errors, ζ Norma (0, σ 2 ), so if mode M is the mode, β is the parameter, x is the design with predictor matrix y = y (x), then the output Z has an mutivariate norma distribution, p(z M, β, σ 2, x) Norma ( y β, σ 2 I n ), where I n is the identity matrix. For prior distributions, we presume a conjugate prior distribution [5] for the unknown θ = (β, σ 2 ), conditiona on the th mode M, π(β M, σ 2 ) Norma ( ) β µ, σ 2 V ( π(σ 2 M ) InvertedGamma σ 2 ν 2, νλ 2 where the conditiona prior mean vector µ and covariance matrix σ 2 V for β may depend on the mode M. The parameters ν and λ are seected by the modeer. The InvertedGamma (x α, β) distribution has pdf x (α+1) e β/x β α /Γ(α) and mean β/(α 1). Raftery et a. [34] suggest vaues of µ, V, ν and λ that minimize the infuence of the priors in numerica experiments. The distributions in Eq. (2) can either be based on prior information aone, or can incude information gained during initia stages of experimentation. Data z 0 from an initia stage of n 0 observations with predictor matrix y 0 is straightforward to incorporate because of the conjugate ( ) 1 ( form [5]. Repace the mean µ with µ = V 1 + y T 0 y 0 V 1 µ + y T 0 z 0 ); repace V with ( ) 1; V 1 + y T 0 y 0 repace ν/2 with (ν + n0 )/2; and repace νλ/2 with (νλ + (z 0 y 0 µ )T z 0 + (µ µ )T V 1 µ )/2. Choices used here for the prior distribution of M incude the discrete uniform (p(m ) = 1/s) and the independence prior p(m i ) = ω t i (1 ω) p t i, where t i is the number of predictors in mode i, (i = 1,..., s), and ω is the prior probabiity that a predictor is active. Raftery et a. [34] provide cosed form formuas to update the probabiities p(m z 0, y 0 ). 4 ), (2)
5 In the rest of the paper, the prior distribution for the foow up stage is based on a prior distribution from Eq. (2) in combination with data from an initia stage. The optima baance of the amount of initia stage data versus the amount of foowup data is beyond the scope of this paper Modeing Remarks Considering posterior probabiities is a usefu way to assess the reative merits of the modes [20]. Seecting modes according to p(m Z) is consistent in that if one of the entertained modes is actuay the true mode, then it wi seect the true mode if enough data is observed. When the true mode is not among those being considered, Bayesian mode seection chooses the candidate that is cosest to the true mode in terms of KubackLeiber divergence [4, 5, 18]. In practice, the true mode is typicay not known and is potentiay not in M. Despite this, carefu seection of a cass of approximating modes is important in the understanding of many probems. Here we seek a mode within the cass that is approximatey correct (containing ony significant predictors) and that approximates the parameters of the true underying response mode with ow variance [9]. Atkinson [1] raises severa concerns about inference for regression modes. Our prior probabiity framework avoids by design his concern about improper prior distributions for modes. A concern about nesting, so that two modes may be true, is resoved by noting that the simper mode wi be identified as more data is coected, and that simper modes are more desirabe expanations [19, 28]. Atkinson [1] aso indicates that if two response modes are compared, the true mode and an incorrect mode with fewer parameters, then asymptoticay the correct mode wi be seected, but that for finite numbers of sampes the posterior probabiities may support the incorrect mode in the absence of strong evidence from the data. This is a cause for care, but is not a vioation of the ikeihood principa, and negative consequences for seecting a mode when the data do not provide enough evidence is a probem for any seection criterion. A goodnessoffit test may be usefu to provide further post hoc vaidation EntropyBased Criteria Severa authors [6, 16, 26] proposed the use of the expected gain in Shannon information (or decrease in entropy) given by an experiment as an optima design criterion. This expected gain is a natura measure of the utiity of an experiment. The choice of design infuences the expected gain in information as the predictive distribution of future output Z is determined by the design x, mode M, and the prior distribution in Eq. (2) p(z M, σ 2, x) Norma 5 (yµ, σ 2 [yv y T + I n ]). (3)
6 The margina distribution of Z given M, y, obtained by integrating out σ 2, is a mutivariate t distribution. Entropy is different for discrete (mode seection) and continuous (parameter estimation) random variabes, so each is discussed in turn. For mode seection, Box and Hi [10] use the expected increase in Shannon information J as a design criterion. The criterion was derived from information theory where the information (entropy) was used as a measure of uncertainty for distinguishing the s candidate modes. ( ) J = p(m ) og p(m ) + p(m Z, y ) og p(m Z, y ) p(z y )dz (4) = =1 p(m ) =1 og =1 p(z M, y ) s =1 p(z M, y )p(m ) p(z M, y )dz An expicit soution is unknown in genera, so J may be evauated numericay or approximated. Aternatey, Box and Hi [10] gave an upper bound approximation, the expected gain in Shannon information between the predictive distributions of each pair of candidate modes M i and M. This approximation was originay named the Dcriterion, but we use the notation MD, as in [28]. MD = ( p(m i )p(m ) p(z M i, y i ) og p(z M ) i, y i ) p(z M, y ) dz 0 i s The MD criterion is effective in practice and popuar with research workers [21]. We use MD for the mode discrimination portion of our joint criterion. For the norma inear mode, Meyer et a. [28] show that MD reduces to a cosed form if a noninformative prior 1/σ on σ and a conditionay norma prior for β given σ are assumed. A cosed form aso resuts if the conjugate prior is assumed. Proposition 1. Assume the conjugate norma gamma prior in Eq. (2). Let ẑ = y µ, and V = [y V y T + I ]. Then for the inear mode, MD simpifies to MD = 0 i s Proof. See Appendix A.1 [ 1 2 p(m i)p(m ) n + tr(v 1 V i ) + 1 ] λ (ẑ i ẑ ) T V 1 (ẑ i ẑ ) For parameter estimation, Bernardo [6] and Smith and Verdinei [38] adopted an entropy based method to ensure precise estimates for parameters that have aready been identified as important. They choose the design that maximizes the expected gain in Shannon information (or equivaenty, (5) (6) 6
7 maximizes the expected KubackLeiber distance) between the posterior and prior distributions of the parameters θ = (β, σ 2 ). BD = [ ] p(θ Z) p(z)p(θ Z) og dθdz (7) p(θ) Eq. (7) simpifies consideraby for the norma inear mode into a form known as the Bayesian Doptima criterion (hence the choice of name BD). Proposition 2. For a inear mode M of the form Eq. (1), the prior probabiity mode Eq. (2), and a given design y, Proof. See Appendix A.2. BD = 1 y 2 og T y + V og V 1 Foowing Borth [8], the entropy criterion S P for parameter uncertainty generaizes when there are mutipe candidate modes. S P = + =1 p(m ) =1 p(m Z) p(θ M ) og p(θ M )dθ (8) p(θ Z, M ) og p(θ Z, M )dθ Proposition 3. For the norma inear mode, S P simpifies to S P = =1 p(m ) 2 for some K that does not depend on the design. Proof. See Appendix A Joint Criterion p(m )p(z M )dz =1 og y T y + V 1 + K (9) In order to account for both mode discrimination and parameter estimation simutaneousy, Hi et a. [22] proposed a joint criterion that adds a weighted measure of discrimination and precision, C = w 1 D 0 + w 2 E 0, (10) where D 0 is some measure of discrimination and E 0 is some measure of precision in parameter estimation. A nonunique choice of D 0 and E 0 they suggest is the mode discrimination criterion 7
8 proposed by Box and Hi [10], and the determinant of the regression matrix for estimating the parameters for mode i. where MD and E i C = w MD MD + (1 w) i=1 p(m i ) E i, (11) Ei are the maximum vaues of MD and E i over the design region, and w is a nonnegative weight paced on mode discrimination. They assumed that σ 2 is known or can be estimated when computing C. As the two criteria are summed together and weighted by w, the maxima MD and E i may be ess reevant than the range of the criteria over the design space. Borth [8] treated the two objectives using the idea of the change in tota entropy. He showed that it decomposed into the mode discrimination term J and parameter estimation term S P. We denote Borth s criterion as B hereafter. The scae for entropy for continuous random variabes (parameters) may not be wecaibrated with entropy for a discrete random variabe (mode seection): their range may differ when evauated throughout the design space. Borth s method aso requires computationay expensive numerica integration. Here we aso use the idea of the expected gain in entropy of an experiment, but normaize J and S P over their range of vaues, and simpify the weight factor. Instead of numericay evauating the J criterion, we approximate it with the MD criterion. So an upper bound approximation of the joint criterion for mode discrimination and parameter estimation is where MD min, MD max, S Pmin, S Pmax S Q = w MD MD min + (1 w) S P S Pmin, (12) MD max MD min S Pmax S Pmin are the smaest and argest MD vaues and the smaest and argest S P vaues respectivey over a designs in D, and w [0, 1] is a weight factor. This is simiar in form to criterion C, but reduces to a cosed form if the prior setup in Eq. (2) is used, as a resut of Eq. (6) and Eq. (9). Eq. (12) does not require σ 2 to be known (see the propositions above for inear modes, and comments beow for noninear modes), and incorporates prior information and data from initia experiments. The weight w shoud be seected based on the resuts of the initia experiments and the focus of the foowup experiment. If the initia experiment was insufficient to identify the important parameters, then more weight shoud be paced on mode discrimination. If the mode is reasonaby determined, then more focus can be paced on parameter estimation. Hi et a. [22] suggested w = [ s s 1 (1 p(m max)) ] ξ where Mmax is the a priori most probabe mode. Another choice is w = [(1 (p(m max ) p(m max2 ))] ξ, where M max2 is the second most probabe mode. Sma vaues of ξ paces more weight on mode discrimination. To equay baance the two caibrated 8
9 entropy measures, w can aso be set at 1/2. The numerica exampes in Section 3 use both the weighting function of Hi et a. [22] and w = 1/2. Exampes 1 and 2 assess the dependence upon the optima design on the weight. The exampes show that rescaing the entropies can be important, but that the fina design may be somewhat insensitive to a misspecification in w. To achieve the joint objectives of mode discrimination and parameter estimation, we seek a design x D that maximizes S Q in Eq. (12). For norma inear modes, S Q simpifies to a cosed form through Eq. (6) and Eq. (9). The criterion is aso appicabe to noninear modes. When a noninear mode can be approximated by a inear mode in the neighborhood of θ 0, S Q can be appied by substituting the initia estimates of the parameters [12]. For nonnorma modes, S Q requires numericay integrating Eq. (5) and Eq. (7). For generaized inear and noninear modes, Bayesian methods [3, 17] can be used to approximate the terms in Eq. (5) and Eq. (7). Shannon information is not the ony possibe approach to deveop a joint mode seection and parameter estimation design criterion. Bingham and Chipman [7] propose a weighted average of Heinger distances between predictive densities of a possibe pairs of competing modes as a criterion for mode discrimination. A inear combination of Bingham and Chipman [7] s criterion and a weighted average of the Heinger distances between the prior and posterior distributions of each mode s parameters can aso be used as a joint criterion. For the prior setup in Eq. (2), this reduces to a cosed form. The weighting functions described above can be used to weight the importance of each objective. The upper bound on the Heinger distances for each individua term can be usefu for rescaing, but the maximum vaues of each term for a particuar finite n, can be quite far from the upper bound and rescaing each term by its upper bound may not be appropriate. We do not consider that combined criterion further here. 2.3 Some Computationa Issues Athough S Q simpifies to a cosed form for the norma inear mode, there are computationa chaenges. We consider three here. First, the number of modes grows exponentiay in the number of predictors. Second, the min and max vaues of the two entropy measures that comprise S Q are required. Third, the number of designs grows combinatoriay in the number of candidate runs. To address the first issue, the summands for MD and S P are computed by using ony the most ikey modes. There are typicay far fewer than s = 2 p different modes whose probabiity p(m ) ead it to be a competitor for the best after the initia stage of experimentation. By considering ony the most ikey modes, Eq. (12) becomes tractabe. There are severa ways one can chose a subset of probabe modes: (i) Pick a modes h so that p(m h ) E, (ii) Pick the h most ikey 9
10 modes, where h is the smaest integer so that p(m (1) ) + p(m (2) ) + + p(m (h) ) F. Raftery et a. [34] take a simiar approach to mode averaging. Exampes 2 and 3 of Section 3 use (i) with E = The top modes have higher posterior probabiities, so we set E = In Section 3.1, no mode ceary stands out after the initia runs, so we use E = When direct enumeration is not computationay feasibe, these more important modes can be identified heuristicay by using Markov Chain Monte Caro methods ike MC 3 (Markov Chain Monte Caro Mode Composition) [27] to estimate the p(m ). The state space for MC 3 is the set of s modes, and a sampe path visits a sequence of different modes, M. Candidate states for transitions are chosen from the set of modes with one more or one fewer active predictors. The reative probabiities for the current and candidate states, needed to impement the Metropois Hastings step of MC 3, can be computed from cosedform formuas in Raftery et a. [34]. The number of times a mode is visited during MC 3 divided by the number of iterations of MC 3 is a consistent estimate of the mode s posterior probabiity. Chipman et a. [15] and Ng [31] discuss some practicaities of Markov chain Monte Caro methods for mode seection. Second, we use an optimization heuristic to estimate MD min, MD max, S Pmin, S Pmax. We use the kexchange agorithm of Johnson and Nachtsheim [23] to search for the maximum and minimum vaues. The kexchange agorithm was first proposed to construct Doptima designs, but because it is a genera agorithm, it can be used to seect from a finite set of designs as ong as an optimaity criterion is given. Numerica resuts [23, 33] show that it is efficient and effective in constructing optima designs, and the agorithm has been widey used. In the numerica exampes we considered, the kexchange agorithm was very efficient in identifying the optima designs. In addition to increasing k as suggested in [23], we aso found that for the kexchange agorithm, increasing the number of starting designs from scattered points in the design space improves the search for the optima. Aternativey, the branch and bound agorithm in [39], or nested partitions [37] can be used to find the goba optima. Third, we generaize the kexchange agorithm (Appendix A.4) to identify a design with a high vaue of S Q to improve the scaing of the entropy measures. The agorithm is a greedy agorithm that swaps in and out design points one at a time. More work on computationa issues is an avenue for future research. 10
11 3. Numerica Resuts Three numerica experiments compare the new criterion, S Q, with the two other joint criteria in the iterature, as we as the MD and S P criteria. The optima S Q foowup design x (w, z 0 ) depends upon the weight w and previous observations z 0. Let x (z 0 ) = {x (w, z 0 ) : w [0, 1]} be the set of designs that the S Q criterion identifies, given z Chemica Reactor Experiment Box et a. [11, p. 377] gave data for a chemicareactor experiment that used a 2 5 fu factoria design. From this data, we extracted runs that correspond to five coumns of a PackettBurman 12 run (PB12) design. We treated those runs (see Tabe 1) as an initia experiment. The foowup design was simuated by extracting the remaining runs from the compete experiment. Tabe 1: PB12 design and data extracted from the fu 2 5 reactor experiment Run i A B C D E z i We considered fifteen predictors (five factors and their two factor interactions) to get 2 15 distinct inear modes in the mode space M, each differing by the absence or presence of each predictor. We used the equa probabiity prior for mode uncertainty, p(m ) = 2 15, and the prior for parameters suggested by Raftery et a. [34]. Tabe 2 shows the probabiities for the top 8 modes, given that prior distribution and the PB12 data. No mode ceary stands out, but the mode identified in the origina anaysis of a 32 runs [11], with factors (B, D, E, BD, DE), is ranked best. To distinguish between the top eight modes, n = 3 additiona runs were seected from the remaining 20 runs. The best designs for each joint criterion (S Q with w = 0.5; B; and C with ξ = 2 as in [22]) were computed by evauating the criteria over each possibe design. The joint 11
12 Tabe 2: Probabiity of the eight most probabe modes after 12 runs Mode Posterior Probabiity B, D, E, BD, DE B, C, D, E, BD, DE B, D, E, BC, BD, DE B, D, E, BD, BE, DE A, B, D, E, BD, DE B, D, E, AE, BD, DE B, D, E, AC, BD, DE B, D, E, BD, CD, DE Tabe 3: Posterior probabiity (Post.) of the three most probabe modes with PB12, + 3 runs determined by the best design obtained from fu enumeration of the S Q, B, and C criteria. New S Q Criterion Borth s B Criterion Hi s C Criterion Mode Post. Mode Post. Mode Post. B, D, E, BD, DE B, D, E, BD, DE B, D, E, BD, DE B, D, E, BC, BD, DE B, C, D, E, BD, DE B, D, E, BC, BD, DE A, B, D, E, BD, DE A, B, D, E, BD, DE B, D, E, BD, BE, DE 0.04 criterion S Q resuts in different designs than the B and C criteria. The posterior probabiities of a modes were then recomputed using a 15 runs, and the top 3 modes are shown in the eft portion of Tabe 3. A three designs identified the same top mode identified in the origina anaysis of a 32 runs. S Q discriminated in favor of the top mode more than criteria B and C. Tabe 4 indicates that S Q reduced the parameter generaized variance (the determinant of the posterior covariance matrix of the parameter estimates, V (β) ) of the top mode more than B and C. To compare the computationa burden, each criterion was evauated for a possibe designs for one, two, three and four additiona runs using Mape8 (sow, because it is interpreted, but reative CPU times are iustrative). Tabe 5 shows the computation times for the S Q and B criterion. The computation times for S Q and C were simiar. The curse of dimensionaity made quadrature an inefficient approach for the numerica integrations required by B. Tabe 6 shows the posterior probabiities of the top three modes with the mode discrimina Tabe 4: Parameter generaized variance V (β) for the a posteriori top mode (B, D, E, BD, DE), given PB runs, based on the S Q, B and C criteria. Criterion V (β) S Q B C
13 Tabe 5: CPU time for computing S Q, C and B (hours). Additiona runs S Q and C B Tabe 6: Posterior probabiity (Post.) of the three most probabe modes with PB runs determined by the best design obtained by fu enumeration for S Q, MD, and S P. S Q Criterion MD Criterion S P Criterion Mode Post. Mode Post. Mode Post. B, D, E, BD, DE B, D, E, BD, DE B, D, E, BD, DE B, D, E, BC, BD, DE B, D, E, BD, BE, DE B, C, D, E, BD, DE A, B, D, E, BD, DE B, C, D, E, BD, DE A, B, D, E, BD, DE tion MD and parameter estimation S P criteria. As expected, MD did a better job than S P at distinguishing the top mode from the others, and Tabe 7 indicates that S P outperformed MD at reducing the parameter generaized variance of the top mode. S Q on the other hand outperformed MD in favoring the top mode, and was ony sighty poorer than S P in parameter estimation. This exampe aso iustrates the importance of normaizing that we suggest, as one of the subcriteria woud be ignored without recaibration. With the equa probabiity prior for each mode, the range of uncaibrated MD scores over the design space ranged from to 0.062, whie the uncaibrated S P scores range from 0.88 to Without recaibration, the joint criterion woud have seected the best S P design, and ignored the mode discrimination objective. We tested the sensitivity to the prior distribution by rerunning the experiment with the independence prior p(m ) = ω t (1 ω) p t, where t is the number of predictors in mode, ( = 1,..., 2 15 ), with ω = The mode with factors (B, D, E, BD, DE) is ranked third when ony 12 runs are used, but the S Q criterion again identified (B, D, E, BD, DE) as the most probabe mode after the n = 3 run foow up was competed. To test the sensitivity of the designs to the weights, w was varied from 0 to 1. In this exampe, Tabe 7: Parameter generaized variance V (β) for the a posteriori top mode (B, D, E, BD, DE), given PB runs, based on the S Q, MD and S P criteria. Criterion V (β) S Q MD S P 13
14 the top S Q design is robust over a range of weights. There were ony three different top designs obtained as w was varied from 0 to 1. When 0 w <.27, the top S P design is obtained. The same design obtained for S Q when w = 0.5 is obtained when 0.27 w < The top MD design is obtained when w.78. For the weighting function suggested in [22], the best MD design is seected when ξ gets sma, ξ 4, and the best S P design is seected when ξ gets arge, ξ > 27, and the same design seected for any ξ in between. In this exampe, sma changes in w or ξ do not significanty change the optima design. The Entropy Baancing kexchange Agorithm in Appendix A.4 was impemented to evauate its effectiveness to find good designs in this probem. The best design is known for this probem because an exhaustive evauation of a 1140 designs is possibe. The Entropy Baancing agorithm consists of two steps. In the initia step of the agorithm, the maximum and minimum vaues of MD and S p are estimated using the kexchange max and kexchange min agorithms. In the second step, the kexchange max agorithm is used to search for a good S Q design. We first evauated how we the initia step performs in determining the maximum and minimum vaues. We varied the number r of initia random designs to start the agorithm, r = 100 and r = 200, and conducted 10 independent repications of the agorithm for each r (each repication sampes an independent set of r design points). When r = 100, the estimates of (MD max, MD min, S Pmin, S Pmax ) were a equa to the actua vaue 70% of the time. When r = 200, a four estimates were correct 80% of the time. In the remaining cases, ony one of the four actua vaues was not obtained, but the estimate was cose to the actua vaue. We next tested the how we the second step of the agorithm identifies the known best optima S Q design. The known best optima S Q design was identified 70% of the time with r = 100, and 90% of the time with r = 200. The remaining nonoptima designs seected were among the top 8 designs. 3.2 Finding a Known Mode To determine how we the criteria performs in detecting a known mode, and how the best S Q mode depends upon w in repeated sampes, we ran 50 repications of an experiment using the S Q criterion on a known mode with 4 potentia factors (A, B, C, D), namey Z = 10A + 15B + 6AB +7AC +ζ i, where ζ i Norma(0, 5). For each repication i, we ran an initia fractiona factoria design to generate preiminary data z 0,i, which was then used to create a prior distribution for a foowup design with n = 3 runs as described in Fig. 1. In each repication, the true (known) mode was among the top eight candidate modes after the initia = 8 runs but was competey confounded with three other modes. The three additiona runs seected by S Q deaiased 14
15 Figure 1: Agorithm to assess the identification of a known mode in Section 3.2 For i = 1, 2,..., 50: 1. Generate independent preiminary data z 0,i with a factoria design. 2. Update the distributions for the unknown modes and parameters as in Section Determine the best S Q design x (w, z 0,i ) for n = 3 additiona runs, as a function of w. 4. Run the best foowup design, x (w, z 0,i ), for each w. 5. Compute the posterior probabiity and ordina rank of each mode, as a function of w. End for oop the confounded effects and distinguished between the top competing modes. In each repication, three different designs were obtained ( x (z 0,i ) = 3) as w varied between 0 and 1. The best S Q design x (w, z 0,i ) was the top S P design (x (0, z 0,i )) for sma w. For a certain range between 0 and 1, a unique top S Q design was obtained that baances mode discrimination and parameter estimation. For arger w, the best S Q design was the best MD design (x (1, z 0,i )). In 70% of the repications, the same three designs x (z 0,i ) were seected and the same S Q was seected for w in approximatey the range (0.1, 0.55). The other 30% of the repications resuted in 3 other sets of S Q designs with the same quaitative features: the designs with w in the range of about 0.1 up to baanced mode discrimination and parameter estimation. In 49 out of 50 repications, the true mode was identified as the best mode when the S Q design with intermediate vaues of w was used to baance discrimination and estimation. In the remaining repication, the true mode had the third highest posterior probabiity. Averaging over 50 repications, the probabiity that the true mode was best improved from 0.04 after the initia stage (8 runs), to 0.21 (after the 3 foowup runs). The average range of MD was 7.42, the average range of S P was 0.59, and MD max > S Pmax for a repications. If the individua measures were not recaibrated by their ranges, the S Q criterion woud have seected the best MD design and ignored parameter estimation uness a very sma weight were paced on mode discrimination. This experiment gave the same quaitative concusions as Section 3.1. Borth s criterion took orders of magnitude more time to compute due to numerica integration issues (curse of dimensionaity). Rebaancing the entropy measures was important for assuring a baance between discrimination and estimation. The optima design was not highy sensitive to the choice of w. 3.3 Critica Care Faciity The critica care faciity iustrated in Fig. 2 was originay studied by Schruben and Margoin [36]. Patients arrive according to a Poisson process and are routed through the system depending 15
16 Figure 2: Estimated fraction of patients routed through the units of a critica care faciity. upon their specific heath condition. Stays in the intensive care (ICU), coronary care (CCU), and intermediate care faciities are presumed to be ognormay distributed. This section compares S Q with C and the individua criteria, MD and S P. Borth s criterion B took too much time to evauate and was therefore not compared. We initiay ran a 64 run design using the Bayesian mode average to sampe uncertain input parameters [32]. We considered tweve input parameters, resuting in 2 12 distinct inear modes in the mode space M, each differing by the absence and presence of each predictor. Tabe 8 shows the posterior probabiities for the top 5 modes. The mode identified in a 128 run study in [32] is ranked fifth here. Schruben and Margoin [36] studied how to aocate random number streams to reduce variabiity in response surface parameter estimates. Their response mode predicts the expected number of patients per month E[Z] that are denied entry to the faciity as a function of the number of beds in the ICU, CCU, and intermediate care faciities. They presume fixed point estimates for k = 6 input parameters, one per source of randomness, to describe the patient arriva process (Poisson arrivas, mean ˆλ = 3.3/day), ICU stay duration (ognorma, mean 3.4 and standard deviation 3.5 days), intermediate ICU stay duration (ognorma, mean 15.0, standard deviation 7.0), intermediate CCU stay duration (ognorma, mean 17.0, standard deviation 3.0), CCU stay duration (ognorma, mean 3.8, standard deviation 1.6), and routing probabiities (mutinomia, ˆp 1 = 0.2, ˆp 3 = 0.2, ˆp 4 = 0.05). Some parameters are mutivariate, and there are a tota of = 12 dimensions of parameters. For the ognorma service times, the og of the service times has mean µ and precision λ = 1/σ 2. Subscripts distinguish the parameters of each service type (e.g., µ icu, µ iicu, µ iccu, µccu, λ icu ). The anaysis here presumes a inear response mode in these 12 parameters. The actua system parameters are not known with certainty, and the estimated system performance wi be in error if the actua parameter vaues differ from their point estimates. As in Ng and 16
17 Tabe 8: The five most probabe modes after 64 runs. Mode Post. Prob. λsys, µ iicu, λ iicu, p 1, p λsys, µ iicu, λ iicu, p 1, p 3, p λsys, µ iicu, λ iicu, λ iccu, p 1, p λsys, µ iicu, λ iicu, µ iccu, p 1, p λsys, µ iicu, λ iicu, µ iccu, p 1, p 3, p Tabe 9: The most probabe modes with runs determined by S Q with w = Mode Post. Prob. λsys, µ iicu, λ iicu, µ iccu, p 1, p 3, p λsys, µ iicu, λ iicu, p 1, p 3, p λsys, µ iicu, λ iicu, p 1, p Chick [32], who used naive Monte Caro samping for unknown inputs to do an uncertainty anaysis, we fix the number of beds in each of the three units (14 in ICU, 5 in CCU, 16 in intermediate care), and study how the expected number of patients per month that are denied entry depends on the unknown parameters. Design points for the unknown parameter vaues coud take on vaues of the MLE ± one standard error. The approach of Raftery et a. [34] was used to obtain prior distributions for the unknown response parameters. We used the S Q criterion with w = 0.55 (or ξ = 1) to avoid focusing on parameter estimation too eary. The design points of a fu factoria for the 12 parameters were candidates for the 32 run foowup design. The number of possibe 32 run designs from the 2 12 candidate runs is arge, we used the kexchange agorithm to search for the best S Q design (r = 50, k = 5), then ran the critica care simuations again with that design. The posterior probabiities for the top three modes, given the data from the combined design (64+32), are shown in Tabe 9. The top mode is the same mode identified in the 128 runs anaysis in [32], but the S Q criterion identified this mode with fewer runs. We aso used the kexchange agorithm (r = 50 and k = 5) to determine a good C design. Tabe 10 shows the posterior probabiities after running the simuations with the C design. The C design identified the same mode as S Q, but S Q did sighty better in discriminating the top two modes. The best designs for MD and S P are different than the best S Q design with w = The MD design identified the same top mode as the S Q design, and discriminated between the top two modes sighty better than the S Q design (Tabe 9 and Tabe 11). Tabe 12 indicates that design S Q did a better job than C and MD at reducing the parameter generaized variance of the top mode. The S Q criterion with w = 0.75 (or ξ = 0.5) resuted in better mode discrimination than with 17
18 Tabe 10: Most probabe modes with runs with the C criterion. Mode Post. Prob. λsys, µ iicu, λ iicu, µ iccu, p 1, p 3, p λsys, µ iicu, λ iicu, p 1, p 3, p λsys, µ iicu, λ iicu, λ iccu, p 1, p 3, p Tabe 11: Most probabe modes with runs with the MD criterion. Mode Post. Prob. λsys, µ iicu, λ iicu, µ iccu, p 1, p 3, p λsys, µ iicu, λ iicu, µ iccu, λ iccu, p 1, p 3, p λsys, µ iicu, λ iicu, µ iccu, p 1, p 3, p w = 0.55 at the cost of sighty ess effective parameter estimation (in this case, x (z 0,i ) > 3). The top two modes identified in the S P design were the same modes identified in the origina 64 run anaysis, and the top mode identified by the S Q and MD design is ony ranked fourth when the S P design is used. The S P criterion focused on designs that had good parameter estimation primariy for modes with higher posterior probabiity. Using the S P criterion too eary in the experimentation process can prematurey focus the design and experimentation on a few modes that may or may not be good approximations to the system (because of the sma number of runs), an issue raised by Atkinson [1]. An eary focus on M D can better distinguish competitors for the best mode, but at the expense of poorer parameter estimates. S Q baanced both of those needs. 4. Discussion and Concusions The purpose of many experiments is to distinguish between ikey mathematica modes and obtain precise estimates for the mode parameters. The three joint design criteria examined here each use an additive measure for entropy measures or bounds for mode and parameter uncertainty. Our Tabe 12: Parameter generaized variance V (β) after runs for the mode with λsys, µ iicu, λ iicu, µ iccu, p 1, p 3, p 4. Runs Criterion V (β) 96 S P S Qw= S Qw= C MD
19 proposa for the new S Q criterion to normaize each entropy measure by the amount each varies over the design space provides an insight that the other joint criteria do not: It indicates how rich the design space is for improving each entropy measure. If the range for one of the component criteria is much smaer than for the other (e.g. MD max MD min >> S Pmin S Pmax ), or if the number of potentiay optima designs, x (z 0 ), is sma, then a richer design space might be considered. S Q is computationay more efficient than Borth s joint criterion especiay when the panned foowup designs get arger. In the first two exampes, the S Q design performs as we as Borth s criterion, but it is computationay more efficient and practica. S Q extends the C criterion as it considers the reevant range of the individua criteria, does not require initia estimates of the variance, and accounts for avaiabe prior information. These two exampes aso show that there are 3 different designs for S Q as w varies from 0 to 1. The optimum MD design is seected when the mode discrimination term is heaviy weighted and the optimum S p design is seected when the parameter estimation term is heaviy weighted. For each experiment, the S Q design that baances both objectives is shown to be insensitive to a range of w, and this best design seected performs more efficienty than the other criteria. Three numerica experiments iustrated the compromise between mode discrimination and parameter estimation obtained when using the joint criterion S Q. Compared with the individua criteria, the baanced S Q design was about as good as the MD design for mode discrimination, and was amost as good as the S P design for parameter estimation. The MD design fared ess we for parameter estimation, and the S P design was east effective for mode discrimination. Athough S Q is easier to compute for the inear mode than Borth s criterion, the arge number of matrix cacuations required to compute the S Q criterion may need to be baanced against the cost of actuay running the experiments. In a simuation context, CPU cyces might be better spent running repications rather than computing S Q if the simuations run quicky. For expensive industria experiments or compex simuations with ong run times, the S Q criterion may be an effective mechanism to baance the needs of factor identification and parameter estimation. Sequentia designs and criteria based on the Heinger distance are avenues for further research. 19
20 A. Mathematica Detais A.1 Proof of Prop. 1 Conditioning on σ 2, the MD criterion can be rewritten MD = p(m i )p(m ) p(σ 2 ) p(z M i, y i, σ 2 ) og p(z M i, y i, σ 2 ) 0 p(z M, y, σ 2 ) dzdσ2 (13) 0 i s Meyer et a. [28] substituted the predictive distribution of the norma form in Eq. (3) into Eq. (13) and integrated with respect to p(z M i, σ 2 ) to obtain: MD = [ [ 1 p(m i )p(m ) π(σ 2 ) 2 og 0 i s ( 0 i s nσ 2 σ 2 tr(v 1 0 ( V ) V 1 i 2σ 2 V i ) (ẑ i ẑ ) T V 1 (ẑ i ẑ ) We now isoate the dependence on the noninformative prior. MD = [ ( 1 V ) p(m i )p(m ) 2 og V i ( nσ 2 σ 2 tr ( V 1 = 0 i s V i 1 2 p(m i)p(m ) + (ẑ i ẑ ) T V 1 (ẑ i ẑ ) 0 1 2σ 2 ) ] dσ 2 ] ) ) ] (ẑi ẑ ) T V 1 (ẑ i ẑ ) π(σ 2 )dσ 2 [ ( V ) og n + tr(v 1 V i ) 0 V i ] 1 σ 2 π(σ2 )dσ 2 The doube sum means that pairs i, can be matched to make the og terms cance out. And 1 π(σ 2 )dσ 2 = [(ν/2)/(νλ/2)] InvertedGamma ( σ 2 ( ν + 1) ), νλ 0 σ dσ 2 = 1/λ. Substitute this into Eq. (14) to justify the caim in Eq. (6). A.2 Proof of Prop. 2 (14) Condition on mode M and et θ = (β, σ 2 ). [ ] p(θ Z) BD = p(z)p(θ Z) og dθdz p(θ) [ ] = p(z)p(θ Z) og p(θ Z) dθdz [ ] p(z)p(θ Z) og p(θ) dθdz 20
Face Hallucination and Recognition
Face Haucination and Recognition Xiaogang Wang and Xiaoou Tang Department of Information Engineering, The Chinese University of Hong Kong {xgwang1, xtang}@ie.cuhk.edu.hk http://mmab.ie.cuhk.edu.hk Abstract.
More informationMath: Fundamentals 100
Math: Fundamentas 100 Wecome to the Tooing University. This course is designed to be used in conjunction with the onine version of this cass. The onine version can be found at http://www.tooingu.com. We
More informationAustralian Bureau of Statistics Management of Business Providers
Purpose Austraian Bureau of Statistics Management of Business Providers 1 The principa objective of the Austraian Bureau of Statistics (ABS) in respect of business providers is to impose the owest oad
More informationNormalization of Database Tables. Functional Dependency. Examples of Functional Dependencies: So Now what is Normalization? Transitive Dependencies
ISM 602 Dr. Hamid Nemati Objectives The idea Dependencies Attributes and Design Understand concepts normaization (HigherLeve Norma Forms) Learn how to normaize tabes Understand normaization and database
More informationArtificial neural networks and deep learning
February 20, 2015 1 Introduction Artificia Neura Networks (ANNs) are a set of statistica modeing toos originay inspired by studies of bioogica neura networks in animas, for exampe the brain and the centra
More informationSecure Network Coding with a Cost Criterion
Secure Network Coding with a Cost Criterion Jianong Tan, Murie Médard Laboratory for Information and Decision Systems Massachusetts Institute of Technoogy Cambridge, MA 0239, USA Emai: {jianong, medard}@mit.edu
More informationHYBRID FUZZY LOGIC PID CONTROLLER. Abstract
HYBRID FUZZY LOGIC PID CONTROLLER Thomas Brehm and Kudip S. Rattan Department of Eectrica Engineering Wright State University Dayton, OH 45435 Abstract This paper investigates two fuzzy ogic PID controers
More informationTERM INSURANCE CALCULATION ILLUSTRATED. This is the U.S. Social Security Life Table, based on year 2007.
This is the U.S. Socia Security Life Tabe, based on year 2007. This is avaiabe at http://www.ssa.gov/oact/stats/tabe4c6.htm. The ife eperiences of maes and femaes are different, and we usuay do separate
More informationTeamwork. Abstract. 2.1 Overview
2 Teamwork Abstract This chapter presents one of the basic eements of software projects teamwork. It addresses how to buid teams in a way that promotes team members accountabiity and responsibiity, and
More informationBetting Strategies, Market Selection, and the Wisdom of Crowds
Betting Strategies, Market Seection, and the Wisdom of Crowds Wiemien Kets Northwestern University wkets@keogg.northwestern.edu David M. Pennock Microsoft Research New York City dpennock@microsoft.com
More informationCOMPARISON OF DIFFUSION MODELS IN ASTRONOMICAL OBJECT LOCALIZATION
COMPARISON OF DIFFUSION MODELS IN ASTRONOMICAL OBJECT LOCALIZATION Františe Mojžíš Department of Computing and Contro Engineering, ICT Prague, Technicá, 8 Prague frantise.mojzis@vscht.cz Abstract This
More information3.3 SOFTWARE RISK MANAGEMENT (SRM)
93 3.3 SOFTWARE RISK MANAGEMENT (SRM) Fig. 3.2 SRM is a process buit in five steps. The steps are: Identify Anayse Pan Track Resove The process is continuous in nature and handed dynamicay throughout ifecyce
More informationOligopoly in Insurance Markets
Oigopoy in Insurance Markets June 3, 2008 Abstract We consider an oigopoistic insurance market with individuas who differ in their degrees of accident probabiities. Insurers compete in coverage and premium.
More informationPayondelivery investing
Payondeivery investing EVOLVE INVESTment range 1 EVOLVE INVESTMENT RANGE EVOLVE INVESTMENT RANGE 2 Picture a word where you ony pay a company once they have deivered Imagine striking oi first, before
More informationELEVATING YOUR GAME FROM TRADE SPEND TO TRADE INVESTMENT
Initiatives Strategic Mapping Success in The Food System: Discover. Anayze. Strategize. Impement. Measure. ELEVATING YOUR GAME FROM TRADE SPEND TO TRADE INVESTMENT Foodservice manufacturers aocate, in
More informationSimultaneous Routing and Power Allocation in CDMA Wireless Data Networks
Simutaneous Routing and Power Aocation in CDMA Wireess Data Networks Mikae Johansson *,LinXiao and Stephen Boyd * Department of Signas, Sensors and Systems Roya Institute of Technoogy, SE 00 Stockhom,
More informationA Supplier Evaluation System for Automotive Industry According To Iso/Ts 16949 Requirements
A Suppier Evauation System for Automotive Industry According To Iso/Ts 16949 Requirements DILEK PINAR ÖZTOP 1, ASLI AKSOY 2,*, NURSEL ÖZTÜRK 2 1 HONDA TR Purchasing Department, 41480, Çayırova  Gebze,
More informationFast Robust Hashing. ) [7] will be remapped (and therefore discarded), due to the loadbalancing property of hashing.
Fast Robust Hashing Manue Urueña, David Larrabeiti and Pabo Serrano Universidad Caros III de Madrid E89 Leganés (Madrid), Spain Emai: {muruenya,darra,pabo}@it.uc3m.es Abstract As statefu fowaware services
More informationRisk Margin for a NonLife Insurance RunOff
Risk Margin for a NonLife Insurance RunOff Mario V. Wüthrich, Pau Embrechts, Andreas Tsanakas August 15, 2011 Abstract For sovency purposes insurance companies need to cacuate socaed bestestimate reserves
More informationRisk Margin for a NonLife Insurance RunOff
Risk Margin for a NonLife Insurance RunOff Mario V. Wüthrich, Pau Embrechts, Andreas Tsanakas February 2, 2011 Abstract For sovency purposes insurance companies need to cacuate socaed bestestimate
More informationFinance 360 Problem Set #6 Solutions
Finance 360 Probem Set #6 Soutions 1) Suppose that you are the manager of an opera house. You have a constant margina cost of production equa to $50 (i.e. each additiona person in the theatre raises your
More informationThe guaranteed selection. For certainty in uncertain times
The guaranteed seection For certainty in uncertain times Making the right investment choice If you can t afford to take a ot of risk with your money it can be hard to find the right investment, especiay
More informationDynamic Pricing Trade Market for Shared Resources in IIU Federated Cloud
Dynamic Pricing Trade Market or Shared Resources in IIU Federated Coud Tongrang Fan 1, Jian Liu 1, Feng Gao 1 1Schoo o Inormation Science and Technoogy, Shiiazhuang Tiedao University, Shiiazhuang, 543,
More informationFixed income managers: evolution or revolution
Fixed income managers: evoution or revoution Traditiona approaches to managing fixed interest funds rey on benchmarks that may not represent optima risk and return outcomes. New techniques based on separate
More informationMaintenance activities planning and grouping for complex structure systems
Maintenance activities panning and grouping for compex structure systems Hai Canh u, Phuc Do an, Anne Barros, Christophe Berenguer To cite this version: Hai Canh u, Phuc Do an, Anne Barros, Christophe
More informationLife Contingencies Study Note for CAS Exam S. Tom Struppeck
Life Contingencies Study Note for CAS Eam S Tom Struppeck (Revised 9/19/2015) Introduction Life contingencies is a term used to describe surviva modes for human ives and resuting cash fows that start or
More informationMultiRobot Task Scheduling
Proc of IEEE Internationa Conference on Robotics and Automation, Karsruhe, Germany, 013 MutiRobot Tas Scheduing Yu Zhang and Lynne E Parer Abstract The scheduing probem has been studied extensivey in
More informationBetting on the Real Line
Betting on the Rea Line Xi Gao 1, Yiing Chen 1,, and David M. Pennock 2 1 Harvard University, {xagao,yiing}@eecs.harvard.edu 2 Yahoo! Research, pennockd@yahooinc.com Abstract. We study the probem of designing
More informationPricing and hedging of variable annuities
Cutting Edge Pricing and hedging of variabe annuities Variabe annuity products are unitinked investments with some form of guarantee, traditionay sod by insurers or banks into the retirement and investment
More informationVendor Performance Measurement Using Fuzzy Logic Controller
The Journa of Mathematics and Computer Science Avaiabe onine at http://www.tjmcs.com The Journa of Mathematics and Computer Science Vo.2 No.2 (2011) 311318 Performance Measurement Using Fuzzy Logic Controer
More informationLeakage detection in water pipe networks using a Bayesian probabilistic framework
Probabiistic Engineering Mechanics 18 (2003) 315 327 www.esevier.com/ocate/probengmech Leakage detection in water pipe networks using a Bayesian probabiistic framework Z. Pouakis, D. Vaougeorgis, C. Papadimitriou*
More informationA Latent Variable Pairwise Classification Model of a Clustering Ensemble
A atent Variabe Pairwise Cassification Mode of a Custering Ensembe Vadimir Berikov Soboev Institute of mathematics, Novosibirsk State University, Russia berikov@math.nsc.ru http://www.math.nsc.ru Abstract.
More informationSubject: Corns of En gineers and Bureau of Reclamation: Information on Potential Budgetarv Reductions for Fiscal Year 1998
GAO United States Genera Accounting Office Washington, D.C. 20548 Resources, Community, and Economic Deveopment Division B276660 Apri 25, 1997 The Honorabe Pete V. Domenici Chairman The Honorabe Harry
More informationWith the arrival of Java 2 Micro Edition (J2ME) and its industry
Knowedgebased Autonomous Agents for Pervasive Computing Using AgentLight Fernando L. Koch and JohnJues C. Meyer Utrecht University Project AgentLight is a mutiagent systembuiding framework targeting
More informationAdvanced ColdFusion 4.0 Application Development  3  Server Clustering Using Bright Tiger
Advanced CodFusion 4.0 Appication Deveopment  CH 3  Server Custering Using Bri.. Page 1 of 7 [Figures are not incuded in this sampe chapter] Advanced CodFusion 4.0 Appication Deveopment  3  Server
More informationArt of Java Web Development By Neal Ford 624 pages US$44.95 Manning Publications, 2004 ISBN: 1932394060
IEEE DISTRIBUTED SYSTEMS ONLINE 15414922 2005 Pubished by the IEEE Computer Society Vo. 6, No. 5; May 2005 Editor: Marcin Paprzycki, http://www.cs.okstate.edu/%7emarcin/ Book Reviews: Java Toos and Frameworks
More informationOrdertoCash Processes
TMI170 ING info pat 2:Info pat.qxt 01/12/2008 09:25 Page 1 Section Two: OrdertoCash Processes Gregory Cronie, Head Saes, Payments and Cash Management, ING O rdertocash and purchasetopay processes
More informationLADDER SAFETY Table of Contents
Tabe of Contents SECTION 1. TRAINING PROGRAM INTRODUCTION..................3 Training Objectives...........................................3 Rationae for Training.........................................3
More informationBusiness schools are the academic setting where. The current crisis has highlighted the need to redefine the role of senior managers in organizations.
c r o s os r oi a d s REDISCOVERING THE ROLE OF BUSINESS SCHOOLS The current crisis has highighted the need to redefine the roe of senior managers in organizations. JORDI CANALS Professor and Dean, IESE
More informationPricing Internet Services With Multiple Providers
Pricing Internet Services With Mutipe Providers Linhai He and Jean Warand Dept. of Eectrica Engineering and Computer Science University of Caifornia at Berkeey Berkeey, CA 94709 inhai, wr@eecs.berkeey.edu
More informationEstimation of Tail Development Factors in the PaidIncurred Chain Reserving Method
Estimation of Tai Deveopment Factors in the aidincurred Chain Reserving Method by Michae Merz and Mario V Wüthrich AbSTRACT In many appied caims reserving probems in &C insurance the caims settement process
More informationLearning framework for NNs. Introduction to Neural Networks. Learning goal: Inputs/outputs. x 1 x 2. y 1 y 2
Introduction to Neura Networks Learning framework for NNs What are neura networks? Noninear function approimators How do they reate to pattern recognition/cassification? Noninear discriminant functions
More informationGREEN: An Active Queue Management Algorithm for a Self Managed Internet
: An Active Queue Management Agorithm for a Sef Managed Internet Bartek Wydrowski and Moshe Zukerman ARC Specia Research Centre for UtraBroadband Information Networks, EEE Department, The University of
More informationThe Computation of the Inverse of a Square Polynomial Matrix
The Computation of the Inverse of a Square Poynomia Matrix Ky M. Vu, PhD. AuLac Technoogies Inc. c 2008 Emai: kymvu@auactechnoogies.com Abstract An approach to cacuate the inverse of a square poynomia
More informationSELECTING THE SUITABLE ERP SYSTEM: A FUZZY AHP APPROACH. Ufuk Cebeci
SELECTING THE SUITABLE ERP SYSTEM: A FUZZY AHP APPROACH Ufuk Cebeci Department of Industria Engineering, Istanbu Technica University, Macka, Istanbu, Turkey  ufuk_cebeci@yahoo.com Abstract An Enterprise
More informationCERTIFICATE COURSE ON CLIMATE CHANGE AND SUSTAINABILITY. Course Offered By: Indian Environmental Society
CERTIFICATE COURSE ON CLIMATE CHANGE AND SUSTAINABILITY Course Offered By: Indian Environmenta Society INTRODUCTION The Indian Environmenta Society (IES) a dynamic and fexibe organization with a goba vision
More informationA quantum model for the stock market
A quantum mode for the stock market Authors: Chao Zhang a,, Lu Huang b Affiiations: a Schoo of Physics and Engineering, Sun Yatsen University, Guangzhou 5175, China b Schoo of Economics and Business Administration,
More informationEarly access to FAS payments for members in poor health
Financia Assistance Scheme Eary access to FAS payments for members in poor heath Pension Protection Fund Protecting Peope s Futures The Financia Assistance Scheme is administered by the Pension Protection
More informationAn Interactive Fuzzy Satisficing Method for Multiobjective Stochastic Integer Programming Problems through a Probability Maximization Model
Cahit Perkgoz et a./asia Pacific Management Review (005) 10(1), 935 An Interactive Fuzzy Satisficing Method for Mutiobective Stochastic Integer Programming Probems through a Probabiity Maximization Mode
More informationRecognition of Prior Learning
Recognition of Prior Learning Information Guideines for Students EXTENDED CAMPUS This subject materia is issued by Cork Institute of Technoogy on the understanding that: Cork Institute of Technoogy expressy
More informationSorting, Merge Sort and the DivideandConquer Technique
Inf2B gorithms and Data Structures Note 7 Sorting, Merge Sort and the DivideandConquer Technique This and a subsequent next ecture wi mainy be concerned with sorting agorithms. Sorting is an extremey
More informationPerfect competition. By the end of this chapter, you should be able to: 7 Perfect competition. 1 Microeconomics. 1 Microeconomics
erfect 7 12 By the end of this chapter, you shoud be abe to: HL expain the assumptions of perfect HL distinguish between the demand curve for the industry and for the firm in perfect HL expain how the
More informationThe Radix4 and the Class of Radix2 s FFTs
Chapter 11 The Radix and the Cass of Radix s FFTs The divideandconuer paradigm introduced in Chapter 3 is not restricted to dividing a probem into two subprobems. In fact, as expained in Section. and
More informationComparison of Traditional and OpenAccess Appointment Scheduling for Exponentially Distributed Service Time
Journa of Heathcare Engineering Vo. 6 No. 3 Page 34 376 34 Comparison of Traditiona and OpenAccess Appointment Scheduing for Exponentiay Distributed Service Chongjun Yan, PhD; Jiafu Tang *, PhD; Bowen
More informationSimulationBased Booking Limits for Airline Revenue Management
OPERATIONS RESEARCH Vo. 53, No. 1, January February 2005, pp. 90 106 issn 0030364X eissn 15265463 05 5301 0090 informs doi 10.1287/opre.1040.0164 2005 INFORMS SimuationBased Booking Limits for Airine
More informationFRAME BASED TEXTURE CLASSIFICATION BY CONSIDERING VARIOUS SPATIAL NEIGHBORHOODS. Karl Skretting and John Håkon Husøy
FRAME BASED TEXTURE CLASSIFICATION BY CONSIDERING VARIOUS SPATIAL NEIGHBORHOODS Kar Skretting and John Håkon Husøy University of Stavanger, Department of Eectrica and Computer Engineering N4036 Stavanger,
More informationAN APPROACH TO THE STANDARDISATION OF ACCIDENT AND INJURY REGISTRATION SYSTEMS (STAIRS) IN EUROPE
AN APPROACH TO THE STANDARDSATON OF ACCDENT AND NJURY REGSTRATON SYSTEMS (STARS) N EUROPE R. Ross P. Thomas Vehice Safety Research Centre Loughborough University B. Sexton Transport Research Laboratory
More informationChapter 3: ebusiness Integration Patterns
Chapter 3: ebusiness Integration Patterns Page 1 of 9 Chapter 3: ebusiness Integration Patterns "Consistency is the ast refuge of the unimaginative." Oscar Wide In This Chapter What Are Integration Patterns?
More informationUniversity of Southern California
Master of Science in Financia Engineering Viterbi Schoo of Engineering University of Southern Caifornia Dia 18664693239 (Meeting number 924 898 113) to hear the audio portion, or isten through your
More informationA New Statistical Approach to Network Anomaly Detection
A New Statistica Approach to Network Anomay Detection Christian Caegari, Sandrine Vaton 2, and Michee Pagano Dept of Information Engineering, University of Pisa, ITALY Emai: {christiancaegari,mpagano}@ietunipiit
More informationDistribution of Income Sources of Recent Retirees: Findings From the New Beneficiary Survey
Distribution of Income Sources of Recent Retirees: Findings From the New Beneficiary Survey by Linda Drazga Maxfied and Virginia P. Rena* Using data from the New Beneficiary Survey, this artice examines
More informationNetwork/Communicational Vulnerability
Automated teer machines (ATMs) are a part of most of our ives. The major appea of these machines is convenience The ATM environment is changing and that change has serious ramifications for the security
More informationCONTRIBUTION OF INTERNAL AUDITING IN THE VALUE OF A NURSING UNIT WITHIN THREE YEARS
Dehi Business Review X Vo. 4, No. 2, Juy  December 2003 CONTRIBUTION OF INTERNAL AUDITING IN THE VALUE OF A NURSING UNIT WITHIN THREE YEARS John N.. Var arvatsouakis atsouakis DURING the present time,
More informationPricing and Revenue Sharing Strategies for Internet Service Providers
Pricing and Revenue Sharing Strategies for Internet Service Providers Linhai He and Jean Warand Department of Eectrica Engineering and Computer Sciences University of Caifornia at Berkeey {inhai,wr}@eecs.berkeey.edu
More informationMarket Design & Analysis for a P2P Backup System
Market Design & Anaysis for a P2P Backup System Sven Seuken Schoo of Engineering & Appied Sciences Harvard University, Cambridge, MA seuken@eecs.harvard.edu Denis Chares, Max Chickering, Sidd Puri Microsoft
More informationeffect on major accidents
An Investigation into a weekend (or bank hoiday) effect on major accidents Nicoa C. Heaey 1 and Andrew G. Rushton 2 1 Heath and Safety Laboratory, Harpur Hi, Buxton, Derbyshire, SK17 9JN 2 Hazardous Instaations
More informationAn Idiot s guide to Support vector machines (SVMs)
An Idiot s guide to Support vector machines (SVMs) R. Berwick, Viage Idiot SVMs: A New Generation of Learning Agorithms Pre 1980: Amost a earning methods earned inear decision surfaces. Linear earning
More informationAUSTRALIA S GAMBLING INDUSTRIES  INQUIRY
Mr Gary Banks Chairman Productivity Commission PO Box 80 BELCONNEN ACT 2616 Dear Mr Banks AUSTRALIA S GAMBLING INDUSTRIES  INQUIRY I refer to the Issues Paper issued September 1998 seeking submissions
More information500 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 31, NO. 3, MARCH 2013
500 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 3, NO. 3, MARCH 203 Cognitive Radio Network Duaity Agorithms for Utiity Maximization Liang Zheng Chee Wei Tan, Senior Member, IEEE Abstract We
More informationFigure 1. A Simple Centrifugal Speed Governor.
ENGINE SPEED CONTROL Peter Westead and Mark Readman, contro systems principes.co.uk ABSTRACT: This is one of a series of white papers on systems modeing, anaysis and contro, prepared by Contro Systems
More information3.5 Pendulum period. 20090210 19:40:05 UTC / rev 4d4a39156f1e. g = 4π2 l T 2. g = 4π2 x1 m 4 s 2 = π 2 m s 2. 3.5 Pendulum period 68
68 68 3.5 Penduum period 68 3.5 Penduum period Is it coincidence that g, in units of meters per second squared, is 9.8, very cose to 2 9.87? Their proximity suggests a connection. Indeed, they are connected
More informationConstrained Optimization: Step by Step. Maximizing Subject to a set of constraints:
Constrained Optimization: Step by Step Most (if not a) economic decisions are the resut of an optimization probem subject to one or a series of constraints: Consumers make decisions on what to buy constrained
More informationELECTRONIC FUND TRANSFERS YOUR RIGHTS AND RESPONSIBILITIES
About ELECTRONIC FUND TRANSFERS YOUR RIGHTS AND RESPONSIBILITIES The Eectronic Fund Transfers we are capabe of handing for consumers are indicated beow, some of which may not appy your account. Some of
More informationComputing the depth of an arrangement of axisaligned rectangles in parallel
Computing the depth of an arrangement of axisaigned rectanges in parae Hemut At Ludmia Scharf Abstract We consider the probem of computing the depth of the arrangement of n axisaigned rectanges in the
More informationONE of the most challenging problems addressed by the
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 44, NO. 9, SEPTEMBER 2006 2587 A Mutieve ContextBased System for Cassification of Very High Spatia Resoution Images Lorenzo Bruzzone, Senior Member,
More informationMinimum Support Size of the Defender s Strong Stackelberg Equilibrium Strategies in Security Games
Minimum Support Size o the Deender s Strong Stackeberg Equiibrium Strategies in Security Games Jiarui Gan University o Chinese Academy o Sciences The Key Lab o Inteigent Inormation Processing, ICT, CAS
More informationRicoh Healthcare. Process Optimized. Healthcare Simplified.
Ricoh Heathcare Process Optimized. Heathcare Simpified. Rather than a destination that concudes with the eimination of a paper, the Paperess Maturity Roadmap is a continuous journey to strategicay remove
More informationPracticing Reference... Learning from Library Science *
Practicing Reference... Learning from Library Science * Mary Whisner ** Ms. Whisner describes the method and some of the resuts reported in a recenty pubished book about the reference interview written
More informationModeling a Scenariobased Distribution Network Design Problem in a Physical Internetenabled open Logistics Web
4 th Internationa conference on Information Systems, Logistics and Suppy Chain Quebec City August 2629, 2012 Modeing a Scenariobased Distribution Network Design Probem in a Physica Internetenabed open
More informationJournal of Economic Behavior & Organization
Journa of Economic Behavior & Organization 85 (23 79 96 Contents ists avaiabe at SciVerse ScienceDirect Journa of Economic Behavior & Organization j ourna ho me pag e: www.esevier.com/ocate/j ebo Heath
More informationInfrastructure for Business
Infrastructure for Business The IoD Member Broadband Survey Infrastructure for Business 2013 #5 The IoD Member Broadband Survey The IoD Member Broadband Survey Written by: Corin Tayor, Senior Economic
More informationChapter 2 Traditional Software Development
Chapter 2 Traditiona Software Deveopment 2.1 History of Project Management Large projects from the past must aready have had some sort of project management, such the Pyramid of Giza or Pyramid of Cheops,
More informationELECTRONIC FUND TRANSFERS YOUR RIGHTS AND RESPONSIBILITIES. l l
ELECTRONIC FUND TRANSFERS YOUR RIGHTS AND RESPONSIBILITIES The Eectronic Fund Transfers we are capabe of handing for consumers are indicated beow some of which may not appy your account Some of these may
More informationInsertion and deletion correcting DNA barcodes based on watermarks
Kracht and Schober BMC Bioinformatics (2015) 16:50 DOI 10.1186/s1285901504827 METHODOLOGY ARTICLE Open Access Insertion and deetion correcting DNA barcodes based on watermarks David Kracht * and Steffen
More informationELECTRONIC FUND TRANSFERS YOUR RIGHTS AND RESPONSIBILITIES. l l
ELECTRONIC FUND TRANSFERS YOUR RIGHTS AND RESPONSIBILITIES The Eectronic Fund Transfers we are capabe of handing for consumers are indicated beow some of which may not appy your account Some of these may
More informationComparison of Misspecification Tests Designed for Nonlinear Time Series Models
ömmföäfsäafaäsfassfassfas fffffffffffffffffffffffffffffffffff Discussion Papers Comparison of Misspecification Tests Designed for Noninear Time Series Modes Leena Kaiovirta University of Hesinki and HECER
More informationPREFACE. Comptroller General of the United States. Page i
 I PREFACE T he (+nera Accounting Office (GAO) has ong beieved that the federa government urgenty needs to improve the financia information on which it bases many important decisions. To run our compex
More informationPERFORMANCE ANALYSIS OF GANG SCHEDULING IN A PARTITIONABLE PARALLEL SYSTEM
PERFORMANCE ANALYSIS OF GANG SCHEDULING IN A PARTITIONABLE PARALLEL SYSTEM Heen D. Karatza Department of Informatics Aristote University of Thessaoniki 54124 Thessaoniki, Greece Emai: karatza@csd.auth.gr
More informationIntegrating Risk into your Plant Lifecycle A next generation software architecture for risk based
Integrating Risk into your Pant Lifecyce A next generation software architecture for risk based operations Dr Nic Cavanagh 1, Dr Jeremy Linn 2 and Coin Hickey 3 1 Head of Safeti Product Management, DNV
More informationA BranchandPrice Algorithm for Parallel Machine Scheduling with Time Windows and Job Priorities
A BranchandPrice Agorithm for Parae Machine Scheduing with Time Windows and Job Priorities Jonathan F. Bard, 1 Siwate Rojanasoonthon 2 1 Graduate Program in Operations Research and Industria Engineering,
More informationMeasuring operational risk in financial institutions
Measuring operationa risk in financia institutions Operationa risk is now seen as a major risk for financia institutions. This paper considers the various methods avaiabe to measure operationa risk, and
More informationLoad Balancing in Distributed Web Server Systems with Partial Document Replication *
Load Baancing in Distributed Web Server Systems with Partia Document Repication * Ling Zhuo ChoLi Wang Francis C. M. Lau Department of Computer Science and Information Systems The University of Hong Kong
More informationDetermining the User Intent of ChineseEnglish Mixed Language Queries Based On Search Logs
Determining the User Intent of ChineseEngish Mixed Language Queries Based On Search Logs Hengyi Fu, Forida State University, City University of New York Abstract With the increasing number of mutiingua
More informationExperiment #11 BJT filtering
Jonathan Roderick Hakan Durmus Experiment #11 BJT fitering Introduction: Now that the BJT has bn expored in static and dynamic operation, the BJT, combined with what has bn aready presented, may be used
More informationENERGY AND BOLTZMANN DISTRIBUTIONS
MISN159 NRGY AND BOLTZMANN DISTRIBUTIONS NRGY AND BOLTZMANN DISTRIBUTIONS by J. S. Kovacs and O. McHarris Michigan State University 1. Introduction.............................................. 1 2.
More informationOracle Project Financial Planning. User's Guide Release 11.1.2.2
Orace Project Financia Panning User's Guide Reease 11.1.2.2 Project Financia Panning User's Guide, 11.1.2.2 Copyright 2012, Orace and/or its affiiates. A rights reserved. Authors: EPM Information Deveopment
More informationA Similarity Search Scheme over Encrypted Cloud Images based on Secure Transformation
A Simiarity Search Scheme over Encrypted Coud Images based on Secure Transormation Zhihua Xia, Yi Zhu, Xingming Sun, and Jin Wang Jiangsu Engineering Center o Network Monitoring, Nanjing University o Inormation
More informationREADING A CREDIT REPORT
Name Date CHAPTER 6 STUDENT ACTIVITY SHEET READING A CREDIT REPORT Review the sampe credit report. Then search for a sampe credit report onine, print it off, and answer the questions beow. This activity
More informationChapter 3: Investing: Your Options, Your Risks, Your Rewards
Chapter 3: Investing: Your Options, Your Risks, Your Rewards Page 1 of 10 Chapter 3: Investing: Your Options, Your Risks, Your Rewards In This Chapter What is inside a mutua fund? What is a stock? What
More informationEstimation of Liabilities Due to Inactive Hazardous Waste Sites. by Raja Bhagavatula, Brian Brown, and Kevin Murphy
Estimation of Liabiities Due to Inactive Hazardous Waste Sites by Raja Bhagavatua, Brian Brown, and Kevin Murphy ESTIMATION OF LIABILITIES DUE TO INACTIVE HAZARDOUS WASTE SITJZS Abstract: The potentia
More information