# What mathematical optimization can, and cannot, do for biologists. Steven Kelk Department of Knowledge Engineering (DKE) Maastricht University, NL

Save this PDF as:

Size: px
Start display at page:

Download "What mathematical optimization can, and cannot, do for biologists. Steven Kelk Department of Knowledge Engineering (DKE) Maastricht University, NL"

## Transcription

1 What mathematical optimization can, and cannot, do for biologists Steven Kelk Department of Knowledge Engineering (DKE) Maastricht University, NL

2 Introduction There is no shortage of literature about the role of mathematics and computer science in computational biology. The bottom line is well-known: computers, and the software and algorithms that run on them, have become a fundamental part of modern biology. However, for people not working in mathematics or computer science, it remains rather mystical how software and algorithms are developed, and what they can - and cannot - do.

3 Introduction In this talk I want to try and de-mystify these issues a little, because I think this is important for the effective use of computers in biology. I ll do this by drawing on my own experiences in computational biology. Particularly inspirational are those moments when you realise that, outside your own corner of expertise, very few concepts are obvious or universal

4 A disclaimer I m a discrete mathematician, mainly interested in combinatorial optimization, so there will inevitably be a bias in that direction during this talk. In particular, I feel a bit guilty that I have not said more about statistical methods. These methods are extremely important in computational biology, and phylogenetics is no exception. On the other hand, the principles of mathematical optimization often apply here too. For example, How do I compute the maximum likelihood tree? or` How long should I run my Monte Carlo Markov Chain to guarantee that it is close to its uniform distribution?

5 Two motivating examples

6 Sequence 1 T G C Sequence 2 T A C Sequence 3 A G G Sequence 4 A A G Input Maximum Parsimony (MP) Space of feasible solutions From:

7 Sequence 1 T G C Sequence 2 T A C Sequence 3 A G G Sequence 4 A A G Input Maximum Parsimony (MP) Space of feasible solutions What is the best solution? From:

8 Sequence 1 T G C Sequence 2 T A C Sequence 3 A G G Sequence 4 A A G Input Maximum Parsimony (MP) Space of feasible solutions Before we can determine the best tree, we need to formalize what best means From:

9 Sequence 1 T G C Sequence 2 T A C Sequence 3 A G G Sequence 4 A A G Input Maximum Parsimony (MP) Space of feasible solutions In MP, the quality of a tree is the number of mutations along its edges. The best tree is the one that minimizes this number. From:

10 Sequence 1 T G C Sequence 2 T A C Sequence 3 A G G Sequence 4 A A G Input Maximum Parsimony (MP) From:

11 Sequence 1 T G C Sequence 2 T A C Sequence 3 A G G Sequence 4 A A G Input Maximum Parsimony (MP) 4 So the optimal tree is the first one (i.e. fewest mutations), with a parsimony score of 4. Here the word optimal says nothing about the biological relevance of the tree. It simply means: the tree that minimizes the parsimony score. 5 6 From:

12 MP as an optimization problem Input: a set of n sequences Space of feasible solutions: all unrooted phylogenetic trees on n leaves, where the leaves are labelled by the input sequences Objective function: the quality of a tree is defined to be its parsimony score i.e. the number of mutations along its edges Goal: find a tree that minimizes the objective function. This is the optimal tree.

13 MP as an optimization problem Note that this does not say anything about how the optimal tree should be constructed, or even if it is computationally realistic. In an ideal world, there would be a perfect separation between the mathematical model (which is supposed to be an approximation of biological reality), and the question of how to efficiently find the optimal solution within the mathematical model. In practice mathematical models are heavily influenced by the limits of computation ( tractability ). More about this later.

14 Maximum Likelihood (ML) as an optimization problem Input: a set of n sequences and a probability distribution on nucleotide mutations Space of feasible solutions: all unrooted phylogenetic trees on n leaves, where the leaves are labelled by the input sequences Objective function: the quality of a tree is defined to be the likelihood of observing that tree given the input probability distribution Goal: find a tree that maximizes the objective function. This is the optimal tree.

15 Abstraction

16 Input (e.g.character data, distance data, trees, networks) Objective function (e.g. parsimony, likelihood, minimizing reticulation) Space of feasible solutions (e.g. space of trees, networks)

17 Input (e.g.character data, distance data, trees, networks) Objective function (e.g. parsimony, likelihood, minimizing reticulation) Space of feasible solutions (e.g. space of trees, networks)

18 Input (e.g.character data, distance data, trees, networks) Objective function (e.g. parsimony, likelihood, minimizing reticulation) The objective function imposes a hierarchy (i.e. an ordering) on the space of feasible solutions Space of feasible solutions (e.g. space of trees, networks)

19 Input (e.g.character data, distance data, trees, networks) Objective function (e.g. parsimony, likelihood, minimizing reticulation) The objective function imposes a hierarchy (i.e. an ordering) on the space of feasible solutions. Here: darker = better Space of feasible solutions (e.g. space of trees, networks)

20 Input (e.g.character data, distance data, trees, networks) Objective function (e.g. parsimony, likelihood, minimizing reticulation) Solutions grouped according to increasing quality Space of feasible solutions (e.g. space of trees, networks)

21 Input (e.g.character data, distance data, trees, networks) Objective function (e.g. parsimony, likelihood, minimizing reticulation) There are two optimal solutions. So in this case there is no unique optimal solution. It can be very dangerous to make biological inferences based on seeing only a single optimal solution! Space of feasible solutions (e.g. space of trees, networks)

22 Input (e.g.character data, distance data, trees, networks) Objective function (e.g. parsimony, likelihood, minimizing reticulation) There are two optimal solutions. So in this case there is no unique optimal solution. It can be very dangerous to make biological inferences based on seeing only a single optimal solution! Space of feasible solutions (e.g. space of trees, networks)

23 Input (e.g.character data, distance data, trees, networks) Objective function (e.g. parsimony, likelihood, minimizing reticulation) There are two optimal solutions. So in this case there is no unique optimal solution. It can be very dangerous to make biological inferences based on seeing only a single optimal solution! Space of feasible solutions (e.g. space of trees, networks)

24 Input (e.g.character data, distance data, trees, networks) Objective function (e.g. parsimony, likelihood, minimizing reticulation) There are two optimal solutions. So in this case there is no unique optimal solution. Ideally we want software to accurately describe the common characteristics of all optimal solutions. Space of feasible solutions (e.g. space of trees, networks)

25 Input (e.g.character data, distance data, trees, networks) Objective function (e.g. parsimony, likelihood, minimizing reticulation) There are two optimal solutions. So in this case there is no unique optimal solution. Unfortunately, in many cases finding even one optimal (or near-optimal) solution is already a major computational challenge. Space of feasible solutions (e.g. space of trees, networks)

26 Input (e.g.character data, distance data, trees, networks) Objective function (e.g. parsimony, likelihood, minimizing reticulation) There are two optimal solutions. So in this case there is no unique optimal solution. Also, there is often also a vast number of optimal solutions, and summarizing them is also computationally problematic (due to symmetries). Space of feasible solutions (e.g. space of trees, networks)

27 Input (e.g.character data, distance data, trees, networks) Objective function (e.g. parsimony, likelihood, minimizing reticulation) There are two optimal solutions. So in this case there is no unique optimal solution. Also, there is often also a vast number of optimal solutions, and summarizing them is also computationally problematic (due to symmetries). Four optimal solutions that are mathematically distinct, but are in fact trivial symmetries of each other can distort enumeration and sampling algorithms Space of feasible solutions (e.g. space of trees, networks)

28

29 Some modelling issues

30 Modelling issues (1) every cloud has a silver lining The exact form of the input, the objective function, and the space of feasible solutions is a modelling choice that should be made very carefully. Due to computational limitations, mathematicians often encourage biologists to make the model as simple, and well-defined, as possible. This attempt to limit model complexity is pragmatic, and restrictive. But it arguably has a positive side too.

31 Modelling issues (1) every cloud has a silver lining It forces us to ask ourselves what biologically plausible really means (i.e. to identify implicit assumptions). The modelling phase is also a good moment to clarify whether software is being used for hypothesis generation, or for hypothesis testing: a subtle but important distinction.

32 Modelling issues (2) if nothing else, lower bounds Some mathematical optimization models (e.g. Maximum Likelihood) are considered to be quite accurate at reconstructing phylogenies. This is based on many years of practice and validation. Without that level of validation it is dangerous to use a mathematical optimization model to draw direct biological conclusions. But even in the absence of validation, mathematical optimization models can give useful lower bounds on the complexity of the hypothesis required to explain the input data (i.e. the observed phenomena). MP is a classic example of this.

33 Modelling issues (3) Mixed-up messages An (at the time, for me unexpected) danger of mathematical modelling is nicely illustrated by the following. In the rooted phylogenetic networks community, there has been quite a bit of interest from mathematicians in level-k networks. The level of a network is just a measurement of how locally reticulate it is. The higher the level, the more reticulate the network is. It is just a measurement, not a value judgement on the plausibility of the network.

34

35 Modelling issues (3) Mixed-up messages One of the reasons why mathematicians liked this way of ordering the space of phylogenetic networks, is that it is computationally easier to optimize over the space of lower-level networks, than higher-level networks. This became hip and people started asking, but is it biologically plausible to assume that real phylogenetic networks are low level? But this was never the point: level was intended as a roadmap to chart and understand the space of phylogenetic networks. Not as a hypothesis of biological relevance. But this is how it was interpreted!

36 Attaining optimality

37 Tractability Suppose we have chosen a mathematical optimization model. How do we find an optimal solution? Is it even possible to compute an optimal solution without using an impossible amount of time or computer memory? That depends on many factors. In applied mathematics, an enormous amount of research time is spent trying to understand which mathematical optimization problems are tractable and which are intractable.

38 Tractability The good news is that some mathematical optimization models are tractable. That is, you can construct optimal solutions quite quickly and without using a huge amount of computer memory. There was big progress on this front in the 1950s-1980s. Arguably the biggest breakthrough was the realisation that any mathematical optimization model that could be expressed as a linear program, could in practice be solved efficiently.

39

40 Intractability The bottom line is that if a biological model can be squeezed into a tractable mathematical form that we understand well (e.g. linear programming), then this is great news. Unfortunately, a great many mathematical models that arise in practice are at least in theory intractable. This means that the amount of computational time required to construct optimal solutions, grows explosively as the size of the input data increases. The most well-known way of showing that a problem is intractable, is to give a mathematical proof that it is NP-hard. Models such as MP and ML actually belong to this group.

41 Running time Size of input

42 Unfortunately most biologically expressive mathematical models behave like the green line Running time Size of input

43 Intractability So what should we do if a mathematical optimization model is intractable? Is it the end of the road? No. It s actually only the beginning of the story. We are helped by the fact that intractability is a worst-case, pessimistic concept. That is, in many cases it will still be possible to compute optimal solutions. There is a huge amount of interest (both research and applied) in developing general software tools that, within limits, can compute optimal solutions to intractable problems. E.g. Integer Linear Programming (ILP).

44

45

46 Intractability - suboptimality Even with the availability of very powerful general tools for tackling intractable problems (such as ILP), there is a limit to what can be done. Sometimes you simply have to give up, and accept sub-optimal solutions. The good news is that if we no longer insist on finding optimal solutions, it is always possible to find some solution. The obvious question is then: is it a good solution? This is the fundamental difference between heuristics and approximation algorithms.

47 Input (e.g.character data, distance data, trees, networks) Objective function (e.g. parsimony, likelihood, minimizing reticulation) Solutions grouped according to increasing quality Space of feasible solutions (e.g. space of trees, networks)

48 Input (e.g.character data, distance data, trees, networks) Objective function (e.g. parsimony, likelihood, minimizing reticulation) A heuristic will give you a solution, but conveys no information about how far the solution is from optimality Space of feasible solutions (e.g. space of trees, networks)

49 Input (e.g.character data, distance data, trees, networks) Objective function (e.g. parsimony, likelihood, minimizing reticulation) A heuristic will give you a solution, but conveys no information about how far the solution is from optimality Space of feasible solutions (e.g. space of trees, networks)

50 Input (e.g.character data, distance data, trees, networks) Objective function (e.g. parsimony, likelihood, minimizing reticulation) A heuristic will give you a solution, but conveys no information about how far the solution is from optimality I ve found a solution. Parsimony score 200! Great! Space of feasible solutions (e.g. space of trees, networks)

51 Input (e.g.character data, distance data, trees, networks) Objective function (e.g. parsimony, likelihood, minimizing reticulation) Best Score = 190 A heuristic will give you a solution, but conveys no information about how far the solution is from optimality I ve found a solution. Parsimony score 200! Great! indeed, not bad at all Space of feasible solutions (e.g. space of trees, networks)

52 Input (e.g.character data, distance data, trees, networks) Objective function (e.g. parsimony, likelihood, minimizing reticulation) Best Score = 5 A heuristic will give you a solution, but conveys no information about how far the solution is from optimality I ve found a solution. Parsimony score 200! Great! no, really bad Space of feasible solutions (e.g. space of trees, networks)

53 Input (e.g.character data, distance data, trees, networks) Objective function (e.g. parsimony, likelihood, minimizing reticulation) An approximation algorithm will give you a solution and some (pessimistic) measure of how far you are from optimality Space of feasible solutions (e.g. space of trees, networks)

54 Input (e.g.character data, distance data, trees, networks) Objective function (e.g. parsimony, likelihood, minimizing reticulation) I ve found a solution. Parsimony score 200. Furthermore I can guarantee that it is within 20% of optimality. Space of feasible solutions (e.g. space of trees, networks)

55 Input (e.g.character data, distance data, trees, networks) Objective function (e.g. parsimony, likelihood, minimizing reticulation) I ve found a solution. Parsimony score 200. Furthermore I can guarantee that it is within 20% of optimality. We obtain information about how large this gap can be Space of feasible solutions (e.g. space of trees, networks)

56 Intractability - suboptimality The obvious question is then: is it a good solution? This is the fundamental difference between heuristics and approximation algorithms. Certainly in my field, heuristic is a very dirty word, because they give us no reliable quality guarantees. So how do we know that the solution is good? There is of course still a role for heuristics, but for them to have any meaning they have to be experimentally/empirically validated. When this happens, they can be (much) more useful in practice than approximation algorithms which are anyway difficult to develop.

57 Conclusions

58 Mathematical optimization models can be a useful tool whenever biological questions can be framed in terms of optimality. The necessarily simplistic and restrictive character of such models is also a strength: it forces us to codify and rank implicit, qualitative and complex knowledge. Be careful not to over-interpret the output of such models, especially when they have not been experimentally validated. Arguably, the safest approach is to use mathematical optimization as an instrument in a more conventional biological analysis. Computational intractability is a problem that threatens our ability to compute optimal solutions, and inevitably influences the modelling process. However, many techniques are available for dealing with intractable problems. Be wary of any software package which cannot explain what exactly it is optimizing, or the proximity of its solutions to optimality!

59 Thanks for listening

### S = {1, 2,..., n}. P (1, 1) P (1, 2)... P (1, n) P (2, 1) P (2, 2)... P (2, n) P = . P (n, 1) P (n, 2)... P (n, n)

last revised: 26 January 2009 1 Markov Chains A Markov chain process is a simple type of stochastic process with many social science applications. We ll start with an abstract description before moving

### Gerry Hobbs, Department of Statistics, West Virginia University

Decision Trees as a Predictive Modeling Method Gerry Hobbs, Department of Statistics, West Virginia University Abstract Predictive modeling has become an important area of interest in tasks such as credit

### Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 13. Random Variables: Distribution and Expectation

CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 3 Random Variables: Distribution and Expectation Random Variables Question: The homeworks of 20 students are collected

### A Working Knowledge of Computational Complexity for an Optimizer

A Working Knowledge of Computational Complexity for an Optimizer ORF 363/COS 323 Instructor: Amir Ali Ahmadi TAs: Y. Chen, G. Hall, J. Ye Fall 2014 1 Why computational complexity? What is computational

### Chapter 29: Natural Monopoly and Discrimination

Chapter 29: Natural Monopoly and Discrimination 29.1: Introduction This chapter discusses two things, both related to the fact that, in the presence of a monopoly, there is less surplus generated in the

### The first problem we're going to tackle is quicksort. Just as all of the complexity of mergesort is in the merge step, not in the recursion, all of

1 The first problem we're going to tackle is quicksort. Just as all of the complexity of mergesort is in the merge step, not in the recursion, all of the complexity of quicksort is found in the partition

### Bayesian Phylogeny and Measures of Branch Support

Bayesian Phylogeny and Measures of Branch Support Bayesian Statistics Imagine we have a bag containing 100 dice of which we know that 90 are fair and 10 are biased. The

### The Strategic Planning Process

Linking Strategy and Process, Part 3 The Strategic Planning Process The second article of this series explained how to gather strategically critical information via a comprehensive planning process. This

### IS YOUR DATA WAREHOUSE SUCCESSFUL? Developing a Data Warehouse Process that responds to the needs of the Enterprise.

IS YOUR DATA WAREHOUSE SUCCESSFUL? Developing a Data Warehouse Process that responds to the needs of the Enterprise. Peter R. Welbrock Smith-Hanley Consulting Group Philadelphia, PA ABSTRACT Developing

### Contracting for Agile Software Projects

Contracting for Agile Software Projects Author: Peter Stevens As a customer or supplier of software services at the beginning of a Software Development Project, you know that there is too much at stake

### Data Coding and Entry Lessons Learned

Chapter 7 Data Coding and Entry Lessons Learned Pércsich Richárd Introduction In this chapter we give an overview of the process of coding and entry of the 1999 pilot test data for the English examination

### U.C. Berkeley CS276: Cryptography Handout 0.1 Luca Trevisan January, 2009. Notes on Algebra

U.C. Berkeley CS276: Cryptography Handout 0.1 Luca Trevisan January, 2009 Notes on Algebra These notes contain as little theory as possible, and most results are stated without proof. Any introductory

### Factoring & Primality

Factoring & Primality Lecturer: Dimitris Papadopoulos In this lecture we will discuss the problem of integer factorization and primality testing, two problems that have been the focus of a great amount

### Sample Problems in Discrete Mathematics

Sample Problems in Discrete Mathematics This handout lists some sample problems that you should be able to solve as a pre-requisite to Computer Algorithms Try to solve all of them You should also read

### Consumer Awareness Guide. Using Recruitment Agencies

Consumer Awareness Guide Using Recruitment Agencies Prepared By Ian M Campbell Avenue Scotland Introduction At Avenue Scotland, we take great pride in the honest, professional service we provide. We have

### Markov chains and Markov Random Fields (MRFs)

Markov chains and Markov Random Fields (MRFs) 1 Why Markov Models We discuss Markov models now. This is the simplest statistical model in which we don t assume that all variables are independent; we assume

### each college c i C has a capacity q i - the maximum number of students it will admit

n colleges in a set C, m applicants in a set A, where m is much larger than n. each college c i C has a capacity q i - the maximum number of students it will admit each college c i has a strict order i

### PRINCIPLES OF ORGANIZATIONAL GOAL SETTING. Author, Birge D. Reichard Ph.D.

PRINCIPLES OF ORGANIZATIONAL GOAL SETTING Author, Birge D. Reichard Ph.D. Introduction Most operational planning processes require that organizational goals be set as a starting point for fiscal year planning

### Project and Production Management Prof. Arun Kanda Department of Mechanical Engineering Indian Institute of Technology, Delhi

Project and Production Management Prof. Arun Kanda Department of Mechanical Engineering Indian Institute of Technology, Delhi Lecture - 15 Limited Resource Allocation Today we are going to be talking about

### 2DI36 Statistics. 2DI36 Part II (Chapter 7 of MR)

2DI36 Statistics 2DI36 Part II (Chapter 7 of MR) What Have we Done so Far? Last time we introduced the concept of a dataset and seen how we can represent it in various ways But, how did this dataset came

### SIMS 255 Foundations of Software Design. Complexity and NP-completeness

SIMS 255 Foundations of Software Design Complexity and NP-completeness Matt Welsh November 29, 2001 mdw@cs.berkeley.edu 1 Outline Complexity of algorithms Space and time complexity ``Big O'' notation Complexity

### Outline. NP-completeness. When is a problem easy? When is a problem hard? Today. Euler Circuits

Outline NP-completeness Examples of Easy vs. Hard problems Euler circuit vs. Hamiltonian circuit Shortest Path vs. Longest Path 2-pairs sum vs. general Subset Sum Reducing one problem to another Clique

### (Refer Slide Time: 01:52)

Software Engineering Prof. N. L. Sarda Computer Science & Engineering Indian Institute of Technology, Bombay Lecture - 2 Introduction to Software Engineering Challenges, Process Models etc (Part 2) This

### Lecture notes 1: Introduction to linear and (mixed) integer programs

Lecture notes 1: Introduction to linear and (mixed) integer programs Vincent Conitzer 1 An example We will start with a simple example. Suppose we are in the business of selling reproductions of two different

### Hidden Markov Models

8.47 Introduction to omputational Molecular Biology Lecture 7: November 4, 2004 Scribe: Han-Pang hiu Lecturer: Ross Lippert Editor: Russ ox Hidden Markov Models The G island phenomenon The nucleotide frequencies

### Optimal Auctions Continued

Lecture 6 Optimal Auctions Continued 1 Recap Last week, we... Set up the Myerson auction environment: n risk-neutral bidders independent types t i F i with support [, b i ] residual valuation of t 0 for

### Lecture 6 Online and streaming algorithms for clustering

CSE 291: Unsupervised learning Spring 2008 Lecture 6 Online and streaming algorithms for clustering 6.1 On-line k-clustering To the extent that clustering takes place in the brain, it happens in an on-line

### University of Hull Department of Computer Science. Wrestling with Python Week 01 Playing with Python

Introduction Welcome to our Python sessions. University of Hull Department of Computer Science Wrestling with Python Week 01 Playing with Python Vsn. 1.0 Rob Miles 2013 Please follow the instructions carefully.

### What Is A Security Program? How Do I Build A Successful Program?

What Is A Security Program? How Do I Build A Successful Program? White Paper A Security Program is like building a house, the standards provide you with a list of parts needed to build the house and a

### JUST-IN-TIME SCHEDULING WITH PERIODIC TIME SLOTS. Received December May 12, 2003; revised February 5, 2004

Scientiae Mathematicae Japonicae Online, Vol. 10, (2004), 431 437 431 JUST-IN-TIME SCHEDULING WITH PERIODIC TIME SLOTS Ondřej Čepeka and Shao Chin Sung b Received December May 12, 2003; revised February

### NEW VERSION OF DECISION SUPPORT SYSTEM FOR EVALUATING TAKEOVER BIDS IN PRIVATIZATION OF THE PUBLIC ENTERPRISES AND SERVICES

NEW VERSION OF DECISION SUPPORT SYSTEM FOR EVALUATING TAKEOVER BIDS IN PRIVATIZATION OF THE PUBLIC ENTERPRISES AND SERVICES Silvija Vlah Kristina Soric Visnja Vojvodic Rosenzweig Department of Mathematics

### Reading 7 : Program Correctness

CS/Math 240: Introduction to Discrete Mathematics Fall 2015 Instructors: Beck Hasti, Gautam Prakriya Reading 7 : Program Correctness 7.1 Program Correctness Showing that a program is correct means that

### A single minimal complement for the c.e. degrees

A single minimal complement for the c.e. degrees Andrew Lewis Leeds University, April 2002 Abstract We show that there exists a single minimal (Turing) degree b < 0 s.t. for all c.e. degrees 0 < a < 0,

### Integer programming solution methods - introduction

Integer programming solution methods - introduction J E Beasley Capital budgeting There are four possible projects, which each run for 3 years and have the following characteristics. Capital requirements

### 136 CHAPTER 4. INDUCTION, GRAPHS AND TREES

136 TER 4. INDUCTION, GRHS ND TREES 4.3 Graphs In this chapter we introduce a fundamental structural idea of discrete mathematics, that of a graph. Many situations in the applications of discrete mathematics

### Guido Sciavicco. 11 Novembre 2015

classical and new techniques Università degli Studi di Ferrara 11 Novembre 2015 in collaboration with dr. Enrico Marzano, CIO Gap srl Active Contact System Project 1/27 Contents What is? Embedded Wrapper

### Introduction to Bioinformatics AS 250.265 Laboratory Assignment 6

Introduction to Bioinformatics AS 250.265 Laboratory Assignment 6 In the last lab, you learned how to perform basic multiple sequence alignments. While useful in themselves for determining conserved residues

### Problem Analysis Techniques

Introduction Problem Analysis Techniques Derrick Brown, Director Jan Kusiak, General Manager IRM Training Pty Ltd ABN 56 007 219 589 Suite 209, 620 St Kilda Rd, Melbourne, Vic. 3004, Australia 03 9533

### Lecture 4 Online and streaming algorithms for clustering

CSE 291: Geometric algorithms Spring 2013 Lecture 4 Online and streaming algorithms for clustering 4.1 On-line k-clustering To the extent that clustering takes place in the brain, it happens in an on-line

### Student Guide for Mesquite

MESQUITE Student User Guide 1 Student Guide for Mesquite This guide describes how to 1. create a project file, 2. construct phylogenetic trees, and 3. map trait evolution on branches (e.g. morphological

### Lecture 10: Regression Trees

Lecture 10: Regression Trees 36-350: Data Mining October 11, 2006 Reading: Textbook, sections 5.2 and 10.5. The next three lectures are going to be about a particular kind of nonlinear predictive model,

### The Relative Worst Order Ratio for On-Line Algorithms

The Relative Worst Order Ratio for On-Line Algorithms Joan Boyar 1 and Lene M. Favrholdt 2 1 Department of Mathematics and Computer Science, University of Southern Denmark, Odense, Denmark, joan@imada.sdu.dk

### 6.852: Distributed Algorithms Fall, 2009. Class 2

.8: Distributed Algorithms Fall, 009 Class Today s plan Leader election in a synchronous ring: Lower bound for comparison-based algorithms. Basic computation in general synchronous networks: Leader election

### Theorem 2. If x Q and y R \ Q, then. (a) x + y R \ Q, and. (b) xy Q.

Math 305 Fall 011 The Density of Q in R The following two theorems tell us what happens when we add and multiply by rational numbers. For the first one, we see that if we add or multiply two rational numbers

### Bayesian Learning. CSL465/603 - Fall 2016 Narayanan C Krishnan

Bayesian Learning CSL465/603 - Fall 2016 Narayanan C Krishnan ckn@iitrpr.ac.in Outline Bayes Theorem MAP Learners Bayes optimal classifier Naïve Bayes classifier Example text classification Bayesian networks

### Applied Algorithm Design Lecture 5

Applied Algorithm Design Lecture 5 Pietro Michiardi Eurecom Pietro Michiardi (Eurecom) Applied Algorithm Design Lecture 5 1 / 86 Approximation Algorithms Pietro Michiardi (Eurecom) Applied Algorithm Design

### 8. KNOWLEDGE BASED SYSTEMS IN MANUFACTURING SIMULATION

- 1-8. KNOWLEDGE BASED SYSTEMS IN MANUFACTURING SIMULATION 8.1 Introduction 8.1.1 Summary introduction The first part of this section gives a brief overview of some of the different uses of expert systems

### ON THE COMPLEXITY OF THE GAME OF SET. {kamalika,pbg,dratajcz,hoeteck}@cs.berkeley.edu

ON THE COMPLEXITY OF THE GAME OF SET KAMALIKA CHAUDHURI, BRIGHTEN GODFREY, DAVID RATAJCZAK, AND HOETECK WEE {kamalika,pbg,dratajcz,hoeteck}@cs.berkeley.edu ABSTRACT. Set R is a card game played with a

### Concept Learning. Machine Learning 1

Concept Learning Inducing general functions from specific training examples is a main issue of machine learning. Concept Learning: Acquiring the definition of a general category from given sample positive

### arxiv:1112.0829v1 [math.pr] 5 Dec 2011

How Not to Win a Million Dollars: A Counterexample to a Conjecture of L. Breiman Thomas P. Hayes arxiv:1112.0829v1 [math.pr] 5 Dec 2011 Abstract Consider a gambling game in which we are allowed to repeatedly

### NP-completeness and the real world. NP completeness. NP-completeness and the real world (2) NP-completeness and the real world

-completeness and the real world completeness Course Discrete Biological Models (Modelli Biologici Discreti) Zsuzsanna Lipták Imagine you are working for a biotech company. One day your boss calls you

HYBRID GENETIC ALGORITHMS FOR SCHEDULING ADVERTISEMENTS ON A WEB PAGE Subodha Kumar University of Washington subodha@u.washington.edu Varghese S. Jacob University of Texas at Dallas vjacob@utdallas.edu

### 10.1 Integer Programming and LP relaxation

CS787: Advanced Algorithms Lecture 10: LP Relaxation and Rounding In this lecture we will design approximation algorithms using linear programming. The key insight behind this approach is that the closely

### PloneSurvey User Guide (draft 3)

- 1 - PloneSurvey User Guide (draft 3) This short document will hopefully contain enough information to allow people to begin creating simple surveys using the new Plone online survey tool. Caveat PloneSurvey

### Mathematical Induction

Mathematical Induction MAT30 Discrete Mathematics Fall 016 MAT30 (Discrete Math) Mathematical Induction Fall 016 1 / 19 Outline 1 Mathematical Induction Strong Mathematical Induction MAT30 (Discrete Math)

### 2.3 Scheduling jobs on identical parallel machines

2.3 Scheduling jobs on identical parallel machines There are jobs to be processed, and there are identical machines (running in parallel) to which each job may be assigned Each job = 1,,, must be processed

### What is Application Lifecycle Management?

What is Application Lifecycle Management? David Chappell Sponsored by Microsoft Corporation Copyright 2014 Chappell & Associates Defining application lifecycle management (ALM) isn t easy. Different people

### Involve-Project Manager

Involve-Project Manager This article will describe: What is Project Management Why is Project Management so important to community and voluntary organisations The Key Phases of Project Management: o Initiation

### Prime Numbers. Chapter Primes and Composites

Chapter 2 Prime Numbers The term factoring or factorization refers to the process of expressing an integer as the product of two or more integers in a nontrivial way, e.g., 42 = 6 7. Prime numbers are

### A Non-Linear Schema Theorem for Genetic Algorithms

A Non-Linear Schema Theorem for Genetic Algorithms William A Greene Computer Science Department University of New Orleans New Orleans, LA 70148 bill@csunoedu 504-280-6755 Abstract We generalize Holland

### 3. Finding the Median

3. Finding the Median We now address the following question: We have N keys, that are completely unsorted, and we wish to find the key at rank m so that there are m 1 keys bigger than it, and N m keys

### CSC 310: Information Theory

CSC 310: Information Theory University of Toronto, Fall 2011 Instructor: Radford M. Neal Week 2 What s Needed for a Theory of (Lossless) Data Compression? A context for the problem. What are we trying

### Chapter 13: Binary and Mixed-Integer Programming

Chapter 3: Binary and Mixed-Integer Programming The general branch and bound approach described in the previous chapter can be customized for special situations. This chapter addresses two special situations:

### OUTLINE OF PRINCIPLES OF IMPACT EVALUATION

OUTLINE OF PRINCIPLES OF IMPACT EVALUATION PART I KEY CONCEPTS Definition Impact evaluation is an assessment of how the intervention being evaluated affects outcomes, whether these effects are intended

### CONTENTS OF DAY 2. II. Why Random Sampling is Important 9 A myth, an urban legend, and the real reason NOTES FOR SUMMER STATISTICS INSTITUTE COURSE

1 2 CONTENTS OF DAY 2 I. More Precise Definition of Simple Random Sample 3 Connection with independent random variables 3 Problems with small populations 8 II. Why Random Sampling is Important 9 A myth,

### HOW TO USE THIS TOOLKIT

IF POLITICS IS THE ART OF THE POSSIBLE, CAMPAIGNING IS THE ART AND SCIENCE OF CHANGING WHAT IS POSSIBLE. HOW TO USE THIS TOOLKIT The purpose of this toolkit is to inform you on all things campaigns, to

### INTRODUCTORY STATISTICS

INTRODUCTORY STATISTICS FIFTH EDITION Thomas H. Wonnacott University of Western Ontario Ronald J. Wonnacott University of Western Ontario WILEY JOHN WILEY & SONS New York Chichester Brisbane Toronto Singapore

### Introduction to Strategic Supply Chain Network Design Perspectives and Methodologies to Tackle the Most Challenging Supply Chain Network Dilemmas

Introduction to Strategic Supply Chain Network Design Perspectives and Methodologies to Tackle the Most Challenging Supply Chain Network Dilemmas D E L I V E R I N G S U P P L Y C H A I N E X C E L L E

### Markov Chains, part I

Markov Chains, part I December 8, 2010 1 Introduction A Markov Chain is a sequence of random variables X 0, X 1,, where each X i S, such that P(X i+1 = s i+1 X i = s i, X i 1 = s i 1,, X 0 = s 0 ) = P(X

### Induction. Margaret M. Fleck. 10 October These notes cover mathematical induction and recursive definition

Induction Margaret M. Fleck 10 October 011 These notes cover mathematical induction and recursive definition 1 Introduction to induction At the start of the term, we saw the following formula for computing

### Cost Model: Work, Span and Parallelism. 1 The RAM model for sequential computation:

CSE341T 08/31/2015 Lecture 3 Cost Model: Work, Span and Parallelism In this lecture, we will look at how one analyze a parallel program written using Cilk Plus. When we analyze the cost of an algorithm

### Dedekind s forgotten axiom and why we should teach it (and why we shouldn t teach mathematical induction in our calculus classes)

Dedekind s forgotten axiom and why we should teach it (and why we shouldn t teach mathematical induction in our calculus classes) by Jim Propp (UMass Lowell) March 14, 2010 1 / 29 Completeness Three common

### [Refer Slide Time: 05:10]

Principles of Programming Languages Prof: S. Arun Kumar Department of Computer Science and Engineering Indian Institute of Technology Delhi Lecture no 7 Lecture Title: Syntactic Classes Welcome to lecture

### CONTRIBUTIONS TO ZERO SUM PROBLEMS

CONTRIBUTIONS TO ZERO SUM PROBLEMS S. D. ADHIKARI, Y. G. CHEN, J. B. FRIEDLANDER, S. V. KONYAGIN AND F. PAPPALARDI Abstract. A prototype of zero sum theorems, the well known theorem of Erdős, Ginzburg

### Statistics Review PSY379

Statistics Review PSY379 Basic concepts Measurement scales Populations vs. samples Continuous vs. discrete variable Independent vs. dependent variable Descriptive vs. inferential stats Common analyses

### Part 1: Project Portfolio Management Tools

Part 1: Project Portfolio Management Tools To survive and prosper in today's competitive, cost-conscious, and risky business environment, organizations must derive greater value from the projects that

### diffy boxes (iterations of the ducci four number game) 1

diffy boxes (iterations of the ducci four number game) 1 Peter Trapa September 27, 2006 Begin at the beginning and go on till you come to the end: then stop. Lewis Carroll Consider the following game.

### Co-Creation of Models and Metamodels for Enterprise. Architecture Projects.

Co-Creation of Models and Metamodels for Enterprise Architecture Projects Paola Gómez pa.gomez398@uniandes.edu.co Hector Florez ha.florez39@uniandes.edu.co ABSTRACT The linguistic conformance and the ontological

### Kotter and Bridges handouts for participants who did not attend Workshop 1.

MATERIALS AND PREPARATION: Kotter and Bridges handouts for participants who did not attend Workshop 1. Change Management and Leadership Skills handout for participants who did not attend Workshop 1. Strategic

### Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 10

CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 10 Introduction to Discrete Probability Probability theory has its origins in gambling analyzing card games, dice,

### Computer Algorithms. NP-Complete Problems. CISC 4080 Yanjun Li

Computer Algorithms NP-Complete Problems NP-completeness The quest for efficient algorithms is about finding clever ways to bypass the process of exhaustive search, using clues from the input in order

### ECONOMETRIC THEORY. MODULE I Lecture - 1 Introduction to Econometrics

ECONOMETRIC THEORY MODULE I Lecture - 1 Introduction to Econometrics Dr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur 2 Econometrics deals with the measurement

### Amajor benefit of Monte-Carlo schedule analysis is to

2005 AACE International Transactions RISK.10 The Benefits of Monte- Carlo Schedule Analysis Mr. Jason Verschoor, P.Eng. Amajor benefit of Monte-Carlo schedule analysis is to expose underlying risks to

### Permutation Groups. Rubik s Cube

Permutation Groups and Rubik s Cube Tom Davis tomrdavis@earthlink.net May 6, 2000 Abstract In this paper we ll discuss permutations (rearrangements of objects), how to combine them, and how to construct

### Chapter 6: Episode discovery process

Chapter 6: Episode discovery process Algorithmic Methods of Data Mining, Fall 2005, Chapter 6: Episode discovery process 1 6. Episode discovery process The knowledge discovery process KDD process of analyzing

### Security-Aware Beacon Based Network Monitoring

Security-Aware Beacon Based Network Monitoring Masahiro Sasaki, Liang Zhao, Hiroshi Nagamochi Graduate School of Informatics, Kyoto University, Kyoto, Japan Email: {sasaki, liang, nag}@amp.i.kyoto-u.ac.jp

### LABEL PROPAGATION ON GRAPHS. SEMI-SUPERVISED LEARNING. ----Changsheng Liu 10-30-2014

LABEL PROPAGATION ON GRAPHS. SEMI-SUPERVISED LEARNING ----Changsheng Liu 10-30-2014 Agenda Semi Supervised Learning Topics in Semi Supervised Learning Label Propagation Local and global consistency Graph

### The Contextualization of Project Management Practice and Best Practice

The Contextualization of Project Management Practice and Best Practice Claude Besner PhD, University of Quebec at Montreal Brian Hobbs PhD, University of Quebec at Montreal Abstract This research aims

### Students in their first advanced mathematics classes are often surprised

CHAPTER 8 Proofs Involving Sets Students in their first advanced mathematics classes are often surprised by the extensive role that sets play and by the fact that most of the proofs they encounter are

### Static IP Routing and Aggregation Exercises

Politecnico di Torino Static IP Routing and Aggregation xercises Fulvio Risso August 0, 0 Contents I. Methodology 4. Static routing and routes aggregation 5.. Main concepts........................................

### Memoization/Dynamic Programming. The String reconstruction problem. CS125 Lecture 5 Fall 2016

CS125 Lecture 5 Fall 2016 Memoization/Dynamic Programming Today s lecture discusses memoization, which is a method for speeding up algorithms based on recursion, by using additional memory to remember

### Feature Selection with Monte-Carlo Tree Search

Feature Selection with Monte-Carlo Tree Search Robert Pinsler 20.01.2015 20.01.2015 Fachbereich Informatik DKE: Seminar zu maschinellem Lernen Robert Pinsler 1 Agenda 1 Feature Selection 2 Feature Selection

### Bioinformatics Resources at a Glance

Bioinformatics Resources at a Glance A Note about FASTA Format There are MANY free bioinformatics tools available online. Bioinformaticists have developed a standard format for nucleotide and protein sequences

### Revised Version of Chapter 23. We learned long ago how to solve linear congruences. ax c (mod m)

Chapter 23 Squares Modulo p Revised Version of Chapter 23 We learned long ago how to solve linear congruences ax c (mod m) (see Chapter 8). It s now time to take the plunge and move on to quadratic equations.

### Measuring performance in credit management

Measuring performance in credit management Ludo Theunissen Prof. Ghent University Instituut voor Kredietmanagement e-mail: ludo.theunissen@ivkm.be Josef Busuttil MBA (Henley); DipM MCIM; FICM Director

### Prescriptive Analytics. A business guide

Prescriptive Analytics A business guide May 2014 Contents 3 The Business Value of Prescriptive Analytics 4 What is Prescriptive Analytics? 6 Prescriptive Analytics Methods 7 Integration 8 Business Applications

### Introduction. Book Structure

Fundamentals of Project Management, 4 th Edition Simple Solutions for Busy People By Joseph Heagney (A Book review by R. Max Wideman, FPMI) The views expressed in this article are strictly those of Max

### Cryptography and Network Security Department of Computer Science and Engineering Indian Institute of Technology Kharagpur

Cryptography and Network Security Department of Computer Science and Engineering Indian Institute of Technology Kharagpur Module No. # 01 Lecture No. # 05 Classic Cryptosystems (Refer Slide Time: 00:42)