Constraint solving meets machine learning and data mining. Algorithm portfolios
|
|
|
- Cori Hamilton
- 7 years ago
- Views:
Transcription
1 Constraint solving meets machine learning and data mining Algorithm portfolios Kustaa Kangas University of Helsinki, Finland November 8, 2012 K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 1 / 29
2 Constraint problems Boolean satisifiability, integer linear programming etc K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 2 / 29
3 Constraint problems Boolean satisifiability, integer linear programming etc NP-hard K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 2 / 29
4 Constraint problems Boolean satisifiability, integer linear programming etc NP-hard Many instances solvable with heuristic algorithms K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 2 / 29
5 Constraint problems Boolean satisifiability, integer linear programming etc NP-hard Many instances solvable with heuristic algorithms High variance in performance, from milliseconds to weeks K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 2 / 29
6 Constraint problems Boolean satisifiability, integer linear programming etc NP-hard Many instances solvable with heuristic algorithms High variance in performance, from milliseconds to weeks Different algorithms are fast on different instances K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 2 / 29
7 Constraint problems Boolean satisifiability, integer linear programming etc NP-hard Many instances solvable with heuristic algorithms High variance in performance, from milliseconds to weeks Different algorithms are fast on different instances Typically no single best algorithm K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 2 / 29
8 Algorithm selection Algorithm selection problem Given a problem instance, which algorithm should we run? Ideally: run the algorithm that s fastest on the instance Problem: we cannot know without running the algorithms Traditional solution: run the average-case best algorithm Might be reasonably good Can be very bad on some instances Ignores algorithms that are good on some instances K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 3 / 29
9 Algorithm portfolios Idea: Use several algorithms to improve expected performance Algorithm portfolio 1 a collection of algorithms 2 a strategy for running them A variety of strategies K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 4 / 29
10 Algorithm portfolios Idea: Use several algorithms to improve expected performance Algorithm portfolio 1 a collection of algorithms 2 a strategy for running them A variety of strategies Run all algorithms (sequentially / in parallel) K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 4 / 29
11 Algorithm portfolios Idea: Use several algorithms to improve expected performance Algorithm portfolio 1 a collection of algorithms 2 a strategy for running them A variety of strategies Run all algorithms (sequentially / in parallel) Select one algorithm based on the instance K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 4 / 29
12 Algorithm portfolios Idea: Use several algorithms to improve expected performance Algorithm portfolio 1 a collection of algorithms 2 a strategy for running them A variety of strategies Run all algorithms (sequentially / in parallel) Select one algorithm based on the instance Anything from between K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 4 / 29
13 SATzilla A highly successful SAT portfolio solver Uses state-of-the-art SAT solvers K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 5 / 29
14 SATzilla A highly successful SAT portfolio solver Uses state-of-the-art SAT solvers Trains an empirical hardness model for each algorithm K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 5 / 29
15 SATzilla A highly successful SAT portfolio solver Uses state-of-the-art SAT solvers Trains an empirical hardness model for each algorithm explains how hard instances are and why K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 5 / 29
16 SATzilla A highly successful SAT portfolio solver Uses state-of-the-art SAT solvers Trains an empirical hardness model for each algorithm explains how hard instances are and why an approximate predictor of running time K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 5 / 29
17 SATzilla A highly successful SAT portfolio solver Uses state-of-the-art SAT solvers Trains an empirical hardness model for each algorithm explains how hard instances are and why an approximate predictor of running time predicts hardness based on instance features K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 5 / 29
18 SATzilla A highly successful SAT portfolio solver Uses state-of-the-art SAT solvers Trains an empirical hardness model for each algorithm explains how hard instances are and why an approximate predictor of running time predicts hardness based on instance features Selects the algorithm predicted to be fastest K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 5 / 29
19 SATzilla A highly successful SAT portfolio solver Uses state-of-the-art SAT solvers Trains an empirical hardness model for each algorithm explains how hard instances are and why an approximate predictor of running time predicts hardness based on instance features Selects the algorithm predicted to be fastest Performed well in 2007 SAT Competition 1st place in 3 categories, one 2nd place and one 3rd place K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 5 / 29
20 Empirical hardness models Where do empirical hardness models come from? K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 6 / 29
21 Empirical hardness models Where do empirical hardness models come from? They must be learned from data: 1 a set of algorithms 2 a set of training instances 3 a set of instance features K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 6 / 29
22 Empirical hardness models Where do empirical hardness models come from? They must be learned from data: 1 a set of algorithms 2 a set of training instances 3 a set of instance features We use machine learning to exploit correlations between features and algorithm performance K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 6 / 29
23 Empirical hardness models problem instance x 1 x 2 x 3 x 4 x 5 x 6 running time instance 1 instance 2 instance 3 instance 4 instance 5 instance 6 instance 7 instance 8 instance 9 instance 10 instance 11 instance 12 K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 7 / 29
24 Empirical hardness models problem instance x 1 x 2 x 3 x 4 x 5 x 6 running time instance instance instance instance instance instance instance instance instance instance instance instance K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 8 / 29
25 Empirical hardness models problem instance x 1 x 2 x 3 x 4 x 5 x 6 running time instance instance instance instance instance instance instance instance instance instance instance instance K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 9 / 29
26 Empirical hardness models problem instance x 1 x 2 x 3 x 4 x 5 x 6 running time instance instance instance instance instance instance instance instance instance instance instance instance new instance K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 10 / 29
27 Empirical hardness models problem instance x 1 x 2 x 3 x 4 x 5 x 6 running time instance instance instance instance instance instance instance instance instance instance instance instance new instance K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 11 / 29
28 Empirical hardness models problem instance x 1 x 2 x 3 x 4 x 5 x 6 running time instance instance instance instance instance instance instance instance instance instance instance instance new instance ? K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 12 / 29
29 Linear regression running time feature K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 13 / 29
30 Linear regression running time feature K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 14 / 29
31 Linear regression Not limited to just one variable For features x 1, x 2,, x m we fit a hyperplane f w of form f w (x) = w 1 x 1 + w 2 x w m x m We fit w 1, w 2,, w m to minimize prediction error, eg n (f w (x i ) y i ) 2 i=1 where y i is the running time on instance i K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 15 / 29
32 Linear regression Easily minimized by setting w = (Φ T Φ) 1 Φ T y where Φ is the feature matrix Dominated by matrix inversion, which is O(n 3 ) Used by SATzilla Simple and works well in practice K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 16 / 29
33 Identifying features Success of the model depends crucially on the features Features must be Correlated with running time Cheap to compute Feature computation is part of portfolio s running time! How do we find such features? Features are problem-specific No automatic way to find them Requires domain expertise K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 17 / 29
34 Linear regression running time feature K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 18 / 29
35 SAT features SATzilla uses 84 features related to eg instance size number of variables, number of clauses ratio between these two balance ratio of positive and negative literals fraction of binary and ternary clauses variable clause graph variable degrees: average, min, max clause degrees: average, min, max local search probe statistics number of steps to a local optimum average improvement per step proximity to Horn formula K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 19 / 29
36 Feature selection Features can be uninformative: no correlation with running time redundant: highly correlated with other features Problematic: K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 20 / 29
37 Feature selection Features can be uninformative: no correlation with running time redundant: highly correlated with other features Problematic: Unnecessary feature computation K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 20 / 29
38 Feature selection Features can be uninformative: no correlation with running time redundant: highly correlated with other features Problematic: Unnecessary feature computation Learned models are harder to interpret K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 20 / 29
39 Feature selection Features can be uninformative: no correlation with running time redundant: highly correlated with other features Problematic: Unnecessary feature computation Learned models are harder to interpret Regression becomes unstable K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 20 / 29
40 Feature selection Useless features can be pruned automatically A subset of features can be evaluated by cross-validation K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 21 / 29
41 Feature selection Useless features can be pruned automatically A subset of features can be evaluated by cross-validation Exhaustive search of all subsets Infeasible for many features K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 21 / 29
42 Feature selection Useless features can be pruned automatically A subset of features can be evaluated by cross-validation Exhaustive search of all subsets Infeasible for many features Greedy heuristic search K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 21 / 29
43 Feature selection Useless features can be pruned automatically A subset of features can be evaluated by cross-validation Exhaustive search of all subsets Infeasible for many features Greedy heuristic search Forward selection: start with no features, add greedily K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 21 / 29
44 Feature selection Useless features can be pruned automatically A subset of features can be evaluated by cross-validation Exhaustive search of all subsets Infeasible for many features Greedy heuristic search Forward selection: start with no features, add greedily Backward elimination: start with all features, remove greedily K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 21 / 29
45 Feature selection Useless features can be pruned automatically A subset of features can be evaluated by cross-validation Exhaustive search of all subsets Infeasible for many features Greedy heuristic search Forward selection: start with no features, add greedily Backward elimination: start with all features, remove greedily Sequential replacement: add and replace greedily K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 21 / 29
46 Basis function expansion Linear regression is limited to linear correlations Problematic: not all useful correlations are linear K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 22 / 29
47 Basis function expansion Linear regression is limited to linear correlations Problematic: not all useful correlations are linear Generalizing regression gets complicated K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 22 / 29
48 Basis function expansion Linear regression is limited to linear correlations Problematic: not all useful correlations are linear Generalizing regression gets complicated Quadratically dependent on x linearly dependent on x 2 K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 22 / 29
49 Basis function expansion Linear regression is limited to linear correlations Problematic: not all useful correlations are linear Generalizing regression gets complicated Quadratically dependent on x linearly dependent on x 2 Solution: add functions of original features Known as basis function expansion K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 22 / 29
50 Basis function expansion SATzilla uses quadratic expansion: Add all pairwise products of features K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 23 / 29
51 Basis function expansion SATzilla uses quadratic expansion: Add all pairwise products of features Number of features can explode: 84 2 = 7056 K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 23 / 29
52 Basis function expansion SATzilla uses quadratic expansion: Add all pairwise products of features Number of features can explode: 84 2 = 7056 Regression becomes slow K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 23 / 29
53 Basis function expansion SATzilla uses quadratic expansion: Add all pairwise products of features Number of features can explode: 84 2 = 7056 Regression becomes slow Can lead to overfitting K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 23 / 29
54 Basis function expansion SATzilla uses quadratic expansion: Add all pairwise products of features Number of features can explode: 84 2 = 7056 Regression becomes slow Can lead to overfitting Many new features are useless: feature selection before and after expansion! K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 23 / 29
55 Terminated runs What if gathering running time data takes too long? Algorithms can run literally for weeks Such runs must be terminated prematurely How to use these runs to build models? K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 24 / 29
56 Terminated runs Solution 1: discard all such runs Not very sensible We want to learn that such instances are hard Solution 2: pretend they stopped at the cutoff limit Better: takes hardness into account Still systematically underestimates hardness K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 25 / 29
57 Terminated runs Solution 3: treat cutoff times as lower bounds Known as censoring in statistics Makes use of all information available K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 26 / 29
58 K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 27 / 29
59 Algorithm portfolios Have been applied to various constraint problems: Boolean satisfiability (SAT) MaxSAT Mixed integer programming Constraint satisfaction (CSP) Combinatorial auctions Answer set programming Zero-one integer programming K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 28 / 29
60 Conclusions Algorithm portfolios can improve expected performance when no single algorithm is dominant Particularly useful for NP-hard constraint problems where the running times exhibit high variance In addition to predicting running time, empirical hardness models are valuable tools for understanding the hardness of problems K Kangas (U Helsinki) Constraint solving meetsmachine learning and data mining November Algorithm 8, 2012 portfolios 29 / 29
CNFSAT: Predictive Models, Dimensional Reduction, and Phase Transition
CNFSAT: Predictive Models, Dimensional Reduction, and Phase Transition Neil P. Slagle College of Computing Georgia Institute of Technology Atlanta, GA npslagle@gatech.edu Abstract CNFSAT embodies the P
npsolver A SAT Based Solver for Optimization Problems
npsolver A SAT Based Solver for Optimization Problems Norbert Manthey and Peter Steinke Knowledge Representation and Reasoning Group Technische Universität Dresden, 01062 Dresden, Germany peter@janeway.inf.tu-dresden.de
Part 2: Analysis of Relationship Between Two Variables
Part 2: Analysis of Relationship Between Two Variables Linear Regression Linear correlation Significance Tests Multiple regression Linear Regression Y = a X + b Dependent Variable Independent Variable
5 INTEGER LINEAR PROGRAMMING (ILP) E. Amaldi Fondamenti di R.O. Politecnico di Milano 1
5 INTEGER LINEAR PROGRAMMING (ILP) E. Amaldi Fondamenti di R.O. Politecnico di Milano 1 General Integer Linear Program: (ILP) min c T x Ax b x 0 integer Assumption: A, b integer The integrality condition
Guessing Game: NP-Complete?
Guessing Game: NP-Complete? 1. LONGEST-PATH: Given a graph G = (V, E), does there exists a simple path of length at least k edges? YES 2. SHORTEST-PATH: Given a graph G = (V, E), does there exists a simple
Acknowledgments. Data Mining with Regression. Data Mining Context. Overview. Colleagues
Data Mining with Regression Teaching an old dog some new tricks Acknowledgments Colleagues Dean Foster in Statistics Lyle Ungar in Computer Science Bob Stine Department of Statistics The School of the
Optimization Modeling for Mining Engineers
Optimization Modeling for Mining Engineers Alexandra M. Newman Division of Economics and Business Slide 1 Colorado School of Mines Seminar Outline Linear Programming Integer Linear Programming Slide 2
Linear smoother. ŷ = S y. where s ij = s ij (x) e.g. s ij = diag(l i (x)) To go the other way, you need to diagonalize S
Linear smoother ŷ = S y where s ij = s ij (x) e.g. s ij = diag(l i (x)) To go the other way, you need to diagonalize S 2 Online Learning: LMS and Perceptrons Partially adapted from slides by Ryan Gabbard
Lecture 10: Regression Trees
Lecture 10: Regression Trees 36-350: Data Mining October 11, 2006 Reading: Textbook, sections 5.2 and 10.5. The next three lectures are going to be about a particular kind of nonlinear predictive model,
NP-Completeness I. Lecture 19. 19.1 Overview. 19.2 Introduction: Reduction and Expressiveness
Lecture 19 NP-Completeness I 19.1 Overview In the past few lectures we have looked at increasingly more expressive problems that we were able to solve using efficient algorithms. In this lecture we introduce
Discuss the size of the instance for the minimum spanning tree problem.
3.1 Algorithm complexity The algorithms A, B are given. The former has complexity O(n 2 ), the latter O(2 n ), where n is the size of the instance. Let n A 0 be the size of the largest instance that can
Why? A central concept in Computer Science. Algorithms are ubiquitous.
Analysis of Algorithms: A Brief Introduction Why? A central concept in Computer Science. Algorithms are ubiquitous. Using the Internet (sending email, transferring files, use of search engines, online
Multiple Linear Regression in Data Mining
Multiple Linear Regression in Data Mining Contents 2.1. A Review of Multiple Linear Regression 2.2. Illustration of the Regression Process 2.3. Subset Selection in Linear Regression 1 2 Chap. 2 Multiple
Dynamic programming formulation
1.24 Lecture 14 Dynamic programming: Job scheduling Dynamic programming formulation To formulate a problem as a dynamic program: Sort by a criterion that will allow infeasible combinations to be eli minated
Integrating Benders decomposition within Constraint Programming
Integrating Benders decomposition within Constraint Programming Hadrien Cambazard, Narendra Jussien email: {hcambaza,jussien}@emn.fr École des Mines de Nantes, LINA CNRS FRE 2729 4 rue Alfred Kastler BP
. P. 4.3 Basic feasible solutions and vertices of polyhedra. x 1. x 2
4. Basic feasible solutions and vertices of polyhedra Due to the fundamental theorem of Linear Programming, to solve any LP it suffices to consider the vertices (finitely many) of the polyhedron P of the
Data Mining Practical Machine Learning Tools and Techniques
Ensemble learning Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 8 of Data Mining by I. H. Witten, E. Frank and M. A. Hall Combining multiple models Bagging The basic idea
! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. !-approximation algorithm.
Approximation Algorithms Chapter Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of
Backdoors to Combinatorial Optimization: Feasibility and Optimality
Backdoors to Combinatorial Optimization: Feasibility and Optimality Bistra Dilkina, Carla P. Gomes, Yuri Malitsky 2, Ashish Sabharwal, and Meinolf Sellmann 2 Department of Computer Science, Cornell University,
STATISTICA Formula Guide: Logistic Regression. Table of Contents
: Table of Contents... 1 Overview of Model... 1 Dispersion... 2 Parameterization... 3 Sigma-Restricted Model... 3 Overparameterized Model... 4 Reference Coding... 4 Model Summary (Summary Tab)... 5 Summary
! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. #-approximation algorithm.
Approximation Algorithms 11 Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of three
Ant Colony Optimization and Constraint Programming
Ant Colony Optimization and Constraint Programming Christine Solnon Series Editor Narendra Jussien WILEY Table of Contents Foreword Acknowledgements xi xiii Chapter 1. Introduction 1 1.1. Overview of the
Support Vector Machines Explained
March 1, 2009 Support Vector Machines Explained Tristan Fletcher www.cs.ucl.ac.uk/staff/t.fletcher/ Introduction This document has been written in an attempt to make the Support Vector Machines (SVM),
The Effects of Start Prices on the Performance of the Certainty Equivalent Pricing Policy
BMI Paper The Effects of Start Prices on the Performance of the Certainty Equivalent Pricing Policy Faculty of Sciences VU University Amsterdam De Boelelaan 1081 1081 HV Amsterdam Netherlands Author: R.D.R.
Big Data Analytics CSCI 4030
High dim. data Graph data Infinite data Machine learning Apps Locality sensitive hashing PageRank, SimRank Filtering data streams SVM Recommen der systems Clustering Community Detection Web advertising
Performance of Dynamic Load Balancing Algorithms for Unstructured Mesh Calculations
Performance of Dynamic Load Balancing Algorithms for Unstructured Mesh Calculations Roy D. Williams, 1990 Presented by Chris Eldred Outline Summary Finite Element Solver Load Balancing Results Types Conclusions
Discrete Optimization
Discrete Optimization [Chen, Batson, Dang: Applied integer Programming] Chapter 3 and 4.1-4.3 by Johan Högdahl and Victoria Svedberg Seminar 2, 2015-03-31 Todays presentation Chapter 3 Transforms using
not possible or was possible at a high cost for collecting the data.
Data Mining and Knowledge Discovery Generating knowledge from data Knowledge Discovery Data Mining White Paper Organizations collect a vast amount of data in the process of carrying out their day-to-day
Minesweeper as a Constraint Satisfaction Problem
Minesweeper as a Constraint Satisfaction Problem by Chris Studholme Introduction To Minesweeper Minesweeper is a simple one player computer game commonly found on machines with popular operating systems
Chapter 13: Binary and Mixed-Integer Programming
Chapter 3: Binary and Mixed-Integer Programming The general branch and bound approach described in the previous chapter can be customized for special situations. This chapter addresses two special situations:
University of Potsdam Faculty of Computer Science. Clause Learning in SAT Seminar Automatic Problem Solving WS 2005/06
University of Potsdam Faculty of Computer Science Clause Learning in SAT Seminar Automatic Problem Solving WS 2005/06 Authors: Richard Tichy, Thomas Glase Date: 25th April 2006 Contents 1 Introduction
Optimal Scheduling for Dependent Details Processing Using MS Excel Solver
BULGARIAN ACADEMY OF SCIENCES CYBERNETICS AND INFORMATION TECHNOLOGIES Volume 8, No 2 Sofia 2008 Optimal Scheduling for Dependent Details Processing Using MS Excel Solver Daniela Borissova Institute of
Decision Trees from large Databases: SLIQ
Decision Trees from large Databases: SLIQ C4.5 often iterates over the training set How often? If the training set does not fit into main memory, swapping makes C4.5 unpractical! SLIQ: Sort the values
Invited Applications Paper
Invited Applications Paper - - Thore Graepel Joaquin Quiñonero Candela Thomas Borchert Ralf Herbrich Microsoft Research Ltd., 7 J J Thomson Avenue, Cambridge CB3 0FB, UK THOREG@MICROSOFT.COM JOAQUINC@MICROSOFT.COM
Cloud Branching. Timo Berthold. joint work with Domenico Salvagnin (Università degli Studi di Padova)
Cloud Branching Timo Berthold Zuse Institute Berlin joint work with Domenico Salvagnin (Università degli Studi di Padova) DFG Research Center MATHEON Mathematics for key technologies 21/May/13, CPAIOR
Chapter 11. 11.1 Load Balancing. Approximation Algorithms. Load Balancing. Load Balancing on 2 Machines. Load Balancing: Greedy Scheduling
Approximation Algorithms Chapter Approximation Algorithms Q. Suppose I need to solve an NP-hard problem. What should I do? A. Theory says you're unlikely to find a poly-time algorithm. Must sacrifice one
Lecture 7: NP-Complete Problems
IAS/PCMI Summer Session 2000 Clay Mathematics Undergraduate Program Basic Course on Computational Complexity Lecture 7: NP-Complete Problems David Mix Barrington and Alexis Maciel July 25, 2000 1. Circuit
What is Linear Programming?
Chapter 1 What is Linear Programming? An optimization problem usually has three essential ingredients: a variable vector x consisting of a set of unknowns to be determined, an objective function of x to
GAMS, Condor and the Grid: Solving Hard Optimization Models in Parallel. Michael C. Ferris University of Wisconsin
GAMS, Condor and the Grid: Solving Hard Optimization Models in Parallel Michael C. Ferris University of Wisconsin Parallel Optimization Aid search for global solutions (typically in non-convex or discrete)
Lecture 2: The SVM classifier
Lecture 2: The SVM classifier C19 Machine Learning Hilary 2015 A. Zisserman Review of linear classifiers Linear separability Perceptron Support Vector Machine (SVM) classifier Wide margin Cost function
Module 3: Correlation and Covariance
Using Statistical Data to Make Decisions Module 3: Correlation and Covariance Tom Ilvento Dr. Mugdim Pašiƒ University of Delaware Sarajevo Graduate School of Business O ften our interest in data analysis
Minerva Access is the Institutional Repository of The University of Melbourne
Minerva Access is the Institutional Repository of The University of Melbourne Author/s: Chu, Geoffrey G. Title: Improving combinatorial optimization Date: 2011 Citation: Chu, G. G. (2011). Improving combinatorial
Mathematics Online Instructional Materials Correlation to the 2009 Algebra I Standards of Learning and Curriculum Framework
Provider York County School Division Course Syllabus URL http://yorkcountyschools.org/virtuallearning/coursecatalog.aspx Course Title Algebra I AB Last Updated 2010 - A.1 The student will represent verbal
Introduction to Machine Learning and Data Mining. Prof. Dr. Igor Trajkovski trajkovski@nyus.edu.mk
Introduction to Machine Learning and Data Mining Prof. Dr. Igor Trakovski trakovski@nyus.edu.mk Neural Networks 2 Neural Networks Analogy to biological neural systems, the most robust learning systems
Bounded Treewidth in Knowledge Representation and Reasoning 1
Bounded Treewidth in Knowledge Representation and Reasoning 1 Reinhard Pichler Institut für Informationssysteme Arbeitsbereich DBAI Technische Universität Wien Luminy, October 2010 1 Joint work with G.
Using Business Intelligence to Mitigate Graduation Delay Issues
Using Business Intelligence to Mitigate Graduation Delay Issues Khaled Almgren PhD Candidate Department of Computer science and Engineering University of Bridgeport Abstract Graduate master students usually
Proximal mapping via network optimization
L. Vandenberghe EE236C (Spring 23-4) Proximal mapping via network optimization minimum cut and maximum flow problems parametric minimum cut problem application to proximal mapping Introduction this lecture:
Probabilistic Models for Big Data. Alex Davies and Roger Frigola University of Cambridge 13th February 2014
Probabilistic Models for Big Data Alex Davies and Roger Frigola University of Cambridge 13th February 2014 The State of Big Data Why probabilistic models for Big Data? 1. If you don t have to worry about
Chapter 7. Feature Selection. 7.1 Introduction
Chapter 7 Feature Selection Feature selection is not used in the system classification experiments, which will be discussed in Chapter 8 and 9. However, as an autonomous system, OMEGA includes feature
Lecture 3. Linear Programming. 3B1B Optimization Michaelmas 2015 A. Zisserman. Extreme solutions. Simplex method. Interior point method
Lecture 3 3B1B Optimization Michaelmas 2015 A. Zisserman Linear Programming Extreme solutions Simplex method Interior point method Integer programming and relaxation The Optimization Tree Linear Programming
SIMS 255 Foundations of Software Design. Complexity and NP-completeness
SIMS 255 Foundations of Software Design Complexity and NP-completeness Matt Welsh November 29, 2001 mdw@cs.berkeley.edu 1 Outline Complexity of algorithms Space and time complexity ``Big O'' notation Complexity
D-optimal plans in observational studies
D-optimal plans in observational studies Constanze Pumplün Stefan Rüping Katharina Morik Claus Weihs October 11, 2005 Abstract This paper investigates the use of Design of Experiments in observational
CAD Algorithms. P and NP
CAD Algorithms The Classes P and NP Mohammad Tehranipoor ECE Department 6 September 2010 1 P and NP P and NP are two families of problems. P is a class which contains all of the problems we solve using
Approximation Algorithms
Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NP-Completeness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms
ALGEBRA. sequence, term, nth term, consecutive, rule, relationship, generate, predict, continue increase, decrease finite, infinite
ALGEBRA Pupils should be taught to: Generate and describe sequences As outcomes, Year 7 pupils should, for example: Use, read and write, spelling correctly: sequence, term, nth term, consecutive, rule,
Statistical Machine Learning
Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes
Statistical Learning Theory Meets Big Data
Statistical Learning Theory Meets Big Data Randomized algorithms for frequent itemsets Eli Upfal Brown University Data, data, data In God we trust, all others (must) bring data Prof. W.E. Deming, Statistician,
Industrial Optimization
Industrial Optimization Lessons learned from Optimization in Practice Marco Lübbecke Chair of Operations Research RWTH Aachen University, Germany SICS Stockholm Feb 11, 2013 Discrete Optimization: Some
Applied Algorithm Design Lecture 5
Applied Algorithm Design Lecture 5 Pietro Michiardi Eurecom Pietro Michiardi (Eurecom) Applied Algorithm Design Lecture 5 1 / 86 Approximation Algorithms Pietro Michiardi (Eurecom) Applied Algorithm Design
Correlational Research. Correlational Research. Stephen E. Brock, Ph.D., NCSP EDS 250. Descriptive Research 1. Correlational Research: Scatter Plots
Correlational Research Stephen E. Brock, Ph.D., NCSP California State University, Sacramento 1 Correlational Research A quantitative methodology used to determine whether, and to what degree, a relationship
Regression III: Advanced Methods
Lecture 16: Generalized Additive Models Regression III: Advanced Methods Bill Jacoby Michigan State University http://polisci.msu.edu/jacoby/icpsr/regress3 Goals of the Lecture Introduce Additive Models
Sequences. A sequence is a list of numbers, or a pattern, which obeys a rule.
Sequences A sequence is a list of numbers, or a pattern, which obeys a rule. Each number in a sequence is called a term. ie the fourth term of the sequence 2, 4, 6, 8, 10, 12... is 8, because it is the
Empirical Hardness for Mixed Auctions
Empirical Hardness for Mixed Auctions Pablo Almajano 1,Jesús Cerquides 2, and Juan A. Rodriguez-Aguilar 1 1 IIIA, Artificial Intelligence Research Institute CSIC, Spanish National Research Council {palmajano,jar}@iiia.csic.es
Borne inférieure de circuit : une application des expanders
Borne inférieure de circuit : une application des expanders Simone Bova 1 Florent Capelli 2 Friedrich Slivovski 1 Stefan Mengel 3 1 TU Wien, 2 IMJ-PRG, Paris 7, 3 CRIL, Lens 6 Novembre 2015 JGA 2015 Motivation
Page 1. CSCE 310J Data Structures & Algorithms. CSCE 310J Data Structures & Algorithms. P, NP, and NP-Complete. Polynomial-Time Algorithms
CSCE 310J Data Structures & Algorithms P, NP, and NP-Complete Dr. Steve Goddard goddard@cse.unl.edu CSCE 310J Data Structures & Algorithms Giving credit where credit is due:» Most of the lecture notes
Cross Validation. Dr. Thomas Jensen Expedia.com
Cross Validation Dr. Thomas Jensen Expedia.com About Me PhD from ETH Used to be a statistician at Link, now Senior Business Analyst at Expedia Manage a database with 720,000 Hotels that are not on contract
Federated Optimization: Distributed Optimization Beyond the Datacenter
Federated Optimization: Distributed Optimization Beyond the Datacenter Jakub Konečný School of Mathematics University of Edinburgh J.Konecny@sms.ed.ac.uk H. Brendan McMahan Google, Inc. Seattle, WA 98103
Optimization applications in finance, securities, banking and insurance
IBM Software IBM ILOG Optimization and Analytical Decision Support Solutions White Paper Optimization applications in finance, securities, banking and insurance 2 Optimization applications in finance,
Online Adwords Allocation
Online Adwords Allocation Shoshana Neuburger May 6, 2009 1 Overview Many search engines auction the advertising space alongside search results. When Google interviewed Amin Saberi in 2004, their advertisement
Big Data - Lecture 1 Optimization reminders
Big Data - Lecture 1 Optimization reminders S. Gadat Toulouse, Octobre 2014 Big Data - Lecture 1 Optimization reminders S. Gadat Toulouse, Octobre 2014 Schedule Introduction Major issues Examples Mathematics
CFA Examination PORTFOLIO MANAGEMENT Page 1 of 6
PORTFOLIO MANAGEMENT A. INTRODUCTION RETURN AS A RANDOM VARIABLE E(R) = the return around which the probability distribution is centered: the expected value or mean of the probability distribution of possible
Solving NP Hard problems in practice lessons from Computer Vision and Computational Biology
Solving NP Hard problems in practice lessons from Computer Vision and Computational Biology Yair Weiss School of Computer Science and Engineering The Hebrew University of Jerusalem www.cs.huji.ac.il/ yweiss
A Simple Introduction to Support Vector Machines
A Simple Introduction to Support Vector Machines Martin Law Lecture for CSE 802 Department of Computer Science and Engineering Michigan State University Outline A brief history of SVM Large-margin linear
Data Mining Part 5. Prediction
Data Mining Part 5. Prediction 5.1 Spring 2010 Instructor: Dr. Masoud Yaghini Outline Classification vs. Numeric Prediction Prediction Process Data Preparation Comparing Prediction Methods References Classification
Music Mood Classification
Music Mood Classification CS 229 Project Report Jose Padial Ashish Goel Introduction The aim of the project was to develop a music mood classifier. There are many categories of mood into which songs may
Big Data Analytics. Lucas Rego Drumond
Big Data Analytics Lucas Rego Drumond Information Systems and Machine Learning Lab (ISMLL) Institute of Computer Science University of Hildesheim, Germany Going For Large Scale Going For Large Scale 1
On the Relationship between Classes P and NP
Journal of Computer Science 8 (7): 1036-1040, 2012 ISSN 1549-3636 2012 Science Publications On the Relationship between Classes P and NP Anatoly D. Plotnikov Department of Computer Systems and Networks,
Equilibrium computation: Part 1
Equilibrium computation: Part 1 Nicola Gatti 1 Troels Bjerre Sorensen 2 1 Politecnico di Milano, Italy 2 Duke University, USA Nicola Gatti and Troels Bjerre Sørensen ( Politecnico di Milano, Italy, Equilibrium
Chapter 6: Sensitivity Analysis
Chapter 6: Sensitivity Analysis Suppose that you have just completed a linear programming solution which will have a major impact on your company, such as determining how much to increase the overall production
1.204 Lecture 10. Greedy algorithms: Job scheduling. Greedy method
1.204 Lecture 10 Greedy algorithms: Knapsack (capital budgeting) Job scheduling Greedy method Local improvement method Does not look at problem globally Takes best immediate step to find a solution Useful
Risk pricing for Australian Motor Insurance
Risk pricing for Australian Motor Insurance Dr Richard Brookes November 2012 Contents 1. Background Scope How many models? 2. Approach Data Variable filtering GLM Interactions Credibility overlay 3. Model
5.1 Bipartite Matching
CS787: Advanced Algorithms Lecture 5: Applications of Network Flow In the last lecture, we looked at the problem of finding the maximum flow in a graph, and how it can be efficiently solved using the Ford-Fulkerson
Linear Classification. Volker Tresp Summer 2015
Linear Classification Volker Tresp Summer 2015 1 Classification Classification is the central task of pattern recognition Sensors supply information about an object: to which class do the object belong
Algebra Unpacked Content For the new Common Core standards that will be effective in all North Carolina schools in the 2012-13 school year.
This document is designed to help North Carolina educators teach the Common Core (Standard Course of Study). NCDPI staff are continually updating and improving these tools to better serve teachers. Algebra
Factor Analysis. Chapter 420. Introduction
Chapter 420 Introduction (FA) is an exploratory technique applied to a set of observed variables that seeks to find underlying factors (subsets of variables) from which the observed variables were generated.
11. APPROXIMATION ALGORITHMS
11. APPROXIMATION ALGORITHMS load balancing center selection pricing method: vertex cover LP rounding: vertex cover generalized load balancing knapsack problem Lecture slides by Kevin Wayne Copyright 2005
CHAPTER 9. Integer Programming
CHAPTER 9 Integer Programming An integer linear program (ILP) is, by definition, a linear program with the additional constraint that all variables take integer values: (9.1) max c T x s t Ax b and x integral
Supervised Feature Selection & Unsupervised Dimensionality Reduction
Supervised Feature Selection & Unsupervised Dimensionality Reduction Feature Subset Selection Supervised: class labels are given Select a subset of the problem features Why? Redundant features much or
Multimedia Databases. Wolf-Tilo Balke Philipp Wille Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs.
Multimedia Databases Wolf-Tilo Balke Philipp Wille Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs.de 14 Previous Lecture 13 Indexes for Multimedia Data 13.1
Global Multiprocessor Real-Time Scheduling as a Constraint Satisfaction Problem
Global Multiprocessor Real-Time Scheduling as a Constraint Satisfaction Problem Liliana Cucu-Grosean & Olivier Buffet INRIA Nancy Grand-Est 615 rue du Jardin Botanique 54600 Villers-lès-Nancy, France firstname.lastname@loria.fr
Towards running complex models on big data
Towards running complex models on big data Working with all the genomes in the world without changing the model (too much) Daniel Lawson Heilbronn Institute, University of Bristol 2013 1 / 17 Motivation
Classification - Examples
Lecture 2 Scheduling 1 Classification - Examples 1 r j C max given: n jobs with processing times p 1,...,p n and release dates r 1,...,r n jobs have to be scheduled without preemption on one machine taking
Why do statisticians "hate" us?
Why do statisticians "hate" us? David Hand, Heikki Mannila, Padhraic Smyth "Data mining is the analysis of (often large) observational data sets to find unsuspected relationships and to summarize the data
BEHAVIOR BASED CREDIT CARD FRAUD DETECTION USING SUPPORT VECTOR MACHINES
BEHAVIOR BASED CREDIT CARD FRAUD DETECTION USING SUPPORT VECTOR MACHINES 123 CHAPTER 7 BEHAVIOR BASED CREDIT CARD FRAUD DETECTION USING SUPPORT VECTOR MACHINES 7.1 Introduction Even though using SVM presents
Social Media Mining. Data Mining Essentials
Introduction Data production rate has been increased dramatically (Big Data) and we are able store much more data than before E.g., purchase data, social media data, mobile phone data Businesses and customers
A Constraint-Based Method for Project Scheduling with Time Windows
A Constraint-Based Method for Project Scheduling with Time Windows Amedeo Cesta 1 and Angelo Oddi 1 and Stephen F. Smith 2 1 ISTC-CNR, National Research Council of Italy Viale Marx 15, I-00137 Rome, Italy,
Simplex method summary
Simplex method summary Problem: optimize a linear objective, subject to linear constraints 1. Step 1: Convert to standard form: variables on right-hand side, positive constant on left slack variables for
Arithmetic Coding: Introduction
Data Compression Arithmetic coding Arithmetic Coding: Introduction Allows using fractional parts of bits!! Used in PPM, JPEG/MPEG (as option), Bzip More time costly than Huffman, but integer implementation
Notes on Factoring. MA 206 Kurt Bryan
The General Approach Notes on Factoring MA 26 Kurt Bryan Suppose I hand you n, a 2 digit integer and tell you that n is composite, with smallest prime factor around 5 digits. Finding a nontrivial factor
Improving SAT-Based Weighted MaxSAT Solvers CP 2012 (Quebec)
CP 2012 (Quebec) Carlos Ansótegui 1 María Luisa Bonet 2 Joel Gabàs 1 Jordi Levy 3 Universitat de Lleida (DIEI, UdL) 1. Universitat Politècnica de Catalunya (LSI, UPC) 2. Artificial Intelligence Research