A numerically adaptive implementation of the simplex method

Transcription

1 A numerically adaptive implementation of the simplex method József Smidla, Péter Tar, István Maros Department of Computer Science and Systems Technology University of Pannonia 17th of December / 21 József Smidla, Péter Tar, István Maros A numerically adaptive implementation of the simplex method

2 Outline Introduction 1 Introduction Pannon Optimizer Numerical errors 2 Hilbert matrix Condition number Large condition number aware logic 3 Tests Solving the Hilbert matrix Primary large condition number detector 2 / 21 József Smidla, Péter Tar, István Maros A numerically adaptive implementation of the simplex method

3 Pannon Optimizer Introduction Pannon Optimizer Numerical errors Optimizer library: Linear algebraic kernel Auxiliary classes Primal and dual simplex implementation Alternative sub-algorithms Our simplex solver can solve the problems of the NETLIB Next steps: Improve the efficiency Develop a parallel simplex solver Implement more advanced algorithms Develop and test new solutions Solve excessively instable problems 3 / 21 József Smidla, Péter Tar, István Maros A numerically adaptive implementation of the simplex method

4 Linear algebraic kernel Pannon Optimizer Numerical errors Notations: Linear programming problem min c T x Ax = b x j 0, j = 1...n Linear algebraic kernel Provides linear algebraic algorithms and data structures: Vector operations (e.g. dot product) FTRAN: α = B 1 a BTRAN: π T = h T B 1 where B is the actual basis 4 / 21 József Smidla, Péter Tar, István Maros A numerically adaptive implementation of the simplex method

5 Numerical errors Introduction Pannon Optimizer Numerical errors Floating point numbers: ( 1) s 1.m 1 m 2...m n 2 e Errors s {0,1}: sign m i : i th bit of the mantissa e: exponent Rounding error: A» B, and B 0 A = A + B Cancellation: Given A and B 0, A -B C = A + B Expectation: C = 0 Error: C = ±ε These errors can create a lot of fake nonzeros, lead to wrong results and slow down the solution process. 5 / 21 József Smidla, Péter Tar, István Maros A numerically adaptive implementation of the simplex method

6 Relative errors Introduction Pannon Optimizer Numerical errors x and y are floating point numbers, x, y > 0 δ x : relative error of x Relative error of the summation: Relative error of the subtraction: δ x+y = max{δ x,δ y } δ x y = x x y δ x + y x y δ y, where x > y If x y < 1 the relative error is amplified! Relative errors can accumulate. 6 / 21 József Smidla, Péter Tar, István Maros A numerically adaptive implementation of the simplex method

7 Hilbert matrix Introduction Hilbert matrix Condition number Large condition number aware logic Hilbert matrix: H n,n, where h i,j = 1 Example: H 4,4 = i+j , i,j = 1,...,n We can construct the following LP problem: min 0 H n,n x = b x j 0,j = 1...n, and b i = n j=1 1 i + j 1 7 / 21 József Smidla, Péter Tar, István Maros A numerically adaptive implementation of the simplex method

8 Hilbert matrix Condition number Large condition number aware logic Open source solvers and the Hilbert matrix It is clear that if and only if x j = 1,j = 1...n, the solution is optimal We have tested CLP and GLPK Size GLPK Exact GLPK CLP 3 3 x j = 1± x j = 1 x j = x j = 1± x j = 1 x j = x j = 1± x j = 1 x j = x j = 1± x j = 1 INFEASIBLE 7 7 x j = 1±1.157 x j = 1 INFEASIBLE 8 8 x j = 1± x j = 1±0.201 INFEASIBLE x j = 1±6.298 x j = 1±4.124 INFEASIBLE x j x j = 1± INFEASIBLE We have used CLP ang GLPK as libraries, the models were generated and solved by C++ programs. 8 / 21 József Smidla, Péter Tar, István Maros A numerically adaptive implementation of the simplex method

9 Condition number Introduction Hilbert matrix Condition number Large condition number aware logic Measures, how much the output changes if the input changes ( ) κ(a) = A A, where A S 1 0 = V U T, 0 0 ( S 0 and the singular value decomposition of A = U 0 0 ) V T If κ(a) is large, computing A is difficult The condition number of the n*n Hilbert matrix is very large, it grows as ( (1+ ) 2) 4n O n κ(h 10,10 ) κ(h 100,100 ) / 21 József Smidla, Péter Tar, István Maros A numerically adaptive implementation of the simplex method

10 Exact solver Introduction Hilbert matrix Condition number Large condition number aware logic Idea: use an exact arithmetic solver For example: open source tools, like mpf_class or mpq_class Problem: too slow use adaptive methods! One hour on this planet is 7 years on Earth. Great, we ll wait here until the solver finishes. 10 / 21 József Smidla, Péter Tar, István Maros A numerically adaptive implementation of the simplex method

11 Compute the condition number Hilbert matrix Condition number Large condition number aware logic Computing the condition number is difficult Assume, there exists an efficient and reliable algorithm for computing the condition number The condition number of the following matrix is large, but the problem is stable: min 2n j=n+1 x j [I H] = b x j 0,j = 1...2n, and b i = 1 If the starting basis is the unit matrix, columns of H will never enter to the basis if the condition number of the matrix is large, it does not necessary causes troubles 11 / 21 József Smidla, Péter Tar, István Maros A numerically adaptive implementation of the simplex method

12 Control logic Introduction Hilbert matrix Condition number Large condition number aware logic Requirements: Recognize the instable bases Adjust the precision if necessary If the problem is stable, the overhead of monitoring the problem must be close to zero Solution: Return to lower precision arithmetic if the basis becomes stable Primary large condition number detector: heuristic, quick algorithm, indicates that the current basis is instable Secondary large condition number detector: slow, but exact sensitivity analysis The primary detector minimizes the frequency of running the secondary detector minimizes the overhead 12 / 21 József Smidla, Péter Tar, István Maros A numerically adaptive implementation of the simplex method

13 Hilbert matrix Condition number Large condition number aware logic Primary large condition number detector We can not compute the condition number directly However, we can detect the effect of the large condition number We propose: The input of the classic FTRAN is vector a: B 1 a = α Create the perturbed ā copy of a Use a modified FTRAN, which computes B 1 a = α and B 1 ā = ᾱ The modified FTRAN perturbs every sum during the computation of ᾱ If r = max{ α, ᾱ } min{ α, ᾱ } is greater than a threshold, it means that the condition number is too large primary alarm 13 / 21 József Smidla, Péter Tar, István Maros A numerically adaptive implementation of the simplex method

14 Large condition number aware logic Hilbert matrix Condition number Large condition number aware logic The primary detector is executed if an error occurs, for example: fallback to phase-1 If a primary alarm occurs, the algorithm performs primary detector in the following iterations If primary alarms occur in every next iteration and r does not decrease secondary alarm the algorithm terminates If a primary alarm occurs, the algorithm performs a sensitivity analysis If the sensitivity analysis finds that the result is extremely instable secondary alarm If a secondary alarm occurs, the software restarts from the last basis with modified parameters (enabled scaling, switching to LU decomposition, etc.) In the last resort: The software restarts from the last basis with enhanced precision arithmetic 14 / 21 József Smidla, Péter Tar, István Maros A numerically adaptive implementation of the simplex method

15 Tests Introduction Tests Solving the Hilbert matrix Primary large condition number detector The tests were performed on the following environment: CPU: Intel(R) Core(TM) i5-3210m 2.50GHz Memory: 8 GiB Operating system: Debian 7, 64 bit Tests with the following Open MP types: multiple precision floating point arithmetic: mpf_class 128 bit mpf_class 256 bit mpf_class 512 bit rational number arithmetic: mpq_class The output of the primary detector has been tested, too 15 / 21 József Smidla, Péter Tar, István Maros A numerically adaptive implementation of the simplex method

16 Solving the Hilbert matrix: mpf_class Tests Solving the Hilbert matrix Primary large condition number detector 16 / 21 József Smidla, Péter Tar, István Maros A numerically adaptive implementation of the simplex method

17 Solving large scale Hilbert matrix Tests Solving the Hilbert matrix Primary large condition number detector time unit: seconds mpq_class: rational number arithmetic is the most precise but the slowest Size mpf_class 128 mpf_class 256 mpf_class 512 mpq_class ? 17 / 21 József Smidla, Péter Tar, István Maros A numerically adaptive implementation of the simplex method

18 Solving the Hilbert matrix: solutions Tests Solving the Hilbert matrix Primary large condition number detector Size IEEE 754 mpf_class 128 bit 3 x j = 1± x j = 1± x j = 1± x j = 1± x j = 1± x j = 1± x j = 1± x j = 1± INFEASIBLE x j = 1± INFEASIBLE x j = 1± INFEASIBLE x j = 1± INFEASIBLE x j = 1± INFEASIBLE x j = 1± INFEASIBLE x j = 1± / 21 József Smidla, Péter Tar, István Maros A numerically adaptive implementation of the simplex method

19 Primary large condition number detector Tests Solving the Hilbert matrix Primary large condition number detector Output of the detector: r = max{ α, ᾱ } min{ α, ᾱ } δ = r 1 Problem 25FV47.MPS STOCFOR3.MPS PILOT.MPS MAROS-R7.MPS Value of δ after the last iteration e e e e-10 Hilbert 7* Hilbert 8* Hilbert 20* Hilbert 26* Hilbert 100* / 21 József Smidla, Péter Tar, István Maros A numerically adaptive implementation of the simplex method

20 Conclusions, next steps Tests Solving the Hilbert matrix Primary large condition number detector The primary and secondary detectors can minimize the overhead of the adaptivity Multiple precision arithmetic can be used for the excessively instable problems We have to integrate the enhanced precision arithmetic to the Pannon Optimizer We have to integrate the large condition number recognizer algorithm The large condition number recognizer can be accelerated with low-level optimization (SIMD architecture) Our goal: Implement a solver which runs fast on the stable problems, but recognizes the excessively instable problems Switches to more precise arithmetic, and solves these problems too 20 / 21 József Smidla, Péter Tar, István Maros A numerically adaptive implementation of the simplex method

21 Tests Solving the Hilbert matrix Primary large condition number detector Thank you for your attention! The research and publication has been supported by the European Union and Hungary and co-financed by the European Social Fund through the project TAMOP C-11/1/KONV National Research Center for Development and Market Introduction of Advanced Information and Communication Technologies. 21 / 21 József Smidla, Péter Tar, István Maros A numerically adaptive implementation of the simplex method