GAMS Productivity - Performance - Reliability

Transcription

1 GAMS Productivity - Performance - Reliability Jan-H. Jagla, Lutz Westermann GAMS Software GmbH Annual Review Meeting CAPD, CMU Pittsburgh, PA, March 12 13, 2007

2 Agenda GAMS Productivity Performance Reliability 2

3 GAMS Roots: Research project World Bank 1976 Pioneer in Algebraic Modeling Systems used for economic modeling Went commercial in 1987 Offices in Washington, D.C and Cologne Professional software tool provider, not a consulting company Operating in a segmented niche market Broad academic & commercial user base and network 3

4 Typical Application Annual Review Meeting 2007 Areas * Agricultural Economics Chemical Engineering Econometrics Environmental Economics Finance International Trade Macro Economics Management Science/OR Micro Economics Applied General Equilibrium Economic Development Energy Engineering Forestry Logistics Military Mathematics Physics * Illustrative examples in the GAMS Model Library 4

5 GAMS at a Glance General Algebraic Modeling System: Algebraic Modeling Language, Integrated Solver, Model Libraries, Connectivity- & Productivity Tools Design Principles: Balanced mix of declarative and procedural elements Open architecture and interfaces to other systems Different layers with separation of: model and data model and solution methods model and operating system model and interface 5

6 Multiple Solvers & Annual Review Meeting 2007 Platforms 6

7 Downloads Downloads to aix 13 axu 47 dar 6 hp7 46 lex 100 lnx 4 sgi 17 sol 321 wex 1400 win ~ 500 downloads a week Total:

8 System Overview Interactive API / Batch Connectivity Tools Uniform Data Exchange: ASCII GDX (ODBC, SQL, XLS, XML) GDX Tools Data API Ext. programs EXCEL MATLAB GNUPLOT,... C, Delphi,... User Interfaces GAMS Language Compiler and Execution System Solvers LP-MIP-QCP-MIQCP-NLP-MINLP-CNS- MCP-MPEC MPSGE, global, and stochastic optimization Productivity Tools Integrated Development Environment Model Debugger and Profiler Model Libraries Data Browser Charting Engine Benchmarking Deployment System Quality Assurance and Testing BARON, COIN, CONOPT, CPLEX, DECIS, DICOPT, KNITRO, LGO,MINOS, MOSEK, OQNLP, PATH, SNOPT, XA, XPRESS, 8

9 Who are the GAMS Annual Review Meeting 2007 Users? (Academic) Researcher One time use (Research Paper) Model Expert Model Results used for Consulting Black Box User Model integrated in (Optimization) Application Each Category has its own needs 9

10 What are their Risks? Relevant new features for productivity increase Performance gains Long term reliability User s Risks 10

11 Agenda GAMS Productivity Performance Reliability 11

12 Integrated Development Annual Review Meeting 2007 Environment 12

13 Solver Convert DEMO 13

14 Integrated Development Annual Review Meeting 2007 Environment DEMO 14

15 Gams Data exchange Binary Data Exchange GDX Tools Application GDX GAMS SOLVER GDX API IDE GDX Viewer GDXRank - Complements ASCII data exchange - Data exchange at any stage - Supports ODBC and XML - Direct Excel connectivity - General API - Visual Inspection - Scenario Management Support - Full Support of Batch Runs GAMS GDXxrw (MS Office) GDX GDXDiff GDXMerge GDXDump 15

16 GAMS in Control External Database Import Text Files Direct Interface (Office Appl.) GAMS Text Files Direct Interface (Office Appl.) Visualization Tools Export 16

17 U.S. Energy Model Goal: -Single annual model -Unique detail level -Long term energy outlook Coal Energy Emissions Size: -MIP, 7.5 million var., 3.5 million con. -Reduction of almost 80 %, while optimal solution changed by only 0.5 % 17

18 How Clients want to use Annual Review Meeting 2007 our interfaces Energy Model 18

19 GAMS in Control DEMO 19

20 Application in Control External Application Programming Language or other Application Call GAMS GAMS Call external program (including GAMS) External Application Application 20

21 Samurai Sudoku DEMO 21

22 Application in Control DEMO 22

23 Interfacing with GIS Annual Review Meeting 2007 Applications Increase in Ktons Per Year Less Than

24 Interfacing with Annual Review Meeting 2007 MATLAB 24

25 Interfacing with Annual Review Meeting 2007 GNUPLOT 25

26 Interfacing with Annual Review Meeting 2007 Web Applications 26

27 Individual Front Ends SAT-PROPHET 27

28 Deployment DEMO 28

29 Deployment DEMO 29

30 Agenda GAMS Productivity Performance Reliability 30

31 Performance Analysis Solver robustness and correctness Solver efficiency Quality of solution (nonconvex and discrete models) Cross comparison of solver resource times 31

32 GAMS World Cone World Global World MINLP World MPEC World MPSGE World Performance World PAVER Translation World Convert 32

33 PAVER Performance Analysis and Visualization for Effortless Reproducibility Online server to facilitate performance testing and analysis/visualization Results sent via in HTML format System independent 33

34 Open Testing Annual Review Meeting 2007 Architecture GAMS Models I. Models Translate: GAMS/Convert Solve with other systems Solve with GAMS II. Data Collection Web Web PAVER Server III. Analysis & Visualization 34

35 Analysis / Visualization DEMO 35

36 Analysis / Visualization DEMO 36

37 Agenda GAMS Productivity Performance Grid Computing Reliability 37

38 Grid Computing Imagine.. you have to solve s of independent scenarios.. and you can do this very rapidly for little additional money.. without having to do lots of cumbersome programming work... Grid Computing 38

39 What is Grid Annual Review Meeting 2007 Computing? A pool of connected computers managed and available as a common computing resource Effective sharing of CPU power Massive parallel task execution Scheduler handles management tasks E.g. Condor, Sun Grid Engine, Globus Can be rented or owned in common Licensing & security issues 39

40 Simple Serial Solve Loop Loop(p(pp), ret.fx = rmin +(rmax-rmin) /(card(pp)+1)*ord(pp) ; Solve minvar min var using miqcp; xres(i,p) = x.l(i); report(p,i,'inc') = xi.l(i); report(p,i,'dec') = xd.l(i) ); Loop Generation Solution Update How do we get to parallel and distributed computing? 40

41 GRID Specific Annual Review Meeting 2007 Enhancements 1. Submission of jobs 2. Grid Middleware Distribution of jobs Job execution 3. Collection of solutions 4. Processing of results 41

42 Grid Facilities DEMO 42

43 GAMS & Annual Review Meeting 2007 Grid Computing Scalable: support of massive grids, but also multi-cpu / multiple cores desktop machines 1 CPU - Grid Platform independent Only minor changes to model required Separation of model and solution method Model stays maintainable 43

44 Agenda GAMS Productivity Performance Branch & Cut & Heuristic Facility Reliability 44

45 Example Oil Pipeline Design Problem J. Brimberg, P. Hansen, K.-W. Lih, N. Mladenovic, M. Breton An Oil Pipeline Design Problem. Operations Research, Vol 51, No

46 Convergence Annual Review Meeting 2007 Pipeline Design 46

47 BCH Facility Cut Generator and Heuristic Represented in terms of original GAMS problem formulation Independent of the specific MIP solver Use any other model type and solver available in GAMS Design Principle: GAMS Solver Link MIP Solver (e.g. CPLEX) GAMS System BCH Facility User Cut Generator & Heuristics 47

48 BCH Facility DEMO 48

49 Agenda GAMS Productivity Performance Reliability 49

50 Long Term Annual Review Meeting 2007 Commitment Backward compatibility Performance comparison tools: Bench / Paver Model converter and encryption tool: Convert Solution verification tool: Examiner Software Quality Assurance (SQA) Software configuration management Quality control and tests of the product Client model testing SQA becomes more and more important for MP industry 50

51 SQA at GAMS Annual Review Meeting 2007 SCM Software configuration management Audit strings KNITRO Nov 27, 2006 WIN.KN.NA VIS KNITRO 5.0 KNITRO ALFA 5Mar07 WIN.KN.NA VIS KNITRO Build automation tools: Automatic build of the whole system every week - build early and build often Simple source management system (automatic version and audit string checks) 51

52 SQA at GAMS Annual Review Meeting 2007 Testing Quality control and tests of the product Goal: Continuous quality improvement using automated and reproducible tests Test libraries (available online): GAMS Model Library (solver tests) GAMS Quality Test Models Library (modeling system test) Continuous addition of new test models Bugtracking System 52

53 SQA at GAMS Annual Review Meeting 2007 Client Model Testing Clients with complex application Need assurance that application will work with new GAMS releases Only limited ressources to do major testing themselves Confidentiality issues 53

54 SQA at GAMS Annual Review Meeting 2007 Client Model Testing Includes: Ability to solve (= no bugs) Returns the same solution back Similar or better performance Improves communication between development team and clients Benefits both for solver developers, for clients and for GAMS 54

55 Summary GAMS increases productivity Continuous gain in performance Proven reliability (+30) Opens doors to a large network of GAMS developers and clients with modeling needs 55

56 Thank you! Performance of bchoil example: Cplex/BCH: 17 seconds Regular Xpress: 124 seconds Regular Cplex: 311 seconds Regular Coincbc: > 10,000 seconds 56