Large-Scale Reservoir Simulation and Big Data Visualization

Transcription

1 Large-Scale Reservoir Simulation and Big Data Visualization Dr. Zhangxing John Chen NSERC/Alberta Innovates Energy Environment Solutions/Foundation CMG Chair Alberta Innovates Technology Future (icore) Chair Director, Foundation CMG/Frank-Sarah Meyer Collaboration Center, University of Calgary Slide 1

2 Outlines Motivations - Importance Simulation and Visualization Case Studies Summary Remarks Slide 2

3 Outlines Motivations - Importance Parallelization Means Case Studies Summary Remarks Slide 3

4 Motivations Accuracy Stability Robustness Speed capital savings Slide 4

5 Speed Issue 3D thermal models: with 500,000 cells geomodels and multiprocessor computers, runs take hours to weeks. For typical optimization run, 400-1,000 runs required. If done serially, hundreds to thousands of days for each optimization run required. With parallelization, optimization runtimes will drop. More geological descriptions can be added. Slide 5

6 Motivations Grid (scale) requirements Physics requirements Process requirements Slide 6

7 Grid Requirements Reservoir (rock and fluid) heterogeneity Thin moving thermal front Thin mobile solvent-rich layers ~ 1cm Discontinuity: presence of faults, fractures, and mud and shale layers Small dispersion/diffusion Local reaction (chemistry) zones Slide 7

8 Grid Requirements Rock heterogeneity Slide 8

9 Fluid heterogeneity Grid Requirements Slide 9

10 Grid Requirements Thin moving thermal and solvent fronts Slide 10

11 Grid Requirements Discontinuity: presence of faults, fractures, and mud and shale layers Slide 11

12 Grid Requirements Small dispersion/diffusion Slide 12

13 Grid Requirements Local reaction (chemistry) zones Slide 13

14 Physics Requirements Disparate data with different scales Thermal and solvent effects Mass and heat transfer Phase behavior Geomechanics Wellbore flow Slide 14

15 Process Requirements Thermal Recovery Processes - SAGD (steam assisted gravity drainage) - CSS (cyclic steam stimulation) - ISC (in situ combustion) Solvent Recovery Processes - VAPEX (vapor extraction) - SAP (solvent aided process) Slide 15

16 Business Competiveness Field management Equipment management Cost management SIMULATION TECHNOLOGY Strategy Safety Slide 16

17 Outlines Motivations - Importance Simulation and Visualization Case Studies Summary Remarks Slide 17

18 Petroleum Reservoir Simulators Black oil simulator (water flooding, fractured reservoirs) Compositional simulator (CO 2, N 2, CBM, tight and shale oil and gas) Thermal simulator (CSS, SAGD, ISC, THAI, Steam flooding) Chemical simulator (SAP+Foam) Wellbore module Geomechanical module Slide 18

19 Acceleration Means Software Acceleration Hardware Acceleration Slide 19

20 Software Acceleration Grid Management Discretization (Numerical) Methods System Solvers Parallelization (Matrices and Vectors) Software Design Slide 20

21 Grid Management The most critical modules - Type, choice of numerical methods - Partition, workload, communication - Data structure, info and data distribution - Numbering, bandwidth, and input Slide 21

22 Discretization Methods Finite difference methods Finite volume (control volume) methods Finite element methods Slide 22

23 System Solvers Preconditioners Linear Solvers A x = b Nonlinear Solvers Slide 23

24 Polynomial Approximate Inverse ILUT(p, tol) ILU(k) Preconditioners Domain Decomposition (Restricted Additive Schwarz) Algebraic Multigrid: classical and smoothed aggregation Slide 24

25 Linear Solvers GMRES, CG, BICGSTAB and GCR CGS, Orthomin and Orthodir Classical AMG Smoothed Aggregation AMG Slide 25

26 Nonlinear Solvers Newton Iterations Newton-Raphson Iterations Slide 26

27 Software Design OpenMP - Multi-core, easy to use, limited scalability MPI - communication - MPI-IO Programming languages - C, C++, Fortran, Slide 27

28 Hardware Acceleration CPUs (central processing units) GPUs (graph processing units) Slide 28

29 CPU VS GPU GPU (C2050) CPU (X5570) Cores Memory 144 GB/s 10 GB/s Float Performance 1030G (s) / 515G (d) ~20G GPU is around 10 times faster than CPU! Slide 29

30 Outlines Motivations Simulation and Visualization Case Studies Summary Remarks Slide 30

31 AMG: Example 1 Two-dimensional elliptic problem Grid size: 1,000x1,000, non-zeros: 4,996,000 Tol: 1e-6, level: 8 36 iterations GPU: 1.30s CPU: 12.26s Speedup: 9.33 Slide 31

32 AMG: Example 2 Two-dimensional elliptic problem Grid size: 1,500x1,500, non-zeros: 11,244,000 Tol: 1e-6, level: 8 31 iterations GPU: 2.58s CPU: 24.50s Speedup: 9.42 Slide 32

33 AMG: Example 3 Three-dimensional elliptic problem Grid size: 100x100x100, non-zeros: 6,940,000 Tol: 1e-6, level: 8 21 iterations GPU: 1.13s CPU: 10.80s Speedup: 9.58 Slide 33

34 AMG: Example 4 Three-dimensional elliptic problem Grid size: 130x130x130, non-zeros: 15,277,000 Tol: 1e-6, level: 8 25 iterations GPU: 3.02s CPU: 37.86s Speedup: Slide 34

35 Numerical Studies on CPUs Parallel (University of Calgary), Westgrid 528 standard nodes - 26-core Intel Xeon E5649 processors - 24 G memory InfiniBand 4X QDR, 40 Gbit/s Slide 35

36 15M Case: Example 5 GMRES + DDM Grid: 250x250x250 Unknowns: 15 million # processors Grid time (s) Overall time (s) Slide 36

37 125M Case: Example 6 GMRES + DDM Grid: 500x500x500 Unknowns: 125 million # processors Grid time (s) Overall time (s) Slide 37

38 200M Case: Example 7 GMRES + DDM Grid: 585x585x585 Unknowns: 200 million # processors Grid time (s) Overall time (s) Slide 38

39 Billion Case: Example 8 GMRES + DDM Grid size: billion (B) # processors Grid size 1 B 1B 2 B 3 B Grid time (s) Overall time (s) Slide 39

40 Stereo 3D and Immersive Visualization Immersive visualization has been shown to enable interactions that are more intuitive and that let users focus on analysis. Slide 40

41 Stereo 3D and Immersive Visualization Slide 41

42 Stereo 3D and Immersive Visualization Slide 42

43 Case Study T.Q.C. Dang, L.X. Nghiem, Z. Chen, and T.B.N. Nguyen, CO 2 low-salinity water alternating gas: A promising new approach for EOR, Journal of Petroleum Technology, January 2015, Slide 43

44 Case Study Geological Modeling Facies Modeling Clay Mapping Advanced Field Scale LSW Modeling Sensitivity Analysis History Matching Optimization Uncertainty Assessment Reservoir Simulation Flow, Ion Exchange, Geochemistry, Wettability Alteration CEC & Porosity Modeling Permeability Modeling Slide 44

45 Case Study Slide 45

46 Case Study Optimal Pattern Slide 46

47 Case Study Slide 47

48 Outlines Motivations Simulation and Visualization Case Studies Summary Remarks Slide 48

49 Summary Remarks Ultimate Goals Increasing reserves Reducing operating costs Enhancing petroleum recovery Capital savings Slide 49

50 Sponsors Slide 50