Le langage OCaml et la programmation des GPU
|
|
|
- Timothy Carpenter
- 8 years ago
- Views:
Transcription
1 Le langage OCaml et la programmation des GPU GPU programming with OCaml Mathias Bourgoin - Emmanuel Chailloux - Jean-Luc Lamotte Le projet OpenGPU : un an plus tard Ecole Polytechnique - 8 juin 2011
2 Outline 1 GPGPU Programming 2 OCaml 3 Motivations 4 GPGPU programming with OCaml SPOC Overview A Little Example Tools MultiGPU 5 Benchmarks 6 Conclusion OCaml meets GPGPU Future Work M. Bourgoin - E. Chailloux - J-L. Lamotte (UPMC-LIP6) Le langage OCaml et la programmation des GPU 8 juin / 28
3 GPGPU Programming Two main frameworks Cuda OpenCL Different Languages To write kernels Assembly (ptx, il,... ) subsets of C/C++ To manage kernels C/C++/Objective-C Fortran Python Scala Java Scilab... M. Bourgoin - E. Chailloux - J-L. Lamotte (UPMC-LIP6) Le langage OCaml et la programmation des GPU 8 juin / 28
4 Who needs GPGPU Two kinds of programmers HPC / Scientific Known Hardware Heavy Optimisation Problem Driven General Purpose Unknown Hardware/Multiplatform Light Optimisation User Driven Main Difficulties From the managing program Memory transfers Multiple Devices management Many different kind of Devices From the kernel Highly parallel Different levels of parallism Different kinds of memory (global, local,... ) What for? Data parallelism Distributed Computation Hopefully High Speed-Ups M. Bourgoin - E. Chailloux - J-L. Lamotte (UPMC-LIP6) Le langage OCaml et la programmation des GPU 8 juin / 28
5 OCaml High-Level language Portable Statically Typed Type inference Multiparadigm (imperative, object, functionnal, modular) Compile to Bytecode/native Code Memory Manager (very efficient Garbage Collector) Interactive Toplevel (to learn, test and debug) Interoperability with C System : Windows - Unix (OS-X, Linux... ) Architecture : x86, x86-64, PowerPC, ARM... M. Bourgoin - E. Chailloux - J-L. Lamotte (UPMC-LIP6) Le langage OCaml et la programmation des GPU 8 juin / 28
6 OCaml - Usage Domains Education Research Industry Software Examples The Coq Proof Assistant LexiFi s Modeling Language for Finance FFTW Scade Suite The Caml Consortium: M. Bourgoin - E. Chailloux - J-L. Lamotte (UPMC-LIP6) Le langage OCaml et la programmation des GPU 8 juin / 28
7 Motivations OCaml and GPGPU complement each other GPGPU frameworks are Ocaml is Highly Parallel Mainly Sequential Architecture Sensitive Multi-platform/architecture Very Low-Level Very High-Level Idea Allow OCaml developers to use GPGPU with their favorite language. Use OCaml to develop high level abstractions for GPGPU. Make GPGPU programming safer and easier M. Bourgoin - E. Chailloux - J-L. Lamotte (UPMC-LIP6) Le langage OCaml et la programmation des GPU 8 juin / 28
8 Main Objectives Goals Allow complete use of Cuda/OpenCL frameworks with OCaml Abstract these two frameworks Abstract memory Abstract memory transfers Use OCaml type-checking to ensure kernels type safety Propose Abstractions for GPGPU programming Solution SPOC (Stream Programming OCaml) M. Bourgoin - E. Chailloux - J-L. Lamotte (UPMC-LIP6) Le langage OCaml et la programmation des GPU 8 juin / 28
9 SPOC: Abstracting frameworks Our choice Dynamic linking. The Cuda implementation uses the Cuda Driver API instead of the Runtime Library (lower level API, does not need the cudart library which is only provided with the Cuda SDK). Compilation doesn t need any specific hardware (no need of a Cuda/OpenCL compatible Device) or SDK. Allows development for multiple architectures from a single system; executables to use any OpenCL/Cuda Devices conjointly; distribution of a single executable for multiple architectures. M. Bourgoin - E. Chailloux - J-L. Lamotte (UPMC-LIP6) Le langage OCaml et la programmation des GPU 8 juin / 28
10 SPOC: Abstracting Transfers Automatic Transfers Vectors automatically move from CPU to Devices When a cpu function uses a vector, SPOC moves it to the CPU RAM When a kernel uses a vector, SPOC moves it to the Device Global Memory Unused vectors do not move SPOC allows users to explicitely force transfers OCaml memory manager Vectors are managed by the OCaml memory manager Automatic allocation when created The garbage collector automatically frees vectors (on the CPU or on Devices) Allocation failure during a transfer triggers a collection M. Bourgoin - E. Chailloux - J-L. Lamotte (UPMC-LIP6) Le langage OCaml et la programmation des GPU 8 juin / 28
11 A Little Example CPU RAM GPU0 RAM GPU1 RAM 1 open Spoc 2 let dev = Devices.init () 3 let n = let v1 = Vector.create Vector.float32 n 5 let v2 = Vector.create Vector.float32 n 6 let v3 = Vector.create Vector.float32 n 7 let k = vector add v3 v1 v2 n 8 let block = {blockx = 1024; blocky = 1; blockz = 1} 9 let grid = {gridx = (n )/1024; gridy = 1; gridz = 1} let main () = 12 random fill(v1); 13 random fill(v2); 14 Kernel.run dev.(0) (block,grid) k; 15 for i = 0 to Vector.length v3 do 16 Printf.printf res[%d] = %f; 17 i (Mem.get v3 i) 18 done; M. Bourgoin - E. Chailloux - J-L. Lamotte (UPMC-LIP6) Le langage OCaml et la programmation des GPU 8 juin / 28
12 A Little Example CPU RAM GPU0 RAM GPU1 RAM 1 open Spoc 2 let dev = Devices.init () 3 let n = let v1 = Vector.create Vector.float32 n 5 let v2 = Vector.create Vector.float32 n 6 let v3 = Vector.create Vector.float32 n let k = vector add v3 v1 v2 n 7 let block = {blockx = 1024; blocky = 1; blockz = 1} 8 let grid = {gridx = (n )/1024; gridy = 1; gridz = 1} 9 let main () = 10 random fill(v1); 11 random fill(v2); 12 Kernel.run dev.(0) (block,grid) k; 13 for i = 0 to Vector.length v3 do 14 Printf.printf res[%d] = %f; 15 i (Mem.get v3 i) 16 done; M. Bourgoin - E. Chailloux - J-L. Lamotte (UPMC-LIP6) Le langage OCaml et la programmation des GPU 8 juin / 28
13 A Little Example v1 v2 v3 CPU RAM GPU0 RAM GPU1 RAM 1 open Spoc 2 let dev = Devices.init () 3 let n = let v1 = Vector.create Vector.float32 n 5 let v2 = Vector.create Vector.float32 n 6 let v3 = Vector.create Vector.float32 n 7 let k = vector add v3 v1 v2 n 8 let block = {blockx = 1024; blocky = 1; blockz = 1} 9 let grid = {gridx = (n )/1024; gridy = 1; gridz = 1} 10 let main () = 11 random fill(v1); 12 random fill(v2); 13 Kernel.run dev.(0) (block,grid) k; 14 for i = 0 to Vector.length v3 do 15 Printf.printf res[%d] = %f; 16 i (Mem.get v3 i) 17 done; M. Bourgoin - E. Chailloux - J-L. Lamotte (UPMC-LIP6) Le langage OCaml et la programmation des GPU 8 juin / 28
14 A Little Example v1 v2 v3 CPU RAM GPU0 RAM GPU1 RAM 1 open Spoc 2 let dev = Devices.init () 3 let n = let v1 = Vector.create Vector.float32 n 5 let v2 = Vector.create Vector.float32 n 6 let v3 = Vector.create Vector.float32 n 7 let k = vector add v3 v1 v2 n 8 let block = {blockx = 1024; blocky = 1; blockz = 1} 9 let grid = {gridx = (n )/1024; gridy = 1; gridz = 1} 10 let main () = 11 random fill(v1); 12 random fill(v2); 13 Kernel.run dev.(0) (block,grid) k; 14 for i = 0 to Vector.length v3 do 15 Printf.printf res[%d] = %f; 16 i (Mem.get v3 i) 17 done; M. Bourgoin - E. Chailloux - J-L. Lamotte (UPMC-LIP6) Le langage OCaml et la programmation des GPU 8 juin / 28
15 A Little Example v1 v2 v3 CPU RAM GPU0 RAM GPU1 RAM 1 open Spoc 2 let dev = Devices.init () 3 let n = let v1 = Vector.create Vector.float32 n 5 let v2 = Vector.create Vector.float32 n 6 let v3 = Vector.create Vector.float32 n 7 let k = vector add v3 v1 v2 n 8 let block = {blockx = 1024; blocky = 1; blockz = 1} 9 let grid = {gridx = (n )/1024; gridy = 1; gridz = 1} 10 let main () = 11 random fill(v1); 12 random fill(v2); 13 Kernel.run dev.(0) (block,grid) k; 14 for i = 0 to Vector.length v3 do 15 Printf.printf res[%d] = %f; 16 i (Mem.get v3 i) 17 done; M. Bourgoin - E. Chailloux - J-L. Lamotte (UPMC-LIP6) Le langage OCaml et la programmation des GPU 8 juin / 28
16 A Little Example CPU RAM v1 v2 v3 GPU0 RAM GPU1 RAM 1 open Spoc 2 let dev = Devices.init () 3 let n = let v1 = Vector.create Vector.float32 n 5 let v2 = Vector.create Vector.float32 n 6 let v3 = Vector.create Vector.float32 n 7 let k = vector add v3 v1 v2 n 8 let block = {blockx = 1024; blocky = 1; blockz = 1} 9 let grid = {gridx = (n )/1024; gridy = 1; gridz = 1} 10 let main () = 11 random fill(v1); 12 random fill(v2); 13 Kernel.run dev.(0) (block,grid) k; 14 for i = 0 to Vector.length v3 do 15 Printf.printf res[%d] = %f; 16 i (Mem.get v3 i) 17 done; M. Bourgoin - E. Chailloux - J-L. Lamotte (UPMC-LIP6) Le langage OCaml et la programmation des GPU 8 juin / 28
17 A Little Example v3 CPU RAM v1 v2 GPU0 RAM GPU1 RAM 1 open Spoc 2 let dev = Devices.init () 3 let n = let v1 = Vector.create Vector.float32 n 5 let v2 = Vector.create Vector.float32 n 6 let v3 = Vector.create Vector.float32 n 7 let k = vector add v3 v1 v2 n 8 let block = {blockx = 1024; blocky = 1; blockz = 1} 9 let grid = {gridx = (n )/1024; gridy = 1; gridz = 1} 10 let main () = 11 random fill(v1); 12 random fill(v2); 13 Kernel.run dev.(0) (block,grid) k; 14 for i = 0 to Vector.length v3 do 15 Printf.printf res[%d] = %f; 16 i (Mem.get v3 i) 17 done; M. Bourgoin - E. Chailloux - J-L. Lamotte (UPMC-LIP6) Le langage OCaml et la programmation des GPU 8 juin / 28
18 Kernels Type-Safe Kernel Declaration kernel vector add : Vector.vfloat64 Vector.vfloat64 Vector.vfloat64 unit = source file kernel name Static arguments types checking (compilation time) Kernel.run compiles kernel from source (.ptx /.cl) kernel vec_add : Vector.vfloat64!> Vector.vfloat64!> Vector.vfloat64!> unit = kernels vec_add kernel call dev compilation/execution Kernel.run dev vec_add Cuda OpenCL global void vec_add (...){... } kernels.ptx kernel void vec_add (...){... } kernels.cl for i = 0 to Vector.length v3 do" printf res[%d] = %f; i (Mem.get v3 i) done; M. Bourgoin - E. Chailloux - J-L. Lamotte (UPMC-LIP6) Le langage OCaml et la programmation des GPU 8 juin / 28
19 Errors Type-Checking: Error at compile-time 1 kernel vector sum: Vector.float64 unit = my file kernel sum 2 let v = Vector.create Vector.float in 3 Kernel.run device (block, grid) vector sum v; Type-Checking : Correct 1 kernel vector sum: Vector.float64 unit = my file kernel sum 2 let v = Vector.create Vector.float in 3 Kernel.run device (block, grid) vector sum v; Exceptions SPOC raises OCaml exceptions when Kernel compilation/execution fails Not enough memory on devices M. Bourgoin - E. Chailloux - J-L. Lamotte (UPMC-LIP6) Le langage OCaml et la programmation des GPU 8 juin / 28
20 Tools Development tools : OCaml Interactive Toplevel M. Bourgoin - E. Chailloux - J-L. Lamotte (UPMC-LIP6) Le langage OCaml et la programmation des GPU 8 juin / 28
21 Tools Development tools IDE (OCaide plugin for Eclipse) Cublas (v1) Optimized vector iterators M. Bourgoin - E. Chailloux - J-L. Lamotte (UPMC-LIP6) Le langage OCaml et la programmation des GPU 8 juin / 28
22 MultiGPU? M. Bourgoin - E. Chailloux - J-L. Lamotte (UPMC-LIP6) Le langage OCaml et la programmation des GPU 8 juin / 28
23 Multi-GPU Devices/Frameworks SPOC allows to use any Device from both frameworks indifferently SPOC allows to use any Device from both frameworks conjointly Tested with Cuda used conjointly with OpenCL Tested with Tesla C2070 used conjointly with AMD 6970 Transfers Automatic Transfers from CPU to Device Automatic Transfers from Device to CPU Automatic Transfers from Device to Device M. Bourgoin - E. Chailloux - J-L. Lamotte (UPMC-LIP6) Le langage OCaml et la programmation des GPU 8 juin / 28
24 Benchmarks - 1 Spoc easily speeds OCaml programs up Mandelbrot Naive implementation Non optimized kernels Graphic display handled by CPU Mandelbrot OCaml C Spoc - Cuda Spoc - OpenCL Cuda + OpenCL Corei7 960 i7 960 NVIDIA C2070 AMD 6970 C Time 913s 741s 6.49s 9.70s 4.74s speedups x1 x1.2 x140.7 x94.2 x192.6 opencl kernel not vectorized M. Bourgoin - E. Chailloux - J-L. Lamotte (UPMC-LIP6) Le langage OCaml et la programmation des GPU 8 juin / 28
25 Benchmarks - 2 OCaml+Spoc actually useable as a kernel composition language Matrix Multiply SP 2 optimized kernels Nvidia Cublas sgemm AMD kernel from AMD OpenCL SDK Matrix Multiply SP Average Cuda OpenCL Cuda + OpenCL Sequential NVIDIA C2070 AMD 6970 NVIDIA C AMD 6970 C GFlops 330 GFlops 657 GFlops OCaml GFlops 322 GFlops 678 GFlops Overhead 25.5% 5.7% 2.4% -3.2% M. Bourgoin - E. Chailloux - J-L. Lamotte (UPMC-LIP6) Le langage OCaml et la programmation des GPU 8 juin / 28
26 Conclusion - SPOC OCaml meets GPGPU OCaml developers can now use GPGPU programming SPOC allows to easily develop efficient GPGPU applications Abstracted frameworks (Cuda/Opencl) Automatic transfers Kernel type safety Efficient memory manager Can also be used as a tool for non OCaml developers OCaml Toplevel allows to test kernels OCaml can be used to quickly express new algorithms Still possible to use C externals... M. Bourgoin - E. Chailloux - J-L. Lamotte (UPMC-LIP6) Le langage OCaml et la programmation des GPU 8 juin / 28
27 Conclusion - Future Work Real world use case: PROP 2DRMP : Dimensional R-matrix propagation (Computer Physics Communications) Simulates electron scattering from H-like atoms and ions at intermediate energies Multi-Architecture: MultiCore, GPGPU, Clusters, GPU Clusters Translate from Fortran + Cuda to OCaml+SPOC + Cuda/OpenCL Test on Bull GPU Cluster Objectives Use OCaml+Spoc to simplify parallelism and transfers Verify that OCaml and SPOC enhance development speed, safety and maintainability while keeping high performances M. Bourgoin - E. Chailloux - J-L. Lamotte (UPMC-LIP6) Le langage OCaml et la programmation des GPU 8 juin / 28
28 Thanks SPOC sources : Spoc is compatible with x86 64: Unix (Linux, Mac OS X), Windows For more information mathias.bourgoin@lip6.fr M. Bourgoin - E. Chailloux - J-L. Lamotte (UPMC-LIP6) Le langage OCaml et la programmation des GPU 8 juin / 28
Retour d expérience : portage d une application haute-performance vers un langage de haut niveau
Retour d expérience : portage d une application haute-performance vers un langage de haut niveau ComPAS/RenPar 2013 Mathias Bourgoin - Emmanuel Chailloux - Jean-Luc Lamotte 16 Janvier 2013 Our Goals Globally
HIGH PERFORMANCE CONSULTING COURSE OFFERINGS
Performance 1(6) HIGH PERFORMANCE CONSULTING COURSE OFFERINGS LEARN TO TAKE ADVANTAGE OF POWERFUL GPU BASED ACCELERATOR TECHNOLOGY TODAY 2006 2013 Nvidia GPUs Intel CPUs CONTENTS Acronyms and Terminology...
Introduction to GPU hardware and to CUDA
Introduction to GPU hardware and to CUDA Philip Blakely Laboratory for Scientific Computing, University of Cambridge Philip Blakely (LSC) GPU introduction 1 / 37 Course outline Introduction to GPU hardware
Introducing PgOpenCL A New PostgreSQL Procedural Language Unlocking the Power of the GPU! By Tim Child
Introducing A New PostgreSQL Procedural Language Unlocking the Power of the GPU! By Tim Child Bio Tim Child 35 years experience of software development Formerly VP Oracle Corporation VP BEA Systems Inc.
Applications to Computational Financial and GPU Computing. May 16th. Dr. Daniel Egloff +41 44 520 01 17 +41 79 430 03 61
F# Applications to Computational Financial and GPU Computing May 16th Dr. Daniel Egloff +41 44 520 01 17 +41 79 430 03 61 Today! Why care about F#? Just another fashion?! Three success stories! How Alea.cuBase
Programming models for heterogeneous computing. Manuel Ujaldón Nvidia CUDA Fellow and A/Prof. Computer Architecture Department University of Malaga
Programming models for heterogeneous computing Manuel Ujaldón Nvidia CUDA Fellow and A/Prof. Computer Architecture Department University of Malaga Talk outline [30 slides] 1. Introduction [5 slides] 2.
PARALLEL JAVASCRIPT. Norm Rubin (NVIDIA) Jin Wang (Georgia School of Technology)
PARALLEL JAVASCRIPT Norm Rubin (NVIDIA) Jin Wang (Georgia School of Technology) JAVASCRIPT Not connected with Java Scheme and self (dressed in c clothing) Lots of design errors (like automatic semicolon
Overview. Lecture 1: an introduction to CUDA. Hardware view. Hardware view. hardware view software view CUDA programming
Overview Lecture 1: an introduction to CUDA Mike Giles mike.giles@maths.ox.ac.uk hardware view software view Oxford University Mathematical Institute Oxford e-research Centre Lecture 1 p. 1 Lecture 1 p.
Introduction GPU Hardware GPU Computing Today GPU Computing Example Outlook Summary. GPU Computing. Numerical Simulation - from Models to Software
GPU Computing Numerical Simulation - from Models to Software Andreas Barthels JASS 2009, Course 2, St. Petersburg, Russia Prof. Dr. Sergey Y. Slavyanov St. Petersburg State University Prof. Dr. Thomas
HPC with Multicore and GPUs
HPC with Multicore and GPUs Stan Tomov Electrical Engineering and Computer Science Department University of Tennessee, Knoxville CS 594 Lecture Notes March 4, 2015 1/18 Outline! Introduction - Hardware
Last Class: OS and Computer Architecture. Last Class: OS and Computer Architecture
Last Class: OS and Computer Architecture System bus Network card CPU, memory, I/O devices, network card, system bus Lecture 3, page 1 Last Class: OS and Computer Architecture OS Service Protection Interrupts
Accelerating Simulation & Analysis with Hybrid GPU Parallelization and Cloud Computing
Accelerating Simulation & Analysis with Hybrid GPU Parallelization and Cloud Computing Innovation Intelligence Devin Jensen August 2012 Altair Knows HPC Altair is the only company that: makes HPC tools
Evaluation of CUDA Fortran for the CFD code Strukti
Evaluation of CUDA Fortran for the CFD code Strukti Practical term report from Stephan Soller High performance computing center Stuttgart 1 Stuttgart Media University 2 High performance computing center
ANDROID DEVELOPER TOOLS TRAINING GTC 2014. Sébastien Dominé, NVIDIA
ANDROID DEVELOPER TOOLS TRAINING GTC 2014 Sébastien Dominé, NVIDIA AGENDA NVIDIA Developer Tools Introduction Multi-core CPU tools Graphics Developer Tools Compute Developer Tools NVIDIA Developer Tools
Cross-Platform GP with Organic Vectory BV Project Services Consultancy Services Expertise Markets 3D Visualization Architecture/Design Computing Embedded Software GIS Finance George van Venrooij Organic
Part I Courses Syllabus
Part I Courses Syllabus This document provides detailed information about the basic courses of the MHPC first part activities. The list of courses is the following 1.1 Scientific Programming Environment
ST810 Advanced Computing
ST810 Advanced Computing Lecture 17: Parallel computing part I Eric B. Laber Hua Zhou Department of Statistics North Carolina State University Mar 13, 2013 Outline computing Hardware computing overview
Turbomachinery CFD on many-core platforms experiences and strategies
Turbomachinery CFD on many-core platforms experiences and strategies Graham Pullan Whittle Laboratory, Department of Engineering, University of Cambridge MUSAF Colloquium, CERFACS, Toulouse September 27-29
The High Performance Internet of Things: using GVirtuS for gluing cloud computing and ubiquitous connected devices
WS on Models, Algorithms and Methodologies for Hierarchical Parallelism in new HPC Systems The High Performance Internet of Things: using GVirtuS for gluing cloud computing and ubiquitous connected devices
GPGPU accelerated Computational Fluid Dynamics
t e c h n i s c h e u n i v e r s i t ä t b r a u n s c h w e i g Carl-Friedrich Gauß Faculty GPGPU accelerated Computational Fluid Dynamics 5th GACM Colloquium on Computational Mechanics Hamburg Institute
Retargeting PLAPACK to Clusters with Hardware Accelerators
Retargeting PLAPACK to Clusters with Hardware Accelerators Manuel Fogué 1 Francisco Igual 1 Enrique S. Quintana-Ortí 1 Robert van de Geijn 2 1 Departamento de Ingeniería y Ciencia de los Computadores.
Last Class: OS and Computer Architecture. Last Class: OS and Computer Architecture
Last Class: OS and Computer Architecture System bus Network card CPU, memory, I/O devices, network card, system bus Lecture 3, page 1 Last Class: OS and Computer Architecture OS Service Protection Interrupts
Introduction to GPU Programming Languages
CSC 391/691: GPU Programming Fall 2011 Introduction to GPU Programming Languages Copyright 2011 Samuel S. Cho http://www.umiacs.umd.edu/ research/gpu/facilities.html Maryland CPU/GPU Cluster Infrastructure
The Fastest Way to Parallel Programming for Multicore, Clusters, Supercomputers and the Cloud.
White Paper 021313-3 Page 1 : A Software Framework for Parallel Programming* The Fastest Way to Parallel Programming for Multicore, Clusters, Supercomputers and the Cloud. ABSTRACT Programming for Multicore,
Lecture 1 Introduction to Android
These slides are by Dr. Jaerock Kwon at. The original URL is http://kettering.jrkwon.com/sites/default/files/2011-2/ce-491/lecture/alecture-01.pdf so please use that instead of pointing to this local copy
ACCELERATING SELECT WHERE AND SELECT JOIN QUERIES ON A GPU
Computer Science 14 (2) 2013 http://dx.doi.org/10.7494/csci.2013.14.2.243 Marcin Pietroń Pawe l Russek Kazimierz Wiatr ACCELERATING SELECT WHERE AND SELECT JOIN QUERIES ON A GPU Abstract This paper presents
Embedded Systems: map to FPGA, GPU, CPU?
Embedded Systems: map to FPGA, GPU, CPU? Jos van Eijndhoven jos@vectorfabrics.com Bits&Chips Embedded systems Nov 7, 2013 # of transistors Moore s law versus Amdahl s law Computational Capacity Hardware
Equalizer. Parallel OpenGL Application Framework. Stefan Eilemann, Eyescale Software GmbH
Equalizer Parallel OpenGL Application Framework Stefan Eilemann, Eyescale Software GmbH Outline Overview High-Performance Visualization Equalizer Competitive Environment Equalizer Features Scalability
OpenACC 2.0 and the PGI Accelerator Compilers
OpenACC 2.0 and the PGI Accelerator Compilers Michael Wolfe The Portland Group michael.wolfe@pgroup.com This presentation discusses the additions made to the OpenACC API in Version 2.0. I will also present
High Performance Cloud: a MapReduce and GPGPU Based Hybrid Approach
High Performance Cloud: a MapReduce and GPGPU Based Hybrid Approach Beniamino Di Martino, Antonio Esposito and Andrea Barbato Department of Industrial and Information Engineering Second University of Naples
GPU Hardware and Programming Models. Jeremy Appleyard, September 2015
GPU Hardware and Programming Models Jeremy Appleyard, September 2015 A brief history of GPUs In this talk Hardware Overview Programming Models Ask questions at any point! 2 A Brief History of GPUs 3 Once
GPU Parallel Computing Architecture and CUDA Programming Model
GPU Parallel Computing Architecture and CUDA Programming Model John Nickolls Outline Why GPU Computing? GPU Computing Architecture Multithreading and Arrays Data Parallel Problem Decomposition Parallel
Case Study on Productivity and Performance of GPGPUs
Case Study on Productivity and Performance of GPGPUs Sandra Wienke wienke@rz.rwth-aachen.de ZKI Arbeitskreis Supercomputing April 2012 Rechen- und Kommunikationszentrum (RZ) RWTH GPU-Cluster 56 Nvidia
HPC Wales Skills Academy Course Catalogue 2015
HPC Wales Skills Academy Course Catalogue 2015 Overview The HPC Wales Skills Academy provides a variety of courses and workshops aimed at building skills in High Performance Computing (HPC). Our courses
Code generation under Control
Code generation under Control Rencontres sur la compilation / Saint Hippolyte Henri-Pierre Charles CEA Laboratoire LaSTRE / Grenoble 12 décembre 2011 Introduction Présentation Henri-Pierre Charles, two
GPU System Architecture. Alan Gray EPCC The University of Edinburgh
GPU System Architecture EPCC The University of Edinburgh Outline Why do we want/need accelerators such as GPUs? GPU-CPU comparison Architectural reasons for GPU performance advantages GPU accelerated systems
Release Notes for Open Grid Scheduler/Grid Engine. Version: Grid Engine 2011.11
Release Notes for Open Grid Scheduler/Grid Engine Version: Grid Engine 2011.11 New Features Berkeley DB Spooling Directory Can Be Located on NFS The Berkeley DB spooling framework has been enhanced such
PERFORMANCE ENHANCEMENTS IN TreeAge Pro 2014 R1.0
PERFORMANCE ENHANCEMENTS IN TreeAge Pro 2014 R1.0 15 th January 2014 Al Chrosny Director, Software Engineering TreeAge Software, Inc. achrosny@treeage.com Andrew Munzer Director, Training and Customer
Parallel Computing with MATLAB
Parallel Computing with MATLAB Scott Benway Senior Account Manager Jiro Doke, Ph.D. Senior Application Engineer 2013 The MathWorks, Inc. 1 Acceleration Strategies Applied in MATLAB Approach Options Best
Parallel Computing: Strategies and Implications. Dori Exterman CTO IncrediBuild.
Parallel Computing: Strategies and Implications Dori Exterman CTO IncrediBuild. In this session we will discuss Multi-threaded vs. Multi-Process Choosing between Multi-Core or Multi- Threaded development
Multi-core Programming System Overview
Multi-core Programming System Overview Based on slides from Intel Software College and Multi-Core Programming increasing performance through software multi-threading by Shameem Akhter and Jason Roberts,
Porting the Plasma Simulation PIConGPU to Heterogeneous Architectures with Alpaka
Porting the Plasma Simulation PIConGPU to Heterogeneous Architectures with Alpaka René Widera1, Erik Zenker1,2, Guido Juckeland1, Benjamin Worpitz1,2, Axel Huebl1,2, Andreas Knüpfer2, Wolfgang E. Nagel2,
CSE 373: Data Structure & Algorithms Lecture 25: Programming Languages. Nicki Dell Spring 2014
CSE 373: Data Structure & Algorithms Lecture 25: Programming Languages Nicki Dell Spring 2014 What is a Programming Language? A set of symbols and associated tools that translate (if necessary) collections
An Easier Way for Cross-Platform Data Acquisition Application Development
An Easier Way for Cross-Platform Data Acquisition Application Development For industrial automation and measurement system developers, software technology continues making rapid progress. Software engineers
Graphics Cards and Graphics Processing Units. Ben Johnstone Russ Martin November 15, 2011
Graphics Cards and Graphics Processing Units Ben Johnstone Russ Martin November 15, 2011 Contents Graphics Processing Units (GPUs) Graphics Pipeline Architectures 8800-GTX200 Fermi Cayman Performance Analysis
Performance Evaluation of NAS Parallel Benchmarks on Intel Xeon Phi
Performance Evaluation of NAS Parallel Benchmarks on Intel Xeon Phi ICPP 6 th International Workshop on Parallel Programming Models and Systems Software for High-End Computing October 1, 2013 Lyon, France
A general-purpose virtualization service for HPC on cloud computing: an application to GPUs
A general-purpose virtualization service for HPC on cloud computing: an application to GPUs R.Montella, G.Coviello, G.Giunta* G. Laccetti #, F. Isaila, J. Garcia Blas *Department of Applied Science University
COSCO 2015 Heterogeneous Computing Programming
COSCO 2015 Heterogeneous Computing Programming Michael Meyer, Shunsuke Ishikuro Supporters: Kazuaki Sasamoto, Ryunosuke Murakami July 24th, 2015 Heterogeneous Computing Programming 1. Overview 2. Methodology
Example of Standard API
16 Example of Standard API System Call Implementation Typically, a number associated with each system call System call interface maintains a table indexed according to these numbers The system call interface
HIGH PERFORMANCE BIG DATA ANALYTICS
HIGH PERFORMANCE BIG DATA ANALYTICS Kunle Olukotun Electrical Engineering and Computer Science Stanford University June 2, 2014 Explosion of Data Sources Sensors DoD is swimming in sensors and drowning
GPU Tools Sandra Wienke
Sandra Wienke Center for Computing and Communication, RWTH Aachen University MATSE HPC Battle 2012/13 Rechen- und Kommunikationszentrum (RZ) Agenda IDE Eclipse Debugging (CUDA) TotalView Profiling (CUDA
NVIDIA CUDA Software and GPU Parallel Computing Architecture. David B. Kirk, Chief Scientist
NVIDIA CUDA Software and GPU Parallel Computing Architecture David B. Kirk, Chief Scientist Outline Applications of GPU Computing CUDA Programming Model Overview Programming in CUDA The Basics How to Get
RWTH GPU Cluster. Sandra Wienke wienke@rz.rwth-aachen.de November 2012. Rechen- und Kommunikationszentrum (RZ) Fotos: Christian Iwainsky
RWTH GPU Cluster Fotos: Christian Iwainsky Sandra Wienke wienke@rz.rwth-aachen.de November 2012 Rechen- und Kommunikationszentrum (RZ) The RWTH GPU Cluster GPU Cluster: 57 Nvidia Quadro 6000 (Fermi) innovative
GPU Computing - CUDA
GPU Computing - CUDA A short overview of hardware and programing model Pierre Kestener 1 1 CEA Saclay, DSM, Maison de la Simulation Saclay, June 12, 2012 Atelier AO and GPU 1 / 37 Content Historical perspective
22S:295 Seminar in Applied Statistics High Performance Computing in Statistics
22S:295 Seminar in Applied Statistics High Performance Computing in Statistics Luke Tierney Department of Statistics & Actuarial Science University of Iowa August 30, 2007 Luke Tierney (U. of Iowa) HPC
U N C L A S S I F I E D
CUDA and Java GPU Computing in a Cross Platform Application Scot Halverson sah@lanl.gov LA-UR-13-20719 Slide 1 What s the goal? Run GPU code alongside Java code Take advantage of high parallelization Utilize
Introduction to GP-GPUs. Advanced Computer Architectures, Cristina Silvano, Politecnico di Milano 1
Introduction to GP-GPUs Advanced Computer Architectures, Cristina Silvano, Politecnico di Milano 1 GPU Architectures: How do we reach here? NVIDIA Fermi, 512 Processing Elements (PEs) 2 What Can It Do?
PAPI - PERFORMANCE API. ANDRÉ PEREIRA ampereira@di.uminho.pt
1 PAPI - PERFORMANCE API ANDRÉ PEREIRA ampereira@di.uminho.pt 2 Motivation Application and functions execution time is easy to measure time gprof valgrind (callgrind) It is enough to identify bottlenecks,
System Structures. Services Interface Structure
System Structures Services Interface Structure Operating system services (1) Operating system services (2) Functions that are helpful to the user User interface Command line interpreter Batch interface
Stream Processing on GPUs Using Distributed Multimedia Middleware
Stream Processing on GPUs Using Distributed Multimedia Middleware Michael Repplinger 1,2, and Philipp Slusallek 1,2 1 Computer Graphics Lab, Saarland University, Saarbrücken, Germany 2 German Research
CUDA programming on NVIDIA GPUs
p. 1/21 on NVIDIA GPUs Mike Giles mike.giles@maths.ox.ac.uk Oxford University Mathematical Institute Oxford-Man Institute for Quantitative Finance Oxford eresearch Centre p. 2/21 Overview hardware view
Multi-GPU Load Balancing for Simulation and Rendering
Multi- Load Balancing for Simulation and Rendering Yong Cao Computer Science Department, Virginia Tech, USA In-situ ualization and ual Analytics Instant visualization and interaction of computing tasks
Advanced compiler construction. General course information. Teacher & assistant. Course goals. Evaluation. Grading scheme. Michel Schinz 2007 03 16
Advanced compiler construction Michel Schinz 2007 03 16 General course information Teacher & assistant Course goals Teacher: Michel Schinz Michel.Schinz@epfl.ch Assistant: Iulian Dragos INR 321, 368 64
Next Generation GPU Architecture Code-named Fermi
Next Generation GPU Architecture Code-named Fermi The Soul of a Supercomputer in the Body of a GPU Why is NVIDIA at Super Computing? Graphics is a throughput problem paint every pixel within frame time
Rootbeer: Seamlessly using GPUs from Java
Rootbeer: Seamlessly using GPUs from Java Phil Pratt-Szeliga. Dr. Jim Fawcett. Dr. Roy Welch. Syracuse University. Rootbeer Overview and Motivation Rootbeer allows a developer to program a GPU in Java
OpenPOWER Outlook AXEL KOEHLER SR. SOLUTION ARCHITECT HPC
OpenPOWER Outlook AXEL KOEHLER SR. SOLUTION ARCHITECT HPC Driving industry innovation The goal of the OpenPOWER Foundation is to create an open ecosystem, using the POWER Architecture to share expertise,
High Performance Computing in CST STUDIO SUITE
High Performance Computing in CST STUDIO SUITE Felix Wolfheimer GPU Computing Performance Speedup 18 16 14 12 10 8 6 4 2 0 Promo offer for EUC participants: 25% discount for K40 cards Speedup of Solver
OpenACC Basics Directive-based GPGPU Programming
OpenACC Basics Directive-based GPGPU Programming Sandra Wienke, M.Sc. wienke@rz.rwth-aachen.de Center for Computing and Communication RWTH Aachen University Rechen- und Kommunikationszentrum (RZ) PPCES,
OPERATING SYSTEM SERVICES
OPERATING SYSTEM SERVICES USER INTERFACE Command line interface(cli):uses text commands and a method for entering them Batch interface(bi):commands and directives to control those commands are entered
LBM BASED FLOW SIMULATION USING GPU COMPUTING PROCESSOR
LBM BASED FLOW SIMULATION USING GPU COMPUTING PROCESSOR Frédéric Kuznik, frederic.kuznik@insa lyon.fr 1 Framework Introduction Hardware architecture CUDA overview Implementation details A simple case:
Benchmark Hadoop and Mars: MapReduce on cluster versus on GPU
Benchmark Hadoop and Mars: MapReduce on cluster versus on GPU Heshan Li, Shaopeng Wang The Johns Hopkins University 3400 N. Charles Street Baltimore, Maryland 21218 {heshanli, shaopeng}@cs.jhu.edu 1 Overview
Lecture 1: an introduction to CUDA
Lecture 1: an introduction to CUDA Mike Giles mike.giles@maths.ox.ac.uk Oxford University Mathematical Institute Oxford e-research Centre Lecture 1 p. 1 Overview hardware view software view CUDA programming
Accelerating CFD using OpenFOAM with GPUs
Accelerating CFD using OpenFOAM with GPUs Authors: Saeed Iqbal and Kevin Tubbs The OpenFOAM CFD Toolbox is a free, open source CFD software package produced by OpenCFD Ltd. Its user base represents a wide
Amazon EC2 Product Details Page 1 of 5
Amazon EC2 Product Details Page 1 of 5 Amazon EC2 Functionality Amazon EC2 presents a true virtual computing environment, allowing you to use web service interfaces to launch instances with a variety of
OpenCL for programming shared memory multicore CPUs
Akhtar Ali, Usman Dastgeer and Christoph Kessler. OpenCL on shared memory multicore CPUs. Proc. MULTIPROG-212 Workshop at HiPEAC-212, Paris, Jan. 212. OpenCL for programming shared memory multicore CPUs
10- High Performance Compu5ng
10- High Performance Compu5ng (Herramientas Computacionales Avanzadas para la Inves6gación Aplicada) Rafael Palacios, Fernando de Cuadra MRE Contents Implemen8ng computa8onal tools 1. High Performance
MapGraph. A High Level API for Fast Development of High Performance Graphic Analytics on GPUs. http://mapgraph.io
MapGraph A High Level API for Fast Development of High Performance Graphic Analytics on GPUs http://mapgraph.io Zhisong Fu, Michael Personick and Bryan Thompson SYSTAP, LLC Outline Motivations MapGraph
NVIDIA CUDA GETTING STARTED GUIDE FOR MAC OS X
NVIDIA CUDA GETTING STARTED GUIDE FOR MAC OS X DU-05348-001_v5.5 July 2013 Installation and Verification on Mac OS X TABLE OF CONTENTS Chapter 1. Introduction...1 1.1. System Requirements... 1 1.2. About
Crosswalk: build world class hybrid mobile apps
Crosswalk: build world class hybrid mobile apps Ningxin Hu Intel Today s Hybrid Mobile Apps Application HTML CSS JS Extensions WebView of Operating System (Tizen, Android, etc.,) 2 State of Art HTML5 performance
1 Bull, 2011 Bull Extreme Computing
1 Bull, 2011 Bull Extreme Computing Table of Contents HPC Overview. Cluster Overview. FLOPS. 2 Bull, 2011 Bull Extreme Computing HPC Overview Ares, Gerardo, HPC Team HPC concepts HPC: High Performance
NVIDIA CUDA GETTING STARTED GUIDE FOR MAC OS X
NVIDIA CUDA GETTING STARTED GUIDE FOR MAC OS X DU-05348-001_v6.5 August 2014 Installation and Verification on Mac OS X TABLE OF CONTENTS Chapter 1. Introduction...1 1.1. System Requirements... 1 1.2. About
5x in 5 hours Porting SEISMIC_CPML using the PGI Accelerator Model
5x in 5 hours Porting SEISMIC_CPML using the PGI Accelerator Model C99, C++, F2003 Compilers Optimizing Vectorizing Parallelizing Graphical parallel tools PGDBG debugger PGPROF profiler Intel, AMD, NVIDIA
GPUs for Scientific Computing
GPUs for Scientific Computing p. 1/16 GPUs for Scientific Computing Mike Giles mike.giles@maths.ox.ac.uk Oxford-Man Institute of Quantitative Finance Oxford University Mathematical Institute Oxford e-research
Android Development: Part One
Android Development: Part One This workshop will introduce you to the nature of the Android development platform. We begin with an overview of the platform s development history and some discussion of
Program Grid and HPC5+ workshop
Program Grid and HPC5+ workshop 24-30, Bahman 1391 Tuesday Wednesday 9.00-9.45 9.45-10.30 Break 11.00-11.45 11.45-12.30 Lunch 14.00-17.00 Workshop Rouhani Karimi MosalmanTabar Karimi G+MMT+K Opening IPM_Grid
Applying Parallel and Distributed Computing for Image Reconstruction in 3D Electrical Capacitance Tomography
AUTOMATYKA 2010 Tom 14 Zeszyt 3/2 Pawe³ Kapusta*, Micha³ Majchrowicz*, Robert Banasiak* Applying Parallel and Distributed Computing for Image Reconstruction in 3D Electrical Capacitance Tomography 1. Introduction
Reminders. Lab opens from today. Many students want to use the extra I/O pins on
Reminders Lab opens from today Wednesday 4:00-5:30pm, Friday 1:00-2:30pm Location: MK228 Each student checks out one sensor mote for your Lab 1 The TA will be there to help your lab work Many students
AMD WHITE PAPER GETTING STARTED WITH SEQUENCEL. AMD Embedded Solutions 1
AMD WHITE PAPER GETTING STARTED WITH SEQUENCEL AMD Embedded Solutions 1 Optimizing Parallel Processing Performance and Coding Efficiency with AMD APUs and Texas Multicore Technologies SequenceL Auto-parallelizing
Graphic Processing Units: a possible answer to High Performance Computing?
4th ABINIT Developer Workshop RESIDENCE L ESCANDILLE AUTRANS HPC & Graphic Processing Units: a possible answer to High Performance Computing? Luigi Genovese ESRF - Grenoble 26 March 2009 http://inac.cea.fr/l_sim/
High Efficiency Video Coding (HEVC) or H.265 is a next generation video coding standard developed by ITU-T (VCEG) and ISO/IEC (MPEG).
HEVC - Introduction High Efficiency Video Coding (HEVC) or H.265 is a next generation video coding standard developed by ITU-T (VCEG) and ISO/IEC (MPEG). HEVC / H.265 reduces bit-rate requirement by 50%
GPGPU Computing. Yong Cao
GPGPU Computing Yong Cao Why Graphics Card? It s powerful! A quiet trend Copyright 2009 by Yong Cao Why Graphics Card? It s powerful! Processor Processing Units FLOPs per Unit Clock Speed Processing Power
High Performance GPGPU Computer for Embedded Systems
High Performance GPGPU Computer for Embedded Systems Author: Dan Mor, Aitech Product Manager September 2015 Contents 1. Introduction... 3 2. Existing Challenges in Modern Embedded Systems... 3 2.1. Not
NVIDIA CUDA GETTING STARTED GUIDE FOR MICROSOFT WINDOWS
NVIDIA CUDA GETTING STARTED GUIDE FOR MICROSOFT WINDOWS DU-05349-001_v6.0 February 2014 Installation and Verification on TABLE OF CONTENTS Chapter 1. Introduction...1 1.1. System Requirements... 1 1.2.
On-Demand Supercomputing Multiplies the Possibilities
Microsoft Windows Compute Cluster Server 2003 Partner Solution Brief Image courtesy of Wolfram Research, Inc. On-Demand Supercomputing Multiplies the Possibilities Microsoft Windows Compute Cluster Server
Optimizing Application Performance with CUDA Profiling Tools
Optimizing Application Performance with CUDA Profiling Tools Why Profile? Application Code GPU Compute-Intensive Functions Rest of Sequential CPU Code CPU 100 s of cores 10,000 s of threads Great memory
Kernel Types System Calls. Operating Systems. Autumn 2013 CS4023
Operating Systems Autumn 2013 Outline 1 2 Types of 2.4, SGG The OS Kernel The kernel is the central component of an OS It has complete control over everything that occurs in the system Kernel overview
L20: GPU Architecture and Models
L20: GPU Architecture and Models scribe(s): Abdul Khalifa 20.1 Overview GPUs (Graphics Processing Units) are large parallel structure of processing cores capable of rendering graphics efficiently on displays.
Exploiting GPU Hardware Saturation for Fast Compiler Optimization
Exploiting GPU Hardware Saturation for Fast Compiler Optimization Alberto Magni School of Informatics University of Edinburgh United Kingdom a.magni@sms.ed.ac.uk Christophe Dubach School of Informatics
GPGPU Parallel Merge Sort Algorithm
GPGPU Parallel Merge Sort Algorithm Jim Kukunas and James Devine May 4, 2009 Abstract The increasingly high data throughput and computational power of today s Graphics Processing Units (GPUs), has led
What is a programming language?
Overview Introduction Motivation Why study programming languages? Some key concepts What is a programming language? Artificial language" Computers" Programs" Syntax" Semantics" What is a programming language?...there