Lecture 1 Introduction to Parallel Programming

Transcription

1 Lecture 1 Introduction to Parallel Programming EN /420 Instructor: Randal Burns 4 September 2008 Department of Computer Science, Johns Hopkins University

2

3 Pipelined Processor From

4 Multi-Core Super-scalar the wrong(ish) term Cray T3e Independent processing units on one chip

5 Why this class? Parallelism is everywhere! Every program/programmer needs to address it Traditional CS curriculum totally misses the point Who should take it? Anyone who needs to program ever. Period! Excalamation point!!

6 Whatʼs a Parallel Computer?

7 Whatʼs a Parallel Computer?

8 Whatʼs a Parallel Computer?

9 Whatʼs a Parallel Computer?

10 Whatʼs a Parallel Computer?

11 Whatʼs a Parallel Computer?

12 Whatʼs a Parallel Computer?

13 Whatʼs a Parallel Computer?

14 Concurrency in Operating Systems Informal perspective (1980s and 1990s) Threaded and multi-process programming to hide I/O network latencies To avoid stalls on Web servers, file servers For responsiveness of GUIs Important for scalar architectures (2000s) Parallel to exercise processors Pipeline/multi-core capabilites Multi-processors (game consoles and video cards) Adapt techniques from HPC and SMP communities Otherwise we throw the cycles away

15 Weʼve been duped! Mooreʼs law The number of transistors that can be inexpensively placed on an integrated circuit is increasing exponentially, doubling approximately every two years. The observation has held for half a century Itʼs a fallacy: pipelined multicore not the same N k flop cores is not the same as an Nxk flop processor But the chip vendors tell us it is So we (the programmers) must deal with parallelism

16 Weʼve been duped!

17 About the Class: Goals The few, the proud, the parallel programmers This is a systems class, not a PL class Focus on understanding concurrency Algorithmic thinking not algorithms Play with toys Languages (MPI, concurrent Java, Hadoop!, maybe OpenMP) Hardware (uh oh! Thereʼs a lot of you!)

18 About the Class: Caveats New course. Material will co-evolve. Maybe a little chaotic at times Our systems may not always work New hardware New software We learn as you learn Two parallel themes Follow the book (cover to cover), learn ParProg. idioms Learn about languages and technology (unstructured)

19 Course Structure Exams: 1 or 2 Programming assignments: 2-3 major (Java, MPI, Hadoop!) 1-4 minor Problem sets (I doubt it) If we need to support exams Quizzes: only if you make me

20 Discussion (you first)

21 Discussion (OK me) What are the hardware trends of note? GPU processor convergence What is cloud computing? How and why do Google!/IBM, Yahoo! And Amazon feel differently? Canʼt my compiler do it for me? Or why is MPI the assembly language of parallelism successful? This is why itʼs a systems class The Tim Sweeney (Epic Games) question Why do game consoles drive in one lane on an 8-lane superhighway?