Parallel Programming in C with MPI and OpenMP Michael J. Quinn
Chapter 1 Motivation and History
Outline
Motivation
Modern scientific method
Evolution of supercomputing
Modern parallel computers
Seeking concurrency
Data clustering case study
Programming parallel computers
Why Faster Computers?
Solve compute-intensive problems faster
Make infeasible problems feasible
Reduce design time
Solve larger problems in same amount of time
Improve answer’s precision
Reduce design time
Gain competitive advantage
Definitions
Parallel computing
Using parallel computer to solve single problems faster
Parallel computer
Multiple-processor system supporting parallel programming
Parallel programming
Programming in a language that supports concurrency explicitly
Why MPI?
MPI = “Message Passing Interface”
Standard specification for message-passing libraries
Libraries available on virtually all parallel computers
Free libraries also available for networks of workstations or commodity clusters
Why OpenMP?
OpenMP an application programming interface (API) for shared-memory systems
Supports higher performance parallel programming of symmetrical multiprocessors
Classical Science
Nature Observation Theory Physical
Experimentation
Modern Scientific Method
Nature Observation Theory Physical
Experimentation Numerical
Simulation
Evolution of Supercomputing
World War II
Hand-computed artillery tables
Need to speed computations
ENIAC
Cold War
Nuclear weapon design
Intelligence gathering
Code-breaking
Supercomputer
General-purpose computer
Solves individual problems at high speeds, compared with contemporary systems
Typically costs $10 million or more
Traditionally found in government labs
Commercial Supercomputing
Started in capital-intensive industries
Petroleum exploration
Automobile manufacturing
Other companies followed suit
Pharmaceutical design
Consumer products
50 Years of Speed Increases
ENIAC
350 flops Today
> 1 trillion flops
CPUs 1 Million Times Faster
Faster clock speeds
Greater system concurrency
Multiple functional units
Concurrent instruction execution
Speculative instruction execution
Systems 1 Billion Times Faster
Processors are 1 million times faster
Combine thousands of processors
Parallel computer
Multiple processors
Supports parallel programming
Parallel computing = Using a parallel computer to execute a program faster
Microprocessor Revolution
Micros Minis Mainframes Speed (log scale) Time Supercomputers
Modern Parallel Computers
Caltech’s Cosmic Cube (Seitz and Fox)
Commercial copy-cats
nCUBE Corporation
Intel’s Supercomputer Systems Division
Lots more
Thinking Machines Corporation
Copy-cat Strategy
Microprocessor
1% speed of supercomputer
0.1% cost of supercomputer
Parallel computer = 1000 microprocessors
10 x speed of supercomputer
Same cost as supercomputer
Comments