Parallel Computing - Lec 2
- 1. Parallel Computing Mohamed Zahran (aka Z) mzahran@cs.nyu.edu http://www.mzahran.com CSCI-UA.0480-003 Lecture 2: Parallel Hardware: Basics
- 2. Computer History Eckert and Mauchly • 1st working electronic computer (1946) • To reprogram it you need to re-arrange the cords • 18,000 Vacuum tubes • 1,800 instructions/sec • 3,000 ft3
- 3. Computer History Programming the ENIAC!
- 4. Computer History • Von Neumann presented his idea of stored program concept. • Maurice Wilkes built it. 1st stored program computer 650 instructions/sec 1,400 ft3http://www.cl.cam.ac.uk/UoCCL/misc/EDSAC99/ EDSAC 1 (1949)
- 5. Computer History • After the vacuum tubes, transistors were invented (1947) 2nd generations of computers • UNIVAC (UNIversal Automatic Computer) • Introduced in the 50s
- 6. Computer History • From transistors to integrated circuits(IC) 3rd generation of computers • One IC can host hundreds (then thousands, then millions) of transistors computers are getting smaller
- 7. Intel 4004 Die Photo • Introduced in 1970 – First microprocessor • 2,250 transistors • 12 mm2 • 108 KHz
- 8. Intel 8086 Die Scan • 29,000 transistors • 33 mm2 • 5 MHz • Introduced in 1979 – Basic architecture of the IA32 PC
- 9. Intel 80486 Die Scan • 1,200,000 transistors • 81 mm2 • 25 MHz • Introduced in 1989 – 1st pipelined implementation of IA32 – 1st processor with on-chip cache
- 10. Pentium Die Photo • 3,100,000 transistors • 296 mm2 • 60 MHz • Introduced in 1993 – 1st superscalar implementation of IA32
- 11. Pentium 4 • 55,000,000 transistors • 146 mm2 • 3 GHz • Introduced in 2000 http://www.chip-architect.com
- 12. Pentium 4 IBM Power 9 (24 cores) Core 2 Duo (2 cores) AMD RyZen(8 cores to 32 cores) Intel Core i9 (22 cores )
- 13. How did the hardware evolve like that? Let’s look at different waves (generations of architectures)
- 14. First Generation (1970s) Single Cycle Implementation
- 15. Copyright © 2010, Elsevier Inc. All rights Reserved The Von Neumann Architecture
- 16. Second Generation (1980s) DF CI E •Pipelinining: •the hardware divided into stages •temporal parallelism •Number of stages increases with each generation •Maximum CPI (Cycles Per Instruction) = 1 •Due to dependencies (i.e. an instruction must wait for the result of another instruction) Fetch ExecuteIssueDecode Commit
- 17. Some Enhancements Cache Memory Virtual Memory TLBMulti-level caches
- 18. Third Generation (1990s) DF CI E E E •ILP (Instruction Level Parallelism) •Spatial parallelism •Executing several instructions at the same time is called superscalar capability. •performance = instructions per cycle (IPC) •Speculative Execution (prediction of branch direction) is introduced to make the best use of superscalar capability This can make some instructions execute out-of-order!!
- 19. Fourth Generation (2000s) DF CI E E E DF CI E E E Simultaneous Multithreading (SMT) (aka Hyperthreading Technology)
- 20. Some definitions before we proceed
- 21. An operating system “process” • An instance of a computer program that is being executed. • Components of a process: – The executable machine language program – A block of memory – Descriptors of resources the OS has allocated to the process – Security information – Information about the state of the process Copyright © 2010, Elsevier Inc. All rights Reserved
- 22. Multitasking • Gives the illusion that a single processor system is running multiple programs simultaneously. • Each process takes turns running time slice • After its time is up, it waits until it has a turn again. Copyright © 2010, Elsevier Inc. All rights Reserved
- 23. Threading • Threads are contained within processes. • They allow programmers to divide their programs into (more or less) independent tasks. • The hope is that when one thread blocks because it is waiting on a resource, another thread will have work to do and can run. Copyright © 2010, Elsevier Inc. All rights Reserved
- 24. As you can see … We can have several processes, executed in a multitasking fashion, and each process can consist of several threads.
- 25. The Status-Quo • We moved from single core to multicore to manycore: – for technological reasons, as we saw last lecture. • Free lunch is over for software folks – The software will not become faster with every new generation of processors • Not enough experience in parallel programming – Parallel programs of old days were restricted to some elite applications -> very few programmers – Now we need parallel programs for many different applications
- 26. How Did These Advances Happen? Computer Architecture Software Community Process Technology Wishes • Performance • Restrictions • Restrictions • Capabilities Design
- 28. Conclusions • The hardware evolution, driven by Moore’s law, was geared toward two things: – Exploiting parallelism – Dealing with memory (latency, capacity) Powered by TCPDF (www.tcpdf.org)Powered by TCPDF (www.tcpdf.org)