![[ Team LiB ]
• Table of Contents
Introduction to Parallel Computing, Second Edition
By Ananth Grama, Anshul Gupta, George Karypis, Vipin Kumar
Publisher: Addison Wesley
Pub Date: January 16, 2003
ISBN: 0-201-64865-2
Pages: 856
Increasingly, parallel processing is being seen as the only cost-effective method for the fast solution of computationally large and
data-intensive problems. The emergence of inexpensive parallel computers such as commodity desktop multiprocessors and clusters of
workstations or PCs has made such parallel methods generally applicable, as have software standards for portable parallel
programming. This sets the stage for substantial growth in parallel software.
Data-intensive applications such as transaction processing and information retrieval, data mining and analysis and multimedia services
have provided a new challenge for the modern generation of parallel platforms. Emerging areas such as computational biology and
nanotechnology have implications for algorithms and systems development, while changes in architectures, programming models and
applications have implications for how parallel platforms are made available to users in the form of grid-based services.
This book takes into account these new developments as well as covering the more traditional problems addressed by parallel
computers.Where possible it employs an architecture-independent view of the underlying platforms and designs algorithms for an
abstract model. Message Passing Interface (MPI), POSIX threads and OpenMP have been selected as programming models and the
evolving application mix of parallel computing is reflected in various examples throughout the book.
[ Team LiB ]
This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks .](https://image.slidesharecdn.com/64399-250726042431-2351f906/85/Introduction-to-Parallel-Computing-2nd-Edition-Ananth-Grama-6-320.jpg)
![[ Team LiB ]
• Table of Contents
Introduction to Parallel Computing, Second Edition
By Ananth Grama, Anshul Gupta, George Karypis, Vipin Kumar
Publisher: Addison Wesley
Pub Date: January 16, 2003
ISBN: 0-201-64865-2
Pages: 856
Copyright
Pearson Education
Preface
Acknowledgments
Chapter 1. Introduction to Parallel Computing
Section 1.1. Motivating Parallelism
Section 1.2. Scope of Parallel Computing
Section 1.3. Organization and Contents of the Text
Section 1.4. Bibliographic Remarks
Problems
Chapter 2. Parallel Programming Platforms
Section 2.1. Implicit Parallelism: Trends in Microprocessor Architectures*
Section 2.2. Limitations of Memory System Performance*
Section 2.3. Dichotomy of Parallel Computing Platforms
Section 2.4. Physical Organization of Parallel Platforms
Section 2.5. Communication Costs in Parallel Machines
Section 2.6. Routing Mechanisms for Interconnection Networks
Section 2.7. Impact of Process-Processor Mapping and Mapping Techniques
Section 2.8. Bibliographic Remarks
Problems
Chapter 3. Principles of Parallel Algorithm Design
Section 3.1. Preliminaries
Section 3.2. Decomposition Techniques
Section 3.3. Characteristics of Tasks and Interactions
Section 3.4. Mapping Techniques for Load Balancing
Section 3.5. Methods for Containing Interaction Overheads
This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks.](https://image.slidesharecdn.com/64399-250726042431-2351f906/85/Introduction-to-Parallel-Computing-2nd-Edition-Ananth-Grama-7-320.jpg)
![[ Team LiB ]
This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks .](https://image.slidesharecdn.com/64399-250726042431-2351f906/85/Introduction-to-Parallel-Computing-2nd-Edition-Ananth-Grama-10-320.jpg)
![[ Team LiB ]
This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks .](https://image.slidesharecdn.com/64399-250726042431-2351f906/85/Introduction-to-Parallel-Computing-2nd-Edition-Ananth-Grama-11-320.jpg)
![To Joanna, Rinku, Krista, and Renu
[ Team LiB ]
This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks .](https://image.slidesharecdn.com/64399-250726042431-2351f906/85/Introduction-to-Parallel-Computing-2nd-Edition-Ananth-Grama-13-320.jpg)
![[ Team LiB ]
Pearson Education
We work with leading authors to develop the strongest educational materials in computing, bringing cutting-edge thinking and best
learning practice to a global market.
Under a range of well-known imprints, including Addison-Wesley, we craft high-quality print and electronic publications which help
readers to understand and apply their content, whether studying or at work.
To find out more about the complete range of our publishing, please visit us on the World Wide Web at:
www.pearsoneduc.com
[ Team LiB ]
This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks .](https://image.slidesharecdn.com/64399-250726042431-2351f906/85/Introduction-to-Parallel-Computing-2nd-Edition-Ananth-Grama-14-320.jpg)
![[ Team LiB ]
Preface
Since the 1994 release of the text "Introduction to Parallel Computing: Design and Analysis of Algorithms" by the same authors, the field
of parallel computing has undergone significant changes. Whereas tightly coupled scalable message-passing platforms were the norm a
decade ago, a significant portion of the current generation of platforms consists of inexpensive clusters of workstations, and
multiprocessor workstations and servers. Programming models for these platforms have also evolved over this time. Whereas most
machines a decade back relied on custom APIs for messaging and loop-based parallelism, current models standardize these APIs
across platforms. Message passing libraries such as PVM and MPI, thread libraries such as POSIX threads, and directive based models
such as OpenMP are widely accepted as standards, and have been ported to a variety of platforms.
With respect to applications, fluid dynamics, structural mechanics, and signal processing formed dominant applications a decade back.
These applications continue to challenge the current generation of parallel platforms. However, a variety of new applications have also
become important. These include data-intensive applications such as transaction processing and information retrieval, data mining and
analysis, and multimedia services. Applications in emerging areas of computational biology and nanotechnology pose tremendous
challenges for algorithms and systems development. Changes in architectures, programming models, and applications are also being
accompanied by changes in how parallel platforms are made available to the users in the form of grid-based services.
This evolution has a profound impact on the process of design, analysis, and implementation of parallel algorithms. Whereas the
emphasis of parallel algorithm design a decade back was on precise mapping of tasks to specific topologies such as meshes and
hypercubes, current emphasis is on programmability and portability, both from points of view of algorithm design and implementation. To
this effect, where possible, this book employs an architecture independent view of the underlying platforms and designs algorithms for an
abstract model. With respect to programming models, Message Passing Interface (MPI), POSIX threads, and OpenMP have been
selected. The evolving application mix for parallel computing is also reflected in various examples in the book.
This book forms the basis for a single concentrated course on parallel computing or a two-part sequence. Some suggestions for such a
two-part sequence are:
Introduction to Parallel Computing: Chapters1–6. This course would provide the basics of algorithm design and parallel
programming.
1.
Design and Analysis of Parallel Algorithms: Chapters 2 and 3 followed by Chapters8–12. This course would provide an
in-depth coverage of design and analysis of various parallel algorithms.
2.
The material in this book has been tested in Parallel Algorithms and Parallel Computing courses at the University of Minnesota and
Purdue University. These courses are taken primarily by graduate students and senior-level undergraduate students in Computer
Science. In addition, related courses in Scientific Computation, for which this material has also been tested, are taken by graduate
students in science and engineering, who are interested in solving computationally intensive problems.
Most chapters of the book include (i) examples and illustrations; (ii) problems that supplement the text and test students' understanding
of the material; and (iii) bibliographic remarks to aid researchers and students interested in learning more about related and advanced
topics. The comprehensive subject index helps the reader locate terms they might be interested in. The page number on which a term is
defined is highlighted in boldface in the index. Furthermore, the term itself appears in bold italics where it is defined. The sections that
deal with relatively complex material are preceded by a '*'. An instructors' manual containing slides of the figures and solutions to
selected problems is also available from the publisher (http://www.booksites.net/kumar).
As with our previous book, we view this book as a continually evolving resource. We thank all the readers who have kindly shared
critiques, opinions, problems, code, and other information relating to our first book. It is our sincere hope that we can continue this
interaction centered around this new book. We encourage readers to address communication relating to this book to
book-vk@cs.umn.edu. All relevant reader input will be added to the information archived at the site
http://www.cs.umn.edu/~parbook with
due credit to (and permission of) the sender(s). An on-line errata of the book will also be maintained at the site. We believe that in a
highly dynamic field such as ours, a lot is to be gained from a healthy exchange of ideas and material in this manner.
[ Team LiB ]
This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks .](https://image.slidesharecdn.com/64399-250726042431-2351f906/85/Introduction-to-Parallel-Computing-2nd-Edition-Ananth-Grama-15-320.jpg)
![[ Team LiB ]
Acknowledgments
We would like to begin by acknowledging our spouses, Joanna, Rinku, Krista, and Renu to whom this book is dedicated. Without their
sacrifices this project would not have been seen completion. We also thank our parents, and family members, Akash, Avi, Chethan,
Eleni, Larry, Mary-Jo, Naina, Petros, Samir, Subhasish, Varun, Vibhav, and Vipasha for their affectionate support and encouragement
throughout this project.
Our respective institutions, Computer Sciences and Computing Research Institute (CRI) at Purdue University, Department of Computer
Science & Engineering, the Army High Performance Computing Research Center (AHPCRC), and the Digital Technology Center (DTC)
at the University of Minnesota, and the IBM T. J. Watson Research Center at Yorktown Heights, provided computing resources and
active and nurturing environments for the completion of this project.
This project evolved from our first book. We would therefore like to acknowledge all of the people who helped us with both editions. Many
people contributed to this project in different ways. We would like to thank Ahmed Sameh for his constant encouragement and support,
and Dan Challou, Michael Heath, Dinesh Mehta, Tom Nurkkala, Paul Saylor, and Shang-Hua Teng for the valuable input they provided
to the various versions of the book. We thank the students of the introduction to parallel computing classes at the University of
Minnesota and Purdue university for identifying and working through the errors in the early drafts of the book. In particular, we
acknowledge the patience and help of Jim Diehl and Rasit Eskicioglu, who worked through several early drafts of the manuscript to
identify numerous errors. Ramesh Agarwal, David Bailey, Rupak Biswas, Jim Bottum, Thomas Downar, Rudolf Eigenmann, Sonia
Fahmy, Greg Frederickson, John Gunnels, Fred Gustavson, Susanne Hambrusch, Bruce Hendrickson, Christoph Hoffmann, Kai Hwang,
Ioannis Ioannidis, Chandrika Kamath, David Keyes, Mehmet Koyuturk, Piyush Mehrotra, Zhiyuan Li, Jens Palsberg, Voicu Popescu, Alex
Pothen, Viktor Prasanna, Sanjay Ranka, Naren Ramakrishnan, Elisha Sacks, Vineet Singh, Sartaj Sahni, Vivek Sarin, Wojciech
Szpankowski, Srikanth Thirumalai, Jan Vitek, and David Yau have been great technical resources. It was a pleasure working with the
cooperative and helpful staff at Pearson Education. In particular, we would like to thank Keith Mansfield and Mary Lince for their
professional handling of the project.
The Army Research Laboratory, ARO, DOE, NASA, and NSF provided parallel computing research support for Ananth Grama, George
Karypis, and Vipin Kumar. In particular, Kamal Abdali, Michael Coyle, Jagdish Chandra, Frederica Darema, Stephen Davis, Wm
Randolph Franklin, Richard Hirsch, Charles Koelbel, Raju Namburu, N. Radhakrishnan, John Van Rosendale, Subhash Saini, and
Xiaodong Zhang have been supportive of our research programs in the area of parallel computing. Andrew Conn, Brenda Dietrich, John
Forrest, David Jensen, and Bill Pulleyblank at IBM supported the work of Anshul Gupta over the years.
[ Team LiB ]
This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks .](https://image.slidesharecdn.com/64399-250726042431-2351f906/85/Introduction-to-Parallel-Computing-2nd-Edition-Ananth-Grama-16-320.jpg)
![[ Team LiB ]
Chapter 1. Introduction to Parallel Computing
The past decade has seen tremendous advances in microprocessor technology. Clock rates of processors have increased from about
40 MHz (e.g., a MIPS R3000, circa 1988) to over 2.0 GHz (e.g., a Pentium 4, circa 2002). At the same time, processors are now capable
of executing multiple instructions in the same cycle. The average number of cycles per instruction (CPI) of high end processors has
improved by roughly an order of magnitude over the past 10 years. All this translates to an increase in the peak floating point operation
execution rate (floating point operations per second, or FLOPS) of several orders of magnitude. A variety of other issues have also
become important over the same period. Perhaps the most prominent of these is the ability (or lack thereof) of the memory system to
feed data to the processor at the required rate. Significant innovations in architecture and software have addressed the alleviation of
bottlenecks posed by the datapath and the memory.
The role of concurrency in accelerating computing elements has been recognized for several decades. However, their role in providing
multiplicity of datapaths, increased access to storage elements (both memory and disk), scalable performance, and lower costs is
reflected in the wide variety of applications of parallel computing. Desktop machines, engineering workstations, and compute servers
with two, four, or even eight processors connected together are becoming common platforms for design applications. Large scale
applications in science and engineering rely on larger configurations of parallel computers, often comprising hundreds of processors.
Data intensive platforms such as database or web servers and applications such as transaction processing and data mining often use
clusters of workstations that provide high aggregate disk bandwidth. Applications in graphics and visualization use multiple rendering
pipes and processing elements to compute and render realistic environments with millions of polygons in real time. Applications requiring
high availability rely on parallel and distributed platforms for redundancy. It is therefore extremely important, from the point of view of
cost, performance, and application requirements, to understand the principles, tools, and techniques for programming the wide variety of
parallel platforms currently available.
[ Team LiB ]
This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks .](https://image.slidesharecdn.com/64399-250726042431-2351f906/85/Introduction-to-Parallel-Computing-2nd-Edition-Ananth-Grama-17-320.jpg)
![[ Team LiB ]
1.1 Motivating Parallelism
Development of parallel software has traditionally been thought of as time and effort intensive. This can be largely attributed to the
inherent complexity of specifying and coordinating concurrent tasks, a lack of portable algorithms, standardized environments, and
software development toolkits. When viewed in the context of the brisk rate of development of microprocessors, one is tempted to
question the need for devoting significant effort towards exploiting parallelism as a means of accelerating applications. After all, if it takes
two years to develop a parallel application, during which time the underlying hardware and/or software platform has become obsolete,
the development effort is clearly wasted. However, there are some unmistakable trends in hardware design, which indicate that
uniprocessor (or implicitly parallel) architectures may not be able to sustain the rate of realizable performance increments in the future.
This is a result of lack of implicit parallelism as well as other bottlenecks such as the datapath and the memory. At the same time,
standardized hardware interfaces have reduced the turnaround time from the development of a microprocessor to a parallel machine
based on the microprocessor. Furthermore, considerable progress has been made in standardization of programming environments to
ensure a longer life-cycle for parallel applications. All of these present compelling arguments in favor of parallel computing platforms.
1.1.1 The Computational Power Argument – from Transistors to FLOPS
In 1965, Gordon Moore made the following simple observation:
"The complexity for minimum component costs has increased at a rate of roughly a factor of two per year.
Certainly over the short term this rate can be expected to continue, if not to increase. Over the longer term, the
rate of increase is a bit more uncertain, although there is no reason to believe it will not remain nearly constant for
at least 10 years. That means by 1975, the number of components per integrated circuit for minimum cost will be
65,000."
His reasoning was based on an empirical log-linear relationship between device complexity and time, observed over three data points.
He used this to justify that by 1975, devices with as many as 65,000 components would become feasible on a single silicon chip
occupying an area of only about one-fourth of a square inch. This projection turned out to be accurate with the fabrication of a 16K CCD
memory with about 65,000 components in 1975. In a subsequent paper in 1975, Moore attributed the log-linear relationship to
exponential behavior of die sizes, finer minimum dimensions, and "circuit and device cleverness". He went on to state that:
"There is no room left to squeeze anything out by being clever. Going forward from here we have to depend on
the two size factors - bigger dies and finer dimensions."
He revised his rate of circuit complexity doubling to 18 months and projected from 1975 onwards at this reduced rate. This curve came to
be known as "Moore's Law". Formally, Moore's Law states that circuit complexity doubles every eighteen months. This empirical
relationship has been amazingly resilient over the years both for microprocessors as well as for DRAMs. By relating component density
and increases in die-size to the computing power of a device, Moore's law has been extrapolated to state that the amount of computing
power available at a given cost doubles approximately every 18 months.
The limits of Moore's law have been the subject of extensive debate in the past few years. Staying clear of this debate, the issue of
translating transistors into useful OPS (operations per second) is the critical one. It is possible to fabricate devices with very large
transistor counts. How we use these transistors to achieve increasing rates of computation is the key architectural challenge. A logical
recourse to this is to rely on parallelism – both implicit and explicit. We will briefly discuss implicit parallelism in Section 2.1 and devote
the rest of this book to exploiting explicit parallelism.
This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks](https://image.slidesharecdn.com/64399-250726042431-2351f906/85/Introduction-to-Parallel-Computing-2nd-Edition-Ananth-Grama-18-320.jpg)
![1.1.2 The Memory/Disk Speed Argument
The overall speed of computation is determined not just by the speed of the processor, but also by the ability of the memory system to
feed data to it. While clock rates of high-end processors have increased at roughly 40% per year over the past decade, DRAM access
times have only improved at the rate of roughly 10% per year over this interval. Coupled with increases in instructions executed per clock
cycle, this gap between processor speed and memory presents a tremendous performance bottleneck. This growing mismatch between
processor speed and DRAM latency is typically bridged by a hierarchy of successively faster memory devices called caches that rely on
locality of data reference to deliver higher memory system performance. In addition to the latency, the net effective bandwidth between
DRAM and the processor poses other problems for sustained computation rates.
The overall performance of the memory system is determined by the fraction of the total memory requests that can be satisfied from the
cache. Memory system performance is addressed in greater detail in Section 2.2. Parallel platforms typically yield better memory system
performance because they provide (i) larger aggregate caches, and (ii) higher aggregate bandwidth to the memory system (both typically
linear in the number of processors). Furthermore, the principles that are at the heart of parallel algorithms, namely locality of data
reference, also lend themselves to cache-friendly serial algorithms. This argument can be extended to disks where parallel platforms can
be used to achieve high aggregate bandwidth to secondary storage. Here, parallel algorithms yield insights into the development of
out-of-core computations. Indeed, some of the fastest growing application areas of parallel computing in data servers (database servers,
web servers) rely not so much on their high aggregate computation rates but rather on the ability to pump data out at a faster rate.
1.1.3 The Data Communication Argument
As the networking infrastructure evolves, the vision of using the Internet as one large heterogeneous parallel/distributed computing
environment has begun to take shape. Many applications lend themselves naturally to such computing paradigms. Some of the most
impressive applications of massively parallel computing have been in the context of wide-area distributed platforms. The SETI (Search
for Extra Terrestrial Intelligence) project utilizes the power of a large number of home computers to analyze electromagnetic signals from
outer space. Other such efforts have attempted to factor extremely large integers and to solve large discrete optimization problems.
In many applications there are constraints on the location of data and/or resources across the Internet. An example of such an
application is mining of large commercial datasets distributed over a relatively low bandwidth network. In such applications, even if the
computing power is available to accomplish the required task without resorting to parallel computing, it is infeasible to collect the data at
a central location. In these cases, the motivation for parallelism comes not just from the need for computing resources but also from the
infeasibility or undesirability of alternate (centralized) approaches.
[ Team LiB ]
This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks .](https://image.slidesharecdn.com/64399-250726042431-2351f906/85/Introduction-to-Parallel-Computing-2nd-Edition-Ananth-Grama-19-320.jpg)
![[ Team LiB ]
This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks .](https://image.slidesharecdn.com/64399-250726042431-2351f906/85/Introduction-to-Parallel-Computing-2nd-Edition-Ananth-Grama-20-320.jpg)
![With the widespread use of the web and associated static and dynamic content, there is increasing emphasis on cost-effective servers
capable of providing scalable performance. Parallel platforms ranging from multiprocessors to linux clusters are frequently used as web
and database servers. For instance, on heavy volume days, large brokerage houses on Wall Street handle hundreds of thousands of
simultaneous user sessions and millions of orders. Platforms such as IBMs SP supercomputers and Sun Ultra HPC servers power these
business-critical sites. While not highly visible, some of the largest supercomputing networks are housed on Wall Street.
The availability of large-scale transaction data has also sparked considerable interest in data mining and analysis for optimizing business
and marketing decisions. The sheer volume and geographically distributed nature of this data require the use of effective parallel
algorithms for such problems as association rule mining, clustering, classification, and time-series analysis.
1.2.4 Applications in Computer Systems
As computer systems become more pervasive and computation spreads over the network, parallel processing issues become engrained
into a variety of applications. In computer security, intrusion detection is an outstanding challenge. In the case of network intrusion
detection, data is collected at distributed sites and must be analyzed rapidly for signaling intrusion. The infeasibility of collecting this data
at a central location for analysis requires effective parallel and distributed algorithms. In the area of cryptography, some of the most
spectacular applications of Internet-based parallel computing have focused on factoring extremely large integers.
Embedded systems increasingly rely on distributed control algorithms for accomplishing a variety of tasks. A modern automobile consists
of tens of processors communicating to perform complex tasks for optimizing handling and performance. In such systems, traditional
parallel and distributed algorithms for leader selection, maximal independent set, etc., are frequently used.
While parallel computing has traditionally confined itself to platforms with well behaved compute and network elements in which faults
and errors do not play a significant role, there are valuable lessons that extend to computations on ad-hoc, mobile, or faulty
environments.
[ Team LiB ]
This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks .](https://image.slidesharecdn.com/64399-250726042431-2351f906/85/Introduction-to-Parallel-Computing-2nd-Edition-Ananth-Grama-22-320.jpg)
![[ Team LiB ]
1.3 Organization and Contents of the Text
This book provides a comprehensive and self-contained exposition of problem solving using parallel computers. Algorithms and metrics
focus on practical and portable models of parallel machines. Principles of algorithm design focus on desirable attributes of parallel
algorithms and techniques for achieving these in the contest of a large class of applications and architectures. Programming techniques
cover standard paradigms such as MPI and POSIX threads that are available across a range of parallel platforms.
Chapters in this book can be grouped into four main parts as illustrated in Figure 1.1. These parts are as follows:
Figure 1.1. Recommended sequence for reading the chapters.
This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks](https://image.slidesharecdn.com/64399-250726042431-2351f906/85/Introduction-to-Parallel-Computing-2nd-Edition-Ananth-Grama-23-320.jpg)
![Fundamentals This section spans Chapters2 through 4 of the book.Chapter 2, Parallel Programming Platforms, discusses the physical
organization of parallel platforms. It establishes cost metrics that can be used for algorithm design. The objective of this chapter is not to
provide an exhaustive treatment of parallel architectures; rather, it aims to provide sufficient detail required to use these machines
efficiently. Chapter 3, Principles of Parallel Algorithm Design, addresses key factors that contribute to efficient parallel algorithms and
presents a suite of techniques that can be applied across a wide range of applications. Chapter 4, Basic Communication Operations,
presents a core set of operations that are used throughout the book for facilitating efficient data transfer in parallel algorithms. Finally,
Chapter 5, Analytical Modeling of Parallel Programs, deals with metrics for quantifying the performance of a parallel algorithm.
Parallel Programming This section includes Chapters6 and 7 of the book.Chapter 6, Programming Using the Message-Passing
Paradigm, focuses on the Message Passing Interface (MPI) for programming message passing platforms, including clusters. Chapter 7,
Programming Shared Address Space Platforms, deals with programming paradigms such as threads and directive based approaches.
Using paradigms such as POSIX threads and OpenMP, it describes various features necessary for programming shared-address-space
parallel machines. Both of these chapters illustrate various programming concepts using a variety of examples of parallel programs.
Non-numerical Algorithms Chapters 9–12 present parallel non-numerical algorithms. Chapter 9 addresses sorting algorithms such as
bitonic sort, bubble sort and its variants, quicksort, sample sort, and shellsort. Chapter 10 describes algorithms for various graph theory
problems such as minimum spanning tree, shortest paths, and connected components. Algorithms for sparse graphs are also discussed.
Chapter 11 addresses search-based methods such as branch-and-bound and heuristic search for combinatorial problems.
Chapter 12
classifies and presents parallel formulations for a variety of dynamic programming algorithms.
Numerical Algorithms Chapters 8 and 13 present parallel numerical algorithms. Chapter 8 covers basic operations on dense matrices
such as matrix multiplication, matrix-vector multiplication, and Gaussian elimination. This chapter is included before non-numerical
algorithms, as the techniques for partitioning and assigning matrices to processors are common to many non-numerical algorithms.
Furthermore, matrix-vector and matrix-matrix multiplication algorithms form the kernels of many graph algorithms. Chapter 13 describes
algorithms for computing Fast Fourier Transforms.
[ Team LiB ]
This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks .](https://image.slidesharecdn.com/64399-250726042431-2351f906/85/Introduction-to-Parallel-Computing-2nd-Edition-Ananth-Grama-24-320.jpg)
![[ Team LiB ]
1.4 Bibliographic Remarks
Many books discuss aspects of parallel processing at varying levels of detail. Hardware aspects of parallel computers have been
discussed extensively in several textbooks and monographs [CSG98, LW95, HX98, AG94, Fly95, AG94, Sto93, DeC89, HB84, RF89,
Sie85, Tab90, Tab91, WF84, Woo86]. A number of texts discuss paradigms and languages for programming parallel computers [
LB98,
Pac98, GLS99, GSNL98, CDK
+
00, WA98, And91, BA82, Bab88, Ble90, Con89, CT92, Les93, Per87, Wal91]. Akl [Akl97], Cole [Col89],
Gibbons and Rytter [GR90], Foster [Fos95], Leighton [Lei92], Miller and Stout [
MS96], and Quinn [
Qui94] discuss various aspects of
parallel algorithm design and analysis. Buyya (Editor) [Buy99] and Pfister [Pfi98] discuss various aspects of parallel computing using
clusters. Jaja [Jaj92] covers parallel algorithms for the PRAM model of computation. Hillis [
Hil85, HS86] and Hatcher and Quinn [
HQ91]
discuss data-parallel programming. Agha [Agh86] discusses a model of concurrent computation based onactors. Sharp [Sha85]
addresses data-flow computing. Some books provide a general overview of topics in parallel computing [CL93, Fou94, Zom96, JGD87,
LER92, Mol93, Qui94]. Many books address parallel processing applications in numerical analysis and scientific computing [
DDSV99,
FJDS96, GO93, Car89]. Foxet al. [FJL
+
88] and Angus et al. [AFKW90] provide an application-oriented view of algorithm design for
problems in scientific computing. Bertsekas and Tsitsiklis [BT97] discuss parallel algorithms, with emphasis on numerical applications.
Akl and Lyons [AL93] discuss parallel algorithms in computational geometry. Ranka and Sahni [
RS90b] and Dew, Earnshaw, and
Heywood [DEH89] address parallel algorithms for use in computer vision. Green [
Gre91] covers parallel algorithms for graphics
applications. Many books address the use of parallel processing in artificial intelligence applications [Gup87, HD89b, KGK90, KKKS94,
Kow88, RZ89].
A useful collection of reviews, bibliographies and indexes has been put together by the Association for Computing Machinery [ACM91].
Messina and Murli [MM91] present a collection of papers on various aspects of the application and potential of parallel computing. The
scope of parallel processing and various aspects of US government support have also been discussed in National Science Foundation
reports [NSF91, GOV99].
A number of conferences address various aspects of parallel computing. A few important ones are the Supercomputing Conference,
ACM Symposium on Parallel Algorithms and Architectures, the International Conference on Parallel Processing, the International
Parallel and Distributed Processing Symposium, Parallel Computing, and the SIAM Conference on Parallel Processing. Important
journals in parallel processing include IEEE Transactions on Parallel and Distributed Systems, International Journal of Parallel
Programming, Journal of Parallel and Distributed Computing, Parallel Computing, IEEE Concurrency, and Parallel Processing Letters.
These proceedings and journals provide a rich source of information on the state of the art in parallel processing.
[ Team LiB ]
This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks .](https://image.slidesharecdn.com/64399-250726042431-2351f906/85/Introduction-to-Parallel-Computing-2nd-Edition-Ananth-Grama-25-320.jpg)
![MR. JUSTICE JACKSON: You testified that the movement into the
Rhineland had not been planned in advance.
GÖRING: Only a short time in advance, I emphasized.
MR. JUSTICE JACKSON: How long?
GÖRING: As far as I recall, at the most 2 to 3 weeks.
MR. JUSTICE JACKSON: Now, I call your attention to the minutes
of the 10th meeting of the Working Committee of the Reich Defense
Council, Document Number EC-405 toward the end of that
document, the discussion on 6th month, 26th day of 1935, which
reads as follows . . .
GÖRING: May I ask what page? This document is very long and
is new to me. What page, please, otherwise I shall have to read the
whole document.
MR. JUSTICE JACKSON: Turn to the last paragraph and we will
work backwards.
“Commitment to writing of directives for mobilization
purposes is permissible only insofar as it is absolutely
necessary for the smooth execution of the measures
provided for the demilitarized zone. Without exception such
material must be kept in safes.”
Do you find that part?
GÖRING: This document that has been handed to me contains
alternating statements of various individuals, that is, a dialogue. May
I ask once more . . . The last paragraph contains nothing of what
you have stated, apparently there must be a difference between the
German and English texts. The last paragraph here is altogether
irrelevant. Where, please, am I to read in the document?
MR. JUSTICE JACKSON: Do you find the third paragraph from the
end? If my document is correct we have got the same document.
GÖRING: You must tell me who was speaking, for different
persons speak here.
[The place in the document was indicated to the defendant.]](https://image.slidesharecdn.com/64399-250726042431-2351f906/85/Introduction-to-Parallel-Computing-2nd-Edition-Ananth-Grama-29-320.jpg)
![see, it figures among small quite general, ordinary and usual
preparations for mobilization. The phrase used by the Prosecution
. . .
MR. JUSTICE JACKSON: Mobilization, exactly.
GÖRING: That, if you remember, I stressed clearly in my
statement, that in the demilitarized zone general preparations for
mobilization were made. I mentioned the purchase of horses, et
cetera. I wanted only to point out the mistake regarding “clearing of
the Rhine,” which has nothing to do with the Rhineland, but only
with the river.
MR. JUSTICE JACKSON: Well, those preparations were
preparations for armed occupation of the Rhineland, were they not?
GÖRING: No, that is altogether wrong. If Germany had become
involved in a war, no matter from which side, let us assume from the
East, then mobilization measures would have had to be carried out
for security reasons throughout the Reich, in this event even in the
demilitarized Rhineland; but not for the purpose of occupation, of
liberating the Rhineland.
MR. JUSTICE JACKSON: You mean the preparations were not
military preparations?
GÖRING: Those were general preparations for mobilization, such
as every country makes, and not for the purpose of the occupation
of the Rhineland.
MR. JUSTICE JACKSON: But were of a character which had to be
kept entirely secret from foreign powers?
GÖRING: I do not think I can recall reading beforehand the
publication of the mobilization preparations of the United States.
MR. JUSTICE JACKSON: Well, I respectfully submit to the
Tribunal that this witness is not being responsive, and has not been
in his examination, and that it is . . .
[The defendant interposed a few words which were not
recorded.]
It is perfectly futile to spend our time if we cannot have
responsive answers to our questions.](https://image.slidesharecdn.com/64399-250726042431-2351f906/85/Introduction-to-Parallel-Computing-2nd-Edition-Ananth-Grama-31-320.jpg)
![[The defendant interposed a few words which were not
recorded.]
We can strike these things out. I do not want to spend time
doing that, but this witness, it seems to me, is adopting, and has
adopted, in the witness box and in the dock, an arrogant and
contemptuous attitude toward the Tribunal which is giving him the
trial which he never gave a living soul, nor dead ones either.
I respectfully submit that the witness be instructed to make
notes, if he wishes, of his explanations, but that he be required to
answer my questions and reserve his explanations for his counsel to
bring out.
THE PRESIDENT: I have already laid down the general rule,
which is binding upon this defendant as upon other witnesses.
Perhaps we had better adjourn now at this state.
[The Tribunal adjourned until 20 March 1946 at 1000 hours.]](https://image.slidesharecdn.com/64399-250726042431-2351f906/85/Introduction-to-Parallel-Computing-2nd-Edition-Ananth-Grama-32-320.jpg)
![other nation, then the remark would have been considered
harmless.
In my opinion the answer was quite justified. The witness should
be given the possibility not only to answer “yes” or “no,” but to give
reasons for his answer, as ruled by the Court.
THE PRESIDENT: Mr. Justice Jackson, the Tribunal considers that
the rule which it has laid down is the only possible rule and that the
witness must be confined strictly to answering the question directly
where the question admits of a direct answer, and that he must not
make his explanation before he gives a direct answer; but, after
having given a direct answer to any question which admits of a
direct answer, he may make a short explanation; and that he is not
to be confined simply to making direct answers “yes” or “no,” and
leaving the explanation until his counsel puts it to him in his re-
examination.
As to this particular observation of the defendant, the defendant
ought not to have referred to the United States, but it is a matter
which I think you might well ignore.
MR. JUSTICE JACKSON: I shall bow to the ruling, of course.
I wish to make a statement to the Tribunal about one of the
documents. At the conclusion of the session yesterday we were
considering Document Number EC-405. The Defendant Göring
challenged the use of a word which he said should have been
translated “clearance” rather than “liberation.” We have since had
the translation checked and find that the defendant is correct. This
document was introduced under Exhibit Number GB-160 on the 9th
of January, at Page 2396 of the Tribunal’s records (Volume V, Page
28), and since it has already been received in evidence and it is
before the Tribunal, we think it incumbent upon the Prosecution to
make that correction now for the record.
[Turning to the witness.] You stated yesterday that the minutes
of the Reich Defense Council with which you were presented were
not minutes of a meeting of the Reich Defense Council as such?
GÖRING: Yes, I said that.](https://image.slidesharecdn.com/64399-250726042431-2351f906/85/Introduction-to-Parallel-Computing-2nd-Edition-Ananth-Grama-37-320.jpg)
![MR. JUSTICE JACKSON: And your testimony, notwithstanding
that document, still stands, I take it, that the Reich Defense Council
never met?
GÖRING: I said that also, yes.
MR. JUSTICE JACKSON: I now ask to have you shown a
document which has just come into our possession, the minutes of
the second session of the Reich Defense Council. I should have said,
just come to us for translation. We have not had it translated; we
just discovered it among our great collection of documents.
THE PRESIDENT: Could Doctor Stahmer have a copy in English or
not?
MR. JUSTICE JACKSON: We have not even had a chance to get it
into English. I do not know what it says except that it is the minutes
of their meeting. We have a photostat.
[Turning to the witness.] Are those not the minutes of the second
meeting of the Reich Defense Council held on the 23rd of June
1939?
GÖRING: I must read it first.
MR. JUSTICE JACKSON: I call your attention to the fact that the
chairman is Minister President General Field Marshal Göring. You will
find that on Page 1.
GÖRING: I have never disputed that. It was fixed by law. This
deals with the second Reich Defense Council, not the first one.
Besides, I was not present at this meeting; and I point out that on
the left is a list of the authorities who took part in the meeting, and
in my case it says “Minister President Field Marshal Göring,” and on
the right, as representative for him, “State Secretary Körner and
State Secretary Neumann.” But I shall have to look through the
document first in order to find out whether I took part personally.
MR. JUSTICE JACKSON: Does it not say on Page 1, directly under
the place of meeting, “Chairman: Minister President Göring”?
GÖRING: Yes. I have to read it first.
MR. JUSTICE JACKSON: Do you deny the authenticity of those
minutes?](https://image.slidesharecdn.com/64399-250726042431-2351f906/85/Introduction-to-Parallel-Computing-2nd-Edition-Ananth-Grama-38-320.jpg)
![GÖRING: If you have it there before you, then it must be correct.
MR. JUSTICE JACKSON: You have no recollection of that?
GÖRING: Not of all the separate laws and decrees. That is
impossible.
MR. JUSTICE JACKSON: Then, it was you, was it not, who
signed, on the 31st day of July 1941, a decree asking Himmler, and
the Chief of Security Police and the SS Gruppenführer Heydrich to
make the plans for the complete solution of the Jewish question?
GÖRING: No, that is not correct. I know that decree very well.
MR. JUSTICE JACKSON: I ask to have you shown Document 710,
Exhibit Number USA-509.
THE PRESIDENT: Is that 710-PS?
MR. JUSTICE JACKSON: 710-PS, Your Honor.
[Turning to the witness.] That document is signed by you, is it
not?
GÖRING: That is correct.
MR. JUSTICE JACKSON: And it is addressed to the Chief of the
Security Police and the Security Service, and to SS Gruppenführer
Heydrich, isn’t it?
GÖRING: That is also correct.
MR. JUSTICE JACKSON: I am not certain whether the entire thing
has been read into the record, but I think it should be; and, that we
may have no difficulty about the translation of this, you correct me if
I am wrong:
“Completing the task that was assigned to you on the 24th
of January 1939 . . .”
GÖRING: Here is a mistake already. It says: “Complementing”
not “completing” the task which has been assigned to you.
MR. JUSTICE JACKSON: Very well, I will accept that.
“. . . which dealt with arriving at a thorough furtherance of
emigration and evacuation, a solution of the Jewish
problem, as advantageously as possible, I hereby charge](https://image.slidesharecdn.com/64399-250726042431-2351f906/85/Introduction-to-Parallel-Computing-2nd-Edition-Ananth-Grama-45-320.jpg)
![MR. JUSTICE JACKSON: And mentioning his rank in the SS was
just superfluous and has nothing to do with the case?
GÖRING: I have to explain that. For instance, if I write to the
Commander-in-Chief of the Army, then I write: “To the Commander-
in-Chief of the Army, Colonel General or Field Marshal Von
Brauchitsch.” And if I write to the Chief of the Security Police, then I
must address it: “To the Chief of the Security Police, SS
Gruppenführer Heydrich.” That was his rank and his title. However,
that does not mean that the SS had anything to do with it.
MR. JUSTICE JACKSON: Now, at the time that you issued this
order you had received complete reports as to the 1938 riots and
Heydrich’s part in them, hadn’t you?
GÖRING: At that time I had no knowledge of Heydrich’s part in
the riots—only Heydrich’s report on the riots, for which I had asked.
MR. JUSTICE, JACKSON: All right. Now we will show you
Document Number 3058-PS, in evidence as Exhibit Number USA-
508.
[Document 3058-PS was submitted to the witness.]
That is the report written by Heydrich which you say you had
received, and it is dated 11 November 1938, is it not?
GÖRING: That is correct.
MR. JUSTICE JACKSON: And it recited to you the looting of
Jewish shops, the arrest of 174 persons for looting, the destruction
of 815 shops, 171 dwellings set on fire or destroyed, and that this
indicated only a fraction of the actual damage caused; 191
synagogues were set on fire, and another 76 completely destroyed;
in addition, 11 parish halls, cemetery chapels, and similar buildings
were set on fire, and 3 more completely destroyed; 20,000 Jews
were arrested; also, 7 Aryans and 3 foreigners—the latter were
arrested for their own safety; 36 deaths were reported, and the
seriously injured were also numbered at 36. Those killed and injured
are Jews. One Jew is still missing. The Jews killed include 1 Polish
national, and those injured include 2 Poles.](https://image.slidesharecdn.com/64399-250726042431-2351f906/85/Introduction-to-Parallel-Computing-2nd-Edition-Ananth-Grama-48-320.jpg)
Introduction to Parallel Computing 2nd Edition Ananth Grama
- 1. Introduction to Parallel Computing 2nd Edition Ananth Grama - PDF Download (2025) https://ebookultra.com/download/introduction-to-parallel- computing-2nd-edition-ananth-grama/ Visit ebookultra.com today to download the complete set of ebooks or textbooks
- 2. We have selected some products that you may be interested in Click the link to download now or visit ebookultra.com for more options!. Introduction to Parallel Computing 2nd Edition Ananth Grama https://ebookultra.com/download/introduction-to-parallel- computing-2nd-edition-ananth-grama-2/ An introduction to parallel and vector scientific computing 1st Edition Ronald W. Shonkwiler https://ebookultra.com/download/an-introduction-to-parallel-and- vector-scientific-computing-1st-edition-ronald-w-shonkwiler/ Elements of Parallel Computing 1st Edition Eric Aubanel https://ebookultra.com/download/elements-of-parallel-computing-1st- edition-eric-aubanel/ Introduction to grid computing with Globus 2nd ed Edition Luis Ferreira https://ebookultra.com/download/introduction-to-grid-computing-with- globus-2nd-ed-edition-luis-ferreira/
- 3. Parallel Computing in Quantum Chemistry 1st Edition Curtis L. Janssen https://ebookultra.com/download/parallel-computing-in-quantum- chemistry-1st-edition-curtis-l-janssen/ High Performance Parallel Database Processing and Grid Databases Wiley Series on Parallel and Distributed Computing 1st Edition David Taniar https://ebookultra.com/download/high-performance-parallel-database- processing-and-grid-databases-wiley-series-on-parallel-and- distributed-computing-1st-edition-david-taniar/ An Introduction to Quantum Computing 1st Edition Phillip Kaye https://ebookultra.com/download/an-introduction-to-quantum- computing-1st-edition-phillip-kaye/ The dRuby Book Distributed and Parallel Computing with Ruby 1st Edition Masatoshi Seki https://ebookultra.com/download/the-druby-book-distributed-and- parallel-computing-with-ruby-1st-edition-masatoshi-seki/ Mobile Intelligence Wiley Series on Parallel and Distributed Computing 1st Edition Laurence T. Yang https://ebookultra.com/download/mobile-intelligence-wiley-series-on- parallel-and-distributed-computing-1st-edition-laurence-t-yang/
- 5. Introduction to Parallel Computing 2nd Edition Ananth Grama Digital Instant Download Author(s): Ananth Grama ISBN(s): 9780849344961, 0849344964 Edition: 2 File Details: PDF, 12.32 MB Year: 2003 Language: english
- 6. [ Team LiB ] • Table of Contents Introduction to Parallel Computing, Second Edition By Ananth Grama, Anshul Gupta, George Karypis, Vipin Kumar Publisher: Addison Wesley Pub Date: January 16, 2003 ISBN: 0-201-64865-2 Pages: 856 Increasingly, parallel processing is being seen as the only cost-effective method for the fast solution of computationally large and data-intensive problems. The emergence of inexpensive parallel computers such as commodity desktop multiprocessors and clusters of workstations or PCs has made such parallel methods generally applicable, as have software standards for portable parallel programming. This sets the stage for substantial growth in parallel software. Data-intensive applications such as transaction processing and information retrieval, data mining and analysis and multimedia services have provided a new challenge for the modern generation of parallel platforms. Emerging areas such as computational biology and nanotechnology have implications for algorithms and systems development, while changes in architectures, programming models and applications have implications for how parallel platforms are made available to users in the form of grid-based services. This book takes into account these new developments as well as covering the more traditional problems addressed by parallel computers.Where possible it employs an architecture-independent view of the underlying platforms and designs algorithms for an abstract model. Message Passing Interface (MPI), POSIX threads and OpenMP have been selected as programming models and the evolving application mix of parallel computing is reflected in various examples throughout the book. [ Team LiB ] This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks .
- 7. [ Team LiB ] • Table of Contents Introduction to Parallel Computing, Second Edition By Ananth Grama, Anshul Gupta, George Karypis, Vipin Kumar Publisher: Addison Wesley Pub Date: January 16, 2003 ISBN: 0-201-64865-2 Pages: 856 Copyright Pearson Education Preface Acknowledgments Chapter 1. Introduction to Parallel Computing Section 1.1. Motivating Parallelism Section 1.2. Scope of Parallel Computing Section 1.3. Organization and Contents of the Text Section 1.4. Bibliographic Remarks Problems Chapter 2. Parallel Programming Platforms Section 2.1. Implicit Parallelism: Trends in Microprocessor Architectures* Section 2.2. Limitations of Memory System Performance* Section 2.3. Dichotomy of Parallel Computing Platforms Section 2.4. Physical Organization of Parallel Platforms Section 2.5. Communication Costs in Parallel Machines Section 2.6. Routing Mechanisms for Interconnection Networks Section 2.7. Impact of Process-Processor Mapping and Mapping Techniques Section 2.8. Bibliographic Remarks Problems Chapter 3. Principles of Parallel Algorithm Design Section 3.1. Preliminaries Section 3.2. Decomposition Techniques Section 3.3. Characteristics of Tasks and Interactions Section 3.4. Mapping Techniques for Load Balancing Section 3.5. Methods for Containing Interaction Overheads This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks.
- 8. Section 3.6. Parallel Algorithm Models Section 3.7. Bibliographic Remarks Problems Chapter 4. Basic Communication Operations Section 4.1. One-to-All Broadcast and All-to-One Reduction Section 4.2. All-to-All Broadcast and Reduction Section 4.3. All-Reduce and Prefix-Sum Operations Section 4.4. Scatter and Gather Section 4.5. All-to-All Personalized Communication Section 4.6. Circular Shift Section 4.7. Improving the Speed of Some Communication Operations Section 4.8. Summary Section 4.9. Bibliographic Remarks Problems Chapter 5. Analytical Modeling of Parallel Programs Section 5.1. Sources of Overhead in Parallel Programs Section 5.2. Performance Metrics for Parallel Systems Section 5.3. The Effect of Granularity on Performance Section 5.4. Scalability of Parallel Systems Section 5.5. Minimum Execution Time and Minimum Cost-Optimal Execution Time Section 5.6. Asymptotic Analysis of Parallel Programs Section 5.7. Other Scalability Metrics Section 5.8. Bibliographic Remarks Problems Chapter 6. Programming Using the Message-Passing Paradigm Section 6.1. Principles of Message-Passing Programming Section 6.2. The Building Blocks: Send and Receive Operations Section 6.3. MPI: the Message Passing Interface Section 6.4. Topologies and Embedding Section 6.5. Overlapping Communication with Computation Section 6.6. Collective Communication and Computation Operations Section 6.7. Groups and Communicators Section 6.8. Bibliographic Remarks Problems Chapter 7. Programming Shared Address Space Platforms Section 7.1. Thread Basics Section 7.2. Why Threads? Section 7.3. The POSIX Thread API Section 7.4. Thread Basics: Creation and Termination Section 7.5. Synchronization Primitives in Pthreads Section 7.6. Controlling Thread and Synchronization Attributes Section 7.7. Thread Cancellation Section 7.8. Composite Synchronization Constructs Section 7.9. Tips for Designing Asynchronous Programs Section 7.10. OpenMP: a Standard for Directive Based Parallel Programming Section 7.11. Bibliographic Remarks Problems Chapter 8. Dense Matrix Algorithms Section 8.1. Matrix-Vector Multiplication Section 8.2. Matrix-Matrix Multiplication This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks.
- 9. Section 8.3. Solving a System of Linear Equations Section 8.4. Bibliographic Remarks Problems Chapter 9. Sorting Section 9.1. Issues in Sorting on Parallel Computers Section 9.2. Sorting Networks Section 9.3. Bubble Sort and its Variants Section 9.4. Quicksort Section 9.5. Bucket and Sample Sort Section 9.6. Other Sorting Algorithms Section 9.7. Bibliographic Remarks Problems Chapter 10. Graph Algorithms Section 10.1. Definitions and Representation Section 10.2. Minimum Spanning Tree: Prim's Algorithm Section 10.3. Single-Source Shortest Paths: Dijkstra's Algorithm Section 10.4. All-Pairs Shortest Paths Section 10.5. Transitive Closure Section 10.6. Connected Components Section 10.7. Algorithms for Sparse Graphs Section 10.8. Bibliographic Remarks Problems Chapter 11. Search Algorithms for Discrete Optimization Problems Section 11.1. Definitions and Examples Section 11.2. Sequential Search Algorithms Section 11.3. Search Overhead Factor Section 11.4. Parallel Depth-First Search Section 11.5. Parallel Best-First Search Section 11.6. Speedup Anomalies in Parallel Search Algorithms Section 11.7. Bibliographic Remarks Problems Chapter 12. Dynamic Programming Section 12.1. Overview of Dynamic Programming Section 12.2. Serial Monadic DP Formulations Section 12.3. Nonserial Monadic DP Formulations Section 12.4. Serial Polyadic DP Formulations Section 12.5. Nonserial Polyadic DP Formulations Section 12.6. Summary and Discussion Section 12.7. Bibliographic Remarks Problems Chapter 13. Fast Fourier Transform Section 13.1. The Serial Algorithm Section 13.2. The Binary-Exchange Algorithm Section 13.3. The Transpose Algorithm Section 13.4. Bibliographic Remarks Problems Appendix A. Complexity of Functions and Order Analysis Section A.1. Complexity of Functions Section A.2. Order Analysis of Functions Bibliography This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks.
- 10. [ Team LiB ] This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks .
- 11. [ Team LiB ] This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks .
- 12. Copyright Pearson Education Limited Edinburgh Gate Harlow Essex CM20 2JE England and Associated Companies throughout the world Visit us on the World Wide Web at: www.pearsoneduc.com First published by The Benjamin/Cummings Publishing Company, Inc. 1994 Second edition published 2003 © The Benjamin/Cummings Publishing Company, Inc. 1994 © Pearson Education Limited 2003 The rights of Ananth Grama, Anshul Gupta, George Karypis and Vipin Kumar to be identified as authors of this work have been asserted by them in accordance with the Copyright, Designs and Patents Act 1988. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without either the prior written permission of the publisher or a licence permitting restricted copying in the United Kingdom issued by the Copyright Licensing Agency Ltd, 90 Tottenham Court Road, London W1T 4LP. The programs in this book have been included for their instructional value. They have been tested with care but are not guaranteed for any particular purpose. The publisher does not offer any warranties or representations nor does it accept any liabilities with respect to the programs. All trademarks used herein are the property of their respective owners. The use of any trademark in this text does not vest in the author or publisher any trademark ownership rights in such trademarks, nor does the use of such trademarks imply any affiliation with or endorsement of this book by such owners. British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library Library of Congress Cataloging-in-Publication Data A catalog record for this book is available from the Library of Congress 10 9 8 7 6 5 4 3 2 1 07 06 05 04 03 Printed and bound in the United States of America Dedication This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks
- 13. To Joanna, Rinku, Krista, and Renu [ Team LiB ] This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks .
- 14. [ Team LiB ] Pearson Education We work with leading authors to develop the strongest educational materials in computing, bringing cutting-edge thinking and best learning practice to a global market. Under a range of well-known imprints, including Addison-Wesley, we craft high-quality print and electronic publications which help readers to understand and apply their content, whether studying or at work. To find out more about the complete range of our publishing, please visit us on the World Wide Web at: www.pearsoneduc.com [ Team LiB ] This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks .
- 15. [ Team LiB ] Preface Since the 1994 release of the text "Introduction to Parallel Computing: Design and Analysis of Algorithms" by the same authors, the field of parallel computing has undergone significant changes. Whereas tightly coupled scalable message-passing platforms were the norm a decade ago, a significant portion of the current generation of platforms consists of inexpensive clusters of workstations, and multiprocessor workstations and servers. Programming models for these platforms have also evolved over this time. Whereas most machines a decade back relied on custom APIs for messaging and loop-based parallelism, current models standardize these APIs across platforms. Message passing libraries such as PVM and MPI, thread libraries such as POSIX threads, and directive based models such as OpenMP are widely accepted as standards, and have been ported to a variety of platforms. With respect to applications, fluid dynamics, structural mechanics, and signal processing formed dominant applications a decade back. These applications continue to challenge the current generation of parallel platforms. However, a variety of new applications have also become important. These include data-intensive applications such as transaction processing and information retrieval, data mining and analysis, and multimedia services. Applications in emerging areas of computational biology and nanotechnology pose tremendous challenges for algorithms and systems development. Changes in architectures, programming models, and applications are also being accompanied by changes in how parallel platforms are made available to the users in the form of grid-based services. This evolution has a profound impact on the process of design, analysis, and implementation of parallel algorithms. Whereas the emphasis of parallel algorithm design a decade back was on precise mapping of tasks to specific topologies such as meshes and hypercubes, current emphasis is on programmability and portability, both from points of view of algorithm design and implementation. To this effect, where possible, this book employs an architecture independent view of the underlying platforms and designs algorithms for an abstract model. With respect to programming models, Message Passing Interface (MPI), POSIX threads, and OpenMP have been selected. The evolving application mix for parallel computing is also reflected in various examples in the book. This book forms the basis for a single concentrated course on parallel computing or a two-part sequence. Some suggestions for such a two-part sequence are: Introduction to Parallel Computing: Chapters1–6. This course would provide the basics of algorithm design and parallel programming. 1. Design and Analysis of Parallel Algorithms: Chapters 2 and 3 followed by Chapters8–12. This course would provide an in-depth coverage of design and analysis of various parallel algorithms. 2. The material in this book has been tested in Parallel Algorithms and Parallel Computing courses at the University of Minnesota and Purdue University. These courses are taken primarily by graduate students and senior-level undergraduate students in Computer Science. In addition, related courses in Scientific Computation, for which this material has also been tested, are taken by graduate students in science and engineering, who are interested in solving computationally intensive problems. Most chapters of the book include (i) examples and illustrations; (ii) problems that supplement the text and test students' understanding of the material; and (iii) bibliographic remarks to aid researchers and students interested in learning more about related and advanced topics. The comprehensive subject index helps the reader locate terms they might be interested in. The page number on which a term is defined is highlighted in boldface in the index. Furthermore, the term itself appears in bold italics where it is defined. The sections that deal with relatively complex material are preceded by a '*'. An instructors' manual containing slides of the figures and solutions to selected problems is also available from the publisher (http://www.booksites.net/kumar). As with our previous book, we view this book as a continually evolving resource. We thank all the readers who have kindly shared critiques, opinions, problems, code, and other information relating to our first book. It is our sincere hope that we can continue this interaction centered around this new book. We encourage readers to address communication relating to this book to book-vk@cs.umn.edu. All relevant reader input will be added to the information archived at the site http://www.cs.umn.edu/~parbook with due credit to (and permission of) the sender(s). An on-line errata of the book will also be maintained at the site. We believe that in a highly dynamic field such as ours, a lot is to be gained from a healthy exchange of ideas and material in this manner. [ Team LiB ] This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks .
- 16. [ Team LiB ] Acknowledgments We would like to begin by acknowledging our spouses, Joanna, Rinku, Krista, and Renu to whom this book is dedicated. Without their sacrifices this project would not have been seen completion. We also thank our parents, and family members, Akash, Avi, Chethan, Eleni, Larry, Mary-Jo, Naina, Petros, Samir, Subhasish, Varun, Vibhav, and Vipasha for their affectionate support and encouragement throughout this project. Our respective institutions, Computer Sciences and Computing Research Institute (CRI) at Purdue University, Department of Computer Science & Engineering, the Army High Performance Computing Research Center (AHPCRC), and the Digital Technology Center (DTC) at the University of Minnesota, and the IBM T. J. Watson Research Center at Yorktown Heights, provided computing resources and active and nurturing environments for the completion of this project. This project evolved from our first book. We would therefore like to acknowledge all of the people who helped us with both editions. Many people contributed to this project in different ways. We would like to thank Ahmed Sameh for his constant encouragement and support, and Dan Challou, Michael Heath, Dinesh Mehta, Tom Nurkkala, Paul Saylor, and Shang-Hua Teng for the valuable input they provided to the various versions of the book. We thank the students of the introduction to parallel computing classes at the University of Minnesota and Purdue university for identifying and working through the errors in the early drafts of the book. In particular, we acknowledge the patience and help of Jim Diehl and Rasit Eskicioglu, who worked through several early drafts of the manuscript to identify numerous errors. Ramesh Agarwal, David Bailey, Rupak Biswas, Jim Bottum, Thomas Downar, Rudolf Eigenmann, Sonia Fahmy, Greg Frederickson, John Gunnels, Fred Gustavson, Susanne Hambrusch, Bruce Hendrickson, Christoph Hoffmann, Kai Hwang, Ioannis Ioannidis, Chandrika Kamath, David Keyes, Mehmet Koyuturk, Piyush Mehrotra, Zhiyuan Li, Jens Palsberg, Voicu Popescu, Alex Pothen, Viktor Prasanna, Sanjay Ranka, Naren Ramakrishnan, Elisha Sacks, Vineet Singh, Sartaj Sahni, Vivek Sarin, Wojciech Szpankowski, Srikanth Thirumalai, Jan Vitek, and David Yau have been great technical resources. It was a pleasure working with the cooperative and helpful staff at Pearson Education. In particular, we would like to thank Keith Mansfield and Mary Lince for their professional handling of the project. The Army Research Laboratory, ARO, DOE, NASA, and NSF provided parallel computing research support for Ananth Grama, George Karypis, and Vipin Kumar. In particular, Kamal Abdali, Michael Coyle, Jagdish Chandra, Frederica Darema, Stephen Davis, Wm Randolph Franklin, Richard Hirsch, Charles Koelbel, Raju Namburu, N. Radhakrishnan, John Van Rosendale, Subhash Saini, and Xiaodong Zhang have been supportive of our research programs in the area of parallel computing. Andrew Conn, Brenda Dietrich, John Forrest, David Jensen, and Bill Pulleyblank at IBM supported the work of Anshul Gupta over the years. [ Team LiB ] This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks .
- 17. [ Team LiB ] Chapter 1. Introduction to Parallel Computing The past decade has seen tremendous advances in microprocessor technology. Clock rates of processors have increased from about 40 MHz (e.g., a MIPS R3000, circa 1988) to over 2.0 GHz (e.g., a Pentium 4, circa 2002). At the same time, processors are now capable of executing multiple instructions in the same cycle. The average number of cycles per instruction (CPI) of high end processors has improved by roughly an order of magnitude over the past 10 years. All this translates to an increase in the peak floating point operation execution rate (floating point operations per second, or FLOPS) of several orders of magnitude. A variety of other issues have also become important over the same period. Perhaps the most prominent of these is the ability (or lack thereof) of the memory system to feed data to the processor at the required rate. Significant innovations in architecture and software have addressed the alleviation of bottlenecks posed by the datapath and the memory. The role of concurrency in accelerating computing elements has been recognized for several decades. However, their role in providing multiplicity of datapaths, increased access to storage elements (both memory and disk), scalable performance, and lower costs is reflected in the wide variety of applications of parallel computing. Desktop machines, engineering workstations, and compute servers with two, four, or even eight processors connected together are becoming common platforms for design applications. Large scale applications in science and engineering rely on larger configurations of parallel computers, often comprising hundreds of processors. Data intensive platforms such as database or web servers and applications such as transaction processing and data mining often use clusters of workstations that provide high aggregate disk bandwidth. Applications in graphics and visualization use multiple rendering pipes and processing elements to compute and render realistic environments with millions of polygons in real time. Applications requiring high availability rely on parallel and distributed platforms for redundancy. It is therefore extremely important, from the point of view of cost, performance, and application requirements, to understand the principles, tools, and techniques for programming the wide variety of parallel platforms currently available. [ Team LiB ] This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks .
- 18. [ Team LiB ] 1.1 Motivating Parallelism Development of parallel software has traditionally been thought of as time and effort intensive. This can be largely attributed to the inherent complexity of specifying and coordinating concurrent tasks, a lack of portable algorithms, standardized environments, and software development toolkits. When viewed in the context of the brisk rate of development of microprocessors, one is tempted to question the need for devoting significant effort towards exploiting parallelism as a means of accelerating applications. After all, if it takes two years to develop a parallel application, during which time the underlying hardware and/or software platform has become obsolete, the development effort is clearly wasted. However, there are some unmistakable trends in hardware design, which indicate that uniprocessor (or implicitly parallel) architectures may not be able to sustain the rate of realizable performance increments in the future. This is a result of lack of implicit parallelism as well as other bottlenecks such as the datapath and the memory. At the same time, standardized hardware interfaces have reduced the turnaround time from the development of a microprocessor to a parallel machine based on the microprocessor. Furthermore, considerable progress has been made in standardization of programming environments to ensure a longer life-cycle for parallel applications. All of these present compelling arguments in favor of parallel computing platforms. 1.1.1 The Computational Power Argument – from Transistors to FLOPS In 1965, Gordon Moore made the following simple observation: "The complexity for minimum component costs has increased at a rate of roughly a factor of two per year. Certainly over the short term this rate can be expected to continue, if not to increase. Over the longer term, the rate of increase is a bit more uncertain, although there is no reason to believe it will not remain nearly constant for at least 10 years. That means by 1975, the number of components per integrated circuit for minimum cost will be 65,000." His reasoning was based on an empirical log-linear relationship between device complexity and time, observed over three data points. He used this to justify that by 1975, devices with as many as 65,000 components would become feasible on a single silicon chip occupying an area of only about one-fourth of a square inch. This projection turned out to be accurate with the fabrication of a 16K CCD memory with about 65,000 components in 1975. In a subsequent paper in 1975, Moore attributed the log-linear relationship to exponential behavior of die sizes, finer minimum dimensions, and "circuit and device cleverness". He went on to state that: "There is no room left to squeeze anything out by being clever. Going forward from here we have to depend on the two size factors - bigger dies and finer dimensions." He revised his rate of circuit complexity doubling to 18 months and projected from 1975 onwards at this reduced rate. This curve came to be known as "Moore's Law". Formally, Moore's Law states that circuit complexity doubles every eighteen months. This empirical relationship has been amazingly resilient over the years both for microprocessors as well as for DRAMs. By relating component density and increases in die-size to the computing power of a device, Moore's law has been extrapolated to state that the amount of computing power available at a given cost doubles approximately every 18 months. The limits of Moore's law have been the subject of extensive debate in the past few years. Staying clear of this debate, the issue of translating transistors into useful OPS (operations per second) is the critical one. It is possible to fabricate devices with very large transistor counts. How we use these transistors to achieve increasing rates of computation is the key architectural challenge. A logical recourse to this is to rely on parallelism – both implicit and explicit. We will briefly discuss implicit parallelism in Section 2.1 and devote the rest of this book to exploiting explicit parallelism. This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks
- 19. 1.1.2 The Memory/Disk Speed Argument The overall speed of computation is determined not just by the speed of the processor, but also by the ability of the memory system to feed data to it. While clock rates of high-end processors have increased at roughly 40% per year over the past decade, DRAM access times have only improved at the rate of roughly 10% per year over this interval. Coupled with increases in instructions executed per clock cycle, this gap between processor speed and memory presents a tremendous performance bottleneck. This growing mismatch between processor speed and DRAM latency is typically bridged by a hierarchy of successively faster memory devices called caches that rely on locality of data reference to deliver higher memory system performance. In addition to the latency, the net effective bandwidth between DRAM and the processor poses other problems for sustained computation rates. The overall performance of the memory system is determined by the fraction of the total memory requests that can be satisfied from the cache. Memory system performance is addressed in greater detail in Section 2.2. Parallel platforms typically yield better memory system performance because they provide (i) larger aggregate caches, and (ii) higher aggregate bandwidth to the memory system (both typically linear in the number of processors). Furthermore, the principles that are at the heart of parallel algorithms, namely locality of data reference, also lend themselves to cache-friendly serial algorithms. This argument can be extended to disks where parallel platforms can be used to achieve high aggregate bandwidth to secondary storage. Here, parallel algorithms yield insights into the development of out-of-core computations. Indeed, some of the fastest growing application areas of parallel computing in data servers (database servers, web servers) rely not so much on their high aggregate computation rates but rather on the ability to pump data out at a faster rate. 1.1.3 The Data Communication Argument As the networking infrastructure evolves, the vision of using the Internet as one large heterogeneous parallel/distributed computing environment has begun to take shape. Many applications lend themselves naturally to such computing paradigms. Some of the most impressive applications of massively parallel computing have been in the context of wide-area distributed platforms. The SETI (Search for Extra Terrestrial Intelligence) project utilizes the power of a large number of home computers to analyze electromagnetic signals from outer space. Other such efforts have attempted to factor extremely large integers and to solve large discrete optimization problems. In many applications there are constraints on the location of data and/or resources across the Internet. An example of such an application is mining of large commercial datasets distributed over a relatively low bandwidth network. In such applications, even if the computing power is available to accomplish the required task without resorting to parallel computing, it is infeasible to collect the data at a central location. In these cases, the motivation for parallelism comes not just from the need for computing resources but also from the infeasibility or undesirability of alternate (centralized) approaches. [ Team LiB ] This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks .
- 20. [ Team LiB ] This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks .
- 21. 1.2 Scope of Parallel Computing Parallel computing has made a tremendous impact on a variety of areas ranging from computational simulations for scientific and engineering applications to commercial applications in data mining and transaction processing. The cost benefits of parallelism coupled with the performance requirements of applications present compelling arguments in favor of parallel computing. We present a small sample of the diverse applications of parallel computing. 1.2.1 Applications in Engineering and Design Parallel computing has traditionally been employed with great success in the design of airfoils (optimizing lift, drag, stability), internal combustion engines (optimizing charge distribution, burn), high-speed circuits (layouts for delays and capacitive and inductive effects), and structures (optimizing structural integrity, design parameters, cost, etc.), among others. More recently, design of microelectromechanical and nanoelectromechanical systems (MEMS and NEMS) has attracted significant attention. While most applications in engineering and design pose problems of multiple spatial and temporal scales and coupled physical phenomena, in the case of MEMS/NEMS design these problems are particularly acute. Here, we often deal with a mix of quantum phenomena, molecular dynamics, and stochastic and continuum models with physical processes such as conduction, convection, radiation, and structural mechanics, all in a single system. This presents formidable challenges for geometric modeling, mathematical modeling, and algorithm development, all in the context of parallel computers. Other applications in engineering and design focus on optimization of a variety of processes. Parallel computers have been used to solve a variety of discrete and continuous optimization problems. Algorithms such as Simplex, Interior Point Method for linear optimization and Branch-and-bound, and Genetic programming for discrete optimization have been efficiently parallelized and are frequently used. 1.2.2 Scientific Applications The past few years have seen a revolution in high performance scientific computing applications. The sequencing of the human genome by the International Human Genome Sequencing Consortium and Celera, Inc. has opened exciting new frontiers in bioinformatics. Functional and structural characterization of genes and proteins hold the promise of understanding and fundamentally influencing biological processes. Analyzing biological sequences with a view to developing new drugs and cures for diseases and medical conditions requires innovative algorithms as well as large-scale computational power. Indeed, some of the newest parallel computing technologies are targeted specifically towards applications in bioinformatics. Advances in computational physics and chemistry have focused on understanding processes ranging in scale from quantum phenomena to macromolecular structures. These have resulted in design of new materials, understanding of chemical pathways, and more efficient processes. Applications in astrophysics have explored the evolution of galaxies, thermonuclear processes, and the analysis of extremely large datasets from telescopes. Weather modeling, mineral prospecting, flood prediction, etc., rely heavily on parallel computers and have very significant impact on day-to-day life. Bioinformatics and astrophysics also present some of the most challenging problems with respect to analyzing extremely large datasets. Protein and gene databases (such as PDB, SwissProt, and ENTREZ and NDB) along with Sky Survey datasets (such as the Sloan Digital Sky Surveys) represent some of the largest scientific datasets. Effectively analyzing these datasets requires tremendous computational power and holds the key to significant scientific discoveries. 1.2.3 Commercial Applications This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks
- 22. With the widespread use of the web and associated static and dynamic content, there is increasing emphasis on cost-effective servers capable of providing scalable performance. Parallel platforms ranging from multiprocessors to linux clusters are frequently used as web and database servers. For instance, on heavy volume days, large brokerage houses on Wall Street handle hundreds of thousands of simultaneous user sessions and millions of orders. Platforms such as IBMs SP supercomputers and Sun Ultra HPC servers power these business-critical sites. While not highly visible, some of the largest supercomputing networks are housed on Wall Street. The availability of large-scale transaction data has also sparked considerable interest in data mining and analysis for optimizing business and marketing decisions. The sheer volume and geographically distributed nature of this data require the use of effective parallel algorithms for such problems as association rule mining, clustering, classification, and time-series analysis. 1.2.4 Applications in Computer Systems As computer systems become more pervasive and computation spreads over the network, parallel processing issues become engrained into a variety of applications. In computer security, intrusion detection is an outstanding challenge. In the case of network intrusion detection, data is collected at distributed sites and must be analyzed rapidly for signaling intrusion. The infeasibility of collecting this data at a central location for analysis requires effective parallel and distributed algorithms. In the area of cryptography, some of the most spectacular applications of Internet-based parallel computing have focused on factoring extremely large integers. Embedded systems increasingly rely on distributed control algorithms for accomplishing a variety of tasks. A modern automobile consists of tens of processors communicating to perform complex tasks for optimizing handling and performance. In such systems, traditional parallel and distributed algorithms for leader selection, maximal independent set, etc., are frequently used. While parallel computing has traditionally confined itself to platforms with well behaved compute and network elements in which faults and errors do not play a significant role, there are valuable lessons that extend to computations on ad-hoc, mobile, or faulty environments. [ Team LiB ] This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks .
- 23. [ Team LiB ] 1.3 Organization and Contents of the Text This book provides a comprehensive and self-contained exposition of problem solving using parallel computers. Algorithms and metrics focus on practical and portable models of parallel machines. Principles of algorithm design focus on desirable attributes of parallel algorithms and techniques for achieving these in the contest of a large class of applications and architectures. Programming techniques cover standard paradigms such as MPI and POSIX threads that are available across a range of parallel platforms. Chapters in this book can be grouped into four main parts as illustrated in Figure 1.1. These parts are as follows: Figure 1.1. Recommended sequence for reading the chapters. This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks
- 24. Fundamentals This section spans Chapters2 through 4 of the book.Chapter 2, Parallel Programming Platforms, discusses the physical organization of parallel platforms. It establishes cost metrics that can be used for algorithm design. The objective of this chapter is not to provide an exhaustive treatment of parallel architectures; rather, it aims to provide sufficient detail required to use these machines efficiently. Chapter 3, Principles of Parallel Algorithm Design, addresses key factors that contribute to efficient parallel algorithms and presents a suite of techniques that can be applied across a wide range of applications. Chapter 4, Basic Communication Operations, presents a core set of operations that are used throughout the book for facilitating efficient data transfer in parallel algorithms. Finally, Chapter 5, Analytical Modeling of Parallel Programs, deals with metrics for quantifying the performance of a parallel algorithm. Parallel Programming This section includes Chapters6 and 7 of the book.Chapter 6, Programming Using the Message-Passing Paradigm, focuses on the Message Passing Interface (MPI) for programming message passing platforms, including clusters. Chapter 7, Programming Shared Address Space Platforms, deals with programming paradigms such as threads and directive based approaches. Using paradigms such as POSIX threads and OpenMP, it describes various features necessary for programming shared-address-space parallel machines. Both of these chapters illustrate various programming concepts using a variety of examples of parallel programs. Non-numerical Algorithms Chapters 9–12 present parallel non-numerical algorithms. Chapter 9 addresses sorting algorithms such as bitonic sort, bubble sort and its variants, quicksort, sample sort, and shellsort. Chapter 10 describes algorithms for various graph theory problems such as minimum spanning tree, shortest paths, and connected components. Algorithms for sparse graphs are also discussed. Chapter 11 addresses search-based methods such as branch-and-bound and heuristic search for combinatorial problems. Chapter 12 classifies and presents parallel formulations for a variety of dynamic programming algorithms. Numerical Algorithms Chapters 8 and 13 present parallel numerical algorithms. Chapter 8 covers basic operations on dense matrices such as matrix multiplication, matrix-vector multiplication, and Gaussian elimination. This chapter is included before non-numerical algorithms, as the techniques for partitioning and assigning matrices to processors are common to many non-numerical algorithms. Furthermore, matrix-vector and matrix-matrix multiplication algorithms form the kernels of many graph algorithms. Chapter 13 describes algorithms for computing Fast Fourier Transforms. [ Team LiB ] This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks .
- 25. [ Team LiB ] 1.4 Bibliographic Remarks Many books discuss aspects of parallel processing at varying levels of detail. Hardware aspects of parallel computers have been discussed extensively in several textbooks and monographs [CSG98, LW95, HX98, AG94, Fly95, AG94, Sto93, DeC89, HB84, RF89, Sie85, Tab90, Tab91, WF84, Woo86]. A number of texts discuss paradigms and languages for programming parallel computers [ LB98, Pac98, GLS99, GSNL98, CDK + 00, WA98, And91, BA82, Bab88, Ble90, Con89, CT92, Les93, Per87, Wal91]. Akl [Akl97], Cole [Col89], Gibbons and Rytter [GR90], Foster [Fos95], Leighton [Lei92], Miller and Stout [ MS96], and Quinn [ Qui94] discuss various aspects of parallel algorithm design and analysis. Buyya (Editor) [Buy99] and Pfister [Pfi98] discuss various aspects of parallel computing using clusters. Jaja [Jaj92] covers parallel algorithms for the PRAM model of computation. Hillis [ Hil85, HS86] and Hatcher and Quinn [ HQ91] discuss data-parallel programming. Agha [Agh86] discusses a model of concurrent computation based onactors. Sharp [Sha85] addresses data-flow computing. Some books provide a general overview of topics in parallel computing [CL93, Fou94, Zom96, JGD87, LER92, Mol93, Qui94]. Many books address parallel processing applications in numerical analysis and scientific computing [ DDSV99, FJDS96, GO93, Car89]. Foxet al. [FJL + 88] and Angus et al. [AFKW90] provide an application-oriented view of algorithm design for problems in scientific computing. Bertsekas and Tsitsiklis [BT97] discuss parallel algorithms, with emphasis on numerical applications. Akl and Lyons [AL93] discuss parallel algorithms in computational geometry. Ranka and Sahni [ RS90b] and Dew, Earnshaw, and Heywood [DEH89] address parallel algorithms for use in computer vision. Green [ Gre91] covers parallel algorithms for graphics applications. Many books address the use of parallel processing in artificial intelligence applications [Gup87, HD89b, KGK90, KKKS94, Kow88, RZ89]. A useful collection of reviews, bibliographies and indexes has been put together by the Association for Computing Machinery [ACM91]. Messina and Murli [MM91] present a collection of papers on various aspects of the application and potential of parallel computing. The scope of parallel processing and various aspects of US government support have also been discussed in National Science Foundation reports [NSF91, GOV99]. A number of conferences address various aspects of parallel computing. A few important ones are the Supercomputing Conference, ACM Symposium on Parallel Algorithms and Architectures, the International Conference on Parallel Processing, the International Parallel and Distributed Processing Symposium, Parallel Computing, and the SIAM Conference on Parallel Processing. Important journals in parallel processing include IEEE Transactions on Parallel and Distributed Systems, International Journal of Parallel Programming, Journal of Parallel and Distributed Computing, Parallel Computing, IEEE Concurrency, and Parallel Processing Letters. These proceedings and journals provide a rich source of information on the state of the art in parallel processing. [ Team LiB ] This document was created by an unregistered ChmMagic, please go to http://www.bisenter.com to register it. Thanks .
- 26. Other documents randomly have different content
- 27. mimeographing it makes no difference at all whether 20, 40, 80, or 150 copies are produced. It makes no difference from the point of view of time, except perhaps 4 or 5 minutes. I consider for this reason that one can hardly refer to technical difficulties in this matter. THE PRESIDENT: Counsel for the Prosecution will consider what you say, but no rule has been made by the Tribunal that every document should be supplied to every counsel during cross- examination. GÖRING: I should like to say again in regard to the document that this is not . . . MR. JUSTICE JACKSON: May I respectfully ask that the witness be instructed to answer the question and reserve his explanations until his counsel takes him on. Otherwise, this cross-examination cannot successfully be conducted, in the sense of being reasonable in time. THE PRESIDENT: I have already explained, on several occasions, that it is the duty of defendants when they are in the witness box, and the duty of witnesses, to answer questions directly, if they are capable of being answered directly, in the affirmative or in the negative; and if they have any explanation to make afterwards, they can make it after answering the question directly. MR. JUSTICE JACKSON: I call your attention to Item 3, under II, “Finances,” reading as follows: “Very critical situation of the Reich Exchequer. Relief initially through the milliard imposed on the Jews and through profits accruing to the Reich from the Aryanization of Jewish enterprises.” You find that in the minutes, do you not? GÖRING: Yes, that is there. MR. JUSTICE JACKSON: And you find the minutes signed by Woermann, do you not?
- 28. GÖRING: No, that is not true. I beg your pardon? Here on the photostat Woermann has signed it, that is not Bormann. I know Bormann’s signature well, it is quite different. MR. JUSTICE JACKSON: I said Woermann. GÖRING: Woermann, yes. MR. JUSTICE JACKSON: All right, my poor pronunciation. Well, was it not a fact that you set up a working committee under the Reich Defense Council which did meet from time to time and did carry on certain work? GÖRING: I have already explained recently: That was the committee of departmental chiefs. MR. JUSTICE JACKSON: And I call your attention to Document Number EC-405, minutes of a meeting of the Working Committee of the Reich Defense Council, Meeting Number 10. GÖRING: I understood the President to say before that when I have answered the question, I can add an explanation that seems necessary to me. Now that I have clearly answered your question with regard to the first document, I want to stress once again that this was not a meeting of the close Reich Defense Council but a general calling together of all ministers, state secretaries and numerous other persons. And that I began my statements as follows: “I. Organization of the Reich Defense Council: The Reich Defense Council was already, by decision of the Cabinet of 1933 and 1934, called into being; but it has never met. Through the Reich Defense Law of 4 September 1938 it was re-established. The Chairman is the Führer, who has appointed General Field Marshal Göring his permanent deputy.” Concerning the Reich Defense Council, about which we have been talking, consisting of Schacht—or rather of the triumvirate—it is attested here in writing once more, as I have correctly said, that this Council never met. I ask to have the question about the second document repeated, as I have forgotten it.
- 29. MR. JUSTICE JACKSON: You testified that the movement into the Rhineland had not been planned in advance. GÖRING: Only a short time in advance, I emphasized. MR. JUSTICE JACKSON: How long? GÖRING: As far as I recall, at the most 2 to 3 weeks. MR. JUSTICE JACKSON: Now, I call your attention to the minutes of the 10th meeting of the Working Committee of the Reich Defense Council, Document Number EC-405 toward the end of that document, the discussion on 6th month, 26th day of 1935, which reads as follows . . . GÖRING: May I ask what page? This document is very long and is new to me. What page, please, otherwise I shall have to read the whole document. MR. JUSTICE JACKSON: Turn to the last paragraph and we will work backwards. “Commitment to writing of directives for mobilization purposes is permissible only insofar as it is absolutely necessary for the smooth execution of the measures provided for the demilitarized zone. Without exception such material must be kept in safes.” Do you find that part? GÖRING: This document that has been handed to me contains alternating statements of various individuals, that is, a dialogue. May I ask once more . . . The last paragraph contains nothing of what you have stated, apparently there must be a difference between the German and English texts. The last paragraph here is altogether irrelevant. Where, please, am I to read in the document? MR. JUSTICE JACKSON: Do you find the third paragraph from the end? If my document is correct we have got the same document. GÖRING: You must tell me who was speaking, for different persons speak here. [The place in the document was indicated to the defendant.]
- 30. Now it has been shown to me. Under the name Jodl; I have to read through it first. MR. JUSTICE JACKSON: Do you find this: “The demilitarized zone requires special treatment. In his speech of 21 May 1935 and in other statements, the Führer and Reich Chancellor declared that the stipulations of the Versailles Treaty and the Locarno Pact regarding the demilitarized zone would be observed.” Do you find this? GÖRING: Yes. MR. JUSTICE JACKSON: And do you find the next paragraph, “Since at present international entanglements must be avoided under all circumstances, all urgently needed preparations may be made. The preparations as such, or their planning, must be kept in strictest secrecy in the zone itself as well as in the rest of the Reich.” Do you find this? GÖRING: Yes. MR. JUSTICE JACKSON: And you also find, “These preparations include in particular”—a) and b) are not important to my present question—“c) Preparation for the liberation of the Rhine.” GÖRING: Oh, no, here you have made a great mistake. The original phrase—and this alone is the point in question—is: “c) Preparation for the clearing of the Rhine.” It is a purely technical preparation that has nothing at all to do with the liberation of the Rhineland. Here it says, first, mobilization measures for transportation and communications, then “c) Preparation for the clearing of the Rhine,” that is, in case of mobilization preparations the Rhine is not to be overburdened with freighters, tugboats, et cetera, but the river has to be clear for military measures. Then it continues: “d) Preparation for local defense,” et cetera. Thus you
- 31. see, it figures among small quite general, ordinary and usual preparations for mobilization. The phrase used by the Prosecution . . . MR. JUSTICE JACKSON: Mobilization, exactly. GÖRING: That, if you remember, I stressed clearly in my statement, that in the demilitarized zone general preparations for mobilization were made. I mentioned the purchase of horses, et cetera. I wanted only to point out the mistake regarding “clearing of the Rhine,” which has nothing to do with the Rhineland, but only with the river. MR. JUSTICE JACKSON: Well, those preparations were preparations for armed occupation of the Rhineland, were they not? GÖRING: No, that is altogether wrong. If Germany had become involved in a war, no matter from which side, let us assume from the East, then mobilization measures would have had to be carried out for security reasons throughout the Reich, in this event even in the demilitarized Rhineland; but not for the purpose of occupation, of liberating the Rhineland. MR. JUSTICE JACKSON: You mean the preparations were not military preparations? GÖRING: Those were general preparations for mobilization, such as every country makes, and not for the purpose of the occupation of the Rhineland. MR. JUSTICE JACKSON: But were of a character which had to be kept entirely secret from foreign powers? GÖRING: I do not think I can recall reading beforehand the publication of the mobilization preparations of the United States. MR. JUSTICE JACKSON: Well, I respectfully submit to the Tribunal that this witness is not being responsive, and has not been in his examination, and that it is . . . [The defendant interposed a few words which were not recorded.] It is perfectly futile to spend our time if we cannot have responsive answers to our questions.
- 32. [The defendant interposed a few words which were not recorded.] We can strike these things out. I do not want to spend time doing that, but this witness, it seems to me, is adopting, and has adopted, in the witness box and in the dock, an arrogant and contemptuous attitude toward the Tribunal which is giving him the trial which he never gave a living soul, nor dead ones either. I respectfully submit that the witness be instructed to make notes, if he wishes, of his explanations, but that he be required to answer my questions and reserve his explanations for his counsel to bring out. THE PRESIDENT: I have already laid down the general rule, which is binding upon this defendant as upon other witnesses. Perhaps we had better adjourn now at this state. [The Tribunal adjourned until 20 March 1946 at 1000 hours.]
- 33. EIGHTY-SIXTH DAY Wednesday, 20 March 1946 Morning Session MR. JUSTICE JACKSON: If the Tribunal please, the last question which I asked last night referring to mobilization preparations in the Rhineland, as shown in the official transcript, was this: “But of a character which had to be kept entirely secret from foreign powers?” The answer was: “I do not believe I can recall the publication of the preparations of the United States for mobilization.” Now, representing the United States of America, I am confronted with these choices—to ignore that remark and allow it to stand for people who do not understand our system; or to develop, at considerable expense of time, its falsity; or to answer it in rebuttal. The difficulty arises from this, Your Honor, that if the witness is permitted to volunteer statements in cross-examination there is no opportunity to make objection until they are placed on the record. Of course, if such an answer had been indicated by a question of counsel, as I respectfully submit would be the orderly procedure, there would have been objection; the Tribunal would have been in a position to discharge its duty under the Charter and I would have been in a position to have shortened the case by not having that remark placed. The Charter in Article 18 provides that the Tribunal shall rule out irrelevant issues and statements of any kind whatsoever. We are squarely confronted with that question; we cannot discharge those duties if the defendant is to volunteer these statements without questions which bring them up. I respectfully submit that, if the
- 34. ruling of the Tribunal that the defendant may volunteer questions of this kind is to prevail, the control of these proceedings is put in the hands of this defendant, and the United States has been substantially denied its right of cross-examination under the Charter, because cross-examination cannot be effective under this kind of procedure. Since we cannot anticipate, we cannot meet . . . THE PRESIDENT: I quite agree with you that any reference to the United States’ secrecy with reference to mobilization is entirely irrelevant, and that the answer ought not to have been made, but the only rule which the Tribunal can lay down as a general rule is the rule—already laid down—that the witness must answer if possible “yes” or “no,” and that he may make such explanations as may be necessary after answering questions directly in that way, and that such explanations must be brief and not be speeches. As far as this particular answer goes, I think it is entirely irrelevant. MR. JUSTICE JACKSON: I must, of course, bow to the ruling of the Tribunal, but it is to the second part, I quite recall the admonition of the Court that there shall be answers “yes” or “no.” This witness, of course, pays not the slightest attention to that, and I must say I cannot blame him; he is pursuing his interests. But we have no way of anticipating, and here we are confronted with this statement in the record, because when these statements are volunteered they are in the record before the Tribunal can rule upon them and I have no opportunity to make objections, and the Tribunal have no opportunity to rule. And it puts, as I said before, the control of these proceedings in the hands of the defendant, if he first makes the charges and then puts it up to us to ignore them or answer them by long cross-examination in rebuttal; and I think the specific charge made against the United States of America from the witness stand presents that. Your Honor now advises the United States that it is an improper answer, but it is in the record and we must deal with it. I respectfully submit that unless we have . . . THE PRESIDENT: What exactly is the motion you are making? Are you asking the Tribunal to strike the answer out of the record?
- 35. MR. JUSTICE JACKSON: Well, no; in a Trial of this kind, where propaganda is one of the purposes of the defendant, striking out does no good after the answer is made, and Göring knows that as well as I. The charge has been made against the United States and it is in the record. I am now moving that this witness be instructed that he must answer my questions “yes” or “no” if they permit an answer, and that the explanation be brought out by his counsel in a fashion that will permit us to make objections, if they are irrelevant, and to obtain rulings of the Tribunal, so that the Tribunal can discharge its functions of ruling out irrelevant issues and statements of any kind whatsoever. We must not let the Trial degenerate into a bickering contest between counsel and the witness. That is not what the United States would expect me to participate in. I respectfully suggest that if he can draw any kind of challenge . . . THE PRESIDENT: Are you submitting to the Tribunal that the witness has to answer every question “yes” or “no” and wait until he is re-examined for the purpose of making any explanations at all? MR. JUSTICE JACKSON: I think that is the rule of cross- examination under ordinary circumstances. The witness, if the question permits it, must answer, and if there are relevant explanations they should be reserved until later. Now let me come back to the specific problem I have right here this morning. Here is an answer given which the Tribunal now rules is irrelevant. But we have no opportunity to object to it. The Tribunal had no opportunity to rule upon it. The witness asks, “Did you ever hear of the United States publishing its plan of mobilization?” Of course, we would have objected. The difficulty is that the Tribunal loses control of these proceedings if the defendant, in a case of this kind where we all know propaganda is one of the purposes of the defendant, is permitted to put his propaganda in, and then we have to meet it afterwards. I really feel that the United States is deprived of the opportunity of the technique of cross-examination if this is the procedure. THE PRESIDENT: Surely it is making too much of a sentence the witness has said, whether the United States makes its orders for
- 36. mobilization public or not. Surely that is not a matter of very great importance. Every country keeps certain things secret. Certainly it would be much wiser to ignore a statement of that sort. But as to the general rule, the Tribunal will now consider the matter. I have already laid down what I believe to be the rule, and I think with the assent of the Tribunal, but I will ascertain . . . MR. JUSTICE JACKSON: Let me say that I agree with Your Honor that as far as the United States is concerned we are not worried by anything the witness can say about it—and we expected plenty. The point is, do we answer these things or leave them, apart from the control of the Trial? And it does seem to me that this is the beginning of this Trial’s getting out of hand, if I may say so, if we do not have control of this situation. I trust the Tribunal will pardon my earnestness in presenting this. I think it is a very vital thing. THE PRESIDENT: I have never heard it suggested that the Counsel for the Prosecution have to answer every irrelevant observation made in cross-examination. MR. JUSTICE JACKSON: That would be true in a private litigation, but I trust the Court is not unaware that outside of this courtroom is a great social question of the revival of Nazism and that one of the purposes of the Defendant Göring—I think he would be the first to admit—is to revive and perpetuate it by propaganda from this Trial now in process. THE PRESIDENT: Yes, Doctor Stahmer? DR. STAHMER: I just wanted to explain the following: An accusation has been made as if we intended to make propaganda here for Nazism, or in some other direction. I do not think this accusation is justified. Neither do I believe that the defendant intended to make an accusation against the United States. I think we have to consider the question that was put to him. That is, it was pointed out to him by the Prosecution that this document which was submitted to him was marked “secret.” Then he stated that he had never heard that a document of that kind would have been made public in the United States. If instead of the U.S.A, he had said any
- 37. other nation, then the remark would have been considered harmless. In my opinion the answer was quite justified. The witness should be given the possibility not only to answer “yes” or “no,” but to give reasons for his answer, as ruled by the Court. THE PRESIDENT: Mr. Justice Jackson, the Tribunal considers that the rule which it has laid down is the only possible rule and that the witness must be confined strictly to answering the question directly where the question admits of a direct answer, and that he must not make his explanation before he gives a direct answer; but, after having given a direct answer to any question which admits of a direct answer, he may make a short explanation; and that he is not to be confined simply to making direct answers “yes” or “no,” and leaving the explanation until his counsel puts it to him in his re- examination. As to this particular observation of the defendant, the defendant ought not to have referred to the United States, but it is a matter which I think you might well ignore. MR. JUSTICE JACKSON: I shall bow to the ruling, of course. I wish to make a statement to the Tribunal about one of the documents. At the conclusion of the session yesterday we were considering Document Number EC-405. The Defendant Göring challenged the use of a word which he said should have been translated “clearance” rather than “liberation.” We have since had the translation checked and find that the defendant is correct. This document was introduced under Exhibit Number GB-160 on the 9th of January, at Page 2396 of the Tribunal’s records (Volume V, Page 28), and since it has already been received in evidence and it is before the Tribunal, we think it incumbent upon the Prosecution to make that correction now for the record. [Turning to the witness.] You stated yesterday that the minutes of the Reich Defense Council with which you were presented were not minutes of a meeting of the Reich Defense Council as such? GÖRING: Yes, I said that.
- 38. MR. JUSTICE JACKSON: And your testimony, notwithstanding that document, still stands, I take it, that the Reich Defense Council never met? GÖRING: I said that also, yes. MR. JUSTICE JACKSON: I now ask to have you shown a document which has just come into our possession, the minutes of the second session of the Reich Defense Council. I should have said, just come to us for translation. We have not had it translated; we just discovered it among our great collection of documents. THE PRESIDENT: Could Doctor Stahmer have a copy in English or not? MR. JUSTICE JACKSON: We have not even had a chance to get it into English. I do not know what it says except that it is the minutes of their meeting. We have a photostat. [Turning to the witness.] Are those not the minutes of the second meeting of the Reich Defense Council held on the 23rd of June 1939? GÖRING: I must read it first. MR. JUSTICE JACKSON: I call your attention to the fact that the chairman is Minister President General Field Marshal Göring. You will find that on Page 1. GÖRING: I have never disputed that. It was fixed by law. This deals with the second Reich Defense Council, not the first one. Besides, I was not present at this meeting; and I point out that on the left is a list of the authorities who took part in the meeting, and in my case it says “Minister President Field Marshal Göring,” and on the right, as representative for him, “State Secretary Körner and State Secretary Neumann.” But I shall have to look through the document first in order to find out whether I took part personally. MR. JUSTICE JACKSON: Does it not say on Page 1, directly under the place of meeting, “Chairman: Minister President Göring”? GÖRING: Yes. I have to read it first. MR. JUSTICE JACKSON: Do you deny the authenticity of those minutes?
- 39. GÖRING: I have not looked them through yet. It seems to be an absolutely authentic copy of the minutes; I admit that. But here again we are dealing with a meeting not, as I said when answering my counsel, of the Reich Defense Council, but of a larger meeting in which many other departments participated; and it is a matter of the second Reich Defense Council, which was set up after 1938, not a secret council such as was the case from 1933-38. MR. JUSTICE JACKSON: In other words, in interpreting your testimony, we must understand that, when you say there was no meeting of the Reich Defense Council, you mean only that there were no meetings at which no other people were present? GÖRING: No, that is not correct. There were two Reich defense laws concerning the Reich Defense Council, which I tried to explain in my statement: the Secret Council of 1933 to 1938, which was not made public, and the Reich Defense Council which was created in 1938 and converted into the Ministerial Council in 1939; the latter held meetings which were in no way confined to its own members. MR. JUSTICE JACKSON: Then you say that this was not the Defense Council that met under the ban of secrecy? GÖRING: The Prosecution want me to answer first with “yes” or “no.” It is hard to answer this question with “yes” or “no.” I assert that the Secret Defense Council, which was not made public and which arose out of a meeting of ministers in 1933, never met. After 1938 a new Reich defense law created a new council. At that time it was clear that our military sovereignty had already been declared. This first council, which the Prosecution called the secret one, never met, and the document of yesterday proved that. MR. JUSTICE JACKSON: Will you refer to Page 19 of this document, please, and tell me whether one of the very things with which this meeting concerned itself was not the lifting of the secrecy ban from the Reich defense law? GÖRING: No, that is not the way it reads here. If I may translate it, the last point on the agenda: Consequences resulting from the
- 40. lifting of the secrecy ban on the Reich defense law and measures to expedite procedures have already been dealt with by a letter from the Reich Defense Committee on 26 June: “Consequences resulting from the lifting of the secrecy ban with a view to expediting written communications.” MR. JUSTICE JACKSON: You have stated that on the Jewish question, some of the members of the government were more radical than you. Would you state who these were? GÖRING: Broadly speaking, when we took over the government, we only demanded their removal from political and other leading positions in the State. MR. JUSTICE JACKSON: That is not what I asked you. THE PRESIDENT: That is not a direct answer to the question. The question was that you said some members of the government were more radical toward Jews than you were. Would you tell us which of the members of the government were more radical than you were? GÖRING: Excuse me, I did not understand the question to mean who were more radical, but in what way they were more radical. If you ask who, then I would say that those were primarily Minister Goebbels and Himmler. MR. JUSTICE JACKSON: Do you also include your co-defendant, Streicher, as more radical than you? GÖRING: Yes, but he was not a member of the government. MR. JUSTICE JACKSON: He was the Gauleiter, was he not, for this very territory in which we are sitting. GÖRING: That is correct; but he had very little or no influence on government measures. MR. JUSTICE JACKSON: What about Heydrich? GÖRING: Heydrich was subordinate to Himmler. If I said Himmler, I, of course, include Heydrich. MR. JUSTICE JACKSON: Heydrich is then included in the list of the more radical ones to whom you refer? GÖRING: That is right; yes.
- 41. MR. JUSTICE JACKSON: What about Bormann? GÖRING: It was only during the later years that I observed that Bormann was becoming more radical. I do not know anything about his attitude in the beginning. MR. JUSTICE JACKSON: Now, I want to review with you briefly what the Prosecution understands to be public acts taken by you in reference to the Jewish question. From the very beginning you regarded the elimination of the Jews from the economic life of Germany as one phase of the Four Year Plan under your jurisdiction, did you not? GÖRING: The elimination, yes; that is partly correct. The elimination as far as the large industries were concerned, because there were continual disturbances due to the fact that there were large industries, also armament industries, still partly under Jewish directors, or with Jewish shareholders, and that gave rise to a certain anxiety among the lower ranks. MR. JUSTICE JACKSON: Now, do I understand that you want the Tribunal to believe that all you were concerned about was the big Jewish enterprises? That is the way you want to be understood? GÖRING: I was not at first disturbed by the small stores. They did not come into the Four Year Plan. MR. JUSTICE JACKSON: When did you become disturbed by the small stores? GÖRING: When trade had to be limited, it was pointed out that this could be done first by closing the Jewish stores. MR. JUSTICE JACKSON: Now, let us go through the public acts which you performed on the Jewish question. First, did you proclaim the Nuremberg Laws? GÖRING: As President of the Reichstag, yes. I have already stated that. MR. JUSTICE JACKSON: What date was that? GÖRING: 1935, I believe; here in Nuremberg, in September. MR. JUSTICE JACKSON: That was the beginning of the legal measures taken against the Jews, was it not?
- 42. GÖRING: That was a legal measure. MR. JUSTICE JACKSON: That was the first of the legal measures taken by your government against the Jews, was it not? GÖRING: No, I believe the removal from office was before. MR. JUSTICE JACKSON: When was that? GÖRING: I could not state the exact date, but I believe that happened in 1933. MR. JUSTICE JACKSON: Then on the first day of December 1936, you promulgated an act making it a death penalty for Germans to transfer property abroad or leave it abroad; the property of a culprit to be forfeited to the State, and the People’s Court given jurisdiction to prosecute, did you not? GÖRING: That is correct; the “Decree Governing Restriction on Foreign Currency.” That is to say, whoever had an account in a foreign country without permission of the government. MR. JUSTICE JACKSON: Then, your third public act was on 22 April 1938 when you published penalties for veiling the character of a Jewish enterprise within the Reich, was it not? GÖRING: Yes. MR. JUSTICE JACKSON: Then on 28 July 1939, you, Hermann Göring, published certain prescriptions on the competence of the courts to handle those matters by the decree, did you not? GÖRING: Please, would you kindly read the law to me? I cannot recall it. MR. JUSTICE JACKSON: I will not take time reading it. Do you deny that you published the Reichsgesetzblatt law, 1939, found on Page 1370, referring to the competence of the courts to handle penalties against Jews? If you do not remember, say so. GÖRING: Yes, I say that I cannot remember the law. If it is in the Reichsgesetzblatt and bears my name, then, of course, it is so; but I do not remember the contents. MR. JUSTICE JACKSON: Now, on 26 April 1938 you, under the Four Year Plan, published a decree providing for the registration of
- 43. Jewish property and provided that Jews inside and outside Germany must register their property, did you not? GÖRING: I assume so. I no longer remember it, but if you have the decree there, and if it is signed by me, there cannot be any doubt. MR. JUSTICE JACKSON: On 26 April 1938 you published a decree under the Four Year Plan, did you not, that all acts of disposal of Jewish enterprises required the permission of the authorities? GÖRING: That I remember. MR. JUSTICE JACKSON: Then you published on 12 November 1938 a decree, also under the Four Year Plan, imposing a fine of a billion marks for atonement on all Jews? GÖRING: I have already explained that all these decrees at that time were signed by me, and I assume responsibility for them. MR. JUSTICE JACKSON: Well, I am asking you if you did not sign that particular decree? I am going to ask you some further questions about it later. GÖRING: Yes. MR. JUSTICE JACKSON: Then on the 12th of November 1938, you also signed a decree that, under the Four Year Plan, all damage caused to Jewish property by the riots of 1938 must be repaired immediately by the Jews, and at their own expense; and their insurance claims were forfeited to the Reich. Did you personally sign that law? GÖRING: I did sign a similar law. Whether it was exactly the same as you have just read, I could not say. MR. JUSTICE JACKSON: You do not disagree that that was the substance of the law, do you? GÖRING: No. MR. JUSTICE JACKSON: And on the 12th of November 1938, did you not also personally sign a decree, also under the Four Year Plan, that Jews may not own retail stores, or engage independently in handicrafts or offer goods, or services, for sale at markets, fairs, or
- 44. exhibitions; or act as leaders of enterprises or as members of co- operatives? Do you recall all of that? GÖRING: Yes. Those are all parts of the decrees for the elimination of Jewry from economic life. MR. JUSTICE JACKSON: Then, on the 21st of February 1939, you personally signed a decree, did you not, that the Jews must surrender all objects of precious metals and jewels purchased, to the public office within 2 weeks? GÖRING: I do not remember that, but without doubt, that is correct. MR. JUSTICE JACKSON: I refer to Volume I of the Reichsgesetzblatt, 1939, Page 282. You have no recollection of that? GÖRING: I have not the Reichsgesetzblatt in front of me now, but if there is a decree in the Reichsgesetzblatt, or a law signed with my name, then I signed that law and decreed it. MR. JUSTICE JACKSON: Did you not also, on the 3rd of March 1939, sign a further decree concerning the period within which items of jewelry must be surrendered by Jews—Reichsgesetzblatt, Volume I, 1939, Page 387? GÖRING: I assume that was the decree for the execution of the decree for surrender previously mentioned. A law sometimes requires regulations and decrees for execution consequent upon the law. Taken together, this is one single measure. MR. JUSTICE JACKSON: Did you not also sign personally a decree under the Four Year Plan, of the 17th of September 1940, ordering the sequestration of Jewish property in Poland? GÖRING: Yes, as I stated before, in that part of Poland which, I may say, as an old German province, was to return to Germany. MR. JUSTICE JACKSON: Did you not also, on the 30th day of November 1940, personally sign a decree which provided that the Jews should receive no compensation for damages caused by enemy attacks or by German forces, and did you not sign that in the capacity of President of the Reich Defense Council? I refer to the Reichsgesetzblatt, Volume I, 1940, Page 1547.
- 45. GÖRING: If you have it there before you, then it must be correct. MR. JUSTICE JACKSON: You have no recollection of that? GÖRING: Not of all the separate laws and decrees. That is impossible. MR. JUSTICE JACKSON: Then, it was you, was it not, who signed, on the 31st day of July 1941, a decree asking Himmler, and the Chief of Security Police and the SS Gruppenführer Heydrich to make the plans for the complete solution of the Jewish question? GÖRING: No, that is not correct. I know that decree very well. MR. JUSTICE JACKSON: I ask to have you shown Document 710, Exhibit Number USA-509. THE PRESIDENT: Is that 710-PS? MR. JUSTICE JACKSON: 710-PS, Your Honor. [Turning to the witness.] That document is signed by you, is it not? GÖRING: That is correct. MR. JUSTICE JACKSON: And it is addressed to the Chief of the Security Police and the Security Service, and to SS Gruppenführer Heydrich, isn’t it? GÖRING: That is also correct. MR. JUSTICE JACKSON: I am not certain whether the entire thing has been read into the record, but I think it should be; and, that we may have no difficulty about the translation of this, you correct me if I am wrong: “Completing the task that was assigned to you on the 24th of January 1939 . . .” GÖRING: Here is a mistake already. It says: “Complementing” not “completing” the task which has been assigned to you. MR. JUSTICE JACKSON: Very well, I will accept that. “. . . which dealt with arriving at a thorough furtherance of emigration and evacuation, a solution of the Jewish problem, as advantageously as possible, I hereby charge
- 46. you with making all necessary preparations in regard to organizational and financial matters for bringing about a complete solution of the Jewish question in the German sphere of influence in Europe.” Am I correct so far? GÖRING: No, that is in no way correctly translated. MR. JUSTICE JACKSON: Give us your translation of it? GÖRING: May I read it as it is written here? “Complementing the task which was conferred upon you already on 24 January 1939, to solve the Jewish problem by means of emigration and evacuation in the best possible way according to present conditions, I charge you herewith to make all necessary preparations as regards organizational, factual, and material matters. . . .” Now comes the decisive word which has been mistranslated: “for a total solution,” not “for a final solution.” “. . . for a total solution of the Jewish question within the area of German influence in Europe. Should these come within the competence of other governmental departments, then such departments are to co-operate. “I charge you further to submit to me as soon as possible a general plan showing the organizational and material measures for reaching the desired total solution of the Jewish question. . . . Complementing the task assigned to you on 24 January 1939. . . .” That was at a time when there was no war or prospect of a war. MR. JUSTICE JACKSON: Now are you reporting the instrument or are you making an explanation? GÖRING: I wanted to add an explanation to the quotation and just to point out the date. MR. JUSTICE JACKSON: Yes. Well, I just did not want it to appear that it was a part of the instrument. The last that is contained in the
- 47. instrument is: “I charge you furthermore to send me, before long, an over-all plan concerning the organizational, factual, and material measures necessary for the accomplishment of the desired solution of the Jewish question.” Is that not a substantially accurate translation of your order to Heydrich and Himmler? GÖRING: To Heydrich and the other government departments which had anything to do with it. That can be seen from the first part of the letter, the last sentence. MR. JUSTICE JACKSON: Let us have no misunderstanding about this translation now. This letter was directed to the Chief of the Security Police and the Security Service, and SS Gruppenführer Heydrich. We are right about that, are we not? GÖRING: That is correct, but I have to make an explanation in connection with that. MR. JUSTICE JACKSON: All right. GÖRING: The reason I sent this letter to him was that, by the decree of 24 January 1939, Heydrich, or it may have been Himmler, had been given the task of dealing with the emigration of the Jews. Therefore, this was the government department concerned, and it was to the department which had been given the task that I had to apply concerning all material and economic matters arising therefrom. MR. JUSTICE JACKSON: Yes. And you ordered all other governmental agencies to co-operate with the Security Police and the SS in the final solution of the Jewish question, did you not? GÖRING: There is nothing about the SS here; only about the Sicherheitspolizei, a governmental agency. The fact that Heydrich was SS Gruppenführer had no direct bearing on it, because it was sent to the Chief of the Security Police—mentioning his rank as SS Gruppenführer Heydrich.
- 48. MR. JUSTICE JACKSON: And mentioning his rank in the SS was just superfluous and has nothing to do with the case? GÖRING: I have to explain that. For instance, if I write to the Commander-in-Chief of the Army, then I write: “To the Commander- in-Chief of the Army, Colonel General or Field Marshal Von Brauchitsch.” And if I write to the Chief of the Security Police, then I must address it: “To the Chief of the Security Police, SS Gruppenführer Heydrich.” That was his rank and his title. However, that does not mean that the SS had anything to do with it. MR. JUSTICE JACKSON: Now, at the time that you issued this order you had received complete reports as to the 1938 riots and Heydrich’s part in them, hadn’t you? GÖRING: At that time I had no knowledge of Heydrich’s part in the riots—only Heydrich’s report on the riots, for which I had asked. MR. JUSTICE, JACKSON: All right. Now we will show you Document Number 3058-PS, in evidence as Exhibit Number USA- 508. [Document 3058-PS was submitted to the witness.] That is the report written by Heydrich which you say you had received, and it is dated 11 November 1938, is it not? GÖRING: That is correct. MR. JUSTICE JACKSON: And it recited to you the looting of Jewish shops, the arrest of 174 persons for looting, the destruction of 815 shops, 171 dwellings set on fire or destroyed, and that this indicated only a fraction of the actual damage caused; 191 synagogues were set on fire, and another 76 completely destroyed; in addition, 11 parish halls, cemetery chapels, and similar buildings were set on fire, and 3 more completely destroyed; 20,000 Jews were arrested; also, 7 Aryans and 3 foreigners—the latter were arrested for their own safety; 36 deaths were reported, and the seriously injured were also numbered at 36. Those killed and injured are Jews. One Jew is still missing. The Jews killed include 1 Polish national, and those injured include 2 Poles.
- 49. Welcome to our website – the ideal destination for book lovers and knowledge seekers. With a mission to inspire endlessly, we offer a vast collection of books, ranging from classic literary works to specialized publications, self-development books, and children's literature. Each book is a new journey of discovery, expanding knowledge and enriching the soul of the reade Our website is not just a platform for buying books, but a bridge connecting readers to the timeless values of culture and wisdom. With an elegant, user-friendly interface and an intelligent search system, we are committed to providing a quick and convenient shopping experience. Additionally, our special promotions and home delivery services ensure that you save time and fully enjoy the joy of reading. Let us accompany you on the journey of exploring knowledge and personal growth! ebookultra.com