Introduction to Parallel Processing

Computer architects have always strived to increase the performance of their
computer architectures. High performance may come from fast dense circuitry,
packaging technology, and parallelism. Single-processor supercomputers have
achieved unheard of speeds and have been pushing hardware technology to the physical
limit of chip manufacturing. However, this trend will soon come to an end,
because there are physical and architectural bounds that limit the computational
power that can be achieved with a single-processor system. In this book we will
study advanced computer architectures that utilize parallelism via multiple processing
units.
Parallel processors are computer systems consisting of multiple processing units
connected via some interconnection network plus the software needed to make the
processing units work together. There are two major factors used to categorize such
systems: the processing units themselves, and the interconnection network that ties
them together. The processing units can communicate and interact with each other
using either shared memory or message passing methods. The interconnection network
for shared memory systems can be classified as bus-based versus switch-based.
In message passing systems, the interconnection network is divided into static and
dynamic. Static connections have a fixed topology that does not change while
programs are running. Dynamic connections create links on the fly as the program
executes.