Notes for Parallel and Distributed Systems - PDS by Sankarsan Sahoo

Introduction to parallel computing Parallel programming platforms: Trends in microprocessor Architectures, Limitations of memory system performance, Dichotomy of parallel computing platforms, physical organization of parallel platforms, communication costs in parallel machines, Routing mechanisms for interconnection network, Impact of process processors mapping and mapping techniques

Principles of parallel algorithm design Preliminaries, Decomposition techniques, Characteristics of tasks and interactions, Mapping techniques for load balancing, Methods for containing. Interactions overheads, Parallel algorithm models. Basic communication operations: One-to-All Broadcast and All-to-One Reduction, All-to-All broadcast and reduction All-Reduce and prefix sum operations, scatter and gather, All-to-All personalized communication, circular shift, Improving the speed of some communication operation

One-to-All Broadcast and All-to-One Reduction, All-to-All broadcast and reduction All-Reduce and prefix sum operations, scatter and gather, All-to-All personalized communication, circular shift, Improving the speed of some communication operation.

Analytical Modeling of Parallel Systems