CSC/ECE 506 Fall 2007/wiki1 12 dp3
Sections 1.3.3 and 1.3.4: Most changes here are probably related to performance metrics. Cite other models for measuring artifacts such as data-transfer time, overhead, occupancy, and communication cost. Focus on the models that are most useful in practice.
Communication and Replication
Performance
Introduction
- What is this?
- Why is this important to measure?
- How this is measured?
In this introduction section, we describe why performance issue is important for parallel computer architects and what are the metrices of performance.
As we already know performance is a very important issue in uniprocessor system where architects always try to improve performance of system in terms of execution time by using several techniques such as minimizing memory access time (to access the data fast), designing hardware which can execute many instruction in parallel and possibly faster ( micro level parallelism extraction).Consequently, performance issue plays a big role in parallel computing for several reasons like data is shared among many processors and hence processes on different processors need to communicate efficiently and coherently. To make our point more precise, let us consider the following example : Assume we want to run a program which takes 100ms on uniprocessor with no pipeline scheme. However, we also know that the full program can be decomposed in many tasks and again tasks can be grouped to form many processes and these processes can be run on many processors (not necesaarily one-to-one mapping).Suppose we have three processors which can support at least one parallel programming model and we end up hafour processes
Artifacts of Measuring Performance
Data Transfer
Overhead and Occupancy
One of the three components of Processor execution time, apart from Computation Time and Idle Time, is Communication Time. Communication time is the time spent by the processor on sending and receiving messages. There can be two different types of communication, interprocessor and intraprocessor. In interprocessor communication the two communicating tasks are handled by two different processors. While in intraprocessor communication the communicating tasks are handled by the same processor. Generally both intraprocessor and interprocessor communication costs same, provided the intraprocessor communication is not highly optimized.
The communication time is function of number of bytes transferred across. It can be given as below
T(n) = Overhead + Occupancy + Network Delay + Message Size / Bandwidth + Delay due to Contention
Communication Overhead includes time spent to
- Create messages
- Execute communications protocols
- Physically send messages
- Run through the protocol sets and decode the message on the receiving node.
During this period the processor cannot do any useful or computational work. Parallel programs, running on different processors, need coordination of work among themselves. This results in increased rate of interprocessor communication, which in turn increases the net overhead cost. Some of the recent trends/designs helped reduce overhead communication cost. IBM blue gene (L) uses Global Collective Network, which carries out operations within the network itself. This saves the processors time to decode messages with intermediate values, calculate new intermediate values, create new messages, and send them on to other nodes. The overhead now is because of communication protocol. It also has a dedicated communications network to speed up task-to-task coordination activities.
The occupancy is the time spent at the slowest component in the communication assist, affects performance in couple of ways. It delays the current request and indirectly contributes to the delays of subsequent requests. The occupancy gets to set the upper limit on how frequently communication operation can be initiated by the processor. From processor’s point of view number of other kinds of network delays can be categorized as occupancy. Contention for resources can be view as occupancy. The net bandwidth reduces as a result of this. If P concurrent processors are using a network of Bandwidth B, then the net bandwidth would be B/P.
Communication Cost
- Linking to previous Artifacts.
- How to measure- Mathematical Equation
- Quantitative Measure
- Implication of such measurement
- Advantage and Disadvantage
- New Advances
New Artifact
- not decided yet!