Expertiza_Wiki - User contributions [en]

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-19T23:56:41Z

Refuller: /* SSCI and IEE SCI Protocol Similarities */

== Topic Background ==
The protocols discussed here were created to allow cache coherence to be maintained on a system that has no global bus system that allows snooping to take place, and, has distributed memory that is physically separated. These systems usually consist of PCM (Processor-Control-Memory) nodes connected by a network; the systems also have network assists to allow communication to take place.

For a system to be coherent it must:
#Provide a state machine for the protocol
#Manage the coherence protocol
#*Determine when the coherence protocol should be used
#*Find information on the current cache block in other caches
#*Find out where other copies of the current cache block are located
#*Communicate with the other copies

The protocols discussed here are managed by the scalable system by using a directory-based approach. Information on all cache blocks is kept in a directory that is associated with each block of memory. When a cache miss arises the directory information is checked and all other caches containing a copy of the desired block are contacted. The information contained in the directories consists of a dirty bit, and some state information that will allow action to be taken when there is a cache miss and the directory information needs to be referenced.
 

== SSCI (Simple Scalable Coherence Interface) protocol ==
The IEEE SCI protocol is a very large and in-depth protocol that is used in real world scalable machines when coherence is a necessity. Due to its size and complexity it is much easier to gain a basic understanding of the SCI protocol by studying a similar, smaller protocol. The SSCI protocol, like its name implies, is a simpler version of the SCI protocol. Below is a list of characteristics that define the SSCI protocol. Note that it is readily apparent from the characteristics below that the SSCI protocol is very similar to the SCI protocol, more information on this can be viewed in section 4 of this page.

SSCI characteristics:
*Full-bit vector protocol
*MESI protocol states used in the cache
*Memory directory states used:
**U - Unowned
**S - Shared
**EM - Exclusive/Modified
*Presence bits are replaced by pointers

More information on, and examples involving, the SSCI protocol can be viewed the CSC/ECE 506 lecture 12 notes [http://courses.ncsu.edu/csc506/common/lectures/notes/lec12.pdf here].
 

== IEEE SCI (Scalable Coherence Interface) protocol ==
 SCI has 2 main configurations: 

'''Ring configuration''' consists of nodes that are doubly linked in a
ring and all nodes have equal priority. The nodes administer the
system as well as do the actual processing.

'''Switch Configuration''' consists nodes don't connect directly they go through
switches that direct packets directly from the source to desination node
(possibly traveling through other switches but not through other nodes).
This is more complicated, but necessary when systems get larger.

The two configurations can also be combined
so there would be rings connected with switches,
this would be best for large performance setups.

 Packets in SCI consist of 3 main sections: 

Header - contains the ID code of the target node, then the control flow
and command, then the third is the id code of the sender (kind of a return
address), followed by the sequence number

Address & Data - contains between 16 and 256 bytes.

Error check word - when transmitted a CRC code is attached to the packet
for error checking and transmission verification.

A transaction requires a transaction handshake, the sender sends a
request to the receiver, the receiver sends an echo of the
request then a response to the sender, then the sender sends an echo of the
response to the receiver.

To ensure cache coherence only 1 node connects to memory and this allows
each node to have its own cache while not generating inconsistency which
would obviously be a problem.

== SSCI and IEE SCI Protocol Similarities ==
*The overall protocol operation of SSCI is exactly like the overall protocol operation of SCI.
*SSCI and SCI both use doubly linked lists among the nodes.

== Links ==
IEEE SCI Standard Abstract and Content Summary -
http://standards.ieee.org/reading/ieee/std_public/description/busarch/1596-1992_desc.html
 
IEEE SCI on ieexplore -
http://ieeexplore.ieee.org/iel5/285/3365/00113656.pdf?tp=&arnumber=113656&isnumber=3365
 
4BA2 Technology Survery on SCI (good overview) -
http://ntrg.cs.tcd.ie/undergrad/4ba2.05/group12/index.html
 
CSC/ECE 506 Lecture Notes - http://www.csc.ncsu.edu/faculty/efg/506/f07/material/

== References ==
CSC/ECE 506 Lecture Notes, Fall 2007

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-19T23:53:57Z

Refuller:

== Topic Background ==
The protocols discussed here were created to allow cache coherence to be maintained on a system that has no global bus system that allows snooping to take place, and, has distributed memory that is physically separated. These systems usually consist of PCM (Processor-Control-Memory) nodes connected by a network; the systems also have network assists to allow communication to take place.

For a system to be coherent it must:
#Provide a state machine for the protocol
#Manage the coherence protocol
#*Determine when the coherence protocol should be used
#*Find information on the current cache block in other caches
#*Find out where other copies of the current cache block are located
#*Communicate with the other copies

The protocols discussed here are managed by the scalable system by using a directory-based approach. Information on all cache blocks is kept in a directory that is associated with each block of memory. When a cache miss arises the directory information is checked and all other caches containing a copy of the desired block are contacted. The information contained in the directories consists of a dirty bit, and some state information that will allow action to be taken when there is a cache miss and the directory information needs to be referenced.
 

== SSCI (Simple Scalable Coherence Interface) protocol ==
The IEEE SCI protocol is a very large and in-depth protocol that is used in real world scalable machines when coherence is a necessity. Due to its size and complexity it is much easier to gain a basic understanding of the SCI protocol by studying a similar, smaller protocol. The SSCI protocol, like its name implies, is a simpler version of the SCI protocol. Below is a list of characteristics that define the SSCI protocol. Note that it is readily apparent from the characteristics below that the SSCI protocol is very similar to the SCI protocol, more information on this can be viewed in section 4 of this page.

SSCI characteristics:
*Full-bit vector protocol
*MESI protocol states used in the cache
*Memory directory states used:
**U - Unowned
**S - Shared
**EM - Exclusive/Modified
*Presence bits are replaced by pointers

More information on, and examples involving, the SSCI protocol can be viewed the CSC/ECE 506 lecture 12 notes [http://courses.ncsu.edu/csc506/common/lectures/notes/lec12.pdf here].
 

== IEEE SCI (Scalable Coherence Interface) protocol ==
 SCI has 2 main configurations: 

'''Ring configuration''' consists of nodes that are doubly linked in a
ring and all nodes have equal priority. The nodes administer the
system as well as do the actual processing.

'''Switch Configuration''' consists nodes don't connect directly they go through
switches that direct packets directly from the source to desination node
(possibly traveling through other switches but not through other nodes).
This is more complicated, but necessary when systems get larger.

The two configurations can also be combined
so there would be rings connected with switches,
this would be best for large performance setups.

 Packets in SCI consist of 3 main sections: 

Header - contains the ID code of the target node, then the control flow
and command, then the third is the id code of the sender (kind of a return
address), followed by the sequence number

Address & Data - contains between 16 and 256 bytes.

Error check word - when transmitted a CRC code is attached to the packet
for error checking and transmission verification.

A transaction requires a transaction handshake, the sender sends a
request to the receiver, the receiver sends an echo of the
request then a response to the sender, then the sender sends an echo of the
response to the receiver.

To ensure cache coherence only 1 node connects to memory and this allows
each node to have its own cache while not generating inconsistency which
would obviously be a problem.

== SSCI and IEE SCI Protocol Similarities ==

== Links ==
IEEE SCI Standard Abstract and Content Summary -
http://standards.ieee.org/reading/ieee/std_public/description/busarch/1596-1992_desc.html
 
IEEE SCI on ieexplore -
http://ieeexplore.ieee.org/iel5/285/3365/00113656.pdf?tp=&arnumber=113656&isnumber=3365
 
4BA2 Technology Survery on SCI (good overview) -
http://ntrg.cs.tcd.ie/undergrad/4ba2.05/group12/index.html
 
CSC/ECE 506 Lecture Notes - http://www.csc.ncsu.edu/faculty/efg/506/f07/material/

== References ==
CSC/ECE 506 Lecture Notes, Fall 2007

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-18T04:11:20Z

Refuller: /* Links */

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-18T04:10:15Z

Refuller:

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-18T04:05:07Z

Refuller: /* SSCI (Simple Scalable Coherence Interface) protocol */

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-18T04:04:44Z

Refuller: /* Topic Background */

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-18T04:04:05Z

Refuller: /* SSCI (Simple Scalable Coherence Interface) protocol */

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-18T04:03:05Z

Refuller:

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-18T03:48:30Z

Refuller: /* SSCI (Simple Scalable Coherence Interface) protocol */

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-17T01:34:02Z

Refuller: /* IEEE SCI (Scalable Coherence Interface) protocol */

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-17T01:30:56Z

Refuller: /* Links */

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-17T01:30:33Z

Refuller: /* Links */

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-17T01:30:15Z

Refuller: /* Links */

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-17T01:29:57Z

Refuller: /* Links */

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-17T01:15:18Z

Refuller: /* IEEE SCI (Scalable Coherence Interface) protocol */

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-17T01:12:15Z

Refuller: /* References */

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-17T01:11:00Z

Refuller: /* Topic Background */

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-17T01:10:10Z

Refuller: /* Topic Background */

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-17T01:09:54Z

Refuller: /* Topic Background */

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-17T01:09:05Z

Refuller: /* Topic Background */

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-17T01:07:33Z

Refuller: /* Background */

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-17T01:07:02Z

Refuller: /* Background */

== Background ==
The protocols discussed here were created to allow cache coherence to be maintained on a system that has no global bus system that allows snooping to take place, and, has distributed memory that is physically separated. These systems usually consist of PCM (Processor-Control-Memory) nodes connected by a network; the systems also have network assists to allow communication to take place.

For a system to be coherent it must:
#Provide a state machine
#Manage the coherence protocol
#*Determine when the coherence protocol should be used
#*Find information on the current cache block in other caches
#*Find out where other copies of the current cache block are located
#*Communicate with the other copies

The protocols discussed here are managed by the scalable system by using a directory-based approach. Information on all cache blocks is kept in a directory that is associated with each block of memory. When a cache miss arises the directory information is checked and all other caches containing a copy of the desired block are contacted. The information contained in the directories consists of a dirty bit, and some state information that will allow action to be taken when there is a cache miss and the directory information needs to be referenced.

== SSCI (Simple Scalable Coherence Interface) protocol ==

== IEEE SCI (Scalable Coherence Interface) protocol ==

== SSCI and IEE SCI Protocol Similarities ==

== IEE SCI Protocol Additional States ==

== Links ==
IEEE SCI Standard Abstract and Content Summary -
http://standards.ieee.org/reading/ieee/std_public/description/busarch/1596-1992_desc.html
 
IEEE SCI on ieexplore -
http://ieeexplore.ieee.org/iel5/285/3365/00113656.pdf?tp=&arnumber=113656&isnumber=3365

== References ==

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-17T01:02:48Z

Refuller: /* Background */

== Background ==
The protocols discussed here were created to allow cache coherence to be maintained on a system that has no global bus system that allows snooping to take place, and, has distributed memory that is physically separated. These systems usually consist of PCM (Processor-Control-Memory) nodes connected by a network; the systems also have network assists to allow communication to take place.

For a system to be coherent it must:
#Provide a state machine
#Manage the coherence protocol
#*Determine when the coherence protocol should be used
#*Find information on the current cache block in other caches
#*Find out where other copies of the current cache block are located
#*Communicate with the other copies

The protocols discussed here are managed by the scalable system by using a directory-based approach. Information on all cache blocks is kept in a directory that is associated with each block of memory. When a cache miss arises the directory information is checked and all other caches containing a copy of the desired block are contacted.

== SSCI (Simple Scalable Coherence Interface) protocol ==

== IEEE SCI (Scalable Coherence Interface) protocol ==

== SSCI and IEE SCI Protocol Similarities ==

== IEE SCI Protocol Additional States ==

== Links ==
IEEE SCI Standard Abstract and Content Summary -
http://standards.ieee.org/reading/ieee/std_public/description/busarch/1596-1992_desc.html
 
IEEE SCI on ieexplore -
http://ieeexplore.ieee.org/iel5/285/3365/00113656.pdf?tp=&arnumber=113656&isnumber=3365

== References ==

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-17T01:00:34Z

Refuller: /* Background */

== Background ==
The protocols discussed here were created to allow cache coherence to be maintained on a system that has no global bus system that allows snooping to take place, and, has distributed memory that is physically separated. These systems usually consist of PCM (Processor-Control-Memory) nodes connected by a network; the systems also have network assists to allow communication to take place.

For a system to be coherent it must:
#Provide a state machine
#Manage the coherence protocol
#*Determine when the coherence protocol should be used
#*Find information on the current cache block in other caches
#*Find out where other copies of the current cache block are located
#*Communicate with the other copies

The protocols discussed here are managed by the scalable system by using a directory-based approach. This approach stores information on all cache blocks in a directory that associated with each block of memory.

== SSCI (Simple Scalable Coherence Interface) protocol ==

== IEEE SCI (Scalable Coherence Interface) protocol ==

== SSCI and IEE SCI Protocol Similarities ==

== IEE SCI Protocol Additional States ==

== Links ==
IEEE SCI Standard Abstract and Content Summary -
http://standards.ieee.org/reading/ieee/std_public/description/busarch/1596-1992_desc.html
 
IEEE SCI on ieexplore -
http://ieeexplore.ieee.org/iel5/285/3365/00113656.pdf?tp=&arnumber=113656&isnumber=3365

== References ==

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-17T00:54:45Z

Refuller:

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-17T00:49:42Z

Refuller:

== Background ==
The protocols discussed here were created to allow coherence to be maintained on a system that has no global bus system that allows snooping to take place, and, has distributed memory that is physically separated. These systems usually consist of PCM (Processor-Control-Memory) nodes connected by a network; the systems also have network assists to allow communication to take place.

For a system to be coherent it must:
#Provide a state machine
#Manage the coherence protocol
#*Determine when the coherence protocol should be used
#*

== SSCI (Simple Scalable Coherence Interface) protocol ==

== IEEE SCI (Scalable Cohttp://pg.ece.ncsu.edu/mediawiki/index.php?title=CSC/ECE_506_Fall_2007/wiki3_8_a1&action=edit
Editing CSC/ECE 506 Fall 2007/wiki3 8 a1 - PG_MediaWikiherence Interface) protocol ==

== SSCI and IEE SCI Protocol Similarities ==

== IEE SCI Protocol Additional States ==

== Links ==
IEEE SCI Standard Abstract and Content Summary -
http://standards.ieee.org/reading/ieee/std_public/description/busarch/1596-1992_desc.html
 
IEEE SCI on ieexplore -
http://ieeexplore.ieee.org/iel5/285/3365/00113656.pdf?tp=&arnumber=113656&isnumber=3365

== References ==

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-17T00:29:18Z

Refuller:

== Background ==
The protocols discussed here were created to allow coherence to be maintained on a system that has no global bus system that allows snooping to take place and has distributed memory that is physically separated.

== SSCI (Simple Scalable Coherence Interface) protocol ==

== IEEE SCI (Scalable Coherence Interface) protocol ==

== SSCI and IEE SCI Protocol Similarities ==

== IEE SCI Protocol Additional States ==

== Links ==
IEEE SCI Standard Abstract and Content Summary -
http://standards.ieee.org/reading/ieee/std_public/description/busarch/1596-1992_desc.html
 
IEEE SCI on ieexplore -
http://ieeexplore.ieee.org/iel5/285/3365/00113656.pdf?tp=&arnumber=113656&isnumber=3365

== References ==

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-16T22:52:28Z

Refuller:

== Background ==

== SSCI (Simple Scalable Coherence Interface) protocol ==

== IEEE SCI (Scalable Coherence Interface) protocol ==

== SSCI and IEE SCI Protocol Similarities ==

== IEE SCI Protocol Additional States ==

== Links ==
IEEE SCI Standard Abstract and Content Summary -
http://standards.ieee.org/reading/ieee/std_public/description/busarch/1596-1992_desc.html
 
IEEE SCI on ieexplore -
http://ieeexplore.ieee.org/iel5/285/3365/00113656.pdf?tp=&arnumber=113656&isnumber=3365

== References ==

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-16T22:52:09Z

Refuller:

== Background ==

== SSCI (Simple Scalable Coherence Interface) protocol ==

== IEEE SCI (Scalable Coherence Interface) protocol ==

== SSCI and IEE SCI Protocol Similarities ==

== IEE SCI Protocol Additional States ==

== Links ==
IEEE SCI Standard Abstract and Content Summary -
http://standards.ieee.org/reading/ieee/std_public/description/busarch/1596-1992_desc.html
IEEE SCI on ieexplore -
http://ieeexplore.ieee.org/iel5/285/3365/00113656.pdf?tp=&arnumber=113656&isnumber=3365

== References ==

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-16T22:48:27Z

Refuller:

== Background ==

== SSCI (Simple Scalable Coherence Interface) protocol ==

== IEEE SCI (Scalable Coherence Interface) protocol ==

== SSCI and IEE SCI Protocol Similarities ==

== IEE SCI Protocol Additional States ==

== Links ==
IEE SCI Standard Abstract and Content Summary:
http://standards.ieee.org/reading/ieee/std_public/description/busarch/1596-1992_desc.html

== References ==

CSC/ECE 506 Fall 2007/wiki3 8 a1

2007-10-16T22:38:57Z

Refuller:

== SSCI (Simple Scalable Coherence Interface) protocol ==

== IEEE SCI (Scalable Coherence Interface) protocol ==

== SSCI and IEE SCI Protocol Similarities ==

== IEE SCI Protocol Additional States ==

== Links ==

== References ==

CSC/ECE 506 Fall 2007/wiki1 6 r8e

2007-09-11T00:24:14Z

Refuller:

== Communication Architecture ==
Parallel computers must have two types of architectures. One of these is this computer architecture, or that of the individual processors that are interconnected. Each of these processors has an instruction set architecture and microarchitecture, or a system of computer organization at a very low level that make up the computer architecture. The second type of architecture that is needed for parallel computing is that of a communication architecture, or a way in which the individual processors can communicate and work together to do useful things, this includes synchronization among the processors. Also, the communication architecture is closely linked with the hardware of the parallel computer because the hardware must be able to accommodate the communication operations.

To see the trends of parallel computers becoming widespread in use this [http://www.islandnet.com/~kpolsson/micropro/ site] provides a chronology of microprocessors.

== Parallel Programming Models ==
The communication architecture dictates communication operations that the user software is allowed to perform. Parallel programming models layout the framework for the way in which the communication operations take place. In the beginning of parallel computing a single programming model was used for one system. The hardware was constructed to support only the communication operations that the specific programming model used. However, As parallel computing became more widespread there has been a convergence of programming models. Many models have become integrated and have been used together. Many parallel programming models exist, the most common of which are described below.

=== Shared Address Space ===
Shared address space programming can be understood best when related to a message or bulletin board. Anyone can post a message on this board, and anyone can read what others have written. The key to the shared address space model is the fact that all of the vital information is posted in shared locations that can be accessed by all with memory operations such as loads and stores.

[http://www.sas.com/grid SAS grid computing.]

[http://www.openmp.org/drupal/ OpenMP - shared-memory parallel programming.]

=== Message Passing ===
Message passing machines convey information in a way similar to that of a phone call or a letter. There are very specific events that trigger the movement of information from a unique sender to a unique receiver. The sending process sends the data along with information that directs the message to the correct receiving process.

A widespread interface for message passing machines is the message passing interface, or MPI. More information on the MPI can be found at the [http://www.mpi-forum.org MPI forum].

=== Data Parallel ===
Data parallel programming is by far the most regulated of the three parallel programming models. In this type of programming work is carried out on different elements of a data set by different operators. Once all operations are finished information is exchanged among all, basically after the work is done a global data organization is performed.

Intel's [http://www.intel.com/design/archives/Processors/mmx/ MMX] processors are a good example of data parallel processing.

These [http://www.vcpc.univie.ac.at/activities/tutorials/HPF/lectures/html/jhm.2.html lectures] speak on the topic of data parallelism in the F90 and HPF (fortran).

== Convergence ==
As parallel computer architecture has matured it has become apparent that the parallel programming models have all been created to accomplish the same sorts of problems. For this reason the division between the programming models has become unclear since hardware for recent parallel computers is able to support a variety of programming models. This is due to the fact that the hardware
primitives have basically become the same for all parallel architectures. A good example of this convergence is the way in which the message-passing machines of today are very similar to nonuniform memory access machines (NUMA), a type of machine that uses the shared address space model.

A 2003 comparison of parallel computing problems solved with message passing and shared address space models can be viewed [http://portal.acm.org/citation.cfm?id=763446 here].

This [http://www.cs.princeton.edu/picasso/seminarsS04/MPI_Day1.pdf paper], which was written in 2005, discusses the need to integrate message passing into shared address space models.

A very good 1999 [http://www.cs.berkeley.edu/~culler/cs258-s99/slides/lec02/index.htm presentation] on the convergence of parallel architectures.

== Layers of Abstraction ==
There are many layers of abstraction between the application and the hardware. The user program is written using a certain programming model. This programming model specifies the way in which the information is communicated among different pieces of the program. The compiler and/or libraries provide the layer of abstraction between the programming model and the available hardware primitives. Figure 1.13 on pg. 27 of "Parallel Computer Architecture: A Hardware/Software Approach" displays these layers of abstraction well. However, it seems that with the convergence of programming models and the use of data parallelism in message passing and shared address space programming the layer of abstraction containing the programming models should be revised if not edited out completely.

== Links to Trends in Parallel Computing ==
[http://www-unix.mcs.anl.gov/dbpp/text/node7.html Parallelism and Computing] 
[http://www.intel.com/cd/ids/developer/asmo-na/eng/segments/hpc/95223.htm?page=3 Intel - Trends in Distributed Computing] 
[http://css.psu.edu/news/nlsp98/progtrends.html Programming Trends in High Performance Computing] 
[http://www.cs.cmu.edu/~scandal/research-groups.html Supercomputing and Parallel Computing Research Groups]

== References ==
Culler, David and Singh, Jaswinder. "Parallel Computer Architecture: A Hardware/Software Approach". ISBN 1-55860-343-3

CSC/ECE 506 Fall 2007/wiki1 6 r8e

2007-09-11T00:22:44Z

Refuller: /* Bibliography */

== Communication Architecture ==
Parallel computers must have two types of architectures. One of these is this computer architecture, or that of the individual processors that are interconnected. Each of these processors has an instruction set architecture and microarchitecture, or a system of computer organization at a very low level that make up the computer architecture. The second type of architecture that is needed for parallel computing is that of a communication architecture, or a way in which the individual processors can communicate and work together to do useful things, this includes synchronization among the processors. Also, the communication architecture is closely linked with the hardware of the parallel computer because the hardware must be able to accommodate the communication operations.

To see the trends of parallel computers becoming widespread in use this [http://www.islandnet.com/~kpolsson/micropro/ site] provides a chronology of microprocessors.

== Parallel Programming Models ==
The communication architecture dictates communication operations that the user software is allowed to perform. Parallel programming models layout the framework for the way in which the communication operations take place. In the beginning of parallel computing a single programming model was used for one system. The hardware was constructed to support only the communication operations that the specific programming model used. However, As parallel computing became more widespread there has been a convergence of programming models. Many models have become integrated and have been used together. Many parallel programming models exist, the most common of which are described below.

=== Shared Address Space ===
Shared address space programming can be understood best when related to a message or bulletin board. Anyone can post a message on this board, and anyone can read what others have written. The key to the shared address space model is the fact that all of the vital information is posted in shared locations that can be accessed by all with memory operations such as loads and stores.

[http://www.sas.com/grid SAS grid computing.]

[http://www.openmp.org/drupal/ OpenMP - shared-memory parallel programming.]

=== Message Passing ===
Message passing machines convey information in a way similar to that of a phone call or a letter. There are very specific events that trigger the movement of information from a unique sender to a unique receiver. The sending process sends the data along with information that directs the message to the correct receiving process.

A widespread interface for message passing machines is the message passing interface, or MPI. More information on the MPI can be found at the [http://www.mpi-forum.org MPI forum].

=== Data Parallel ===
Data parallel programming is by far the most regulated of the three parallel programming models. In this type of programming work is carried out on different elements of a data set by different operators. Once all operations are finished information is exchanged among all, basically after the work is done a global data organization is performed.

Intel's [http://www.intel.com/design/archives/Processors/mmx/ MMX] processors are a good example of data parallel processing.

These [http://www.vcpc.univie.ac.at/activities/tutorials/HPF/lectures/html/jhm.2.html lectures] speak on the topic of data parallelism in the F90 and HPF (fortran).

== Convergence ==
As parallel computer architecture has matured it has become apparent that the parallel programming models have all been created to accomplish the same sorts of problems. For this reason the division between the programming models has become unclear since hardware for recent parallel computers is able to support a variety of programming models. This is due to the fact that the hardware
primitives have basically become the same for all parallel architectures. A good example of this convergence is the way in which the message-passing machines of today are very similar to nonuniform memory access machines (NUMA), a type of machine that uses the shared address space model.

A 2003 comparison of parallel computing problems solved with message passing and shared address space models can be viewed [http://portal.acm.org/citation.cfm?id=763446 here].

This [http://www.cs.princeton.edu/picasso/seminarsS04/MPI_Day1.pdf paper], which was written in 2005, discusses the need to integrate message passing into shared address space models.

A 1999 [http://www.cs.berkeley.edu/~culler/cs258-s99/slides/lec02/index.htm presentation] on the convergence of parallel architectures.

== Layers of Abstraction ==
There are many layers of abstraction between the application and the hardware. The user program is written using a certain programming model. This programming model specifies the way in which the information is communicated among different pieces of the program. The compiler and/or libraries provide the layer of abstraction between the programming model and the available hardware primitives. Figure 1.13 on pg. 27 of "Parallel Computer Architecture: A Hardware/Software Approach" displays these layers of abstraction well. However, it seems that with the convergence of programming models and the use of data parallelism in message passing and shared address space programming the layer of abstraction containing the programming models should be revised if not edited out completely.

== Links to Trends in Parallel Computing ==
[http://www-unix.mcs.anl.gov/dbpp/text/node7.html Parallelism and Computing] 
[http://www.intel.com/cd/ids/developer/asmo-na/eng/segments/hpc/95223.htm?page=3 Intel - Trends in Distributed Computing] 
[http://css.psu.edu/news/nlsp98/progtrends.html Programming Trends in High Performance Computing] 
[http://www.cs.cmu.edu/~scandal/research-groups.html Supercomputing and Parallel Computing Research Groups]

== References ==
Culler, David and Singh, Jaswinder. "Parallel Computer Architecture: A Hardware/Software Approach". ISBN 1-55860-343-3

CSC/ECE 506 Fall 2007/wiki1 6 r8e

2007-09-11T00:09:49Z

Refuller:

== Communication Architecture ==
Parallel computers must have two types of architectures. One of these is this computer architecture, or that of the individual processors that are interconnected. Each of these processors has an instruction set architecture and microarchitecture, or a system of computer organization at a very low level that make up the computer architecture. The second type of architecture that is needed for parallel computing is that of a communication architecture, or a way in which the individual processors can communicate and work together to do useful things, this includes synchronization among the processors. Also, the communication architecture is closely linked with the hardware of the parallel computer because the hardware must be able to accommodate the communication operations.

To see the trends of parallel computers becoming widespread in use this [http://www.islandnet.com/~kpolsson/micropro/ site] provides a chronology of microprocessors.

== Parallel Programming Models ==
The communication architecture dictates communication operations that the user software is allowed to perform. Parallel programming models layout the framework for the way in which the communication operations take place. In the beginning of parallel computing a single programming model was used for one system. The hardware was constructed to support only the communication operations that the specific programming model used. However, As parallel computing became more widespread there has been a convergence of programming models. Many models have become integrated and have been used together. Many parallel programming models exist, the most common of which are described below.

=== Shared Address Space ===
Shared address space programming can be understood best when related to a message or bulletin board. Anyone can post a message on this board, and anyone can read what others have written. The key to the shared address space model is the fact that all of the vital information is posted in shared locations that can be accessed by all with memory operations such as loads and stores.

[http://www.sas.com/grid SAS grid computing.]

[http://www.openmp.org/drupal/ OpenMP - shared-memory parallel programming.]

=== Message Passing ===
Message passing machines convey information in a way similar to that of a phone call or a letter. There are very specific events that trigger the movement of information from a unique sender to a unique receiver. The sending process sends the data along with information that directs the message to the correct receiving process.

A widespread interface for message passing machines is the message passing interface, or MPI. More information on the MPI can be found at the [http://www.mpi-forum.org MPI forum].

=== Data Parallel ===
Data parallel programming is by far the most regulated of the three parallel programming models. In this type of programming work is carried out on different elements of a data set by different operators. Once all operations are finished information is exchanged among all, basically after the work is done a global data organization is performed.

Intel's [http://www.intel.com/design/archives/Processors/mmx/ MMX] processors are a good example of data parallel processing.

These [http://www.vcpc.univie.ac.at/activities/tutorials/HPF/lectures/html/jhm.2.html lectures] speak on the topic of data parallelism in the F90 and HPF (fortran).

== Convergence ==
As parallel computer architecture has matured it has become apparent that the parallel programming models have all been created to accomplish the same sorts of problems. For this reason the division between the programming models has become unclear since hardware for recent parallel computers is able to support a variety of programming models. This is due to the fact that the hardware
primitives have basically become the same for all parallel architectures. A good example of this convergence is the way in which the message-passing machines of today are very similar to nonuniform memory access machines (NUMA), a type of machine that uses the shared address space model.

A 2003 comparison of parallel computing problems solved with message passing and shared address space models can be viewed [http://portal.acm.org/citation.cfm?id=763446 here].

This [http://www.cs.princeton.edu/picasso/seminarsS04/MPI_Day1.pdf paper], which was written in 2005, discusses the need to integrate message passing into shared address space models.

A 1999 [http://www.cs.berkeley.edu/~culler/cs258-s99/slides/lec02/index.htm presentation] on the convergence of parallel architectures.

== Layers of Abstraction ==
There are many layers of abstraction between the application and the hardware. The user program is written using a certain programming model. This programming model specifies the way in which the information is communicated among different pieces of the program. The compiler and/or libraries provide the layer of abstraction between the programming model and the available hardware primitives. Figure 1.13 on pg. 27 of "Parallel Computer Architecture: A Hardware/Software Approach" displays these layers of abstraction well. However, it seems that with the convergence of programming models and the use of data parallelism in message passing and shared address space programming the layer of abstraction containing the programming models should be revised if not edited out completely.

== Links to Trends in Parallel Computing ==
[http://www-unix.mcs.anl.gov/dbpp/text/node7.html Parallelism and Computing] 
[http://www.intel.com/cd/ids/developer/asmo-na/eng/segments/hpc/95223.htm?page=3 Intel - Trends in Distributed Computing] 
[http://css.psu.edu/news/nlsp98/progtrends.html Programming Trends in High Performance Computing] 
[http://www.cs.cmu.edu/~scandal/research-groups.html Supercomputing and Parallel Computing Research Groups]

== Bibliography ==

CSC/ECE 506 Fall 2007/wiki1 6 r8e

2007-09-10T23:33:42Z

Refuller:

== Communication Architecture ==
Parallel computers must have two types of architectures. One of these is this computer architecture, or that of the individual processors that are interconnected. Each of these processors has an instruction set architecture and microarchitecture, or a system of computer organization at a very low level that make up the computer architecture. The second type of architecture that is needed for parallel computing is that of a communication architecture, or a way in which the individual processors can communicate and work together to do useful things, this includes synchronization among the processors. Also, the communication architecture is closely linked with the hardware of the parallel computer because the hardware must be able to accommodate the communication operations.

To see the trends of parallel computers becoming widespread in use this [http://www.islandnet.com/~kpolsson/micropro/ site] provides a chronology of microprocessors.

=== Layers of Abstraction ===

== Parallel Programming Models ==
The communication architecture dictates communication operations that the user software is allowed to perform. Parallel programming models layout the framework for the way in which the communication operations take place. In the beginning of parallel computing a single programming model was used for one sytem. The hardware was constructed to support only the communication operations that the specific programming model used. However, As parallel computing became more widespread there has been a convergence of programming models. Many models have become integrated and have been used together. Many parallel programming models exist, the most common of which are described below.

=== Shared Address Space ===
Shared address space programming can be understood best when related to a message or bulletin board. Anyone can post a message on this board, and anyone can read what others have written. The key to the shared address space model is the fact that all of the vital information is posted in shared locations that can be accessed by all with memory operations such as loads and stores.

[http://www.sas.com/grid SAS grid computing.]

[http://www.openmp.org/drupal/ OpenMP - shared-memory parallel programming.]

=== Message Passing ===
Message passing machines convey information in a way similar to that of a phone call or a letter. There are very specific events that trigger the movement of information from a unique sender to a unique receiver. The sending process sends the data along with information that directs the message to the correct receiving process.

A widespread interface for message passing machines is the message passing interface, or MPI. More information on the MPI can be found at the [http://www.mpi-forum.org MPI forum].

=== Data Parallel ===
Data parallel programming is by far the most regulated of the three parallel programming models. In this type of programming work is carried out on different elements of a data set by different operators. Once all operations are finished information is exchanged among all, basically after the work is done a global data organization is performed. Data parallelism

Intel's [http://www.intel.com/design/archives/Processors/mmx/ MMX] processors are a good example of data parallel processing.

These [http://www.vcpc.univie.ac.at/activities/tutorials/HPF/lectures/html/jhm.2.html lectures] speak on the topic of data parallelism in the F90 and HPF (fortran).

== Convergence ==
As parallel computer architecture has matured it has become apparent that the parallel programming models have all been created to accomplish the same sorts of problems. For this reason the division between the programming models has become unclear since hardware for recent parallel computers is able to support a variety of programming models. This is due to the fact that the hardware
primitives have basically become the same for all parallel architectures. A good example of this convergence is the way in which the message-passing machines of today are very similar to nonuniform memory access machines (NUMA), a type of machine that uses the shared address space model.

A 2003 comparison of parallel computing problems solved with message passing and shared address space models can be viewed [http://portal.acm.org/citation.cfm?id=763446 here].

This [http://www.cs.princeton.edu/picasso/seminarsS04/MPI_Day1.pdf paper], which was written in 2005, discusses the need to integrate message passing into shared address space models.

A 1999 [http://www.cs.berkeley.edu/~culler/cs258-s99/slides/lec02/index.htm presentation] on the convergence of parallel architectures.

== Links to Trends in Parallel Computing ==
[http://www-unix.mcs.anl.gov/dbpp/text/node7.html Parallelism and Computing] 
[http://www.intel.com/cd/ids/developer/asmo-na/eng/segments/hpc/95223.htm?page=3 Intel - Trends in Distributed Computing] 
[http://css.psu.edu/news/nlsp98/progtrends.html Programming Trends in High Performance Computing] 
[http://www.cs.cmu.edu/~scandal/research-groups.html Supercomputing and Parallel Computing Research Groups]

== Bibliography ==

CSC/ECE 506 Fall 2007/wiki1 6 r8e

2007-09-10T23:27:24Z

Refuller:

== Communication Architecture ==
Parallel computers must have two types of architectures. One of these is this computer architecture, or that of the individual processors that are interconnected. Each of these processors has an instruction set architecture and microarchitecture, or a system of computer organization at a very low level that make up the computer architecture. The second type of architecture that is needed for parallel computing is that of a communication architecture, or a way in which the individual processors can communicate and work together to do useful things, this includes synchronization among the processors. Also, the communication architecture is closely linked with the hardware of the parallel computer because the hardware must be able to accommodate the communication operations.

To see the trends of parallel computers becoming widespread in use this [http://www.islandnet.com/~kpolsson/micropro/ site] provides a chronology of microprocessors.

== Parallel Programming Models ==
The communication architecture dictates communication operations that the user software is allowed to perform. Parallel programming models layout the framework for the way in which the communication operations take place. In the beginning of parallel computing a single programming model was used for one sytem. The hardware was constructed to support only the communication operations that the specific programming model used. However, As parallel computing became more widespread there has been a convergence of programming models. Many models have become integrated and have been used together. Many parallel programming models exist, the most common of which are described below.

=== Shared Address Space ===
Shared address space programming can be understood best when related to a message or bulletin board. Anyone can post a message on this board, and anyone can read what others have written. The key to the shared address space model is the fact that all of the vital information is posted in shared locations that can be accessed by all with memory operations such as loads and stores.

[http://www.sas.com/grid SAS grid computing.]

[http://www.openmp.org/drupal/ OpenMP - shared-memory parallel programming.]

=== Message Passing ===
Message passing machines convey information in a way similar to that of a phone call or a letter. There are very specific events that trigger the movement of information from a unique sender to a unique receiver. The sending process sends the data along with information that directs the message to the correct receiving process.

A widespread interface for message passing machines is the message passing interface, or MPI. More information on the MPI can be found at the [http://www.mpi-forum.org MPI forum].

=== Data Parallel ===
Data parallel programming is by far the most regulated of the three parallel programming models. In this type of programming work is carried out on different elements of a data set by different operators. Once all operations are finished information is exchanged among all, basically after the work is done a global data organization is performed. Data parallelism

Intel's [http://www.intel.com/design/archives/Processors/mmx/ MMX] processors are a good example of data parallel processing.

These [http://www.vcpc.univie.ac.at/activities/tutorials/HPF/lectures/html/jhm.2.html lectures] speak on the topic of data parallelism in the F90 and HPF (fortran).

== Convergence ==
As parallel computer architecture has matured it has become apparent that the parallel programming models have all been created to accomplish the same sorts of problems. For this reason the division between the programming models has become unclear since hardware for recent parallel computers is able to support a variety of programming models. This is due to the fact that the hardware
primitives have basically become the same for all parallel architectures. A good example of this convergence is the way in which the message-passing machines of today are very similar to nonuniform memory access machines (NUMA), a type of machine that uses the shared address space model.

A 2003 comparison of parallel computing problems solved with message passing and shared address space models can be viewed [http://portal.acm.org/citation.cfm?id=763446 here].

This [http://www.cs.princeton.edu/picasso/seminarsS04/MPI_Day1.pdf paper], which was written in 2005, discusses the need to integrate message passing into shared address space models.

A 1999 [http://www.cs.berkeley.edu/~culler/cs258-s99/slides/lec02/index.htm presentation] on the convergence of parallel architectures.

== Links to Trends in Parallel Computing ==
[http://www-unix.mcs.anl.gov/dbpp/text/node7.html Parallelism and Computing] 
[http://www.intel.com/cd/ids/developer/asmo-na/eng/segments/hpc/95223.htm?page=3 Intel - Trends in Distributed Computing] 
[http://css.psu.edu/news/nlsp98/progtrends.html Programming Trends in High Performance Computing]

CSC/ECE 506 Fall 2007/wiki1 6 r8e

2007-09-10T23:21:52Z

Refuller: /* Convergence */

== Communication Architecture ==
Parallel computers must have two types of architectures. One of these is this computer architecture, or that of the individual processors that are interconnected. Each of these processors has an instruction set architecture and microarchitecture, or a system of computer organization at a very low level that make up the computer architecture. The second type of architecture that is needed for parallel computing is that of a communication architecture, or a way in which the individual processors can communicate and work together to do useful things, this includes synchronization among the processors. Also, the communication architecture is closely linked with the hardware of the parallel computer because the hardware must be able to accommodate the communication operations.

To see the trends of parallel computers becoming widespread in use this [http://www.islandnet.com/~kpolsson/micropro/ site] provides a chronology of microprocessors.

== Parallel Programming Models ==
The communication architecture dictates communication operations that the user software is allowed to perform. Parallel programming models layout the framework for the way in which the communication operations take place. In the beginning of parallel computing a single programming model was used for one sytem. The hardware was constructed to support only the communication operations that the specific programming model used. However, As parallel computing became more widespread there has been a convergence of programming models. Many models have become integrated and have been used together. Many parallel programming models exist, the most common of which are described below.

=== Shared Address Space ===
Shared address space programming can be understood best when related to a message or bulletin board. Anyone can post a message on this board, and anyone can read what others have written. The key to the shared address space model is the fact that all of the vital information is posted in shared locations that can be accessed by all with memory operations such as loads and stores.

[http://www.sas.com/grid SAS grid computing.]

[http://www.openmp.org/drupal/ OpenMP - shared-memory parallel programming.]

=== Message Passing ===
Message passing machines convey information in a way similar to that of a phone call or a letter. There are very specific events that trigger the movement of information from a unique sender to a unique receiver. The sending process sends the data along with information that directs the message to the correct receiving process.

A widespread interface for message passing machines is the message passing interface, or MPI. More information on the MPI can be found at the [http://www.mpi-forum.org MPI forum].

=== Data Parallel ===
Data parallel programming is by far the most regulated of the three parallel programming models. In this type of programming work is carried out on different elements of a data set by different operators. Once all operations are finished information is exchanged among all, basically after the work is done a global data organization is performed.

Intel's [http://www.intel.com/design/archives/Processors/mmx/ MMX] processors are a good example of data parallel processing.

These [http://www.vcpc.univie.ac.at/activities/tutorials/HPF/lectures/html/jhm.2.html lectures] speak on the topic of data parallelism in the F90 and HPF (fortran).

== Convergence ==
As parallel computer architecture has matured it has become apparent that the parallel programming models have all been created to accomplish the same sorts of problems. For this reason the division between the programming models has become unclear since hardware for recent parallel computers is able to support a variety of programming models. This is due to the fact that the hardware
primitives have basically become the same for all parallel architectures. A good example of this convergence is the way in which the message-passing machines of today are very similar to nonuniform memory access machines (NUMA), a type of machine that uses the shared address space model.

A 2003 comparison of parallel computing problems solved with message passing and shared address space models can be viewed [http://portal.acm.org/citation.cfm?id=763446 here].

This [http://www.cs.princeton.edu/picasso/seminarsS04/MPI_Day1.pdf paper], which was written in 2005, discusses the need to integrate message passing into shared address space models.

A 1999 [http://www.cs.berkeley.edu/~culler/cs258-s99/slides/lec02/index.htm presentation] on the convergence of parallel architectures.

=== Layers of Abstraction ===

== Links to Trends in Parallel Computing ==
[http://www-unix.mcs.anl.gov/dbpp/text/node7.html Parallelism and Computing] 
[http://www.intel.com/cd/ids/developer/asmo-na/eng/segments/hpc/95223.htm?page=3 Intel - Trends in Distributed Computing] 
[http://css.psu.edu/news/nlsp98/progtrends.html Programming Trends in High Performance Computing]

CSC/ECE 506 Fall 2007/wiki1 6 r8e

2007-09-10T23:06:52Z

Refuller:

== Communication Architecture ==
Parallel computers must have two types of architectures. One of these is this computer architecture, or that of the individual processors that are interconnected. Each of these processors has an instruction set architecture and microarchitecture, or a system of computer organization at a very low level that make up the computer architecture. The second type of architecture that is needed for parallel computing is that of a communication architecture, or a way in which the individual processors can communicate and work together to do useful things, this includes synchronization among the processors. Also, the communication architecture is closely linked with the hardware of the parallel computer because the hardware must be able to accommodate the communication operations.

To see the trends of parallel computers becoming widespread in use this [http://www.islandnet.com/~kpolsson/micropro/ site] provides a chronology of microprocessors.

== Parallel Programming Models ==
The communication architecture dictates communication operations that the user software is allowed to perform. Parallel programming models layout the framework for the way in which the communication operations take place. In the beginning of parallel computing a single programming model was used for one sytem. The hardware was constructed to support only the communication operations that the specific programming model used. However, As parallel computing became more widespread there has been a convergence of programming models. Many models have become integrated and have been used together. Many parallel programming models exist, the most common of which are described below.

=== Shared Address Space ===
Shared address space programming can be understood best when related to a message or bulletin board. Anyone can post a message on this board, and anyone can read what others have written. The key to the shared address space model is the fact that all of the vital information is posted in shared locations that can be accessed by all with memory operations such as loads and stores.

[http://www.sas.com/grid SAS grid computing.]

[http://www.openmp.org/drupal/ OpenMP - shared-memory parallel programming.]

=== Message Passing ===
Message passing machines convey information in a way similar to that of a phone call or a letter. There are very specific events that trigger the movement of information from a unique sender to a unique receiver. The sending process sends the data along with information that directs the message to the correct receiving process.

A widespread interface for message passing machines is the message passing interface, or MPI. More information on the MPI can be found at the [http://www.mpi-forum.org MPI forum].

=== Data Parallel ===
Data parallel programming is by far the most regulated of the three parallel programming models. In this type of programming work is carried out on different elements of a data set by different operators. Once all operations are finished information is exchanged among all, basically after the work is done a global data organization is performed.

Intel's [http://www.intel.com/design/archives/Processors/mmx/ MMX] processors are a good example of data parallel processing.

These [http://www.vcpc.univie.ac.at/activities/tutorials/HPF/lectures/html/jhm.2.html lectures] speak on the topic of data parallelism in the F90 and HPF (fortran).

== Convergence ==
As parallel computer architecture has matured it has become apparent that the parallel programming models have all been created to accomplish the same sorts of problems. For this reason the division between the programming models has become unclear since hardware for recent parallel computers is able to support a variety of programming models. This is due to the fact that the hardware
primitives have basically become the same for all parallel architectures. A good example of this convergence is the way in which the message-passing machines of today are very similar to nonuniform memory access machines (NUMA), a type of machine that uses the shared address space model.

A 2003 comparison of parallel computing problems solved with message passing and shared address space models can be viewed [http://portal.acm.org/citation.cfm?id=763446 here].

This [http://www.cs.princeton.edu/picasso/seminarsS04/MPI_Day1.pdf paper], which was written in 2005, discusses the need to integrate message passing into shared address space models.

=== Layers of Abstraction ===

== Links to Trends in Parallel Computing ==
[http://www-unix.mcs.anl.gov/dbpp/text/node7.html Parallelism and Computing] 
[http://www.intel.com/cd/ids/developer/asmo-na/eng/segments/hpc/95223.htm?page=3 Intel - Trends in Distributed Computing] 
[http://css.psu.edu/news/nlsp98/progtrends.html Programming Trends in High Performance Computing]

CSC/ECE 506 Fall 2007/wiki1 6 r8e

2007-09-10T23:06:07Z

Refuller:

== Communication Architecture ==
Parallel computers must have two types of architectures. One of these is this computer architecture, or that of the individual processors that are interconnected. Each of these processors has an instruction set architecture and microarchitecture, or a system of computer organization at a very low level that make up the computer architecture. The second type of architecture that is needed for parallel computing is that of a communication architecture, or a way in which the individual processors can communicate and work together to do useful things, this includes synchronization among the processors. Also, the communication architecture is closely linked with the hardware of the parallel computer because the hardware must be able to accommodate the communication operations.

To see the trends of parallel computers becoming widespread in use this [http://www.islandnet.com/~kpolsson/micropro/ site] provides a chronology of microprocessors.

== Parallel Programming Models ==
The communication architecture dictates communication operations that the user software is allowed to perform. Parallel programming models layout the framework for the way in which the communication operations take place. In the beginning of parallel computing a single programming model was used for one sytem. The hardware was constructed to support only the communication operations that the specific programming model used. However, As parallel computing became more widespread there has been a convergence of programming models. Many models have become integrated and have been used together. Many parallel programming models exist, the most common of which are described below.

=== Shared Address Space ===
Shared address space programming can be understood best when related to a message or bulletin board. Anyone can post a message on this board, and anyone can read what others have written. The key to the shared address space model is the fact that all of the vital information is posted in shared locations that can be accessed by all with memory operations such as loads and stores.

[http://www.sas.com/grid SAS grid computing.]

[http://www.openmp.org/drupal/ OpenMP - shared-memory parallel programming.]

=== Message Passing ===
Message passing machines convey information in a way similar to that of a phone call or a letter. There are very specific events that trigger the movement of information from a unique sender to a unique receiver. The sending process sends the data along with information that directs the message to the correct receiving process.

A widespread interface for message passing machines is the message passing interface, or MPI. More information on the MPI can be found at the [http://www.mpi-forum.org MPI forum].

=== Data Parallel ===
Data parallel programming is by far the most regulated of the three parallel programming models. In this type of programming work is carried out on different elements of a data set by different operators. Once all operations are finished information is exchanged among all, basically after the work is done a global data organization is performed.

Intel's [http://www.intel.com/design/archives/Processors/mmx/ MMX] processors are a good example of data parallel processing.

These [http://www.vcpc.univie.ac.at/activities/tutorials/HPF/lectures/html/jhm.2.html lectures] speak on the topic of data parallelism in the F90 and HPF (fortran).

== Convergence ==
As parallel computer architecture has matured it has become apparent that the parallel programming models have all been created to accomplish the same sorts of problems. For this reason the division between the programming models has become unclear since hardware for recent parallel computers is able to support a variety of programming models. This is due to the fact that the hardware
primitives have basically become the same for all parallel architectures. A good example of this convergence is the way in which the message-passing machines of today are very similar to nonuniform memory access machines (NUMA), a type of machine that uses the shared address space model.

A 2003 comparison of parallel computing problems solved with message passing and shared address space models can be viewed [http://portal.acm.org/citation.cfm?id=763446 here].

This [http://www.cs.princeton.edu/picasso/seminarsS04/MPI_Day1.pdf paper], which was written in 2005, discusses the need to integrate message passing into shared address space models.

=== Layers of Abstraction ===

== Links to Trends in Parallel Computing ==
[http://www-unix.mcs.anl.gov/dbpp/text/node7.html Parallelism and Computing]
[http://www.intel.com/cd/ids/developer/asmo-na/eng/segments/hpc/95223.htm?page=3 Intel - Trends in Distributed Computing]
[http://css.psu.edu/news/nlsp98/progtrends.html Programming Trends in High Performance Computing]

CSC/ECE 506 Fall 2007/wiki1 6 r8e

2007-09-10T22:47:07Z

Refuller:

== Communication Architecture ==
Parallel computers must have two types of architectures. One of these is this computer architecture, or that of the individual processors that are interconnected. Each of these processors has an instruction set architecture and microarchitecture, or a system of computer organization at a very low level that make up the computer architecture. The second type of architecture that is needed for parallel computing is that of a communication architecture, or a way in which the individual processors can communicate and work together to do useful things, this includes synchronization among the processors. Also, the communication architecture is closely linked with the hardware of the parallel computer because the hardware must be able to accommodate the communication operations.

To see the trends of parallel computers becoming widespread in use this [http://www.islandnet.com/~kpolsson/micropro/ site] provides a chronology of microprocessors.

== Parallel Programming Models ==
The communication architecture dictates communication operations that the user software is allowed to perform. Parallel programming models layout the framework for the way in which the communication operations take place. In the beginning of parallel computing a single programming model was used for one sytem. The hardware was constructed to support only the communication operations that the specific programming model used. However, As parallel computing became more widespread there has been a convergence of programming models. Many models have become integrated and have been used together. Many parallel programming models exist, the most common of which are described below.

=== Shared Address Space ===
Shared address space programming can be understood best when related to a message or bulletin board. Anyone can post a message on this board, and anyone can read what others have written. The key to the shared address space model is the fact that all of the vital information is posted in shared locations that can be accessed by all with memory operations such as loads and stores.

[http://www.sas.com/grid SAS grid computing.]

=== Message Passing ===
Message passing machines convey information in a way similar to that of a phone call or a letter. There are very specific events that trigger the movement of information from a unique sender to a unique receiver. The sending process sends the data along with information that directs the message to the correct receiving process.

A widespread interface for message passing machines is the message passing interface, or MPI. More information on the MPI can be found at the [http://www.mpi-forum.org MPI forum].

=== Data Parallel ===
Data parallel programming is by far the most regulated of the three parallel programming models. In this type of programming work is carried out on different elements of a data set by different operators. Once all operations are finished information is exchanged among all, basically after the work is done a global data organization is performed.

Intel's [http://www.intel.com/design/archives/Processors/mmx/ MMX] processors are a good example of using data parallel processing.

These [http://www.vcpc.univie.ac.at/activities/tutorials/HPF/lectures/html/jhm.2.html lectures] speak on the topic of data parallelism in the F90 and HPF (fortran).

== Convergence ==
As parallel computer architecture has matured it has become apparent that the parallel programming models have all been created to accomplish the same sorts of problems. For this reason the division between the programming models has become unclear since hardware for recent parallel computers is able to support a variety of programming models. A good example of this convergence is the way in which the message-passing machines of today are very similar to nonuniform memory access machines (NUMA), a type of machine that uses the shared address space model.

A 2003 comparison of parallel computing problems solved with message passing and shared address space models can be viewed [http://portal.acm.org/citation.cfm?id=763446 here].

This [http://www.cs.princeton.edu/picasso/seminarsS04/MPI_Day1.pdf paper], which was written in 2005, discusses the need to integrate message passing into shared address space models.

=== Layers of Abstraction ===

== Links to Trends in Parallel Computing ==
[http://www-unix.mcs.anl.gov/dbpp/text/node7.html]

CSC/ECE 506 Fall 2007/wiki1 6 r8e

2007-09-10T22:44:32Z

Refuller:

== Communication Architecture ==
Parallel computers must have two types of architectures. One of these is this computer architecture, or that of the individual processors that are interconnected. Each of these processors has an instruction set architecture and microarchitecture, or a system of computer organization at a very low level that make up the computer architecture. The second type of architecture that is needed for parallel computing is that of a communication architecture, or a way in which the individual processors can communicate and work together to do useful things, this includes synchronization among the processors. Also, the communication architecture is closely linked with the hardware of the parallel computer because the hardware must be able to accommodate the communication operations.

To see the trends of parallel computers becoming widespread in use this [http://www.islandnet.com/~kpolsson/micropro/ site] provides a chronology of microprocessors.

== Parallel Programming Models ==
The communication architecture dictates communication operations that the user software is allowed to perform. Parallel programming models layout the framework for the way in which the communication operations take place. In the beginning of parallel computing a single programming model was used for one sytem. The hardware was constructed to support only the communication operations that the specific programming model used. However, As parallel computing became more widespread there has been a convergence of programming models. Many models have become integrated and have been used together. Many parallel programming models exist, the most common of which are described below.

=== Shared Address Space ===
Shared address space programming can be understood best when related to a message or bulletin board. Anyone can post a message on this board, and anyone can read what others have written. The key to the shared address space model is the fact that all of the vital information is posted in shared locations that can be accessed by all with memory operations such as loads and stores.

[http://www.sas.com/grid SAS grid computing.]

=== Message Passing ===
Message passing machines convey information in a way similar to that of a phone call or a letter. There are very specific events that trigger the movement of information from a unique sender to a unique receiver. The sending process sends the data along with information that directs the message to the correct receiving process.

A widespread interface for message passing machines is the message passing interface, or MPI. More information on the MPI can be found at the [http://www.mpi-forum.org MPI forum].

=== Data Parallel ===
Data parallel programming is by far the most regulated of the three parallel programming models. In this type of programming work is carried out on different elements of a data set by different operators. Once all operations are finished information is exchanged among all, basically after the work is done a global data organization is performed.

Intel's [http://www.intel.com/design/archives/Processors/mmx/ MMX] processors are a good example of using data parallel processing.

These [http://www.vcpc.univie.ac.at/activities/tutorials/HPF/lectures/html/jhm.2.html lectures] speak on the topic of data parallelism in the F90 and HPF (fortran).

== Convergence ==
As parallel computer architecture has matured it has become apparent that the parallel programming models have all been created to accomplish the same sorts of problems. For this reason the division between the programming models has become unclear since hardware for recent parallel computers is able to support a variety of programming models. A good example of this convergence is the way in which the message-passing machines of today are very similar to nonuniform memory access machines (NUMA), a type of machine that uses the shared address space model.

A 2003 comparison of parallel computing problems solved with message passing and shared address space models can be viewed [http://portal.acm.org/citation.cfm?id=763446 here].

This [http://www.cs.princeton.edu/picasso/seminarsS04/MPI_Day1.pdf paper], which was written in 2005, discusses the need to integrate message passing into shared address space models.

=== Layers of Abstraction ===

CSC/ECE 506 Fall 2007/wiki1 6 r8e

2007-09-10T22:39:05Z

Refuller:

== Communication Architecture ==
Parallel computers must have two types of architectures. One of these is this computer architecture, or that of the individual processors that are interconnected. Each of these processors has an instruction set architecture and microarchitecture, or a system of computer organization at a very low level that make up the computer architecture. The second type of architecture that is needed for parallel computing is that of a communication architecture, or a way in which the individual processors can communicate and work together to do useful things, this includes synchronization among the processors. Also, the communication architecture is closely linked with the hardware of the parallel computer because the hardware must be able to accommodate the communication operations.

To see the trends of parallel computers becoming widespread in use this [http://www.islandnet.com/~kpolsson/micropro/ site] provides a chronology of microprocessors.

== Parallel Programming Models ==
The communication architecture dictates communication operations that the user software is allowed to perform. Parallel programming models layout the framework for the way in which the communication operations take place. In the beginning of parallel computing a single programming model was used for one sytem. The hardware was constructed to support only the communication operations that the specific programming model used. However, As parallel computing became more widespread there has been a convergence of programming models. Many models have become integrated and have been used together. Many parallel programming models exist, the most common of which are described below.

=== Shared Address Space ===
Shared address space programming can be understood best when related to a message or bulletin board. Anyone can post a message on this board, and anyone can read what others have written. The key to the shared address space model is the fact that all of the vital information is posted in shared locations that can be accessed by all with memory operations such as loads and stores.

[http://www.sas.com/grid SAS grid computing.]

=== Message Passing ===
Message passing machines convey information in a way similar to that of a phone call or a letter. There are very specific events that trigger the movement of information from a unique sender to a unique receiver. The sending process sends the data along with information that directs the message to the correct receiving process.

A widespread interface for message passing machines is the message passing interface, or MPI. More information on the MPI can be found at the [http://www.mpi-forum.org MPI forum].

=== Data Parallel ===
Data parallel programming is by far the most regulated of the three parallel programming models. In this type of programming work is carried out on different elements of a data set by different operators. Once all operations are finished information is exchanged among all, basically after the work is done a global data organization is performed.

Intel's [http://www.intel.com/design/archives/Processors/mmx/ MMX] processors are a good example of using data parallel processing.

These [http://www.vcpc.univie.ac.at/activities/tutorials/HPF/lectures/html/jhm.2.html lectures] speak on the topic of data parallelism in the F90 and HPF (fortran).

== Convergence ==
As parallel computer architecture has matured it has become apparent that the parallel programming models have all been created to accomplish the same sorts of problems. For this reason the division between the programming models has become unclear since hardware for recent parallel computers is able to support a variety of programming models. A good example of this convergence is the way in which the message-passing machines of today are very similar to nonuniform memory access machines (NUMA), a type of machine that uses the shared address space model.

A 2003 comparison of parallel computing problems solved with message passing and shared address space models can be viewed [http://portal.acm.org/citation.cfm?id=763446 here].

This [http://www.cs.princeton.edu/picasso/seminarsS04/MPI_Day1.pdf paper], which was written in 2005, discusses the need to integrate message passing into shared address space models.

CSC/ECE 506 Fall 2007/wiki1 6 r8e

2007-09-10T22:37:00Z

Refuller:

== Communication Architecture ==
Parallel computers must have two types of architectures. One of these is this computer architecture, or that of the individual processors that are interconnected. Each of these processors has an instruction set architecture and microarchitecture, or a system of computer organization at a very low level that make up the computer architecture. The second type of architecture that is needed for parallel computing is that of a communication architecture, or a way in which the individual processors can communicate and work together to do useful things, this includes synchronization among the processors. Also, the communication architecture is closely linked with the hardware of the parallel computer because the hardware must be able to accommodate the communication operations.

To see the trends of parallel computers becoming widespread in use this [http://www.islandnet.com/~kpolsson/micropro/ site] provides a chronology of microprocessors.

== Parallel Programming Models ==
The communication architecture dictates communication operations that the user software is allowed to perform. Parallel programming models layout the framework for the way in which the communication operations take place. In the beginning of parallel computing a single programming model was used for one sytem. The hardware was constructed to support only the communication operations that the specific programming model used. However, As parallel computing became more widespread there has been a convergence of programming models. Many models have become integrated and have been used together. Many parallel programming models exist, the most common of which are described below.

=== Shared Address Space ===
Shared address space programming can be understood best when related to a message or bulletin board. Anyone can post a message on this board, and anyone can read what others have written. The key to the shared address space model is the fact that all of the vital information is posted in shared locations that can be accessed by all with memory operations such as loads and stores.

=== Message Passing ===
Message passing machines convey information in a way similar to that of a phone call or a letter. There are very specific events that trigger the movement of information from a unique sender to a unique receiver. The sending process sends the data along with information that directs the message to the correct receiving process.

A widespread interface for message passing machines is the message passing interface, or MPI. More information on the MPI can be found at the [http://www.mpi-forum.org MPI forum].

=== Data Parallel ===
Data parallel programming is by far the most regulated of the three parallel programming models. In this type of programming work is carried out on different elements of a data set by different operators. Once all operations are finished information is exchanged among all, basically after the work is done a global data organization is performed.

Intel's [http://www.intel.com/design/archives/Processors/mmx/ MMX] processors are a good example of using data parallel processing.

These [http://www.vcpc.univie.ac.at/activities/tutorials/HPF/lectures/html/jhm.2.html lectures] speak on the topic of data parallelism in the F90 and HPF (fortran).

== Convergence ==
As parallel computer architecture has matured it has become apparent that the parallel programming models have all been created to accomplish the same sorts of problems. For this reason the division between the programming models has become unclear since hardware for recent parallel computers is able to support a variety of programming models. A good example of this convergence is the way in which the message-passing machines of today are very similar to nonuniform memory access machines (NUMA), a type of machine that uses the shared address space model.

A 2003 comparison of parallel computing problems solved with message passing and shared address space models can be viewed [http://portal.acm.org/citation.cfm?id=763446 here].

This [http://www.cs.princeton.edu/picasso/seminarsS04/MPI_Day1.pdf paper], which was written in 2005, discusses the need to integrate message passing into shared address space models.

CSC/ECE 506 Fall 2007/wiki1 6 r8e

2007-09-06T02:33:38Z

Refuller:

CSC/ECE 506 Fall 2007/wiki1 6 r8e

2007-09-06T02:27:38Z

Refuller:

CSC/ECE 506 Fall 2007/wiki1 6 r8e

2007-09-06T01:55:13Z

Refuller:

CSC/ECE 506 Fall 2007/wiki1 6 r8e

2007-09-06T01:45:04Z

Refuller: /* Message Passing */

CSC/ECE 506 Fall 2007/wiki1 6 r8e

2007-09-06T01:44:37Z

Refuller: /* Message Passing */

CSC/ECE 506 Fall 2007/wiki1 6 r8e

2007-09-06T01:35:58Z

Refuller:

CSC/ECE 506 Fall 2007/wiki1 6 r8e

2007-09-05T22:39:59Z

Refuller:

== Communication Architecture Background ==
Parallel computers must have two types of architectures. One of these is this computer architecture, or that of the individual processors that are interconnected. Each of these processors has an instruction set architecture and microarchitecture, or a system of computer organization at a very low level that make up the computer architecture. The second type of architecture that is needed for parallel computing is that of a commuincation architecture, or a way in which the individual processors can communicate and work together to do useful things, this includes synchronization among the processors. Also, the commuincation architecture is closely linked with the hardware of the parallel computer because the hardware must be able to accomodate the communication operations.

== Parallel Programming Models ==
The communication architecture dictates communication operations that the user software is allowed to perform. Parallel programming models layout the framework for the way in which the communication operations take place. In the beginning of parallel computing a single programming model was used for one sytem. The hardware was constructed to support only the communication operations that the specific programming model used. However, As parallel computing became more widespread there has been a convergence of programming models. Many models have become integrated and have been used together. Many parallel programming models exist, the most common of which are described below.

=== Shared Address Space ===
Shared address space programming can be understood best when related to a message or bulletin board. Anyone can post a message on this board, and anyone can read what others have written. The key to the shared address space model is the fact that all of the vital information is posted in shared locations that can be accessed by all.

=== Message Passing ===
Message passing machines convey information in a way similar to that of a phone call or a letter. There are very specific events that trigger the movement of information from a unique sender to a unique receiver.

=== Data Parallel ===
Data parallel programming is by far the most regulated of the three parallel programming models. In this type of programming work is carried out on different elements of a data set by different operators. Once all operations are finished information is exchanged among all, basically after the work is done a global data organization is performed.