CSC/ECE 506 Spring 2011/ch2 JR - Revision history

Admin: /* Summary */

2011-04-18T04:44:00Z

Summary

← Older revision		Revision as of 04:44, 18 April 2011
Line 148:		Line 148:
	= Summary =		= Summary =

	As computer architectures have evolved, so have the parallel computing models. As discussed in [http://www.cesr.ncsu.edu/solihin Solihin], two models that have been prominent throughout computer history have been the ~~Shared Memory Model~~ and the ~~Message Passing Model~~. These are, by no means, the only programming models in use. In this supplemental, we discussed two other important models: the ~~Task Parallel Model~~ and the ~~Data Parallel Model~~. We also gave some context for how each of the programming models have emerged throughout the history of computer architecture. Each model has different strengths and weaknesses as discussed above.		As computer architectures have evolved, so have the parallel computing models. As discussed in [http://www.cesr.ncsu.edu/solihin Solihin], two models that have been prominent throughout computer history have been the shared-memory model and the message-passing model. These are, by no means, the only programming models in use. In this supplemental, we discussed two other important models: the task-parallel model and the data-parallel model. We also gave some context for how each of the programming models have emerged throughout the history of computer architecture. Each model has different strengths and weaknesses as discussed above.

	= Definitions =		= Definitions =

Admin: /* Summary */

2011-04-18T04:42:47Z

Summary

← Older revision		Revision as of 04:42, 18 April 2011
Line 148:		Line 148:
	= Summary =		= Summary =

	As computer architectures have evolved, so have the parallel computing models. As discussed in ~~Solihan~~ [~~SITE~~], two models that have been prominent throughout computer history have been the Shared Memory Model and the Message Passing Model. These are, by no means, the only programming models in use. In this supplemental, we discussed two other important models: the Task Parallel Model and the Data Parallel Model. We also gave some context for how each of the programming models have emerged throughout the history of computer architecture. Each model has different strengths and weaknesses as discussed above.		As computer architectures have evolved, so have the parallel computing models. As discussed in [http://www.cesr.ncsu.edu/solihin Solihin], two models that have been prominent throughout computer history have been the Shared Memory Model and the Message Passing Model. These are, by no means, the only programming models in use. In this supplemental, we discussed two other important models: the Task Parallel Model and the Data Parallel Model. We also gave some context for how each of the programming models have emerged throughout the history of computer architecture. Each model has different strengths and weaknesses as discussed above.

	= Definitions =		= Definitions =

Admin: /* Data Parallel Model vs Task Parallel Model */

2011-04-18T04:41:14Z

Data Parallel Model vs Task Parallel Model

← Older revision		Revision as of 04:41, 18 April 2011
Line 103:		Line 103:
	In the code above, work is divided into two parallel tasks. The first performs the element-wise addition of arrays ''b'' and ''c'' and stores the result in ''a.'' The other sums the elements of ''a.'' These tasks both operate on all elements of ''a'' (rather than on separate chunks), and the code executed by each thread is different (rather than identical).		In the code above, work is divided into two parallel tasks. The first performs the element-wise addition of arrays ''b'' and ''c'' and stores the result in ''a.'' The other sums the elements of ''a.'' These tasks both operate on all elements of ''a'' (rather than on separate chunks), and the code executed by each thread is different (rather than identical).

	= Data Parallel Model vs Task Parallel Model =		= Data-Parallel Model vs. Task-Parallel Model =
	One important feature of data-parallel programming model or data parallelism (SIMD) is the single control flow: there is only one control processor that directs the activities of all the processing elements. In stark contrast to this is task parallelism (MIMD: Multiple Instruction, Multiple Data): characterized by its multiple control flows, it allows the concurrent execution of multiple instruction streams, each manipulates its own data and services separate functions. Below is a contrast between the data parallelism and task parallelism models from wikipedia: [http://en.wikipedia.org/wiki/SIMD SIMD] and [http://en.wikipedia.org/wiki/MIMD MIMD]. In the following subsections we continue to compare and contrast different features of data-parallel model and task-parallel model to help reader understand the unique characteristics of data-parallel programming model.		One important feature of data-parallel programming model or data parallelism (SIMD) is the single control flow: there is only one control processor that directs the activities of all the processing elements. In stark contrast to this is task parallelism (MIMD: Multiple Instruction, Multiple Data): characterized by its multiple control flows, it allows the concurrent execution of multiple instruction streams, each manipulates its own data and services separate functions. Below is a contrast between the data parallelism and task parallelism models from wikipedia: [http://en.wikipedia.org/wiki/SIMD SIMD] and [http://en.wikipedia.org/wiki/MIMD MIMD]. In the following subsections we continue to compare and contrast different features of data-parallel model and task-parallel model to help reader understand the unique characteristics of data-parallel programming model.
	[[Image:Smid.png\|frame\|center\|425px\|contrast between data parallelism and task parallelism]]		[[Image:Smid.png\|frame\|center\|425px\|contrast between data parallelism and task parallelism]]

Admin: /* Combining with message passing and shared memory */

2011-04-18T04:40:41Z

Combining with message passing and shared memory

← Older revision		Revision as of 04:40, 18 April 2011
Line 65:		Line 65:

	== Combining with message passing and shared memory ==		== Combining with message passing and shared memory ==
	Although the shared memory and message passing models may be combined into hybrid approaches, the two models are fundamentally different ways of addressing the same problem (of access control to common data). In contrast, the data parallel model is concerned with a fundamentally different problem (how to divide work into parallel tasks). As such, the data parallel model may be used in conjunction with either the shared memory or the message passing model without conflict. In fact, Klaiber (1994) compares the performance of a number of data parallel programs implemented with both shared memory and message passing models.		Although the shared-memory and message-passing models may be combined into hybrid approaches, the two models are fundamentally different ways of addressing the same problem (of access control to common data). In contrast, the data parallel model is concerned with a fundamentally different problem (how to divide work into parallel tasks). As such, the data parallel model may be used in conjunction with either the shared-memory or the message-passing model without conflict. In fact, Klaiber (1994) compares the performance of a number of data parallel programs implemented with both shared-memory and message-passing models.


	One of the major advantages of combining the data parallel and message passing models is a reduction in the amount and complexity of communication required relative to a task parallel approach. Similarly, combining the data parallel and shared memory models tends to simplify and reduce the amount of synchronization required. Much as the shared memory model can benefit from specialized hardware, the data parallel programming model can as well. [[#Definitions \|''SIMD'']] processors are specifically designed to run data parallel algorithms. These processors perform a single instruction on many different data locations simultaneously. Modern examples include CUDA processors developed by nVidia and Cell processors developed by STI (Sony, Toshiba, and IBM). For the curious, example code for CUDA processors is provided in the Appendix. However, whereas the shared memory model can be a difficult and costly abstraction in the absence of hardware support, the data parallel model—like the message passing model—does not require hardware support.		One of the major advantages of combining the data parallel and message-passing models is a reduction in the amount and complexity of communication required relative to a task parallel approach. Similarly, combining the data parallel and shared-memory models tends to simplify and reduce the amount of synchronization required. Much as the shared-memory model can benefit from specialized hardware, the data parallel programming model can as well. [[#Definitions \|''SIMD'']] processors are specifically designed to run data parallel algorithms. These processors perform a single instruction on many different data locations simultaneously. Modern examples include CUDA processors developed by nVidia and Cell processors developed by STI (Sony, Toshiba, and IBM). For the curious, example code for CUDA processors is provided in the Appendix. However, whereas the shared-memory model can be a difficult and costly abstraction in the absence of hardware support, the data parallel model—like the message-passing model—does not require hardware support.

	= Task-Parallel Model =		= Task-Parallel Model =

Admin: /* Move to message passing in the 1980's */

2011-04-18T04:37:59Z

Move to message passing in the 1980's

← Older revision		Revision as of 04:37, 18 April 2011
Line 20:		Line 20:
	== Move to message passing in the 1980's ==		== Move to message passing in the 1980's ==

	Increasing the word size above 32 bits offered diminishing returns in terms of performance ([[#References\|Culler (1999), p. 15.]]), so there were fewer gains to be had for performance from processor improvements. At the same time, processors were becoming smaller and connections more efficient and connecting processors to do work in parallel became more viable. In the late 70's and early 80's Massively Parallel Processors (MPPs) emerged [http://www.intel.com/pressroom/kits/upcrc/parallelcomputing_backgrounder.pdf]. These consisted of separate computational units with their own memory and a link to the network that connects each unit. This structure allowed separate units to communicate the results of computations to each other without there needing to be a direct connection to each memory location. This change in architecture shifted the emphasis from bit-level parallelism to instruction-level parallelism, which involved increasing the number of instructions that could be executed at one time ([[#References\|Culler (1999), p. 15.]]). The message-passing model allowed programmers the ability to divide up instructions in order to take advantage of this architecture. This gave rise to the message passing model which allowed programmers the ability to divide up instructions in order to take advantage of this architecture.		Increasing the word size above 32 bits offered diminishing returns in terms of performance ([[#References\|Culler (1999), p. 15.]]), so there were fewer gains to be had for performance from processor improvements. At the same time, processors were becoming smaller and connections more efficient and connecting processors to do work in parallel became more viable. In the late 70's and early 80's Massively Parallel Processors (MPPs) emerged [http://www.intel.com/pressroom/kits/upcrc/parallelcomputing_backgrounder.pdf]. These consisted of separate computational units with their own memory and a link to the network that connects each unit. This structure allowed separate units to communicate the results of computations to each other without there needing to be a direct connection to each memory location. This change in architecture shifted the emphasis from bit-level parallelism to instruction-level parallelism, which involved increasing the number of instructions that could be executed at one time ([[#References\|Culler (1999), p. 15.]]). The message-passing model allowed programmers the ability to divide up instructions in order to take advantage of this architecture. This gave rise to the message-passing model which allowed programmers the ability to divide up instructions in order to take advantage of this architecture.

	Organizing each of the nodes in a MPP posed its own problems. Some MPPs organized the connections into hypercubes, but these proved difficult to build [http://en.wikipedia.org/wiki/MIMD]. One of the most successful were the Connection Machines [http://ed-thelen.org/comp-hist/vs-cm-1-2-5.html]. Other architectures used 2-D meshes. All of these strategies meant that each message might have to pass through a number of nodes before reaching its final destination. This introduced its own restrictions on performance becaues each node had to handle routing duties. As networking technology became faster and faster, and individual processors became more and more efficient, it became reasonable to connect separate computers across a network, which gave rise to cluster machines.		Organizing each of the nodes in a MPP posed its own problems. Some MPPs organized the connections into hypercubes, but these proved difficult to build [http://en.wikipedia.org/wiki/MIMD]. One of the most successful were the Connection Machines [http://ed-thelen.org/comp-hist/vs-cm-1-2-5.html]. Other architectures used 2-D meshes. All of these strategies meant that each message might have to pass through a number of nodes before reaching its final destination. This introduced its own restrictions on performance becaues each node had to handle routing duties. As networking technology became faster and faster, and individual processors became more and more efficient, it became reasonable to connect separate computers across a network, which gave rise to cluster machines.

Admin: /* Move to message passing in the 1980's */

2011-04-18T04:37:11Z

Move to message passing in the 1980's

← Older revision		Revision as of 04:37, 18 April 2011
Line 20:		Line 20:
	== Move to message passing in the 1980's ==		== Move to message passing in the 1980's ==

	Increasing the word size above 32 bits offered diminishing returns in terms of performance ([[#References\|Culler (1999), p. 15.]]), so there were fewer gains to be had for performance from processor improvements. At the same time, processors were becoming smaller and connections more efficient and connecting processors to do work in parallel became more viable. In the late 70's and early 80's Massively Parallel Processors (MPPs) emerged [http://www.intel.com/pressroom/kits/upcrc/parallelcomputing_backgrounder.pdf]. These consisted of separate computational units with their own memory and a link to the network that connects each unit. This structure allowed separate units to communicate the results of computations to each other without there needing to be a direct connection to each memory location. This change in architecture shifted the emphasis from bit-level parallelism to instruction-level parallelism, which involved increasing the number of instructions that could be executed at one time ([[#References\|Culler (1999), p. 15.]]). The message passing model allowed programmers the ability to divide up instructions in order to take advantage of this architecture. This gave rise to the message passing model which allowed programmers the ability to divide up instructions in order to take advantage of this architecture.		Increasing the word size above 32 bits offered diminishing returns in terms of performance ([[#References\|Culler (1999), p. 15.]]), so there were fewer gains to be had for performance from processor improvements. At the same time, processors were becoming smaller and connections more efficient and connecting processors to do work in parallel became more viable. In the late 70's and early 80's Massively Parallel Processors (MPPs) emerged [http://www.intel.com/pressroom/kits/upcrc/parallelcomputing_backgrounder.pdf]. These consisted of separate computational units with their own memory and a link to the network that connects each unit. This structure allowed separate units to communicate the results of computations to each other without there needing to be a direct connection to each memory location. This change in architecture shifted the emphasis from bit-level parallelism to instruction-level parallelism, which involved increasing the number of instructions that could be executed at one time ([[#References\|Culler (1999), p. 15.]]). The message-passing model allowed programmers the ability to divide up instructions in order to take advantage of this architecture. This gave rise to the message passing model which allowed programmers the ability to divide up instructions in order to take advantage of this architecture.

	Organizing each of the nodes in a MPP posed its own problems. Some MPPs organized the connections into hypercubes, but these proved difficult to build [http://en.wikipedia.org/wiki/MIMD]. One of the most successful were the Connection Machines [http://ed-thelen.org/comp-hist/vs-cm-1-2-5.html]. Other architectures used 2-D meshes. All of these strategies meant that each message might have to pass through a number of nodes before reaching its final destination. This introduced its own restrictions on performance becaues each node had to handle routing duties. As networking technology became faster and faster, and individual processors became more and more efficient, it became reasonable to connect separate computers across a network, which gave rise to cluster machines.		Organizing each of the nodes in a MPP posed its own problems. Some MPPs organized the connections into hypercubes, but these proved difficult to build [http://en.wikipedia.org/wiki/MIMD]. One of the most successful were the Connection Machines [http://ed-thelen.org/comp-hist/vs-cm-1-2-5.html]. Other architectures used 2-D meshes. All of these strategies meant that each message might have to pass through a number of nodes before reaching its final destination. This introduced its own restrictions on performance becaues each node had to handle routing duties. As networking technology became faster and faster, and individual processors became more and more efficient, it became reasonable to connect separate computers across a network, which gave rise to cluster machines.

Admin: /* Shared memory in the 1970's */

2011-04-18T04:33:04Z

Shared memory in the 1970's

← Older revision		Revision as of 04:33, 18 April 2011
Line 14:		Line 14:
	Development of the ILLIAC IV began in 1964 and wasn't finished until 1975 [http://en.wikipedia.org/wiki/ILLIAC_IV]. A central processor was connected to the main memory and delegated tasks to individual PE's, which each had their own memory cache. [http://archive.computerhistory.org/resources/text/Burroughs/Burroughs.ILLIAC%20IV.1974.102624911.pdf]. Each PE could operate either an 8-, 32- or 64-bit operand at a given time [http://archive.computerhistory.org/resources/text/Burroughs/Burroughs.ILLIAC%20IV.1974.102624911.pdf].		Development of the ILLIAC IV began in 1964 and wasn't finished until 1975 [http://en.wikipedia.org/wiki/ILLIAC_IV]. A central processor was connected to the main memory and delegated tasks to individual PE's, which each had their own memory cache. [http://archive.computerhistory.org/resources/text/Burroughs/Burroughs.ILLIAC%20IV.1974.102624911.pdf]. Each PE could operate either an 8-, 32- or 64-bit operand at a given time [http://archive.computerhistory.org/resources/text/Burroughs/Burroughs.ILLIAC%20IV.1974.102624911.pdf].

	The Cray machine was installed at Los Alamos National Laborartory in1976 by Cray Research, an offshoot of Control Data Corporation, and had similar performance to the ILLIAC IV [http://en.wikipedia.org/wiki/ILLIAC_IV]. ~~The~~ Cray machine relied heavily on the use of registers ~~for memory~~ instead of individual caches connected directly to processors like the ILLIAC IV. Each processor was connected to main memory and had a number of 64-bit registers used to perform operations [http://www.eecg.toronto.edu/~moshovos/ACA05/read/cray1.pdf].		The Cray machine was installed at Los Alamos National Laborartory in1976 by Cray Research, an offshoot of Control Data Corporation, and had similar performance to the ILLIAC IV [http://en.wikipedia.org/wiki/ILLIAC_IV]. For frequently-accessed data, the Cray machine relied heavily on the use of registers, instead of individual caches connected directly to processors like the ILLIAC IV. Each processor was connected to main memory and had a number of 64-bit registers used to perform operations [http://www.eecg.toronto.edu/~moshovos/ACA05/read/cray1.pdf].

	All of these early commercial machines relied on there being a direct connection between each processor and the memory. Not only did there have to be a direct connection, but each connection had to allow relatively similar access to each part of memory. As you increased the number of processors, you had to increase the number of connections, which meant that this architecture did not scale well. At the same time, processors were becoming more and more advanced. The first single chip microprocessor was introduced in 1971 [http://en.wikipedia.org/wiki/Intel_4004]. Due to these two pressures, there was a movement away from large shared memory supercomputers and towards distributed memory systems.		All of these early commercial machines relied on there being a direct connection between each processor and the memory. Not only did there have to be a direct connection, but each connection had to allow relatively similar access to each part of memory. As you increased the number of processors, you had to increase the number of connections, which meant that this architecture did not scale well. At the same time, processors were becoming more and more advanced. The first single chip microprocessor was introduced in 1971 [http://en.wikipedia.org/wiki/Intel_4004]. Due to these two pressures, there was a movement away from large shared memory supercomputers and towards distributed memory systems.

Admin: /* History */

2011-04-18T04:30:47Z

History

← Older revision		Revision as of 04:30, 18 April 2011
Line 8:		Line 8:

	= History =		= History =
	As computer architectures have evolved, so have parallel programming models. The two factors that influenced parallel computing performance improvement the most were the speed of the individual processor and the speed of the communication connections. These communication connections include access to memory (local and main), as well as communication between processors. The earliest advancements in parallel computers took advantage of bit-level parallelism from improvements made to chip design. These computers mainly used vector processing and each processor had direct, fast conections to memory. This gave rise to the shared memory programming model. As performance returns from this architecture diminished and the complexity of building machines with direct access to memory increased, the emphasis was placed on instruction-level parallelism, distributed memory systems, and the message passing model began to dominate. Most recently, with the move to cluster-based machines, there has been an increased emphasis on thread-level parallelism. This has corresponded to an increase interest in the data parallel programming model.		As computer architectures have evolved, so have parallel programming models. The two factors that influenced parallel computing performance improvement the most were the speed of the individual processor and the speed of the communication connections. These communication connections include access to memory (local and main), as well as communication between processors. The earliest advancements in parallel computers took advantage of bit-level parallelism from improvements made to chip design. These computers mainly used vector processing and each processor had direct, fast conections to memory. This gave rise to the shared memory programming model. As performance returns from this architecture diminished and the complexity of building machines with direct access to memory increased, the emphasis was placed on instruction-level parallelism, distributed memory systems, and the message passing model began to dominate. Most recently, with the move to cluster-based machines, there has been an increased emphasis on thread-level parallelism. This has corresponded to an increase interest in the data-parallel programming model.

	== Shared memory in the 1970's ==		== Shared memory in the 1970's ==

Jmfoste2 at 23:52, 13 April 2011

2011-04-13T23:52:35Z

← Older revision		Revision as of 23:52, 13 April 2011
Line 145:		Line 145:
	\| Higher		\| Higher
	\|}		\|}

			= Summary =

			As computer architectures have evolved, so have the parallel computing models. As discussed in Solihan [SITE], two models that have been prominent throughout computer history have been the Shared Memory Model and the Message Passing Model. These are, by no means, the only programming models in use. In this supplemental, we discussed two other important models: the Task Parallel Model and the Data Parallel Model. We also gave some context for how each of the programming models have emerged throughout the history of computer architecture. Each model has different strengths and weaknesses as discussed above.

	= Definitions =		= Definitions =

Jmfoste2 at 23:44, 13 April 2011

2011-04-13T23:44:33Z

← Older revision		Revision as of 23:44, 13 April 2011
Line 14:		Line 14:
	Development of the ILLIAC IV began in 1964 and wasn't finished until 1975 [http://en.wikipedia.org/wiki/ILLIAC_IV]. A central processor was connected to the main memory and delegated tasks to individual PE's, which each had their own memory cache. [http://archive.computerhistory.org/resources/text/Burroughs/Burroughs.ILLIAC%20IV.1974.102624911.pdf]. Each PE could operate either an 8-, 32- or 64-bit operand at a given time [http://archive.computerhistory.org/resources/text/Burroughs/Burroughs.ILLIAC%20IV.1974.102624911.pdf].		Development of the ILLIAC IV began in 1964 and wasn't finished until 1975 [http://en.wikipedia.org/wiki/ILLIAC_IV]. A central processor was connected to the main memory and delegated tasks to individual PE's, which each had their own memory cache. [http://archive.computerhistory.org/resources/text/Burroughs/Burroughs.ILLIAC%20IV.1974.102624911.pdf]. Each PE could operate either an 8-, 32- or 64-bit operand at a given time [http://archive.computerhistory.org/resources/text/Burroughs/Burroughs.ILLIAC%20IV.1974.102624911.pdf].

	The Cray machine was installed at Los Alamos National Laborartory in1976 by Cray Research, an offshoot of Control Data Corporation, and had similar performance to the ILLIAC IV [http://en.wikipedia.org/wiki/ILLIAC_IV]. The Cray machine relied heavily on the use of registers instead of individual processors like the ILLIAC IV. Each processor was connected to main memory and had a number of 64-bit registers used to perform operations [http://www.eecg.toronto.edu/~moshovos/ACA05/read/cray1.pdf].		The Cray machine was installed at Los Alamos National Laborartory in1976 by Cray Research, an offshoot of Control Data Corporation, and had similar performance to the ILLIAC IV [http://en.wikipedia.org/wiki/ILLIAC_IV]. The Cray machine relied heavily on the use of registers for memory instead of individual caches connected directly to processors like the ILLIAC IV. Each processor was connected to main memory and had a number of 64-bit registers used to perform operations [http://www.eecg.toronto.edu/~moshovos/ACA05/read/cray1.pdf].

	All of these early machines relied on there being a direct connection between each processor and the memory. Not only did there have to be a direct connection, but each connection had to allow relatively similar access to each part of memory. As you increased the number of processors, you had to increase the number of connections, which meant that this architecture did not scale well. At the same time, processors were becoming more and more advanced. The first single chip microprocessor was introduced in 1971 [http://en.wikipedia.org/wiki/Intel_4004]. Due to these two pressures, there was a movement away from large shared memory supercomputers and towards distributed memory systems.		All of these early commercial machines relied on there being a direct connection between each processor and the memory. Not only did there have to be a direct connection, but each connection had to allow relatively similar access to each part of memory. As you increased the number of processors, you had to increase the number of connections, which meant that this architecture did not scale well. At the same time, processors were becoming more and more advanced. The first single chip microprocessor was introduced in 1971 [http://en.wikipedia.org/wiki/Intel_4004]. Due to these two pressures, there was a movement away from large shared memory supercomputers and towards distributed memory systems.

	== Move to message passing in the 1980's ==		== Move to message passing in the 1980's ==
Line 25:		Line 25:

	== Current trend to cluster machines ==		== Current trend to cluster machines ==
	In the 1990's, ~~multi~~-~~core~~ computers ~~became the dominant trend in computer architecture~~. At the same time, network connections were becoming faster and faster. These two trends meant that it was no longer necessary to build custom hardware for parallel computing, like the bit-serial processor machines. Off-the-shelf computers connected via networks could offer similar performance. These cluster-based machines added another layer of complexity to parallelism. Since computers could be located across a network from each other, there is more emphasis on software acting as a bridge [http://cobweb.ecn.purdue.edu/~pplinux/ppcluster.html]. This has led to a greater emphasis on thread- or task-level parallelism [http://en.wikipedia.org/wiki/Thread-level_parallelism] and the addition of the data parallelism programming model to existing message passing or shared memory models [http://en.wikipedia.org/wiki/Thread-level_parallelism].		In the 1990's, off-the-shelf computers were becoming more and more powerful. At the same time, network connections were becoming faster and faster. These two trends meant that it was no longer necessary to build custom hardware for parallel computing, like the bit-serial processor machines. Off-the-shelf computers connected via networks could offer similar performance. These cluster-based machines added another layer of complexity to parallelism. Since computers could be located across a network from each other, there is more emphasis on software acting as a bridge [http://cobweb.ecn.purdue.edu/~pplinux/ppcluster.html]. This has led to a greater emphasis on thread- or task-level parallelism [http://en.wikipedia.org/wiki/Thread-level_parallelism] and the addition of the data parallelism programming model to existing message passing or shared memory models [http://en.wikipedia.org/wiki/Thread-level_parallelism].

	= Data-Parallel Model =		= Data-Parallel Model =