CSC/ECE 506 Spring 2013/2a lm - Revision history

Mchen4: Added new reference

2013-02-15T20:44:36Z

Added new reference

← Older revision		Revision as of 20:44, 15 February 2013
Line 357:		Line 357:
	*David R. Cheriton. 1985. [http://doi.acm.org.prox.lib.ncsu.edu/10.1145/858336.858338 Preliminary thoughts on problem-oriented shared memory: a decentralized approach to distributed systems.] SIGOPS Oper. Syst. Rev. 19, 4 (October 1985), 26-33.		*David R. Cheriton. 1985. [http://doi.acm.org.prox.lib.ncsu.edu/10.1145/858336.858338 Preliminary thoughts on problem-oriented shared memory: a decentralized approach to distributed systems.] SIGOPS Oper. Syst. Rev. 19, 4 (October 1985), 26-33.
	*Matthew Chapman and Gernot Heiser. 2009. [http://www.usenix.org/event/usenix09/tech/full_papers/chapman/chapman_html/index.html vNUMA: a virtual shared-memory multiprocessor.] In Proceedings of the 2009 conference on USENIX Annual technical conference (USENIX'09). USENIX Association, Berkeley, CA, USA, 2-2.		*Matthew Chapman and Gernot Heiser. 2009. [http://www.usenix.org/event/usenix09/tech/full_papers/chapman/chapman_html/index.html vNUMA: a virtual shared-memory multiprocessor.] In Proceedings of the 2009 conference on USENIX Annual technical conference (USENIX'09). USENIX Association, Berkeley, CA, USA, 2-2.

			*Protic, Jelica, Milo Tomagevic, and Veljk Milutinovic. [http://www.cs.rit.edu/~pns6910/docs/Distributed%20Shared%20Memory%20Systems/A%20survey%20of%20distributed%20shared%20memory%20systems.pdf "A Survey of Distributed Shared Memory Systems."] 28th Annual Hawaii International Conference on System Sciences. IEEE. Hawaii, 1995. Reading.

	== Quiz ==		== Quiz ==

Mchen4: Updated background section

2013-02-15T20:36:16Z

Updated background section

← Older revision		Revision as of 20:36, 15 February 2013
Line 9:		Line 9:

	=== Background ===		=== Background ===
	Distributed memory systems started to flourish in the 1980s. The increasing performance in processors and network connectivity offered the perfect environment for parallel processing over a network of computers. This was a cheap way to put together massive computing power. The main drawback was going from sequential programs made for local memory to parallel programming in shared memory. This was where SAS provided the means to simplify programming by hiding the mechanisms to access distant memory located in other computers of the cluster.		Distributed memory systems are multi-processor systems in which each processor has its own individual memory. Tasks can only operate on a processor's local memory and if non-local data is required, the processor must communicate with one or more remote processors. Distributed memory systems started to flourish in the 1980s. The increasing performance in processors and network connectivity offered the perfect environment for parallel processing over a network of computers. This was a cheap way to put together massive computing power. The main drawback was going from sequential programs made for local memory to parallel programming in shared memory. This was where SAS provided the means to simplify programming by hiding the mechanisms to access distant memory located in other computers of the cluster.

	In 1985, Cheriton, in his article [http://doi.acm.org.prox.lib.ncsu.edu/10.1145/858336.858338 "Preliminary thoughts on problem-oriented shared memory: a decentralized approach to distributed systems"], introduced ideas for the application of shared memory techniques in distributed memory systems. Cheriton envisioned a system of nodes with a pool of shared memory using a common file namespace that could "decentralize the implementation of a service."		In 1985, Cheriton, in his article [http://doi.acm.org.prox.lib.ncsu.edu/10.1145/858336.858338 "Preliminary thoughts on problem-oriented shared memory: a decentralized approach to distributed systems"], introduced ideas for the application of shared memory techniques in distributed memory systems. Cheriton envisioned a system of nodes with a pool of shared memory using a common file namespace that could "decentralize the implementation of a service."

Ljblackw: /* Distributed Shared Memory (DSM) */

2013-02-12T01:51:23Z

Distributed Shared Memory (DSM)

← Older revision		Revision as of 01:51, 12 February 2013
Line 17:		Line 17:
	=== Distributed Shared Memory (DSM) ===		=== Distributed Shared Memory (DSM) ===
	[[File:Dsmd.jpg\|400px\|thumb\|right\|Distributed Shared Memory]]		[[File:Dsmd.jpg\|400px\|thumb\|right\|Distributed Shared Memory]]
	Most commonly, a distributed system utilizing SAS will consist of a set of nodes connected by a network. Nodes may be comprised of individual processors or a multiprocessor system (e.g. [http://en.wikipedia.org/wiki/Symmetric_multiprocessing '''Symmetric Multiprocessor'''] (SMP)), the latter typically sharing a system bus. Each node itself contains a local memory, which maps partially to the distributed address space. Relevant design elements of early SAS implementations included scalability, coherence, structure and granularity. Most early examples did not structure memory, that is the layout of shared memory was simply a linear array of words. Some, however, structured data as objects or language types. '''IVY''' , an early example of a DSM system, implemented shared memory as virtual memory. The granularity, or unit share size, for IVY was in 1-Kbyte pages and the memory was unstructured. A problem when considering optimal page size is the balance between a process likely needing quick access to a large range of the shared address space, which argues for a larger page size, countered by the greater contention for individual pages that the larger page may cause amongst processes and the [http://en.wikipedia.org/wiki/False_sharing '''false sharing'''] it may lead to. [http://en.wikipedia.org/wiki/Memory_coherence Memory coherence} is another important design element consideration, and semantics can be instituted that run gradations of strict to weak consistencies. The strictest consistency guarantees that a read returns the most recently written value. Weaker consistencies may use synchronization operations to guarantee sequential consistency.		Most commonly, a distributed system utilizing SAS will consist of a set of nodes connected by a network. Nodes may be comprised of individual processors or a multiprocessor system (e.g. [http://en.wikipedia.org/wiki/Symmetric_multiprocessing '''Symmetric Multiprocessor'''] (SMP)), the latter typically sharing a system bus. Each node itself contains a local memory, which maps partially to the distributed address space. Relevant design elements of early SAS implementations included scalability, coherence, structure and granularity. Most early examples did not structure memory, that is the layout of shared memory was simply a linear array of words. Some, however, structured data as objects or language types. '''IVY''' , an early example of a DSM system, implemented shared memory as virtual memory. The granularity, or unit share size, for IVY was in 1-Kbyte pages and the memory was unstructured. A problem when considering optimal page size is the balance between a process likely needing quick access to a large range of the shared address space, which argues for a larger page size, countered by the greater contention for individual pages that the larger page may cause amongst processes and the [http://en.wikipedia.org/wiki/False_sharing '''false sharing'''] it may lead to. [http://en.wikipedia.org/wiki/Memory_coherence Memory coherence] is another important design element consideration, and semantics can be instituted that run gradations of strict to weak consistencies. The strictest consistency guarantees that a read returns the most recently written value. Weaker consistencies may use synchronization operations to guarantee sequential consistency.

	==== Cache-Coherent DSM ====		==== Cache-Coherent DSM ====

Ljblackw: /* Distributed Shared Memory (DSM) */

2013-02-12T01:51:06Z

Distributed Shared Memory (DSM)

← Older revision		Revision as of 01:51, 12 February 2013
Line 17:		Line 17:
	=== Distributed Shared Memory (DSM) ===		=== Distributed Shared Memory (DSM) ===
	[[File:Dsmd.jpg\|400px\|thumb\|right\|Distributed Shared Memory]]		[[File:Dsmd.jpg\|400px\|thumb\|right\|Distributed Shared Memory]]
	Most commonly, a distributed system utilizing SAS will consist of a set of nodes connected by a network. Nodes may be comprised of individual processors or a multiprocessor system (e.g. [http://en.wikipedia.org/wiki/Symmetric_multiprocessing '''Symmetric Multiprocessor'''] (SMP)), the latter typically sharing a system bus. Each node itself contains a local memory, which maps partially to the distributed address space. Relevant design elements of early SAS implementations included scalability, coherence, structure and granularity. Most early examples did not structure memory, that is the layout of shared memory was simply a linear array of words. Some, however, structured data as objects or language types. '''IVY''' , an early example of a DSM system, implemented shared memory as virtual memory. The granularity, or unit share size, for IVY was in 1-Kbyte pages and the memory was unstructured. A problem when considering optimal page size is the balance between a process likely needing quick access to a large range of the shared address space, which argues for a larger page size, countered by the greater contention for individual pages that the larger page may cause amongst processes and the [http://en.wikipedia.org/wiki/False_sharing '''false sharing'''] it may lead to. [http://en.wikipedia.org/wiki/Memory_coherence,Memory coherence} is another important design element consideration, and semantics can be instituted that run gradations of strict to weak consistencies. The strictest consistency guarantees that a read returns the most recently written value. Weaker consistencies may use synchronization operations to guarantee sequential consistency.		Most commonly, a distributed system utilizing SAS will consist of a set of nodes connected by a network. Nodes may be comprised of individual processors or a multiprocessor system (e.g. [http://en.wikipedia.org/wiki/Symmetric_multiprocessing '''Symmetric Multiprocessor'''] (SMP)), the latter typically sharing a system bus. Each node itself contains a local memory, which maps partially to the distributed address space. Relevant design elements of early SAS implementations included scalability, coherence, structure and granularity. Most early examples did not structure memory, that is the layout of shared memory was simply a linear array of words. Some, however, structured data as objects or language types. '''IVY''' , an early example of a DSM system, implemented shared memory as virtual memory. The granularity, or unit share size, for IVY was in 1-Kbyte pages and the memory was unstructured. A problem when considering optimal page size is the balance between a process likely needing quick access to a large range of the shared address space, which argues for a larger page size, countered by the greater contention for individual pages that the larger page may cause amongst processes and the [http://en.wikipedia.org/wiki/False_sharing '''false sharing'''] it may lead to. [http://en.wikipedia.org/wiki/Memory_coherence Memory coherence} is another important design element consideration, and semantics can be instituted that run gradations of strict to weak consistencies. The strictest consistency guarantees that a read returns the most recently written value. Weaker consistencies may use synchronization operations to guarantee sequential consistency.

	==== Cache-Coherent DSM ====		==== Cache-Coherent DSM ====

Ljblackw: /* Distributed Shared Memory (DSM) */

2013-02-12T01:50:46Z

Distributed Shared Memory (DSM)

← Older revision		Revision as of 01:50, 12 February 2013
Line 17:		Line 17:
	=== Distributed Shared Memory (DSM) ===		=== Distributed Shared Memory (DSM) ===
	[[File:Dsmd.jpg\|400px\|thumb\|right\|Distributed Shared Memory]]		[[File:Dsmd.jpg\|400px\|thumb\|right\|Distributed Shared Memory]]
	Most commonly, a distributed system utilizing SAS will consist of a set of nodes connected by a network. Nodes may be comprised of individual processors or a multiprocessor system (e.g. [http://en.wikipedia.org/wiki/Symmetric_multiprocessing '''Symmetric Multiprocessor'''] (SMP)), the latter typically sharing a system bus. Each node itself contains a local memory, which maps partially to the distributed address space. Relevant design elements of early SAS implementations included scalability, coherence, structure and granularity. Most early examples did not structure memory, that is the layout of shared memory was simply a linear array of words. Some, however, structured data as objects or language types. '''IVY''' , an early example of a DSM system, implemented shared memory as virtual memory. The granularity, or unit share size, for IVY was in 1-Kbyte pages and the memory was unstructured. A problem when considering optimal page size is the balance between a process likely needing quick access to a large range of the shared address space, which argues for a larger page size, countered by the greater contention for individual pages that the larger page may cause amongst processes and the [http://en.wikipedia.org/wiki/False_sharing '''false sharing'''] it may lead to. Memory coherence is another important design element consideration, and semantics can be instituted that run gradations of strict to weak consistencies. The strictest consistency guarantees that a read returns the most recently written value. Weaker consistencies may use synchronization operations to guarantee sequential consistency.		Most commonly, a distributed system utilizing SAS will consist of a set of nodes connected by a network. Nodes may be comprised of individual processors or a multiprocessor system (e.g. [http://en.wikipedia.org/wiki/Symmetric_multiprocessing '''Symmetric Multiprocessor'''] (SMP)), the latter typically sharing a system bus. Each node itself contains a local memory, which maps partially to the distributed address space. Relevant design elements of early SAS implementations included scalability, coherence, structure and granularity. Most early examples did not structure memory, that is the layout of shared memory was simply a linear array of words. Some, however, structured data as objects or language types. '''IVY''' , an early example of a DSM system, implemented shared memory as virtual memory. The granularity, or unit share size, for IVY was in 1-Kbyte pages and the memory was unstructured. A problem when considering optimal page size is the balance between a process likely needing quick access to a large range of the shared address space, which argues for a larger page size, countered by the greater contention for individual pages that the larger page may cause amongst processes and the [http://en.wikipedia.org/wiki/False_sharing '''false sharing'''] it may lead to. [http://en.wikipedia.org/wiki/Memory_coherence,Memory coherence} is another important design element consideration, and semantics can be instituted that run gradations of strict to weak consistencies. The strictest consistency guarantees that a read returns the most recently written value. Weaker consistencies may use synchronization operations to guarantee sequential consistency.

	==== Cache-Coherent DSM ====		==== Cache-Coherent DSM ====

Ljblackw: /* SAS programming on distributed-memory machines */

2013-02-05T04:12:54Z

SAS programming on distributed-memory machines

← Older revision		Revision as of 04:12, 5 February 2013
Line 4:		Line 4:
	[https://docs.google.com/a/ncsu.edu/document/d/1898MW7jXRhuz40HXXiTsobSUDdUVBZ-aUjEyLdeQdNc/edit#, Topic Writeup]		[https://docs.google.com/a/ncsu.edu/document/d/1898MW7jXRhuz40HXXiTsobSUDdUVBZ-aUjEyLdeQdNc/edit#, Topic Writeup]

	[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_506_Spring_2012/2a_bm, Original Page]		[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_506_Spring_2012/2a_bm Original Page]

	[http://en.wikipedia.org/wiki/Shared_memory '''Shared Address Space'''] (SAS) programming on distributed memory machines is a programming abstraction that provides less development effort than that of the traditional method of [http://en.wikipedia.org/wiki/Message_passing '''Message Passing'''] (MP) on distributed memory machines, such as clusters of servers. Distributed systems are groups of computers that communicate through a network and share a common work goal. Distributed systems typically do not physically share the same memory (are not [http://en.wikipedia.org/wiki/Coupling_%28computer_programming%29 '''tightly coupled''']) but rather each processor or group of processors must depend on mechanisms other than direct memory access the shared memory, this arrangement is called Distributed Shared Memory and is discussed below. Relevant issues that come to bear include [http://en.wikipedia.org/wiki/Memory_coherence '''memory coherence'''], types of memory access, data and process [http://en.wikipedia.org/wiki/Synchronization_%28computer_science%29 '''synchronization'''], and performance.		[http://en.wikipedia.org/wiki/Shared_memory '''Shared Address Space'''] (SAS) programming on distributed memory machines is a programming abstraction that provides less development effort than that of the traditional method of [http://en.wikipedia.org/wiki/Message_passing '''Message Passing'''] (MP) on distributed memory machines, such as clusters of servers. Distributed systems are groups of computers that communicate through a network and share a common work goal. Distributed systems typically do not physically share the same memory (are not [http://en.wikipedia.org/wiki/Coupling_%28computer_programming%29 '''tightly coupled''']) but rather each processor or group of processors must depend on mechanisms other than direct memory access the shared memory, this arrangement is called Distributed Shared Memory and is discussed below. Relevant issues that come to bear include [http://en.wikipedia.org/wiki/Memory_coherence '''memory coherence'''], types of memory access, data and process [http://en.wikipedia.org/wiki/Synchronization_%28computer_science%29 '''synchronization'''], and performance.

Ljblackw: /* SAS programming on distributed-memory machines */

2013-02-05T04:11:11Z

SAS programming on distributed-memory machines

← Older revision		Revision as of 04:11, 5 February 2013
Line 2:		Line 2:

	== SAS programming on distributed-memory machines ==		== SAS programming on distributed-memory machines ==
			[https://docs.google.com/a/ncsu.edu/document/d/1898MW7jXRhuz40HXXiTsobSUDdUVBZ-aUjEyLdeQdNc/edit#, Topic Writeup]

			[http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_506_Spring_2012/2a_bm, Original Page]

	[http://en.wikipedia.org/wiki/Shared_memory '''Shared Address Space'''] (SAS) programming on distributed memory machines is a programming abstraction that provides less development effort than that of the traditional method of [http://en.wikipedia.org/wiki/Message_passing '''Message Passing'''] (MP) on distributed memory machines, such as clusters of servers. Distributed systems are groups of computers that communicate through a network and share a common work goal. Distributed systems typically do not physically share the same memory (are not [http://en.wikipedia.org/wiki/Coupling_%28computer_programming%29 '''tightly coupled''']) but rather each processor or group of processors must depend on mechanisms other than direct memory access the shared memory, this arrangement is called Distributed Shared Memory and is discussed below. Relevant issues that come to bear include [http://en.wikipedia.org/wiki/Memory_coherence '''memory coherence'''], types of memory access, data and process [http://en.wikipedia.org/wiki/Synchronization_%28computer_science%29 '''synchronization'''], and performance.		[http://en.wikipedia.org/wiki/Shared_memory '''Shared Address Space'''] (SAS) programming on distributed memory machines is a programming abstraction that provides less development effort than that of the traditional method of [http://en.wikipedia.org/wiki/Message_passing '''Message Passing'''] (MP) on distributed memory machines, such as clusters of servers. Distributed systems are groups of computers that communicate through a network and share a common work goal. Distributed systems typically do not physically share the same memory (are not [http://en.wikipedia.org/wiki/Coupling_%28computer_programming%29 '''tightly coupled''']) but rather each processor or group of processors must depend on mechanisms other than direct memory access the shared memory, this arrangement is called Distributed Shared Memory and is discussed below. Relevant issues that come to bear include [http://en.wikipedia.org/wiki/Memory_coherence '''memory coherence'''], types of memory access, data and process [http://en.wikipedia.org/wiki/Synchronization_%28computer_science%29 '''synchronization'''], and performance.

Ljblackw: /* DSM Implementations */

2013-02-05T04:05:02Z

DSM Implementations

← Older revision		Revision as of 04:05, 5 February 2013
Line 142:		Line 142:
	increasing the ''Shortest_path'' a lock is acquired to ensure [http://en.wikipedia.org/wiki/Mutual_exclusion '''mutual exclusion'''] to update this shared data as well.		increasing the ''Shortest_path'' a lock is acquired to ensure [http://en.wikipedia.org/wiki/Mutual_exclusion '''mutual exclusion'''] to update this shared data as well.

	=== DSM Implementations ===		=== Notable DSM Implementations ===
	From an architectural point of view, DSMs are composed of several nodes connected via a network. Each of the nodes can be an individual machine or a cluster of machines. Each system has local memory modules that are either partially or completely part of the shared memory. There are many characteristics that can be used to classify DSM implementations. One of them is based on the nature of the memory demarcation: Software, Hardware, and Hybrid. This historical classification has been extracted from [http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=494605&isnumber=10721 Distributed shared memory: concepts and systems].		From an architectural point of view, DSMs are composed of several nodes connected via a network. Each of the nodes can be an individual machine or a cluster of machines. Each system has local memory modules that are either partially or completely part of the shared memory. There are many characteristics that can be used to classify DSM implementations. One of them is based on the nature of the memory demarcation: Software, Hardware, and Hybrid. This historical classification has been extracted from [http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=494605&isnumber=10721 Distributed shared memory: concepts and systems].

Ljblackw: /* DSM Implementations */

2013-02-05T04:04:42Z

DSM Implementations

← Older revision		Revision as of 04:04, 5 February 2013
Line 145:		Line 145:
	From an architectural point of view, DSMs are composed of several nodes connected via a network. Each of the nodes can be an individual machine or a cluster of machines. Each system has local memory modules that are either partially or completely part of the shared memory. There are many characteristics that can be used to classify DSM implementations. One of them is based on the nature of the memory demarcation: Software, Hardware, and Hybrid. This historical classification has been extracted from [http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=494605&isnumber=10721 Distributed shared memory: concepts and systems].		From an architectural point of view, DSMs are composed of several nodes connected via a network. Each of the nodes can be an individual machine or a cluster of machines. Each system has local memory modules that are either partially or completely part of the shared memory. There are many characteristics that can be used to classify DSM implementations. One of them is based on the nature of the memory demarcation: Software, Hardware, and Hybrid. This historical classification has been extracted from [http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=494605&isnumber=10721 Distributed shared memory: concepts and systems].

	Software DSM implementations refer to the DSM implemented by using user-level software, OS, programming language, or combination of ~~all or some of them~~.		Software DSM implementations refer to the DSM implemented by using user-level software, OS, programming language, or combination of these.

	{\| {{table}}		{\| {{table}}

Ljblackw: /* DSM Implementations */

2013-02-05T04:04:12Z

DSM Implementations

← Older revision		Revision as of 04:04, 5 February 2013
Line 143:		Line 143:

	=== DSM Implementations ===		=== DSM Implementations ===
	From an architectural point of view, DSMs are composed of several nodes connected via a network. Each of the nodes can be an individual machine or a cluster of machines. Each system has local memory modules that are either partially or completely part of the shared memory. There are many characteristics that can be used to classify DSM implementations. One of them ~~is obvious and~~ is based on the nature of the ~~implementation as~~ demarcation: Software, Hardware, and Hybrid. This historical classification has been extracted from [http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=494605&isnumber=10721 Distributed shared memory: concepts and systems].		From an architectural point of view, DSMs are composed of several nodes connected via a network. Each of the nodes can be an individual machine or a cluster of machines. Each system has local memory modules that are either partially or completely part of the shared memory. There are many characteristics that can be used to classify DSM implementations. One of them is based on the nature of the memory demarcation: Software, Hardware, and Hybrid. This historical classification has been extracted from [http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=494605&isnumber=10721 Distributed shared memory: concepts and systems].

	Software DSM implementations refer to the DSM implemented by using user-level software, OS, programming language, or combination of all or some of them.		Software DSM implementations refer to the DSM implemented by using user-level software, OS, programming language, or combination of all or some of them.