Expertiza_Wiki - User contributions [en]

CSC 456 Spring 2012/10a AJ

2012-04-25T01:24:47Z

Jmmohund2: /* References */

=Prefetching and Consistency Models=
==Intro==
While memory consistency models ensure instructions are executed in correct order, these models can hinder efficiency. Since consistency models dictate order of execution, prefetching allows operations to complete quicker once their turn comes by bringing the necessary data closer into the cache before it is needed. Prefetching is a hardware optimization technique in which the processor automatically prefetches ownership for any write operations that are delayed. These delays are due to the program order requirement. Using prefetching as well as sequential consistency creates delayed writes that become partially overlapped with the operations preceding them in program order. This technique is only applicable to cache-based systems that use an invalidation-based protocol. This technique is suitable for statically scheduled processors.

==Methods==

===Fixed vs. Adaptive Sequential Prefetching===
Fixed sequential prefetching refers to prefetching that occurs at a constant rate over time. Because the amount of prefetching remains constant throughout execution, fixed sequential is the simplest method of prefetching to implement. When block n is read, a fixed number of consecutive blocks (K), are prefetched if they are not already in the cache regardless of a read miss or read hit. Finding the best value for K remains tricky, however. The K value must be large enough to lower cold miss rates while at the same time small enough in order to not load the cache with useless data and unnecessary traffic.

Adaptive sequential prefetching, on the other hand, changes the rate of prefetching allowed over time. The prefetching rate is increased/decreased based on the count of successful prefetches.

The rate is therefore dependent on workload and application (a start-up process will have a higher rate of cold misses). While both methods improve efficiency, adaptive sequential prefetching is the most efficient but also the most costly. (Dahlgren)

===Exclusive Mode-Prefetching===
Exclusive-mode prefetching helps to reduce both the miss latencies and the message traffic associated with writes. Unlike read misses, which directly stall the processor for their entire duration. Write misses affect performance[[File:exclusive_prefetch.jpg|200px|thumb|right| exclusive prefetch]] more indirectly, since writes can be buffered. A processor stalls while waiting for writes to complete in two situations: (i) when executing a write instruction if the write buffer is full, and (ii) during a read miss if previous writes must complete before the read miss can proceed. The impact of the former effect can be reduced through larger write buffers. In summary, exclusive-mode prefetching can provide significant performance benefits in architectures that have not already eliminated write stall times through aggressive implementations of weaker consistency models with lockup-free caches. Even if write stall times cannot be further reduced, exclusive-mode prefetching can improve performance somewhat by reducing the traffic associated with cache coherency. The main effect of exclusive mode prefetching is that the item that is prefetched, is done so as though its exclusive to that cache.

==Where they stand now==
Prefetching began to fade due to its many disadvantages and memory speeds starting to catch up with processor transfer rates. One issue with prefetching is the increased complexity and overhead of handing the prefetching algorithms. There is great risk that this overhead can be larger than any benefits if the prefetching algorithm used is not accurate, and fetches too early or fetches too late to be effective. Performance must be improved significantly to overcome the overhead and complexity or be a waste. Another problem comes about with the introduction of multicore architectures. In a single core architecture, prefetching requests are able to come from one core. With multiple cores, prefetching requests can originate from any of the different cores. This puts additional stress on memory to not only deal with regular prefetch requests but also to handle prefetch from different sources, and greatly increases the overhead and complexity of logic. Coherence algorithms must account not only for sequential consistency issues, but also account for the data change in another location, the location of prefetched data. Flushing data becomes significantly more complicated. If prefetched data is stored in the data cache, then cache conflict, can become a significant problem. This is because the current and predictive sets of data must exist in the cache at the same time. Without prefetching, you could use the additional space to simply increase the cache size itself. The solution here would be to add extra hardware to act as a buffer to prevent utilizing this cache space.

==References==

*http://impact.asu.edu/cse520fa08/Srivatsan.pdf
*http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/ch10_sj
*http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.126.3128&rep=rep1&type=pdf
*http://www.cesr.ncsu.edu/solihin/Main.html

*Kourosh Gharachorloo, Anoop Gupta, and John Hennessy of Stanford University, "Two techniques to enhance the performance of memory consistency models," International Conference on Parallel Processing, 1991

*Frederick Dahlgren, Michel Dubois, and Per Stenstrom, "Sequential hardware prefetching in shared-memory multiprocessors," IEEE Trans. Parallel and Distributed Sytems, July 1995

*Mark Hill, "Multiprocessors should support simple memory consistency models," IEEE Computer, August 1998

*Amir Kamil Jimmy Su, and Katherine Yelick, "Making sequential consistency practical in Titanium," Supercomputing, SC-2005
*http://download.intel.com/education/highered/multicore/Lecture27.pdf

CSC 456 Spring 2012/10a AJ

2012-04-25T01:22:54Z

Jmmohund2: /* Exclusive Mode-Prefetching */

CSC 456 Spring 2012/10a AJ

2012-04-23T18:03:12Z

Jmmohund2: /* Intro */

=Prefetching and Consistency Models=
==Intro==
While memory consistency models ensure instructions are executed in correct order, these models can hinder efficiency. Since consistency models dictate order of execution, prefetching allows operations to complete quicker once their turn comes by bringing the necessary data closer into the cache before it is needed. Prefetching is a hardware optimization technique in which the processor automatically prefetches ownership for any write operations that are delayed. These delays are due to the program order requirement. Using prefetching as well as sequential consistency creates delayed writes that become partially overlapped with the operations preceding them in program order. This technique is only applicable to cache-based systems that use an invalidation-based protocol. This technique is suitable for statically scheduled processors.

==Methods==

===Fixed vs. Adaptive Sequential Prefetching===
Fixed sequential prefetching refers to prefetching that occurs at a constant rate over time. Adaptive sequential prefetching, on the other hand, changes the rate of prefetching allowed over time. The prefetching rate is increased/decreased based on the count of successful prefetches. The rate is therefore dependent on workload and application (a start-up process will have a higher rate of cold misses). While both methods improve efficiency, adaptive sequential prefetching is the most efficient but also the most costly. (Dahlgren)

===Exclusive Mode-Prefetching===
Exclusive-mode prefetching helps to reduce both the miss latencies and the message traffic associated with writes. Unlike read misses, which directly stall the processor for their entire duration. Write misses affect performance[[File:exclusive_prefetch.jpg|200px|thumb|right| exclusive prefetch]] more indirectly, since writes can be buffered. A processor stalls while waiting for writes to complete in two situations: (i) when executing a write instruction if the write buffer is full, and (ii) during a read miss if previous writes must complete before the read miss can proceed. The impact of the former effect can be reduced through larger write buffers. In summary, exclusive-mode prefetching can provide significant performance benefits in architectures that have not already eliminated write stall times through aggressive implementations of weaker consistency models with lockup-free caches. Even if write stall times cannot be further reduced, exclusive-mode prefetching can improve performance somewhat by reducing the traffic associated with cache coherency.

==Where they stand now==
Prefetching began to fade due to its many disadvantages and memory speeds starting to catch up with processor transfer rates. One issue with prefetching is the increased complexity and overhead of handing the prefetching algorithms. There is great risk that this overhead can be larger than any benefits if the prefetching algorithm used is not accurate, and fetches too early or fetches too late to be effective. Performance must be improved significantly to overcome the overhead and complexity or be a waste. Another problem comes about with the introduction of multicore architectures. In a single core architecture, prefetching requests are able to come from one core. With multiple cores, prefetching requests can originate from any of the different cores. This puts additional stress on memory to not only deal with regular prefetch requests but also to handle prefetch from different sources, and greatly increases the overhead and complexity of logic. Coherence algorithms must account not only for sequential consistency issues, but also account for the data change in another location, the location of prefetched data. Flushing data becomes significantly more complicated. If prefetched data is stored in the data cache, then cache conflict, can become a significant problem. This is because the current and predictive sets of data must exist in the cache at the same time. Without prefetching, you could use the additional space to simply increase the cache size itself. The solution here would be to add extra hardware to act as a buffer to prevent utilizing this cache space.

==References==

*http://impact.asu.edu/cse520fa08/Srivatsan.pdf
*http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/ch10_sj
*http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.126.3128&rep=rep1&type=pdf
*http://www.cesr.ncsu.edu/solihin/Main.html

*Kourosh Gharachorloo, Anoop Gupta, and John Hennessy of Stanford University, "Two techniques to enhance the performance of memory consistency models," International Conference on Parallel Processing, 1991

*Frederick Dahlgren, Michel Dubois, and Per Stenstrom, "Sequential hardware prefetching in shared-memory multiprocessors," IEEE Trans. Parallel and Distributed Sytems, July 1995

*Mark Hill, "Multiprocessors should support simple memory consistency models," IEEE Computer, August 1998

*Amir Kamil Jimmy Su, and Katherine Yelick, "Making sequential consistency practical in Titanium," Supercomputing, SC-2005

CSC 456 Spring 2012/10a AJ

2012-04-23T18:00:05Z

Jmmohund2: /* Exclusive Mode-Prefetching */

=Prefetching and Consistency Models=
==Intro==
While memory consistency models insure instructions are executed in correct order, these models can hinder efficiency. Since consistency models dictate order of execution, prefetching allows operations to complete quicker once their turn comes by bringing the necessary data closer into the cache before it is needed. Prefetching is a hardware optimization technique in which the processor automatically prefetches ownership for any write operations that are delayed. These delays are due to the program order requirement. Using prefetching as well as sequential consistency creates delayed writes that become partially overlapped with the operations preceding them in program order. This technique is only applicable to cache-based systems that use an invalidation-based protocol. This technique is suitable for statically scheduled processors.

==Methods==

===Fixed vs. Adaptive Sequential Prefetching===
Fixed sequential prefetching refers to prefetching that occurs at a constant rate over time. Adaptive sequential prefetching, on the other hand, changes the rate of prefetching allowed over time. The prefetching rate is increased/decreased based on the count of successful prefetches. The rate is therefore dependent on workload and application (a start-up process will have a higher rate of cold misses). While both methods improve efficiency, adaptive sequential prefetching is the most efficient but also the most costly. (Dahlgren)

===Exclusive Mode-Prefetching===
Exclusive-mode prefetching helps to reduce both the miss latencies and the message traffic associated with writes. Unlike read misses, which directly stall the processor for their entire duration. Write misses affect performance[[File:exclusive_prefetch.jpg|200px|thumb|right| exclusive prefetch]] more indirectly, since writes can be buffered. A processor stalls while waiting for writes to complete in two situations: (i) when executing a write instruction if the write buffer is full, and (ii) during a read miss if previous writes must complete before the read miss can proceed. The impact of the former effect can be reduced through larger write buffers. In summary, exclusive-mode prefetching can provide significant performance benefits in architectures that have not already eliminated write stall times through aggressive implementations of weaker consistency models with lockup-free caches. Even if write stall times cannot be further reduced, exclusive-mode prefetching can improve performance somewhat by reducing the traffic associated with cache coherency.

==Where they stand now==
Prefetching began to fade due to its many disadvantages and memory speeds starting to catch up with processor transfer rates. One issue with prefetching is the increased complexity and overhead of handing the prefetching algorithms. There is great risk that this overhead can be larger than any benefits if the prefetching algorithm used is not accurate, and fetches too early or fetches too late to be effective. Performance must be improved significantly to overcome the overhead and complexity or be a waste. Another problem comes about with the introduction of multicore architectures. In a single core architecture, prefetching requests are able to come from one core. With multiple cores, prefetching requests can originate from any of the different cores. This puts additional stress on memory to not only deal with regular prefetch requests but also to handle prefetch from different sources, and greatly increases the overhead and complexity of logic. Coherence algorithms must account not only for sequential consistency issues, but also account for the data change in another location, the location of prefetched data. Flushing data becomes significantly more complicated. If prefetched data is stored in the data cache, then cache conflict, can become a significant problem. This is because the current and predictive sets of data must exist in the cache at the same time. Without prefetching, you could use the additional space to simply increase the cache size itself. The solution here would be to add extra hardware to act as a buffer to prevent utilizing this cache space.

==References==

*http://impact.asu.edu/cse520fa08/Srivatsan.pdf
*http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/ch10_sj
*http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.126.3128&rep=rep1&type=pdf
*http://www.cesr.ncsu.edu/solihin/Main.html

*Kourosh Gharachorloo, Anoop Gupta, and John Hennessy of Stanford University, "Two techniques to enhance the performance of memory consistency models," International Conference on Parallel Processing, 1991

*Frederick Dahlgren, Michel Dubois, and Per Stenstrom, "Sequential hardware prefetching in shared-memory multiprocessors," IEEE Trans. Parallel and Distributed Sytems, July 1995

*Mark Hill, "Multiprocessors should support simple memory consistency models," IEEE Computer, August 1998

*Amir Kamil Jimmy Su, and Katherine Yelick, "Making sequential consistency practical in Titanium," Supercomputing, SC-2005

CSC 456 Spring 2012/10a AJ

2012-04-23T17:50:53Z

Jmmohund2: /* Intro */

=Prefetching and Consistency Models=
==Intro==
While memory consistency models insure instructions are executed in correct order, these models can hinder efficiency. Since consistency models dictate order of execution, prefetching allows operations to complete quicker once their turn comes by bringing the necessary data closer into the cache before it is needed. Prefetching is a hardware optimization technique in which the processor automatically prefetches ownership for any write operations that are delayed. These delays are due to the program order requirement. Using prefetching as well as sequential consistency creates delayed writes that become partially overlapped with the operations preceding them in program order. This technique is only applicable to cache-based systems that use an invalidation-based protocol. This technique is suitable for statically scheduled processors.

==Methods==

===Fixed vs. Adaptive Sequential Prefetching===
Fixed sequential prefetching refers to prefetching that occurs at a constant rate over time. Adaptive sequential prefetching, on the other hand, changes the rate of prefetching allowed over time. The prefetching rate is increased/decreased based on the count of successful prefetches. The rate is therefore dependent on workload and application (a start-up process will have a higher rate of cold misses). While both methods improve efficiency, adaptive sequential prefetching is the most efficient but also the most costly. (Dahlgren)

===Exclusive Mode-Prefetching===
Exclusive-mode prefetching helps to reduce both the miss latencies and the message traffic associated with writes. Unlike read misses, which directly stall the processor for their entire duration, write misses affect performance[[File:exclusive_prefetch.jpg|200px|thumb|right| exclusive prefetch]] more indirectly, since writes can be buffered. A processor stalls while waiting for writes to complete in two situations: (i) when executing a write instruction if the write buffer is full, and (ii) during a read miss if previous writes must complete before the read miss can proceed. The impact of the former effect can be reduced through larger write buffers. In summary, exclusive-mode prefetching can provide significant performance benefits in architectures that have not already eliminated write stall times through aggressive implementations of weaker consistency models with lockup-free caches. Even if write stall times cannot be further reduced, exclusive-mode prefetching can improve performance somewhat by reducing the traffic associated with cache coherency.

==Where they stand now==
Prefetching began to fade due to its many disadvantages and memory speeds starting to catch up with processor transfer rates. One issue with prefetching is the increased complexity and overhead of handing the prefetching algorithms. There is great risk that this overhead can be larger than any benefits if the prefetching algorithm used is not accurate, and fetches too early or fetches too late to be effective. Performance must be improved significantly to overcome the overhead and complexity or be a waste. Another problem comes about with the introduction of multicore architectures. In a single core architecture, prefetching requests are able to come from one core. With multiple cores, prefetching requests can originate from any of the different cores. This puts additional stress on memory to not only deal with regular prefetch requests but also to handle prefetch from different sources, and greatly increases the overhead and complexity of logic. Coherence algorithms must account not only for sequential consistency issues, but also account for the data change in another location, the location of prefetched data. Flushing data becomes significantly more complicated. If prefetched data is stored in the data cache, then cache conflict, can become a significant problem. This is because the current and predictive sets of data must exist in the cache at the same time. Without prefetching, you could use the additional space to simply increase the cache size itself. The solution here would be to add extra hardware to act as a buffer to prevent utilizing this cache space.

==References==

*http://impact.asu.edu/cse520fa08/Srivatsan.pdf
*http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/ch10_sj
*http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.126.3128&rep=rep1&type=pdf
*http://www.cesr.ncsu.edu/solihin/Main.html

*Kourosh Gharachorloo, Anoop Gupta, and John Hennessy of Stanford University, "Two techniques to enhance the performance of memory consistency models," International Conference on Parallel Processing, 1991

*Frederick Dahlgren, Michel Dubois, and Per Stenstrom, "Sequential hardware prefetching in shared-memory multiprocessors," IEEE Trans. Parallel and Distributed Sytems, July 1995

*Mark Hill, "Multiprocessors should support simple memory consistency models," IEEE Computer, August 1998

*Amir Kamil Jimmy Su, and Katherine Yelick, "Making sequential consistency practical in Titanium," Supercomputing, SC-2005

CSC 456 Spring 2012/10a AJ

2012-04-23T17:50:11Z

Jmmohund2: /* Intro */

=Prefetching and Consistency Models=
==Intro==
While memory consistency models insure instructions are executed in correct order, these models can hinder efficiency. Since consistency models dictate order of execution, prefetching allows operations to complete quicker once their turn comes by bringing the necessarily data closer into the cache before it is needed. Prefetching is a hardware optimization technique in which the processor automatically prefetches ownership for any write operations that are delayed. These delays are due to the program order requirement. Using prefetching as well as sequential consistency creates delayed writes that become partially overlapped with the operations preceding them in program order. This technique is only applicable to cache-based systems that use an invalidation-based protocol. This technique is suitable for statically scheduled processors.

==Methods==

===Fixed vs. Adaptive Sequential Prefetching===
Fixed sequential prefetching refers to prefetching that occurs at a constant rate over time. Adaptive sequential prefetching, on the other hand, changes the rate of prefetching allowed over time. The prefetching rate is increased/decreased based on the count of successful prefetches. The rate is therefore dependent on workload and application (a start-up process will have a higher rate of cold misses). While both methods improve efficiency, adaptive sequential prefetching is the most efficient but also the most costly. (Dahlgren)

===Exclusive Mode-Prefetching===
Exclusive-mode prefetching helps to reduce both the miss latencies and the message traffic associated with writes. Unlike read misses, which directly stall the processor for their entire duration, write misses affect performance[[File:exclusive_prefetch.jpg|200px|thumb|right| exclusive prefetch]] more indirectly, since writes can be buffered. A processor stalls while waiting for writes to complete in two situations: (i) when executing a write instruction if the write buffer is full, and (ii) during a read miss if previous writes must complete before the read miss can proceed. The impact of the former effect can be reduced through larger write buffers. In summary, exclusive-mode prefetching can provide significant performance benefits in architectures that have not already eliminated write stall times through aggressive implementations of weaker consistency models with lockup-free caches. Even if write stall times cannot be further reduced, exclusive-mode prefetching can improve performance somewhat by reducing the traffic associated with cache coherency.

==Where they stand now==
Prefetching began to fade due to its many disadvantages and memory speeds starting to catch up with processor transfer rates. One issue with prefetching is the increased complexity and overhead of handing the prefetching algorithms. There is great risk that this overhead can be larger than any benefits if the prefetching algorithm used is not accurate, and fetches too early or fetches too late to be effective. Performance must be improved significantly to overcome the overhead and complexity or be a waste. Another problem comes about with the introduction of multicore architectures. In a single core architecture, prefetching requests are able to come from one core. With multiple cores, prefetching requests can originate from any of the different cores. This puts additional stress on memory to not only deal with regular prefetch requests but also to handle prefetch from different sources, and greatly increases the overhead and complexity of logic. Coherence algorithms must account not only for sequential consistency issues, but also account for the data change in another location, the location of prefetched data. Flushing data becomes significantly more complicated. If prefetched data is stored in the data cache, then cache conflict, can become a significant problem. This is because the current and predictive sets of data must exist in the cache at the same time. Without prefetching, you could use the additional space to simply increase the cache size itself. The solution here would be to add extra hardware to act as a buffer to prevent utilizing this cache space.

==References==

*http://impact.asu.edu/cse520fa08/Srivatsan.pdf
*http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/ch10_sj
*http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.126.3128&rep=rep1&type=pdf
*http://www.cesr.ncsu.edu/solihin/Main.html

*Kourosh Gharachorloo, Anoop Gupta, and John Hennessy of Stanford University, "Two techniques to enhance the performance of memory consistency models," International Conference on Parallel Processing, 1991

*Frederick Dahlgren, Michel Dubois, and Per Stenstrom, "Sequential hardware prefetching in shared-memory multiprocessors," IEEE Trans. Parallel and Distributed Sytems, July 1995

*Mark Hill, "Multiprocessors should support simple memory consistency models," IEEE Computer, August 1998

*Amir Kamil Jimmy Su, and Katherine Yelick, "Making sequential consistency practical in Titanium," Supercomputing, SC-2005

CSC 456 Spring 2012/10a AJ

2012-04-23T17:45:25Z

Jmmohund2: /* References */

=Prefetching and Consistency Models=
==Intro==
While memory consistency models insure instructions are executed in correct order, these models can hinder efficiency. Since consistency models dictate order of execution, prefetching allows operations to complete quicker once their turn comes by bringing the necessarily data closer into the cache before it is needed. Prefetching is a hardware optimization technique in which the processor automatically prefetches ownership for any write operations that are delayed. These delays are due to the program order requirement (e.g., by issuing prefetch-exclusive requests for any writes delayed in the write buffer). Using prefetching as well as sequential consistency creates delayed writes that become partially overlapped with the operations preceding them in program order. This technique is only applicable to cache-based systems that use an invalidation-based protocol. This technique is suitable for statically scheduled processors.

==Methods==

===Fixed vs. Adaptive Sequential Prefetching===
Fixed sequential prefetching refers to prefetching that occurs at a constant rate over time. Adaptive sequential prefetching, on the other hand, changes the rate of prefetching allowed over time. The prefetching rate is increased/decreased based on the count of successful prefetches. The rate is therefore dependent on workload and application (a start-up process will have a higher rate of cold misses). While both methods improve efficiency, adaptive sequential prefetching is the most efficient but also the most costly.

===Exclusive Mode-Prefetching===
Exclusive-mode prefetching helps to reduce both the miss latencies and the message traffic associated with writes. Unlike read misses, which directly stall the processor for their entire duration, write misses affect performance[[File:exclusive_prefetch.jpg|200px|thumb|right| exclusive prefetch]] more indirectly, since writes can be buffered. A processor stalls while waiting for writes to complete in two situations: (i) when executing a write instruction if the write buffer is full, and (ii) during a read miss if previous writes must complete before the read miss can proceed. The impact of the former effect can be reduced through larger write buffers. In summary, exclusive-mode prefetching can provide significant performance benefits in architectures that have not already eliminated write stall times through aggressive implementations of weaker consistency models with lockup-free caches. Even if write stall times cannot be further reduced, exclusive-mode prefetching can improve performance somewhat by reducing the traffic associated with cache coherency.

==Where they stand now==
Prefetching began to fade due to its many disadvantages and memory speeds starting to catch up with processor transfer rates. One issue with prefetching is the increased complexity and overhead of handing the prefetching algorithms. There is great risk that this overhead can be larger than any benefits if the prefetching algorithm used is not accurate, and fetches too early or fetches too late to be effective. Performance must be improved significantly to overcome the overhead and complexity or be a waste. Another problem comes about with the introduction of multicore architectures. In a single core architecture, prefetching requests are able to come from one core. With multiple cores, prefetching requests can originate from any of the different cores. This puts additional stress on memory to not only deal with regular prefetch requests but also to handle prefetch from different sources, and greatly increases the overhead and complexity of logic. Coherence algorithms must account not only for sequential consistency issues, but also account for the data change in another location, the location of prefetched data. Flushing data becomes significantly more complicated. If prefetched data is stored in the data cache, then cache conflict, can become a significant problem. This is because the current and predictive sets of data must exist in the cache at the same time. Without prefetching, you could use the additional space to simply increase the cache size itself. The solution here would be to add extra hardware to act as a buffer to prevent utilizing this cache space.

==References==

*http://impact.asu.edu/cse520fa08/Srivatsan.pdf
*http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/ch10_sj
*http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.126.3128&rep=rep1&type=pdf
*http://www.cesr.ncsu.edu/solihin/Main.html

*Kourosh Gharachorloo, Anoop Gupta, and John Hennessy of Stanford University, "Two techniques to enhance the performance of memory consistency models," International Conference on Parallel Processing, 1991

*Frederick Dahlgren, Michel Dubois, and Per Stenstrom, "Sequential hardware prefetching in shared-memory multiprocessors," IEEE Trans. Parallel and Distributed Sytems, July 1995

*Mark Hill, "Multiprocessors should support simple memory consistency models," IEEE Computer, August 1998

*Amir Kamil Jimmy Su, and Katherine Yelick, "Making sequential consistency practical in Titanium," Supercomputing, SC-2005

CSC 456 Spring 2012/10a AJ

2012-04-23T17:44:10Z

Jmmohund2: /* References */

=Prefetching and Consistency Models=
==Intro==
While memory consistency models insure instructions are executed in correct order, these models can hinder efficiency. Since consistency models dictate order of execution, prefetching allows operations to complete quicker once their turn comes by bringing the necessarily data closer into the cache before it is needed. Prefetching is a hardware optimization technique in which the processor automatically prefetches ownership for any write operations that are delayed. These delays are due to the program order requirement (e.g., by issuing prefetch-exclusive requests for any writes delayed in the write buffer). Using prefetching as well as sequential consistency creates delayed writes that become partially overlapped with the operations preceding them in program order. This technique is only applicable to cache-based systems that use an invalidation-based protocol. This technique is suitable for statically scheduled processors.

==Methods==

===Fixed vs. Adaptive Sequential Prefetching===
Fixed sequential prefetching refers to prefetching that occurs at a constant rate over time. Adaptive sequential prefetching, on the other hand, changes the rate of prefetching allowed over time. The prefetching rate is increased/decreased based on the count of successful prefetches. The rate is therefore dependent on workload and application (a start-up process will have a higher rate of cold misses). While both methods improve efficiency, adaptive sequential prefetching is the most efficient but also the most costly.

===Exclusive Mode-Prefetching===
Exclusive-mode prefetching helps to reduce both the miss latencies and the message traffic associated with writes. Unlike read misses, which directly stall the processor for their entire duration, write misses affect performance[[File:exclusive_prefetch.jpg|200px|thumb|right| exclusive prefetch]] more indirectly, since writes can be buffered. A processor stalls while waiting for writes to complete in two situations: (i) when executing a write instruction if the write buffer is full, and (ii) during a read miss if previous writes must complete before the read miss can proceed. The impact of the former effect can be reduced through larger write buffers. In summary, exclusive-mode prefetching can provide significant performance benefits in architectures that have not already eliminated write stall times through aggressive implementations of weaker consistency models with lockup-free caches. Even if write stall times cannot be further reduced, exclusive-mode prefetching can improve performance somewhat by reducing the traffic associated with cache coherency.

==Where they stand now==
Prefetching began to fade due to its many disadvantages and memory speeds starting to catch up with processor transfer rates. One issue with prefetching is the increased complexity and overhead of handing the prefetching algorithms. There is great risk that this overhead can be larger than any benefits if the prefetching algorithm used is not accurate, and fetches too early or fetches too late to be effective. Performance must be improved significantly to overcome the overhead and complexity or be a waste. Another problem comes about with the introduction of multicore architectures. In a single core architecture, prefetching requests are able to come from one core. With multiple cores, prefetching requests can originate from any of the different cores. This puts additional stress on memory to not only deal with regular prefetch requests but also to handle prefetch from different sources, and greatly increases the overhead and complexity of logic. Coherence algorithms must account not only for sequential consistency issues, but also account for the data change in another location, the location of prefetched data. Flushing data becomes significantly more complicated. If prefetched data is stored in the data cache, then cache conflict, can become a significant problem. This is because the current and predictive sets of data must exist in the cache at the same time. Without prefetching, you could use the additional space to simply increase the cache size itself. The solution here would be to add extra hardware to act as a buffer to prevent utilizing this cache space.

==References==

*http://impact.asu.edu/cse520fa08/Srivatsan.pdf
*http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/ch10_sj
*http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.126.3128&rep=rep1&type=pdf

*Kourosh Gharachorloo, Anoop Gupta, and John Hennessy of Stanford University, "Two techniques to enhance the performance of memory consistency models," International Conference on Parallel Processing, 1991

*Frederick Dahlgren, Michel Dubois, and Per Stenstrom, "Sequential hardware prefetching in shared-memory multiprocessors," IEEE Trans. Parallel and Distributed Sytems, July 1995

*Frederick Dahlgren, Michel Dubois, and Per Stenstrom, "Sequential hardware prefetching in shared-memory multiprocessors," IEEE Computer, August 1998

*Mark Hill, "Multiprocessors should support simple memory consistency models," IEEE Computer, August 1998

*Amir Kamil Jimmy Su, and Katherine Yelick, "Making sequential consistency practical in Titanium," Supercomputing, SC-2005

CSC 456 Spring 2012/10a AJ

2012-04-23T17:43:52Z

Jmmohund2: /* References */

=Prefetching and Consistency Models=
==Intro==
While memory consistency models insure instructions are executed in correct order, these models can hinder efficiency. Since consistency models dictate order of execution, prefetching allows operations to complete quicker once their turn comes by bringing the necessarily data closer into the cache before it is needed. Prefetching is a hardware optimization technique in which the processor automatically prefetches ownership for any write operations that are delayed. These delays are due to the program order requirement (e.g., by issuing prefetch-exclusive requests for any writes delayed in the write buffer). Using prefetching as well as sequential consistency creates delayed writes that become partially overlapped with the operations preceding them in program order. This technique is only applicable to cache-based systems that use an invalidation-based protocol. This technique is suitable for statically scheduled processors.

==Methods==

===Fixed vs. Adaptive Sequential Prefetching===
Fixed sequential prefetching refers to prefetching that occurs at a constant rate over time. Adaptive sequential prefetching, on the other hand, changes the rate of prefetching allowed over time. The prefetching rate is increased/decreased based on the count of successful prefetches. The rate is therefore dependent on workload and application (a start-up process will have a higher rate of cold misses). While both methods improve efficiency, adaptive sequential prefetching is the most efficient but also the most costly.

===Exclusive Mode-Prefetching===
Exclusive-mode prefetching helps to reduce both the miss latencies and the message traffic associated with writes. Unlike read misses, which directly stall the processor for their entire duration, write misses affect performance[[File:exclusive_prefetch.jpg|200px|thumb|right| exclusive prefetch]] more indirectly, since writes can be buffered. A processor stalls while waiting for writes to complete in two situations: (i) when executing a write instruction if the write buffer is full, and (ii) during a read miss if previous writes must complete before the read miss can proceed. The impact of the former effect can be reduced through larger write buffers. In summary, exclusive-mode prefetching can provide significant performance benefits in architectures that have not already eliminated write stall times through aggressive implementations of weaker consistency models with lockup-free caches. Even if write stall times cannot be further reduced, exclusive-mode prefetching can improve performance somewhat by reducing the traffic associated with cache coherency.

==Where they stand now==
Prefetching began to fade due to its many disadvantages and memory speeds starting to catch up with processor transfer rates. One issue with prefetching is the increased complexity and overhead of handing the prefetching algorithms. There is great risk that this overhead can be larger than any benefits if the prefetching algorithm used is not accurate, and fetches too early or fetches too late to be effective. Performance must be improved significantly to overcome the overhead and complexity or be a waste. Another problem comes about with the introduction of multicore architectures. In a single core architecture, prefetching requests are able to come from one core. With multiple cores, prefetching requests can originate from any of the different cores. This puts additional stress on memory to not only deal with regular prefetch requests but also to handle prefetch from different sources, and greatly increases the overhead and complexity of logic. Coherence algorithms must account not only for sequential consistency issues, but also account for the data change in another location, the location of prefetched data. Flushing data becomes significantly more complicated. If prefetched data is stored in the data cache, then cache conflict, can become a significant problem. This is because the current and predictive sets of data must exist in the cache at the same time. Without prefetching, you could use the additional space to simply increase the cache size itself. The solution here would be to add extra hardware to act as a buffer to prevent utilizing this cache space.

==References==

*http://impact.asu.edu/cse520fa08/Srivatsan.pdf
*http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/ch10_sj

*Kourosh Gharachorloo, Anoop Gupta, and John Hennessy of Stanford University, "Two techniques to enhance the performance of memory consistency models," International Conference on Parallel Processing, 1991

*Frederick Dahlgren, Michel Dubois, and Per Stenstrom, "Sequential hardware prefetching in shared-memory multiprocessors," IEEE Trans. Parallel and Distributed Sytems, July 1995

*Frederick Dahlgren, Michel Dubois, and Per Stenstrom, "Sequential hardware prefetching in shared-memory multiprocessors," IEEE Computer, August 1998

*Mark Hill, "Multiprocessors should support simple memory consistency models," IEEE Computer, August 1998

*Amir Kamil Jimmy Su, and Katherine Yelick, "Making sequential consistency practical in Titanium," Supercomputing, SC-2005

CSC 456 Spring 2012/10a AJ

2012-04-16T13:42:25Z

Jmmohund2: /* Exclusive Mode-Prefetching */

=Prefetching and Consistency Models=
==Intro==
While memory consistency models insure instructions are executed in correct order, these models can hinder efficiency. Since consistency models dictate order of execution, prefetching allows operations to complete quicker once their turn comes by bringing the necessarily data closer into the cache before it is needed. Prefetching is a hardware optimization technique in which the processor automatically prefetches ownership for any write operations that are delayed due to the program order requirement (e.g., by issuing prefetch-exclusive requests for any writes delayed in the write buffer), thus partially overlapping the service of the delayed writes with the operations preceding them in program order. This technique is only applicable to cache-based systems that use an invalidation-based protocol. This technique is suitable for statically scheduled processors.

==Methods==

===Fixed vs. Adaptive Sequential Prefetching===
Fixed sequential prefetching refers to prefetching of that occurs at a constant rate over time. Adaptive sequential prefetching, on the other hand, changes the rate of prefetching allowed over time. The prefetching rate is increased/decreased based on the count of successful prefetches. The rate is therefore dependent on workload and application (a start-up process will have a high rate of cold misses). While both methods improve efficiency, adaptive sequential prefetching is the most efficient as well as the most costly.

===Exclusive Mode-Prefetching===
Exclusive-mode prefetching helps to reduce both the miss latencies and the message traffic associated with writes. Unlike read misses, which directly stall the processor for their entire duration, write misses affect performance[[File:exclusive_prefetch.jpg|200px|thumb|right| exclusive prefetch]] more indirectly, since writes can be buffered. A processor stalls while waiting for writes to complete in two situations: (i) when executing a write instruction if the write buffer is full, and (ii) during a read miss if previous writes must complete before the read miss can proceed. The impact of the former effect can be reduced through larger write buffers. In summary, exclusive-mode prefetching can provide significant performance benefits in architectures that have not already eliminated write stall times through aggressive implementations of weaker consistency models with lockup-free caches. Even if write stall times cannot be further reduced, exclusive-mode prefetching can improve performance somewhat by reducing the traffic associated with cache coherency.

==Where they stand now==
Prefetching began to fade due to its many disadvantages and memory speeds starting to catch up with processor transfer rates. One issue with prefetching is the increased complexity and overhead of handing the prefetching algorithms. There is great risk that this overhead can be larger than any benefits if the prefetching algorithm used is not accurate, and fetches too early or fetches too late to be effective. Performance must be improved significantly to overcome the overhead and complexity or be a waste. Another problem comes about with the introduction of multicore architectures. In a single core architecture, prefetching requests are able to come from one core. With multiple cores, prefetching requests can originate from any of the different cores. This puts additional stress on memory to not only deal with regular prefetch requests but also to handle prefetch from different sources, and greatly increases the overhead and complexity of logic. Coherence algorithms must account not only for sequential consistency issues, but also account for the data change in another location, the location of prefetched data. Flushing data becomes significantly more complicated. If prefetched data is stored in the data cache, then cache conflict, can become a significant problem. This is because the current and predictive sets of data must exist in the cache at the same time. Without prefetching, you could use the additional space to simply increase the cache size itself. The solution here would be to add extra hardware to act as a buffer to prevent utilizing this cache space.

==References==

*http://impact.asu.edu/cse520fa08/Srivatsan.pdf
*http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/ch10_sj

CSC 456 Spring 2012/10a AJ

2012-04-16T13:41:57Z

Jmmohund2: /* Exclusive Mode-Prefetching */

=Prefetching and Consistency Models=
==Intro==
While memory consistency models insure instructions are executed in correct order, these models can hinder efficiency. Since consistency models dictate order of execution, prefetching allows operations to complete quicker once their turn comes by bringing the necessarily data closer into the cache before it is needed. Prefetching is a hardware optimization technique in which the processor automatically prefetches ownership for any write operations that are delayed due to the program order requirement (e.g., by issuing prefetch-exclusive requests for any writes delayed in the write buffer), thus partially overlapping the service of the delayed writes with the operations preceding them in program order. This technique is only applicable to cache-based systems that use an invalidation-based protocol. This technique is suitable for statically scheduled processors.

==Methods==

===Fixed vs. Adaptive Sequential Prefetching===
Fixed sequential prefetching refers to prefetching of that occurs at a constant rate over time. Adaptive sequential prefetching, on the other hand, changes the rate of prefetching allowed over time. The prefetching rate is increased/decreased based on the count of successful prefetches. The rate is therefore dependent on workload and application (a start-up process will have a high rate of cold misses). While both methods improve efficiency, adaptive sequential prefetching is the most efficient as well as the most costly.

===Exclusive Mode-Prefetching===
Exclusive-mode prefetching helps to reduce both the miss latencies and the message traffic associated with writes. Unlike read misses, which directly stall the processor for their entire duration, write misses affect performance more indirectly, since writes can be buffered. A processor stalls while waiting for writes to complete in two situations: (i) when executing a write instruction if the write buffer is full, and (ii) during a read miss if previous writes must complete before the read miss can proceed. The impact of the former effect can be reduced through larger write buffers. In summary, exclusive-mode prefetching can provide significant performance benefits in [[File:exclusive_prefetch.jpg|200px|thumb|right| exclusive prefetch]]architectures that have not already eliminated write stall times through aggressive implementations of weaker consistency models with lockup-free caches. Even if write stall times cannot be further reduced, exclusive-mode prefetching can improve performance somewhat by reducing the traffic associated with cache coherency.

==Where they stand now==
Prefetching began to fade due to its many disadvantages and memory speeds starting to catch up with processor transfer rates. One issue with prefetching is the increased complexity and overhead of handing the prefetching algorithms. There is great risk that this overhead can be larger than any benefits if the prefetching algorithm used is not accurate, and fetches too early or fetches too late to be effective. Performance must be improved significantly to overcome the overhead and complexity or be a waste. Another problem comes about with the introduction of multicore architectures. In a single core architecture, prefetching requests are able to come from one core. With multiple cores, prefetching requests can originate from any of the different cores. This puts additional stress on memory to not only deal with regular prefetch requests but also to handle prefetch from different sources, and greatly increases the overhead and complexity of logic. Coherence algorithms must account not only for sequential consistency issues, but also account for the data change in another location, the location of prefetched data. Flushing data becomes significantly more complicated. If prefetched data is stored in the data cache, then cache conflict, can become a significant problem. This is because the current and predictive sets of data must exist in the cache at the same time. Without prefetching, you could use the additional space to simply increase the cache size itself. The solution here would be to add extra hardware to act as a buffer to prevent utilizing this cache space.

==References==

*http://impact.asu.edu/cse520fa08/Srivatsan.pdf
*http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/ch10_sj

CSC 456 Spring 2012/10a AJ

2012-04-16T13:41:28Z

Jmmohund2: /* Exclusive Mode-Prefetching */

=Prefetching and Consistency Models=
==Intro==
While memory consistency models insure instructions are executed in correct order, these models can hinder efficiency. Since consistency models dictate order of execution, prefetching allows operations to complete quicker once their turn comes by bringing the necessarily data closer into the cache before it is needed. Prefetching is a hardware optimization technique in which the processor automatically prefetches ownership for any write operations that are delayed due to the program order requirement (e.g., by issuing prefetch-exclusive requests for any writes delayed in the write buffer), thus partially overlapping the service of the delayed writes with the operations preceding them in program order. This technique is only applicable to cache-based systems that use an invalidation-based protocol. This technique is suitable for statically scheduled processors.

==Methods==

===Fixed vs. Adaptive Sequential Prefetching===
Fixed sequential prefetching refers to prefetching of that occurs at a constant rate over time. Adaptive sequential prefetching, on the other hand, changes the rate of prefetching allowed over time. The prefetching rate is increased/decreased based on the count of successful prefetches. The rate is therefore dependent on workload and application (a start-up process will have a high rate of cold misses). While both methods improve efficiency, adaptive sequential prefetching is the most efficient as well as the most costly.

===Exclusive Mode-Prefetching===
Exclusive-mode prefetching helps to reduce both the miss latencies and the message traffic associated with writes. Unlike read misses, which directly stall the processor for their entire duration, write misses affect performance more indirectly, since writes can be buffered. A processor stalls while waiting for writes to complete in two situations: (i) when executing a write instruction if the write buffer is full, and (ii) during a read miss if previous writes must complete before the read miss can proceed. The impact of the former effect can be reduced through larger write buffers. In summary, exclusive-mode prefetching can provide significant performance benefits in architectures that have not already eliminated write stall times through aggressive implementations of weaker consistency models with lockup-free caches. Even if write stall times cannot be further reduced, exclusive-mode prefetching can improve performance somewhat by reducing the traffic associated with cache coherency.[[File:exclusive_prefetch.jpg|200px|thumb|right| exclusive prefetch]]

==Where they stand now==
Prefetching began to fade due to its many disadvantages and memory speeds starting to catch up with processor transfer rates. One issue with prefetching is the increased complexity and overhead of handing the prefetching algorithms. There is great risk that this overhead can be larger than any benefits if the prefetching algorithm used is not accurate, and fetches too early or fetches too late to be effective. Performance must be improved significantly to overcome the overhead and complexity or be a waste. Another problem comes about with the introduction of multicore architectures. In a single core architecture, prefetching requests are able to come from one core. With multiple cores, prefetching requests can originate from any of the different cores. This puts additional stress on memory to not only deal with regular prefetch requests but also to handle prefetch from different sources, and greatly increases the overhead and complexity of logic. Coherence algorithms must account not only for sequential consistency issues, but also account for the data change in another location, the location of prefetched data. Flushing data becomes significantly more complicated. If prefetched data is stored in the data cache, then cache conflict, can become a significant problem. This is because the current and predictive sets of data must exist in the cache at the same time. Without prefetching, you could use the additional space to simply increase the cache size itself. The solution here would be to add extra hardware to act as a buffer to prevent utilizing this cache space.

==References==

*http://impact.asu.edu/cse520fa08/Srivatsan.pdf
*http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/ch10_sj

File:Exclusive prefetch.jpg

2012-04-16T13:40:21Z

Jmmohund2:

CSC 456 Spring 2012/10a AJ

2012-04-15T21:17:42Z

Jmmohund2: /* Intro */

=Prefetching and Consistency Models=
==Intro==
While memory consistency models insure instructions are executed in correct order, these models can hinder efficiency. Since consistency models dictate order of execution, prefetching allows operations to complete quicker once their turn comes by bringing the necessarily data closer into the cache before it is needed. Prefetching is a hardware optimization technique in which the processor automatically prefetches ownership for any write operations that are delayed due to the program order requirement (e.g., by issuing prefetch-exclusive requests for any writes delayed in the write buffer), thus partially overlapping the service of the delayed writes with the operations preceding them in program order. This technique is only applicable to cache-based systems that use an invalidation-based protocol. This technique is suitable for statically scheduled processors.

==Methods==

===Fixed vs. Adaptive Sequential Prefetching===
Fixed sequential prefetching refers to prefetching of that occurs at a constant rate over time. Adaptive sequential prefetching, on the other hand, changes the rate of prefetching allowed over time. The prefetching rate is increased/decreased based on the count of successful prefetches. The rate is therefore dependent on workload and application (a start-up process will have a high rate of cold misses). While both methods improve efficiency, adaptive sequential prefetching is the most efficient as well as the most costly.

===Exclusive Mode-Prefetching===
Exclusive-mode prefetching helps to reduce both the miss latencies and the message traffic associated with writes. Unlike read misses, which directly stall the processor for their entire duration, write misses affect performance more indirectly, since writes can be buffered. A processor stalls while waiting for writes to complete in two situations: (i) when executing a write instruction if the write buffer is full, and (ii) during a read miss if previous writes must complete before the read miss can proceed. The impact of the former effect can be reduced through larger write buffers. In summary, exclusive-mode prefetching can provide significant performance benefits in architectures that have not already eliminated write stall times through aggressive implementations of weaker consistency models with lockup-free caches. Even if write stall times cannot be further reduced, exclusive-mode prefetching can improve performance somewhat by reducing the traffic associated with cache coherency.

==Where they stand now==
Prefetching began to fade due to its many disadvantages and memory speeds starting to catch up with processor transfer rates. One issue with prefetching is the increased complexity and overhead of handing the prefetching algorithms. There is great risk that this overhead can be larger than any benefits if the prefetching algorithm used is not accurate, and fetches too early or fetches too late to be effective. Performance must be improved significantly to overcome the overhead and complexity or be a waste. Another problem comes about with the introduction of multicore architectures. In a single core architecture, prefetching requests are able to come from one core. With multiple cores, prefetching requests can originate from any of the different cores. This puts additional stress on memory to not only deal with regular prefetch requests but also to handle prefetch from different sources, and greatly increases the overhead and complexity of logic. Coherence algorithms must account not only for sequential consistency issues, but also account for the data change in another location, the location of prefetched data. Flushing data becomes significantly more complicated. If prefetched data is stored in the data cache, then cache conflict, can become a significant problem. This is because the current and predictive sets of data must exist in the cache at the same time. Without prefetching, you could use the additional space to simply increase the cache size itself. The solution here would be to add extra hardware to act as a buffer to prevent utilizing this cache space.

==References==

*http://impact.asu.edu/cse520fa08/Srivatsan.pdf
*http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/ch10_sj

CSC 456 Spring 2012/10a AJ

2012-04-15T21:15:40Z

Jmmohund2: /* Why? */

=Prefetching and Consistency Models=
==Intro==
While memory consistency models insure instructions are executed in correct order, these models can hinder efficiency. Since consistency models dictate order of execution, prefetching allows operations to complete quicker once their turn comes by bringing the necessarily data closer into the cache before it is needed.

A solution that was being explored in the 1990 was prefetching which is a hardware optimization technique in which the processor automatically prefetches ownership for any write operations that are delayed due to the program order requirement (e.g., by issuing prefetch-exclusive requests for any writes delayed in the write buffer), thus partially overlapping the service of the delayed writes with the operations preceding them in program order. This technique is only applicable to cache-based systems that use an invalidation-based protocol. This technique is suitable for statically scheduled processors.

==Methods==

===Fixed vs. Adaptive Sequential Prefetching===
Fixed sequential prefetching refers to prefetching of that occurs at a constant rate over time. Adaptive sequential prefetching, on the other hand, changes the rate of prefetching allowed over time. The prefetching rate is increased/decreased based on the count of successful prefetches. The rate is therefore dependent on workload and application (a start-up process will have a high rate of cold misses). While both methods improve efficiency, adaptive sequential prefetching is the most efficient as well as the most costly.

===Exclusive Mode-Prefetching===
Exclusive-mode prefetching helps to reduce both the miss latencies and the message traffic associated with writes. Unlike read misses, which directly stall the processor for their entire duration, write misses affect performance more indirectly, since writes can be buffered. A processor stalls while waiting for writes to complete in two situations: (i) when executing a write instruction if the write buffer is full, and (ii) during a read miss if previous writes must complete before the read miss can proceed. The impact of the former effect can be reduced through larger write buffers. In summary, exclusive-mode prefetching can provide significant performance benefits in architectures that have not already eliminated write stall times through aggressive implementations of weaker consistency models with lockup-free caches. Even if write stall times cannot be further reduced, exclusive-mode prefetching can improve performance somewhat by reducing the traffic associated with cache coherency.

==Where they stand now==
Prefetching began to fade due to its many disadvantages and memory speeds starting to catch up with processor transfer rates. One issue with prefetching is the increased complexity and overhead of handing the prefetching algorithms. There is great risk that this overhead can be larger than any benefits if the prefetching algorithm used is not accurate, and fetches too early or fetches too late to be effective. Performance must be improved significantly to overcome the overhead and complexity or be a waste. Another problem comes about with the introduction of multicore architectures. In a single core architecture, prefetching requests are able to come from one core. With multiple cores, prefetching requests can originate from any of the different cores. This puts additional stress on memory to not only deal with regular prefetch requests but also to handle prefetch from different sources, and greatly increases the overhead and complexity of logic. Coherence algorithms must account not only for sequential consistency issues, but also account for the data change in another location, the location of prefetched data. Flushing data becomes significantly more complicated. If prefetched data is stored in the data cache, then cache conflict, can become a significant problem. This is because the current and predictive sets of data must exist in the cache at the same time. Without prefetching, you could use the additional space to simply increase the cache size itself. The solution here would be to add extra hardware to act as a buffer to prevent utilizing this cache space.

==References==

*http://impact.asu.edu/cse520fa08/Srivatsan.pdf
*http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/ch10_sj

CSC 456 Spring 2012/10a AJ

2012-04-15T20:47:36Z

Jmmohund2: /* Methods */

CSC 456 Spring 2012/10a AJ

2012-04-15T20:20:50Z

Jmmohund2: /* References */

CSC 456 Spring 2012/10a AJ

2012-04-15T20:20:15Z

Jmmohund2: /* References */

CSC 456 Spring 2012/10a AJ

2012-04-15T20:20:01Z

Jmmohund2: /* Intro */

CSC 456 Spring 2012/10a AJ

2012-04-11T17:06:03Z

Jmmohund2: /* Prefetching and Consistency Models */

=Prefetching and Consistency Models=
==Intro==
==Methods==
==Where they stand now==
===Why?===

CSC 456 Spring 2012/ch7 MN

2012-03-19T18:03:19Z

Jmmohund2: /* MOESI */

==Introduction==
Though the migration from [http://en.wikipedia.org/wiki/Uniprocessor_system uniprocessor system] to [http://en.wikipedia.org/wiki/Multiprocessing multiprocessing] systems is not new, the world of parallel computers is undergoing a continuous change. Parallel computers, which started as high-end super-computing systems for carrying out huge calculations, are now ubiquitous and are present in all mainstream architectures for servers, desktops, and embedded systems. In order to design parallel architectures to meet programmer's needs and expectations more closely, exciting and challenging changes exist. The three main areas which are being considered by scientists today are: [http://en.wikipedia.org/wiki/Cache_coherence cache coherence], memory consistency and [http://en.wikipedia.org/wiki/Synchronization_%28computer_science%29 synchronization].

=Cache Coherence Problem=

[[Image:sharedmem.jpg|thumbnail|right|600px|Shared Memory system with dedicated Cache for each processor[[#4foot|[4]]]]]

In a system with single processor (single core), maintaining cache coherence is simple and easy but in a multiprocessor system, it is much more complicated. Data can be present in any processor's cache and the protocol needs to ensure that the data is same in all caches. If it cannot ensure that all the caches are same, then it needs to flag a cache line indicating that it is not updated.

In the figure shown here, this is a 4 processor shared memory system where each processor has its own cache. Suppose processor P1 reads memory location M1 and stores it in its local cache. Then, if P2 reads same location memory location then M1 gets stored in P2’s cache. Now, if P1 changes value of M1, two copies of same data, residing in different caches will become different. When P2 operates on M1, it uses the stale value of M1 that was stored in its cache. It is the responsibility of cache coherence protocol to prevent this. Hardware support is needed to provide a coherent view of data in multiple caches. This is known as write propagation requirement.

One may think that cache write policy can provide cache coherence, but this is incorrect. Cache write policy only controls how a change in value of cache is propagated to lower level cache or main memory. It is not responsible for propagating changes to other caches.

==Cache Coherence Protocols==

The two basic methods to utilize the inter-core bus to notify other cores when a core changes something in its cache are '''update''' and '''invalidate'''. In the update method, if variable 'x' is modified by core 1, core 1 has to send the updated value of 'x' onto the inter-core bus. Each cache listens to the inter-core bus and if a cache sees a variable on the bus which it has a copy of, it will read the updated value. This ensures that all caches have the most up-to-date value of the variable.[[#3foot|[3]]]

In case of invalidation, an invalidation message is sent onto the inter-core bus when a variable is changed. The other caches will read this invalidation signal and if its core attempts to access that variable, it will result in a cache miss and the variable will be read from main memory.

The update method results in significant amount of traffic on the inter-core bus as the update signal is sent onto the bus every time the variable is updated. The invalidation method only requires that an invalidation signal be sent the first time a variable is altered; this is why the invalidation method is the preferred method.

In order to improve cache coherence performance over the years, several protocols have been proposed.

===MSI===
[[Image:MSI.jpg|thumbnail|right|150px|MSI State Diagram]]
MSI stands for Modified, Shared, and Invalid, based on the three states that a line of cache can be in. The modified state means that a variable in the cache has been modified and therefore has a different value than that found in main memory; the cache is responsible for writing the variable back to main memory. The shared state means that the variable exists in at least one cache and is not modified; the cache can evict the variable without writing it back to the main memory. The invalid state means that the value of the variable has been modified by another cache and this value is invalid; the cache must read a new value from main memory (or another cache).

A drawback to this protocol occurs when a single processor wants to read blocks and then write to them without another processor sharing that block. After reading the block, a bus transaction places the block into a shared state. The write then occurs and another bus transaction is sent to invalidate the shared copy. This second transaction is useless as no other processors are sharing the block, but the MSI protocol has no way to specify this.

===MESI===
[[Image:MESI.jpg|thumbnail|right|150px|MSI State Diagram]]
MESI stands for Modified, Exclusive, Shared, and Invalid. The modified and invalid states are the same for this protocol as they are for the MSI protocol. This protocol introduces a new state; the exclusive state. The exclusive state means that the variable is in only this cache and the value of it matches the value within the main memory. This now means that the shared state indicates that the variable is contained in more than one cache.

===MOSI===

The MOSI protocol is identical to the MSI protocol except that it adds an owned state. The owned state means that the processor "owns" the variable and will provide the current value to other caches when requested (or at least it will decide if it will provide it when asked). This is useful because another cache will not have to read the value from main memory and will receive it from the owning cache much, much, faster.

===MOESI===
[[Image:MOESI.jpg|thumbnail|right|150px|MSI State Diagram]]
The MOESI protocol is a combination of the MESI and MOSI protocols.

=Memory Consistency Problem=

Memory consistency deals with the ordering of memory operations (load and store) to different memory locations. In a single processor system, code will execute correctly if the compiler preserves the order of the access to synchronization variables and other dependent variables. But in shared memory model with multiple processors, two threads could access a shared data element, such as a synchronization variable, the output of the threads would change based on which thread, accesses the shared data element earlier. If this were to occur, then the program output may not be the value expected. Maintaining program order is very important for memory consistency but it comes with performance degradation.

== Memory Consistency Models ==

The memory consistency model of a shared-memory multiprocessor is a formal speciﬁcation of how the memory system appears to the programmer. It eliminates the gap between the behavior expected by the programmer and the actual behavior supported by a system. Effectively, the consistency model places restrictions on the values that can be returned by a read, in a shared-memory program execution.

In a single processor system, in order to maintain memory consistency, it needs to ensure that the compiler preserves the program order when accessing synchronization variables. But in a multiprocessor system, it is required to ensure that accesses of one processor appear to execute in program order to all other processors, at least partially.

=== Memory semantics in Uniprocessor systems ===

Uniprocessor languages use simple sequential semantics for memory operations, which allow the programmer to assume that all memory operations will occur one at a time in the sequential order speciﬁed by the program. Thus, one can expect for the value in a particular location be the same for a read as it was for the last write because of the sequential program execution. It is sufﬁcient to only maintain uniprocessor data and control dependences. The compiler and hardware can freely reorder operations to different locations if the uniprocessor data and control dependences are respected. This enables compiler optimizations such as register allocation, code motion, and loop transformations, and hardware optimizations, such as pipelining, multiple issue, write buffer bypassing and forwarding, and lockup-free caches, all of which lead to overlapping and reordering of memory operations.[[#4foot|[4]]]

=== Memory semantics in multiprocessor systems ===

Programmer's implicit expectations are:

* memory accesses in a processor takes place according to the program order.
* Each memory access is performed atomically.

A strong consistency model attempting uniprocessor-like consistency could cause global bottleneck, costing performance. Thus, '''''weak''''' consistency models are deployed to improve performance. The advanatges of such models are:

* They support out-of-order execution within individual CPUs
* Relaxes latency issues with near-simultaneous accesses by different CPUs

The following are the various consistency models and it is the programmer who must take into account the memory consistency model to create correct software:
{| class="wikitable sortable" border="1"
|+ Sortable Table of Memory Consistency Models
|-
! scope="col" class="unsortable" | Type of Constancy Modle
! scope="col" | Strictness rank, 1-10(lowest-highest)
|-
| [http://en.wikipedia.org/wiki/Linearizability linearizability (also known as strict or atomic consistency)]|| 3
|-
| [http://en.wikipedia.org/wiki/Sequential_consistency sequential consistency]|| 3
|-
| [http://en.wikipedia.org/wiki/Causal_consistency causal consistency]|| 1
|-
|[http://en.wikipedia.org/wiki/Release_consistency release consistency]|| 3
|-
| [http://en.wikipedia.org/wiki/Eventual_consistency eventual consistency ]|| 2
|-
|[http://en.wikipedia.org/wiki/Delta_consistency delta consistency]|| 1
|-
| [http://en.wikipedia.org/wiki/PRAM_consistency PRAM consistency (also known as FIFO consistency)]|| 1
|-
| [http://en.wikipedia.org/wiki/Weak_consistency weak consistency]|| 1
|-
|[http://en.wikipedia.org/wiki/Vector-field_consistency vector-field consistency]|| 3
|-
| fork consistency|| 2
|-
|[http://en.wikipedia.org/wiki/Serializability serializability]|| 3
|-
|one-copy serializability|| 3
|-
|entry consistency|| 3
|}

== Memory Coherence and Shared Virtual Memory ==

[[Image:Centralized.png|thumbnail|right|350px|'''Figure 2''' Example of a centralized system. Here, one processor monitors all pages in the cache.]]
[[Image:Distributed.png|thumbnail|right|350px|'''Figure 3''' Example of a distributed system. Here, each processor monitors a different subset of pages in the cache.]]
The memory coherence problem in a shared virtual memory system and in multicache systems are different. In a multicache multiprocessor, there are processors sharing a physical memory through their private caches. The relatively small size of a cache and the fast bus connection to the shared memory, enables using a sophisticated coherence protocol for the multicache hardware such that the time delay of conflicting writes to a memory location is small.

In contrast, in a shared virtual memory on a loosely coupled multiprocessor which has no physically shared memory, and having a nontrivial communication cost between processors, conflicts are not likely to be solved with negligible delay, and they resemble much more a “page
fault” in a traditional virtual memory system. Thus, there are two design choices that greatly influence the implementation of a shared virtual memory: the granularity of the memory units (i.e., the “page size”) and the strategy for maintaining coherence.

Memory coherence strategies are classified based on how they deal with '''''page synchronization''''' and '''''page ownership'''''. The algorithms for memory coherence depend on the page fault handlers, their servers and the data structures used. So ''page table'' becomes an important part of these protocols.

Two classes of algorithms for solving the memory coherence problem are centralized manager (figure 2) and distributed manager (figure 3). In the centralized system, one processor is designated the "monitor". This processor contains a list of information for each page in the cache. The list includes the owner of the page (which processor accessed it last) and all the processors that have copies of the page. When a page is invalidated, then invalidation message is only sent to processors that have copies of the page. This differs from a bus-based system, as the invalidation message is broadcasted to all processors. A drawback to the centralized manager is that there is a bottleneck at the monitor processor. This bottleneck is alleviated by using a distributed manager.

The distributed manager is similar to a centralized manager, but instead of one processor monitoring all pages, a subset of the pages is given to each processor. So, processor 0 would only monitor pages 1 through i, and processor 1 would only monitor pages i+1 through n.[[#5foot|[5]]]

= Synchronization =

Two related concepts dealing with synchronization are '''process synchronization''' and '''data synchronization'''. Process synchronization is concerned with different processes committing to a certain sequence of actions. Data synchronization deals with maintaining data integrity across various copies of a dataset. Process synchronization primitives can be used to implement data synchronization.

[http://en.wikipedia.org/wiki/Mutual_exclusion Mutual exclusion] is the main requirement to be fulfilled in order to synchronize processes, and is needed in both single-processor and multiprocessor systems. There are various approaches to provide mutual exclusion in a system:

* Disabling interrupts
* Locks
* Mutex
* Semaphores
* Barriers
* Test and Set

The next section discusses if there is an alternative to implement mutual exclusion without requiring any hardware support. Peterson's algorithm is one such software solution for guaranteeing mutual exclusion.

== Hardware support ==

Exclusive locking assumes the worst and proceeds only after acquiring all locks such that no other thread can interfere. This is a ''pessimistic'' approach. In contrast, the ''optimistic'' approach proceeds with an update, hoping that it can be completed without any interference. This requires ''collision detection'' during the update. The optimistic approach is thus, more efficient in fine-grained operations.

Special instructions are provided by processors designed for multiprocessor operations in order to manage concurrent access to shared variables. Atomic instructions like [http://en.wikipedia.org/wiki/Test_and_Test-and-set test-and-set], [http://en.wikipedia.org/wiki/Fetch-and-add fetch-and-increment] and [http://en.wikipedia.org/wiki/Swap_%28computer_science%29 swap] were sufficient for early processors to implement mutexes for concurrent objects. Today, every modern processor relies on some form of read-modify-write atomic instruction such as [http://en.wikipedia.org/wiki/Compare-and-swap compare-and-swap], [http://en.wikipedia.org/wiki/Load-link/store-conditional LL/SC] etc. for the same.

=Resources=

[[#1body|1.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_jp#Memory_Consistency_Problem 
[[#2body|2.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_ss#Cache_Coherence 
[[#3body|3.]] http://www.windowsnetworking.com/articles_tutorials/Cache-Coherency.html 
[[#4body|4.]]http://web.sfc.keio.ac.jp/~rdv/keio/sfc/teaching/architecture/architecture-2007/lec08.html 
[[#5body|5.]]http://dl.acm.org/citation.cfm?id=75105

CSC 456 Spring 2012/ch7 MN

2012-03-19T18:03:08Z

Jmmohund2: /* MESI */

==Introduction==
Though the migration from [http://en.wikipedia.org/wiki/Uniprocessor_system uniprocessor system] to [http://en.wikipedia.org/wiki/Multiprocessing multiprocessing] systems is not new, the world of parallel computers is undergoing a continuous change. Parallel computers, which started as high-end super-computing systems for carrying out huge calculations, are now ubiquitous and are present in all mainstream architectures for servers, desktops, and embedded systems. In order to design parallel architectures to meet programmer's needs and expectations more closely, exciting and challenging changes exist. The three main areas which are being considered by scientists today are: [http://en.wikipedia.org/wiki/Cache_coherence cache coherence], memory consistency and [http://en.wikipedia.org/wiki/Synchronization_%28computer_science%29 synchronization].

=Cache Coherence Problem=

[[Image:sharedmem.jpg|thumbnail|right|600px|Shared Memory system with dedicated Cache for each processor[[#4foot|[4]]]]]

In a system with single processor (single core), maintaining cache coherence is simple and easy but in a multiprocessor system, it is much more complicated. Data can be present in any processor's cache and the protocol needs to ensure that the data is same in all caches. If it cannot ensure that all the caches are same, then it needs to flag a cache line indicating that it is not updated.

In the figure shown here, this is a 4 processor shared memory system where each processor has its own cache. Suppose processor P1 reads memory location M1 and stores it in its local cache. Then, if P2 reads same location memory location then M1 gets stored in P2’s cache. Now, if P1 changes value of M1, two copies of same data, residing in different caches will become different. When P2 operates on M1, it uses the stale value of M1 that was stored in its cache. It is the responsibility of cache coherence protocol to prevent this. Hardware support is needed to provide a coherent view of data in multiple caches. This is known as write propagation requirement.

One may think that cache write policy can provide cache coherence, but this is incorrect. Cache write policy only controls how a change in value of cache is propagated to lower level cache or main memory. It is not responsible for propagating changes to other caches.

==Cache Coherence Protocols==

The two basic methods to utilize the inter-core bus to notify other cores when a core changes something in its cache are '''update''' and '''invalidate'''. In the update method, if variable 'x' is modified by core 1, core 1 has to send the updated value of 'x' onto the inter-core bus. Each cache listens to the inter-core bus and if a cache sees a variable on the bus which it has a copy of, it will read the updated value. This ensures that all caches have the most up-to-date value of the variable.[[#3foot|[3]]]

In case of invalidation, an invalidation message is sent onto the inter-core bus when a variable is changed. The other caches will read this invalidation signal and if its core attempts to access that variable, it will result in a cache miss and the variable will be read from main memory.

The update method results in significant amount of traffic on the inter-core bus as the update signal is sent onto the bus every time the variable is updated. The invalidation method only requires that an invalidation signal be sent the first time a variable is altered; this is why the invalidation method is the preferred method.

In order to improve cache coherence performance over the years, several protocols have been proposed.

===MSI===
[[Image:MSI.jpg|thumbnail|right|150px|MSI State Diagram]]
MSI stands for Modified, Shared, and Invalid, based on the three states that a line of cache can be in. The modified state means that a variable in the cache has been modified and therefore has a different value than that found in main memory; the cache is responsible for writing the variable back to main memory. The shared state means that the variable exists in at least one cache and is not modified; the cache can evict the variable without writing it back to the main memory. The invalid state means that the value of the variable has been modified by another cache and this value is invalid; the cache must read a new value from main memory (or another cache).

A drawback to this protocol occurs when a single processor wants to read blocks and then write to them without another processor sharing that block. After reading the block, a bus transaction places the block into a shared state. The write then occurs and another bus transaction is sent to invalidate the shared copy. This second transaction is useless as no other processors are sharing the block, but the MSI protocol has no way to specify this.

===MESI===
[[Image:MESI.jpg|thumbnail|right|150px|MSI State Diagram]]
MESI stands for Modified, Exclusive, Shared, and Invalid. The modified and invalid states are the same for this protocol as they are for the MSI protocol. This protocol introduces a new state; the exclusive state. The exclusive state means that the variable is in only this cache and the value of it matches the value within the main memory. This now means that the shared state indicates that the variable is contained in more than one cache.

===MOSI===

The MOSI protocol is identical to the MSI protocol except that it adds an owned state. The owned state means that the processor "owns" the variable and will provide the current value to other caches when requested (or at least it will decide if it will provide it when asked). This is useful because another cache will not have to read the value from main memory and will receive it from the owning cache much, much, faster.

===MOESI===
[[Image:MOESI.jpg|thumbnail|right|350px|MSI State Diagram]]
The MOESI protocol is a combination of the MESI and MOSI protocols.

=Memory Consistency Problem=

Memory consistency deals with the ordering of memory operations (load and store) to different memory locations. In a single processor system, code will execute correctly if the compiler preserves the order of the access to synchronization variables and other dependent variables. But in shared memory model with multiple processors, two threads could access a shared data element, such as a synchronization variable, the output of the threads would change based on which thread, accesses the shared data element earlier. If this were to occur, then the program output may not be the value expected. Maintaining program order is very important for memory consistency but it comes with performance degradation.

== Memory Consistency Models ==

The memory consistency model of a shared-memory multiprocessor is a formal speciﬁcation of how the memory system appears to the programmer. It eliminates the gap between the behavior expected by the programmer and the actual behavior supported by a system. Effectively, the consistency model places restrictions on the values that can be returned by a read, in a shared-memory program execution.

In a single processor system, in order to maintain memory consistency, it needs to ensure that the compiler preserves the program order when accessing synchronization variables. But in a multiprocessor system, it is required to ensure that accesses of one processor appear to execute in program order to all other processors, at least partially.

=== Memory semantics in Uniprocessor systems ===

Uniprocessor languages use simple sequential semantics for memory operations, which allow the programmer to assume that all memory operations will occur one at a time in the sequential order speciﬁed by the program. Thus, one can expect for the value in a particular location be the same for a read as it was for the last write because of the sequential program execution. It is sufﬁcient to only maintain uniprocessor data and control dependences. The compiler and hardware can freely reorder operations to different locations if the uniprocessor data and control dependences are respected. This enables compiler optimizations such as register allocation, code motion, and loop transformations, and hardware optimizations, such as pipelining, multiple issue, write buffer bypassing and forwarding, and lockup-free caches, all of which lead to overlapping and reordering of memory operations.[[#4foot|[4]]]

=== Memory semantics in multiprocessor systems ===

Programmer's implicit expectations are:

* memory accesses in a processor takes place according to the program order.
* Each memory access is performed atomically.

A strong consistency model attempting uniprocessor-like consistency could cause global bottleneck, costing performance. Thus, '''''weak''''' consistency models are deployed to improve performance. The advanatges of such models are:

* They support out-of-order execution within individual CPUs
* Relaxes latency issues with near-simultaneous accesses by different CPUs

The following are the various consistency models and it is the programmer who must take into account the memory consistency model to create correct software:
{| class="wikitable sortable" border="1"
|+ Sortable Table of Memory Consistency Models
|-
! scope="col" class="unsortable" | Type of Constancy Modle
! scope="col" | Strictness rank, 1-10(lowest-highest)
|-
| [http://en.wikipedia.org/wiki/Linearizability linearizability (also known as strict or atomic consistency)]|| 3
|-
| [http://en.wikipedia.org/wiki/Sequential_consistency sequential consistency]|| 3
|-
| [http://en.wikipedia.org/wiki/Causal_consistency causal consistency]|| 1
|-
|[http://en.wikipedia.org/wiki/Release_consistency release consistency]|| 3
|-
| [http://en.wikipedia.org/wiki/Eventual_consistency eventual consistency ]|| 2
|-
|[http://en.wikipedia.org/wiki/Delta_consistency delta consistency]|| 1
|-
| [http://en.wikipedia.org/wiki/PRAM_consistency PRAM consistency (also known as FIFO consistency)]|| 1
|-
| [http://en.wikipedia.org/wiki/Weak_consistency weak consistency]|| 1
|-
|[http://en.wikipedia.org/wiki/Vector-field_consistency vector-field consistency]|| 3
|-
| fork consistency|| 2
|-
|[http://en.wikipedia.org/wiki/Serializability serializability]|| 3
|-
|one-copy serializability|| 3
|-
|entry consistency|| 3
|}

== Memory Coherence and Shared Virtual Memory ==

[[Image:Centralized.png|thumbnail|right|350px|'''Figure 2''' Example of a centralized system. Here, one processor monitors all pages in the cache.]]
[[Image:Distributed.png|thumbnail|right|350px|'''Figure 3''' Example of a distributed system. Here, each processor monitors a different subset of pages in the cache.]]
The memory coherence problem in a shared virtual memory system and in multicache systems are different. In a multicache multiprocessor, there are processors sharing a physical memory through their private caches. The relatively small size of a cache and the fast bus connection to the shared memory, enables using a sophisticated coherence protocol for the multicache hardware such that the time delay of conflicting writes to a memory location is small.

In contrast, in a shared virtual memory on a loosely coupled multiprocessor which has no physically shared memory, and having a nontrivial communication cost between processors, conflicts are not likely to be solved with negligible delay, and they resemble much more a “page
fault” in a traditional virtual memory system. Thus, there are two design choices that greatly influence the implementation of a shared virtual memory: the granularity of the memory units (i.e., the “page size”) and the strategy for maintaining coherence.

Memory coherence strategies are classified based on how they deal with '''''page synchronization''''' and '''''page ownership'''''. The algorithms for memory coherence depend on the page fault handlers, their servers and the data structures used. So ''page table'' becomes an important part of these protocols.

Two classes of algorithms for solving the memory coherence problem are centralized manager (figure 2) and distributed manager (figure 3). In the centralized system, one processor is designated the "monitor". This processor contains a list of information for each page in the cache. The list includes the owner of the page (which processor accessed it last) and all the processors that have copies of the page. When a page is invalidated, then invalidation message is only sent to processors that have copies of the page. This differs from a bus-based system, as the invalidation message is broadcasted to all processors. A drawback to the centralized manager is that there is a bottleneck at the monitor processor. This bottleneck is alleviated by using a distributed manager.

The distributed manager is similar to a centralized manager, but instead of one processor monitoring all pages, a subset of the pages is given to each processor. So, processor 0 would only monitor pages 1 through i, and processor 1 would only monitor pages i+1 through n.[[#5foot|[5]]]

= Synchronization =

Two related concepts dealing with synchronization are '''process synchronization''' and '''data synchronization'''. Process synchronization is concerned with different processes committing to a certain sequence of actions. Data synchronization deals with maintaining data integrity across various copies of a dataset. Process synchronization primitives can be used to implement data synchronization.

[http://en.wikipedia.org/wiki/Mutual_exclusion Mutual exclusion] is the main requirement to be fulfilled in order to synchronize processes, and is needed in both single-processor and multiprocessor systems. There are various approaches to provide mutual exclusion in a system:

* Disabling interrupts
* Locks
* Mutex
* Semaphores
* Barriers
* Test and Set

The next section discusses if there is an alternative to implement mutual exclusion without requiring any hardware support. Peterson's algorithm is one such software solution for guaranteeing mutual exclusion.

== Hardware support ==

Exclusive locking assumes the worst and proceeds only after acquiring all locks such that no other thread can interfere. This is a ''pessimistic'' approach. In contrast, the ''optimistic'' approach proceeds with an update, hoping that it can be completed without any interference. This requires ''collision detection'' during the update. The optimistic approach is thus, more efficient in fine-grained operations.

Special instructions are provided by processors designed for multiprocessor operations in order to manage concurrent access to shared variables. Atomic instructions like [http://en.wikipedia.org/wiki/Test_and_Test-and-set test-and-set], [http://en.wikipedia.org/wiki/Fetch-and-add fetch-and-increment] and [http://en.wikipedia.org/wiki/Swap_%28computer_science%29 swap] were sufficient for early processors to implement mutexes for concurrent objects. Today, every modern processor relies on some form of read-modify-write atomic instruction such as [http://en.wikipedia.org/wiki/Compare-and-swap compare-and-swap], [http://en.wikipedia.org/wiki/Load-link/store-conditional LL/SC] etc. for the same.

=Resources=

[[#1body|1.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_jp#Memory_Consistency_Problem 
[[#2body|2.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_ss#Cache_Coherence 
[[#3body|3.]] http://www.windowsnetworking.com/articles_tutorials/Cache-Coherency.html 
[[#4body|4.]]http://web.sfc.keio.ac.jp/~rdv/keio/sfc/teaching/architecture/architecture-2007/lec08.html 
[[#5body|5.]]http://dl.acm.org/citation.cfm?id=75105

CSC 456 Spring 2012/ch7 MN

2012-03-19T18:02:55Z

Jmmohund2: /* MSI */

==Introduction==
Though the migration from [http://en.wikipedia.org/wiki/Uniprocessor_system uniprocessor system] to [http://en.wikipedia.org/wiki/Multiprocessing multiprocessing] systems is not new, the world of parallel computers is undergoing a continuous change. Parallel computers, which started as high-end super-computing systems for carrying out huge calculations, are now ubiquitous and are present in all mainstream architectures for servers, desktops, and embedded systems. In order to design parallel architectures to meet programmer's needs and expectations more closely, exciting and challenging changes exist. The three main areas which are being considered by scientists today are: [http://en.wikipedia.org/wiki/Cache_coherence cache coherence], memory consistency and [http://en.wikipedia.org/wiki/Synchronization_%28computer_science%29 synchronization].

=Cache Coherence Problem=

[[Image:sharedmem.jpg|thumbnail|right|600px|Shared Memory system with dedicated Cache for each processor[[#4foot|[4]]]]]

In a system with single processor (single core), maintaining cache coherence is simple and easy but in a multiprocessor system, it is much more complicated. Data can be present in any processor's cache and the protocol needs to ensure that the data is same in all caches. If it cannot ensure that all the caches are same, then it needs to flag a cache line indicating that it is not updated.

In the figure shown here, this is a 4 processor shared memory system where each processor has its own cache. Suppose processor P1 reads memory location M1 and stores it in its local cache. Then, if P2 reads same location memory location then M1 gets stored in P2’s cache. Now, if P1 changes value of M1, two copies of same data, residing in different caches will become different. When P2 operates on M1, it uses the stale value of M1 that was stored in its cache. It is the responsibility of cache coherence protocol to prevent this. Hardware support is needed to provide a coherent view of data in multiple caches. This is known as write propagation requirement.

One may think that cache write policy can provide cache coherence, but this is incorrect. Cache write policy only controls how a change in value of cache is propagated to lower level cache or main memory. It is not responsible for propagating changes to other caches.

==Cache Coherence Protocols==

The two basic methods to utilize the inter-core bus to notify other cores when a core changes something in its cache are '''update''' and '''invalidate'''. In the update method, if variable 'x' is modified by core 1, core 1 has to send the updated value of 'x' onto the inter-core bus. Each cache listens to the inter-core bus and if a cache sees a variable on the bus which it has a copy of, it will read the updated value. This ensures that all caches have the most up-to-date value of the variable.[[#3foot|[3]]]

In case of invalidation, an invalidation message is sent onto the inter-core bus when a variable is changed. The other caches will read this invalidation signal and if its core attempts to access that variable, it will result in a cache miss and the variable will be read from main memory.

The update method results in significant amount of traffic on the inter-core bus as the update signal is sent onto the bus every time the variable is updated. The invalidation method only requires that an invalidation signal be sent the first time a variable is altered; this is why the invalidation method is the preferred method.

In order to improve cache coherence performance over the years, several protocols have been proposed.

===MSI===
[[Image:MSI.jpg|thumbnail|right|150px|MSI State Diagram]]
MSI stands for Modified, Shared, and Invalid, based on the three states that a line of cache can be in. The modified state means that a variable in the cache has been modified and therefore has a different value than that found in main memory; the cache is responsible for writing the variable back to main memory. The shared state means that the variable exists in at least one cache and is not modified; the cache can evict the variable without writing it back to the main memory. The invalid state means that the value of the variable has been modified by another cache and this value is invalid; the cache must read a new value from main memory (or another cache).

A drawback to this protocol occurs when a single processor wants to read blocks and then write to them without another processor sharing that block. After reading the block, a bus transaction places the block into a shared state. The write then occurs and another bus transaction is sent to invalidate the shared copy. This second transaction is useless as no other processors are sharing the block, but the MSI protocol has no way to specify this.

===MESI===
[[Image:MESI.jpg|thumbnail|right|350px|MSI State Diagram]]
MESI stands for Modified, Exclusive, Shared, and Invalid. The modified and invalid states are the same for this protocol as they are for the MSI protocol. This protocol introduces a new state; the exclusive state. The exclusive state means that the variable is in only this cache and the value of it matches the value within the main memory. This now means that the shared state indicates that the variable is contained in more than one cache.

===MOSI===

The MOSI protocol is identical to the MSI protocol except that it adds an owned state. The owned state means that the processor "owns" the variable and will provide the current value to other caches when requested (or at least it will decide if it will provide it when asked). This is useful because another cache will not have to read the value from main memory and will receive it from the owning cache much, much, faster.

===MOESI===
[[Image:MOESI.jpg|thumbnail|right|350px|MSI State Diagram]]
The MOESI protocol is a combination of the MESI and MOSI protocols.

=Memory Consistency Problem=

Memory consistency deals with the ordering of memory operations (load and store) to different memory locations. In a single processor system, code will execute correctly if the compiler preserves the order of the access to synchronization variables and other dependent variables. But in shared memory model with multiple processors, two threads could access a shared data element, such as a synchronization variable, the output of the threads would change based on which thread, accesses the shared data element earlier. If this were to occur, then the program output may not be the value expected. Maintaining program order is very important for memory consistency but it comes with performance degradation.

== Memory Consistency Models ==

The memory consistency model of a shared-memory multiprocessor is a formal speciﬁcation of how the memory system appears to the programmer. It eliminates the gap between the behavior expected by the programmer and the actual behavior supported by a system. Effectively, the consistency model places restrictions on the values that can be returned by a read, in a shared-memory program execution.

In a single processor system, in order to maintain memory consistency, it needs to ensure that the compiler preserves the program order when accessing synchronization variables. But in a multiprocessor system, it is required to ensure that accesses of one processor appear to execute in program order to all other processors, at least partially.

=== Memory semantics in Uniprocessor systems ===

Uniprocessor languages use simple sequential semantics for memory operations, which allow the programmer to assume that all memory operations will occur one at a time in the sequential order speciﬁed by the program. Thus, one can expect for the value in a particular location be the same for a read as it was for the last write because of the sequential program execution. It is sufﬁcient to only maintain uniprocessor data and control dependences. The compiler and hardware can freely reorder operations to different locations if the uniprocessor data and control dependences are respected. This enables compiler optimizations such as register allocation, code motion, and loop transformations, and hardware optimizations, such as pipelining, multiple issue, write buffer bypassing and forwarding, and lockup-free caches, all of which lead to overlapping and reordering of memory operations.[[#4foot|[4]]]

=== Memory semantics in multiprocessor systems ===

Programmer's implicit expectations are:

* memory accesses in a processor takes place according to the program order.
* Each memory access is performed atomically.

A strong consistency model attempting uniprocessor-like consistency could cause global bottleneck, costing performance. Thus, '''''weak''''' consistency models are deployed to improve performance. The advanatges of such models are:

* They support out-of-order execution within individual CPUs
* Relaxes latency issues with near-simultaneous accesses by different CPUs

The following are the various consistency models and it is the programmer who must take into account the memory consistency model to create correct software:
{| class="wikitable sortable" border="1"
|+ Sortable Table of Memory Consistency Models
|-
! scope="col" class="unsortable" | Type of Constancy Modle
! scope="col" | Strictness rank, 1-10(lowest-highest)
|-
| [http://en.wikipedia.org/wiki/Linearizability linearizability (also known as strict or atomic consistency)]|| 3
|-
| [http://en.wikipedia.org/wiki/Sequential_consistency sequential consistency]|| 3
|-
| [http://en.wikipedia.org/wiki/Causal_consistency causal consistency]|| 1
|-
|[http://en.wikipedia.org/wiki/Release_consistency release consistency]|| 3
|-
| [http://en.wikipedia.org/wiki/Eventual_consistency eventual consistency ]|| 2
|-
|[http://en.wikipedia.org/wiki/Delta_consistency delta consistency]|| 1
|-
| [http://en.wikipedia.org/wiki/PRAM_consistency PRAM consistency (also known as FIFO consistency)]|| 1
|-
| [http://en.wikipedia.org/wiki/Weak_consistency weak consistency]|| 1
|-
|[http://en.wikipedia.org/wiki/Vector-field_consistency vector-field consistency]|| 3
|-
| fork consistency|| 2
|-
|[http://en.wikipedia.org/wiki/Serializability serializability]|| 3
|-
|one-copy serializability|| 3
|-
|entry consistency|| 3
|}

== Memory Coherence and Shared Virtual Memory ==

[[Image:Centralized.png|thumbnail|right|350px|'''Figure 2''' Example of a centralized system. Here, one processor monitors all pages in the cache.]]
[[Image:Distributed.png|thumbnail|right|350px|'''Figure 3''' Example of a distributed system. Here, each processor monitors a different subset of pages in the cache.]]
The memory coherence problem in a shared virtual memory system and in multicache systems are different. In a multicache multiprocessor, there are processors sharing a physical memory through their private caches. The relatively small size of a cache and the fast bus connection to the shared memory, enables using a sophisticated coherence protocol for the multicache hardware such that the time delay of conflicting writes to a memory location is small.

In contrast, in a shared virtual memory on a loosely coupled multiprocessor which has no physically shared memory, and having a nontrivial communication cost between processors, conflicts are not likely to be solved with negligible delay, and they resemble much more a “page
fault” in a traditional virtual memory system. Thus, there are two design choices that greatly influence the implementation of a shared virtual memory: the granularity of the memory units (i.e., the “page size”) and the strategy for maintaining coherence.

Memory coherence strategies are classified based on how they deal with '''''page synchronization''''' and '''''page ownership'''''. The algorithms for memory coherence depend on the page fault handlers, their servers and the data structures used. So ''page table'' becomes an important part of these protocols.

Two classes of algorithms for solving the memory coherence problem are centralized manager (figure 2) and distributed manager (figure 3). In the centralized system, one processor is designated the "monitor". This processor contains a list of information for each page in the cache. The list includes the owner of the page (which processor accessed it last) and all the processors that have copies of the page. When a page is invalidated, then invalidation message is only sent to processors that have copies of the page. This differs from a bus-based system, as the invalidation message is broadcasted to all processors. A drawback to the centralized manager is that there is a bottleneck at the monitor processor. This bottleneck is alleviated by using a distributed manager.

The distributed manager is similar to a centralized manager, but instead of one processor monitoring all pages, a subset of the pages is given to each processor. So, processor 0 would only monitor pages 1 through i, and processor 1 would only monitor pages i+1 through n.[[#5foot|[5]]]

= Synchronization =

Two related concepts dealing with synchronization are '''process synchronization''' and '''data synchronization'''. Process synchronization is concerned with different processes committing to a certain sequence of actions. Data synchronization deals with maintaining data integrity across various copies of a dataset. Process synchronization primitives can be used to implement data synchronization.

[http://en.wikipedia.org/wiki/Mutual_exclusion Mutual exclusion] is the main requirement to be fulfilled in order to synchronize processes, and is needed in both single-processor and multiprocessor systems. There are various approaches to provide mutual exclusion in a system:

* Disabling interrupts
* Locks
* Mutex
* Semaphores
* Barriers
* Test and Set

The next section discusses if there is an alternative to implement mutual exclusion without requiring any hardware support. Peterson's algorithm is one such software solution for guaranteeing mutual exclusion.

== Hardware support ==

Exclusive locking assumes the worst and proceeds only after acquiring all locks such that no other thread can interfere. This is a ''pessimistic'' approach. In contrast, the ''optimistic'' approach proceeds with an update, hoping that it can be completed without any interference. This requires ''collision detection'' during the update. The optimistic approach is thus, more efficient in fine-grained operations.

Special instructions are provided by processors designed for multiprocessor operations in order to manage concurrent access to shared variables. Atomic instructions like [http://en.wikipedia.org/wiki/Test_and_Test-and-set test-and-set], [http://en.wikipedia.org/wiki/Fetch-and-add fetch-and-increment] and [http://en.wikipedia.org/wiki/Swap_%28computer_science%29 swap] were sufficient for early processors to implement mutexes for concurrent objects. Today, every modern processor relies on some form of read-modify-write atomic instruction such as [http://en.wikipedia.org/wiki/Compare-and-swap compare-and-swap], [http://en.wikipedia.org/wiki/Load-link/store-conditional LL/SC] etc. for the same.

=Resources=

[[#1body|1.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_jp#Memory_Consistency_Problem 
[[#2body|2.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_ss#Cache_Coherence 
[[#3body|3.]] http://www.windowsnetworking.com/articles_tutorials/Cache-Coherency.html 
[[#4body|4.]]http://web.sfc.keio.ac.jp/~rdv/keio/sfc/teaching/architecture/architecture-2007/lec08.html 
[[#5body|5.]]http://dl.acm.org/citation.cfm?id=75105

CSC 456 Spring 2012/ch7 MN

2012-03-19T18:02:29Z

Jmmohund2: /* MESI */

==Introduction==
Though the migration from [http://en.wikipedia.org/wiki/Uniprocessor_system uniprocessor system] to [http://en.wikipedia.org/wiki/Multiprocessing multiprocessing] systems is not new, the world of parallel computers is undergoing a continuous change. Parallel computers, which started as high-end super-computing systems for carrying out huge calculations, are now ubiquitous and are present in all mainstream architectures for servers, desktops, and embedded systems. In order to design parallel architectures to meet programmer's needs and expectations more closely, exciting and challenging changes exist. The three main areas which are being considered by scientists today are: [http://en.wikipedia.org/wiki/Cache_coherence cache coherence], memory consistency and [http://en.wikipedia.org/wiki/Synchronization_%28computer_science%29 synchronization].

=Cache Coherence Problem=

[[Image:sharedmem.jpg|thumbnail|right|600px|Shared Memory system with dedicated Cache for each processor[[#4foot|[4]]]]]

In a system with single processor (single core), maintaining cache coherence is simple and easy but in a multiprocessor system, it is much more complicated. Data can be present in any processor's cache and the protocol needs to ensure that the data is same in all caches. If it cannot ensure that all the caches are same, then it needs to flag a cache line indicating that it is not updated.

In the figure shown here, this is a 4 processor shared memory system where each processor has its own cache. Suppose processor P1 reads memory location M1 and stores it in its local cache. Then, if P2 reads same location memory location then M1 gets stored in P2’s cache. Now, if P1 changes value of M1, two copies of same data, residing in different caches will become different. When P2 operates on M1, it uses the stale value of M1 that was stored in its cache. It is the responsibility of cache coherence protocol to prevent this. Hardware support is needed to provide a coherent view of data in multiple caches. This is known as write propagation requirement.

One may think that cache write policy can provide cache coherence, but this is incorrect. Cache write policy only controls how a change in value of cache is propagated to lower level cache or main memory. It is not responsible for propagating changes to other caches.

==Cache Coherence Protocols==

The two basic methods to utilize the inter-core bus to notify other cores when a core changes something in its cache are '''update''' and '''invalidate'''. In the update method, if variable 'x' is modified by core 1, core 1 has to send the updated value of 'x' onto the inter-core bus. Each cache listens to the inter-core bus and if a cache sees a variable on the bus which it has a copy of, it will read the updated value. This ensures that all caches have the most up-to-date value of the variable.[[#3foot|[3]]]

In case of invalidation, an invalidation message is sent onto the inter-core bus when a variable is changed. The other caches will read this invalidation signal and if its core attempts to access that variable, it will result in a cache miss and the variable will be read from main memory.

The update method results in significant amount of traffic on the inter-core bus as the update signal is sent onto the bus every time the variable is updated. The invalidation method only requires that an invalidation signal be sent the first time a variable is altered; this is why the invalidation method is the preferred method.

In order to improve cache coherence performance over the years, several protocols have been proposed.

===MSI===
[[Image:MSI.jpg|thumbnail|right|350px|MSI State Diagram]]
MSI stands for Modified, Shared, and Invalid, based on the three states that a line of cache can be in. The modified state means that a variable in the cache has been modified and therefore has a different value than that found in main memory; the cache is responsible for writing the variable back to main memory. The shared state means that the variable exists in at least one cache and is not modified; the cache can evict the variable without writing it back to the main memory. The invalid state means that the value of the variable has been modified by another cache and this value is invalid; the cache must read a new value from main memory (or another cache).

A drawback to this protocol occurs when a single processor wants to read blocks and then write to them without another processor sharing that block. After reading the block, a bus transaction places the block into a shared state. The write then occurs and another bus transaction is sent to invalidate the shared copy. This second transaction is useless as no other processors are sharing the block, but the MSI protocol has no way to specify this.

===MESI===
[[Image:MESI.jpg|thumbnail|right|350px|MSI State Diagram]]
MESI stands for Modified, Exclusive, Shared, and Invalid. The modified and invalid states are the same for this protocol as they are for the MSI protocol. This protocol introduces a new state; the exclusive state. The exclusive state means that the variable is in only this cache and the value of it matches the value within the main memory. This now means that the shared state indicates that the variable is contained in more than one cache.

===MOSI===

The MOSI protocol is identical to the MSI protocol except that it adds an owned state. The owned state means that the processor "owns" the variable and will provide the current value to other caches when requested (or at least it will decide if it will provide it when asked). This is useful because another cache will not have to read the value from main memory and will receive it from the owning cache much, much, faster.

===MOESI===
[[Image:MOESI.jpg|thumbnail|right|350px|MSI State Diagram]]
The MOESI protocol is a combination of the MESI and MOSI protocols.

=Memory Consistency Problem=

Memory consistency deals with the ordering of memory operations (load and store) to different memory locations. In a single processor system, code will execute correctly if the compiler preserves the order of the access to synchronization variables and other dependent variables. But in shared memory model with multiple processors, two threads could access a shared data element, such as a synchronization variable, the output of the threads would change based on which thread, accesses the shared data element earlier. If this were to occur, then the program output may not be the value expected. Maintaining program order is very important for memory consistency but it comes with performance degradation.

== Memory Consistency Models ==

The memory consistency model of a shared-memory multiprocessor is a formal speciﬁcation of how the memory system appears to the programmer. It eliminates the gap between the behavior expected by the programmer and the actual behavior supported by a system. Effectively, the consistency model places restrictions on the values that can be returned by a read, in a shared-memory program execution.

In a single processor system, in order to maintain memory consistency, it needs to ensure that the compiler preserves the program order when accessing synchronization variables. But in a multiprocessor system, it is required to ensure that accesses of one processor appear to execute in program order to all other processors, at least partially.

=== Memory semantics in Uniprocessor systems ===

Uniprocessor languages use simple sequential semantics for memory operations, which allow the programmer to assume that all memory operations will occur one at a time in the sequential order speciﬁed by the program. Thus, one can expect for the value in a particular location be the same for a read as it was for the last write because of the sequential program execution. It is sufﬁcient to only maintain uniprocessor data and control dependences. The compiler and hardware can freely reorder operations to different locations if the uniprocessor data and control dependences are respected. This enables compiler optimizations such as register allocation, code motion, and loop transformations, and hardware optimizations, such as pipelining, multiple issue, write buffer bypassing and forwarding, and lockup-free caches, all of which lead to overlapping and reordering of memory operations.[[#4foot|[4]]]

=== Memory semantics in multiprocessor systems ===

Programmer's implicit expectations are:

* memory accesses in a processor takes place according to the program order.
* Each memory access is performed atomically.

A strong consistency model attempting uniprocessor-like consistency could cause global bottleneck, costing performance. Thus, '''''weak''''' consistency models are deployed to improve performance. The advanatges of such models are:

* They support out-of-order execution within individual CPUs
* Relaxes latency issues with near-simultaneous accesses by different CPUs

The following are the various consistency models and it is the programmer who must take into account the memory consistency model to create correct software:
{| class="wikitable sortable" border="1"
|+ Sortable Table of Memory Consistency Models
|-
! scope="col" class="unsortable" | Type of Constancy Modle
! scope="col" | Strictness rank, 1-10(lowest-highest)
|-
| [http://en.wikipedia.org/wiki/Linearizability linearizability (also known as strict or atomic consistency)]|| 3
|-
| [http://en.wikipedia.org/wiki/Sequential_consistency sequential consistency]|| 3
|-
| [http://en.wikipedia.org/wiki/Causal_consistency causal consistency]|| 1
|-
|[http://en.wikipedia.org/wiki/Release_consistency release consistency]|| 3
|-
| [http://en.wikipedia.org/wiki/Eventual_consistency eventual consistency ]|| 2
|-
|[http://en.wikipedia.org/wiki/Delta_consistency delta consistency]|| 1
|-
| [http://en.wikipedia.org/wiki/PRAM_consistency PRAM consistency (also known as FIFO consistency)]|| 1
|-
| [http://en.wikipedia.org/wiki/Weak_consistency weak consistency]|| 1
|-
|[http://en.wikipedia.org/wiki/Vector-field_consistency vector-field consistency]|| 3
|-
| fork consistency|| 2
|-
|[http://en.wikipedia.org/wiki/Serializability serializability]|| 3
|-
|one-copy serializability|| 3
|-
|entry consistency|| 3
|}

== Memory Coherence and Shared Virtual Memory ==

[[Image:Centralized.png|thumbnail|right|350px|'''Figure 2''' Example of a centralized system. Here, one processor monitors all pages in the cache.]]
[[Image:Distributed.png|thumbnail|right|350px|'''Figure 3''' Example of a distributed system. Here, each processor monitors a different subset of pages in the cache.]]
The memory coherence problem in a shared virtual memory system and in multicache systems are different. In a multicache multiprocessor, there are processors sharing a physical memory through their private caches. The relatively small size of a cache and the fast bus connection to the shared memory, enables using a sophisticated coherence protocol for the multicache hardware such that the time delay of conflicting writes to a memory location is small.

In contrast, in a shared virtual memory on a loosely coupled multiprocessor which has no physically shared memory, and having a nontrivial communication cost between processors, conflicts are not likely to be solved with negligible delay, and they resemble much more a “page
fault” in a traditional virtual memory system. Thus, there are two design choices that greatly influence the implementation of a shared virtual memory: the granularity of the memory units (i.e., the “page size”) and the strategy for maintaining coherence.

Memory coherence strategies are classified based on how they deal with '''''page synchronization''''' and '''''page ownership'''''. The algorithms for memory coherence depend on the page fault handlers, their servers and the data structures used. So ''page table'' becomes an important part of these protocols.

Two classes of algorithms for solving the memory coherence problem are centralized manager (figure 2) and distributed manager (figure 3). In the centralized system, one processor is designated the "monitor". This processor contains a list of information for each page in the cache. The list includes the owner of the page (which processor accessed it last) and all the processors that have copies of the page. When a page is invalidated, then invalidation message is only sent to processors that have copies of the page. This differs from a bus-based system, as the invalidation message is broadcasted to all processors. A drawback to the centralized manager is that there is a bottleneck at the monitor processor. This bottleneck is alleviated by using a distributed manager.

The distributed manager is similar to a centralized manager, but instead of one processor monitoring all pages, a subset of the pages is given to each processor. So, processor 0 would only monitor pages 1 through i, and processor 1 would only monitor pages i+1 through n.[[#5foot|[5]]]

= Synchronization =

Two related concepts dealing with synchronization are '''process synchronization''' and '''data synchronization'''. Process synchronization is concerned with different processes committing to a certain sequence of actions. Data synchronization deals with maintaining data integrity across various copies of a dataset. Process synchronization primitives can be used to implement data synchronization.

[http://en.wikipedia.org/wiki/Mutual_exclusion Mutual exclusion] is the main requirement to be fulfilled in order to synchronize processes, and is needed in both single-processor and multiprocessor systems. There are various approaches to provide mutual exclusion in a system:

* Disabling interrupts
* Locks
* Mutex
* Semaphores
* Barriers
* Test and Set

The next section discusses if there is an alternative to implement mutual exclusion without requiring any hardware support. Peterson's algorithm is one such software solution for guaranteeing mutual exclusion.

== Hardware support ==

Exclusive locking assumes the worst and proceeds only after acquiring all locks such that no other thread can interfere. This is a ''pessimistic'' approach. In contrast, the ''optimistic'' approach proceeds with an update, hoping that it can be completed without any interference. This requires ''collision detection'' during the update. The optimistic approach is thus, more efficient in fine-grained operations.

Special instructions are provided by processors designed for multiprocessor operations in order to manage concurrent access to shared variables. Atomic instructions like [http://en.wikipedia.org/wiki/Test_and_Test-and-set test-and-set], [http://en.wikipedia.org/wiki/Fetch-and-add fetch-and-increment] and [http://en.wikipedia.org/wiki/Swap_%28computer_science%29 swap] were sufficient for early processors to implement mutexes for concurrent objects. Today, every modern processor relies on some form of read-modify-write atomic instruction such as [http://en.wikipedia.org/wiki/Compare-and-swap compare-and-swap], [http://en.wikipedia.org/wiki/Load-link/store-conditional LL/SC] etc. for the same.

=Resources=

[[#1body|1.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_jp#Memory_Consistency_Problem 
[[#2body|2.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_ss#Cache_Coherence 
[[#3body|3.]] http://www.windowsnetworking.com/articles_tutorials/Cache-Coherency.html 
[[#4body|4.]]http://web.sfc.keio.ac.jp/~rdv/keio/sfc/teaching/architecture/architecture-2007/lec08.html 
[[#5body|5.]]http://dl.acm.org/citation.cfm?id=75105

CSC 456 Spring 2012/ch7 MN

2012-03-19T18:02:16Z

Jmmohund2: /* MSI */

==Introduction==
Though the migration from [http://en.wikipedia.org/wiki/Uniprocessor_system uniprocessor system] to [http://en.wikipedia.org/wiki/Multiprocessing multiprocessing] systems is not new, the world of parallel computers is undergoing a continuous change. Parallel computers, which started as high-end super-computing systems for carrying out huge calculations, are now ubiquitous and are present in all mainstream architectures for servers, desktops, and embedded systems. In order to design parallel architectures to meet programmer's needs and expectations more closely, exciting and challenging changes exist. The three main areas which are being considered by scientists today are: [http://en.wikipedia.org/wiki/Cache_coherence cache coherence], memory consistency and [http://en.wikipedia.org/wiki/Synchronization_%28computer_science%29 synchronization].

=Cache Coherence Problem=

[[Image:sharedmem.jpg|thumbnail|right|600px|Shared Memory system with dedicated Cache for each processor[[#4foot|[4]]]]]

In a system with single processor (single core), maintaining cache coherence is simple and easy but in a multiprocessor system, it is much more complicated. Data can be present in any processor's cache and the protocol needs to ensure that the data is same in all caches. If it cannot ensure that all the caches are same, then it needs to flag a cache line indicating that it is not updated.

In the figure shown here, this is a 4 processor shared memory system where each processor has its own cache. Suppose processor P1 reads memory location M1 and stores it in its local cache. Then, if P2 reads same location memory location then M1 gets stored in P2’s cache. Now, if P1 changes value of M1, two copies of same data, residing in different caches will become different. When P2 operates on M1, it uses the stale value of M1 that was stored in its cache. It is the responsibility of cache coherence protocol to prevent this. Hardware support is needed to provide a coherent view of data in multiple caches. This is known as write propagation requirement.

One may think that cache write policy can provide cache coherence, but this is incorrect. Cache write policy only controls how a change in value of cache is propagated to lower level cache or main memory. It is not responsible for propagating changes to other caches.

==Cache Coherence Protocols==

The two basic methods to utilize the inter-core bus to notify other cores when a core changes something in its cache are '''update''' and '''invalidate'''. In the update method, if variable 'x' is modified by core 1, core 1 has to send the updated value of 'x' onto the inter-core bus. Each cache listens to the inter-core bus and if a cache sees a variable on the bus which it has a copy of, it will read the updated value. This ensures that all caches have the most up-to-date value of the variable.[[#3foot|[3]]]

In case of invalidation, an invalidation message is sent onto the inter-core bus when a variable is changed. The other caches will read this invalidation signal and if its core attempts to access that variable, it will result in a cache miss and the variable will be read from main memory.

The update method results in significant amount of traffic on the inter-core bus as the update signal is sent onto the bus every time the variable is updated. The invalidation method only requires that an invalidation signal be sent the first time a variable is altered; this is why the invalidation method is the preferred method.

In order to improve cache coherence performance over the years, several protocols have been proposed.

===MSI===
[[Image:MSI.jpg|thumbnail|right|350px|MSI State Diagram]]
MSI stands for Modified, Shared, and Invalid, based on the three states that a line of cache can be in. The modified state means that a variable in the cache has been modified and therefore has a different value than that found in main memory; the cache is responsible for writing the variable back to main memory. The shared state means that the variable exists in at least one cache and is not modified; the cache can evict the variable without writing it back to the main memory. The invalid state means that the value of the variable has been modified by another cache and this value is invalid; the cache must read a new value from main memory (or another cache).

A drawback to this protocol occurs when a single processor wants to read blocks and then write to them without another processor sharing that block. After reading the block, a bus transaction places the block into a shared state. The write then occurs and another bus transaction is sent to invalidate the shared copy. This second transaction is useless as no other processors are sharing the block, but the MSI protocol has no way to specify this.

===MESI===
[[Image:MESI.jpg|thumbnail|right|600px|MSI State Diagram]]
MESI stands for Modified, Exclusive, Shared, and Invalid. The modified and invalid states are the same for this protocol as they are for the MSI protocol. This protocol introduces a new state; the exclusive state. The exclusive state means that the variable is in only this cache and the value of it matches the value within the main memory. This now means that the shared state indicates that the variable is contained in more than one cache.

===MOSI===

The MOSI protocol is identical to the MSI protocol except that it adds an owned state. The owned state means that the processor "owns" the variable and will provide the current value to other caches when requested (or at least it will decide if it will provide it when asked). This is useful because another cache will not have to read the value from main memory and will receive it from the owning cache much, much, faster.

===MOESI===
[[Image:MOESI.jpg|thumbnail|right|350px|MSI State Diagram]]
The MOESI protocol is a combination of the MESI and MOSI protocols.

=Memory Consistency Problem=

Memory consistency deals with the ordering of memory operations (load and store) to different memory locations. In a single processor system, code will execute correctly if the compiler preserves the order of the access to synchronization variables and other dependent variables. But in shared memory model with multiple processors, two threads could access a shared data element, such as a synchronization variable, the output of the threads would change based on which thread, accesses the shared data element earlier. If this were to occur, then the program output may not be the value expected. Maintaining program order is very important for memory consistency but it comes with performance degradation.

== Memory Consistency Models ==

The memory consistency model of a shared-memory multiprocessor is a formal speciﬁcation of how the memory system appears to the programmer. It eliminates the gap between the behavior expected by the programmer and the actual behavior supported by a system. Effectively, the consistency model places restrictions on the values that can be returned by a read, in a shared-memory program execution.

In a single processor system, in order to maintain memory consistency, it needs to ensure that the compiler preserves the program order when accessing synchronization variables. But in a multiprocessor system, it is required to ensure that accesses of one processor appear to execute in program order to all other processors, at least partially.

=== Memory semantics in Uniprocessor systems ===

Uniprocessor languages use simple sequential semantics for memory operations, which allow the programmer to assume that all memory operations will occur one at a time in the sequential order speciﬁed by the program. Thus, one can expect for the value in a particular location be the same for a read as it was for the last write because of the sequential program execution. It is sufﬁcient to only maintain uniprocessor data and control dependences. The compiler and hardware can freely reorder operations to different locations if the uniprocessor data and control dependences are respected. This enables compiler optimizations such as register allocation, code motion, and loop transformations, and hardware optimizations, such as pipelining, multiple issue, write buffer bypassing and forwarding, and lockup-free caches, all of which lead to overlapping and reordering of memory operations.[[#4foot|[4]]]

=== Memory semantics in multiprocessor systems ===

Programmer's implicit expectations are:

* memory accesses in a processor takes place according to the program order.
* Each memory access is performed atomically.

A strong consistency model attempting uniprocessor-like consistency could cause global bottleneck, costing performance. Thus, '''''weak''''' consistency models are deployed to improve performance. The advanatges of such models are:

* They support out-of-order execution within individual CPUs
* Relaxes latency issues with near-simultaneous accesses by different CPUs

The following are the various consistency models and it is the programmer who must take into account the memory consistency model to create correct software:
{| class="wikitable sortable" border="1"
|+ Sortable Table of Memory Consistency Models
|-
! scope="col" class="unsortable" | Type of Constancy Modle
! scope="col" | Strictness rank, 1-10(lowest-highest)
|-
| [http://en.wikipedia.org/wiki/Linearizability linearizability (also known as strict or atomic consistency)]|| 3
|-
| [http://en.wikipedia.org/wiki/Sequential_consistency sequential consistency]|| 3
|-
| [http://en.wikipedia.org/wiki/Causal_consistency causal consistency]|| 1
|-
|[http://en.wikipedia.org/wiki/Release_consistency release consistency]|| 3
|-
| [http://en.wikipedia.org/wiki/Eventual_consistency eventual consistency ]|| 2
|-
|[http://en.wikipedia.org/wiki/Delta_consistency delta consistency]|| 1
|-
| [http://en.wikipedia.org/wiki/PRAM_consistency PRAM consistency (also known as FIFO consistency)]|| 1
|-
| [http://en.wikipedia.org/wiki/Weak_consistency weak consistency]|| 1
|-
|[http://en.wikipedia.org/wiki/Vector-field_consistency vector-field consistency]|| 3
|-
| fork consistency|| 2
|-
|[http://en.wikipedia.org/wiki/Serializability serializability]|| 3
|-
|one-copy serializability|| 3
|-
|entry consistency|| 3
|}

== Memory Coherence and Shared Virtual Memory ==

[[Image:Centralized.png|thumbnail|right|350px|'''Figure 2''' Example of a centralized system. Here, one processor monitors all pages in the cache.]]
[[Image:Distributed.png|thumbnail|right|350px|'''Figure 3''' Example of a distributed system. Here, each processor monitors a different subset of pages in the cache.]]
The memory coherence problem in a shared virtual memory system and in multicache systems are different. In a multicache multiprocessor, there are processors sharing a physical memory through their private caches. The relatively small size of a cache and the fast bus connection to the shared memory, enables using a sophisticated coherence protocol for the multicache hardware such that the time delay of conflicting writes to a memory location is small.

In contrast, in a shared virtual memory on a loosely coupled multiprocessor which has no physically shared memory, and having a nontrivial communication cost between processors, conflicts are not likely to be solved with negligible delay, and they resemble much more a “page
fault” in a traditional virtual memory system. Thus, there are two design choices that greatly influence the implementation of a shared virtual memory: the granularity of the memory units (i.e., the “page size”) and the strategy for maintaining coherence.

Memory coherence strategies are classified based on how they deal with '''''page synchronization''''' and '''''page ownership'''''. The algorithms for memory coherence depend on the page fault handlers, their servers and the data structures used. So ''page table'' becomes an important part of these protocols.

Two classes of algorithms for solving the memory coherence problem are centralized manager (figure 2) and distributed manager (figure 3). In the centralized system, one processor is designated the "monitor". This processor contains a list of information for each page in the cache. The list includes the owner of the page (which processor accessed it last) and all the processors that have copies of the page. When a page is invalidated, then invalidation message is only sent to processors that have copies of the page. This differs from a bus-based system, as the invalidation message is broadcasted to all processors. A drawback to the centralized manager is that there is a bottleneck at the monitor processor. This bottleneck is alleviated by using a distributed manager.

The distributed manager is similar to a centralized manager, but instead of one processor monitoring all pages, a subset of the pages is given to each processor. So, processor 0 would only monitor pages 1 through i, and processor 1 would only monitor pages i+1 through n.[[#5foot|[5]]]

= Synchronization =

Two related concepts dealing with synchronization are '''process synchronization''' and '''data synchronization'''. Process synchronization is concerned with different processes committing to a certain sequence of actions. Data synchronization deals with maintaining data integrity across various copies of a dataset. Process synchronization primitives can be used to implement data synchronization.

[http://en.wikipedia.org/wiki/Mutual_exclusion Mutual exclusion] is the main requirement to be fulfilled in order to synchronize processes, and is needed in both single-processor and multiprocessor systems. There are various approaches to provide mutual exclusion in a system:

* Disabling interrupts
* Locks
* Mutex
* Semaphores
* Barriers
* Test and Set

The next section discusses if there is an alternative to implement mutual exclusion without requiring any hardware support. Peterson's algorithm is one such software solution for guaranteeing mutual exclusion.

== Hardware support ==

Exclusive locking assumes the worst and proceeds only after acquiring all locks such that no other thread can interfere. This is a ''pessimistic'' approach. In contrast, the ''optimistic'' approach proceeds with an update, hoping that it can be completed without any interference. This requires ''collision detection'' during the update. The optimistic approach is thus, more efficient in fine-grained operations.

Special instructions are provided by processors designed for multiprocessor operations in order to manage concurrent access to shared variables. Atomic instructions like [http://en.wikipedia.org/wiki/Test_and_Test-and-set test-and-set], [http://en.wikipedia.org/wiki/Fetch-and-add fetch-and-increment] and [http://en.wikipedia.org/wiki/Swap_%28computer_science%29 swap] were sufficient for early processors to implement mutexes for concurrent objects. Today, every modern processor relies on some form of read-modify-write atomic instruction such as [http://en.wikipedia.org/wiki/Compare-and-swap compare-and-swap], [http://en.wikipedia.org/wiki/Load-link/store-conditional LL/SC] etc. for the same.

=Resources=

[[#1body|1.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_jp#Memory_Consistency_Problem 
[[#2body|2.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_ss#Cache_Coherence 
[[#3body|3.]] http://www.windowsnetworking.com/articles_tutorials/Cache-Coherency.html 
[[#4body|4.]]http://web.sfc.keio.ac.jp/~rdv/keio/sfc/teaching/architecture/architecture-2007/lec08.html 
[[#5body|5.]]http://dl.acm.org/citation.cfm?id=75105

CSC 456 Spring 2012/ch7 MN

2012-03-19T18:01:59Z

Jmmohund2: /* MOESI */

==Introduction==
Though the migration from [http://en.wikipedia.org/wiki/Uniprocessor_system uniprocessor system] to [http://en.wikipedia.org/wiki/Multiprocessing multiprocessing] systems is not new, the world of parallel computers is undergoing a continuous change. Parallel computers, which started as high-end super-computing systems for carrying out huge calculations, are now ubiquitous and are present in all mainstream architectures for servers, desktops, and embedded systems. In order to design parallel architectures to meet programmer's needs and expectations more closely, exciting and challenging changes exist. The three main areas which are being considered by scientists today are: [http://en.wikipedia.org/wiki/Cache_coherence cache coherence], memory consistency and [http://en.wikipedia.org/wiki/Synchronization_%28computer_science%29 synchronization].

=Cache Coherence Problem=

[[Image:sharedmem.jpg|thumbnail|right|600px|Shared Memory system with dedicated Cache for each processor[[#4foot|[4]]]]]

In a system with single processor (single core), maintaining cache coherence is simple and easy but in a multiprocessor system, it is much more complicated. Data can be present in any processor's cache and the protocol needs to ensure that the data is same in all caches. If it cannot ensure that all the caches are same, then it needs to flag a cache line indicating that it is not updated.

In the figure shown here, this is a 4 processor shared memory system where each processor has its own cache. Suppose processor P1 reads memory location M1 and stores it in its local cache. Then, if P2 reads same location memory location then M1 gets stored in P2’s cache. Now, if P1 changes value of M1, two copies of same data, residing in different caches will become different. When P2 operates on M1, it uses the stale value of M1 that was stored in its cache. It is the responsibility of cache coherence protocol to prevent this. Hardware support is needed to provide a coherent view of data in multiple caches. This is known as write propagation requirement.

One may think that cache write policy can provide cache coherence, but this is incorrect. Cache write policy only controls how a change in value of cache is propagated to lower level cache or main memory. It is not responsible for propagating changes to other caches.

==Cache Coherence Protocols==

The two basic methods to utilize the inter-core bus to notify other cores when a core changes something in its cache are '''update''' and '''invalidate'''. In the update method, if variable 'x' is modified by core 1, core 1 has to send the updated value of 'x' onto the inter-core bus. Each cache listens to the inter-core bus and if a cache sees a variable on the bus which it has a copy of, it will read the updated value. This ensures that all caches have the most up-to-date value of the variable.[[#3foot|[3]]]

In case of invalidation, an invalidation message is sent onto the inter-core bus when a variable is changed. The other caches will read this invalidation signal and if its core attempts to access that variable, it will result in a cache miss and the variable will be read from main memory.

The update method results in significant amount of traffic on the inter-core bus as the update signal is sent onto the bus every time the variable is updated. The invalidation method only requires that an invalidation signal be sent the first time a variable is altered; this is why the invalidation method is the preferred method.

In order to improve cache coherence performance over the years, several protocols have been proposed.

===MSI===
[[Image:MSI.jpg|thumbnail|right|600px|MSI State Diagram]]
MSI stands for Modified, Shared, and Invalid, based on the three states that a line of cache can be in. The modified state means that a variable in the cache has been modified and therefore has a different value than that found in main memory; the cache is responsible for writing the variable back to main memory. The shared state means that the variable exists in at least one cache and is not modified; the cache can evict the variable without writing it back to the main memory. The invalid state means that the value of the variable has been modified by another cache and this value is invalid; the cache must read a new value from main memory (or another cache).

A drawback to this protocol occurs when a single processor wants to read blocks and then write to them without another processor sharing that block. After reading the block, a bus transaction places the block into a shared state. The write then occurs and another bus transaction is sent to invalidate the shared copy. This second transaction is useless as no other processors are sharing the block, but the MSI protocol has no way to specify this.

===MESI===
[[Image:MESI.jpg|thumbnail|right|600px|MSI State Diagram]]
MESI stands for Modified, Exclusive, Shared, and Invalid. The modified and invalid states are the same for this protocol as they are for the MSI protocol. This protocol introduces a new state; the exclusive state. The exclusive state means that the variable is in only this cache and the value of it matches the value within the main memory. This now means that the shared state indicates that the variable is contained in more than one cache.

===MOSI===

The MOSI protocol is identical to the MSI protocol except that it adds an owned state. The owned state means that the processor "owns" the variable and will provide the current value to other caches when requested (or at least it will decide if it will provide it when asked). This is useful because another cache will not have to read the value from main memory and will receive it from the owning cache much, much, faster.

===MOESI===
[[Image:MOESI.jpg|thumbnail|right|350px|MSI State Diagram]]
The MOESI protocol is a combination of the MESI and MOSI protocols.

=Memory Consistency Problem=

Memory consistency deals with the ordering of memory operations (load and store) to different memory locations. In a single processor system, code will execute correctly if the compiler preserves the order of the access to synchronization variables and other dependent variables. But in shared memory model with multiple processors, two threads could access a shared data element, such as a synchronization variable, the output of the threads would change based on which thread, accesses the shared data element earlier. If this were to occur, then the program output may not be the value expected. Maintaining program order is very important for memory consistency but it comes with performance degradation.

== Memory Consistency Models ==

The memory consistency model of a shared-memory multiprocessor is a formal speciﬁcation of how the memory system appears to the programmer. It eliminates the gap between the behavior expected by the programmer and the actual behavior supported by a system. Effectively, the consistency model places restrictions on the values that can be returned by a read, in a shared-memory program execution.

In a single processor system, in order to maintain memory consistency, it needs to ensure that the compiler preserves the program order when accessing synchronization variables. But in a multiprocessor system, it is required to ensure that accesses of one processor appear to execute in program order to all other processors, at least partially.

=== Memory semantics in Uniprocessor systems ===

Uniprocessor languages use simple sequential semantics for memory operations, which allow the programmer to assume that all memory operations will occur one at a time in the sequential order speciﬁed by the program. Thus, one can expect for the value in a particular location be the same for a read as it was for the last write because of the sequential program execution. It is sufﬁcient to only maintain uniprocessor data and control dependences. The compiler and hardware can freely reorder operations to different locations if the uniprocessor data and control dependences are respected. This enables compiler optimizations such as register allocation, code motion, and loop transformations, and hardware optimizations, such as pipelining, multiple issue, write buffer bypassing and forwarding, and lockup-free caches, all of which lead to overlapping and reordering of memory operations.[[#4foot|[4]]]

=== Memory semantics in multiprocessor systems ===

Programmer's implicit expectations are:

* memory accesses in a processor takes place according to the program order.
* Each memory access is performed atomically.

A strong consistency model attempting uniprocessor-like consistency could cause global bottleneck, costing performance. Thus, '''''weak''''' consistency models are deployed to improve performance. The advanatges of such models are:

* They support out-of-order execution within individual CPUs
* Relaxes latency issues with near-simultaneous accesses by different CPUs

The following are the various consistency models and it is the programmer who must take into account the memory consistency model to create correct software:
{| class="wikitable sortable" border="1"
|+ Sortable Table of Memory Consistency Models
|-
! scope="col" class="unsortable" | Type of Constancy Modle
! scope="col" | Strictness rank, 1-10(lowest-highest)
|-
| [http://en.wikipedia.org/wiki/Linearizability linearizability (also known as strict or atomic consistency)]|| 3
|-
| [http://en.wikipedia.org/wiki/Sequential_consistency sequential consistency]|| 3
|-
| [http://en.wikipedia.org/wiki/Causal_consistency causal consistency]|| 1
|-
|[http://en.wikipedia.org/wiki/Release_consistency release consistency]|| 3
|-
| [http://en.wikipedia.org/wiki/Eventual_consistency eventual consistency ]|| 2
|-
|[http://en.wikipedia.org/wiki/Delta_consistency delta consistency]|| 1
|-
| [http://en.wikipedia.org/wiki/PRAM_consistency PRAM consistency (also known as FIFO consistency)]|| 1
|-
| [http://en.wikipedia.org/wiki/Weak_consistency weak consistency]|| 1
|-
|[http://en.wikipedia.org/wiki/Vector-field_consistency vector-field consistency]|| 3
|-
| fork consistency|| 2
|-
|[http://en.wikipedia.org/wiki/Serializability serializability]|| 3
|-
|one-copy serializability|| 3
|-
|entry consistency|| 3
|}

== Memory Coherence and Shared Virtual Memory ==

[[Image:Centralized.png|thumbnail|right|350px|'''Figure 2''' Example of a centralized system. Here, one processor monitors all pages in the cache.]]
[[Image:Distributed.png|thumbnail|right|350px|'''Figure 3''' Example of a distributed system. Here, each processor monitors a different subset of pages in the cache.]]
The memory coherence problem in a shared virtual memory system and in multicache systems are different. In a multicache multiprocessor, there are processors sharing a physical memory through their private caches. The relatively small size of a cache and the fast bus connection to the shared memory, enables using a sophisticated coherence protocol for the multicache hardware such that the time delay of conflicting writes to a memory location is small.

In contrast, in a shared virtual memory on a loosely coupled multiprocessor which has no physically shared memory, and having a nontrivial communication cost between processors, conflicts are not likely to be solved with negligible delay, and they resemble much more a “page
fault” in a traditional virtual memory system. Thus, there are two design choices that greatly influence the implementation of a shared virtual memory: the granularity of the memory units (i.e., the “page size”) and the strategy for maintaining coherence.

Memory coherence strategies are classified based on how they deal with '''''page synchronization''''' and '''''page ownership'''''. The algorithms for memory coherence depend on the page fault handlers, their servers and the data structures used. So ''page table'' becomes an important part of these protocols.

Two classes of algorithms for solving the memory coherence problem are centralized manager (figure 2) and distributed manager (figure 3). In the centralized system, one processor is designated the "monitor". This processor contains a list of information for each page in the cache. The list includes the owner of the page (which processor accessed it last) and all the processors that have copies of the page. When a page is invalidated, then invalidation message is only sent to processors that have copies of the page. This differs from a bus-based system, as the invalidation message is broadcasted to all processors. A drawback to the centralized manager is that there is a bottleneck at the monitor processor. This bottleneck is alleviated by using a distributed manager.

The distributed manager is similar to a centralized manager, but instead of one processor monitoring all pages, a subset of the pages is given to each processor. So, processor 0 would only monitor pages 1 through i, and processor 1 would only monitor pages i+1 through n.[[#5foot|[5]]]

= Synchronization =

Two related concepts dealing with synchronization are '''process synchronization''' and '''data synchronization'''. Process synchronization is concerned with different processes committing to a certain sequence of actions. Data synchronization deals with maintaining data integrity across various copies of a dataset. Process synchronization primitives can be used to implement data synchronization.

[http://en.wikipedia.org/wiki/Mutual_exclusion Mutual exclusion] is the main requirement to be fulfilled in order to synchronize processes, and is needed in both single-processor and multiprocessor systems. There are various approaches to provide mutual exclusion in a system:

* Disabling interrupts
* Locks
* Mutex
* Semaphores
* Barriers
* Test and Set

The next section discusses if there is an alternative to implement mutual exclusion without requiring any hardware support. Peterson's algorithm is one such software solution for guaranteeing mutual exclusion.

== Hardware support ==

Exclusive locking assumes the worst and proceeds only after acquiring all locks such that no other thread can interfere. This is a ''pessimistic'' approach. In contrast, the ''optimistic'' approach proceeds with an update, hoping that it can be completed without any interference. This requires ''collision detection'' during the update. The optimistic approach is thus, more efficient in fine-grained operations.

Special instructions are provided by processors designed for multiprocessor operations in order to manage concurrent access to shared variables. Atomic instructions like [http://en.wikipedia.org/wiki/Test_and_Test-and-set test-and-set], [http://en.wikipedia.org/wiki/Fetch-and-add fetch-and-increment] and [http://en.wikipedia.org/wiki/Swap_%28computer_science%29 swap] were sufficient for early processors to implement mutexes for concurrent objects. Today, every modern processor relies on some form of read-modify-write atomic instruction such as [http://en.wikipedia.org/wiki/Compare-and-swap compare-and-swap], [http://en.wikipedia.org/wiki/Load-link/store-conditional LL/SC] etc. for the same.

=Resources=

[[#1body|1.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_jp#Memory_Consistency_Problem 
[[#2body|2.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_ss#Cache_Coherence 
[[#3body|3.]] http://www.windowsnetworking.com/articles_tutorials/Cache-Coherency.html 
[[#4body|4.]]http://web.sfc.keio.ac.jp/~rdv/keio/sfc/teaching/architecture/architecture-2007/lec08.html 
[[#5body|5.]]http://dl.acm.org/citation.cfm?id=75105

CSC 456 Spring 2012/ch7 MN

2012-03-19T18:01:17Z

Jmmohund2: /* MESI */

==Introduction==
Though the migration from [http://en.wikipedia.org/wiki/Uniprocessor_system uniprocessor system] to [http://en.wikipedia.org/wiki/Multiprocessing multiprocessing] systems is not new, the world of parallel computers is undergoing a continuous change. Parallel computers, which started as high-end super-computing systems for carrying out huge calculations, are now ubiquitous and are present in all mainstream architectures for servers, desktops, and embedded systems. In order to design parallel architectures to meet programmer's needs and expectations more closely, exciting and challenging changes exist. The three main areas which are being considered by scientists today are: [http://en.wikipedia.org/wiki/Cache_coherence cache coherence], memory consistency and [http://en.wikipedia.org/wiki/Synchronization_%28computer_science%29 synchronization].

=Cache Coherence Problem=

[[Image:sharedmem.jpg|thumbnail|right|600px|Shared Memory system with dedicated Cache for each processor[[#4foot|[4]]]]]

In a system with single processor (single core), maintaining cache coherence is simple and easy but in a multiprocessor system, it is much more complicated. Data can be present in any processor's cache and the protocol needs to ensure that the data is same in all caches. If it cannot ensure that all the caches are same, then it needs to flag a cache line indicating that it is not updated.

In the figure shown here, this is a 4 processor shared memory system where each processor has its own cache. Suppose processor P1 reads memory location M1 and stores it in its local cache. Then, if P2 reads same location memory location then M1 gets stored in P2’s cache. Now, if P1 changes value of M1, two copies of same data, residing in different caches will become different. When P2 operates on M1, it uses the stale value of M1 that was stored in its cache. It is the responsibility of cache coherence protocol to prevent this. Hardware support is needed to provide a coherent view of data in multiple caches. This is known as write propagation requirement.

One may think that cache write policy can provide cache coherence, but this is incorrect. Cache write policy only controls how a change in value of cache is propagated to lower level cache or main memory. It is not responsible for propagating changes to other caches.

==Cache Coherence Protocols==

The two basic methods to utilize the inter-core bus to notify other cores when a core changes something in its cache are '''update''' and '''invalidate'''. In the update method, if variable 'x' is modified by core 1, core 1 has to send the updated value of 'x' onto the inter-core bus. Each cache listens to the inter-core bus and if a cache sees a variable on the bus which it has a copy of, it will read the updated value. This ensures that all caches have the most up-to-date value of the variable.[[#3foot|[3]]]

In case of invalidation, an invalidation message is sent onto the inter-core bus when a variable is changed. The other caches will read this invalidation signal and if its core attempts to access that variable, it will result in a cache miss and the variable will be read from main memory.

The update method results in significant amount of traffic on the inter-core bus as the update signal is sent onto the bus every time the variable is updated. The invalidation method only requires that an invalidation signal be sent the first time a variable is altered; this is why the invalidation method is the preferred method.

In order to improve cache coherence performance over the years, several protocols have been proposed.

===MSI===
[[Image:MSI.jpg|thumbnail|right|600px|MSI State Diagram]]
MSI stands for Modified, Shared, and Invalid, based on the three states that a line of cache can be in. The modified state means that a variable in the cache has been modified and therefore has a different value than that found in main memory; the cache is responsible for writing the variable back to main memory. The shared state means that the variable exists in at least one cache and is not modified; the cache can evict the variable without writing it back to the main memory. The invalid state means that the value of the variable has been modified by another cache and this value is invalid; the cache must read a new value from main memory (or another cache).

A drawback to this protocol occurs when a single processor wants to read blocks and then write to them without another processor sharing that block. After reading the block, a bus transaction places the block into a shared state. The write then occurs and another bus transaction is sent to invalidate the shared copy. This second transaction is useless as no other processors are sharing the block, but the MSI protocol has no way to specify this.

===MESI===
[[Image:MESI.jpg|thumbnail|right|600px|MSI State Diagram]]
MESI stands for Modified, Exclusive, Shared, and Invalid. The modified and invalid states are the same for this protocol as they are for the MSI protocol. This protocol introduces a new state; the exclusive state. The exclusive state means that the variable is in only this cache and the value of it matches the value within the main memory. This now means that the shared state indicates that the variable is contained in more than one cache.

===MOSI===

The MOSI protocol is identical to the MSI protocol except that it adds an owned state. The owned state means that the processor "owns" the variable and will provide the current value to other caches when requested (or at least it will decide if it will provide it when asked). This is useful because another cache will not have to read the value from main memory and will receive it from the owning cache much, much, faster.

===MOESI===

The MOESI protocol is a combination of the MESI and MOSI protocols.

=Memory Consistency Problem=

Memory consistency deals with the ordering of memory operations (load and store) to different memory locations. In a single processor system, code will execute correctly if the compiler preserves the order of the access to synchronization variables and other dependent variables. But in shared memory model with multiple processors, two threads could access a shared data element, such as a synchronization variable, the output of the threads would change based on which thread, accesses the shared data element earlier. If this were to occur, then the program output may not be the value expected. Maintaining program order is very important for memory consistency but it comes with performance degradation.

== Memory Consistency Models ==

The memory consistency model of a shared-memory multiprocessor is a formal speciﬁcation of how the memory system appears to the programmer. It eliminates the gap between the behavior expected by the programmer and the actual behavior supported by a system. Effectively, the consistency model places restrictions on the values that can be returned by a read, in a shared-memory program execution.

In a single processor system, in order to maintain memory consistency, it needs to ensure that the compiler preserves the program order when accessing synchronization variables. But in a multiprocessor system, it is required to ensure that accesses of one processor appear to execute in program order to all other processors, at least partially.

=== Memory semantics in Uniprocessor systems ===

Uniprocessor languages use simple sequential semantics for memory operations, which allow the programmer to assume that all memory operations will occur one at a time in the sequential order speciﬁed by the program. Thus, one can expect for the value in a particular location be the same for a read as it was for the last write because of the sequential program execution. It is sufﬁcient to only maintain uniprocessor data and control dependences. The compiler and hardware can freely reorder operations to different locations if the uniprocessor data and control dependences are respected. This enables compiler optimizations such as register allocation, code motion, and loop transformations, and hardware optimizations, such as pipelining, multiple issue, write buffer bypassing and forwarding, and lockup-free caches, all of which lead to overlapping and reordering of memory operations.[[#4foot|[4]]]

=== Memory semantics in multiprocessor systems ===

Programmer's implicit expectations are:

* memory accesses in a processor takes place according to the program order.
* Each memory access is performed atomically.

A strong consistency model attempting uniprocessor-like consistency could cause global bottleneck, costing performance. Thus, '''''weak''''' consistency models are deployed to improve performance. The advanatges of such models are:

* They support out-of-order execution within individual CPUs
* Relaxes latency issues with near-simultaneous accesses by different CPUs

The following are the various consistency models and it is the programmer who must take into account the memory consistency model to create correct software:
{| class="wikitable sortable" border="1"
|+ Sortable Table of Memory Consistency Models
|-
! scope="col" class="unsortable" | Type of Constancy Modle
! scope="col" | Strictness rank, 1-10(lowest-highest)
|-
| [http://en.wikipedia.org/wiki/Linearizability linearizability (also known as strict or atomic consistency)]|| 3
|-
| [http://en.wikipedia.org/wiki/Sequential_consistency sequential consistency]|| 3
|-
| [http://en.wikipedia.org/wiki/Causal_consistency causal consistency]|| 1
|-
|[http://en.wikipedia.org/wiki/Release_consistency release consistency]|| 3
|-
| [http://en.wikipedia.org/wiki/Eventual_consistency eventual consistency ]|| 2
|-
|[http://en.wikipedia.org/wiki/Delta_consistency delta consistency]|| 1
|-
| [http://en.wikipedia.org/wiki/PRAM_consistency PRAM consistency (also known as FIFO consistency)]|| 1
|-
| [http://en.wikipedia.org/wiki/Weak_consistency weak consistency]|| 1
|-
|[http://en.wikipedia.org/wiki/Vector-field_consistency vector-field consistency]|| 3
|-
| fork consistency|| 2
|-
|[http://en.wikipedia.org/wiki/Serializability serializability]|| 3
|-
|one-copy serializability|| 3
|-
|entry consistency|| 3
|}

== Memory Coherence and Shared Virtual Memory ==

[[Image:Centralized.png|thumbnail|right|350px|'''Figure 2''' Example of a centralized system. Here, one processor monitors all pages in the cache.]]
[[Image:Distributed.png|thumbnail|right|350px|'''Figure 3''' Example of a distributed system. Here, each processor monitors a different subset of pages in the cache.]]
The memory coherence problem in a shared virtual memory system and in multicache systems are different. In a multicache multiprocessor, there are processors sharing a physical memory through their private caches. The relatively small size of a cache and the fast bus connection to the shared memory, enables using a sophisticated coherence protocol for the multicache hardware such that the time delay of conflicting writes to a memory location is small.

In contrast, in a shared virtual memory on a loosely coupled multiprocessor which has no physically shared memory, and having a nontrivial communication cost between processors, conflicts are not likely to be solved with negligible delay, and they resemble much more a “page
fault” in a traditional virtual memory system. Thus, there are two design choices that greatly influence the implementation of a shared virtual memory: the granularity of the memory units (i.e., the “page size”) and the strategy for maintaining coherence.

Memory coherence strategies are classified based on how they deal with '''''page synchronization''''' and '''''page ownership'''''. The algorithms for memory coherence depend on the page fault handlers, their servers and the data structures used. So ''page table'' becomes an important part of these protocols.

Two classes of algorithms for solving the memory coherence problem are centralized manager (figure 2) and distributed manager (figure 3). In the centralized system, one processor is designated the "monitor". This processor contains a list of information for each page in the cache. The list includes the owner of the page (which processor accessed it last) and all the processors that have copies of the page. When a page is invalidated, then invalidation message is only sent to processors that have copies of the page. This differs from a bus-based system, as the invalidation message is broadcasted to all processors. A drawback to the centralized manager is that there is a bottleneck at the monitor processor. This bottleneck is alleviated by using a distributed manager.

The distributed manager is similar to a centralized manager, but instead of one processor monitoring all pages, a subset of the pages is given to each processor. So, processor 0 would only monitor pages 1 through i, and processor 1 would only monitor pages i+1 through n.[[#5foot|[5]]]

= Synchronization =

Two related concepts dealing with synchronization are '''process synchronization''' and '''data synchronization'''. Process synchronization is concerned with different processes committing to a certain sequence of actions. Data synchronization deals with maintaining data integrity across various copies of a dataset. Process synchronization primitives can be used to implement data synchronization.

[http://en.wikipedia.org/wiki/Mutual_exclusion Mutual exclusion] is the main requirement to be fulfilled in order to synchronize processes, and is needed in both single-processor and multiprocessor systems. There are various approaches to provide mutual exclusion in a system:

* Disabling interrupts
* Locks
* Mutex
* Semaphores
* Barriers
* Test and Set

The next section discusses if there is an alternative to implement mutual exclusion without requiring any hardware support. Peterson's algorithm is one such software solution for guaranteeing mutual exclusion.

== Hardware support ==

Exclusive locking assumes the worst and proceeds only after acquiring all locks such that no other thread can interfere. This is a ''pessimistic'' approach. In contrast, the ''optimistic'' approach proceeds with an update, hoping that it can be completed without any interference. This requires ''collision detection'' during the update. The optimistic approach is thus, more efficient in fine-grained operations.

Special instructions are provided by processors designed for multiprocessor operations in order to manage concurrent access to shared variables. Atomic instructions like [http://en.wikipedia.org/wiki/Test_and_Test-and-set test-and-set], [http://en.wikipedia.org/wiki/Fetch-and-add fetch-and-increment] and [http://en.wikipedia.org/wiki/Swap_%28computer_science%29 swap] were sufficient for early processors to implement mutexes for concurrent objects. Today, every modern processor relies on some form of read-modify-write atomic instruction such as [http://en.wikipedia.org/wiki/Compare-and-swap compare-and-swap], [http://en.wikipedia.org/wiki/Load-link/store-conditional LL/SC] etc. for the same.

=Resources=

[[#1body|1.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_jp#Memory_Consistency_Problem 
[[#2body|2.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_ss#Cache_Coherence 
[[#3body|3.]] http://www.windowsnetworking.com/articles_tutorials/Cache-Coherency.html 
[[#4body|4.]]http://web.sfc.keio.ac.jp/~rdv/keio/sfc/teaching/architecture/architecture-2007/lec08.html 
[[#5body|5.]]http://dl.acm.org/citation.cfm?id=75105

CSC 456 Spring 2012/ch7 MN

2012-03-19T18:00:47Z

Jmmohund2: /* MSI */

==Introduction==
Though the migration from [http://en.wikipedia.org/wiki/Uniprocessor_system uniprocessor system] to [http://en.wikipedia.org/wiki/Multiprocessing multiprocessing] systems is not new, the world of parallel computers is undergoing a continuous change. Parallel computers, which started as high-end super-computing systems for carrying out huge calculations, are now ubiquitous and are present in all mainstream architectures for servers, desktops, and embedded systems. In order to design parallel architectures to meet programmer's needs and expectations more closely, exciting and challenging changes exist. The three main areas which are being considered by scientists today are: [http://en.wikipedia.org/wiki/Cache_coherence cache coherence], memory consistency and [http://en.wikipedia.org/wiki/Synchronization_%28computer_science%29 synchronization].

=Cache Coherence Problem=

[[Image:sharedmem.jpg|thumbnail|right|600px|Shared Memory system with dedicated Cache for each processor[[#4foot|[4]]]]]

In a system with single processor (single core), maintaining cache coherence is simple and easy but in a multiprocessor system, it is much more complicated. Data can be present in any processor's cache and the protocol needs to ensure that the data is same in all caches. If it cannot ensure that all the caches are same, then it needs to flag a cache line indicating that it is not updated.

In the figure shown here, this is a 4 processor shared memory system where each processor has its own cache. Suppose processor P1 reads memory location M1 and stores it in its local cache. Then, if P2 reads same location memory location then M1 gets stored in P2’s cache. Now, if P1 changes value of M1, two copies of same data, residing in different caches will become different. When P2 operates on M1, it uses the stale value of M1 that was stored in its cache. It is the responsibility of cache coherence protocol to prevent this. Hardware support is needed to provide a coherent view of data in multiple caches. This is known as write propagation requirement.

One may think that cache write policy can provide cache coherence, but this is incorrect. Cache write policy only controls how a change in value of cache is propagated to lower level cache or main memory. It is not responsible for propagating changes to other caches.

==Cache Coherence Protocols==

The two basic methods to utilize the inter-core bus to notify other cores when a core changes something in its cache are '''update''' and '''invalidate'''. In the update method, if variable 'x' is modified by core 1, core 1 has to send the updated value of 'x' onto the inter-core bus. Each cache listens to the inter-core bus and if a cache sees a variable on the bus which it has a copy of, it will read the updated value. This ensures that all caches have the most up-to-date value of the variable.[[#3foot|[3]]]

In case of invalidation, an invalidation message is sent onto the inter-core bus when a variable is changed. The other caches will read this invalidation signal and if its core attempts to access that variable, it will result in a cache miss and the variable will be read from main memory.

The update method results in significant amount of traffic on the inter-core bus as the update signal is sent onto the bus every time the variable is updated. The invalidation method only requires that an invalidation signal be sent the first time a variable is altered; this is why the invalidation method is the preferred method.

In order to improve cache coherence performance over the years, several protocols have been proposed.

===MSI===
[[Image:MSI.jpg|thumbnail|right|600px|MSI State Diagram]]
MSI stands for Modified, Shared, and Invalid, based on the three states that a line of cache can be in. The modified state means that a variable in the cache has been modified and therefore has a different value than that found in main memory; the cache is responsible for writing the variable back to main memory. The shared state means that the variable exists in at least one cache and is not modified; the cache can evict the variable without writing it back to the main memory. The invalid state means that the value of the variable has been modified by another cache and this value is invalid; the cache must read a new value from main memory (or another cache).

A drawback to this protocol occurs when a single processor wants to read blocks and then write to them without another processor sharing that block. After reading the block, a bus transaction places the block into a shared state. The write then occurs and another bus transaction is sent to invalidate the shared copy. This second transaction is useless as no other processors are sharing the block, but the MSI protocol has no way to specify this.

===MESI===

MESI stands for Modified, Exclusive, Shared, and Invalid. The modified and invalid states are the same for this protocol as they are for the MSI protocol. This protocol introduces a new state; the exclusive state. The Exclusive state means that the variable is in only this cache and the value of it matches the value within the main memory. This now means that the shared state indicates that the variable is contained in more than one cache.

===MOSI===

The MOSI protocol is identical to the MSI protocol except that it adds an owned state. The owned state means that the processor "owns" the variable and will provide the current value to other caches when requested (or at least it will decide if it will provide it when asked). This is useful because another cache will not have to read the value from main memory and will receive it from the owning cache much, much, faster.

===MOESI===

The MOESI protocol is a combination of the MESI and MOSI protocols.

=Memory Consistency Problem=

Memory consistency deals with the ordering of memory operations (load and store) to different memory locations. In a single processor system, code will execute correctly if the compiler preserves the order of the access to synchronization variables and other dependent variables. But in shared memory model with multiple processors, two threads could access a shared data element, such as a synchronization variable, the output of the threads would change based on which thread, accesses the shared data element earlier. If this were to occur, then the program output may not be the value expected. Maintaining program order is very important for memory consistency but it comes with performance degradation.

== Memory Consistency Models ==

The memory consistency model of a shared-memory multiprocessor is a formal speciﬁcation of how the memory system appears to the programmer. It eliminates the gap between the behavior expected by the programmer and the actual behavior supported by a system. Effectively, the consistency model places restrictions on the values that can be returned by a read, in a shared-memory program execution.

In a single processor system, in order to maintain memory consistency, it needs to ensure that the compiler preserves the program order when accessing synchronization variables. But in a multiprocessor system, it is required to ensure that accesses of one processor appear to execute in program order to all other processors, at least partially.

=== Memory semantics in Uniprocessor systems ===

Uniprocessor languages use simple sequential semantics for memory operations, which allow the programmer to assume that all memory operations will occur one at a time in the sequential order speciﬁed by the program. Thus, one can expect for the value in a particular location be the same for a read as it was for the last write because of the sequential program execution. It is sufﬁcient to only maintain uniprocessor data and control dependences. The compiler and hardware can freely reorder operations to different locations if the uniprocessor data and control dependences are respected. This enables compiler optimizations such as register allocation, code motion, and loop transformations, and hardware optimizations, such as pipelining, multiple issue, write buffer bypassing and forwarding, and lockup-free caches, all of which lead to overlapping and reordering of memory operations.[[#4foot|[4]]]

=== Memory semantics in multiprocessor systems ===

Programmer's implicit expectations are:

* memory accesses in a processor takes place according to the program order.
* Each memory access is performed atomically.

A strong consistency model attempting uniprocessor-like consistency could cause global bottleneck, costing performance. Thus, '''''weak''''' consistency models are deployed to improve performance. The advanatges of such models are:

* They support out-of-order execution within individual CPUs
* Relaxes latency issues with near-simultaneous accesses by different CPUs

The following are the various consistency models and it is the programmer who must take into account the memory consistency model to create correct software:
{| class="wikitable sortable" border="1"
|+ Sortable Table of Memory Consistency Models
|-
! scope="col" class="unsortable" | Type of Constancy Modle
! scope="col" | Strictness rank, 1-10(lowest-highest)
|-
| [http://en.wikipedia.org/wiki/Linearizability linearizability (also known as strict or atomic consistency)]|| 3
|-
| [http://en.wikipedia.org/wiki/Sequential_consistency sequential consistency]|| 3
|-
| [http://en.wikipedia.org/wiki/Causal_consistency causal consistency]|| 1
|-
|[http://en.wikipedia.org/wiki/Release_consistency release consistency]|| 3
|-
| [http://en.wikipedia.org/wiki/Eventual_consistency eventual consistency ]|| 2
|-
|[http://en.wikipedia.org/wiki/Delta_consistency delta consistency]|| 1
|-
| [http://en.wikipedia.org/wiki/PRAM_consistency PRAM consistency (also known as FIFO consistency)]|| 1
|-
| [http://en.wikipedia.org/wiki/Weak_consistency weak consistency]|| 1
|-
|[http://en.wikipedia.org/wiki/Vector-field_consistency vector-field consistency]|| 3
|-
| fork consistency|| 2
|-
|[http://en.wikipedia.org/wiki/Serializability serializability]|| 3
|-
|one-copy serializability|| 3
|-
|entry consistency|| 3
|}

== Memory Coherence and Shared Virtual Memory ==

[[Image:Centralized.png|thumbnail|right|350px|'''Figure 2''' Example of a centralized system. Here, one processor monitors all pages in the cache.]]
[[Image:Distributed.png|thumbnail|right|350px|'''Figure 3''' Example of a distributed system. Here, each processor monitors a different subset of pages in the cache.]]
The memory coherence problem in a shared virtual memory system and in multicache systems are different. In a multicache multiprocessor, there are processors sharing a physical memory through their private caches. The relatively small size of a cache and the fast bus connection to the shared memory, enables using a sophisticated coherence protocol for the multicache hardware such that the time delay of conflicting writes to a memory location is small.

In contrast, in a shared virtual memory on a loosely coupled multiprocessor which has no physically shared memory, and having a nontrivial communication cost between processors, conflicts are not likely to be solved with negligible delay, and they resemble much more a “page
fault” in a traditional virtual memory system. Thus, there are two design choices that greatly influence the implementation of a shared virtual memory: the granularity of the memory units (i.e., the “page size”) and the strategy for maintaining coherence.

Memory coherence strategies are classified based on how they deal with '''''page synchronization''''' and '''''page ownership'''''. The algorithms for memory coherence depend on the page fault handlers, their servers and the data structures used. So ''page table'' becomes an important part of these protocols.

Two classes of algorithms for solving the memory coherence problem are centralized manager (figure 2) and distributed manager (figure 3). In the centralized system, one processor is designated the "monitor". This processor contains a list of information for each page in the cache. The list includes the owner of the page (which processor accessed it last) and all the processors that have copies of the page. When a page is invalidated, then invalidation message is only sent to processors that have copies of the page. This differs from a bus-based system, as the invalidation message is broadcasted to all processors. A drawback to the centralized manager is that there is a bottleneck at the monitor processor. This bottleneck is alleviated by using a distributed manager.

The distributed manager is similar to a centralized manager, but instead of one processor monitoring all pages, a subset of the pages is given to each processor. So, processor 0 would only monitor pages 1 through i, and processor 1 would only monitor pages i+1 through n.[[#5foot|[5]]]

= Synchronization =

Two related concepts dealing with synchronization are '''process synchronization''' and '''data synchronization'''. Process synchronization is concerned with different processes committing to a certain sequence of actions. Data synchronization deals with maintaining data integrity across various copies of a dataset. Process synchronization primitives can be used to implement data synchronization.

[http://en.wikipedia.org/wiki/Mutual_exclusion Mutual exclusion] is the main requirement to be fulfilled in order to synchronize processes, and is needed in both single-processor and multiprocessor systems. There are various approaches to provide mutual exclusion in a system:

* Disabling interrupts
* Locks
* Mutex
* Semaphores
* Barriers
* Test and Set

The next section discusses if there is an alternative to implement mutual exclusion without requiring any hardware support. Peterson's algorithm is one such software solution for guaranteeing mutual exclusion.

== Hardware support ==

Exclusive locking assumes the worst and proceeds only after acquiring all locks such that no other thread can interfere. This is a ''pessimistic'' approach. In contrast, the ''optimistic'' approach proceeds with an update, hoping that it can be completed without any interference. This requires ''collision detection'' during the update. The optimistic approach is thus, more efficient in fine-grained operations.

Special instructions are provided by processors designed for multiprocessor operations in order to manage concurrent access to shared variables. Atomic instructions like [http://en.wikipedia.org/wiki/Test_and_Test-and-set test-and-set], [http://en.wikipedia.org/wiki/Fetch-and-add fetch-and-increment] and [http://en.wikipedia.org/wiki/Swap_%28computer_science%29 swap] were sufficient for early processors to implement mutexes for concurrent objects. Today, every modern processor relies on some form of read-modify-write atomic instruction such as [http://en.wikipedia.org/wiki/Compare-and-swap compare-and-swap], [http://en.wikipedia.org/wiki/Load-link/store-conditional LL/SC] etc. for the same.

=Resources=

[[#1body|1.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_jp#Memory_Consistency_Problem 
[[#2body|2.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_ss#Cache_Coherence 
[[#3body|3.]] http://www.windowsnetworking.com/articles_tutorials/Cache-Coherency.html 
[[#4body|4.]]http://web.sfc.keio.ac.jp/~rdv/keio/sfc/teaching/architecture/architecture-2007/lec08.html 
[[#5body|5.]]http://dl.acm.org/citation.cfm?id=75105

CSC 456 Spring 2012/ch7 MN

2012-03-19T18:00:12Z

Jmmohund2: /* MSI */

==Introduction==
Though the migration from [http://en.wikipedia.org/wiki/Uniprocessor_system uniprocessor system] to [http://en.wikipedia.org/wiki/Multiprocessing multiprocessing] systems is not new, the world of parallel computers is undergoing a continuous change. Parallel computers, which started as high-end super-computing systems for carrying out huge calculations, are now ubiquitous and are present in all mainstream architectures for servers, desktops, and embedded systems. In order to design parallel architectures to meet programmer's needs and expectations more closely, exciting and challenging changes exist. The three main areas which are being considered by scientists today are: [http://en.wikipedia.org/wiki/Cache_coherence cache coherence], memory consistency and [http://en.wikipedia.org/wiki/Synchronization_%28computer_science%29 synchronization].

=Cache Coherence Problem=

[[Image:sharedmem.jpg|thumbnail|right|600px|Shared Memory system with dedicated Cache for each processor[[#4foot|[4]]]]]

In a system with single processor (single core), maintaining cache coherence is simple and easy but in a multiprocessor system, it is much more complicated. Data can be present in any processor's cache and the protocol needs to ensure that the data is same in all caches. If it cannot ensure that all the caches are same, then it needs to flag a cache line indicating that it is not updated.

In the figure shown here, this is a 4 processor shared memory system where each processor has its own cache. Suppose processor P1 reads memory location M1 and stores it in its local cache. Then, if P2 reads same location memory location then M1 gets stored in P2’s cache. Now, if P1 changes value of M1, two copies of same data, residing in different caches will become different. When P2 operates on M1, it uses the stale value of M1 that was stored in its cache. It is the responsibility of cache coherence protocol to prevent this. Hardware support is needed to provide a coherent view of data in multiple caches. This is known as write propagation requirement.

One may think that cache write policy can provide cache coherence, but this is incorrect. Cache write policy only controls how a change in value of cache is propagated to lower level cache or main memory. It is not responsible for propagating changes to other caches.

==Cache Coherence Protocols==

The two basic methods to utilize the inter-core bus to notify other cores when a core changes something in its cache are '''update''' and '''invalidate'''. In the update method, if variable 'x' is modified by core 1, core 1 has to send the updated value of 'x' onto the inter-core bus. Each cache listens to the inter-core bus and if a cache sees a variable on the bus which it has a copy of, it will read the updated value. This ensures that all caches have the most up-to-date value of the variable.[[#3foot|[3]]]

In case of invalidation, an invalidation message is sent onto the inter-core bus when a variable is changed. The other caches will read this invalidation signal and if its core attempts to access that variable, it will result in a cache miss and the variable will be read from main memory.

The update method results in significant amount of traffic on the inter-core bus as the update signal is sent onto the bus every time the variable is updated. The invalidation method only requires that an invalidation signal be sent the first time a variable is altered; this is why the invalidation method is the preferred method.

In order to improve cache coherence performance over the years, several protocols have been proposed.

===MSI===

MSI stands for Modified, Shared, and Invalid, based on the three states that a line of cache can be in. The modified state means that a variable in the cache has been modified and therefore has a different value than that found in main memory; the cache is responsible for writing the variable back to main memory. The shared state means that the variable exists in at least one cache and is not modified; the cache can evict the variable without writing it back to the main memory. The invalid state means that the value of the variable has been modified by another cache and this value is invalid; the cache must read a new value from main memory (or another cache).[[Image:MSI.jpg|thumbnail|right|600px|MSI State Diagram]]

A drawback to this protocol occurs when a single processor wants to read blocks and then write to them without another processor sharing that block. After reading the block, a bus transaction places the block into a shared state. The write then occurs and another bus transaction is sent to invalidate the shared copy. This second transaction is useless as no other processors are sharing the block, but the MSI protocol has no way to specify this.
[[Image:MSI.jpg|thumbnail|right|600px|MSI State Diagram]]

===MESI===

MESI stands for Modified, Exclusive, Shared, and Invalid. The modified and invalid states are the same for this protocol as they are for the MSI protocol. This protocol introduces a new state; the exclusive state. The Exclusive state means that the variable is in only this cache and the value of it matches the value within the main memory. This now means that the shared state indicates that the variable is contained in more than one cache.

===MOSI===

The MOSI protocol is identical to the MSI protocol except that it adds an owned state. The owned state means that the processor "owns" the variable and will provide the current value to other caches when requested (or at least it will decide if it will provide it when asked). This is useful because another cache will not have to read the value from main memory and will receive it from the owning cache much, much, faster.

===MOESI===

The MOESI protocol is a combination of the MESI and MOSI protocols.

=Memory Consistency Problem=

Memory consistency deals with the ordering of memory operations (load and store) to different memory locations. In a single processor system, code will execute correctly if the compiler preserves the order of the access to synchronization variables and other dependent variables. But in shared memory model with multiple processors, two threads could access a shared data element, such as a synchronization variable, the output of the threads would change based on which thread, accesses the shared data element earlier. If this were to occur, then the program output may not be the value expected. Maintaining program order is very important for memory consistency but it comes with performance degradation.

== Memory Consistency Models ==

The memory consistency model of a shared-memory multiprocessor is a formal speciﬁcation of how the memory system appears to the programmer. It eliminates the gap between the behavior expected by the programmer and the actual behavior supported by a system. Effectively, the consistency model places restrictions on the values that can be returned by a read, in a shared-memory program execution.

In a single processor system, in order to maintain memory consistency, it needs to ensure that the compiler preserves the program order when accessing synchronization variables. But in a multiprocessor system, it is required to ensure that accesses of one processor appear to execute in program order to all other processors, at least partially.

=== Memory semantics in Uniprocessor systems ===

Uniprocessor languages use simple sequential semantics for memory operations, which allow the programmer to assume that all memory operations will occur one at a time in the sequential order speciﬁed by the program. Thus, one can expect for the value in a particular location be the same for a read as it was for the last write because of the sequential program execution. It is sufﬁcient to only maintain uniprocessor data and control dependences. The compiler and hardware can freely reorder operations to different locations if the uniprocessor data and control dependences are respected. This enables compiler optimizations such as register allocation, code motion, and loop transformations, and hardware optimizations, such as pipelining, multiple issue, write buffer bypassing and forwarding, and lockup-free caches, all of which lead to overlapping and reordering of memory operations.[[#4foot|[4]]]

=== Memory semantics in multiprocessor systems ===

Programmer's implicit expectations are:

* memory accesses in a processor takes place according to the program order.
* Each memory access is performed atomically.

A strong consistency model attempting uniprocessor-like consistency could cause global bottleneck, costing performance. Thus, '''''weak''''' consistency models are deployed to improve performance. The advanatges of such models are:

* They support out-of-order execution within individual CPUs
* Relaxes latency issues with near-simultaneous accesses by different CPUs

The following are the various consistency models and it is the programmer who must take into account the memory consistency model to create correct software:
{| class="wikitable sortable" border="1"
|+ Sortable Table of Memory Consistency Models
|-
! scope="col" class="unsortable" | Type of Constancy Modle
! scope="col" | Strictness rank, 1-10(lowest-highest)
|-
| [http://en.wikipedia.org/wiki/Linearizability linearizability (also known as strict or atomic consistency)]|| 3
|-
| [http://en.wikipedia.org/wiki/Sequential_consistency sequential consistency]|| 3
|-
| [http://en.wikipedia.org/wiki/Causal_consistency causal consistency]|| 1
|-
|[http://en.wikipedia.org/wiki/Release_consistency release consistency]|| 3
|-
| [http://en.wikipedia.org/wiki/Eventual_consistency eventual consistency ]|| 2
|-
|[http://en.wikipedia.org/wiki/Delta_consistency delta consistency]|| 1
|-
| [http://en.wikipedia.org/wiki/PRAM_consistency PRAM consistency (also known as FIFO consistency)]|| 1
|-
| [http://en.wikipedia.org/wiki/Weak_consistency weak consistency]|| 1
|-
|[http://en.wikipedia.org/wiki/Vector-field_consistency vector-field consistency]|| 3
|-
| fork consistency|| 2
|-
|[http://en.wikipedia.org/wiki/Serializability serializability]|| 3
|-
|one-copy serializability|| 3
|-
|entry consistency|| 3
|}

== Memory Coherence and Shared Virtual Memory ==

[[Image:Centralized.png|thumbnail|right|350px|'''Figure 2''' Example of a centralized system. Here, one processor monitors all pages in the cache.]]
[[Image:Distributed.png|thumbnail|right|350px|'''Figure 3''' Example of a distributed system. Here, each processor monitors a different subset of pages in the cache.]]
The memory coherence problem in a shared virtual memory system and in multicache systems are different. In a multicache multiprocessor, there are processors sharing a physical memory through their private caches. The relatively small size of a cache and the fast bus connection to the shared memory, enables using a sophisticated coherence protocol for the multicache hardware such that the time delay of conflicting writes to a memory location is small.

In contrast, in a shared virtual memory on a loosely coupled multiprocessor which has no physically shared memory, and having a nontrivial communication cost between processors, conflicts are not likely to be solved with negligible delay, and they resemble much more a “page
fault” in a traditional virtual memory system. Thus, there are two design choices that greatly influence the implementation of a shared virtual memory: the granularity of the memory units (i.e., the “page size”) and the strategy for maintaining coherence.

Memory coherence strategies are classified based on how they deal with '''''page synchronization''''' and '''''page ownership'''''. The algorithms for memory coherence depend on the page fault handlers, their servers and the data structures used. So ''page table'' becomes an important part of these protocols.

Two classes of algorithms for solving the memory coherence problem are centralized manager (figure 2) and distributed manager (figure 3). In the centralized system, one processor is designated the "monitor". This processor contains a list of information for each page in the cache. The list includes the owner of the page (which processor accessed it last) and all the processors that have copies of the page. When a page is invalidated, then invalidation message is only sent to processors that have copies of the page. This differs from a bus-based system, as the invalidation message is broadcasted to all processors. A drawback to the centralized manager is that there is a bottleneck at the monitor processor. This bottleneck is alleviated by using a distributed manager.

The distributed manager is similar to a centralized manager, but instead of one processor monitoring all pages, a subset of the pages is given to each processor. So, processor 0 would only monitor pages 1 through i, and processor 1 would only monitor pages i+1 through n.[[#5foot|[5]]]

= Synchronization =

Two related concepts dealing with synchronization are '''process synchronization''' and '''data synchronization'''. Process synchronization is concerned with different processes committing to a certain sequence of actions. Data synchronization deals with maintaining data integrity across various copies of a dataset. Process synchronization primitives can be used to implement data synchronization.

[http://en.wikipedia.org/wiki/Mutual_exclusion Mutual exclusion] is the main requirement to be fulfilled in order to synchronize processes, and is needed in both single-processor and multiprocessor systems. There are various approaches to provide mutual exclusion in a system:

* Disabling interrupts
* Locks
* Mutex
* Semaphores
* Barriers
* Test and Set

The next section discusses if there is an alternative to implement mutual exclusion without requiring any hardware support. Peterson's algorithm is one such software solution for guaranteeing mutual exclusion.

== Hardware support ==

Exclusive locking assumes the worst and proceeds only after acquiring all locks such that no other thread can interfere. This is a ''pessimistic'' approach. In contrast, the ''optimistic'' approach proceeds with an update, hoping that it can be completed without any interference. This requires ''collision detection'' during the update. The optimistic approach is thus, more efficient in fine-grained operations.

Special instructions are provided by processors designed for multiprocessor operations in order to manage concurrent access to shared variables. Atomic instructions like [http://en.wikipedia.org/wiki/Test_and_Test-and-set test-and-set], [http://en.wikipedia.org/wiki/Fetch-and-add fetch-and-increment] and [http://en.wikipedia.org/wiki/Swap_%28computer_science%29 swap] were sufficient for early processors to implement mutexes for concurrent objects. Today, every modern processor relies on some form of read-modify-write atomic instruction such as [http://en.wikipedia.org/wiki/Compare-and-swap compare-and-swap], [http://en.wikipedia.org/wiki/Load-link/store-conditional LL/SC] etc. for the same.

=Resources=

[[#1body|1.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_jp#Memory_Consistency_Problem 
[[#2body|2.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_ss#Cache_Coherence 
[[#3body|3.]] http://www.windowsnetworking.com/articles_tutorials/Cache-Coherency.html 
[[#4body|4.]]http://web.sfc.keio.ac.jp/~rdv/keio/sfc/teaching/architecture/architecture-2007/lec08.html 
[[#5body|5.]]http://dl.acm.org/citation.cfm?id=75105

CSC 456 Spring 2012/ch7 MN

2012-03-19T17:59:54Z

Jmmohund2: /* MSI */

==Introduction==
Though the migration from [http://en.wikipedia.org/wiki/Uniprocessor_system uniprocessor system] to [http://en.wikipedia.org/wiki/Multiprocessing multiprocessing] systems is not new, the world of parallel computers is undergoing a continuous change. Parallel computers, which started as high-end super-computing systems for carrying out huge calculations, are now ubiquitous and are present in all mainstream architectures for servers, desktops, and embedded systems. In order to design parallel architectures to meet programmer's needs and expectations more closely, exciting and challenging changes exist. The three main areas which are being considered by scientists today are: [http://en.wikipedia.org/wiki/Cache_coherence cache coherence], memory consistency and [http://en.wikipedia.org/wiki/Synchronization_%28computer_science%29 synchronization].

=Cache Coherence Problem=

[[Image:sharedmem.jpg|thumbnail|right|600px|Shared Memory system with dedicated Cache for each processor[[#4foot|[4]]]]]

In a system with single processor (single core), maintaining cache coherence is simple and easy but in a multiprocessor system, it is much more complicated. Data can be present in any processor's cache and the protocol needs to ensure that the data is same in all caches. If it cannot ensure that all the caches are same, then it needs to flag a cache line indicating that it is not updated.

In the figure shown here, this is a 4 processor shared memory system where each processor has its own cache. Suppose processor P1 reads memory location M1 and stores it in its local cache. Then, if P2 reads same location memory location then M1 gets stored in P2’s cache. Now, if P1 changes value of M1, two copies of same data, residing in different caches will become different. When P2 operates on M1, it uses the stale value of M1 that was stored in its cache. It is the responsibility of cache coherence protocol to prevent this. Hardware support is needed to provide a coherent view of data in multiple caches. This is known as write propagation requirement.

One may think that cache write policy can provide cache coherence, but this is incorrect. Cache write policy only controls how a change in value of cache is propagated to lower level cache or main memory. It is not responsible for propagating changes to other caches.

==Cache Coherence Protocols==

The two basic methods to utilize the inter-core bus to notify other cores when a core changes something in its cache are '''update''' and '''invalidate'''. In the update method, if variable 'x' is modified by core 1, core 1 has to send the updated value of 'x' onto the inter-core bus. Each cache listens to the inter-core bus and if a cache sees a variable on the bus which it has a copy of, it will read the updated value. This ensures that all caches have the most up-to-date value of the variable.[[#3foot|[3]]]

In case of invalidation, an invalidation message is sent onto the inter-core bus when a variable is changed. The other caches will read this invalidation signal and if its core attempts to access that variable, it will result in a cache miss and the variable will be read from main memory.

The update method results in significant amount of traffic on the inter-core bus as the update signal is sent onto the bus every time the variable is updated. The invalidation method only requires that an invalidation signal be sent the first time a variable is altered; this is why the invalidation method is the preferred method.

In order to improve cache coherence performance over the years, several protocols have been proposed.

===MSI===

MSI stands for Modified, Shared, and Invalid, based on the three states that a line of cache can be in. The modified state means that a variable in the cache has been modified and therefore has a different value than that found in main memory; the cache is responsible for writing the variable back to main memory. The shared state means that the variable exists in at least one cache and is not modified; the cache can evict the variable without writing it back to the main memory. The invalid state means that the value of the variable has been modified by another cache and this value is invalid; the cache must read a new value from main memory (or another cache).

A drawback to this protocol occurs when a single processor wants to read blocks and then write to them without another processor sharing that block. After reading the block, a bus transaction places the block into a shared state. The write then occurs and another bus transaction is sent to invalidate the shared copy. This second transaction is useless as no other processors are sharing the block, but the MSI protocol has no way to specify this.
[[Image:MSI.jpg|thumbnail|right|600px|MSI State Diagram]]

===MESI===

MESI stands for Modified, Exclusive, Shared, and Invalid. The modified and invalid states are the same for this protocol as they are for the MSI protocol. This protocol introduces a new state; the exclusive state. The Exclusive state means that the variable is in only this cache and the value of it matches the value within the main memory. This now means that the shared state indicates that the variable is contained in more than one cache.

===MOSI===

The MOSI protocol is identical to the MSI protocol except that it adds an owned state. The owned state means that the processor "owns" the variable and will provide the current value to other caches when requested (or at least it will decide if it will provide it when asked). This is useful because another cache will not have to read the value from main memory and will receive it from the owning cache much, much, faster.

===MOESI===

The MOESI protocol is a combination of the MESI and MOSI protocols.

=Memory Consistency Problem=

Memory consistency deals with the ordering of memory operations (load and store) to different memory locations. In a single processor system, code will execute correctly if the compiler preserves the order of the access to synchronization variables and other dependent variables. But in shared memory model with multiple processors, two threads could access a shared data element, such as a synchronization variable, the output of the threads would change based on which thread, accesses the shared data element earlier. If this were to occur, then the program output may not be the value expected. Maintaining program order is very important for memory consistency but it comes with performance degradation.

== Memory Consistency Models ==

The memory consistency model of a shared-memory multiprocessor is a formal speciﬁcation of how the memory system appears to the programmer. It eliminates the gap between the behavior expected by the programmer and the actual behavior supported by a system. Effectively, the consistency model places restrictions on the values that can be returned by a read, in a shared-memory program execution.

In a single processor system, in order to maintain memory consistency, it needs to ensure that the compiler preserves the program order when accessing synchronization variables. But in a multiprocessor system, it is required to ensure that accesses of one processor appear to execute in program order to all other processors, at least partially.

=== Memory semantics in Uniprocessor systems ===

Uniprocessor languages use simple sequential semantics for memory operations, which allow the programmer to assume that all memory operations will occur one at a time in the sequential order speciﬁed by the program. Thus, one can expect for the value in a particular location be the same for a read as it was for the last write because of the sequential program execution. It is sufﬁcient to only maintain uniprocessor data and control dependences. The compiler and hardware can freely reorder operations to different locations if the uniprocessor data and control dependences are respected. This enables compiler optimizations such as register allocation, code motion, and loop transformations, and hardware optimizations, such as pipelining, multiple issue, write buffer bypassing and forwarding, and lockup-free caches, all of which lead to overlapping and reordering of memory operations.[[#4foot|[4]]]

=== Memory semantics in multiprocessor systems ===

Programmer's implicit expectations are:

* memory accesses in a processor takes place according to the program order.
* Each memory access is performed atomically.

A strong consistency model attempting uniprocessor-like consistency could cause global bottleneck, costing performance. Thus, '''''weak''''' consistency models are deployed to improve performance. The advanatges of such models are:

* They support out-of-order execution within individual CPUs
* Relaxes latency issues with near-simultaneous accesses by different CPUs

The following are the various consistency models and it is the programmer who must take into account the memory consistency model to create correct software:
{| class="wikitable sortable" border="1"
|+ Sortable Table of Memory Consistency Models
|-
! scope="col" class="unsortable" | Type of Constancy Modle
! scope="col" | Strictness rank, 1-10(lowest-highest)
|-
| [http://en.wikipedia.org/wiki/Linearizability linearizability (also known as strict or atomic consistency)]|| 3
|-
| [http://en.wikipedia.org/wiki/Sequential_consistency sequential consistency]|| 3
|-
| [http://en.wikipedia.org/wiki/Causal_consistency causal consistency]|| 1
|-
|[http://en.wikipedia.org/wiki/Release_consistency release consistency]|| 3
|-
| [http://en.wikipedia.org/wiki/Eventual_consistency eventual consistency ]|| 2
|-
|[http://en.wikipedia.org/wiki/Delta_consistency delta consistency]|| 1
|-
| [http://en.wikipedia.org/wiki/PRAM_consistency PRAM consistency (also known as FIFO consistency)]|| 1
|-
| [http://en.wikipedia.org/wiki/Weak_consistency weak consistency]|| 1
|-
|[http://en.wikipedia.org/wiki/Vector-field_consistency vector-field consistency]|| 3
|-
| fork consistency|| 2
|-
|[http://en.wikipedia.org/wiki/Serializability serializability]|| 3
|-
|one-copy serializability|| 3
|-
|entry consistency|| 3
|}

== Memory Coherence and Shared Virtual Memory ==

[[Image:Centralized.png|thumbnail|right|350px|'''Figure 2''' Example of a centralized system. Here, one processor monitors all pages in the cache.]]
[[Image:Distributed.png|thumbnail|right|350px|'''Figure 3''' Example of a distributed system. Here, each processor monitors a different subset of pages in the cache.]]
The memory coherence problem in a shared virtual memory system and in multicache systems are different. In a multicache multiprocessor, there are processors sharing a physical memory through their private caches. The relatively small size of a cache and the fast bus connection to the shared memory, enables using a sophisticated coherence protocol for the multicache hardware such that the time delay of conflicting writes to a memory location is small.

In contrast, in a shared virtual memory on a loosely coupled multiprocessor which has no physically shared memory, and having a nontrivial communication cost between processors, conflicts are not likely to be solved with negligible delay, and they resemble much more a “page
fault” in a traditional virtual memory system. Thus, there are two design choices that greatly influence the implementation of a shared virtual memory: the granularity of the memory units (i.e., the “page size”) and the strategy for maintaining coherence.

Memory coherence strategies are classified based on how they deal with '''''page synchronization''''' and '''''page ownership'''''. The algorithms for memory coherence depend on the page fault handlers, their servers and the data structures used. So ''page table'' becomes an important part of these protocols.

Two classes of algorithms for solving the memory coherence problem are centralized manager (figure 2) and distributed manager (figure 3). In the centralized system, one processor is designated the "monitor". This processor contains a list of information for each page in the cache. The list includes the owner of the page (which processor accessed it last) and all the processors that have copies of the page. When a page is invalidated, then invalidation message is only sent to processors that have copies of the page. This differs from a bus-based system, as the invalidation message is broadcasted to all processors. A drawback to the centralized manager is that there is a bottleneck at the monitor processor. This bottleneck is alleviated by using a distributed manager.

The distributed manager is similar to a centralized manager, but instead of one processor monitoring all pages, a subset of the pages is given to each processor. So, processor 0 would only monitor pages 1 through i, and processor 1 would only monitor pages i+1 through n.[[#5foot|[5]]]

= Synchronization =

Two related concepts dealing with synchronization are '''process synchronization''' and '''data synchronization'''. Process synchronization is concerned with different processes committing to a certain sequence of actions. Data synchronization deals with maintaining data integrity across various copies of a dataset. Process synchronization primitives can be used to implement data synchronization.

[http://en.wikipedia.org/wiki/Mutual_exclusion Mutual exclusion] is the main requirement to be fulfilled in order to synchronize processes, and is needed in both single-processor and multiprocessor systems. There are various approaches to provide mutual exclusion in a system:

* Disabling interrupts
* Locks
* Mutex
* Semaphores
* Barriers
* Test and Set

The next section discusses if there is an alternative to implement mutual exclusion without requiring any hardware support. Peterson's algorithm is one such software solution for guaranteeing mutual exclusion.

== Hardware support ==

Exclusive locking assumes the worst and proceeds only after acquiring all locks such that no other thread can interfere. This is a ''pessimistic'' approach. In contrast, the ''optimistic'' approach proceeds with an update, hoping that it can be completed without any interference. This requires ''collision detection'' during the update. The optimistic approach is thus, more efficient in fine-grained operations.

Special instructions are provided by processors designed for multiprocessor operations in order to manage concurrent access to shared variables. Atomic instructions like [http://en.wikipedia.org/wiki/Test_and_Test-and-set test-and-set], [http://en.wikipedia.org/wiki/Fetch-and-add fetch-and-increment] and [http://en.wikipedia.org/wiki/Swap_%28computer_science%29 swap] were sufficient for early processors to implement mutexes for concurrent objects. Today, every modern processor relies on some form of read-modify-write atomic instruction such as [http://en.wikipedia.org/wiki/Compare-and-swap compare-and-swap], [http://en.wikipedia.org/wiki/Load-link/store-conditional LL/SC] etc. for the same.

=Resources=

[[#1body|1.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_jp#Memory_Consistency_Problem 
[[#2body|2.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_ss#Cache_Coherence 
[[#3body|3.]] http://www.windowsnetworking.com/articles_tutorials/Cache-Coherency.html 
[[#4body|4.]]http://web.sfc.keio.ac.jp/~rdv/keio/sfc/teaching/architecture/architecture-2007/lec08.html 
[[#5body|5.]]http://dl.acm.org/citation.cfm?id=75105

CSC 456 Spring 2012/ch7 MN

2012-03-19T17:59:05Z

Jmmohund2: /* MSI */

==Introduction==
Though the migration from [http://en.wikipedia.org/wiki/Uniprocessor_system uniprocessor system] to [http://en.wikipedia.org/wiki/Multiprocessing multiprocessing] systems is not new, the world of parallel computers is undergoing a continuous change. Parallel computers, which started as high-end super-computing systems for carrying out huge calculations, are now ubiquitous and are present in all mainstream architectures for servers, desktops, and embedded systems. In order to design parallel architectures to meet programmer's needs and expectations more closely, exciting and challenging changes exist. The three main areas which are being considered by scientists today are: [http://en.wikipedia.org/wiki/Cache_coherence cache coherence], memory consistency and [http://en.wikipedia.org/wiki/Synchronization_%28computer_science%29 synchronization].

=Cache Coherence Problem=

[[Image:sharedmem.jpg|thumbnail|right|600px|Shared Memory system with dedicated Cache for each processor[[#4foot|[4]]]]]

In a system with single processor (single core), maintaining cache coherence is simple and easy but in a multiprocessor system, it is much more complicated. Data can be present in any processor's cache and the protocol needs to ensure that the data is same in all caches. If it cannot ensure that all the caches are same, then it needs to flag a cache line indicating that it is not updated.

In the figure shown here, this is a 4 processor shared memory system where each processor has its own cache. Suppose processor P1 reads memory location M1 and stores it in its local cache. Then, if P2 reads same location memory location then M1 gets stored in P2’s cache. Now, if P1 changes value of M1, two copies of same data, residing in different caches will become different. When P2 operates on M1, it uses the stale value of M1 that was stored in its cache. It is the responsibility of cache coherence protocol to prevent this. Hardware support is needed to provide a coherent view of data in multiple caches. This is known as write propagation requirement.

One may think that cache write policy can provide cache coherence, but this is incorrect. Cache write policy only controls how a change in value of cache is propagated to lower level cache or main memory. It is not responsible for propagating changes to other caches.

==Cache Coherence Protocols==

The two basic methods to utilize the inter-core bus to notify other cores when a core changes something in its cache are '''update''' and '''invalidate'''. In the update method, if variable 'x' is modified by core 1, core 1 has to send the updated value of 'x' onto the inter-core bus. Each cache listens to the inter-core bus and if a cache sees a variable on the bus which it has a copy of, it will read the updated value. This ensures that all caches have the most up-to-date value of the variable.[[#3foot|[3]]]

In case of invalidation, an invalidation message is sent onto the inter-core bus when a variable is changed. The other caches will read this invalidation signal and if its core attempts to access that variable, it will result in a cache miss and the variable will be read from main memory.

The update method results in significant amount of traffic on the inter-core bus as the update signal is sent onto the bus every time the variable is updated. The invalidation method only requires that an invalidation signal be sent the first time a variable is altered; this is why the invalidation method is the preferred method.

In order to improve cache coherence performance over the years, several protocols have been proposed.

===MSI===

MSI stands for Modified, Shared, and Invalid, based on the three states that a line of cache can be in. The modified state means that a variable in the cache has been modified and therefore has a different value than that found in main memory; the cache is responsible for writing the variable back to main memory. The shared state means that the variable exists in at least one cache and is not modified; the cache can evict the variable without writing it back to the main memory. The invalid state means that the value of the variable has been modified by another cache and this value is invalid; the cache must read a new value from main memory (or another cache).

A drawback to this protocol occurs when a single processor wants to read blocks and then write to them without another processor sharing that block. After reading the block, a bus transaction places the block into a shared state. The write then occurs and another bus transaction is sent to invalidate the shared copy. This second transaction is useless as no other processors are sharing the block, but the MSI protocol has no way to specify this.
[[Image:msi.jpg|thumbnail|right|600px|MSI State Diagram]]

===MESI===

MESI stands for Modified, Exclusive, Shared, and Invalid. The modified and invalid states are the same for this protocol as they are for the MSI protocol. This protocol introduces a new state; the exclusive state. The Exclusive state means that the variable is in only this cache and the value of it matches the value within the main memory. This now means that the shared state indicates that the variable is contained in more than one cache.

===MOSI===

The MOSI protocol is identical to the MSI protocol except that it adds an owned state. The owned state means that the processor "owns" the variable and will provide the current value to other caches when requested (or at least it will decide if it will provide it when asked). This is useful because another cache will not have to read the value from main memory and will receive it from the owning cache much, much, faster.

===MOESI===

The MOESI protocol is a combination of the MESI and MOSI protocols.

=Memory Consistency Problem=

Memory consistency deals with the ordering of memory operations (load and store) to different memory locations. In a single processor system, code will execute correctly if the compiler preserves the order of the access to synchronization variables and other dependent variables. But in shared memory model with multiple processors, two threads could access a shared data element, such as a synchronization variable, the output of the threads would change based on which thread, accesses the shared data element earlier. If this were to occur, then the program output may not be the value expected. Maintaining program order is very important for memory consistency but it comes with performance degradation.

== Memory Consistency Models ==

The memory consistency model of a shared-memory multiprocessor is a formal speciﬁcation of how the memory system appears to the programmer. It eliminates the gap between the behavior expected by the programmer and the actual behavior supported by a system. Effectively, the consistency model places restrictions on the values that can be returned by a read, in a shared-memory program execution.

In a single processor system, in order to maintain memory consistency, it needs to ensure that the compiler preserves the program order when accessing synchronization variables. But in a multiprocessor system, it is required to ensure that accesses of one processor appear to execute in program order to all other processors, at least partially.

=== Memory semantics in Uniprocessor systems ===

Uniprocessor languages use simple sequential semantics for memory operations, which allow the programmer to assume that all memory operations will occur one at a time in the sequential order speciﬁed by the program. Thus, one can expect for the value in a particular location be the same for a read as it was for the last write because of the sequential program execution. It is sufﬁcient to only maintain uniprocessor data and control dependences. The compiler and hardware can freely reorder operations to different locations if the uniprocessor data and control dependences are respected. This enables compiler optimizations such as register allocation, code motion, and loop transformations, and hardware optimizations, such as pipelining, multiple issue, write buffer bypassing and forwarding, and lockup-free caches, all of which lead to overlapping and reordering of memory operations.[[#4foot|[4]]]

=== Memory semantics in multiprocessor systems ===

Programmer's implicit expectations are:

* memory accesses in a processor takes place according to the program order.
* Each memory access is performed atomically.

A strong consistency model attempting uniprocessor-like consistency could cause global bottleneck, costing performance. Thus, '''''weak''''' consistency models are deployed to improve performance. The advanatges of such models are:

* They support out-of-order execution within individual CPUs
* Relaxes latency issues with near-simultaneous accesses by different CPUs

The following are the various consistency models and it is the programmer who must take into account the memory consistency model to create correct software:
{| class="wikitable sortable" border="1"
|+ Sortable Table of Memory Consistency Models
|-
! scope="col" class="unsortable" | Type of Constancy Modle
! scope="col" | Strictness rank, 1-10(lowest-highest)
|-
| [http://en.wikipedia.org/wiki/Linearizability linearizability (also known as strict or atomic consistency)]|| 3
|-
| [http://en.wikipedia.org/wiki/Sequential_consistency sequential consistency]|| 3
|-
| [http://en.wikipedia.org/wiki/Causal_consistency causal consistency]|| 1
|-
|[http://en.wikipedia.org/wiki/Release_consistency release consistency]|| 3
|-
| [http://en.wikipedia.org/wiki/Eventual_consistency eventual consistency ]|| 2
|-
|[http://en.wikipedia.org/wiki/Delta_consistency delta consistency]|| 1
|-
| [http://en.wikipedia.org/wiki/PRAM_consistency PRAM consistency (also known as FIFO consistency)]|| 1
|-
| [http://en.wikipedia.org/wiki/Weak_consistency weak consistency]|| 1
|-
|[http://en.wikipedia.org/wiki/Vector-field_consistency vector-field consistency]|| 3
|-
| fork consistency|| 2
|-
|[http://en.wikipedia.org/wiki/Serializability serializability]|| 3
|-
|one-copy serializability|| 3
|-
|entry consistency|| 3
|}

== Memory Coherence and Shared Virtual Memory ==

[[Image:Centralized.png|thumbnail|right|350px|'''Figure 2''' Example of a centralized system. Here, one processor monitors all pages in the cache.]]
[[Image:Distributed.png|thumbnail|right|350px|'''Figure 3''' Example of a distributed system. Here, each processor monitors a different subset of pages in the cache.]]
The memory coherence problem in a shared virtual memory system and in multicache systems are different. In a multicache multiprocessor, there are processors sharing a physical memory through their private caches. The relatively small size of a cache and the fast bus connection to the shared memory, enables using a sophisticated coherence protocol for the multicache hardware such that the time delay of conflicting writes to a memory location is small.

In contrast, in a shared virtual memory on a loosely coupled multiprocessor which has no physically shared memory, and having a nontrivial communication cost between processors, conflicts are not likely to be solved with negligible delay, and they resemble much more a “page
fault” in a traditional virtual memory system. Thus, there are two design choices that greatly influence the implementation of a shared virtual memory: the granularity of the memory units (i.e., the “page size”) and the strategy for maintaining coherence.

Memory coherence strategies are classified based on how they deal with '''''page synchronization''''' and '''''page ownership'''''. The algorithms for memory coherence depend on the page fault handlers, their servers and the data structures used. So ''page table'' becomes an important part of these protocols.

Two classes of algorithms for solving the memory coherence problem are centralized manager (figure 2) and distributed manager (figure 3). In the centralized system, one processor is designated the "monitor". This processor contains a list of information for each page in the cache. The list includes the owner of the page (which processor accessed it last) and all the processors that have copies of the page. When a page is invalidated, then invalidation message is only sent to processors that have copies of the page. This differs from a bus-based system, as the invalidation message is broadcasted to all processors. A drawback to the centralized manager is that there is a bottleneck at the monitor processor. This bottleneck is alleviated by using a distributed manager.

The distributed manager is similar to a centralized manager, but instead of one processor monitoring all pages, a subset of the pages is given to each processor. So, processor 0 would only monitor pages 1 through i, and processor 1 would only monitor pages i+1 through n.[[#5foot|[5]]]

= Synchronization =

Two related concepts dealing with synchronization are '''process synchronization''' and '''data synchronization'''. Process synchronization is concerned with different processes committing to a certain sequence of actions. Data synchronization deals with maintaining data integrity across various copies of a dataset. Process synchronization primitives can be used to implement data synchronization.

[http://en.wikipedia.org/wiki/Mutual_exclusion Mutual exclusion] is the main requirement to be fulfilled in order to synchronize processes, and is needed in both single-processor and multiprocessor systems. There are various approaches to provide mutual exclusion in a system:

* Disabling interrupts
* Locks
* Mutex
* Semaphores
* Barriers
* Test and Set

The next section discusses if there is an alternative to implement mutual exclusion without requiring any hardware support. Peterson's algorithm is one such software solution for guaranteeing mutual exclusion.

== Hardware support ==

Exclusive locking assumes the worst and proceeds only after acquiring all locks such that no other thread can interfere. This is a ''pessimistic'' approach. In contrast, the ''optimistic'' approach proceeds with an update, hoping that it can be completed without any interference. This requires ''collision detection'' during the update. The optimistic approach is thus, more efficient in fine-grained operations.

Special instructions are provided by processors designed for multiprocessor operations in order to manage concurrent access to shared variables. Atomic instructions like [http://en.wikipedia.org/wiki/Test_and_Test-and-set test-and-set], [http://en.wikipedia.org/wiki/Fetch-and-add fetch-and-increment] and [http://en.wikipedia.org/wiki/Swap_%28computer_science%29 swap] were sufficient for early processors to implement mutexes for concurrent objects. Today, every modern processor relies on some form of read-modify-write atomic instruction such as [http://en.wikipedia.org/wiki/Compare-and-swap compare-and-swap], [http://en.wikipedia.org/wiki/Load-link/store-conditional LL/SC] etc. for the same.

=Resources=

[[#1body|1.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_jp#Memory_Consistency_Problem 
[[#2body|2.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_ss#Cache_Coherence 
[[#3body|3.]] http://www.windowsnetworking.com/articles_tutorials/Cache-Coherency.html 
[[#4body|4.]]http://web.sfc.keio.ac.jp/~rdv/keio/sfc/teaching/architecture/architecture-2007/lec08.html 
[[#5body|5.]]http://dl.acm.org/citation.cfm?id=75105

CSC 456 Spring 2012/ch7 MN

2012-03-19T17:57:46Z

Jmmohund2: /* MSI */

==Introduction==
Though the migration from [http://en.wikipedia.org/wiki/Uniprocessor_system uniprocessor system] to [http://en.wikipedia.org/wiki/Multiprocessing multiprocessing] systems is not new, the world of parallel computers is undergoing a continuous change. Parallel computers, which started as high-end super-computing systems for carrying out huge calculations, are now ubiquitous and are present in all mainstream architectures for servers, desktops, and embedded systems. In order to design parallel architectures to meet programmer's needs and expectations more closely, exciting and challenging changes exist. The three main areas which are being considered by scientists today are: [http://en.wikipedia.org/wiki/Cache_coherence cache coherence], memory consistency and [http://en.wikipedia.org/wiki/Synchronization_%28computer_science%29 synchronization].

=Cache Coherence Problem=

[[Image:sharedmem.jpg|thumbnail|right|600px|Shared Memory system with dedicated Cache for each processor[[#4foot|[4]]]]]

In a system with single processor (single core), maintaining cache coherence is simple and easy but in a multiprocessor system, it is much more complicated. Data can be present in any processor's cache and the protocol needs to ensure that the data is same in all caches. If it cannot ensure that all the caches are same, then it needs to flag a cache line indicating that it is not updated.

In the figure shown here, this is a 4 processor shared memory system where each processor has its own cache. Suppose processor P1 reads memory location M1 and stores it in its local cache. Then, if P2 reads same location memory location then M1 gets stored in P2’s cache. Now, if P1 changes value of M1, two copies of same data, residing in different caches will become different. When P2 operates on M1, it uses the stale value of M1 that was stored in its cache. It is the responsibility of cache coherence protocol to prevent this. Hardware support is needed to provide a coherent view of data in multiple caches. This is known as write propagation requirement.

One may think that cache write policy can provide cache coherence, but this is incorrect. Cache write policy only controls how a change in value of cache is propagated to lower level cache or main memory. It is not responsible for propagating changes to other caches.

==Cache Coherence Protocols==

The two basic methods to utilize the inter-core bus to notify other cores when a core changes something in its cache are '''update''' and '''invalidate'''. In the update method, if variable 'x' is modified by core 1, core 1 has to send the updated value of 'x' onto the inter-core bus. Each cache listens to the inter-core bus and if a cache sees a variable on the bus which it has a copy of, it will read the updated value. This ensures that all caches have the most up-to-date value of the variable.[[#3foot|[3]]]

In case of invalidation, an invalidation message is sent onto the inter-core bus when a variable is changed. The other caches will read this invalidation signal and if its core attempts to access that variable, it will result in a cache miss and the variable will be read from main memory.

The update method results in significant amount of traffic on the inter-core bus as the update signal is sent onto the bus every time the variable is updated. The invalidation method only requires that an invalidation signal be sent the first time a variable is altered; this is why the invalidation method is the preferred method.

In order to improve cache coherence performance over the years, several protocols have been proposed.

===MSI===

MSI stands for Modified, Shared, and Invalid, based on the three states that a line of cache can be in. The modified state means that a variable in the cache has been modified and therefore has a different value than that found in main memory; the cache is responsible for writing the variable back to main memory. The shared state means that the variable exists in at least one cache and is not modified; the cache can evict the variable without writing it back to the main memory. The invalid state means that the value of the variable has been modified by another cache and this value is invalid; the cache must read a new value from main memory (or another cache).

A drawback to this protocol occurs when a single processor wants to read blocks and then write to them without another processor sharing that block. After reading the block, a bus transaction places the block into a shared state. The write then occurs and another bus transaction is sent to invalidate the shared copy. This second transaction is useless as no other processors are sharing the block, but the MSI protocol has no way to specify this.
[[Image:msi.jpg|200px|thumb|left|alt text]]

===MESI===

MESI stands for Modified, Exclusive, Shared, and Invalid. The modified and invalid states are the same for this protocol as they are for the MSI protocol. This protocol introduces a new state; the exclusive state. The Exclusive state means that the variable is in only this cache and the value of it matches the value within the main memory. This now means that the shared state indicates that the variable is contained in more than one cache.

===MOSI===

The MOSI protocol is identical to the MSI protocol except that it adds an owned state. The owned state means that the processor "owns" the variable and will provide the current value to other caches when requested (or at least it will decide if it will provide it when asked). This is useful because another cache will not have to read the value from main memory and will receive it from the owning cache much, much, faster.

===MOESI===

The MOESI protocol is a combination of the MESI and MOSI protocols.

=Memory Consistency Problem=

Memory consistency deals with the ordering of memory operations (load and store) to different memory locations. In a single processor system, code will execute correctly if the compiler preserves the order of the access to synchronization variables and other dependent variables. But in shared memory model with multiple processors, two threads could access a shared data element, such as a synchronization variable, the output of the threads would change based on which thread, accesses the shared data element earlier. If this were to occur, then the program output may not be the value expected. Maintaining program order is very important for memory consistency but it comes with performance degradation.

== Memory Consistency Models ==

The memory consistency model of a shared-memory multiprocessor is a formal speciﬁcation of how the memory system appears to the programmer. It eliminates the gap between the behavior expected by the programmer and the actual behavior supported by a system. Effectively, the consistency model places restrictions on the values that can be returned by a read, in a shared-memory program execution.

In a single processor system, in order to maintain memory consistency, it needs to ensure that the compiler preserves the program order when accessing synchronization variables. But in a multiprocessor system, it is required to ensure that accesses of one processor appear to execute in program order to all other processors, at least partially.

=== Memory semantics in Uniprocessor systems ===

Uniprocessor languages use simple sequential semantics for memory operations, which allow the programmer to assume that all memory operations will occur one at a time in the sequential order speciﬁed by the program. Thus, one can expect for the value in a particular location be the same for a read as it was for the last write because of the sequential program execution. It is sufﬁcient to only maintain uniprocessor data and control dependences. The compiler and hardware can freely reorder operations to different locations if the uniprocessor data and control dependences are respected. This enables compiler optimizations such as register allocation, code motion, and loop transformations, and hardware optimizations, such as pipelining, multiple issue, write buffer bypassing and forwarding, and lockup-free caches, all of which lead to overlapping and reordering of memory operations.[[#4foot|[4]]]

=== Memory semantics in multiprocessor systems ===

Programmer's implicit expectations are:

* memory accesses in a processor takes place according to the program order.
* Each memory access is performed atomically.

A strong consistency model attempting uniprocessor-like consistency could cause global bottleneck, costing performance. Thus, '''''weak''''' consistency models are deployed to improve performance. The advanatges of such models are:

* They support out-of-order execution within individual CPUs
* Relaxes latency issues with near-simultaneous accesses by different CPUs

The following are the various consistency models and it is the programmer who must take into account the memory consistency model to create correct software:
{| class="wikitable sortable" border="1"
|+ Sortable Table of Memory Consistency Models
|-
! scope="col" class="unsortable" | Type of Constancy Modle
! scope="col" | Strictness rank, 1-10(lowest-highest)
|-
| [http://en.wikipedia.org/wiki/Linearizability linearizability (also known as strict or atomic consistency)]|| 3
|-
| [http://en.wikipedia.org/wiki/Sequential_consistency sequential consistency]|| 3
|-
| [http://en.wikipedia.org/wiki/Causal_consistency causal consistency]|| 1
|-
|[http://en.wikipedia.org/wiki/Release_consistency release consistency]|| 3
|-
| [http://en.wikipedia.org/wiki/Eventual_consistency eventual consistency ]|| 2
|-
|[http://en.wikipedia.org/wiki/Delta_consistency delta consistency]|| 1
|-
| [http://en.wikipedia.org/wiki/PRAM_consistency PRAM consistency (also known as FIFO consistency)]|| 1
|-
| [http://en.wikipedia.org/wiki/Weak_consistency weak consistency]|| 1
|-
|[http://en.wikipedia.org/wiki/Vector-field_consistency vector-field consistency]|| 3
|-
| fork consistency|| 2
|-
|[http://en.wikipedia.org/wiki/Serializability serializability]|| 3
|-
|one-copy serializability|| 3
|-
|entry consistency|| 3
|}

== Memory Coherence and Shared Virtual Memory ==

[[Image:Centralized.png|thumbnail|right|350px|'''Figure 2''' Example of a centralized system. Here, one processor monitors all pages in the cache.]]
[[Image:Distributed.png|thumbnail|right|350px|'''Figure 3''' Example of a distributed system. Here, each processor monitors a different subset of pages in the cache.]]
The memory coherence problem in a shared virtual memory system and in multicache systems are different. In a multicache multiprocessor, there are processors sharing a physical memory through their private caches. The relatively small size of a cache and the fast bus connection to the shared memory, enables using a sophisticated coherence protocol for the multicache hardware such that the time delay of conflicting writes to a memory location is small.

In contrast, in a shared virtual memory on a loosely coupled multiprocessor which has no physically shared memory, and having a nontrivial communication cost between processors, conflicts are not likely to be solved with negligible delay, and they resemble much more a “page
fault” in a traditional virtual memory system. Thus, there are two design choices that greatly influence the implementation of a shared virtual memory: the granularity of the memory units (i.e., the “page size”) and the strategy for maintaining coherence.

Memory coherence strategies are classified based on how they deal with '''''page synchronization''''' and '''''page ownership'''''. The algorithms for memory coherence depend on the page fault handlers, their servers and the data structures used. So ''page table'' becomes an important part of these protocols.

Two classes of algorithms for solving the memory coherence problem are centralized manager (figure 2) and distributed manager (figure 3). In the centralized system, one processor is designated the "monitor". This processor contains a list of information for each page in the cache. The list includes the owner of the page (which processor accessed it last) and all the processors that have copies of the page. When a page is invalidated, then invalidation message is only sent to processors that have copies of the page. This differs from a bus-based system, as the invalidation message is broadcasted to all processors. A drawback to the centralized manager is that there is a bottleneck at the monitor processor. This bottleneck is alleviated by using a distributed manager.

The distributed manager is similar to a centralized manager, but instead of one processor monitoring all pages, a subset of the pages is given to each processor. So, processor 0 would only monitor pages 1 through i, and processor 1 would only monitor pages i+1 through n.[[#5foot|[5]]]

= Synchronization =

Two related concepts dealing with synchronization are '''process synchronization''' and '''data synchronization'''. Process synchronization is concerned with different processes committing to a certain sequence of actions. Data synchronization deals with maintaining data integrity across various copies of a dataset. Process synchronization primitives can be used to implement data synchronization.

[http://en.wikipedia.org/wiki/Mutual_exclusion Mutual exclusion] is the main requirement to be fulfilled in order to synchronize processes, and is needed in both single-processor and multiprocessor systems. There are various approaches to provide mutual exclusion in a system:

* Disabling interrupts
* Locks
* Mutex
* Semaphores
* Barriers
* Test and Set

The next section discusses if there is an alternative to implement mutual exclusion without requiring any hardware support. Peterson's algorithm is one such software solution for guaranteeing mutual exclusion.

== Hardware support ==

Exclusive locking assumes the worst and proceeds only after acquiring all locks such that no other thread can interfere. This is a ''pessimistic'' approach. In contrast, the ''optimistic'' approach proceeds with an update, hoping that it can be completed without any interference. This requires ''collision detection'' during the update. The optimistic approach is thus, more efficient in fine-grained operations.

Special instructions are provided by processors designed for multiprocessor operations in order to manage concurrent access to shared variables. Atomic instructions like [http://en.wikipedia.org/wiki/Test_and_Test-and-set test-and-set], [http://en.wikipedia.org/wiki/Fetch-and-add fetch-and-increment] and [http://en.wikipedia.org/wiki/Swap_%28computer_science%29 swap] were sufficient for early processors to implement mutexes for concurrent objects. Today, every modern processor relies on some form of read-modify-write atomic instruction such as [http://en.wikipedia.org/wiki/Compare-and-swap compare-and-swap], [http://en.wikipedia.org/wiki/Load-link/store-conditional LL/SC] etc. for the same.

=Resources=

[[#1body|1.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_jp#Memory_Consistency_Problem 
[[#2body|2.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_ss#Cache_Coherence 
[[#3body|3.]] http://www.windowsnetworking.com/articles_tutorials/Cache-Coherency.html 
[[#4body|4.]]http://web.sfc.keio.ac.jp/~rdv/keio/sfc/teaching/architecture/architecture-2007/lec08.html 
[[#5body|5.]]http://dl.acm.org/citation.cfm?id=75105

File:MSI.jpg

2012-03-19T17:57:26Z

Jmmohund2: uploaded a new version of "File:MSI.jpg": msi diagram

File:MESI.jpg

2012-03-19T17:55:57Z

Jmmohund2: uploaded a new version of "File:MESI.jpg": MESI state diagram

CSC 456 Spring 2012/ch7 MN

2012-03-19T17:38:25Z

Jmmohund2: /* Memory semantics in multiprocessor systems */

==Introduction==
Though the migration from [http://en.wikipedia.org/wiki/Uniprocessor_system uniprocessor system] to [http://en.wikipedia.org/wiki/Multiprocessing multiprocessing] systems is not new, the world of parallel computers is undergoing a continuous change. Parallel computers, which started as high-end super-computing systems for carrying out huge calculations, are now ubiquitous and are present in all mainstream architectures for servers, desktops, and embedded systems. In order to design parallel architectures to meet programmer's needs and expectations more closely, exciting and challenging changes exist. The three main areas which are being considered by scientists today are: [http://en.wikipedia.org/wiki/Cache_coherence cache coherence], memory consistency and [http://en.wikipedia.org/wiki/Synchronization_%28computer_science%29 synchronization].

=Cache Coherence Problem=

[[Image:sharedmem.jpg|thumbnail|right|600px|Shared Memory system with dedicated Cache for each processor[[#4foot|[4]]]]]

In a system with single processor (single core), maintaining cache coherence is simple and easy but in a multiprocessor system, it is much more complicated. Data can be present in any processor's cache and the protocol needs to ensure that the data is same in all caches. If it cannot ensure that all the caches are same, then it needs to flag a cache line indicating that it is not updated.

In the figure shown here, this is a 4 processor shared memory system where each processor has its own cache. Suppose processor P1 reads memory location M1 and stores it in its local cache. Then, if P2 reads same location memory location then M1 gets stored in P2’s cache. Now, if P1 changes value of M1, two copies of same data, residing in different caches will become different. When P2 operates on M1, it uses the stale value of M1 that was stored in its cache. It is the responsibility of cache coherence protocol to prevent this. Hardware support is needed to provide a coherent view of data in multiple caches. This is known as write propagation requirement.

One may think that cache write policy can provide cache coherence, but this is incorrect. Cache write policy only controls how a change in value of cache is propagated to lower level cache or main memory. It is not responsible for propagating changes to other caches.

==Cache Coherence Protocols==

The two basic methods to utilize the inter-core bus to notify other cores when a core changes something in its cache are '''update''' and '''invalidate'''. In the update method, if variable 'x' is modified by core 1, core 1 has to send the updated value of 'x' onto the inter-core bus. Each cache listens to the inter-core bus and if a cache sees a variable on the bus which it has a copy of, it will read the updated value. This ensures that all caches have the most up-to-date value of the variable.[[#3foot|[3]]]

In case of invalidation, an invalidation message is sent onto the inter-core bus when a variable is changed. The other caches will read this invalidation signal and if its core attempts to access that variable, it will result in a cache miss and the variable will be read from main memory.

The update method results in significant amount of traffic on the inter-core bus as the update signal is sent onto the bus every time the variable is updated. The invalidation method only requires that an invalidation signal be sent the first time a variable is altered; this is why the invalidation method is the preferred method.

In order to improve cache coherence performance over the years, several protocols have been proposed.

===MSI===

MSI stands for Modified, Shared, and Invalid, based on the three states that a line of cache can be in. The modified state means that a variable in the cache has been modified and therefore has a different value than that found in main memory; the cache is responsible for writing the variable back to main memory. The shared state means that the variable exists in at least one cache and is not modified; the cache can evict the variable without writing it back to the main memory. The invalid state means that the value of the variable has been modified by another cache and this value is invalid; the cache must read a new value from main memory (or another cache).

A drawback to this protocol occurs when a single processor wants to read blocks and then write to them without another processor sharing that block. After reading the block, a bus transaction places the block into a shared state. The write then occurs and another bus transaction is sent to invalidate the shared copy. This second transaction is useless as no other processors are sharing the block, but the MSI protocol has no way to specify this.

===MESI===

MESI stands for Modified, Exclusive, Shared, and Invalid. The modified and invalid states are the same for this protocol as they are for the MSI protocol. This protocol introduces a new state; the exclusive state. The Exclusive state means that the variable is in only this cache and the value of it matches the value within the main memory. This now means that the shared state indicates that the variable is contained in more than one cache.

===MOSI===

The MOSI protocol is identical to the MSI protocol except that it adds an owned state. The owned state means that the processor "owns" the variable and will provide the current value to other caches when requested (or at least it will decide if it will provide it when asked). This is useful because another cache will not have to read the value from main memory and will receive it from the owning cache much, much, faster.

===MOESI===

The MOESI protocol is a combination of the MESI and MOSI protocols.

=Memory Consistency Problem=

Memory consistency deals with the ordering of memory operations (load and store) to different memory locations. In a single processor system, code will execute correctly if the compiler preserves the order of the access to synchronization variables and other dependent variables. But in shared memory model with multiple processors, two threads could access a shared data element, such as a synchronization variable, the output of the threads would change based on which thread, accesses the shared data element earlier. If this were to occur, then the program output may not be the value expected. Maintaining program order is very important for memory consistency but it comes with performance degradation.

== Memory Consistency Models ==

The memory consistency model of a shared-memory multiprocessor is a formal speciﬁcation of how the memory system appears to the programmer. It eliminates the gap between the behavior expected by the programmer and the actual behavior supported by a system. Effectively, the consistency model places restrictions on the values that can be returned by a read, in a shared-memory program execution.

In a single processor system, in order to maintain memory consistency, it needs to ensure that the compiler preserves the program order when accessing synchronization variables. But in a multiprocessor system, it is required to ensure that accesses of one processor appear to execute in program order to all other processors, at least partially.

=== Memory semantics in Uniprocessor systems ===

Uniprocessor languages use simple sequential semantics for memory operations, which allow the programmer to assume that all memory operations will occur one at a time in the sequential order speciﬁed by the program. Thus, one can expect for the value in a particular location be the same for a read as it was for the last write because of the sequential program execution. It is sufﬁcient to only maintain uniprocessor data and control dependences. The compiler and hardware can freely reorder operations to different locations if the uniprocessor data and control dependences are respected. This enables compiler optimizations such as register allocation, code motion, and loop transformations, and hardware optimizations, such as pipelining, multiple issue, write buffer bypassing and forwarding, and lockup-free caches, all of which lead to overlapping and reordering of memory operations.[[#4foot|[4]]]

=== Memory semantics in multiprocessor systems ===

Programmer's implicit expectations are:

* memory accesses in a processor takes place according to the program order.
* Each memory access is performed atomically.

A strong consistency model attempting uniprocessor-like consistency could cause global bottleneck, costing performance. Thus, '''''weak''''' consistency models are deployed to improve performance. The advanatges of such models are:

* They support out-of-order execution within individual CPUs
* Relaxes latency issues with near-simultaneous accesses by different CPUs

The following are the various consistency models and it is the programmer who must take into account the memory consistency model to create correct software:
{| class="wikitable sortable" border="1"
|+ Sortable Table of Memory Consistency Models
|-
! scope="col" class="unsortable" | Type of Constancy Modle
! scope="col" | Strictness rank, 1-10(lowest-highest)
|-
| [http://en.wikipedia.org/wiki/Linearizability linearizability (also known as strict or atomic consistency)]|| 3
|-
| [http://en.wikipedia.org/wiki/Sequential_consistency sequential consistency]|| 3
|-
| [http://en.wikipedia.org/wiki/Causal_consistency causal consistency]|| 1
|-
|[http://en.wikipedia.org/wiki/Release_consistency release consistency]|| 3
|-
| [http://en.wikipedia.org/wiki/Eventual_consistency eventual consistency ]|| 2
|-
|[http://en.wikipedia.org/wiki/Delta_consistency delta consistency]|| 1
|-
| [http://en.wikipedia.org/wiki/PRAM_consistency PRAM consistency (also known as FIFO consistency)]|| 1
|-
| [http://en.wikipedia.org/wiki/Weak_consistency weak consistency]|| 1
|-
|[http://en.wikipedia.org/wiki/Vector-field_consistency vector-field consistency]|| 3
|-
| fork consistency|| 2
|-
|[http://en.wikipedia.org/wiki/Serializability serializability]|| 3
|-
|one-copy serializability|| 3
|-
|entry consistency|| 3
|}

== Memory Coherence and Shared Virtual Memory ==

The memory coherence problem in a shared virtual memory system and in multicache systems are different. In a multicache multiprocessor, there are processors sharing a physical memory through their private caches. The relatively small size of a cache and the fast bus connection to the shared memory, enables using a sophisticated coherence protocol for the multicache hardware such that the time delay of conflicting writes to a memory location is small.

In contrast, in a shared virtual memory on a loosely coupled multiprocessor which has no physically shared memory, and having a nontrivial communication cost between processors, conflicts are not likely to be solved with negligible delay, and they resemble much more a “page
fault” in a traditional virtual memory system. Thus, there are two design choices that greatly influence the implementation of a shared virtual memory: the granularity of the memory units (i.e., the “page size”) and the strategy for maintaining coherence.

Memory coherence strategies are classified based on how they deal with '''''page synchronization''''' and '''''page ownership'''''. The algorithms for memory coherence depend on the page fault handlers, their servers and the data structures used. So ''page table'' becomes an important part of these protocols.

Two classes of algorithms for solving the memory coherence problem are centralized manager and distributed manager. In the centralized system, one processor is designated the "monitor". This processor contains a list of information for each page in the cache. The list includes the owner of the page (which processor accessed it last) and all the processors that have copies of the page. When a page is invalidated, then invalidation message is only sent to processors that have copies of the page. This differs from a bus-based system, as the invalidation message is broadcasted to all processors. A drawback to the centralized manager is that there is a bottleneck at the monitor processor. This bottleneck is alleviated by using a distributed manager.

The distributed manager is similar to a centralized manager, but instead of one processor monitoring all pages, a subset of the pages is given to each processor. So, processor 0 would only monitor pages 1 through i, and processor 1 would only monitor pages i+1 through n.[[#5foot|[5]]]

= Synchronization =

Two related concepts dealing with synchronization are '''process synchronization''' and '''data synchronization'''. Process synchronization is concerned with different processes committing to a certain sequence of actions. Data synchronization deals with maintaining data integrity across various copies of a dataset. Process synchronization primitives can be used to implement data synchronization.

[http://en.wikipedia.org/wiki/Mutual_exclusion Mutual exclusion] is the main requirement to be fulfilled in order to synchronize processes, and is needed in both single-processor and multiprocessor systems. There are various approaches to provide mutual exclusion in a system:

* Disabling interrupts
* Locks
* Mutex
* Semaphores
* Barriers
* Test and Set

The next section discusses if there is an alternative to implement mutual exclusion without requiring any hardware support. Peterson's algorithm is one such software solution for guaranteeing mutual exclusion.

== Hardware support ==

Exclusive locking assumes the worst and proceeds only after acquiring all locks such that no other thread can interfere. This is a ''pessimistic'' approach. In contrast, the ''optimistic'' approach proceeds with an update, hoping that it can be completed without any interference. This requires ''collision detection'' during the update. The optimistic approach is thus, more efficient in fine-grained operations.

Special instructions are provided by processors designed for multiprocessor operations in order to manage concurrent access to shared variables. Atomic instructions like [http://en.wikipedia.org/wiki/Test_and_Test-and-set test-and-set], [http://en.wikipedia.org/wiki/Fetch-and-add fetch-and-increment] and [http://en.wikipedia.org/wiki/Swap_%28computer_science%29 swap] were sufficient for early processors to implement mutexes for concurrent objects. Today, every modern processor relies on some form of read-modify-write atomic instruction such as [http://en.wikipedia.org/wiki/Compare-and-swap compare-and-swap], [http://en.wikipedia.org/wiki/Load-link/store-conditional LL/SC] etc. for the same.

=Resources=

[[#1body|1.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_jp#Memory_Consistency_Problem 
[[#2body|2.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_ss#Cache_Coherence 
[[#3body|3.]] http://www.windowsnetworking.com/articles_tutorials/Cache-Coherency.html 
[[#4body|4.]]http://web.sfc.keio.ac.jp/~rdv/keio/sfc/teaching/architecture/architecture-2007/lec08.html 
[[#5body|5.]]http://dl.acm.org/citation.cfm?id=75105

CSC 456 Spring 2012/ch7 MN

2012-03-19T17:37:15Z

Jmmohund2: /* Memory semantics in multiprocessor systems */

==Introduction==
Though the migration from [http://en.wikipedia.org/wiki/Uniprocessor_system uniprocessor system] to [http://en.wikipedia.org/wiki/Multiprocessing multiprocessing] systems is not new, the world of parallel computers is undergoing a continuous change. Parallel computers, which started as high-end super-computing systems for carrying out huge calculations, are now ubiquitous and are present in all mainstream architectures for servers, desktops, and embedded systems. In order to design parallel architectures to meet programmer's needs and expectations more closely, exciting and challenging changes exist. The three main areas which are being considered by scientists today are: [http://en.wikipedia.org/wiki/Cache_coherence cache coherence], memory consistency and [http://en.wikipedia.org/wiki/Synchronization_%28computer_science%29 synchronization].

=Cache Coherence Problem=

[[Image:sharedmem.jpg|thumbnail|right|600px|Shared Memory system with dedicated Cache for each processor[[#4foot|[4]]]]]

In a system with single processor (single core), maintaining cache coherence is simple and easy but in a multiprocessor system, it is much more complicated. Data can be present in any processor's cache and the protocol needs to ensure that the data is same in all caches. If it cannot ensure that all the caches are same, then it needs to flag a cache line indicating that it is not updated.

In the figure shown here, this is a 4 processor shared memory system where each processor has its own cache. Suppose processor P1 reads memory location M1 and stores it in its local cache. Then, if P2 reads same location memory location then M1 gets stored in P2’s cache. Now, if P1 changes value of M1, two copies of same data, residing in different caches will become different. When P2 operates on M1, it uses the stale value of M1 that was stored in its cache. It is the responsibility of cache coherence protocol to prevent this. Hardware support is needed to provide a coherent view of data in multiple caches. This is known as write propagation requirement.

One may think that cache write policy can provide cache coherence, but this is incorrect. Cache write policy only controls how a change in value of cache is propagated to lower level cache or main memory. It is not responsible for propagating changes to other caches.

==Cache Coherence Protocols==

The two basic methods to utilize the inter-core bus to notify other cores when a core changes something in its cache are '''update''' and '''invalidate'''. In the update method, if variable 'x' is modified by core 1, core 1 has to send the updated value of 'x' onto the inter-core bus. Each cache listens to the inter-core bus and if a cache sees a variable on the bus which it has a copy of, it will read the updated value. This ensures that all caches have the most up-to-date value of the variable.[[#3foot|[3]]]

In case of invalidation, an invalidation message is sent onto the inter-core bus when a variable is changed. The other caches will read this invalidation signal and if its core attempts to access that variable, it will result in a cache miss and the variable will be read from main memory.

The update method results in significant amount of traffic on the inter-core bus as the update signal is sent onto the bus every time the variable is updated. The invalidation method only requires that an invalidation signal be sent the first time a variable is altered; this is why the invalidation method is the preferred method.

In order to improve cache coherence performance over the years, several protocols have been proposed.

===MSI===

MSI stands for Modified, Shared, and Invalid, based on the three states that a line of cache can be in. The modified state means that a variable in the cache has been modified and therefore has a different value than that found in main memory; the cache is responsible for writing the variable back to main memory. The shared state means that the variable exists in at least one cache and is not modified; the cache can evict the variable without writing it back to the main memory. The invalid state means that the value of the variable has been modified by another cache and this value is invalid; the cache must read a new value from main memory (or another cache).

A drawback to this protocol occurs when a single processor wants to read blocks and then write to them without another processor sharing that block. After reading the block, a bus transaction places the block into a shared state. The write then occurs and another bus transaction is sent to invalidate the shared copy. This second transaction is useless as no other processors are sharing the block, but the MSI protocol has no way to specify this.

===MESI===

MESI stands for Modified, Exclusive, Shared, and Invalid. The modified and invalid states are the same for this protocol as they are for the MSI protocol. This protocol introduces a new state; the exclusive state. The Exclusive state means that the variable is in only this cache and the value of it matches the value within the main memory. This now means that the shared state indicates that the variable is contained in more than one cache.

===MOSI===

The MOSI protocol is identical to the MSI protocol except that it adds an owned state. The owned state means that the processor "owns" the variable and will provide the current value to other caches when requested (or at least it will decide if it will provide it when asked). This is useful because another cache will not have to read the value from main memory and will receive it from the owning cache much, much, faster.

===MOESI===

The MOESI protocol is a combination of the MESI and MOSI protocols.

=Memory Consistency Problem=

Memory consistency deals with the ordering of memory operations (load and store) to different memory locations. In a single processor system, code will execute correctly if the compiler preserves the order of the access to synchronization variables and other dependent variables. But in shared memory model with multiple processors, two threads could access a shared data element, such as a synchronization variable, the output of the threads would change based on which thread, accesses the shared data element earlier. If this were to occur, then the program output may not be the value expected. Maintaining program order is very important for memory consistency but it comes with performance degradation.

== Memory Consistency Models ==

The memory consistency model of a shared-memory multiprocessor is a formal speciﬁcation of how the memory system appears to the programmer. It eliminates the gap between the behavior expected by the programmer and the actual behavior supported by a system. Effectively, the consistency model places restrictions on the values that can be returned by a read, in a shared-memory program execution.

In a single processor system, in order to maintain memory consistency, it needs to ensure that the compiler preserves the program order when accessing synchronization variables. But in a multiprocessor system, it is required to ensure that accesses of one processor appear to execute in program order to all other processors, at least partially.

=== Memory semantics in Uniprocessor systems ===

Uniprocessor languages use simple sequential semantics for memory operations, which allow the programmer to assume that all memory operations will occur one at a time in the sequential order speciﬁed by the program. Thus, one can expect for the value in a particular location be the same for a read as it was for the last write because of the sequential program execution. It is sufﬁcient to only maintain uniprocessor data and control dependences. The compiler and hardware can freely reorder operations to different locations if the uniprocessor data and control dependences are respected. This enables compiler optimizations such as register allocation, code motion, and loop transformations, and hardware optimizations, such as pipelining, multiple issue, write buffer bypassing and forwarding, and lockup-free caches, all of which lead to overlapping and reordering of memory operations.[[#4foot|[4]]]

=== Memory semantics in multiprocessor systems ===

Programmer's implicit expectations are:

* memory accesses in a processor takes place according to the program order.
* Each memory access is performed atomically.

A strong consistency model attempting uniprocessor-like consistency could cause global bottleneck, costing performance. Thus, '''''weak''''' consistency models are deployed to improve performance. The advanatges of such models are:

* They support out-of-order execution within individual CPUs
* Relaxes latency issues with near-simultaneous accesses by different CPUs

The following are the various consistency models and it is the programmer who must take into account the memory consistency model to create correct software:
{| class="wikitable sortable" border="1"
|+ Sortable Table of Memory Consistency Models
|-
! scope="col" class="unsortable" | Type of Constancy Modle
! scope="col" | Strictness rank, 1-10(lowest-highest)
! scope="col" | Requires Programmer annotation
|-
| [http://en.wikipedia.org/wiki/Linearizability linearizability (also known as strict or atomic consistency)]|| 3 ||
|-
| [http://en.wikipedia.org/wiki/Sequential_consistency sequential consistency]|| 3 ||
|-
| [http://en.wikipedia.org/wiki/Causal_consistency causal consistency]|| 1 ||
|-
|[http://en.wikipedia.org/wiki/Release_consistency release consistency]|| 3 ||
|-
| [http://en.wikipedia.org/wiki/Eventual_consistency eventual consistency ]|| 2 ||
|-
|[http://en.wikipedia.org/wiki/Delta_consistency delta consistency]|| 1 ||
|-
| [http://en.wikipedia.org/wiki/PRAM_consistency PRAM consistency (also known as FIFO consistency)]|| 1 ||
|-
| [http://en.wikipedia.org/wiki/Weak_consistency weak consistency]|| 1 ||
|-
|[http://en.wikipedia.org/wiki/Vector-field_consistency vector-field consistency]|| 3 ||
|-
| fork consistency|| 2 ||
|-
|[http://en.wikipedia.org/wiki/Serializability serializability]|| 3 ||
|-
|one-copy serializability|| 3 ||
|-
|entry consistency|| 3 ||
|}

== Memory Coherence and Shared Virtual Memory ==

The memory coherence problem in a shared virtual memory system and in multicache systems are different. In a multicache multiprocessor, there are processors sharing a physical memory through their private caches. The relatively small size of a cache and the fast bus connection to the shared memory, enables using a sophisticated coherence protocol for the multicache hardware such that the time delay of conflicting writes to a memory location is small.

In contrast, in a shared virtual memory on a loosely coupled multiprocessor which has no physically shared memory, and having a nontrivial communication cost between processors, conflicts are not likely to be solved with negligible delay, and they resemble much more a “page
fault” in a traditional virtual memory system. Thus, there are two design choices that greatly influence the implementation of a shared virtual memory: the granularity of the memory units (i.e., the “page size”) and the strategy for maintaining coherence.

Memory coherence strategies are classified based on how they deal with '''''page synchronization''''' and '''''page ownership'''''. The algorithms for memory coherence depend on the page fault handlers, their servers and the data structures used. So ''page table'' becomes an important part of these protocols.

Two classes of algorithms for solving the memory coherence problem are centralized manager and distributed manager. In the centralized system, one processor is designated the "monitor". This processor contains a list of information for each page in the cache. The list includes the owner of the page (which processor accessed it last) and all the processors that have copies of the page. When a page is invalidated, then invalidation message is only sent to processors that have copies of the page. This differs from a bus-based system, as the invalidation message is broadcasted to all processors. A drawback to the centralized manager is that there is a bottleneck at the monitor processor. This bottleneck is alleviated by using a distributed manager.

The distributed manager is similar to a centralized manager, but instead of one processor monitoring all pages, a subset of the pages is given to each processor. So, processor 0 would only monitor pages 1 through i, and processor 1 would only monitor pages i+1 through n.[[#5foot|[5]]]

= Synchronization =

Two related concepts dealing with synchronization are '''process synchronization''' and '''data synchronization'''. Process synchronization is concerned with different processes committing to a certain sequence of actions. Data synchronization deals with maintaining data integrity across various copies of a dataset. Process synchronization primitives can be used to implement data synchronization.

[http://en.wikipedia.org/wiki/Mutual_exclusion Mutual exclusion] is the main requirement to be fulfilled in order to synchronize processes, and is needed in both single-processor and multiprocessor systems. There are various approaches to provide mutual exclusion in a system:

* Disabling interrupts
* Locks
* Mutex
* Semaphores
* Barriers
* Test and Set

The next section discusses if there is an alternative to implement mutual exclusion without requiring any hardware support. Peterson's algorithm is one such software solution for guaranteeing mutual exclusion.

== Hardware support ==

Exclusive locking assumes the worst and proceeds only after acquiring all locks such that no other thread can interfere. This is a ''pessimistic'' approach. In contrast, the ''optimistic'' approach proceeds with an update, hoping that it can be completed without any interference. This requires ''collision detection'' during the update. The optimistic approach is thus, more efficient in fine-grained operations.

Special instructions are provided by processors designed for multiprocessor operations in order to manage concurrent access to shared variables. Atomic instructions like [http://en.wikipedia.org/wiki/Test_and_Test-and-set test-and-set], [http://en.wikipedia.org/wiki/Fetch-and-add fetch-and-increment] and [http://en.wikipedia.org/wiki/Swap_%28computer_science%29 swap] were sufficient for early processors to implement mutexes for concurrent objects. Today, every modern processor relies on some form of read-modify-write atomic instruction such as [http://en.wikipedia.org/wiki/Compare-and-swap compare-and-swap], [http://en.wikipedia.org/wiki/Load-link/store-conditional LL/SC] etc. for the same.

=Resources=

[[#1body|1.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_jp#Memory_Consistency_Problem 
[[#2body|2.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_ss#Cache_Coherence 
[[#3body|3.]] http://www.windowsnetworking.com/articles_tutorials/Cache-Coherency.html 
[[#4body|4.]]http://web.sfc.keio.ac.jp/~rdv/keio/sfc/teaching/architecture/architecture-2007/lec08.html 
[[#5body|5.]]http://dl.acm.org/citation.cfm?id=75105

CSC 456 Spring 2012/ch7 MN

2012-03-19T17:32:19Z

Jmmohund2: /* Memory semantics in multiprocessor systems */

==Introduction==
Though the migration from [http://en.wikipedia.org/wiki/Uniprocessor_system uniprocessor system] to [http://en.wikipedia.org/wiki/Multiprocessing multiprocessing] systems is not new, the world of parallel computers is undergoing a continuous change. Parallel computers, which started as high-end super-computing systems for carrying out huge calculations, are now ubiquitous and are present in all mainstream architectures for servers, desktops, and embedded systems. In order to design parallel architectures to meet programmer's needs and expectations more closely, exciting and challenging changes exist. The three main areas which are being considered by scientists today are: [http://en.wikipedia.org/wiki/Cache_coherence cache coherence], memory consistency and [http://en.wikipedia.org/wiki/Synchronization_%28computer_science%29 synchronization].

=Cache Coherence Problem=

[[Image:sharedmem.jpg|thumbnail|right|600px|Shared Memory system with dedicated Cache for each processor[[#4foot|[4]]]]]

In a system with single processor (single core), maintaining cache coherence is simple and easy but in a multiprocessor system, it is much more complicated. Data can be present in any processor's cache and the protocol needs to ensure that the data is same in all caches. If it cannot ensure that all the caches are same, then it needs to flag a cache line indicating that it is not updated.

In the figure shown here, this is a 4 processor shared memory system where each processor has its own cache. Suppose processor P1 reads memory location M1 and stores it in its local cache. Then, if P2 reads same location memory location then M1 gets stored in P2’s cache. Now, if P1 changes value of M1, two copies of same data, residing in different caches will become different. When P2 operates on M1, it uses the stale value of M1 that was stored in its cache. It is the responsibility of cache coherence protocol to prevent this. Hardware support is needed to provide a coherent view of data in multiple caches. This is known as write propagation requirement.

One may think that cache write policy can provide cache coherence, but this is incorrect. Cache write policy only controls how a change in value of cache is propagated to lower level cache or main memory. It is not responsible for propagating changes to other caches.

==Cache Coherence Protocols==

The two basic methods to utilize the inter-core bus to notify other cores when a core changes something in its cache are '''update''' and '''invalidate'''. In the update method, if variable 'x' is modified by core 1, core 1 has to send the updated value of 'x' onto the inter-core bus. Each cache listens to the inter-core bus and if a cache sees a variable on the bus which it has a copy of, it will read the updated value. This ensures that all caches have the most up-to-date value of the variable.[[#3foot|[3]]]

In case of invalidation, an invalidation message is sent onto the inter-core bus when a variable is changed. The other caches will read this invalidation signal and if its core attempts to access that variable, it will result in a cache miss and the variable will be read from main memory.

The update method results in significant amount of traffic on the inter-core bus as the update signal is sent onto the bus every time the variable is updated. The invalidation method only requires that an invalidation signal be sent the first time a variable is altered; this is why the invalidation method is the preferred method.

In order to improve cache coherence performance over the years, several protocols have been proposed.

===MSI===

MSI stands for Modified, Shared, and Invalid, based on the three states that a line of cache can be in. The modified state means that a variable in the cache has been modified and therefore has a different value than that found in main memory; the cache is responsible for writing the variable back to main memory. The shared state means that the variable exists in at least one cache and is not modified; the cache can evict the variable without writing it back to the main memory. The invalid state means that the value of the variable has been modified by another cache and this value is invalid; the cache must read a new value from main memory (or another cache).

A drawback to this protocol occurs when a single processor wants to read blocks and then write to them without another processor sharing that block. After reading the block, a bus transaction places the block into a shared state. The write then occurs and another bus transaction is sent to invalidate the shared copy. This second transaction is useless as no other processors are sharing the block, but the MSI protocol has no way to specify this.

===MESI===

MESI stands for Modified, Exclusive, Shared, and Invalid. The modified and invalid states are the same for this protocol as they are for the MSI protocol. This protocol introduces a new state; the exclusive state. The Exclusive state means that the variable is in only this cache and the value of it matches the value within the main memory. This now means that the shared state indicates that the variable is contained in more than one cache.

===MOSI===

The MOSI protocol is identical to the MSI protocol except that it adds an owned state. The owned state means that the processor "owns" the variable and will provide the current value to other caches when requested (or at least it will decide if it will provide it when asked). This is useful because another cache will not have to read the value from main memory and will receive it from the owning cache much, much, faster.

===MOESI===

The MOESI protocol is a combination of the MESI and MOSI protocols.

=Memory Consistency Problem=

Memory consistency deals with the ordering of memory operations (load and store) to different memory locations. In a single processor system, code will execute correctly if the compiler preserves the order of the access to synchronization variables and other dependent variables. But in shared memory model with multiple processors, two threads could access a shared data element, such as a synchronization variable, the output of the threads would change based on which thread, accesses the shared data element earlier. If this were to occur, then the program output may not be the value expected. Maintaining program order is very important for memory consistency but it comes with performance degradation.

== Memory Consistency Models ==

The memory consistency model of a shared-memory multiprocessor is a formal speciﬁcation of how the memory system appears to the programmer. It eliminates the gap between the behavior expected by the programmer and the actual behavior supported by a system. Effectively, the consistency model places restrictions on the values that can be returned by a read, in a shared-memory program execution.

In a single processor system, in order to maintain memory consistency, it needs to ensure that the compiler preserves the program order when accessing synchronization variables. But in a multiprocessor system, it is required to ensure that accesses of one processor appear to execute in program order to all other processors, at least partially.

=== Memory semantics in Uniprocessor systems ===

Uniprocessor languages use simple sequential semantics for memory operations, which allow the programmer to assume that all memory operations will occur one at a time in the sequential order speciﬁed by the program. Thus, one can expect for the value in a particular location be the same for a read as it was for the last write because of the sequential program execution. It is sufﬁcient to only maintain uniprocessor data and control dependences. The compiler and hardware can freely reorder operations to different locations if the uniprocessor data and control dependences are respected. This enables compiler optimizations such as register allocation, code motion, and loop transformations, and hardware optimizations, such as pipelining, multiple issue, write buffer bypassing and forwarding, and lockup-free caches, all of which lead to overlapping and reordering of memory operations. [4]

=== Memory semantics in multiprocessor systems ===

Programmer's implicit expectations are:

* memory accesses in a processor takes place according to the program order.
* Each memory access is performed atomically.

A strong consistency model attempting uniprocessor-like consistency could cause global bottleneck, costing performance. Thus, '''''weak''''' consistency models are deployed to improve performance. The advanatges of such models are:

* They support out-of-order execution within individual CPUs
* Relaxes latency issues with near-simultaneous accesses by different CPUs

The following are the various consistency models and it is the programmer who must take into account the memory consistency model to create correct software:
{| class="wikitable sortable" border="1"
|+ Sortable Table of Memory Consistency Models
|-
! scope="col" class="unsortable" | Type of Constancy Modle
! scope="col" | Strictness rank, 1-10(lowest-highest)
! scope="col" | Requires Programmer annotation
|-
| [http://en.wikipedia.org/wiki/Linearizability linearizability (also known as strict or atomic consistency)]|| 3 ||
|-
| [http://en.wikipedia.org/wiki/Sequential_consistency sequential consistency]|| 3 ||
|-
| [http://en.wikipedia.org/wiki/Causal_consistency causal consistency]|| 1 ||
|-
|[http://en.wikipedia.org/wiki/Release_consistency release consistency]|| 3 ||
|-
| [http://en.wikipedia.org/wiki/Eventual_consistency eventual consistency ]|| 2 ||
|-
|[http://en.wikipedia.org/wiki/Delta_consistency delta consistency]|| ||
|-
| [http://en.wikipedia.org/wiki/PRAM_consistency PRAM consistency (also known as FIFO consistency)]|| 1 ||
|-
| [http://en.wikipedia.org/wiki/Weak_consistency weak consistency]|| 1 ||
|-
|[http://en.wikipedia.org/wiki/Vector-field_consistency vector-field consistency]|| ||
|-
| fork consistency|| ||
|-
|[http://en.wikipedia.org/wiki/Serializability serializability]|| 3 ||
|-
|one-copy serializability|| ||
|-
|entry consistency|| ||
|}

== Memory Coherence and Shared Virtual Memory ==

The memory coherence problem in a shared virtual memory system and in multicache systems are different. In a multicache multiprocessor, there are processors sharing a physical memory through their private caches. The relatively small size of a cache and the fast bus connection to the shared memory, enables using a sophisticated coherence protocol for the multicache hardware such that the time delay of conflicting writes to a memory location is small. [5]

In contrast, in a shared virtual memory on a loosely coupled multiprocessor which has no physically shared memory, and having a nontrivial communication cost between processors, conflicts are not likely to be solved with negligible delay, and they resemble much more a “page
fault” in a traditional virtual memory system. Thus, there are two design choices that greatly influence the implementation of a shared virtual memory: the granularity of the memory units (i.e., the “page size”) and the strategy for maintaining coherence.

Memory coherence strategies are classified based on how they deal with '''''page synchronization''''' and '''''page ownership'''''. The algorithms for memory coherence depend on the page fault handlers, their servers and the data structures used. So ''page table'' becomes an important part of these protocols.

Two classes of algorithms for solving the memory coherence problem are centralized manager and distributed manager. In the centralized system, one processor is designated the "monitor". This processor contains a list of information for each page in the cache. The list includes the owner of the page (which processor accessed it last) and all the processors that have copies of the page. When a page is invalidated, then invalidation message is only sent to processors that have copies of the page. This differs from a bus-based system, as the invalidation message is broadcasted to all processors. A drawback to the centralized manager is that there is a bottleneck at the monitor processor. This bottleneck is alleviated by using a distributed manager.

The distributed manager is similar to a centralized manager, but instead of one processor monitoring all pages, a subset of the pages is given to each processor. So, processor 0 would only monitor pages 1 through i, and processor 1 would only monitor pages i+1 through n.

= Synchronization =

Two related concepts dealing with synchronization are '''process synchronization''' and '''data synchronization'''. Process synchronization is concerned with different processes committing to a certain sequence of actions. Data synchronization deals with maintaining data integrity across various copies of a dataset. Process synchronization primitives can be used to implement data synchronization.

[http://en.wikipedia.org/wiki/Mutual_exclusion Mutual exclusion] is the main requirement to be fulfilled in order to synchronize processes, and is needed in both single-processor and multiprocessor systems. There are various approaches to provide mutual exclusion in a system:

* Disabling interrupts
* Locks
* Mutex
* Semaphores
* Barriers
* Test and Set

The next section discusses if there is an alternative to implement mutual exclusion without requiring any hardware support. Peterson's algorithm is one such software solution for guaranteeing mutual exclusion.

== Hardware support ==

Exclusive locking assumes the worst and proceeds only after acquiring all locks such that no other thread can interfere. This is a ''pessimistic'' approach. In contrast, the ''optimistic'' approach proceeds with an update, hoping that it can be completed without any interference. This requires ''collision detection'' during the update. The optimistic approach is thus, more efficient in fine-grained operations.

Special instructions are provided by processors designed for multiprocessor operations in order to manage concurrent access to shared variables. Atomic instructions like [http://en.wikipedia.org/wiki/Test_and_Test-and-set test-and-set], [http://en.wikipedia.org/wiki/Fetch-and-add fetch-and-increment] and [http://en.wikipedia.org/wiki/Swap_%28computer_science%29 swap] were sufficient for early processors to implement mutexes for concurrent objects. Today, every modern processor relies on some form of read-modify-write atomic instruction such as [http://en.wikipedia.org/wiki/Compare-and-swap compare-and-swap], [http://en.wikipedia.org/wiki/Load-link/store-conditional LL/SC] etc. for the same.

=Resources=

[[#1body|1.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_jp#Memory_Consistency_Problem 
[[#2body|2.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_ss#Cache_Coherence 
[[#3body|3.]] http://www.windowsnetworking.com/articles_tutorials/Cache-Coherency.html 
[[#4body|4.]]http://web.sfc.keio.ac.jp/~rdv/keio/sfc/teaching/architecture/architecture-2007/lec08.html

CSC 456 Spring 2012/ch7 MN

2012-03-19T17:26:49Z

Jmmohund2: /* Memory semantics in multiprocessor systems */

==Introduction==
Though the migration from [http://en.wikipedia.org/wiki/Uniprocessor_system uniprocessor system] to [http://en.wikipedia.org/wiki/Multiprocessing multiprocessing] systems is not new, the world of parallel computers is undergoing a continuous change. Parallel computers, which started as high-end super-computing systems for carrying out huge calculations, are now ubiquitous and are present in all mainstream architectures for servers, desktops, and embedded systems. In order to design parallel architectures to meet programmer's needs and expectations more closely, exciting and challenging changes exist. The three main areas which are being considered by scientists today are: [http://en.wikipedia.org/wiki/Cache_coherence cache coherence], memory consistency and [http://en.wikipedia.org/wiki/Synchronization_%28computer_science%29 synchronization].

=Cache Coherence Problem=

[[Image:sharedmem.jpg|thumbnail|right|600px|Shared Memory system with dedicated Cache for each processor[[#4foot|[4]]]]]

In a system with single processor (single core), maintaining cache coherence is simple and easy but in a multiprocessor system, it is much more complicated. Data can be present in any processor's cache and the protocol needs to ensure that the data is same in all caches. If it cannot ensure that all the caches are same, then it needs to flag a cache line indicating that it is not updated.

In the figure shown here, this is a 4 processor shared memory system where each processor has its own cache. Suppose processor P1 reads memory location M1 and stores it in its local cache. Then, if P2 reads same location memory location then M1 gets stored in P2’s cache. Now, if P1 changes value of M1, two copies of same data, residing in different caches will become different. When P2 operates on M1, it uses the stale value of M1 that was stored in its cache. It is the responsibility of cache coherence protocol to prevent this. Hardware support is needed to provide a coherent view of data in multiple caches. This is known as write propagation requirement.

One may think that cache write policy can provide cache coherence, but this is incorrect. Cache write policy only controls how a change in value of cache is propagated to lower level cache or main memory. It is not responsible for propagating changes to other caches.

==Cache Coherence Protocols==

The two basic methods to utilize the inter-core bus to notify other cores when a core changes something in its cache are '''update''' and '''invalidate'''. In the update method, if variable 'x' is modified by core 1, core 1 has to send the updated value of 'x' onto the inter-core bus. Each cache listens to the inter-core bus and if a cache sees a variable on the bus which it has a copy of, it will read the updated value. This ensures that all caches have the most up-to-date value of the variable.[[#3foot|[3]]]

In case of invalidation, an invalidation message is sent onto the inter-core bus when a variable is changed. The other caches will read this invalidation signal and if its core attempts to access that variable, it will result in a cache miss and the variable will be read from main memory.

The update method results in significant amount of traffic on the inter-core bus as the update signal is sent onto the bus every time the variable is updated. The invalidation method only requires that an invalidation signal be sent the first time a variable is altered; this is why the invalidation method is the preferred method.

In order to improve cache coherence performance over the years, several protocols have been proposed.

===MSI===

MSI stands for Modified, Shared, and Invalid, based on the three states that a line of cache can be in. The modified state means that a variable in the cache has been modified and therefore has a different value than that found in main memory; the cache is responsible for writing the variable back to main memory. The shared state means that the variable exists in at least one cache and is not modified; the cache can evict the variable without writing it back to the main memory. The invalid state means that the value of the variable has been modified by another cache and this value is invalid; the cache must read a new value from main memory (or another cache).

A drawback to this protocol occurs when a single processor wants to read blocks and then write to them without another processor sharing that block. After reading the block, a bus transaction places the block into a shared state. The write then occurs and another bus transaction is sent to invalidate the shared copy. This second transaction is useless as no other processors are sharing the block, but the MSI protocol has no way to specify this.

===MESI===

MESI stands for Modified, Exclusive, Shared, and Invalid. The modified and invalid states are the same for this protocol as they are for the MSI protocol. This protocol introduces a new state; the exclusive state. The Exclusive state means that the variable is in only this cache and the value of it matches the value within the main memory. This now means that the shared state indicates that the variable is contained in more than one cache.

===MOSI===

The MOSI protocol is identical to the MSI protocol except that it adds an owned state. The owned state means that the processor "owns" the variable and will provide the current value to other caches when requested (or at least it will decide if it will provide it when asked). This is useful because another cache will not have to read the value from main memory and will receive it from the owning cache much, much, faster.

===MOESI===

The MOESI protocol is a combination of the MESI and MOSI protocols.

=Memory Consistency Problem=

Memory consistency deals with the ordering of memory operations (load and store) to different memory locations. In a single processor system, code will execute correctly if the compiler preserves the order of the access to synchronization variables and other dependent variables. But in shared memory model with multiple processors, two threads could access a shared data element, such as a synchronization variable, the output of the threads would change based on which thread, accesses the shared data element earlier. If this were to occur, then the program output may not be the value expected. Maintaining program order is very important for memory consistency but it comes with performance degradation.

== Memory Consistency Models ==

The memory consistency model of a shared-memory multiprocessor is a formal speciﬁcation of how the memory system appears to the programmer. It eliminates the gap between the behavior expected by the programmer and the actual behavior supported by a system. Effectively, the consistency model places restrictions on the values that can be returned by a read, in a shared-memory program execution.

In a single processor system, in order to maintain memory consistency, it needs to ensure that the compiler preserves the program order when accessing synchronization variables. But in a multiprocessor system, it is required to ensure that accesses of one processor appear to execute in program order to all other processors, at least partially.

=== Memory semantics in Uniprocessor systems ===

Uniprocessor languages use simple sequential semantics for memory operations, which allow the programmer to assume that all memory operations will occur one at a time in the sequential order speciﬁed by the program. Thus, one can expect for the value in a particular location be the same for a read as it was for the last write because of the sequential program execution. It is sufﬁcient to only maintain uniprocessor data and control dependences. The compiler and hardware can freely reorder operations to different locations if the uniprocessor data and control dependences are respected. This enables compiler optimizations such as register allocation, code motion, and loop transformations, and hardware optimizations, such as pipelining, multiple issue, write buffer bypassing and forwarding, and lockup-free caches, all of which lead to overlapping and reordering of memory operations. [4]

=== Memory semantics in multiprocessor systems ===

Programmer's implicit expectations are:

* memory accesses in a processor takes place according to the program order.
* Each memory access is performed atomically.

A strong consistency model attempting uniprocessor-like consistency could cause global bottleneck, costing performance. Thus, '''''weak''''' consistency models are deployed to improve performance. The advanatges of such models are:

* They support out-of-order execution within individual CPUs
* Relaxes latency issues with near-simultaneous accesses by different CPUs

The following are the various consistency models and it is the programmer who must take into account the memory consistency model to create correct software:
{| class="wikitable sortable" border="1"
|+ Sortable Table of Memory Consistency Models
|-
! scope="col" class="unsortable" | Type of Constancy Modle
! scope="col" | Strictness rank, 1-10(lowest-highest)
! scope="col" | Requires Programmer annotation
|-
| [http://en.wikipedia.org/wiki/Linearizability linearizability (also known as strict or atomic consistency)]|| ||
|-
| [http://en.wikipedia.org/wiki/Sequential_consistency sequential consistency]|| 8 ||
|-
| [http://en.wikipedia.org/wiki/Causal_consistency causal consistency]|| 3 ||
|-
|[http://en.wikipedia.org/wiki/Release_consistency release consistency]|| ||
|-
| [http://en.wikipedia.org/wiki/Eventual_consistency eventual consistency ]|| ||
|-
|[http://en.wikipedia.org/wiki/Delta_consistency delta consistency]|| ||
|-
| [http://en.wikipedia.org/wiki/PRAM_consistency PRAM consistency (also known as FIFO consistency)]|| 2 ||
|-
| [http://en.wikipedia.org/wiki/Weak_consistency weak consistency]|| 1 ||
|-
|[http://en.wikipedia.org/wiki/Vector-field_consistency vector-field consistency]|| ||
|-
| fork consistency|| ||
|-
|[http://en.wikipedia.org/wiki/Serializability serializability]|| ||
|-
|one-copy serializability|| ||
|-
|entry consistency|| ||
|}

== Memory Coherence and Shared Virtual Memory ==

The memory coherence problem in a shared virtual memory system and in multicache systems are different. In a multicache multiprocessor, there are processors sharing a physical memory through their private caches. The relatively small size of a cache and the fast bus connection to the shared memory, enables using a sophisticated coherence protocol for the multicache hardware such that the time delay of conflicting writes to a memory location is small. [5]

In contrast, in a shared virtual memory on a loosely coupled multiprocessor which has no physically shared memory, and having a nontrivial communication cost between processors, conflicts are not likely to be solved with negligible delay, and they resemble much more a “page
fault” in a traditional virtual memory system. Thus, there are two design choices that greatly influence the implementation of a shared virtual memory: the granularity of the memory units (i.e., the “page size”) and the strategy for maintaining coherence.

Memory coherence strategies are classified based on how they deal with '''''page synchronization''''' and '''''page ownership'''''. The algorithms for memory coherence depend on the page fault handlers, their servers and the data structures used. So ''page table'' becomes an important part of these protocols.

= Synchronization =

Two related concepts dealing with synchronization are '''process synchronization''' and '''data synchronization'''. Process synchronization is concerned with different processes committing to a certain sequence of actions. Data synchronization deals with maintaining data integrity across various copies of a dataset. Process synchronization primitives can be used to implement data synchronization.

[http://en.wikipedia.org/wiki/Mutual_exclusion Mutual exclusion] is the main requirement to be fulfilled in order to synchronize processes, and is needed in both single-processor and multiprocessor systems. There are various approaches to provide mutual exclusion in a system:

* Disabling interrupts
* Locks
* Mutex
* Semaphores
* Barriers
* Test and Set

The next section discusses if there is an alternative to implement mutual exclusion without requiring any hardware support. Peterson's algorithm is one such software solution for guaranteeing mutual exclusion.

== Hardware support ==

Exclusive locking assumes the worst and proceeds only after acquiring all locks such that no other thread can interfere. This is a ''pessimistic'' approach. In contrast, the ''optimistic'' approach proceeds with an update, hoping that it can be completed without any interference. This requires ''collision detection'' during the update. The optimistic approach is thus, more efficient in fine-grained operations.

Special instructions are provided by processors designed for multiprocessor operations in order to manage concurrent access to shared variables. Atomic instructions like [http://en.wikipedia.org/wiki/Test_and_Test-and-set test-and-set], [http://en.wikipedia.org/wiki/Fetch-and-add fetch-and-increment] and [http://en.wikipedia.org/wiki/Swap_%28computer_science%29 swap] were sufficient for early processors to implement mutexes for concurrent objects. Today, every modern processor relies on some form of read-modify-write atomic instruction such as [http://en.wikipedia.org/wiki/Compare-and-swap compare-and-swap], [http://en.wikipedia.org/wiki/Load-link/store-conditional LL/SC] etc. for the same.

=Resources=

[[#1body|1.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_jp#Memory_Consistency_Problem 
[[#2body|2.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_ss#Cache_Coherence 
[[#3body|3.]] http://www.windowsnetworking.com/articles_tutorials/Cache-Coherency.html 
[[#4body|4.]]http://web.sfc.keio.ac.jp/~rdv/keio/sfc/teaching/architecture/architecture-2007/lec08.html

CSC 456 Spring 2012/ch7 MN

2012-03-19T17:23:13Z

Jmmohund2: /* Memory semantics in multiprocessor systems */

==Introduction==
Though the migration from [http://en.wikipedia.org/wiki/Uniprocessor_system uniprocessor system] to [http://en.wikipedia.org/wiki/Multiprocessing multiprocessing] systems is not new, the world of parallel computers is undergoing a continuous change. Parallel computers, which started as high-end super-computing systems for carrying out huge calculations, are now ubiquitous and are present in all mainstream architectures for servers, desktops, and embedded systems. In order to design parallel architectures to meet programmer's needs and expectations more closely, exciting and challenging changes exist. The three main areas which are being considered by scientists today are: [http://en.wikipedia.org/wiki/Cache_coherence cache coherence], memory consistency and [http://en.wikipedia.org/wiki/Synchronization_%28computer_science%29 synchronization].

=Cache Coherence Problem=

[[Image:sharedmem.jpg|thumbnail|right|600px|Shared Memory system with dedicated Cache for each processor[[#4foot|[4]]]]]

In a system with single processor (single core), maintaining cache coherence is simple and easy but in a multiprocessor system, it is much more complicated. Data can be present in any processor's cache and the protocol needs to ensure that the data is same in all caches. If it cannot ensure that all the caches are same, then it needs to flag a cache line indicating that it is not updated.

In the figure shown here, this is a 4 processor shared memory system where each processor has its own cache. Suppose processor P1 reads memory location M1 and stores it in its local cache. Then, if P2 reads same location memory location then M1 gets stored in P2’s cache. Now, if P1 changes value of M1, two copies of same data, residing in different caches will become different. When P2 operates on M1, it uses the stale value of M1 that was stored in its cache. It is the responsibility of cache coherence protocol to prevent this. Hardware support is needed to provide a coherent view of data in multiple caches. This is known as write propagation requirement.

One may think that cache write policy can provide cache coherence, but this is incorrect. Cache write policy only controls how a change in value of cache is propagated to lower level cache or main memory. It is not responsible for propagating changes to other caches.

==Cache Coherence Protocols==

The two basic methods to utilize the inter-core bus to notify other cores when a core changes something in its cache are '''update''' and '''invalidate'''. In the update method, if variable 'x' is modified by core 1, core 1 has to send the updated value of 'x' onto the inter-core bus. Each cache listens to the inter-core bus and if a cache sees a variable on the bus which it has a copy of, it will read the updated value. This ensures that all caches have the most up-to-date value of the variable.[[#3foot|[3]]]

In case of invalidation, an invalidation message is sent onto the inter-core bus when a variable is changed. The other caches will read this invalidation signal and if its core attempts to access that variable, it will result in a cache miss and the variable will be read from main memory.

The update method results in significant amount of traffic on the inter-core bus as the update signal is sent onto the bus every time the variable is updated. The invalidation method only requires that an invalidation signal be sent the first time a variable is altered; this is why the invalidation method is the preferred method.

In order to improve cache coherence performance over the years, several protocols have been proposed.

===MSI===

MSI stands for Modified, Shared, and Invalid, based on the three states that a line of cache can be in. The modified state means that a variable in the cache has been modified and therefore has a different value than that found in main memory; the cache is responsible for writing the variable back to main memory. The shared state means that the variable exists in at least one cache and is not modified; the cache can evict the variable without writing it back to the main memory. The invalid state means that the value of the variable has been modified by another cache and this value is invalid; the cache must read a new value from main memory (or another cache).

A drawback to this protocol occurs when a single processor wants to read blocks and then write to them without another processor sharing that block. After reading the block, a bus transaction places the block into a shared state. The write then occurs and another bus transaction is sent to invalidate the shared copy. This second transaction is useless as no other processors are sharing the block, but the MSI protocol has no way to specify this.

===MESI===

MESI stands for Modified, Exclusive, Shared, and Invalid. The modified and invalid states are the same for this protocol as they are for the MSI protocol. This protocol introduces a new state; the exclusive state. The Exclusive state means that the variable is in only this cache and the value of it matches the value within the main memory. This now means that the shared state indicates that the variable is contained in more than one cache.

===MOSI===

The MOSI protocol is identical to the MSI protocol except that it adds an owned state. The owned state means that the processor "owns" the variable and will provide the current value to other caches when requested (or at least it will decide if it will provide it when asked). This is useful because another cache will not have to read the value from main memory and will receive it from the owning cache much, much, faster.

===MOESI===

The MOESI protocol is a combination of the MESI and MOSI protocols.

=Memory Consistency Problem=

Memory consistency deals with the ordering of memory operations (load and store) to different memory locations. In a single processor system, code will execute correctly if the compiler preserves the order of the access to synchronization variables and other dependent variables. But in shared memory model with multiple processors, two threads could access a shared data element, such as a synchronization variable, the output of the threads would change based on which thread, accesses the shared data element earlier. If this were to occur, then the program output may not be the value expected. Maintaining program order is very important for memory consistency but it comes with performance degradation.

== Memory Consistency Models ==

The memory consistency model of a shared-memory multiprocessor is a formal speciﬁcation of how the memory system appears to the programmer. It eliminates the gap between the behavior expected by the programmer and the actual behavior supported by a system. Effectively, the consistency model places restrictions on the values that can be returned by a read, in a shared-memory program execution.

In a single processor system, in order to maintain memory consistency, it needs to ensure that the compiler preserves the program order when accessing synchronization variables. But in a multiprocessor system, it is required to ensure that accesses of one processor appear to execute in program order to all other processors, at least partially.

=== Memory semantics in Uniprocessor systems ===

Uniprocessor languages use simple sequential semantics for memory operations, which allow the programmer to assume that all memory operations will occur one at a time in the sequential order speciﬁed by the program. Thus, one can expect for the value in a particular location be the same for a read as it was for the last write because of the sequential program execution. It is sufﬁcient to only maintain uniprocessor data and control dependences. The compiler and hardware can freely reorder operations to different locations if the uniprocessor data and control dependences are respected. This enables compiler optimizations such as register allocation, code motion, and loop transformations, and hardware optimizations, such as pipelining, multiple issue, write buffer bypassing and forwarding, and lockup-free caches, all of which lead to overlapping and reordering of memory operations. [4]

=== Memory semantics in multiprocessor systems ===

Programmer's implicit expectations are:

* memory accesses in a processor takes place according to the program order.
* Each memory access is performed atomically.

A strong consistency model attempting uniprocessor-like consistency could cause global bottleneck, costing performance. Thus, '''''weak''''' consistency models are deployed to improve performance. The advanatges of such models are:

* They support out-of-order execution within individual CPUs
* Relaxes latency issues with near-simultaneous accesses by different CPUs

The following are the various consistency models and it is the programmer who must take into account the memory consistency model to create correct software:
{| class="wikitable sortable" border="1"
|+ Sortable Table of Memory Consistency Models
|-
! scope="col" class="unsortable" | Type of Constancy Modle
! scope="col" | Strictness rank, 1-10(lowest-highest)
! scope="col" | Requires Programmer annotation
|-
| [http://en.wikipedia.org/wiki/Linearizability linearizability (also known as strict or atomic consistency)]|| ||
|-
| [http://en.wikipedia.org/wiki/Sequential_consistency sequential consistency]|| 2 ||
|-
| [http://en.wikipedia.org/wiki/Causal_consistency causal consistency]|| 1 ||
|-
|[http://en.wikipedia.org/wiki/Release_consistency release consistency]|| ||
|-
| [http://en.wikipedia.org/wiki/Eventual_consistency eventual consistency ]|| ||
|-
|[http://en.wikipedia.org/wiki/Delta_consistency delta consistency]|| ||
|-
| [http://en.wikipedia.org/wiki/PRAM_consistency PRAM consistency (also known as FIFO consistency)]|| ||
|-
| [http://en.wikipedia.org/wiki/Weak_consistency weak consistency]|| ||
|-
|[http://en.wikipedia.org/wiki/Vector-field_consistency vector-field consistency]|| ||
|-
| fork consistency|| ||
|-
|[http://en.wikipedia.org/wiki/Serializability serializability]|| ||
|-
|one-copy serializability|| ||
|-
|entry consistency|| ||
|}

== Memory Coherence and Shared Virtual Memory ==

The memory coherence problem in a shared virtual memory system and in multicache systems are different. In a multicache multiprocessor, there are processors sharing a physical memory through their private caches. The relatively small size of a cache and the fast bus connection to the shared memory, enables using a sophisticated coherence protocol for the multicache hardware such that the time delay of conflicting writes to a memory location is small. [5]

In contrast, in a shared virtual memory on a loosely coupled multiprocessor which has no physically shared memory, and having a nontrivial communication cost between processors, conflicts are not likely to be solved with negligible delay, and they resemble much more a “page
fault” in a traditional virtual memory system. Thus, there are two design choices that greatly influence the implementation of a shared virtual memory: the granularity of the memory units (i.e., the “page size”) and the strategy for maintaining coherence.

Memory coherence strategies are classified based on how they deal with '''''page synchronization''''' and '''''page ownership'''''. The algorithms for memory coherence depend on the page fault handlers, their servers and the data structures used. So ''page table'' becomes an important part of these protocols.

= Synchronization =

Two related concepts dealing with synchronization are '''process synchronization''' and '''data synchronization'''. Process synchronization is concerned with different processes committing to a certain sequence of actions. Data synchronization deals with maintaining data integrity across various copies of a dataset. Process synchronization primitives can be used to implement data synchronization.

[http://en.wikipedia.org/wiki/Mutual_exclusion Mutual exclusion] is the main requirement to be fulfilled in order to synchronize processes, and is needed in both single-processor and multiprocessor systems. There are various approaches to provide mutual exclusion in a system:

* Disabling interrupts
* Locks
* Mutex
* Semaphores
* Barriers
* Test and Set

The next section discusses if there is an alternative to implement mutual exclusion without requiring any hardware support. Peterson's algorithm is one such software solution for guaranteeing mutual exclusion.

== Hardware support ==

Exclusive locking assumes the worst and proceeds only after acquiring all locks such that no other thread can interfere. This is a ''pessimistic'' approach. In contrast, the ''optimistic'' approach proceeds with an update, hoping that it can be completed without any interference. This requires ''collision detection'' during the update. The optimistic approach is thus, more efficient in fine-grained operations.

Special instructions are provided by processors designed for multiprocessor operations in order to manage concurrent access to shared variables. Atomic instructions like [http://en.wikipedia.org/wiki/Test_and_Test-and-set test-and-set], [http://en.wikipedia.org/wiki/Fetch-and-add fetch-and-increment] and [http://en.wikipedia.org/wiki/Swap_%28computer_science%29 swap] were sufficient for early processors to implement mutexes for concurrent objects. Today, every modern processor relies on some form of read-modify-write atomic instruction such as [http://en.wikipedia.org/wiki/Compare-and-swap compare-and-swap], [http://en.wikipedia.org/wiki/Load-link/store-conditional LL/SC] etc. for the same.

=Resources=

[[#1body|1.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_jp#Memory_Consistency_Problem 
[[#2body|2.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_ss#Cache_Coherence 
[[#3body|3.]] http://www.windowsnetworking.com/articles_tutorials/Cache-Coherency.html 
[[#4body|4.]]http://web.sfc.keio.ac.jp/~rdv/keio/sfc/teaching/architecture/architecture-2007/lec08.html

CSC 456 Spring 2012/ch7 MN

2012-03-19T17:10:27Z

Jmmohund2: /* Memory semantics in multiprocessor systems */

==Introduction==
Though the migration from [http://en.wikipedia.org/wiki/Uniprocessor_system uniprocessor system] to [http://en.wikipedia.org/wiki/Multiprocessing multiprocessing] systems is not new, the world of parallel computers is undergoing a continuous change. Parallel computers, which started as high-end super-computing systems for carrying out huge calculations, are now ubiquitous and are present in all mainstream architectures for servers, desktops, and embedded systems. In order to design parallel architectures to meet programmer's needs and expectations more closely, exciting and challenging changes exist. The three main areas which are being considered by scientists today are: [http://en.wikipedia.org/wiki/Cache_coherence cache coherence], memory consistency and [http://en.wikipedia.org/wiki/Synchronization_%28computer_science%29 synchronization].

=Cache Coherence Problem=

[[Image:sharedmem.jpg|thumbnail|right|600px|Shared Memory system with dedicated Cache for each processor[[#4foot|[4]]]]]

In a system with single processor (single core), maintaining cache coherence is simple and easy but in a multiprocessor system, it is much more complicated. Data can be present in any processor's cache and the protocol needs to ensure that the data is same in all caches. If it cannot ensure that all the caches are same, then it needs to flag a cache line indicating that it is not updated.

In the figure shown here, this is a 4 processor shared memory system where each processor has its own cache. Suppose processor P1 reads memory location M1 and stores it in its local cache. Then, if P2 reads same location memory location then M1 gets stored in P2’s cache. Now, if P1 changes value of M1, two copies of same data, residing in different caches will become different. When P2 operates on M1, it uses the stale value of M1 that was stored in its cache. It is the responsibility of cache coherence protocol to prevent this. Hardware support is needed to provide a coherent view of data in multiple caches. This is known as write propagation requirement.

One may think that cache write policy can provide cache coherence, but this is incorrect. Cache write policy only controls how a change in value of cache is propagated to lower level cache or main memory. It is not responsible for propagating changes to other caches.

==Cache Coherence Protocols==

The two basic methods to utilize the inter-core bus to notify other cores when a core changes something in its cache are '''update''' and '''invalidate'''. In the update method, if variable 'x' is modified by core 1, core 1 has to send the updated value of 'x' onto the inter-core bus. Each cache listens to the inter-core bus and if a cache sees a variable on the bus which it has a copy of, it will read the updated value. This ensures that all caches have the most up-to-date value of the variable.[[#3foot|[3]]]

In case of invalidation, an invalidation message is sent onto the inter-core bus when a variable is changed. The other caches will read this invalidation signal and if its core attempts to access that variable, it will result in a cache miss and the variable will be read from main memory.

The update method results in significant amount of traffic on the inter-core bus as the update signal is sent onto the bus every time the variable is updated. The invalidation method only requires that an invalidation signal be sent the first time a variable is altered; this is why the invalidation method is the preferred method.

In order to improve cache coherence performance over the years, several protocols have been proposed.

===MSI===

MSI stands for Modified, Shared, and Invalid, based on the three states that a line of cache can be in. The modified state means that a variable in the cache has been modified and therefore has a different value than that found in main memory; the cache is responsible for writing the variable back to main memory. The shared state means that the variable exists in at least one cache and is not modified; the cache can evict the variable without writing it back to the main memory. The invalid state means that the value of the variable has been modified by another cache and this value is invalid; the cache must read a new value from main memory (or another cache).

A drawback to this protocol occurs when a single processor wants to read blocks and then write to them without another processor sharing that block. After reading the block, a bus transaction places the block into a shared state. The write then occurs and another bus transaction is sent to invalidate the shared copy. This second transaction is useless as no other processors are sharing the block, but the MSI protocol has no way to specify this.

===MESI===

MESI stands for Modified, Exclusive, Shared, and Invalid. The modified and invalid states are the same for this protocol as they are for the MSI protocol. This protocol introduces a new state; the exclusive state. The Exclusive state means that the variable is in only this cache and the value of it matches the value within the main memory. This now means that the shared state indicates that the variable is contained in more than one cache.

===MOSI===

The MOSI protocol is identical to the MSI protocol except that it adds an owned state. The owned state means that the processor "owns" the variable and will provide the current value to other caches when requested (or at least it will decide if it will provide it when asked). This is useful because another cache will not have to read the value from main memory and will receive it from the owning cache much, much, faster.

===MOESI===

The MOESI protocol is a combination of the MESI and MOSI protocols.

=Memory Consistency Problem=

Memory consistency deals with the ordering of memory operations (load and store) to different memory locations. In a single processor system, code will execute correctly if the compiler preserves the order of the access to synchronization variables and other dependent variables. But in shared memory model with multiple processors, two threads could access a shared data element, such as a synchronization variable, the output of the threads would change based on which thread, accesses the shared data element earlier. If this were to occur, then the program output may not be the value expected. Maintaining program order is very important for memory consistency but it comes with performance degradation.

== Memory Consistency Models ==

The memory consistency model of a shared-memory multiprocessor is a formal speciﬁcation of how the memory system appears to the programmer. It eliminates the gap between the behavior expected by the programmer and the actual behavior supported by a system. Effectively, the consistency model places restrictions on the values that can be returned by a read, in a shared-memory program execution.

In a single processor system, in order to maintain memory consistency, it needs to ensure that the compiler preserves the program order when accessing synchronization variables. But in a multiprocessor system, it is required to ensure that accesses of one processor appear to execute in program order to all other processors, at least partially.

=== Memory semantics in Uniprocessor systems ===

Uniprocessor languages use simple sequential semantics for memory operations, which allow the programmer to assume that all memory operations will occur one at a time in the sequential order speciﬁed by the program. Thus, one can expect for the value in a particular location be the same for a read as it was for the last write because of the sequential program execution. It is sufﬁcient to only maintain uniprocessor data and control dependences. The compiler and hardware can freely reorder operations to different locations if the uniprocessor data and control dependences are respected. This enables compiler optimizations such as register allocation, code motion, and loop transformations, and hardware optimizations, such as pipelining, multiple issue, write buffer bypassing and forwarding, and lockup-free caches, all of which lead to overlapping and reordering of memory operations. [4]

=== Memory semantics in multiprocessor systems ===

Programmer's implicit expectations are:

* memory accesses in a processor takes place according to the program order.
* Each memory access is performed atomically.

A strong consistency model attempting uniprocessor-like consistency could cause global bottleneck, costing performance. Thus, '''''weak''''' consistency models are deployed to improve performance. The advanatges of such models are:

* They support out-of-order execution within individual CPUs
* Relaxes latency issues with near-simultaneous accesses by different CPUs

The following are the various consistency models and it is the programmer who must take into account the memory consistency model to create correct software:
{| class="wikitable sortable" border="1"
|+ Sortable table
|-
! scope="col" class="unsortable" | Type of Constancy Modle
! scope="col" | Strictness rank, 1-10(lowest-highest)
! scope="col" | Requires Programmer annotation
|-
| [http://en.wikipedia.org/wiki/Linearizability linearizability (also known as strict or atomic consistency)]||||
|-
| [http://en.wikipedia.org/wiki/Sequential_consistency sequential consistency]||||
|-
| [http://en.wikipedia.org/wiki/Causal_consistency causal consistency]||||
|-
|[http://en.wikipedia.org/wiki/Release_consistency release consistency]||||
|-
| [http://en.wikipedia.org/wiki/Eventual_consistency eventual consistency ]||||
|-
|[http://en.wikipedia.org/wiki/Delta_consistency delta consistency]||||
|-
| [http://en.wikipedia.org/wiki/PRAM_consistency PRAM consistency (also known as FIFO consistency)]||||
|-
| [http://en.wikipedia.org/wiki/Weak_consistency weak consistency]||||
|-
|[http://en.wikipedia.org/wiki/Vector-field_consistency vector-field consistency]||||
|-
| fork consistency||||
|-
|[http://en.wikipedia.org/wiki/Serializability serializability]||||
|-
|one-copy serializability||||
|-
|entry consistency||||
|}

== Memory Coherence and Shared Virtual Memory ==

The memory coherence problem in a shared virtual memory system and in multicache systems are different. In a multicache multiprocessor, there are processors sharing a physical memory through their private caches. The relatively small size of a cache and the fast bus connection to the shared memory, enables using a sophisticated coherence protocol for the multicache hardware such that the time delay of conflicting writes to a memory location is small. [5]

In contrast, in a shared virtual memory on a loosely coupled multiprocessor which has no physically shared memory, and having a nontrivial communication cost between processors, conflicts are not likely to be solved with negligible delay, and they resemble much more a “page
fault” in a traditional virtual memory system. Thus, there are two design choices that greatly influence the implementation of a shared virtual memory: the granularity of the memory units (i.e., the “page size”) and the strategy for maintaining coherence.

Memory coherence strategies are classified based on how they deal with '''''page synchronization''''' and '''''page ownership'''''. The algorithms for memory coherence depend on the page fault handlers, their servers and the data structures used. So ''page table'' becomes an important part of these protocols.

= Synchronization =

Two related concepts dealing with synchronization are '''process synchronization''' and '''data synchronization'''. Process synchronization is concerned with different processes committing to a certain sequence of actions. Data synchronization deals with maintaining data integrity across various copies of a dataset. Process synchronization primitives can be used to implement data synchronization.

[http://en.wikipedia.org/wiki/Mutual_exclusion Mutual exclusion] is the main requirement to be fulfilled in order to synchronize processes, and is needed in both single-processor and multiprocessor systems. There are various approaches to provide mutual exclusion in a system:

* Disabling interrupts
* Locks
* Mutex
* Semaphores
* Barriers
* Test and Set

The next section discusses if there is an alternative to implement mutual exclusion without requiring any hardware support. Peterson's algorithm is one such software solution for guaranteeing mutual exclusion.

== Hardware support ==

Exclusive locking assumes the worst and proceeds only after acquiring all locks such that no other thread can interfere. This is a ''pessimistic'' approach. In contrast, the ''optimistic'' approach proceeds with an update, hoping that it can be completed without any interference. This requires ''collision detection'' during the update. The optimistic approach is thus, more efficient in fine-grained operations.

Special instructions are provided by processors designed for multiprocessor operations in order to manage concurrent access to shared variables. Atomic instructions like [http://en.wikipedia.org/wiki/Test_and_Test-and-set test-and-set], [http://en.wikipedia.org/wiki/Fetch-and-add fetch-and-increment] and [http://en.wikipedia.org/wiki/Swap_%28computer_science%29 swap] were sufficient for early processors to implement mutexes for concurrent objects. Today, every modern processor relies on some form of read-modify-write atomic instruction such as [http://en.wikipedia.org/wiki/Compare-and-swap compare-and-swap], [http://en.wikipedia.org/wiki/Load-link/store-conditional LL/SC] etc. for the same.

=Resources=

[[#1body|1.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_jp#Memory_Consistency_Problem 
[[#2body|2.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_ss#Cache_Coherence 
[[#3body|3.]] http://www.windowsnetworking.com/articles_tutorials/Cache-Coherency.html 
[[#4body|4.]]http://web.sfc.keio.ac.jp/~rdv/keio/sfc/teaching/architecture/architecture-2007/lec08.html

CSC 456 Spring 2012/ch7 MN

2012-03-14T17:16:25Z

Jmmohund2: /* ADD INFO ABOUT EACH OF THESE TYPES OF MEM CONSIS MODELS */

==Introduction==
Though the migration from [http://en.wikipedia.org/wiki/Uniprocessor_system uniprocessor system] to [http://en.wikipedia.org/wiki/Multiprocessing multiprocessing] systems is not new, the world of parallel computers is undergoing a continuous change. Parallel computers, which started as high-end super-computing systems for carrying out huge calculations, are now ubiquitous and are present in all mainstream architectures for servers, desktops, and embedded systems. In order to design parallel architectures to meet programmer's needs and expectations more closely, exciting and challenging changes exist. The three main areas which are being considered by scientists today are: [http://en.wikipedia.org/wiki/Cache_coherence cache coherence], memory consistency and [http://en.wikipedia.org/wiki/Synchronization_%28computer_science%29 synchronization].

=Cache Coherence Problem=

[[Image:sharedmem.jpg|thumbnail|right|600px|Shared Memory system with dedicated Cache for each processor[[#4foot|[4]]]]]

In a system with single processor (single core), maintaining cache coherency is simple and easy but in a multiprocessor system, it is much more complicated. Data can be present in any processor's cache and the protocol needs to ensure that the data is same in all caches. If it cannot ensure that all the caches are same, then it needs to flag a cache line indicating that it is not updated.

In the figure shown here, this is a 4 processor shared memory system where each processor has its own cache. Supposed processor P1 reads memory location M1 and stores it in its local cache. Then, if P2 reads same location memory location then M1 gets stored in P2’s cache. Now, if P1 changes value of M1, two copies of same data, residing in different caches will become different. When P2 operates on M1, it uses the stale value of M1 that was stored in its cache. It is responsibility of Cache Coherence Protocol to prevent this. Hardware support is needed to provide a coherent view of data in multiple caches. This is known as write propagation requirement.

One may think that cache write policy can provide cache coherence, but this is incorrect. Cache write policy only controls how a change in value of cache is propagated to lower level cache or main memory. It is not responsible for propagating changes to other caches.

==Cache Coherence Protocols==

The two basic methods to utilize the inter-core bus to notify other cores when a core changes something in its cache are '''update''' and '''invalidate'''. In the update method, if variable 'x' is modified by core 1, core 1 has to send the updated value of 'x' onto the inter-core bus. Each cache listens to the inter-core bus and if a cache sees a variable on the bus which it has a copy of, it will read the updated value. This ensures that all caches have the most up-to-date value of the variable.[[#3foot|[3]]]

In case of invalidation, an invalidation message is sent onto the inter-core bus when a variable is changed. The other caches will read this invalidation signal and if its core attempts to access that variable, it will result in a cache miss and the variable will be read from main memory.

The update method results in significant amount of traffic on the inter-core bus as the update signal is sent onto the bus every time the variable is updated. The invalidation method only requires that an invalidation signal be sent the first time a variable is altered; this is why the invalidation method is the preferred method.

In order to improve cache coherency performance over the years, several protocols have been proposed.

===MSI===

MSI stands for Modified, Shared, and Invalid, based on the three states that a line of cache can be in. The Modified state means that a variable in the cache has been modified and therefore has a different value than that found in main memory; the cache is responsible for writing the variable back to main memory. The Shared state means that the variable exists in at least one cache and is not modified; the cache can evict the variable without writing it back to the main memory. The Invalid state means that the value of the variable has been modified by another cache and this value is invalid; the cache must read a new value from main memory (or another cache).

A drawback to this protocol occurs when a single processor wants to read blocks and then write to them without another processor sharing that block. After reading the block, a bus transaction places the block into a shared state. The write then occurs and another bus transaction is sent to invalidate the shared copy. This second transaction is useless as no other processors are sharing the block, but the MSI protocol has no way to specify this.

===MESI===

MESI stands for Modified, Exclusive, Shared, and Invalid. The Modified and Invalid states are the same for this protocol as they are for the MSI protocol. This protocol introduces a new state; the Exclusive state. The Exclusive state means that the variable is in only this cache and the value of it matches the value within the main memory. This now means that the Shared state indicates that the variable is contained in more than one cache.

===MOSI===

The MOSI protocol is identical to the MSI protocol except that it adds an Owned state. The Owned state means that the processor "Owns" the variable and will provide the current value to other caches when requested (or at least it will decide if it will provide it when asked). This is useful because another cache will not have to read the value from main memory and will receive it from the Owning cache much, much, faster.

===MOESI===

The MOESI protocol is a combination of the MESI and MOSI protocols.

=Memory Consistency Problem=

Memory consistency deals with the ordering of memory operations (load and store) to different memory locations. In a single processor system, code will execute correctly if the compiler preservers the order of the access to synchronization variables and other dependent variables. But in shared memory model with multiple processors, two threads could access a shared data element, such as a synchronization variable, the output of the threads would change based on which thread, accesses the shared data element earlier. If this were to occur, then the program output may not be the value expected. Maintaining program order is very important for memory consistency but it comes with performance degradation.

== Memory Consistency Models ==

The memory consistency model of a shared-memory multiprocessor is a formal speciﬁcation of how the memory system appears to the programmer. It eliminates the gap between the behavior expected by the programmer and the actual behavior supported by a system. Effectively, the consistency model places restrictions on the values that can be returned by a read, in a shared-memory program execution.

In a single processor system, in order to maintain memory consistency, it needs to ensure that the compiler preserves the program order when accessing synchronization variables. But in a multiprocessor system, it is required to ensure that accesses of one processor appear to execute in program order to all other processors, atleast partially.

=== Memory semantics in Uniprocessor systems ===

Uniprocessor languages use simple sequential semantics for memory operations, which allow the programmer to assume that all memory operations will occur one at a time in the sequential order speciﬁed by the program. Thus, one can expect for the value in a particular location be the same for a read as it was for the last write because of the sequential program execution. It is sufﬁcient to only maintain uniprocessor data and control dependences. The compiler and hardware can freely reorder operations to different locations if the uniprocessor data and control dependences are respected. This enables compiler optimizations such as register allocation, code motion, and loop transformations, and hardware optimizations, such as pipelining, multiple issue, write buffer bypassing and forwarding, and lockup-free caches, all of which lead to overlapping and reordering of memory operations. [4]

=== Memory semantics in multiprocessor systems ===

Programmer's implicit expectations are:

* memory accesses in a processor takes place according to the program order.
* Each memory access is performed atomically.

A strong consistency model attempting uniprocessor-like consistency could cause global bottleneck, costing performance. Thus, '''''weak''''' consistency models are deployed to improve performance. The advanatges of such models are:

* They support out-of-order execution within individual CPUs
* Relaxes latency issues with near-simultaneous accesses by different CPUs

The following are the various consistency models and it is the programmer who must take into account the memory consistency model to create correct software:

* [http://en.wikipedia.org/wiki/Linearizability linearizability (also known as strict or atomic consistency)]
* [http://en.wikipedia.org/wiki/Sequential_consistency sequential consistency]
* [http://en.wikipedia.org/wiki/Causal_consistency causal consistency]
* [http://en.wikipedia.org/wiki/Release_consistency release consistency]
* [http://en.wikipedia.org/wiki/Eventual_consistency eventual consistency ]
* [http://en.wikipedia.org/wiki/Delta_consistency delta consistency]
* [http://en.wikipedia.org/wiki/PRAM_consistency PRAM consistency (also known as FIFO consistency)]
* [http://en.wikipedia.org/wiki/Weak_consistency weak consistency]
* [http://en.wikipedia.org/wiki/Vector-field_consistency vector-field consistency]
* fork consistency
* [http://en.wikipedia.org/wiki/Serializability serializability]
* one-copy serializability
* entry consistency

== Memory Coherence and Shared Virtual Memory ==

The memory coherence problem in a shared virtual memory system and in multicache systems are different. In a multicache multiprocessor, there are processors sharing a physical memory through their private caches. The relatively small size of a cache and the fast bus connection to the shared memory, enables using a sophisticated coherence protocol for the multicache hardware such that the time delay of conflicting writes to a memory location is small. [5]

In contrast, in a shared virtual memory on a loosely coupled multiprocessor which has no physically shared memory, and having a nontrivial communication cost between processors, conflicts are not likely to be solved with negligible delay, and they resemble much more a “page
fault” in a traditional virtual memory system. Thus, there are two design choices that greatly influence the implementation of a shared virtual memory: the granularity of the memory units (i.e., the “page size”)and the strategy for maintaining coherence.

Memory coherence strategies are classified based on how they deal with '''''page synchronization''''' and '''''page ownership'''''. The algorithms for memory coherence depend on the page fault handlers, their servers and the data structures used. So ''page table'' becomes an important part of these protocols.

= Synchronization =

'''Process synchronization''', which is the mainly concerned with different processes committing to a a certain sequence of actions, and '''data synchronization''', which deals with maintaining data integrity across various copies of a dataset, are two two distinct but related concepts. Process synchronization primitives can be used to implement data synchronization.

[http://en.wikipedia.org/wiki/Mutual_exclusion Mutual exclusion] is the main requirement to be fulfilled in order to synchronize processes, and is needed in both single-processor and multiprocessor systems. There are various approaches to provide mutual exclusion in a system:

* Disabling interrupts
* Locks
* Mutex
* Semaphores
* Barriers
* Test and Set

The next section discusses if there is an alternative to implement mutual exclusion without requiring any hardware support. Peterson's algorithm is one such software solution for guaranteeing mutual exclusion.

== Peterson's algorithm ==

This algorithm utilizes the simple idea of combining per-thread '''flags''' to indicate the intent to enter the lock and the '''turn''' variable to determine which thread should enter in the rare case that both wish to enter at the same time. The algorithm is as shown below: [3]

<pre>
void init() {
flag[0] = flag[1] = 0; // 1 -> thread wants to acquire lock (intent)
turn = 0; // whose turn is it? (thread 0 or thread 1?)
}

void lock() {
flag[self] = 1;
turn = 1 - self; // be generous: make it the other thread’s turn
while ((flag[1-self] == 1) && (turn == 1 - self))
; // spin-wait while other thread has intent
// AND it is other thread’s turn
}

void unlock() {
flag[self] = 0; // simply undo your intent
}
</pre>

The advantages of this approach are:

* It does not assume much about the underlying hardware. It relies on the fact that load and store instructions are atomic( even across processors) and they execute in an order.
* It does not require special instructions.

Though Peterson's algorithm has the above advantages, there are a few problems that render it impractical for use:

* Spin-waiting is possible which causes a thread to spend lot of its CPU time waiting for another thread to release the lock.
* Algorithm might not work well with out-of-order execution of instructions supported by most of the modern processors.
* The algorithm also lacks scalability.

Thus, only the software solutions cannot work well. Sufficient amount of hardware and OS support is needed to get the locking work properly.

== Hardware support ==

Exclusive locking assumes the worst and proceeds only after acquiring all locks such that no other thread can interfere. This is a ''pessimistic'' approach. In contrast, the ''optimistic'' approach proceeds with an update, hoping that it can be completed without any interference. This requires ''collision detection'' during the update. The optimistic approach is thus, more efficient in fine-grained operations.

Special instructions are provided by processors designed for multiprocessor operations in order to manage concurrent access to shared variables. Atomic instructions like [http://en.wikipedia.org/wiki/Test_and_Test-and-set test-and-set], [http://en.wikipedia.org/wiki/Fetch-and-add fetch-and-increment] and [http://en.wikipedia.org/wiki/Swap_%28computer_science%29 swap] were sufficient for early processors to implement mutexes for concurrent objects. Today, every modern processor relies on some form of read-modify-write atomic instruction such as [http://en.wikipedia.org/wiki/Compare-and-swap compare-and-swap], [http://en.wikipedia.org/wiki/Load-link/store-conditional LL/SC] etc. for the same.

=Resources=

[[#1body|1.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_jp#Memory_Consistency_Problem 
[[#2body|2.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_ss#Cache_Coherence 
[[#3body|3.]] http://www.windowsnetworking.com/articles_tutorials/Cache-Coherency.html 
[[#4body|4.]]http://web.sfc.keio.ac.jp/~rdv/keio/sfc/teaching/architecture/architecture-2007/lec08.html

CSC 456 Spring 2012/ch7 MN

2012-03-14T17:15:50Z

Jmmohund2: /* ADD INFO ABOUT EACH OF THESE TYPES OF MEM CONSIS MODELS */

==Introduction==
Though the migration from [http://en.wikipedia.org/wiki/Uniprocessor_system uniprocessor system] to [http://en.wikipedia.org/wiki/Multiprocessing multiprocessing] systems is not new, the world of parallel computers is undergoing a continuous change. Parallel computers, which started as high-end super-computing systems for carrying out huge calculations, are now ubiquitous and are present in all mainstream architectures for servers, desktops, and embedded systems. In order to design parallel architectures to meet programmer's needs and expectations more closely, exciting and challenging changes exist. The three main areas which are being considered by scientists today are: [http://en.wikipedia.org/wiki/Cache_coherence cache coherence], memory consistency and [http://en.wikipedia.org/wiki/Synchronization_%28computer_science%29 synchronization].

=Cache Coherence Problem=

[[Image:sharedmem.jpg|thumbnail|right|600px|Shared Memory system with dedicated Cache for each processor[[#4foot|[4]]]]]

In a system with single processor (single core), maintaining cache coherency is simple and easy but in a multiprocessor system, it is much more complicated. Data can be present in any processor's cache and the protocol needs to ensure that the data is same in all caches. If it cannot ensure that all the caches are same, then it needs to flag a cache line indicating that it is not updated.

In the figure shown here, this is a 4 processor shared memory system where each processor has its own cache. Supposed processor P1 reads memory location M1 and stores it in its local cache. Then, if P2 reads same location memory location then M1 gets stored in P2’s cache. Now, if P1 changes value of M1, two copies of same data, residing in different caches will become different. When P2 operates on M1, it uses the stale value of M1 that was stored in its cache. It is responsibility of Cache Coherence Protocol to prevent this. Hardware support is needed to provide a coherent view of data in multiple caches. This is known as write propagation requirement.

One may think that cache write policy can provide cache coherence, but this is incorrect. Cache write policy only controls how a change in value of cache is propagated to lower level cache or main memory. It is not responsible for propagating changes to other caches.

==Cache Coherence Protocols==

The two basic methods to utilize the inter-core bus to notify other cores when a core changes something in its cache are '''update''' and '''invalidate'''. In the update method, if variable 'x' is modified by core 1, core 1 has to send the updated value of 'x' onto the inter-core bus. Each cache listens to the inter-core bus and if a cache sees a variable on the bus which it has a copy of, it will read the updated value. This ensures that all caches have the most up-to-date value of the variable.[[#3foot|[3]]]

In case of invalidation, an invalidation message is sent onto the inter-core bus when a variable is changed. The other caches will read this invalidation signal and if its core attempts to access that variable, it will result in a cache miss and the variable will be read from main memory.

The update method results in significant amount of traffic on the inter-core bus as the update signal is sent onto the bus every time the variable is updated. The invalidation method only requires that an invalidation signal be sent the first time a variable is altered; this is why the invalidation method is the preferred method.

In order to improve cache coherency performance over the years, several protocols have been proposed.

===MSI===

MSI stands for Modified, Shared, and Invalid, based on the three states that a line of cache can be in. The Modified state means that a variable in the cache has been modified and therefore has a different value than that found in main memory; the cache is responsible for writing the variable back to main memory. The Shared state means that the variable exists in at least one cache and is not modified; the cache can evict the variable without writing it back to the main memory. The Invalid state means that the value of the variable has been modified by another cache and this value is invalid; the cache must read a new value from main memory (or another cache).

A drawback to this protocol occurs when a single processor wants to read blocks and then write to them without another processor sharing that block. After reading the block, a bus transaction places the block into a shared state. The write then occurs and another bus transaction is sent to invalidate the shared copy. This second transaction is useless as no other processors are sharing the block, but the MSI protocol has no way to specify this.

===MESI===

MESI stands for Modified, Exclusive, Shared, and Invalid. The Modified and Invalid states are the same for this protocol as they are for the MSI protocol. This protocol introduces a new state; the Exclusive state. The Exclusive state means that the variable is in only this cache and the value of it matches the value within the main memory. This now means that the Shared state indicates that the variable is contained in more than one cache.

===MOSI===

The MOSI protocol is identical to the MSI protocol except that it adds an Owned state. The Owned state means that the processor "Owns" the variable and will provide the current value to other caches when requested (or at least it will decide if it will provide it when asked). This is useful because another cache will not have to read the value from main memory and will receive it from the Owning cache much, much, faster.

===MOESI===

The MOESI protocol is a combination of the MESI and MOSI protocols.

=Memory Consistency Problem=

Memory consistency deals with the ordering of memory operations (load and store) to different memory locations. In a single processor system, code will execute correctly if the compiler preservers the order of the access to synchronization variables and other dependent variables. But in shared memory model with multiple processors, two threads could access a shared data element, such as a synchronization variable, the output of the threads would change based on which thread, accesses the shared data element earlier. If this were to occur, then the program output may not be the value expected. Maintaining program order is very important for memory consistency but it comes with performance degradation.

== Memory Consistency Models ==

The memory consistency model of a shared-memory multiprocessor is a formal speciﬁcation of how the memory system appears to the programmer. It eliminates the gap between the behavior expected by the programmer and the actual behavior supported by a system. Effectively, the consistency model places restrictions on the values that can be returned by a read, in a shared-memory program execution.

In a single processor system, in order to maintain memory consistency, it needs to ensure that the compiler preserves the program order when accessing synchronization variables. But in a multiprocessor system, it is required to ensure that accesses of one processor appear to execute in program order to all other processors, atleast partially.

=== Memory semantics in Uniprocessor systems ===

Uniprocessor languages use simple sequential semantics for memory operations, which allow the programmer to assume that all memory operations will occur one at a time in the sequential order speciﬁed by the program. Thus, one can expect for the value in a particular location be the same for a read as it was for the last write because of the sequential program execution. It is sufﬁcient to only maintain uniprocessor data and control dependences. The compiler and hardware can freely reorder operations to different locations if the uniprocessor data and control dependences are respected. This enables compiler optimizations such as register allocation, code motion, and loop transformations, and hardware optimizations, such as pipelining, multiple issue, write buffer bypassing and forwarding, and lockup-free caches, all of which lead to overlapping and reordering of memory operations. [4]

=== Memory semantics in multiprocessor systems ===

Programmer's implicit expectations are:

* memory accesses in a processor takes place according to the program order.
* Each memory access is performed atomically.

A strong consistency model attempting uniprocessor-like consistency could cause global bottleneck, costing performance. Thus, '''''weak''''' consistency models are deployed to improve performance. The advanatges of such models are:

* They support out-of-order execution within individual CPUs
* Relaxes latency issues with near-simultaneous accesses by different CPUs

The following are the various consistency models and it is the programmer who must take into account the memory consistency model to create correct software:
=ADD INFO ABOUT EACH OF THESE TYPES OF MEM CONSIS MODELS=
* [http://en.wikipedia.org/wiki/Linearizability linearizability (also known as strict or atomic consistency)]
* [http://en.wikipedia.org/wiki/Sequential_consistency sequential consistency]
* [http://en.wikipedia.org/wiki/Causal_consistency causal consistency]
* [http://en.wikipedia.org/wiki/Release_consistency release consistency]
* [http://en.wikipedia.org/wiki/Eventual_consistency eventual consistency ]
* [http://en.wikipedia.org/wiki/Delta_consistency delta consistency]
* [http://en.wikipedia.org/wiki/PRAM_consistency PRAM consistency (also known as FIFO consistency)]
* [http://en.wikipedia.org/wiki/Weak_consistency weak consistency]
* [http://en.wikipedia.org/wiki/Vector-field_consistency vector-field consistency]
* fork consistency
* [http://en.wikipedia.org/wiki/Serializability serializability]
* one-copy serializability
* entry consistency

== Memory Coherence and Shared Virtual Memory ==

The memory coherence problem in a shared virtual memory system and in multicache systems are different. In a multicache multiprocessor, there are processors sharing a physical memory through their private caches. The relatively small size of a cache and the fast bus connection to the shared memory, enables using a sophisticated coherence protocol for the multicache hardware such that the time delay of conflicting writes to a memory location is small. [5]

In contrast, in a shared virtual memory on a loosely coupled multiprocessor which has no physically shared memory, and having a nontrivial communication cost between processors, conflicts are not likely to be solved with negligible delay, and they resemble much more a “page
fault” in a traditional virtual memory system. Thus, there are two design choices that greatly influence the implementation of a shared virtual memory: the granularity of the memory units (i.e., the “page size”)and the strategy for maintaining coherence.

Memory coherence strategies are classified based on how they deal with '''''page synchronization''''' and '''''page ownership'''''. The algorithms for memory coherence depend on the page fault handlers, their servers and the data structures used. So ''page table'' becomes an important part of these protocols.

= Synchronization =

'''Process synchronization''', which is the mainly concerned with different processes committing to a a certain sequence of actions, and '''data synchronization''', which deals with maintaining data integrity across various copies of a dataset, are two two distinct but related concepts. Process synchronization primitives can be used to implement data synchronization.

[http://en.wikipedia.org/wiki/Mutual_exclusion Mutual exclusion] is the main requirement to be fulfilled in order to synchronize processes, and is needed in both single-processor and multiprocessor systems. There are various approaches to provide mutual exclusion in a system:

* Disabling interrupts
* Locks
* Mutex
* Semaphores
* Barriers
* Test and Set

The next section discusses if there is an alternative to implement mutual exclusion without requiring any hardware support. Peterson's algorithm is one such software solution for guaranteeing mutual exclusion.

== Peterson's algorithm ==

This algorithm utilizes the simple idea of combining per-thread '''flags''' to indicate the intent to enter the lock and the '''turn''' variable to determine which thread should enter in the rare case that both wish to enter at the same time. The algorithm is as shown below: [3]

<pre>
void init() {
flag[0] = flag[1] = 0; // 1 -> thread wants to acquire lock (intent)
turn = 0; // whose turn is it? (thread 0 or thread 1?)
}

void lock() {
flag[self] = 1;
turn = 1 - self; // be generous: make it the other thread’s turn
while ((flag[1-self] == 1) && (turn == 1 - self))
; // spin-wait while other thread has intent
// AND it is other thread’s turn
}

void unlock() {
flag[self] = 0; // simply undo your intent
}
</pre>

The advantages of this approach are:

* It does not assume much about the underlying hardware. It relies on the fact that load and store instructions are atomic( even across processors) and they execute in an order.
* It does not require special instructions.

Though Peterson's algorithm has the above advantages, there are a few problems that render it impractical for use:

* Spin-waiting is possible which causes a thread to spend lot of its CPU time waiting for another thread to release the lock.
* Algorithm might not work well with out-of-order execution of instructions supported by most of the modern processors.
* The algorithm also lacks scalability.

Thus, only the software solutions cannot work well. Sufficient amount of hardware and OS support is needed to get the locking work properly.

== Hardware support ==

Exclusive locking assumes the worst and proceeds only after acquiring all locks such that no other thread can interfere. This is a ''pessimistic'' approach. In contrast, the ''optimistic'' approach proceeds with an update, hoping that it can be completed without any interference. This requires ''collision detection'' during the update. The optimistic approach is thus, more efficient in fine-grained operations.

Special instructions are provided by processors designed for multiprocessor operations in order to manage concurrent access to shared variables. Atomic instructions like [http://en.wikipedia.org/wiki/Test_and_Test-and-set test-and-set], [http://en.wikipedia.org/wiki/Fetch-and-add fetch-and-increment] and [http://en.wikipedia.org/wiki/Swap_%28computer_science%29 swap] were sufficient for early processors to implement mutexes for concurrent objects. Today, every modern processor relies on some form of read-modify-write atomic instruction such as [http://en.wikipedia.org/wiki/Compare-and-swap compare-and-swap], [http://en.wikipedia.org/wiki/Load-link/store-conditional LL/SC] etc. for the same.

=Resources=

[[#1body|1.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_jp#Memory_Consistency_Problem 
[[#2body|2.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_ss#Cache_Coherence 
[[#3body|3.]] http://www.windowsnetworking.com/articles_tutorials/Cache-Coherency.html 
[[#4body|4.]]http://web.sfc.keio.ac.jp/~rdv/keio/sfc/teaching/architecture/architecture-2007/lec08.html

CSC 456 Spring 2012/ch7 MN

2012-03-14T17:13:44Z

Jmmohund2: /* add links to consitency types

==Introduction==
Though the migration from [http://en.wikipedia.org/wiki/Uniprocessor_system uniprocessor system] to [http://en.wikipedia.org/wiki/Multiprocessing multiprocessing] systems is not new, the world of parallel computers is undergoing a continuous change. Parallel computers, which started as high-end super-computing systems for carrying out huge calculations, are now ubiquitous and are present in all mainstream architectures for servers, desktops, and embedded systems. In order to design parallel architectures to meet programmer's needs and expectations more closely, exciting and challenging changes exist. The three main areas which are being considered by scientists today are: [http://en.wikipedia.org/wiki/Cache_coherence cache coherence], memory consistency and [http://en.wikipedia.org/wiki/Synchronization_%28computer_science%29 synchronization].

=Cache Coherence Problem=

[[Image:sharedmem.jpg|thumbnail|right|600px|Shared Memory system with dedicated Cache for each processor[[#4foot|[4]]]]]

In a system with single processor (single core), maintaining cache coherency is simple and easy but in a multiprocessor system, it is much more complicated. Data can be present in any processor's cache and the protocol needs to ensure that the data is same in all caches. If it cannot ensure that all the caches are same, then it needs to flag a cache line indicating that it is not updated.

In the figure shown here, this is a 4 processor shared memory system where each processor has its own cache. Supposed processor P1 reads memory location M1 and stores it in its local cache. Then, if P2 reads same location memory location then M1 gets stored in P2’s cache. Now, if P1 changes value of M1, two copies of same data, residing in different caches will become different. When P2 operates on M1, it uses the stale value of M1 that was stored in its cache. It is responsibility of Cache Coherence Protocol to prevent this. Hardware support is needed to provide a coherent view of data in multiple caches. This is known as write propagation requirement.

One may think that cache write policy can provide cache coherence, but this is incorrect. Cache write policy only controls how a change in value of cache is propagated to lower level cache or main memory. It is not responsible for propagating changes to other caches.

==Cache Coherence Protocols==

The two basic methods to utilize the inter-core bus to notify other cores when a core changes something in its cache are '''update''' and '''invalidate'''. In the update method, if variable 'x' is modified by core 1, core 1 has to send the updated value of 'x' onto the inter-core bus. Each cache listens to the inter-core bus and if a cache sees a variable on the bus which it has a copy of, it will read the updated value. This ensures that all caches have the most up-to-date value of the variable.[[#3foot|[3]]]

In case of invalidation, an invalidation message is sent onto the inter-core bus when a variable is changed. The other caches will read this invalidation signal and if its core attempts to access that variable, it will result in a cache miss and the variable will be read from main memory.

The update method results in significant amount of traffic on the inter-core bus as the update signal is sent onto the bus every time the variable is updated. The invalidation method only requires that an invalidation signal be sent the first time a variable is altered; this is why the invalidation method is the preferred method.

In order to improve cache coherency performance over the years, several protocols have been proposed.

===MSI===

MSI stands for Modified, Shared, and Invalid, based on the three states that a line of cache can be in. The Modified state means that a variable in the cache has been modified and therefore has a different value than that found in main memory; the cache is responsible for writing the variable back to main memory. The Shared state means that the variable exists in at least one cache and is not modified; the cache can evict the variable without writing it back to the main memory. The Invalid state means that the value of the variable has been modified by another cache and this value is invalid; the cache must read a new value from main memory (or another cache).

A drawback to this protocol occurs when a single processor wants to read blocks and then write to them without another processor sharing that block. After reading the block, a bus transaction places the block into a shared state. The write then occurs and another bus transaction is sent to invalidate the shared copy. This second transaction is useless as no other processors are sharing the block, but the MSI protocol has no way to specify this.

===MESI===

MESI stands for Modified, Exclusive, Shared, and Invalid. The Modified and Invalid states are the same for this protocol as they are for the MSI protocol. This protocol introduces a new state; the Exclusive state. The Exclusive state means that the variable is in only this cache and the value of it matches the value within the main memory. This now means that the Shared state indicates that the variable is contained in more than one cache.

===MOSI===

The MOSI protocol is identical to the MSI protocol except that it adds an Owned state. The Owned state means that the processor "Owns" the variable and will provide the current value to other caches when requested (or at least it will decide if it will provide it when asked). This is useful because another cache will not have to read the value from main memory and will receive it from the Owning cache much, much, faster.

===MOESI===

The MOESI protocol is a combination of the MESI and MOSI protocols.

=Memory Consistency Problem=

Memory consistency deals with the ordering of memory operations (load and store) to different memory locations. In a single processor system, code will execute correctly if the compiler preservers the order of the access to synchronization variables and other dependent variables. But in shared memory model with multiple processors, two threads could access a shared data element, such as a synchronization variable, the output of the threads would change based on which thread, accesses the shared data element earlier. If this were to occur, then the program output may not be the value expected. Maintaining program order is very important for memory consistency but it comes with performance degradation.

== Memory Consistency Models ==

The memory consistency model of a shared-memory multiprocessor is a formal speciﬁcation of how the memory system appears to the programmer. It eliminates the gap between the behavior expected by the programmer and the actual behavior supported by a system. Effectively, the consistency model places restrictions on the values that can be returned by a read, in a shared-memory program execution.

In a single processor system, in order to maintain memory consistency, it needs to ensure that the compiler preserves the program order when accessing synchronization variables. But in a multiprocessor system, it is required to ensure that accesses of one processor appear to execute in program order to all other processors, atleast partially.

=== Memory semantics in Uniprocessor systems ===

Uniprocessor languages use simple sequential semantics for memory operations, which allow the programmer to assume that all memory operations will occur one at a time in the sequential order speciﬁed by the program. Thus, one can expect for the value in a particular location be the same for a read as it was for the last write because of the sequential program execution. It is sufﬁcient to only maintain uniprocessor data and control dependences. The compiler and hardware can freely reorder operations to different locations if the uniprocessor data and control dependences are respected. This enables compiler optimizations such as register allocation, code motion, and loop transformations, and hardware optimizations, such as pipelining, multiple issue, write buffer bypassing and forwarding, and lockup-free caches, all of which lead to overlapping and reordering of memory operations. [4]

=== Memory semantics in multiprocessor systems ===

Programmer's implicit expectations are:

* memory accesses in a processor takes place according to the program order.
* Each memory access is performed atomically.

A strong consistency model attempting uniprocessor-like consistency could cause global bottleneck, costing performance. Thus, '''''weak''''' consistency models are deployed to improve performance. The advanatges of such models are:

* They support out-of-order execution within individual CPUs
* Relaxes latency issues with near-simultaneous accesses by different CPUs

The following are the various consistency models and it is the programmer who must take into account the memory consistency model to create correct software:
=ADD INFO ABOUT EACH OF THESE TYPES OF MEM CONSIS MODELS=
* [http://en.wikipedia.org/wiki/Linearizability linearizability (also known as strict or atomic consistency)]
* [http://en.wikipedia.org/wiki/Sequential_consistency sequential consistency]
* [http://en.wikipedia.org/wiki/Causal_consistency causal consistency]
* [http://en.wikipedia.org/wiki/Release_consistency release consistency]
* [http://en.wikipedia.org/wiki/Eventual_consistency eventual consistency ]
* [http://en.wikipedia.org/wiki/Delta_consistency delta consistency]
* [http://en.wikipedia.org/wiki/PRAM_consistency PRAM consistency (also known as FIFO consistency)]
* [http://en.wikipedia.org/wiki/Weak_consistency weak consistency]
* [http://en.wikipedia.org/wiki/Vector-field_consistency vector-field consistency]
* fork consistency
* serializability
* one-copy serializability
* entry consistency

== Memory Coherence and Shared Virtual Memory ==

The memory coherence problem in a shared virtual memory system and in multicache systems are different. In a multicache multiprocessor, there are processors sharing a physical memory through their private caches. The relatively small size of a cache and the fast bus connection to the shared memory, enables using a sophisticated coherence protocol for the multicache hardware such that the time delay of conflicting writes to a memory location is small. [5]

In contrast, in a shared virtual memory on a loosely coupled multiprocessor which has no physically shared memory, and having a nontrivial communication cost between processors, conflicts are not likely to be solved with negligible delay, and they resemble much more a “page
fault” in a traditional virtual memory system. Thus, there are two design choices that greatly influence the implementation of a shared virtual memory: the granularity of the memory units (i.e., the “page size”)and the strategy for maintaining coherence.

Memory coherence strategies are classified based on how they deal with '''''page synchronization''''' and '''''page ownership'''''. The algorithms for memory coherence depend on the page fault handlers, their servers and the data structures used. So ''page table'' becomes an important part of these protocols.

= Synchronization =

'''Process synchronization''', which is the mainly concerned with different processes committing to a a certain sequence of actions, and '''data synchronization''', which deals with maintaining data integrity across various copies of a dataset, are two two distinct but related concepts. Process synchronization primitives can be used to implement data synchronization.

[http://en.wikipedia.org/wiki/Mutual_exclusion Mutual exclusion] is the main requirement to be fulfilled in order to synchronize processes, and is needed in both single-processor and multiprocessor systems. There are various approaches to provide mutual exclusion in a system:

* Disabling interrupts
* Locks
* Mutex
* Semaphores
* Barriers
* Test and Set

The next section discusses if there is an alternative to implement mutual exclusion without requiring any hardware support. Peterson's algorithm is one such software solution for guaranteeing mutual exclusion.

== Peterson's algorithm ==

This algorithm utilizes the simple idea of combining per-thread '''flags''' to indicate the intent to enter the lock and the '''turn''' variable to determine which thread should enter in the rare case that both wish to enter at the same time. The algorithm is as shown below: [3]

<pre>
void init() {
flag[0] = flag[1] = 0; // 1 -> thread wants to acquire lock (intent)
turn = 0; // whose turn is it? (thread 0 or thread 1?)
}

void lock() {
flag[self] = 1;
turn = 1 - self; // be generous: make it the other thread’s turn
while ((flag[1-self] == 1) && (turn == 1 - self))
; // spin-wait while other thread has intent
// AND it is other thread’s turn
}

void unlock() {
flag[self] = 0; // simply undo your intent
}
</pre>

The advantages of this approach are:

* It does not assume much about the underlying hardware. It relies on the fact that load and store instructions are atomic( even across processors) and they execute in an order.
* It does not require special instructions.

Though Peterson's algorithm has the above advantages, there are a few problems that render it impractical for use:

* Spin-waiting is possible which causes a thread to spend lot of its CPU time waiting for another thread to release the lock.
* Algorithm might not work well with out-of-order execution of instructions supported by most of the modern processors.
* The algorithm also lacks scalability.

Thus, only the software solutions cannot work well. Sufficient amount of hardware and OS support is needed to get the locking work properly.

== Hardware support ==

Exclusive locking assumes the worst and proceeds only after acquiring all locks such that no other thread can interfere. This is a ''pessimistic'' approach. In contrast, the ''optimistic'' approach proceeds with an update, hoping that it can be completed without any interference. This requires ''collision detection'' during the update. The optimistic approach is thus, more efficient in fine-grained operations.

Special instructions are provided by processors designed for multiprocessor operations in order to manage concurrent access to shared variables. Atomic instructions like [http://en.wikipedia.org/wiki/Test_and_Test-and-set test-and-set], [http://en.wikipedia.org/wiki/Fetch-and-add fetch-and-increment] and [http://en.wikipedia.org/wiki/Swap_%28computer_science%29 swap] were sufficient for early processors to implement mutexes for concurrent objects. Today, every modern processor relies on some form of read-modify-write atomic instruction such as [http://en.wikipedia.org/wiki/Compare-and-swap compare-and-swap], [http://en.wikipedia.org/wiki/Load-link/store-conditional LL/SC] etc. for the same.

=Resources=

[[#1body|1.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_jp#Memory_Consistency_Problem 
[[#2body|2.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_ss#Cache_Coherence 
[[#3body|3.]] http://www.windowsnetworking.com/articles_tutorials/Cache-Coherency.html 
[[#4body|4.]]http://web.sfc.keio.ac.jp/~rdv/keio/sfc/teaching/architecture/architecture-2007/lec08.html

CSC 456 Spring 2012/ch7 MN

2012-02-27T18:58:12Z

Jmmohund2: /* Memory semantics in multiprocessor systems */

==Introduction==
Though the migration from [http://en.wikipedia.org/wiki/Uniprocessor_system uniprocessor system] to [http://en.wikipedia.org/wiki/Multiprocessing multiprocessing] systems is not new, the world of parallel computers is undergoing a continuous change. Parallel computers, which started as high-end super-computing systems for carrying out huge calculations, are now ubiquitous and are present in all mainstream architectures for servers, desktops, and embedded systems. In order to design parallel architectures to meet programmer's needs and expectations more closely, exciting and challenging changes exist. The three main areas which are being considered by scientists today are: [http://en.wikipedia.org/wiki/Cache_coherence cache coherence], memory consistency and [http://en.wikipedia.org/wiki/Synchronization_%28computer_science%29 synchronization].

=Cache Coherence Problem=

[[Image:sharedmem.jpg|thumbnail|right|600px|Shared Memory system with dedicated Cache for each processor[[#4foot|[4]]]]]

In a system with single processor (single core), maintaining cache coherency is simple and easy but in a multiprocessor system, it is not as simple. Data can be present in any processors cache and protocol needs to ensure that the data is same in all caches. If it cannot ensure that all the caches are same, then it needs to flag a cache line indicating that it is not updated.

In the figure shown here, this is a 4 processor shared memory system where each processor has its own cache. Supposed processor P1 reads memory location M1 and stores it in its local cache. Then, if P2 reads same location memory location then M1 gets stored in P2’s cache. Now, if P1 changes value of M1, two copies of same data, residing in different caches will become different. When P2 operates on M1, it uses the stale value of M1 that was stored in its cache. It is responsibility of Cache Coherence Protocol to prevent this. Hardware support is needed to provide a coherent view of data in multiple caches. This is known as write propagation requirement.

One may think that cache write policy can provide cache coherence, but this is incorrect. Cache write policy only controls how a change in value of cache is propagated to lower level cache or main memory. It is not responsible for propagating changes to other caches.

==Cache Coherence Protocols==

The two basic methods to utilize the inter-core bus to notify other cores when a core changes something in its cache are '''update''' and '''invalidate'''. In the update method, if variable 'x' is modified by core 1, core 1 has to send the updated value of 'x' onto the inter-core bus. Each cache listens to the inter-core bus and if a cache sees a variable on the bus which it has a copy of, it will read the updated value. This ensures that all caches have the most up-to-date value of the variable.[[#3foot|[3]]]

In case of invalidation, an invalidation message is sent onto the inter-core bus when a variable is changed. The other caches will read this invalidation signal and if its core attempts to access that variable, it will result in a cache miss and the variable will be read from main memory.

The update method results in significant amount of traffic on the inter-core bus as the update signal is sent onto the bus every time the variable is updated. The invalidation method only requires that an invalidation signal be sent the first time a variable is altered; this is why the invalidation method is the preferred method.

In order to improve cache coherency performance over the years, several protocols have been proposed.

===MSI===

MSI stands for Modified, Shared, and Invalid, based on the three states that a line of cache can be in. The Modified state means that a variable in the cache has been modified and therefore has a different value than that found in main memory; the cache is responsible for writing the variable back to main memory. The Shared state means that the variable exists in at least one cache and is not modified; the cache can evict the variable without writing it back to the main memory. The Invalid state means that the value of the variable has been modified by another cache and this value is invalid; the cache must read a new value from main memory (or another cache).

===MESI===

MESI stands for Modified, Exclusive, Shared, and Invalid. The Modified and Invalid states are the same for this protocol as they are for the MSI protocol. This protocol introduces a new state; the Exclusive state. The Exclusive state means that the variable is in only this cache and the value of it matches the value within the main memory. This now means that the Shared state indicates that the variable is contained in more than one cache.

===MOSI===

The MOSI protocol is identical to the MSI protocol except that it adds an Owned state. The Owned state means that the processor "Owns" the variable and will provide the current value to other caches when requested (or at least it will decide if it will provide it when asked). This is useful because another cache will not have to read the value from main memory and will receive it from the Owning cache much, much, faster.

===MOESI===

The MOESI protocol is a combination of the MESI and MOSI protocols.

=Memory Consistency Problem=

Memory consistency deals with the ordering of memory operations (load and store) to different memory locations. In a single processor system, code will execute correctly if the compiler preservers the order of the access to synchronization variables and other dependent variables. But in shared memory model with multiple processors, two threads could access a shared data element, such as a synchronization variable, the output of the threads would change based on which thread, accesses the shared data element earlier. If this were to occur, then the program output may not be the value expected. Maintaining program order is very important for memory consistency but it comes with performance degradation.

== Memory Consistency Models ==

The memory consistency model of a shared-memory multiprocessor is a formal speciﬁcation of how the memory system appears to the programmer. It eliminates the gap between the behavior expected by the programmer and the actual behavior supported by a system. Effectively, the consistency model places restrictions on the values that can be returned by a read, in a shared-memory program execution.

In a single processor system, in order to maintain memory consistency, it needs to ensure that the compiler preserves the program order when accessing synchronization variables. But in a multiprocessor system, it is required to ensure that accesses of one processor appear to execute in program order to all other processors, atleast partially.

=== Memory semantics in Uniprocessor systems ===

Uniprocessor languages use simple sequential semantics for memory operations, which allow the programmer to assume that all memory operations will occur one at a time in the sequential order speciﬁed by the program. Thus, one can expect for the value in a particular location be the same for a read as it was for the last write because of the sequential program execution. It is sufﬁcient to only maintain uniprocessor data and control dependences. The compiler and hardware can freely reorder operations to different locations if the uniprocessor data and control dependences are respected. This enables compiler optimizations such as register allocation, code motion, and loop transformations, and hardware optimizations, such as pipelining, multiple issue, write buffer bypassing and forwarding, and lockup-free caches, all of which lead to overlapping and reordering of memory operations. [4]

=== Memory semantics in multiprocessor systems ===

Programmer's implicit expectations are:

* memory accesses in a processor takes place according to the program order.
* Each memory access is performed atomically.

A strong consistency model attempting uniprocessor-like consistency could cause global bottleneck, costing performance. Thus, '''''weak''''' consistency models are deployed to improve performance. The advanatges of such models are:

* They support out-of-order execution within individual CPUs
* Relaxes latency issues with near-simultaneous accesses by different CPUs

The following are the various consistency models and it is the programmer who must take into account the memory consistency model to create correct software:
=ADD INFO ABOUT EACH OF THESE TYPES OF MEM CONSIS MODELS=
* linearizability (also known as strict or atomic consistency)
* sequential consistency
* causal consistency
* release consistency
* eventual consistency
* delta consistency
* PRAM consistency (also known as FIFO consistency)
* weak consistency
* vector-field consistency
* fork consistency
* serializability
* one-copy serializability
* entry consistency

== Memory Coherence and Shared Virtual Memory ==

The memory coherence problem in a shared virtual memory system and in multicache systems are different. In a multicache multiprocessor, there are processors sharing a physical memory through their private caches. The relatively small size of a cache and the fast bus connection to the shared memory, enables using a sophisticated coherence protocol for the multicache hardware such that the time delay of conflicting writes to a memory location is small. [5]

In contrast, in a shared virtual memory on a loosely coupled multiprocessor which has no physically shared memory, and having a nontrivial communication cost between processors, conflicts are not likely to be solved with negligible delay, and they resemble much more a “page
fault” in a traditional virtual memory system. Thus, there are two design choices that greatly influence the implementation of a shared virtual memory: the granularity of the memory units (i.e., the “page size”)and the strategy for maintaining coherence.

Memory coherence strategies are classified based on how they deal with '''''page synchronization''''' and '''''page ownership'''''. The algorithms for memory coherence depend on the page fault handlers, their servers and the data structures used. So ''page table'' becomes an important part of these protocols.

= Synchronization =

'''Process synchronization''', which is the mainly concerned with different processes committing to a a certain sequence of actions, and '''data synchronization''', which deals with maintaining data integrity across various copies of a dataset, are two two distinct but related concepts. Process synchronization primitives can be used to implement data synchronization.

[http://en.wikipedia.org/wiki/Mutual_exclusion Mutual exclusion] is the main requirement to be fulfilled in order to synchronize processes, and is needed in both single-processor and multiprocessor systems. There are various approaches to provide mutual exclusion in a system:

* Disabling interrupts
* Locks
* Mutex
* Semaphores
* Barriers
* Test and Set

The next section discusses if there is an alternative to implement mutual exclusion without requiring any hardware support. Peterson's algorithm is one such software solution for guaranteeing mutual exclusion.

== Peterson's algorithm ==

This algorithm utilizes the simple idea of combining per-thread '''flags''' to indicate the intent to enter the lock and the '''turn''' variable to determine which thread should enter in the rare case that both wish to enter at the same time. The algorithm is as shown below: [3]

<pre>
void init() {
flag[0] = flag[1] = 0; // 1 -> thread wants to acquire lock (intent)
turn = 0; // whose turn is it? (thread 0 or thread 1?)
}

void lock() {
flag[self] = 1;
turn = 1 - self; // be generous: make it the other thread’s turn
while ((flag[1-self] == 1) && (turn == 1 - self))
; // spin-wait while other thread has intent
// AND it is other thread’s turn
}

void unlock() {
flag[self] = 0; // simply undo your intent
}
</pre>

The advantages of this approach are:

* It does not assume much about the underlying hardware. It relies on the fact that load and store instructions are atomic( even across processors) and they execute in an order.
* It does not require special instructions.

Though Peterson's algorithm has the above advantages, there are a few problems that render it impractical for use:

* Spin-waiting is possible which causes a thread to spend lot of its CPU time waiting for another thread to release the lock.
* Algorithm might not work well with out-of-order execution of instructions supported by most of the modern processors.
* The algorithm also lacks scalability.

Thus, only the software solutions cannot work well. Sufficient amount of hardware and OS support is needed to get the locking work properly.

== Hardware support ==

Exclusive locking assumes the worst and proceeds only after acquiring all locks such that no other thread can interfere. This is a ''pessimistic'' approach. In contrast, the ''optimistic'' approach proceeds with an update, hoping that it can be completed without any interference. This requires ''collision detection'' during the update. The optimistic approach is thus, more efficient in fine-grained operations.

Special instructions are provided by processors designed for multiprocessor operations in order to manage concurrent access to shared variables. Atomic instructions like [http://en.wikipedia.org/wiki/Test_and_Test-and-set test-and-set], [http://en.wikipedia.org/wiki/Fetch-and-add fetch-and-increment] and [http://en.wikipedia.org/wiki/Swap_%28computer_science%29 swap] were sufficient for early processors to implement mutexes for concurrent objects. Today, every modern processor relies on some form of read-modify-write atomic instruction such as [http://en.wikipedia.org/wiki/Compare-and-swap compare-and-swap], [http://en.wikipedia.org/wiki/Load-link/store-conditional LL/SC] etc. for the same.

=Resources=

[[#1body|1.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_jp#Memory_Consistency_Problem 
[[#2body|2.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_ss#Cache_Coherence 
[[#3body|3.]] http://www.windowsnetworking.com/articles_tutorials/Cache-Coherency.html 
[[#4body|4.]]http://web.sfc.keio.ac.jp/~rdv/keio/sfc/teaching/architecture/architecture-2007/lec08.html

CSC 456 Spring 2012/ch7 MN

2012-02-27T18:54:01Z

Jmmohund2: /* Memory semantics in Uniprocessor systems */

==Introduction==
Though the migration from [http://en.wikipedia.org/wiki/Uniprocessor_system uniprocessor system] to [http://en.wikipedia.org/wiki/Multiprocessing multiprocessing] systems is not new, the world of parallel computers is undergoing a continuous change. Parallel computers, which started as high-end super-computing systems for carrying out huge calculations, are now ubiquitous and are present in all mainstream architectures for servers, desktops, and embedded systems. In order to design parallel architectures to meet programmer's needs and expectations more closely, exciting and challenging changes exist. The three main areas which are being considered by scientists today are: [http://en.wikipedia.org/wiki/Cache_coherence cache coherence], memory consistency and [http://en.wikipedia.org/wiki/Synchronization_%28computer_science%29 synchronization].

=Cache Coherence Problem=

[[Image:sharedmem.jpg|thumbnail|right|600px|Shared Memory system with dedicated Cache for each processor[[#4foot|[4]]]]]

In a system with single processor (single core), maintaining cache coherency is simple and easy but in a multiprocessor system, it is not as simple. Data can be present in any processors cache and protocol needs to ensure that the data is same in all caches. If it cannot ensure that all the caches are same, then it needs to flag a cache line indicating that it is not updated.

In the figure shown here, this is a 4 processor shared memory system where each processor has its own cache. Supposed processor P1 reads memory location M1 and stores it in its local cache. Then, if P2 reads same location memory location then M1 gets stored in P2’s cache. Now, if P1 changes value of M1, two copies of same data, residing in different caches will become different. When P2 operates on M1, it uses the stale value of M1 that was stored in its cache. It is responsibility of Cache Coherence Protocol to prevent this. Hardware support is needed to provide a coherent view of data in multiple caches. This is known as write propagation requirement.

One may think that cache write policy can provide cache coherence, but this is incorrect. Cache write policy only controls how a change in value of cache is propagated to lower level cache or main memory. It is not responsible for propagating changes to other caches.

==Cache Coherence Protocols==

The two basic methods to utilize the inter-core bus to notify other cores when a core changes something in its cache are '''update''' and '''invalidate'''. In the update method, if variable 'x' is modified by core 1, core 1 has to send the updated value of 'x' onto the inter-core bus. Each cache listens to the inter-core bus and if a cache sees a variable on the bus which it has a copy of, it will read the updated value. This ensures that all caches have the most up-to-date value of the variable.[[#3foot|[3]]]

In case of invalidation, an invalidation message is sent onto the inter-core bus when a variable is changed. The other caches will read this invalidation signal and if its core attempts to access that variable, it will result in a cache miss and the variable will be read from main memory.

The update method results in significant amount of traffic on the inter-core bus as the update signal is sent onto the bus every time the variable is updated. The invalidation method only requires that an invalidation signal be sent the first time a variable is altered; this is why the invalidation method is the preferred method.

In order to improve cache coherency performance over the years, several protocols have been proposed.

===MSI===

MSI stands for Modified, Shared, and Invalid, based on the three states that a line of cache can be in. The Modified state means that a variable in the cache has been modified and therefore has a different value than that found in main memory; the cache is responsible for writing the variable back to main memory. The Shared state means that the variable exists in at least one cache and is not modified; the cache can evict the variable without writing it back to the main memory. The Invalid state means that the value of the variable has been modified by another cache and this value is invalid; the cache must read a new value from main memory (or another cache).

===MESI===

MESI stands for Modified, Exclusive, Shared, and Invalid. The Modified and Invalid states are the same for this protocol as they are for the MSI protocol. This protocol introduces a new state; the Exclusive state. The Exclusive state means that the variable is in only this cache and the value of it matches the value within the main memory. This now means that the Shared state indicates that the variable is contained in more than one cache.

===MOSI===

The MOSI protocol is identical to the MSI protocol except that it adds an Owned state. The Owned state means that the processor "Owns" the variable and will provide the current value to other caches when requested (or at least it will decide if it will provide it when asked). This is useful because another cache will not have to read the value from main memory and will receive it from the Owning cache much, much, faster.

===MOESI===

The MOESI protocol is a combination of the MESI and MOSI protocols.

=Memory Consistency Problem=

Memory consistency deals with the ordering of memory operations (load and store) to different memory locations. In a single processor system, code will execute correctly if the compiler preservers the order of the access to synchronization variables and other dependent variables. But in shared memory model with multiple processors, two threads could access a shared data element, such as a synchronization variable, the output of the threads would change based on which thread, accesses the shared data element earlier. If this were to occur, then the program output may not be the value expected. Maintaining program order is very important for memory consistency but it comes with performance degradation.

== Memory Consistency Models ==

The memory consistency model of a shared-memory multiprocessor is a formal speciﬁcation of how the memory system appears to the programmer. It eliminates the gap between the behavior expected by the programmer and the actual behavior supported by a system. Effectively, the consistency model places restrictions on the values that can be returned by a read, in a shared-memory program execution.

In a single processor system, in order to maintain memory consistency, it needs to ensure that the compiler preserves the program order when accessing synchronization variables. But in a multiprocessor system, it is required to ensure that accesses of one processor appear to execute in program order to all other processors, atleast partially.

=== Memory semantics in Uniprocessor systems ===

Uniprocessor languages use simple sequential semantics for memory operations, which allow the programmer to assume that all memory operations will occur one at a time in the sequential order speciﬁed by the program. Thus, one can expect for the value in a particular location be the same for a read as it was for the last write because of the sequential program execution. It is sufﬁcient to only maintain uniprocessor data and control dependences. The compiler and hardware can freely reorder operations to different locations if the uniprocessor data and control dependences are respected. This enables compiler optimizations such as register allocation, code motion, and loop transformations, and hardware optimizations, such as pipelining, multiple issue, write buffer bypassing and forwarding, and lockup-free caches, all of which lead to overlapping and reordering of memory operations. [4]

=== Memory semantics in multiprocessor systems ===

Programmer's implicit expectations are:

* memory accesses in a processor takes place according to the program order.
* Each memory access is performed atomically.

A strong consistency model attempting uniprocessor-like consistency could cause global bottleneck, costing performance. Thus, '''''weak''''' consistency models are deployed to improve performance. The advanatges of such models are:

* They support out-of-order execution within individual CPUs
* Relaxes latency issues with near-simultaneous accesses by different CPUs

The following are the various consistency models and it is the programmer who must take into account the memory consistency model to create correct software:

* linearizability (also known as strict or atomic consistency)
* sequential consistency
* causal consistency
* release consistency
* eventual consistency
* delta consistency
* PRAM consistency (also known as FIFO consistency)
* weak consistency
* vector-field consistency
* fork consistency
* serializability
* one-copy serializability
* entry consistency

== Memory Coherence and Shared Virtual Memory ==

The memory coherence problem in a shared virtual memory system and in multicache systems are different. In a multicache multiprocessor, there are processors sharing a physical memory through their private caches. The relatively small size of a cache and the fast bus connection to the shared memory, enables using a sophisticated coherence protocol for the multicache hardware such that the time delay of conflicting writes to a memory location is small. [5]

In contrast, in a shared virtual memory on a loosely coupled multiprocessor which has no physically shared memory, and having a nontrivial communication cost between processors, conflicts are not likely to be solved with negligible delay, and they resemble much more a “page
fault” in a traditional virtual memory system. Thus, there are two design choices that greatly influence the implementation of a shared virtual memory: the granularity of the memory units (i.e., the “page size”)and the strategy for maintaining coherence.

Memory coherence strategies are classified based on how they deal with '''''page synchronization''''' and '''''page ownership'''''. The algorithms for memory coherence depend on the page fault handlers, their servers and the data structures used. So ''page table'' becomes an important part of these protocols.

= Synchronization =

'''Process synchronization''', which is the mainly concerned with different processes committing to a a certain sequence of actions, and '''data synchronization''', which deals with maintaining data integrity across various copies of a dataset, are two two distinct but related concepts. Process synchronization primitives can be used to implement data synchronization.

[http://en.wikipedia.org/wiki/Mutual_exclusion Mutual exclusion] is the main requirement to be fulfilled in order to synchronize processes, and is needed in both single-processor and multiprocessor systems. There are various approaches to provide mutual exclusion in a system:

* Disabling interrupts
* Locks
* Mutex
* Semaphores
* Barriers
* Test and Set

The next section discusses if there is an alternative to implement mutual exclusion without requiring any hardware support. Peterson's algorithm is one such software solution for guaranteeing mutual exclusion.

== Peterson's algorithm ==

This algorithm utilizes the simple idea of combining per-thread '''flags''' to indicate the intent to enter the lock and the '''turn''' variable to determine which thread should enter in the rare case that both wish to enter at the same time. The algorithm is as shown below: [3]

<pre>
void init() {
flag[0] = flag[1] = 0; // 1 -> thread wants to acquire lock (intent)
turn = 0; // whose turn is it? (thread 0 or thread 1?)
}

void lock() {
flag[self] = 1;
turn = 1 - self; // be generous: make it the other thread’s turn
while ((flag[1-self] == 1) && (turn == 1 - self))
; // spin-wait while other thread has intent
// AND it is other thread’s turn
}

void unlock() {
flag[self] = 0; // simply undo your intent
}
</pre>

The advantages of this approach are:

* It does not assume much about the underlying hardware. It relies on the fact that load and store instructions are atomic( even across processors) and they execute in an order.
* It does not require special instructions.

Though Peterson's algorithm has the above advantages, there are a few problems that render it impractical for use:

* Spin-waiting is possible which causes a thread to spend lot of its CPU time waiting for another thread to release the lock.
* Algorithm might not work well with out-of-order execution of instructions supported by most of the modern processors.
* The algorithm also lacks scalability.

Thus, only the software solutions cannot work well. Sufficient amount of hardware and OS support is needed to get the locking work properly.

== Hardware support ==

Exclusive locking assumes the worst and proceeds only after acquiring all locks such that no other thread can interfere. This is a ''pessimistic'' approach. In contrast, the ''optimistic'' approach proceeds with an update, hoping that it can be completed without any interference. This requires ''collision detection'' during the update. The optimistic approach is thus, more efficient in fine-grained operations.

Special instructions are provided by processors designed for multiprocessor operations in order to manage concurrent access to shared variables. Atomic instructions like [http://en.wikipedia.org/wiki/Test_and_Test-and-set test-and-set], [http://en.wikipedia.org/wiki/Fetch-and-add fetch-and-increment] and [http://en.wikipedia.org/wiki/Swap_%28computer_science%29 swap] were sufficient for early processors to implement mutexes for concurrent objects. Today, every modern processor relies on some form of read-modify-write atomic instruction such as [http://en.wikipedia.org/wiki/Compare-and-swap compare-and-swap], [http://en.wikipedia.org/wiki/Load-link/store-conditional LL/SC] etc. for the same.

=Resources=

[[#1body|1.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_jp#Memory_Consistency_Problem 
[[#2body|2.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_ss#Cache_Coherence 
[[#3body|3.]] http://www.windowsnetworking.com/articles_tutorials/Cache-Coherency.html 
[[#4body|4.]]http://web.sfc.keio.ac.jp/~rdv/keio/sfc/teaching/architecture/architecture-2007/lec08.html

CSC 456 Spring 2012/ch7 MN

2012-02-27T18:48:25Z

Jmmohund2: /* Memory Consistency Problem */

==Introduction==
Though the migration from [http://en.wikipedia.org/wiki/Uniprocessor_system uniprocessor system] to [http://en.wikipedia.org/wiki/Multiprocessing multiprocessing] systems is not new, the world of parallel computers is undergoing a continuous change. Parallel computers, which started as high-end super-computing systems for carrying out huge calculations, are now ubiquitous and are present in all mainstream architectures for servers, desktops, and embedded systems. In order to design parallel architectures to meet programmer's needs and expectations more closely, exciting and challenging changes exist. The three main areas which are being considered by scientists today are: [http://en.wikipedia.org/wiki/Cache_coherence cache coherence], memory consistency and [http://en.wikipedia.org/wiki/Synchronization_%28computer_science%29 synchronization].

=Cache Coherence Problem=

[[Image:sharedmem.jpg|thumbnail|right|600px|Shared Memory system with dedicated Cache for each processor[[#4foot|[4]]]]]

In a system with single processor (single core), maintaining cache coherency is simple and easy but in a multiprocessor system, it is not as simple. Data can be present in any processors cache and protocol needs to ensure that the data is same in all caches. If it cannot ensure that all the caches are same, then it needs to flag a cache line indicating that it is not updated.

In the figure shown here, this is a 4 processor shared memory system where each processor has its own cache. Supposed processor P1 reads memory location M1 and stores it in its local cache. Then, if P2 reads same location memory location then M1 gets stored in P2’s cache. Now, if P1 changes value of M1, two copies of same data, residing in different caches will become different. When P2 operates on M1, it uses the stale value of M1 that was stored in its cache. It is responsibility of Cache Coherence Protocol to prevent this. Hardware support is needed to provide a coherent view of data in multiple caches. This is known as write propagation requirement.

One may think that cache write policy can provide cache coherence, but this is incorrect. Cache write policy only controls how a change in value of cache is propagated to lower level cache or main memory. It is not responsible for propagating changes to other caches.

==Cache Coherence Protocols==

The two basic methods to utilize the inter-core bus to notify other cores when a core changes something in its cache are '''update''' and '''invalidate'''. In the update method, if variable 'x' is modified by core 1, core 1 has to send the updated value of 'x' onto the inter-core bus. Each cache listens to the inter-core bus and if a cache sees a variable on the bus which it has a copy of, it will read the updated value. This ensures that all caches have the most up-to-date value of the variable.[[#3foot|[3]]]

In case of invalidation, an invalidation message is sent onto the inter-core bus when a variable is changed. The other caches will read this invalidation signal and if its core attempts to access that variable, it will result in a cache miss and the variable will be read from main memory.

The update method results in significant amount of traffic on the inter-core bus as the update signal is sent onto the bus every time the variable is updated. The invalidation method only requires that an invalidation signal be sent the first time a variable is altered; this is why the invalidation method is the preferred method.

In order to improve cache coherency performance over the years, the following protocols were proposed:

* '''MSI'''

MSI stands for Modified, Shared, and Invalid, based on the three states that a line of cache can be in. The Modified state means that a variable in the cache has been modified and therefore has a different value than that found in main memory; the cache is responsible for writing the variable back to main memory. The Shared state means that the variable exists in at least one cache and is not modified; the cache can evict the variable without writing it back to the main memory. The Invalid state means that the value of the variable has been modified by another cache and this value is invalid; the cache must read a new value from main memory (or another cache).

*'''MESI'''

MESI stands for Modified, Exclusive, Shared, and Invalid. The Modified and Invalid states are the same for this protocol as they are for the MSI protocol. This protocol introduces a new state; the Exclusive state. The Exclusive state means that the variable is in only this cache and the value of it matches the value within the main memory. This now means that the Shared state indicates that the variable is contained in more than one cache.

* '''MOSI'''

The MOSI protocol is identical to the MSI protocol except that it adds an Owned state. The Owned state means that the processor "Owns" the variable and will provide the current value to other caches when requested (or at least it will decide if it will provide it when asked). This is useful because another cache will not have to read the value from main memory and will receive it from the Owning cache much, much, faster.

* '''MOESI'''

The MOESI protocol is a combination of the MESI and MOSI protocols.

=Memory Consistency Problem=

Memory consistency deals with the ordering of memory operations (load and store) to different memory locations. In a single processor system, code will execute correctly if the compiler preservers the order of the access to synchronization variables and other dependent variables. But in shared memory model with multiple processors, two threads could access a shared data element, such as a synchronization variable, the output of the threads would change based on which thread, accesses the shared data element earlier. If this were to occur, then the program output may not be the value expected. Maintaining program order is very important for memory consistency but it comes with performance degradation.

== Memory Consistency Models ==

The memory consistency model of a shared-memory multiprocessor is a formal speciﬁcation of how the memory system appears to the programmer. It eliminates the gap between the behavior expected by the programmer and the actual behavior supported by a system. Effectively, the consistency model places restrictions on the values that can be returned by a read, in a shared-memory program execution.

In a single processor system, in order to maintain memory consistency, it needs to ensure that the compiler preserves the program order when accessing synchronization variables. But in a multiprocessor system, it is required to ensure that accesses of one processor appear to execute in program order to all other processors, atleast partially.

=== Memory semantics in Uniprocessor systems ===

Uniprocessor languages use simple sequential semantics for memory operations, which allow the programmer to assume that all memory operations will occur one at a time in the sequential order speciﬁed by the program. Thus, the programmer can expect a read to return the value of the last write to the same location before it by the sequential program order. It is sufﬁcient to only maintain uniprocessor data and control dependences. The compiler and hardware can freely reorder operations to different locations if the uniprocessor data and control dependences are respected. This enables compiler optimizations such as register allocation, code motion, and loop transformations, and hardware optimizations, such as pipelining, multiple issue, write buffer bypassing and forwarding, and lockup-free caches, all of which lead to overlapping and reordering of memory operations. [4]

=== Memory semantics in multiprocessor systems ===

Programmer's implicit expectations are:

* memory accesses in a processor takes place according to the program order.
* Each memory access is performed atomically.

A strong consistency model attempting uniprocessor-like consistency could cause global bottleneck, costing performance. Thus, '''''weak''''' consistency models are deployed to improve performance. The advanatges of such models are:

* They support out-of-order execution within individual CPUs
* Relaxes latency issues with near-simultaneous accesses by different CPUs

The following are the various consistency models and it is the programmer who must take into account the memory consistency model to create correct software:

* linearizability (also known as strict or atomic consistency)
* sequential consistency
* causal consistency
* release consistency
* eventual consistency
* delta consistency
* PRAM consistency (also known as FIFO consistency)
* weak consistency
* vector-field consistency
* fork consistency
* serializability
* one-copy serializability
* entry consistency

== Memory Coherence and Shared Virtual Memory ==

The memory coherence problem in a shared virtual memory system and in multicache systems are different. In a multicache multiprocessor, there are processors sharing a physical memory through their private caches. The relatively small size of a cache and the fast bus connection to the shared memory, enables using a sophisticated coherence protocol for the multicache hardware such that the time delay of conflicting writes to a memory location is small. [5]

In contrast, in a shared virtual memory on a loosely coupled multiprocessor which has no physically shared memory, and having a nontrivial communication cost between processors, conflicts are not likely to be solved with negligible delay, and they resemble much more a “page
fault” in a traditional virtual memory system. Thus, there are two design choices that greatly influence the implementation of a shared virtual memory: the granularity of the memory units (i.e., the “page size”)and the strategy for maintaining coherence.

Memory coherence strategies are classified based on how they deal with '''''page synchronization''''' and '''''page ownership'''''. The algorithms for memory coherence depend on the page fault handlers, their servers and the data structures used. So ''page table'' becomes an important part of these protocols.

= Synchronization =

'''Process synchronization''', which is the mainly concerned with different processes committing to a a certain sequence of actions, and '''data synchronization''', which deals with maintaining data integrity across various copies of a dataset, are two two distinct but related concepts. Process synchronization primitives can be used to implement data synchronization.

[http://en.wikipedia.org/wiki/Mutual_exclusion Mutual exclusion] is the main requirement to be fulfilled in order to synchronize processes, and is needed in both single-processor and multiprocessor systems. There are various approaches to provide mutual exclusion in a system:

* Disabling interrupts
* Locks
* Mutex
* Semaphores
* Barriers
* Test and Set

The next section discusses if there is an alternative to implement mutual exclusion without requiring any hardware support. Peterson's algorithm is one such software solution for guaranteeing mutual exclusion.

== Peterson's algorithm ==

This algorithm utilizes the simple idea of combining per-thread '''flags''' to indicate the intent to enter the lock and the '''turn''' variable to determine which thread should enter in the rare case that both wish to enter at the same time. The algorithm is as shown below: [3]

<pre>
void init() {
flag[0] = flag[1] = 0; // 1 -> thread wants to acquire lock (intent)
turn = 0; // whose turn is it? (thread 0 or thread 1?)
}

void lock() {
flag[self] = 1;
turn = 1 - self; // be generous: make it the other thread’s turn
while ((flag[1-self] == 1) && (turn == 1 - self))
; // spin-wait while other thread has intent
// AND it is other thread’s turn
}

void unlock() {
flag[self] = 0; // simply undo your intent
}
</pre>

The advantages of this approach are:

* It does not assume much about the underlying hardware. It relies on the fact that load and store instructions are atomic( even across processors) and they execute in an order.
* It does not require special instructions.

Though Peterson's algorithm has the above advantages, there are a few problems that render it impractical for use:

* Spin-waiting is possible which causes a thread to spend lot of its CPU time waiting for another thread to release the lock.
* Algorithm might not work well with out-of-order execution of instructions supported by most of the modern processors.
* The algorithm also lacks scalability.

Thus, only the software solutions cannot work well. Sufficient amount of hardware and OS support is needed to get the locking work properly.

== Hardware support ==

Exclusive locking assumes the worst and proceeds only after acquiring all locks such that no other thread can interfere. This is a ''pessimistic'' approach. In contrast, the ''optimistic'' approach proceeds with an update, hoping that it can be completed without any interference. This requires ''collision detection'' during the update. The optimistic approach is thus, more efficient in fine-grained operations.

Special instructions are provided by processors designed for multiprocessor operations in order to manage concurrent access to shared variables. Atomic instructions like [http://en.wikipedia.org/wiki/Test_and_Test-and-set test-and-set], [http://en.wikipedia.org/wiki/Fetch-and-add fetch-and-increment] and [http://en.wikipedia.org/wiki/Swap_%28computer_science%29 swap] were sufficient for early processors to implement mutexes for concurrent objects. Today, every modern processor relies on some form of read-modify-write atomic instruction such as [http://en.wikipedia.org/wiki/Compare-and-swap compare-and-swap], [http://en.wikipedia.org/wiki/Load-link/store-conditional LL/SC] etc. for the same.

=Resources=

[[#1body|1.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_jp#Memory_Consistency_Problem 
[[#2body|2.]] http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_ss#Cache_Coherence 
[[#3body|3.]] http://www.windowsnetworking.com/articles_tutorials/Cache-Coherency.html

CSC 456 Spring 2012/ch7 MN

2012-02-27T18:32:42Z

Jmmohund2: /* Memory Coherence and Shared Virtual Memory */

==Introduction==
Though the migration from [http://en.wikipedia.org/wiki/Uniprocessor_system uniprocessor system] to [http://en.wikipedia.org/wiki/Multiprocessing multiprocessing] systems is not new, the world of parallel computers is undergoing a continuous change. Parallel computers, which started as high-end super-computing systems for carrying out huge calculations, are now ubiquitous and are present in all mainstream architectures for servers, desktops, and embedded systems. In order to design parallel architectures to meet programmer's needs and expectations more closely, exciting and challenging changes exist. The three main areas which are being considered by scientists today are: [http://en.wikipedia.org/wiki/Cache_coherence cache coherence], memory consistency and [http://en.wikipedia.org/wiki/Synchronization_%28computer_science%29 synchronization].

=Cache Coherence Problem=

[[Image:sharedmem.jpg|thumbnail|right|600px|Shared Memory system with dedicated Cache for each processor[[#4foot|[4]]]]]

In a system with single processor (single core), maintaining cache coherency is simple and easy but in a multiprocessor system, it is not as simple. Data can be present in any processors cache and protocol needs to ensure that the data is same in all caches. If it cannot ensure that all the caches are same, then it needs to flag a cache line indicating that it is not updated.

In the figure shown here, this is a 4 processor shared memory system where each processor has its own cache. Supposed processor P1 reads memory location M1 and stores it in its local cache. Then, if P2 reads same location memory location then M1 gets stored in P2’s cache. Now, if P1 changes value of M1, two copies of same data, residing in different caches will become different. When P2 operates on M1, it uses the stale value of M1 that was stored in its cache. It is responsibility of Cache Coherence Protocol to prevent this. Hardware support is needed to provide a coherent view of data in multiple caches. This is known as write propagation requirement.

One may think that cache write policy can provide cache coherence but it is not true. Cache write policy only controls how a change in value of cache is propagated to lower level cache or main memory. It is not responsible for propagating changes to other caches.

==Cache Coherence Protocols==

The two basic methods to utilize the inter-core bus to notify other cores when a core changes something in its cache are '''update''' and '''invalidate'''. In the update method, if variable 'x' is modified by core 1, core 1 has to send the updated value of 'x' onto the inter-core bus. Each cache listens to the inter-core bus and if a cache sees a variable on the bus which it has a copy of, it will read the updated value. This ensures that all caches have the most up-to-date value of the variable. [2]

In case of invalidation, an invalidation message is sent onto the inter-core bus when a variable is changed. The other caches will read this invalidation signal and if its core attempts to access that variable, it will result in a cache miss and the variable will be read from main memory.

The update method results in significant amount of traffic on the inter-core bus as the update signal is sent onto the bus every time the variable is updated. The invalidation method only requires that an invalidation signal be sent the first time a variable is altered; this is why the invalidation method is the preferred method.

In order to improve cache coherency performance over the years, the following protocols were proposed:

* '''MSI'''

MSI stands for Modified, Shared, and Invalid, based on the three states that a line of cache can be in. The Modified state means that a variable in the cache has been modified and therefore has a different value than that found in main memory; the cache is responsible for writing the variable back to main memory. The Shared state means that the variable exists in at least one cache and is not modified; the cache can evict the variable without writing it back to the main memory. The Invalid state means that the value of the variable has been modified by another cache and this value is invalid; the cache must read a new value from main memory (or another cache).

*'''MESI'''

MESI stands for Modified, Exclusive, Shared, and Invalid. The Modified and Invalid states are the same for this protocol as they are for the MSI protocol. This protocol introduces a new state; the Exclusive state. The Exclusive state means that the variable is in only this cache and the value of it matches the value within the main memory. This now means that the Shared state indicates that the variable is contained in more than one cache.

* '''MOSI'''

The MOSI protocol is identical to the MSI protocol except that it adds an Owned state. The Owned state means that the processor "Owns" the variable and will provide the current value to other caches when requested (or at least it will decide if it will provide it when asked). This is useful because another cache will not have to read the value from main memory and will receive it from the Owning cache much, much, faster.

* '''MOESI'''

The MOESI protocol is a combination of the MESI and MOSI protocols.

=Memory Consistency Problem=

Memory consistency deals with the ordering of memory operations (load and store) to different memory locations. In a single processor system, code will execute correctly if the compiler preservers the order of the access to synchronization variables and other dependent variables. But in shared memory model with multiple processors, two threads could be access a shared data (something like a synchronization variable) and the output of the threads could change based on which thread can get to the shared data earlier. If this happens, then the program output on uni-processor system and multi-processor program will be different.

Maintaining program order is very important for memory consistency but it comes with performance degradation. Various memory consistency models trades off performance to make programming easy.
== Memory Consistency Models ==

The memory consistency model of a shared-memory multiprocessor is a formal speciﬁcation of how the memory system appears to the programmer. It eliminates the gap between the behavior expected by the programmer and the actual behavior supported by a system. Effectively, the consistency model places restrictions on the values that can be returned by a read in a shared-memory program execution.

In a single processor system, in order to maintain memory consistency, it needs to ensure that the compiler preserves the program order when accessing synchronization variables. But in a multiprocessor system, it is required to ensure that accesses of one processor appear to execute in program order to all other processors, atleast partially.

=== Memory semantics in Uniprocessor systems ===

Uniprocessor languages use simple sequential semantics for memory operations, which allow the programmer to assume that all memory operations will occur one at a time in the sequential order speciﬁed by the program. Thus, the programmer can expect a read to return the value of the last write to the same location before it by the sequential program order. It is sufﬁcient to only maintain uniprocessor data and control dependences. The compiler and hardware can freely reorder operations to different locations if the uniprocessor data and control dependences are respected. This enables compiler optimizations such as register allocation, code motion, and loop transformations, and hardware optimizations, such as pipelining, multiple issue, write buffer bypassing and forwarding, and lockup-free caches, all of which lead to overlapping and reordering of memory operations. [4]

=== Memory semantics in multiprocessor systems ===

Programmer's implicit expectations are:

* memory accesses in a processor takes place according to the program order.
* Each memory access is performed atomically.

A strong consistency model attempting uniprocessor-like consistency could cause global bottleneck, costing performance. Thus, '''''weak''''' consistency models are deployed to improve performance. The advanatges of such models are:

* They support out-of-order execution within individual CPUs
* Relaxes latency issues with near-simultaneous accesses by different CPUs

The following are the various consistency models and it is the programmer who must take into account the memory consistency model to create correct software:

* linearizability (also known as strict or atomic consistency)
* sequential consistency
* causal consistency
* release consistency
* eventual consistency
* delta consistency
* PRAM consistency (also known as FIFO consistency)
* weak consistency
* vector-field consistency
* fork consistency
* serializability
* one-copy serializability
* entry consistency

== Memory Coherence and Shared Virtual Memory ==

The memory coherence problem in a shared virtual memory system and in multicache systems are different. In a multicache multiprocessor, there are processors sharing a physical memory through their private caches. The relatively small size of a cache and the fast bus connection to the shared memory, enables using a sophisticated coherence protocol for the multicache hardware such that the time delay of conflicting writes to a memory location is small. [5]

In contrast, in a shared virtual memory on a loosely coupled multiprocessor which has no physically shared memory, and having a nontrivial communication cost between processors, conflicts are not likely to be solved with negligible delay, and they resemble much more a “page
fault” in a traditional virtual memory system. Thus, there are two design choices that greatly influence the implementation of a shared virtual memory: the granularity of the memory units (i.e., the “page size”)and the strategy for maintaining coherence.

Memory coherence strategies are classified based on how they deal with '''''page synchronization''''' and '''''page ownership'''''. The algorithms for memory coherence depend on the page fault handlers, their servers and the data structures used. So ''page table'' becomes an important part of these protocols.

= Synchronization =

'''Process synchronization''', which is the mainly concerned with different processes committing to a a certain sequence of actions, and '''data synchronization''', which deals with maintaining data integrity across various copies of a dataset, are two two distinct but related concepts. Process synchronization primitives can be used to implement data synchronization.

[http://en.wikipedia.org/wiki/Mutual_exclusion Mutual exclusion] is the main requirement to be fulfilled in order to synchronize processes, and is needed in both single-processor and multiprocessor systems. There are various approaches to provide mutual exclusion in a system:

* Disabling interrupts
* Locks
* Mutex
* Semaphores
* Barriers
* Test and Set

The next section discusses if there is an alternative to implement mutual exclusion without requiring any hardware support. Peterson's algorithm is one such software solution for guaranteeing mutual exclusion.

== Peterson's algorithm ==

This algorithm utilizes the simple idea of combining per-thread '''flags''' to indicate the intent to enter the lock and the '''turn''' variable to determine which thread should enter in the rare case that both wish to enter at the same time. The algorithm is as shown below: [3]

<pre>
void init() {
flag[0] = flag[1] = 0; // 1 -> thread wants to acquire lock (intent)
turn = 0; // whose turn is it? (thread 0 or thread 1?)
}

void lock() {
flag[self] = 1;
turn = 1 - self; // be generous: make it the other thread’s turn
while ((flag[1-self] == 1) && (turn == 1 - self))
; // spin-wait while other thread has intent
// AND it is other thread’s turn
}

void unlock() {
flag[self] = 0; // simply undo your intent
}
</pre>

The advantages of this approach are:

* It does not assume much about the underlying hardware. It relies on the fact that load and store instructions are atomic( even across processors) and they execute in an order.
* It does not require special instructions.

Though Peterson's algorithm has the above advantages, there are a few problems that render it impractical for use:

* Spin-waiting is possible which causes a thread to spend lot of its CPU time waiting for another thread to release the lock.
* Algorithm might not work well with out-of-order execution of instructions supported by most of the modern processors.
* The algorithm also lacks scalability.

Thus, only the software solutions cannot work well. Sufficient amount of hardware and OS support is needed to get the locking work properly.

== Hardware support ==

Exclusive locking assumes the worst and proceeds only after acquiring all locks such that no other thread can interfere. This is a ''pessimistic'' approach. In contrast, the ''optimistic'' approach proceeds with an update, hoping that it can be completed without any interference. This requires ''collision detection'' during the update. The optimistic approach is thus, more efficient in fine-grained operations.

Special instructions are provided by processors designed for multiprocessor operations in order to manage concurrent access to shared variables. Atomic instructions like [http://en.wikipedia.org/wiki/Test_and_Test-and-set test-and-set], [http://en.wikipedia.org/wiki/Fetch-and-add fetch-and-increment] and [http://en.wikipedia.org/wiki/Swap_%28computer_science%29 swap] were sufficient for early processors to implement mutexes for concurrent objects. Today, every modern processor relies on some form of read-modify-write atomic instruction such as [http://en.wikipedia.org/wiki/Compare-and-swap compare-and-swap], [http://en.wikipedia.org/wiki/Load-link/store-conditional LL/SC] etc. for the same.

=Resources=
1. http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_jp#Memory_Consistency_Problem 
2. http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2011/ch7_ss#Cache_Coherence

CSC 456 Spring 2012/ch7 MN

2012-02-27T18:31:55Z

Jmmohund2: /* Memory semantics in multiprocessor systems */

CSC 456 Spring 2012/ch7 MN

2012-02-27T18:31:01Z

Jmmohund2: /* Memory Consistency Problem */

CSC 456 Spring 2012/ch7 MN

2012-02-27T18:30:11Z

Jmmohund2: /* Cache Coherence Problem */