Expertiza_Wiki - User contributions [en]

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-19T01:48:26Z

Eapotter:

__TOC__

(Unless otherwise noted, the contents of this article is derived from the IEEE standardization document titled "IEEE Standard for Scalable Coherent Interface (SCI)"[[#References|[1]]] .)

== Background ==
The Scalable Coherent Interface (SCI) cache coherence protocol defines a set of states for memory, a set of states for cache lines, and a set of actions that transition between these states when processors access memory.

Three sets of these attributes are defined for minimal, typical, and full applications:

* '''Minimal''' - for ‘trivial but correct’ applications that require the presence of the memory in only one cache line. It does not enable read-sharing, and is appropriate for small multi-processors or applications that don’t require significant sharing.

* '''Typical''' - enables read-sharing of a memory location and provisions for efficiency, such as DMA transfers, local caching of data, and error recovery. This set adds an additional stable memory state (FRESH) and multiple cache states. This set will be the focus of this article going forward.

* '''Full''' - adds support for pair-wise sharing, QOLB lock bit synchronization, and cleansing and washing of cache lines.

== State Diagrams ==
=== Memory States ===
The memory states define the state of the memory block from the perspective of the home directory. This 2-bit field is maintained in the memory tag by the home directory along with the pointer to the head of the sharing list (forwId). This simple state model includes three stable states and one semi-stable state.

{|border="1" cellspacing="0" cellpadding="2"
!colspan="5"| Table 1: Memory States
|----
!State
!Description
!Minimal
!Typical
!Full
|----
|HOME
|no sharing list
|Y
|Y
|Y
|----
|FRESH
|sharing-list copy is the same as memory
|
|Y
|Y
|----
|GONE
|sharing-list copy may be different from memory
|Y
|Y
|Y
|----
|WASH*
|transitional state (GONE to FRESH)
|
|
|Y
|----
|}

These states are used by the coherency model to determine the state of the cache line when memory transactions are performed by individual processors. Likewise, these memory states are impacted when these memory transactions occur.

Below is a state diagram that depicts the transition between these memory states:

[[Image:memory_state.png|center]]

=== Cache States ===
The cache-line states are maintained by each processors cache-coherency controller. This 7-bit field is stored in each cache line in the sharing list, along with the pointer to the next sharing-list node (forwId) and the previous sharing-list node (backID). Seven bits enable up to 128 possible values, and SCI defines twenty-nine stable-states for use in the minimal, typical, and full sets.

Stable states are those cache states that exist when a memory transaction is not in process. Their names are derived from a combination of
the position of the node in the sharing list, such as ONLY, HEAD, MID, and TAIL, and the state of the data for that node, such as FRESH, CLEAN, DIRTY, VALID, STALE, etc.

====States of the Typical Set====

Following are the states defined for the Typical set. (Note that other states may be supported to provide certain interoperability with nodes implementing the Full set, but are not used when all nodes support the Typical set.)

* '''ONLY_DIRTY''' - only one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and no other processor currently caches the block.
* '''ONLY_FRESH''' - only one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and no other processor currently caches the block.
* '''HEAD_DIRTY''' - more than one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and another processors already caches the block.
* '''HEAD_FRESH''' - more than one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and another processors already caches the block.
* '''MID_VALID''' - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is neither the Head or the Tail of the of the sharing list.
* '''TAIL_VALID''' - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is the Tail of the of the sharing list.

Below is a state diagram that depicts these states:

[[Image:cache_states.png|center]]

The request sub-actions of memory transactions that expect a response from the directory include 17 variations of reads, writes, and locks based on the number of bytes requested and the coherency requirements. For convenience, the notation on the edges of the above graph is derived from the terms used in the SCI specification to describe the memory transactions.

The first value indicates the request to the directory by a node for memory access:
* '''Fetch R''' - indicates a request for a memory block with read privileges.
* '''Fetch RW''' - indicates a request for a memory block with read/write privileges.
* '''Data Modify''' - indicates that a processor (the HEAD or the ONLY processor caching the block) writes to the cache line.

The value in the parenthesis indicates the memory state at the time of the request.

== Race Conditions ==
In a distributed shared memory system with caching, the emergence of race conditions is extremely likely. This is mainly due to the lack of a bus for serialization of actions. It is further compounded by the problem of network errors and congestion.

The early invalidation case from Section 11.4 in Solihin is an excellent example of a race condition that can arise in a distributed system. Recall the diagram from the text, and the cache coherence actions, as shown below.

[[Image:EarlyInValidationRace.png|400px|center]]
The circled actions are as follows:
# ''A'' sends a read request to ''Home''.
# ''Home'' replies with data (but the message gets delayed).
# ''B'' sends a write request to ''Home''.
# ''Home'' sends invalidation to ''A'', and it arrives before the ReplyD

The SCI protocol has a way to handle this race condition, as well as many others, and the following sections will discuss how the SCI protocol design can prevent this race condition.

=== Prevention in the SCI Protocol ===
Race conditions are almost non-existent in the SCI protocol, due primarily to the protocol's design. A brief discussion of how this is accomplished follows, followed by a discussion showing what happens if this condition arises in the SCI protocol.
==== Atomic Transactions ====
SCI's primary method for preventing race conditions is having atomic transactions. A transaction is defined as a set of sub-actions necessary to complete some requested action, such as querying memory for a data or invalidating a node's cache. This usually takes the form a a request-response pair, where the initiating node sends a request to another node and waits for the response. The transaction is not complete until the response returns. Suppose, for example, that node ''A'' is in the middle of a transaction with node ''B''. Node ''C'' then tries to make a request of node ''A''. Node ''A'' will respond to node ''C'' that it is busy, telling node ''C'' to try again, as shown in the following diagram.
[[Image:AtomicBusy.png|center]]
Thus, in the Early Invalidation race, node ''A'' would be in the middle of a transaction, having requested a block of data from memory and not having received a response, which would prevent node ''B'' from invalidating it. Note, however, that nodes will not stay in the busy state indefinitely. Doing so would lead to potential deadlocks, so all transactions have time outs that will eventually cause the transaction to fail, thus moving the node out of the busy state, making it able to respond to requests again.

==== Head Node ====
Another method for preventing race conditions is to have the ''Head'' node of a sharing list perform many of the coherence actions. As a result, only one node is performing actions such as writes and invalidations of other sharers. Since only one node is performing these actions, the possibility of concurrent actions is decreased. If another node wants to write, for instance, it must become the Head Node of the sharing list for the cache line to which it wants to write, displacing the current ''Head'' node as the node that can write. All state changes in the sharing list occur at the behest of the ''Head'' node, with the exception of nodes deleting themselves from the list.

When simply sharing a read-only copy of a node, as in when the memory is a ''FRESH'' state, the ''Head'' node is somewhat irrelevant. All the sharing nodes have their own cached value of the cache line. If any node wants to write, including the ''Head'' node, it must perform an additional action in order to do so. Likewise, when a sharing list is sharing a line of data in the dirty, or ''GONE'', memory state, as long as the ''Head'' node has not written to the line, then all the sharing nodes stay in the list with their cached line. However, at the point the ''Head'' node wants to write, it can write its data immediately, but it then must invalidate all other shared copies, via the forward pointers in the sharing list.

This, however, does not protect against a race condition where the memory is ''FRESH'', the ''Head'' node wants to write, and another node also wants to join the list. The Memory Access mechanism prevents this condition.

==== Memory Access ====
The final way that the SCI protocol minimizes race conditions is by changing the directory structure. In Solihin 11.4, there is a ''Home'' node which keeps track of the cache state and has a certain amount of knowledge about the system as a result. In the SCI protocol, there is still a ''Home'' node of sorts, but this node is only responsible for keeping track of who the current ''Head''. Such a ''Home'' node is usually the node where the memory block physically resides. When a node wants to access a block of memory, it sends a request to the ''Home'' node of that memory block, assuming it isn't already in the sharing list for that block. In the event that the data is already being used, the ''Home'' node will include in its response the address of the ''Head'' node of the sharing list. This requirement necessarily serializes the order of access, as one access request is not serviced at the ''Home'' node until the previous request is finished, and the requests are processed in FIFO order. As well, whichever node is the ''Head'' of the sharing list will be notified by the new ''Head'' when it is demoted. All parties involved in a potential race are thus talking to each other, preventing the race from occurring in the first place.

Because of this requirement, the projected race condition from the previous section, where the ''Head'' node is in a ''FRESH'' state and wants to write while another node also wants to read, cannot occur. This is because both nodes must make their request to the ''Home'' node; the ''Head'' node must request the ability to write, and the other node must request the data from the ''Home'' node. One of these requests will make it to the ''Home'' node first, resulting in the second request being deferred to the first node to reach to ''Home'' node. Such a scenario is pictured below.
[[Image:MemoryAccess.png|center]]

==== Solihin 11.4 Resolution ====
The three previous sections discussed individual parts of how the SCI protocol reduces race conditions. Putting all three of these parts together yields the following scenario which resolves the Early Invalidation race as described in the text.
[[Image:SolihinSCI.png|center]]

# ''A'' sends a request to ''Home'' for access to the memory block. It then goes into a busy state while it waits for a response.
# ''B'' also sends a request to ''Home'' for access to the same memory block. ''A'''s request is received first
# ''Home'' responds to ''A'' with the data, but this response gets caught in network traffic and delayed. This response is sent before ''Home'' processes the request from ''B''.
# ''Home'' responds to the request from ''B'', telling it that ''A'' is the current ''Head'' node.
# ''B'' then sends a request to ''A'' to tell it to demote itself.
# ''A'' still hasn't received the response from ''Home'', so it is still in a transaction, and it tells ''B'' that it is busy. ''B'' will have to retry the request.

It should be noted that ''B'' will be in the ''PENDING''[[#Definitions_and_Terms|[def]]] state, so any requests to demote it from the ''Head'' position will be delayed.. This could lead to a chaining of nodes in the ''PENDING'' state, but as soon as node ''A'' receives its response from ''Home'' or times out, the requests in the chain will be successively accepted and processed.

=== Other Possible Race Conditions ===
The design of the SCI protocol prevents many race conditions from occurring, as has already been shown. Here are two other conditions that may arise, with a discussion of how the protocol resolves the issue.

==== Concurrent List Deletions ====

==== Simultaneous Deletion and Invalidation ====
Even though the ''Head'' node performs most of the coherence actions for a shard list, the individual nodes are able to invalidate themselves, such as when they evict the block from their cache. This is referred to in the SCI standard as a "deletion" from the sharing list. Deletion is accomplished by having the invalidating node "lock" itself and then inform its forward and back nodes that they should now point to each other. This "locking" is essentially another ''PENDING'' state. A problem could arise, then, when two neighboring nodes try to invalidate themselves at the same time, as they would both be locked and not respond to each other's message. In this case, though, the protocol specifies that the node that is closest to the tail takes priority. So, the deadlock and/or race condition is averted by ordering the deletion from the tail towards the head.

== Definitions and Terms ==
* ''Head Node'' - The node at the beginning of the sharing list
* ''Home Node'' - The node responsible for keeping track of the ''Head'' node for a sharing list for a memory block, usually the node at which the memory block resides.
* ''PENDING state'' - The intermediate state for a node while it is attempting to become the ''Head'' node in the sharing list. This includes the transaction involved in querying the ''Home'' node and the transaction involved in demoting the current ''Head'' node.
* ''SCI'' - Scalable Coherent Interface

== References ==
[[#1body|1.]] "IEEE Standard for Scalable Coherent Interface (SCI).," IEEE Std 1596-1992 , vol., no., pp.i, 1993. doi: 10.1109/IEEESTD.1993.120366
URL: http://ieeexplore.ieee.org.proxied.lib.ncsu.edu/stamp/stamp.jsp?tp=&arnumber=347683&isnumber=8049 
[[#2body|2.]] [http://www.scizzl.com/HowSCIcohWorks.html How SCI Coherence Works]

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-19T01:48:10Z

Eapotter:

__TOC__

(Unless otherwise noted, the contents of this article is derived from the IEEE standardization document titled "IEEE Standard for Scalable Coherent Interface (SCI)"[[#References|[def]]] .)

== Background ==
The Scalable Coherent Interface (SCI) cache coherence protocol defines a set of states for memory, a set of states for cache lines, and a set of actions that transition between these states when processors access memory.

Three sets of these attributes are defined for minimal, typical, and full applications:

* '''Minimal''' - for ‘trivial but correct’ applications that require the presence of the memory in only one cache line. It does not enable read-sharing, and is appropriate for small multi-processors or applications that don’t require significant sharing.

* '''Typical''' - enables read-sharing of a memory location and provisions for efficiency, such as DMA transfers, local caching of data, and error recovery. This set adds an additional stable memory state (FRESH) and multiple cache states. This set will be the focus of this article going forward.

* '''Full''' - adds support for pair-wise sharing, QOLB lock bit synchronization, and cleansing and washing of cache lines.

== State Diagrams ==
=== Memory States ===
The memory states define the state of the memory block from the perspective of the home directory. This 2-bit field is maintained in the memory tag by the home directory along with the pointer to the head of the sharing list (forwId). This simple state model includes three stable states and one semi-stable state.

{|border="1" cellspacing="0" cellpadding="2"
!colspan="5"| Table 1: Memory States
|----
!State
!Description
!Minimal
!Typical
!Full
|----
|HOME
|no sharing list
|Y
|Y
|Y
|----
|FRESH
|sharing-list copy is the same as memory
|
|Y
|Y
|----
|GONE
|sharing-list copy may be different from memory
|Y
|Y
|Y
|----
|WASH*
|transitional state (GONE to FRESH)
|
|
|Y
|----
|}

These states are used by the coherency model to determine the state of the cache line when memory transactions are performed by individual processors. Likewise, these memory states are impacted when these memory transactions occur.

Below is a state diagram that depicts the transition between these memory states:

[[Image:memory_state.png|center]]

=== Cache States ===
The cache-line states are maintained by each processors cache-coherency controller. This 7-bit field is stored in each cache line in the sharing list, along with the pointer to the next sharing-list node (forwId) and the previous sharing-list node (backID). Seven bits enable up to 128 possible values, and SCI defines twenty-nine stable-states for use in the minimal, typical, and full sets.

Stable states are those cache states that exist when a memory transaction is not in process. Their names are derived from a combination of
the position of the node in the sharing list, such as ONLY, HEAD, MID, and TAIL, and the state of the data for that node, such as FRESH, CLEAN, DIRTY, VALID, STALE, etc.

====States of the Typical Set====

Following are the states defined for the Typical set. (Note that other states may be supported to provide certain interoperability with nodes implementing the Full set, but are not used when all nodes support the Typical set.)

* '''ONLY_DIRTY''' - only one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and no other processor currently caches the block.
* '''ONLY_FRESH''' - only one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and no other processor currently caches the block.
* '''HEAD_DIRTY''' - more than one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and another processors already caches the block.
* '''HEAD_FRESH''' - more than one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and another processors already caches the block.
* '''MID_VALID''' - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is neither the Head or the Tail of the of the sharing list.
* '''TAIL_VALID''' - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is the Tail of the of the sharing list.

Below is a state diagram that depicts these states:

[[Image:cache_states.png|center]]

The request sub-actions of memory transactions that expect a response from the directory include 17 variations of reads, writes, and locks based on the number of bytes requested and the coherency requirements. For convenience, the notation on the edges of the above graph is derived from the terms used in the SCI specification to describe the memory transactions.

The first value indicates the request to the directory by a node for memory access:
* '''Fetch R''' - indicates a request for a memory block with read privileges.
* '''Fetch RW''' - indicates a request for a memory block with read/write privileges.
* '''Data Modify''' - indicates that a processor (the HEAD or the ONLY processor caching the block) writes to the cache line.

The value in the parenthesis indicates the memory state at the time of the request.

== Race Conditions ==
In a distributed shared memory system with caching, the emergence of race conditions is extremely likely. This is mainly due to the lack of a bus for serialization of actions. It is further compounded by the problem of network errors and congestion.

The early invalidation case from Section 11.4 in Solihin is an excellent example of a race condition that can arise in a distributed system. Recall the diagram from the text, and the cache coherence actions, as shown below.

[[Image:EarlyInValidationRace.png|400px|center]]
The circled actions are as follows:
# ''A'' sends a read request to ''Home''.
# ''Home'' replies with data (but the message gets delayed).
# ''B'' sends a write request to ''Home''.
# ''Home'' sends invalidation to ''A'', and it arrives before the ReplyD

The SCI protocol has a way to handle this race condition, as well as many others, and the following sections will discuss how the SCI protocol design can prevent this race condition.

=== Prevention in the SCI Protocol ===
Race conditions are almost non-existent in the SCI protocol, due primarily to the protocol's design. A brief discussion of how this is accomplished follows, followed by a discussion showing what happens if this condition arises in the SCI protocol.
==== Atomic Transactions ====
SCI's primary method for preventing race conditions is having atomic transactions. A transaction is defined as a set of sub-actions necessary to complete some requested action, such as querying memory for a data or invalidating a node's cache. This usually takes the form a a request-response pair, where the initiating node sends a request to another node and waits for the response. The transaction is not complete until the response returns. Suppose, for example, that node ''A'' is in the middle of a transaction with node ''B''. Node ''C'' then tries to make a request of node ''A''. Node ''A'' will respond to node ''C'' that it is busy, telling node ''C'' to try again, as shown in the following diagram.
[[Image:AtomicBusy.png|center]]
Thus, in the Early Invalidation race, node ''A'' would be in the middle of a transaction, having requested a block of data from memory and not having received a response, which would prevent node ''B'' from invalidating it. Note, however, that nodes will not stay in the busy state indefinitely. Doing so would lead to potential deadlocks, so all transactions have time outs that will eventually cause the transaction to fail, thus moving the node out of the busy state, making it able to respond to requests again.

==== Head Node ====
Another method for preventing race conditions is to have the ''Head'' node of a sharing list perform many of the coherence actions. As a result, only one node is performing actions such as writes and invalidations of other sharers. Since only one node is performing these actions, the possibility of concurrent actions is decreased. If another node wants to write, for instance, it must become the Head Node of the sharing list for the cache line to which it wants to write, displacing the current ''Head'' node as the node that can write. All state changes in the sharing list occur at the behest of the ''Head'' node, with the exception of nodes deleting themselves from the list.

When simply sharing a read-only copy of a node, as in when the memory is a ''FRESH'' state, the ''Head'' node is somewhat irrelevant. All the sharing nodes have their own cached value of the cache line. If any node wants to write, including the ''Head'' node, it must perform an additional action in order to do so. Likewise, when a sharing list is sharing a line of data in the dirty, or ''GONE'', memory state, as long as the ''Head'' node has not written to the line, then all the sharing nodes stay in the list with their cached line. However, at the point the ''Head'' node wants to write, it can write its data immediately, but it then must invalidate all other shared copies, via the forward pointers in the sharing list.

This, however, does not protect against a race condition where the memory is ''FRESH'', the ''Head'' node wants to write, and another node also wants to join the list. The Memory Access mechanism prevents this condition.

==== Memory Access ====
The final way that the SCI protocol minimizes race conditions is by changing the directory structure. In Solihin 11.4, there is a ''Home'' node which keeps track of the cache state and has a certain amount of knowledge about the system as a result. In the SCI protocol, there is still a ''Home'' node of sorts, but this node is only responsible for keeping track of who the current ''Head''. Such a ''Home'' node is usually the node where the memory block physically resides. When a node wants to access a block of memory, it sends a request to the ''Home'' node of that memory block, assuming it isn't already in the sharing list for that block. In the event that the data is already being used, the ''Home'' node will include in its response the address of the ''Head'' node of the sharing list. This requirement necessarily serializes the order of access, as one access request is not serviced at the ''Home'' node until the previous request is finished, and the requests are processed in FIFO order. As well, whichever node is the ''Head'' of the sharing list will be notified by the new ''Head'' when it is demoted. All parties involved in a potential race are thus talking to each other, preventing the race from occurring in the first place.

Because of this requirement, the projected race condition from the previous section, where the ''Head'' node is in a ''FRESH'' state and wants to write while another node also wants to read, cannot occur. This is because both nodes must make their request to the ''Home'' node; the ''Head'' node must request the ability to write, and the other node must request the data from the ''Home'' node. One of these requests will make it to the ''Home'' node first, resulting in the second request being deferred to the first node to reach to ''Home'' node. Such a scenario is pictured below.
[[Image:MemoryAccess.png|center]]

==== Solihin 11.4 Resolution ====
The three previous sections discussed individual parts of how the SCI protocol reduces race conditions. Putting all three of these parts together yields the following scenario which resolves the Early Invalidation race as described in the text.
[[Image:SolihinSCI.png|center]]

# ''A'' sends a request to ''Home'' for access to the memory block. It then goes into a busy state while it waits for a response.
# ''B'' also sends a request to ''Home'' for access to the same memory block. ''A'''s request is received first
# ''Home'' responds to ''A'' with the data, but this response gets caught in network traffic and delayed. This response is sent before ''Home'' processes the request from ''B''.
# ''Home'' responds to the request from ''B'', telling it that ''A'' is the current ''Head'' node.
# ''B'' then sends a request to ''A'' to tell it to demote itself.
# ''A'' still hasn't received the response from ''Home'', so it is still in a transaction, and it tells ''B'' that it is busy. ''B'' will have to retry the request.

It should be noted that ''B'' will be in the ''PENDING''[[#Definitions_and_Terms|[def]]] state, so any requests to demote it from the ''Head'' position will be delayed.. This could lead to a chaining of nodes in the ''PENDING'' state, but as soon as node ''A'' receives its response from ''Home'' or times out, the requests in the chain will be successively accepted and processed.

=== Other Possible Race Conditions ===
The design of the SCI protocol prevents many race conditions from occurring, as has already been shown. Here are two other conditions that may arise, with a discussion of how the protocol resolves the issue.

==== Concurrent List Deletions ====

==== Simultaneous Deletion and Invalidation ====
Even though the ''Head'' node performs most of the coherence actions for a shard list, the individual nodes are able to invalidate themselves, such as when they evict the block from their cache. This is referred to in the SCI standard as a "deletion" from the sharing list. Deletion is accomplished by having the invalidating node "lock" itself and then inform its forward and back nodes that they should now point to each other. This "locking" is essentially another ''PENDING'' state. A problem could arise, then, when two neighboring nodes try to invalidate themselves at the same time, as they would both be locked and not respond to each other's message. In this case, though, the protocol specifies that the node that is closest to the tail takes priority. So, the deadlock and/or race condition is averted by ordering the deletion from the tail towards the head.

== Definitions and Terms ==
* ''Head Node'' - The node at the beginning of the sharing list
* ''Home Node'' - The node responsible for keeping track of the ''Head'' node for a sharing list for a memory block, usually the node at which the memory block resides.
* ''PENDING state'' - The intermediate state for a node while it is attempting to become the ''Head'' node in the sharing list. This includes the transaction involved in querying the ''Home'' node and the transaction involved in demoting the current ''Head'' node.
* ''SCI'' - Scalable Coherent Interface

== References ==
[[#1body|1.]] "IEEE Standard for Scalable Coherent Interface (SCI).," IEEE Std 1596-1992 , vol., no., pp.i, 1993. doi: 10.1109/IEEESTD.1993.120366
URL: http://ieeexplore.ieee.org.proxied.lib.ncsu.edu/stamp/stamp.jsp?tp=&arnumber=347683&isnumber=8049 
[[#2body|2.]] [http://www.scizzl.com/HowSCIcohWorks.html How SCI Coherence Works]

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-19T01:45:57Z

Eapotter: /* Background */

__TOC__

(Unless otherwise noted, the contents of this article is derived from the IEEE standardization document titled "IEEE Standard for Scalable Coherent Interface (SCI)".)

== Background ==
The Scalable Coherent Interface (SCI) cache coherence protocol defines a set of states for memory, a set of states for cache lines, and a set of actions that transition between these states when processors access memory.

Three sets of these attributes are defined for minimal, typical, and full applications:

* '''Minimal''' - for ‘trivial but correct’ applications that require the presence of the memory in only one cache line. It does not enable read-sharing, and is appropriate for small multi-processors or applications that don’t require significant sharing.

* '''Typical''' - enables read-sharing of a memory location and provisions for efficiency, such as DMA transfers, local caching of data, and error recovery. This set adds an additional stable memory state (FRESH) and multiple cache states. This set will be the focus of this article going forward.

* '''Full''' - adds support for pair-wise sharing, QOLB lock bit synchronization, and cleansing and washing of cache lines.

== State Diagrams ==
=== Memory States ===
The memory states define the state of the memory block from the perspective of the home directory. This 2-bit field is maintained in the memory tag by the home directory along with the pointer to the head of the sharing list (forwId). This simple state model includes three stable states and one semi-stable state.

{|border="1" cellspacing="0" cellpadding="2"
!colspan="5"| Table 1: Memory States
|----
!State
!Description
!Minimal
!Typical
!Full
|----
|HOME
|no sharing list
|Y
|Y
|Y
|----
|FRESH
|sharing-list copy is the same as memory
|
|Y
|Y
|----
|GONE
|sharing-list copy may be different from memory
|Y
|Y
|Y
|----
|WASH*
|transitional state (GONE to FRESH)
|
|
|Y
|----
|}

These states are used by the coherency model to determine the state of the cache line when memory transactions are performed by individual processors. Likewise, these memory states are impacted when these memory transactions occur.

Below is a state diagram that depicts the transition between these memory states:

[[Image:memory_state.png|center]]

=== Cache States ===
The cache-line states are maintained by each processors cache-coherency controller. This 7-bit field is stored in each cache line in the sharing list, along with the pointer to the next sharing-list node (forwId) and the previous sharing-list node (backID). Seven bits enable up to 128 possible values, and SCI defines twenty-nine stable-states for use in the minimal, typical, and full sets.

Stable states are those cache states that exist when a memory transaction is not in process. Their names are derived from a combination of
the position of the node in the sharing list, such as ONLY, HEAD, MID, and TAIL, and the state of the data for that node, such as FRESH, CLEAN, DIRTY, VALID, STALE, etc.

====States of the Typical Set====

Following are the states defined for the Typical set. (Note that other states may be supported to provide certain interoperability with nodes implementing the Full set, but are not used when all nodes support the Typical set.)

* '''ONLY_DIRTY''' - only one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and no other processor currently caches the block.
* '''ONLY_FRESH''' - only one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and no other processor currently caches the block.
* '''HEAD_DIRTY''' - more than one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and another processors already caches the block.
* '''HEAD_FRESH''' - more than one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and another processors already caches the block.
* '''MID_VALID''' - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is neither the Head or the Tail of the of the sharing list.
* '''TAIL_VALID''' - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is the Tail of the of the sharing list.

Below is a state diagram that depicts these states:

[[Image:cache_states.png|center]]

The request sub-actions of memory transactions that expect a response from the directory include 17 variations of reads, writes, and locks based on the number of bytes requested and the coherency requirements. For convenience, the notation on the edges of the above graph is derived from the terms used in the SCI specification to describe the memory transactions.

The first value indicates the request to the directory by a node for memory access:
* '''Fetch R''' - indicates a request for a memory block with read privileges.
* '''Fetch RW''' - indicates a request for a memory block with read/write privileges.
* '''Data Modify''' - indicates that a processor (the HEAD or the ONLY processor caching the block) writes to the cache line.

The value in the parenthesis indicates the memory state at the time of the request.

== Race Conditions ==
In a distributed shared memory system with caching, the emergence of race conditions is extremely likely. This is mainly due to the lack of a bus for serialization of actions. It is further compounded by the problem of network errors and congestion.

The early invalidation case from Section 11.4 in Solihin is an excellent example of a race condition that can arise in a distributed system. Recall the diagram from the text, and the cache coherence actions, as shown below.

[[Image:EarlyInValidationRace.png|400px|center]]
The circled actions are as follows:
# ''A'' sends a read request to ''Home''.
# ''Home'' replies with data (but the message gets delayed).
# ''B'' sends a write request to ''Home''.
# ''Home'' sends invalidation to ''A'', and it arrives before the ReplyD

The SCI protocol has a way to handle this race condition, as well as many others, and the following sections will discuss how the SCI protocol design can prevent this race condition.

=== Prevention in the SCI Protocol ===
Race conditions are almost non-existent in the SCI protocol, due primarily to the protocol's design. A brief discussion of how this is accomplished follows, followed by a discussion showing what happens if this condition arises in the SCI protocol.
==== Atomic Transactions ====
SCI's primary method for preventing race conditions is having atomic transactions. A transaction is defined as a set of sub-actions necessary to complete some requested action, such as querying memory for a data or invalidating a node's cache. This usually takes the form a a request-response pair, where the initiating node sends a request to another node and waits for the response. The transaction is not complete until the response returns. Suppose, for example, that node ''A'' is in the middle of a transaction with node ''B''. Node ''C'' then tries to make a request of node ''A''. Node ''A'' will respond to node ''C'' that it is busy, telling node ''C'' to try again, as shown in the following diagram.
[[Image:AtomicBusy.png|center]]
Thus, in the Early Invalidation race, node ''A'' would be in the middle of a transaction, having requested a block of data from memory and not having received a response, which would prevent node ''B'' from invalidating it. Note, however, that nodes will not stay in the busy state indefinitely. Doing so would lead to potential deadlocks, so all transactions have time outs that will eventually cause the transaction to fail, thus moving the node out of the busy state, making it able to respond to requests again.

==== Head Node ====
Another method for preventing race conditions is to have the ''Head'' node of a sharing list perform many of the coherence actions. As a result, only one node is performing actions such as writes and invalidations of other sharers. Since only one node is performing these actions, the possibility of concurrent actions is decreased. If another node wants to write, for instance, it must become the Head Node of the sharing list for the cache line to which it wants to write, displacing the current ''Head'' node as the node that can write. All state changes in the sharing list occur at the behest of the ''Head'' node, with the exception of nodes deleting themselves from the list.

When simply sharing a read-only copy of a node, as in when the memory is a ''FRESH'' state, the ''Head'' node is somewhat irrelevant. All the sharing nodes have their own cached value of the cache line. If any node wants to write, including the ''Head'' node, it must perform an additional action in order to do so. Likewise, when a sharing list is sharing a line of data in the dirty, or ''GONE'', memory state, as long as the ''Head'' node has not written to the line, then all the sharing nodes stay in the list with their cached line. However, at the point the ''Head'' node wants to write, it can write its data immediately, but it then must invalidate all other shared copies, via the forward pointers in the sharing list.

This, however, does not protect against a race condition where the memory is ''FRESH'', the ''Head'' node wants to write, and another node also wants to join the list. The Memory Access mechanism prevents this condition.

==== Memory Access ====
The final way that the SCI protocol minimizes race conditions is by changing the directory structure. In Solihin 11.4, there is a ''Home'' node which keeps track of the cache state and has a certain amount of knowledge about the system as a result. In the SCI protocol, there is still a ''Home'' node of sorts, but this node is only responsible for keeping track of who the current ''Head''. Such a ''Home'' node is usually the node where the memory block physically resides. When a node wants to access a block of memory, it sends a request to the ''Home'' node of that memory block, assuming it isn't already in the sharing list for that block. In the event that the data is already being used, the ''Home'' node will include in its response the address of the ''Head'' node of the sharing list. This requirement necessarily serializes the order of access, as one access request is not serviced at the ''Home'' node until the previous request is finished, and the requests are processed in FIFO order. As well, whichever node is the ''Head'' of the sharing list will be notified by the new ''Head'' when it is demoted. All parties involved in a potential race are thus talking to each other, preventing the race from occurring in the first place.

Because of this requirement, the projected race condition from the previous section, where the ''Head'' node is in a ''FRESH'' state and wants to write while another node also wants to read, cannot occur. This is because both nodes must make their request to the ''Home'' node; the ''Head'' node must request the ability to write, and the other node must request the data from the ''Home'' node. One of these requests will make it to the ''Home'' node first, resulting in the second request being deferred to the first node to reach to ''Home'' node. Such a scenario is pictured below.
[[Image:MemoryAccess.png|center]]

==== Solihin 11.4 Resolution ====
The three previous sections discussed individual parts of how the SCI protocol reduces race conditions. Putting all three of these parts together yields the following scenario which resolves the Early Invalidation race as described in the text.
[[Image:SolihinSCI.png|center]]

# ''A'' sends a request to ''Home'' for access to the memory block. It then goes into a busy state while it waits for a response.
# ''B'' also sends a request to ''Home'' for access to the same memory block. ''A'''s request is received first
# ''Home'' responds to ''A'' with the data, but this response gets caught in network traffic and delayed. This response is sent before ''Home'' processes the request from ''B''.
# ''Home'' responds to the request from ''B'', telling it that ''A'' is the current ''Head'' node.
# ''B'' then sends a request to ''A'' to tell it to demote itself.
# ''A'' still hasn't received the response from ''Home'', so it is still in a transaction, and it tells ''B'' that it is busy. ''B'' will have to retry the request.

It should be noted that ''B'' will be in the ''PENDING''[[#Definitions_and_Terms|[def]]] state, so any requests to demote it from the ''Head'' position will be delayed.. This could lead to a chaining of nodes in the ''PENDING'' state, but as soon as node ''A'' receives its response from ''Home'' or times out, the requests in the chain will be successively accepted and processed.

=== Other Possible Race Conditions ===
The design of the SCI protocol prevents many race conditions from occurring, as has already been shown. Here are two other conditions that may arise, with a discussion of how the protocol resolves the issue.

==== Concurrent List Deletions ====

==== Simultaneous Deletion and Invalidation ====
Even though the ''Head'' node performs most of the coherence actions for a shard list, the individual nodes are able to invalidate themselves, such as when they evict the block from their cache. This is referred to in the SCI standard as a "deletion" from the sharing list. Deletion is accomplished by having the invalidating node "lock" itself and then inform its forward and back nodes that they should now point to each other. This "locking" is essentially another ''PENDING'' state. A problem could arise, then, when two neighboring nodes try to invalidate themselves at the same time, as they would both be locked and not respond to each other's message. In this case, though, the protocol specifies that the node that is closest to the tail takes priority. So, the deadlock and/or race condition is averted by ordering the deletion from the tail towards the head.

== Definitions and Terms ==
* ''Head Node'' - The node at the beginning of the sharing list
* ''Home Node'' - The node responsible for keeping track of the ''Head'' node for a sharing list for a memory block, usually the node at which the memory block resides.
* ''PENDING state'' - The intermediate state for a node while it is attempting to become the ''Head'' node in the sharing list. This includes the transaction involved in querying the ''Home'' node and the transaction involved in demoting the current ''Head'' node.
* ''SCI'' - Scalable Coherent Interface

== References ==
[[#1body|1.]] "IEEE Standard for Scalable Coherent Interface (SCI).," IEEE Std 1596-1992 , vol., no., pp.i, 1993. doi: 10.1109/IEEESTD.1993.120366
URL: http://ieeexplore.ieee.org.proxied.lib.ncsu.edu/stamp/stamp.jsp?tp=&arnumber=347683&isnumber=8049 
[[#2body|2.]] [http://www.scizzl.com/HowSCIcohWorks.html How SCI Coherence Works]

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-19T01:40:25Z

Eapotter: /* Memory States */

__TOC__

== Background ==
The Scalable Coherent Interface (SCI) cache coherence protocol defines a set of states for memory, a set of states for cache lines, and a set of actions that transition between these states when processors access memory.

Three sets of these attributes are defined for minimal, typical, and full applications:

* '''Minimal''' - for ‘trivial but correct’ applications that require the presence of the memory in only one cache line. It does not enable read-sharing, and is appropriate for small multi-processors or applications that don’t require significant sharing.

* '''Typical''' - enables read-sharing of a memory location and provisions for efficiency, such as DMA transfers, local caching of data, and error recovery. This set adds an additional stable memory state (FRESH) and multiple cache states. This set will be the focus of this article going forward.

* '''Full''' - adds support for pair-wise sharing, QOLB lock bit synchronization, and cleansing and washing of cache lines.

== State Diagrams ==
=== Memory States ===
The memory states define the state of the memory block from the perspective of the home directory. This 2-bit field is maintained in the memory tag by the home directory along with the pointer to the head of the sharing list (forwId). This simple state model includes three stable states and one semi-stable state.

{|border="1" cellspacing="0" cellpadding="2"
!colspan="5"| Table 1: Memory States
|----
!State
!Description
!Minimal
!Typical
!Full
|----
|HOME
|no sharing list
|Y
|Y
|Y
|----
|FRESH
|sharing-list copy is the same as memory
|
|Y
|Y
|----
|GONE
|sharing-list copy may be different from memory
|Y
|Y
|Y
|----
|WASH*
|transitional state (GONE to FRESH)
|
|
|Y
|----
|}

These states are used by the coherency model to determine the state of the cache line when memory transactions are performed by individual processors. Likewise, these memory states are impacted when these memory transactions occur.

Below is a state diagram that depicts the transition between these memory states:

[[Image:memory_state.png|center]]

=== Cache States ===
The cache-line states are maintained by each processors cache-coherency controller. This 7-bit field is stored in each cache line in the sharing list, along with the pointer to the next sharing-list node (forwId) and the previous sharing-list node (backID). Seven bits enable up to 128 possible values, and SCI defines twenty-nine stable-states for use in the minimal, typical, and full sets.

Stable states are those cache states that exist when a memory transaction is not in process. Their names are derived from a combination of
the position of the node in the sharing list, such as ONLY, HEAD, MID, and TAIL, and the state of the data for that node, such as FRESH, CLEAN, DIRTY, VALID, STALE, etc.

====States of the Typical Set====

Following are the states defined for the Typical set. (Note that other states may be supported to provide certain interoperability with nodes implementing the Full set, but are not used when all nodes support the Typical set.)

* '''ONLY_DIRTY''' - only one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and no other processor currently caches the block.
* '''ONLY_FRESH''' - only one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and no other processor currently caches the block.
* '''HEAD_DIRTY''' - more than one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and another processors already caches the block.
* '''HEAD_FRESH''' - more than one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and another processors already caches the block.
* '''MID_VALID''' - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is neither the Head or the Tail of the of the sharing list.
* '''TAIL_VALID''' - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is the Tail of the of the sharing list.

Below is a state diagram that depicts these states:

[[Image:cache_states.png|center]]

The request sub-actions of memory transactions that expect a response from the directory include 17 variations of reads, writes, and locks based on the number of bytes requested and the coherency requirements. For convenience, the notation on the edges of the above graph is derived from the terms used in the SCI specification to describe the memory transactions.

The first value indicates the request to the directory by a node for memory access:
* '''Fetch R''' - indicates a request for a memory block with read privileges.
* '''Fetch RW''' - indicates a request for a memory block with read/write privileges.
* '''Data Modify''' - indicates that a processor (the HEAD or the ONLY processor caching the block) writes to the cache line.

The value in the parenthesis indicates the memory state at the time of the request.

== Race Conditions ==
In a distributed shared memory system with caching, the emergence of race conditions is extremely likely. This is mainly due to the lack of a bus for serialization of actions. It is further compounded by the problem of network errors and congestion.

The early invalidation case from Section 11.4 in Solihin is an excellent example of a race condition that can arise in a distributed system. Recall the diagram from the text, and the cache coherence actions, as shown below.

[[Image:EarlyInValidationRace.png|400px|center]]
The circled actions are as follows:
# ''A'' sends a read request to ''Home''.
# ''Home'' replies with data (but the message gets delayed).
# ''B'' sends a write request to ''Home''.
# ''Home'' sends invalidation to ''A'', and it arrives before the ReplyD

The SCI protocol has a way to handle this race condition, as well as many others, and the following sections will discuss how the SCI protocol design can prevent this race condition.

=== Prevention in the SCI Protocol ===
Race conditions are almost non-existent in the SCI protocol, due primarily to the protocol's design. A brief discussion of how this is accomplished follows, followed by a discussion showing what happens if this condition arises in the SCI protocol.
==== Atomic Transactions ====
SCI's primary method for preventing race conditions is having atomic transactions. A transaction is defined as a set of sub-actions necessary to complete some requested action, such as querying memory for a data or invalidating a node's cache. This usually takes the form a a request-response pair, where the initiating node sends a request to another node and waits for the response. The transaction is not complete until the response returns. Suppose, for example, that node ''A'' is in the middle of a transaction with node ''B''. Node ''C'' then tries to make a request of node ''A''. Node ''A'' will respond to node ''C'' that it is busy, telling node ''C'' to try again, as shown in the following diagram.
[[Image:AtomicBusy.png|center]]
Thus, in the Early Invalidation race, node ''A'' would be in the middle of a transaction, having requested a block of data from memory and not having received a response, which would prevent node ''B'' from invalidating it. Note, however, that nodes will not stay in the busy state indefinitely. Doing so would lead to potential deadlocks, so all transactions have time outs that will eventually cause the transaction to fail, thus moving the node out of the busy state, making it able to respond to requests again.

==== Head Node ====
Another method for preventing race conditions is to have the ''Head'' node of a sharing list perform many of the coherence actions. As a result, only one node is performing actions such as writes and invalidations of other sharers. Since only one node is performing these actions, the possibility of concurrent actions is decreased. If another node wants to write, for instance, it must become the Head Node of the sharing list for the cache line to which it wants to write, displacing the current ''Head'' node as the node that can write. All state changes in the sharing list occur at the behest of the ''Head'' node, with the exception of nodes deleting themselves from the list.

When simply sharing a read-only copy of a node, as in when the memory is a ''FRESH'' state, the ''Head'' node is somewhat irrelevant. All the sharing nodes have their own cached value of the cache line. If any node wants to write, including the ''Head'' node, it must perform an additional action in order to do so. Likewise, when a sharing list is sharing a line of data in the dirty, or ''GONE'', memory state, as long as the ''Head'' node has not written to the line, then all the sharing nodes stay in the list with their cached line. However, at the point the ''Head'' node wants to write, it can write its data immediately, but it then must invalidate all other shared copies, via the forward pointers in the sharing list.

This, however, does not protect against a race condition where the memory is ''FRESH'', the ''Head'' node wants to write, and another node also wants to join the list. The Memory Access mechanism prevents this condition.

==== Memory Access ====
The final way that the SCI protocol minimizes race conditions is by changing the directory structure. In Solihin 11.4, there is a ''Home'' node which keeps track of the cache state and has a certain amount of knowledge about the system as a result. In the SCI protocol, there is still a ''Home'' node of sorts, but this node is only responsible for keeping track of who the current ''Head''. Such a ''Home'' node is usually the node where the memory block physically resides. When a node wants to access a block of memory, it sends a request to the ''Home'' node of that memory block, assuming it isn't already in the sharing list for that block. In the event that the data is already being used, the ''Home'' node will include in its response the address of the ''Head'' node of the sharing list. This requirement necessarily serializes the order of access, as one access request is not serviced at the ''Home'' node until the previous request is finished, and the requests are processed in FIFO order. As well, whichever node is the ''Head'' of the sharing list will be notified by the new ''Head'' when it is demoted. All parties involved in a potential race are thus talking to each other, preventing the race from occurring in the first place.

Because of this requirement, the projected race condition from the previous section, where the ''Head'' node is in a ''FRESH'' state and wants to write while another node also wants to read, cannot occur. This is because both nodes must make their request to the ''Home'' node; the ''Head'' node must request the ability to write, and the other node must request the data from the ''Home'' node. One of these requests will make it to the ''Home'' node first, resulting in the second request being deferred to the first node to reach to ''Home'' node. Such a scenario is pictured below.
[[Image:MemoryAccess.png|center]]

==== Solihin 11.4 Resolution ====
The three previous sections discussed individual parts of how the SCI protocol reduces race conditions. Putting all three of these parts together yields the following scenario which resolves the Early Invalidation race as described in the text.
[[Image:SolihinSCI.png|center]]

# ''A'' sends a request to ''Home'' for access to the memory block. It then goes into a busy state while it waits for a response.
# ''B'' also sends a request to ''Home'' for access to the same memory block. ''A'''s request is received first
# ''Home'' responds to ''A'' with the data, but this response gets caught in network traffic and delayed. This response is sent before ''Home'' processes the request from ''B''.
# ''Home'' responds to the request from ''B'', telling it that ''A'' is the current ''Head'' node.
# ''B'' then sends a request to ''A'' to tell it to demote itself.
# ''A'' still hasn't received the response from ''Home'', so it is still in a transaction, and it tells ''B'' that it is busy. ''B'' will have to retry the request.

It should be noted that ''B'' will be in the ''PENDING''[[#Definitions_and_Terms|[def]]] state, so any requests to demote it from the ''Head'' position will be delayed.. This could lead to a chaining of nodes in the ''PENDING'' state, but as soon as node ''A'' receives its response from ''Home'' or times out, the requests in the chain will be successively accepted and processed.

=== Other Possible Race Conditions ===
The design of the SCI protocol prevents many race conditions from occurring, as has already been shown. Here are two other conditions that may arise, with a discussion of how the protocol resolves the issue.

==== Concurrent List Deletions ====

==== Simultaneous Deletion and Invalidation ====
Even though the ''Head'' node performs most of the coherence actions for a shard list, the individual nodes are able to invalidate themselves, such as when they evict the block from their cache. This is referred to in the SCI standard as a "deletion" from the sharing list. Deletion is accomplished by having the invalidating node "lock" itself and then inform its forward and back nodes that they should now point to each other. This "locking" is essentially another ''PENDING'' state. A problem could arise, then, when two neighboring nodes try to invalidate themselves at the same time, as they would both be locked and not respond to each other's message. In this case, though, the protocol specifies that the node that is closest to the tail takes priority. So, the deadlock and/or race condition is averted by ordering the deletion from the tail towards the head.

== Definitions and Terms ==
* ''Head Node'' - The node at the beginning of the sharing list
* ''Home Node'' - The node responsible for keeping track of the ''Head'' node for a sharing list for a memory block, usually the node at which the memory block resides.
* ''PENDING state'' - The intermediate state for a node while it is attempting to become the ''Head'' node in the sharing list. This includes the transaction involved in querying the ''Home'' node and the transaction involved in demoting the current ''Head'' node.
* ''SCI'' - Scalable Coherent Interface

== References ==
[[#1body|1.]] "IEEE Standard for Scalable Coherent Interface (SCI).," IEEE Std 1596-1992 , vol., no., pp.i, 1993. doi: 10.1109/IEEESTD.1993.120366
URL: http://ieeexplore.ieee.org.proxied.lib.ncsu.edu/stamp/stamp.jsp?tp=&arnumber=347683&isnumber=8049 
[[#2body|2.]] [http://www.scizzl.com/HowSCIcohWorks.html How SCI Coherence Works]

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-19T01:38:59Z

Eapotter: /* Memory States */

__TOC__

== Background ==
The Scalable Coherent Interface (SCI) cache coherence protocol defines a set of states for memory, a set of states for cache lines, and a set of actions that transition between these states when processors access memory.

Three sets of these attributes are defined for minimal, typical, and full applications:

* '''Minimal''' - for ‘trivial but correct’ applications that require the presence of the memory in only one cache line. It does not enable read-sharing, and is appropriate for small multi-processors or applications that don’t require significant sharing.

* '''Typical''' - enables read-sharing of a memory location and provisions for efficiency, such as DMA transfers, local caching of data, and error recovery. This set adds an additional stable memory state (FRESH) and multiple cache states. This set will be the focus of this article going forward.

* '''Full''' - adds support for pair-wise sharing, QOLB lock bit synchronization, and cleansing and washing of cache lines.

== State Diagrams ==
=== Memory States ===
The memory states define the state of the memory block from the perspective of the home directory. This 2-bit field is maintained in the memory tag by the home directory along with the pointer to the head of the sharing list (forwId). This simple state model includes three stable states and one semi-stable state.

{|border="1" cellspacing="0" cellpadding="2"
!colspan="5"| Table 1: Memory States
|----
!State
!Description
!Minimal
!Typical
!Full
|----
|HOME
|no sharing list
|Y
|Y
|Y
|----
|FRESH
|sharing-list copy is the same as memory
|
|Y
|Y
|----
|GONE
|sharing-list copy may be different from memory
|Y
|Y
|Y
|----
|WASH*
|transitional state (GONE to FRESH)
|
|
|Y
|----
|}

Below is a state diagram that depicts the transition between these states:

[[Image:memory_state.png|center]]

=== Cache States ===
The cache-line states are maintained by each processors cache-coherency controller. This 7-bit field is stored in each cache line in the sharing list, along with the pointer to the next sharing-list node (forwId) and the previous sharing-list node (backID). Seven bits enable up to 128 possible values, and SCI defines twenty-nine stable-states for use in the minimal, typical, and full sets.

Stable states are those cache states that exist when a memory transaction is not in process. Their names are derived from a combination of
the position of the node in the sharing list, such as ONLY, HEAD, MID, and TAIL, and the state of the data for that node, such as FRESH, CLEAN, DIRTY, VALID, STALE, etc.

====States of the Typical Set====

Following are the states defined for the Typical set. (Note that other states may be supported to provide certain interoperability with nodes implementing the Full set, but are not used when all nodes support the Typical set.)

* '''ONLY_DIRTY''' - only one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and no other processor currently caches the block.
* '''ONLY_FRESH''' - only one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and no other processor currently caches the block.
* '''HEAD_DIRTY''' - more than one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and another processors already caches the block.
* '''HEAD_FRESH''' - more than one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and another processors already caches the block.
* '''MID_VALID''' - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is neither the Head or the Tail of the of the sharing list.
* '''TAIL_VALID''' - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is the Tail of the of the sharing list.

Below is a state diagram that depicts these states:

[[Image:cache_states.png|center]]

The request sub-actions of memory transactions that expect a response from the directory include 17 variations of reads, writes, and locks based on the number of bytes requested and the coherency requirements. For convenience, the notation on the edges of the above graph is derived from the terms used in the SCI specification to describe the memory transactions.

The first value indicates the request to the directory by a node for memory access:
* '''Fetch R''' - indicates a request for a memory block with read privileges.
* '''Fetch RW''' - indicates a request for a memory block with read/write privileges.
* '''Data Modify''' - indicates that a processor (the HEAD or the ONLY processor caching the block) writes to the cache line.

The value in the parenthesis indicates the memory state at the time of the request.

== Race Conditions ==
In a distributed shared memory system with caching, the emergence of race conditions is extremely likely. This is mainly due to the lack of a bus for serialization of actions. It is further compounded by the problem of network errors and congestion.

The early invalidation case from Section 11.4 in Solihin is an excellent example of a race condition that can arise in a distributed system. Recall the diagram from the text, and the cache coherence actions, as shown below.

[[Image:EarlyInValidationRace.png|400px|center]]
The circled actions are as follows:
# ''A'' sends a read request to ''Home''.
# ''Home'' replies with data (but the message gets delayed).
# ''B'' sends a write request to ''Home''.
# ''Home'' sends invalidation to ''A'', and it arrives before the ReplyD

The SCI protocol has a way to handle this race condition, as well as many others, and the following sections will discuss how the SCI protocol design can prevent this race condition.

=== Prevention in the SCI Protocol ===
Race conditions are almost non-existent in the SCI protocol, due primarily to the protocol's design. A brief discussion of how this is accomplished follows, followed by a discussion showing what happens if this condition arises in the SCI protocol.
==== Atomic Transactions ====
SCI's primary method for preventing race conditions is having atomic transactions. A transaction is defined as a set of sub-actions necessary to complete some requested action, such as querying memory for a data or invalidating a node's cache. This usually takes the form a a request-response pair, where the initiating node sends a request to another node and waits for the response. The transaction is not complete until the response returns. Suppose, for example, that node ''A'' is in the middle of a transaction with node ''B''. Node ''C'' then tries to make a request of node ''A''. Node ''A'' will respond to node ''C'' that it is busy, telling node ''C'' to try again, as shown in the following diagram.
[[Image:AtomicBusy.png|center]]
Thus, in the Early Invalidation race, node ''A'' would be in the middle of a transaction, having requested a block of data from memory and not having received a response, which would prevent node ''B'' from invalidating it. Note, however, that nodes will not stay in the busy state indefinitely. Doing so would lead to potential deadlocks, so all transactions have time outs that will eventually cause the transaction to fail, thus moving the node out of the busy state, making it able to respond to requests again.

==== Head Node ====
Another method for preventing race conditions is to have the ''Head'' node of a sharing list perform many of the coherence actions. As a result, only one node is performing actions such as writes and invalidations of other sharers. Since only one node is performing these actions, the possibility of concurrent actions is decreased. If another node wants to write, for instance, it must become the Head Node of the sharing list for the cache line to which it wants to write, displacing the current ''Head'' node as the node that can write. All state changes in the sharing list occur at the behest of the ''Head'' node, with the exception of nodes deleting themselves from the list.

When simply sharing a read-only copy of a node, as in when the memory is a ''FRESH'' state, the ''Head'' node is somewhat irrelevant. All the sharing nodes have their own cached value of the cache line. If any node wants to write, including the ''Head'' node, it must perform an additional action in order to do so. Likewise, when a sharing list is sharing a line of data in the dirty, or ''GONE'', memory state, as long as the ''Head'' node has not written to the line, then all the sharing nodes stay in the list with their cached line. However, at the point the ''Head'' node wants to write, it can write its data immediately, but it then must invalidate all other shared copies, via the forward pointers in the sharing list.

This, however, does not protect against a race condition where the memory is ''FRESH'', the ''Head'' node wants to write, and another node also wants to join the list. The Memory Access mechanism prevents this condition.

==== Memory Access ====
The final way that the SCI protocol minimizes race conditions is by changing the directory structure. In Solihin 11.4, there is a ''Home'' node which keeps track of the cache state and has a certain amount of knowledge about the system as a result. In the SCI protocol, there is still a ''Home'' node of sorts, but this node is only responsible for keeping track of who the current ''Head''. Such a ''Home'' node is usually the node where the memory block physically resides. When a node wants to access a block of memory, it sends a request to the ''Home'' node of that memory block, assuming it isn't already in the sharing list for that block. In the event that the data is already being used, the ''Home'' node will include in its response the address of the ''Head'' node of the sharing list. This requirement necessarily serializes the order of access, as one access request is not serviced at the ''Home'' node until the previous request is finished, and the requests are processed in FIFO order. As well, whichever node is the ''Head'' of the sharing list will be notified by the new ''Head'' when it is demoted. All parties involved in a potential race are thus talking to each other, preventing the race from occurring in the first place.

Because of this requirement, the projected race condition from the previous section, where the ''Head'' node is in a ''FRESH'' state and wants to write while another node also wants to read, cannot occur. This is because both nodes must make their request to the ''Home'' node; the ''Head'' node must request the ability to write, and the other node must request the data from the ''Home'' node. One of these requests will make it to the ''Home'' node first, resulting in the second request being deferred to the first node to reach to ''Home'' node. Such a scenario is pictured below.
[[Image:MemoryAccess.png|center]]

==== Solihin 11.4 Resolution ====
The three previous sections discussed individual parts of how the SCI protocol reduces race conditions. Putting all three of these parts together yields the following scenario which resolves the Early Invalidation race as described in the text.
[[Image:SolihinSCI.png|center]]

# ''A'' sends a request to ''Home'' for access to the memory block. It then goes into a busy state while it waits for a response.
# ''B'' also sends a request to ''Home'' for access to the same memory block. ''A'''s request is received first
# ''Home'' responds to ''A'' with the data, but this response gets caught in network traffic and delayed. This response is sent before ''Home'' processes the request from ''B''.
# ''Home'' responds to the request from ''B'', telling it that ''A'' is the current ''Head'' node.
# ''B'' then sends a request to ''A'' to tell it to demote itself.
# ''A'' still hasn't received the response from ''Home'', so it is still in a transaction, and it tells ''B'' that it is busy. ''B'' will have to retry the request.

It should be noted that ''B'' will be in the ''PENDING''[[#Definitions_and_Terms|[def]]] state, so any requests to demote it from the ''Head'' position will be delayed.. This could lead to a chaining of nodes in the ''PENDING'' state, but as soon as node ''A'' receives its response from ''Home'' or times out, the requests in the chain will be successively accepted and processed.

=== Other Possible Race Conditions ===
The design of the SCI protocol prevents many race conditions from occurring, as has already been shown. Here are two other conditions that may arise, with a discussion of how the protocol resolves the issue.

==== Concurrent List Deletions ====

==== Simultaneous Deletion and Invalidation ====
Even though the ''Head'' node performs most of the coherence actions for a shard list, the individual nodes are able to invalidate themselves, such as when they evict the block from their cache. This is referred to in the SCI standard as a "deletion" from the sharing list. Deletion is accomplished by having the invalidating node "lock" itself and then inform its forward and back nodes that they should now point to each other. This "locking" is essentially another ''PENDING'' state. A problem could arise, then, when two neighboring nodes try to invalidate themselves at the same time, as they would both be locked and not respond to each other's message. In this case, though, the protocol specifies that the node that is closest to the tail takes priority. So, the deadlock and/or race condition is averted by ordering the deletion from the tail towards the head.

== Definitions and Terms ==
* ''Head Node'' - The node at the beginning of the sharing list
* ''Home Node'' - The node responsible for keeping track of the ''Head'' node for a sharing list for a memory block, usually the node at which the memory block resides.
* ''PENDING state'' - The intermediate state for a node while it is attempting to become the ''Head'' node in the sharing list. This includes the transaction involved in querying the ''Home'' node and the transaction involved in demoting the current ''Head'' node.
* ''SCI'' - Scalable Coherent Interface

== References ==
[[#1body|1.]] "IEEE Standard for Scalable Coherent Interface (SCI).," IEEE Std 1596-1992 , vol., no., pp.i, 1993. doi: 10.1109/IEEESTD.1993.120366
URL: http://ieeexplore.ieee.org.proxied.lib.ncsu.edu/stamp/stamp.jsp?tp=&arnumber=347683&isnumber=8049 
[[#2body|2.]] [http://www.scizzl.com/HowSCIcohWorks.html How SCI Coherence Works]

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-19T01:38:27Z

Eapotter: /* Memory States */

__TOC__

== Background ==
The Scalable Coherent Interface (SCI) cache coherence protocol defines a set of states for memory, a set of states for cache lines, and a set of actions that transition between these states when processors access memory.

Three sets of these attributes are defined for minimal, typical, and full applications:

* '''Minimal''' - for ‘trivial but correct’ applications that require the presence of the memory in only one cache line. It does not enable read-sharing, and is appropriate for small multi-processors or applications that don’t require significant sharing.

* '''Typical''' - enables read-sharing of a memory location and provisions for efficiency, such as DMA transfers, local caching of data, and error recovery. This set adds an additional stable memory state (FRESH) and multiple cache states. This set will be the focus of this article going forward.

* '''Full''' - adds support for pair-wise sharing, QOLB lock bit synchronization, and cleansing and washing of cache lines.

== State Diagrams ==
=== Memory States ===
The memory states define the state of the memory block from the perspective of the home directory. This 2-bit field is maintained in the memory tag by the home directory along with the pointer to the head of the sharing list (forwId). This simple state model includes three stable states and one semi-stable state.

{|border="1" cellspacing="0" cellpadding="2"
!colspan="5"| Table 1: Memory States
|----
!State
!Description
!Minimal
!Typical
!Full
|----
|HOME
|no sharing list
|Y
|Y
|Y
|----
|FRESH
|sharing-list copy is the same as memory
|
|Y
|Y
|----
|GONE
|sharing-list copy may be different from memory
|Y
|Y
|Y
|----
|WASH*
|transitional state (GONE to FRESH)
|
|
|Y
|----
|}

Below is a state diagram that depicts the transition between these states:

[[Image:memory_states.png|center]]

=== Cache States ===
The cache-line states are maintained by each processors cache-coherency controller. This 7-bit field is stored in each cache line in the sharing list, along with the pointer to the next sharing-list node (forwId) and the previous sharing-list node (backID). Seven bits enable up to 128 possible values, and SCI defines twenty-nine stable-states for use in the minimal, typical, and full sets.

Stable states are those cache states that exist when a memory transaction is not in process. Their names are derived from a combination of
the position of the node in the sharing list, such as ONLY, HEAD, MID, and TAIL, and the state of the data for that node, such as FRESH, CLEAN, DIRTY, VALID, STALE, etc.

====States of the Typical Set====

Following are the states defined for the Typical set. (Note that other states may be supported to provide certain interoperability with nodes implementing the Full set, but are not used when all nodes support the Typical set.)

* '''ONLY_DIRTY''' - only one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and no other processor currently caches the block.
* '''ONLY_FRESH''' - only one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and no other processor currently caches the block.
* '''HEAD_DIRTY''' - more than one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and another processors already caches the block.
* '''HEAD_FRESH''' - more than one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and another processors already caches the block.
* '''MID_VALID''' - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is neither the Head or the Tail of the of the sharing list.
* '''TAIL_VALID''' - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is the Tail of the of the sharing list.

Below is a state diagram that depicts these states:

[[Image:cache_states.png|center]]

The request sub-actions of memory transactions that expect a response from the directory include 17 variations of reads, writes, and locks based on the number of bytes requested and the coherency requirements. For convenience, the notation on the edges of the above graph is derived from the terms used in the SCI specification to describe the memory transactions.

The first value indicates the request to the directory by a node for memory access:
* '''Fetch R''' - indicates a request for a memory block with read privileges.
* '''Fetch RW''' - indicates a request for a memory block with read/write privileges.
* '''Data Modify''' - indicates that a processor (the HEAD or the ONLY processor caching the block) writes to the cache line.

The value in the parenthesis indicates the memory state at the time of the request.

== Race Conditions ==
In a distributed shared memory system with caching, the emergence of race conditions is extremely likely. This is mainly due to the lack of a bus for serialization of actions. It is further compounded by the problem of network errors and congestion.

The early invalidation case from Section 11.4 in Solihin is an excellent example of a race condition that can arise in a distributed system. Recall the diagram from the text, and the cache coherence actions, as shown below.

[[Image:EarlyInValidationRace.png|400px|center]]
The circled actions are as follows:
# ''A'' sends a read request to ''Home''.
# ''Home'' replies with data (but the message gets delayed).
# ''B'' sends a write request to ''Home''.
# ''Home'' sends invalidation to ''A'', and it arrives before the ReplyD

The SCI protocol has a way to handle this race condition, as well as many others, and the following sections will discuss how the SCI protocol design can prevent this race condition.

=== Prevention in the SCI Protocol ===
Race conditions are almost non-existent in the SCI protocol, due primarily to the protocol's design. A brief discussion of how this is accomplished follows, followed by a discussion showing what happens if this condition arises in the SCI protocol.
==== Atomic Transactions ====
SCI's primary method for preventing race conditions is having atomic transactions. A transaction is defined as a set of sub-actions necessary to complete some requested action, such as querying memory for a data or invalidating a node's cache. This usually takes the form a a request-response pair, where the initiating node sends a request to another node and waits for the response. The transaction is not complete until the response returns. Suppose, for example, that node ''A'' is in the middle of a transaction with node ''B''. Node ''C'' then tries to make a request of node ''A''. Node ''A'' will respond to node ''C'' that it is busy, telling node ''C'' to try again, as shown in the following diagram.
[[Image:AtomicBusy.png|center]]
Thus, in the Early Invalidation race, node ''A'' would be in the middle of a transaction, having requested a block of data from memory and not having received a response, which would prevent node ''B'' from invalidating it. Note, however, that nodes will not stay in the busy state indefinitely. Doing so would lead to potential deadlocks, so all transactions have time outs that will eventually cause the transaction to fail, thus moving the node out of the busy state, making it able to respond to requests again.

==== Head Node ====
Another method for preventing race conditions is to have the ''Head'' node of a sharing list perform many of the coherence actions. As a result, only one node is performing actions such as writes and invalidations of other sharers. Since only one node is performing these actions, the possibility of concurrent actions is decreased. If another node wants to write, for instance, it must become the Head Node of the sharing list for the cache line to which it wants to write, displacing the current ''Head'' node as the node that can write. All state changes in the sharing list occur at the behest of the ''Head'' node, with the exception of nodes deleting themselves from the list.

When simply sharing a read-only copy of a node, as in when the memory is a ''FRESH'' state, the ''Head'' node is somewhat irrelevant. All the sharing nodes have their own cached value of the cache line. If any node wants to write, including the ''Head'' node, it must perform an additional action in order to do so. Likewise, when a sharing list is sharing a line of data in the dirty, or ''GONE'', memory state, as long as the ''Head'' node has not written to the line, then all the sharing nodes stay in the list with their cached line. However, at the point the ''Head'' node wants to write, it can write its data immediately, but it then must invalidate all other shared copies, via the forward pointers in the sharing list.

This, however, does not protect against a race condition where the memory is ''FRESH'', the ''Head'' node wants to write, and another node also wants to join the list. The Memory Access mechanism prevents this condition.

==== Memory Access ====
The final way that the SCI protocol minimizes race conditions is by changing the directory structure. In Solihin 11.4, there is a ''Home'' node which keeps track of the cache state and has a certain amount of knowledge about the system as a result. In the SCI protocol, there is still a ''Home'' node of sorts, but this node is only responsible for keeping track of who the current ''Head''. Such a ''Home'' node is usually the node where the memory block physically resides. When a node wants to access a block of memory, it sends a request to the ''Home'' node of that memory block, assuming it isn't already in the sharing list for that block. In the event that the data is already being used, the ''Home'' node will include in its response the address of the ''Head'' node of the sharing list. This requirement necessarily serializes the order of access, as one access request is not serviced at the ''Home'' node until the previous request is finished, and the requests are processed in FIFO order. As well, whichever node is the ''Head'' of the sharing list will be notified by the new ''Head'' when it is demoted. All parties involved in a potential race are thus talking to each other, preventing the race from occurring in the first place.

Because of this requirement, the projected race condition from the previous section, where the ''Head'' node is in a ''FRESH'' state and wants to write while another node also wants to read, cannot occur. This is because both nodes must make their request to the ''Home'' node; the ''Head'' node must request the ability to write, and the other node must request the data from the ''Home'' node. One of these requests will make it to the ''Home'' node first, resulting in the second request being deferred to the first node to reach to ''Home'' node. Such a scenario is pictured below.
[[Image:MemoryAccess.png|center]]

==== Solihin 11.4 Resolution ====
The three previous sections discussed individual parts of how the SCI protocol reduces race conditions. Putting all three of these parts together yields the following scenario which resolves the Early Invalidation race as described in the text.
[[Image:SolihinSCI.png|center]]

# ''A'' sends a request to ''Home'' for access to the memory block. It then goes into a busy state while it waits for a response.
# ''B'' also sends a request to ''Home'' for access to the same memory block. ''A'''s request is received first
# ''Home'' responds to ''A'' with the data, but this response gets caught in network traffic and delayed. This response is sent before ''Home'' processes the request from ''B''.
# ''Home'' responds to the request from ''B'', telling it that ''A'' is the current ''Head'' node.
# ''B'' then sends a request to ''A'' to tell it to demote itself.
# ''A'' still hasn't received the response from ''Home'', so it is still in a transaction, and it tells ''B'' that it is busy. ''B'' will have to retry the request.

It should be noted that ''B'' will be in the ''PENDING''[[#Definitions_and_Terms|[def]]] state, so any requests to demote it from the ''Head'' position will be delayed.. This could lead to a chaining of nodes in the ''PENDING'' state, but as soon as node ''A'' receives its response from ''Home'' or times out, the requests in the chain will be successively accepted and processed.

=== Other Possible Race Conditions ===
The design of the SCI protocol prevents many race conditions from occurring, as has already been shown. Here are two other conditions that may arise, with a discussion of how the protocol resolves the issue.

==== Concurrent List Deletions ====

==== Simultaneous Deletion and Invalidation ====
Even though the ''Head'' node performs most of the coherence actions for a shard list, the individual nodes are able to invalidate themselves, such as when they evict the block from their cache. This is referred to in the SCI standard as a "deletion" from the sharing list. Deletion is accomplished by having the invalidating node "lock" itself and then inform its forward and back nodes that they should now point to each other. This "locking" is essentially another ''PENDING'' state. A problem could arise, then, when two neighboring nodes try to invalidate themselves at the same time, as they would both be locked and not respond to each other's message. In this case, though, the protocol specifies that the node that is closest to the tail takes priority. So, the deadlock and/or race condition is averted by ordering the deletion from the tail towards the head.

== Definitions and Terms ==
* ''Head Node'' - The node at the beginning of the sharing list
* ''Home Node'' - The node responsible for keeping track of the ''Head'' node for a sharing list for a memory block, usually the node at which the memory block resides.
* ''PENDING state'' - The intermediate state for a node while it is attempting to become the ''Head'' node in the sharing list. This includes the transaction involved in querying the ''Home'' node and the transaction involved in demoting the current ''Head'' node.
* ''SCI'' - Scalable Coherent Interface

== References ==
[[#1body|1.]] "IEEE Standard for Scalable Coherent Interface (SCI).," IEEE Std 1596-1992 , vol., no., pp.i, 1993. doi: 10.1109/IEEESTD.1993.120366
URL: http://ieeexplore.ieee.org.proxied.lib.ncsu.edu/stamp/stamp.jsp?tp=&arnumber=347683&isnumber=8049 
[[#2body|2.]] [http://www.scizzl.com/HowSCIcohWorks.html How SCI Coherence Works]

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-19T01:37:04Z

Eapotter: /* History */

__TOC__

== Background ==
The Scalable Coherent Interface (SCI) cache coherence protocol defines a set of states for memory, a set of states for cache lines, and a set of actions that transition between these states when processors access memory.

Three sets of these attributes are defined for minimal, typical, and full applications:

* '''Minimal''' - for ‘trivial but correct’ applications that require the presence of the memory in only one cache line. It does not enable read-sharing, and is appropriate for small multi-processors or applications that don’t require significant sharing.

* '''Typical''' - enables read-sharing of a memory location and provisions for efficiency, such as DMA transfers, local caching of data, and error recovery. This set adds an additional stable memory state (FRESH) and multiple cache states. This set will be the focus of this article going forward.

* '''Full''' - adds support for pair-wise sharing, QOLB lock bit synchronization, and cleansing and washing of cache lines.

== State Diagrams ==
=== Memory States ===
The memory states define the state of the memory block from the perspective of the home directory. This 2-bit field is maintained in the memory tag by the home directory along with the pointer to the head of the sharing list (forwId). This simple state model includes three stable states and one semi-stable state.

{|border="1" cellspacing="0" cellpadding="2"
!colspan="5"| Table 1: Memory States
|----
!State
!Description
!Minimal
!Typical
!Full
|----
|HOME
|no sharing list
|Y
|Y
|Y
|----
|FRESH
|sharing-list copy is the same as memory
|
|Y
|Y
|----
|GONE
|sharing-list copy may be different from memory
|Y
|Y
|Y
|----
|WASH*
|transitional state (GONE to FRESH)
|
|
|Y
|----
|}

=== Cache States ===
The cache-line states are maintained by each processors cache-coherency controller. This 7-bit field is stored in each cache line in the sharing list, along with the pointer to the next sharing-list node (forwId) and the previous sharing-list node (backID). Seven bits enable up to 128 possible values, and SCI defines twenty-nine stable-states for use in the minimal, typical, and full sets.

Stable states are those cache states that exist when a memory transaction is not in process. Their names are derived from a combination of
the position of the node in the sharing list, such as ONLY, HEAD, MID, and TAIL, and the state of the data for that node, such as FRESH, CLEAN, DIRTY, VALID, STALE, etc.

====States of the Typical Set====

Following are the states defined for the Typical set. (Note that other states may be supported to provide certain interoperability with nodes implementing the Full set, but are not used when all nodes support the Typical set.)

* '''ONLY_DIRTY''' - only one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and no other processor currently caches the block.
* '''ONLY_FRESH''' - only one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and no other processor currently caches the block.
* '''HEAD_DIRTY''' - more than one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and another processors already caches the block.
* '''HEAD_FRESH''' - more than one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and another processors already caches the block.
* '''MID_VALID''' - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is neither the Head or the Tail of the of the sharing list.
* '''TAIL_VALID''' - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is the Tail of the of the sharing list.

Below is a state diagram that depicts these states:

[[Image:cache_states.png|center]]

The request sub-actions of memory transactions that expect a response from the directory include 17 variations of reads, writes, and locks based on the number of bytes requested and the coherency requirements. For convenience, the notation on the edges of the above graph is derived from the terms used in the SCI specification to describe the memory transactions.

The first value indicates the request to the directory by a node for memory access:
* '''Fetch R''' - indicates a request for a memory block with read privileges.
* '''Fetch RW''' - indicates a request for a memory block with read/write privileges.
* '''Data Modify''' - indicates that a processor (the HEAD or the ONLY processor caching the block) writes to the cache line.

The value in the parenthesis indicates the memory state at the time of the request.

== Race Conditions ==
In a distributed shared memory system with caching, the emergence of race conditions is extremely likely. This is mainly due to the lack of a bus for serialization of actions. It is further compounded by the problem of network errors and congestion.

The early invalidation case from Section 11.4 in Solihin is an excellent example of a race condition that can arise in a distributed system. Recall the diagram from the text, and the cache coherence actions, as shown below.

[[Image:EarlyInValidationRace.png|400px|center]]
The circled actions are as follows:
# ''A'' sends a read request to ''Home''.
# ''Home'' replies with data (but the message gets delayed).
# ''B'' sends a write request to ''Home''.
# ''Home'' sends invalidation to ''A'', and it arrives before the ReplyD

The SCI protocol has a way to handle this race condition, as well as many others, and the following sections will discuss how the SCI protocol design can prevent this race condition.

=== Prevention in the SCI Protocol ===
Race conditions are almost non-existent in the SCI protocol, due primarily to the protocol's design. A brief discussion of how this is accomplished follows, followed by a discussion showing what happens if this condition arises in the SCI protocol.
==== Atomic Transactions ====
SCI's primary method for preventing race conditions is having atomic transactions. A transaction is defined as a set of sub-actions necessary to complete some requested action, such as querying memory for a data or invalidating a node's cache. This usually takes the form a a request-response pair, where the initiating node sends a request to another node and waits for the response. The transaction is not complete until the response returns. Suppose, for example, that node ''A'' is in the middle of a transaction with node ''B''. Node ''C'' then tries to make a request of node ''A''. Node ''A'' will respond to node ''C'' that it is busy, telling node ''C'' to try again, as shown in the following diagram.
[[Image:AtomicBusy.png|center]]
Thus, in the Early Invalidation race, node ''A'' would be in the middle of a transaction, having requested a block of data from memory and not having received a response, which would prevent node ''B'' from invalidating it. Note, however, that nodes will not stay in the busy state indefinitely. Doing so would lead to potential deadlocks, so all transactions have time outs that will eventually cause the transaction to fail, thus moving the node out of the busy state, making it able to respond to requests again.

==== Head Node ====
Another method for preventing race conditions is to have the ''Head'' node of a sharing list perform many of the coherence actions. As a result, only one node is performing actions such as writes and invalidations of other sharers. Since only one node is performing these actions, the possibility of concurrent actions is decreased. If another node wants to write, for instance, it must become the Head Node of the sharing list for the cache line to which it wants to write, displacing the current ''Head'' node as the node that can write. All state changes in the sharing list occur at the behest of the ''Head'' node, with the exception of nodes deleting themselves from the list.

When simply sharing a read-only copy of a node, as in when the memory is a ''FRESH'' state, the ''Head'' node is somewhat irrelevant. All the sharing nodes have their own cached value of the cache line. If any node wants to write, including the ''Head'' node, it must perform an additional action in order to do so. Likewise, when a sharing list is sharing a line of data in the dirty, or ''GONE'', memory state, as long as the ''Head'' node has not written to the line, then all the sharing nodes stay in the list with their cached line. However, at the point the ''Head'' node wants to write, it can write its data immediately, but it then must invalidate all other shared copies, via the forward pointers in the sharing list.

This, however, does not protect against a race condition where the memory is ''FRESH'', the ''Head'' node wants to write, and another node also wants to join the list. The Memory Access mechanism prevents this condition.

==== Memory Access ====
The final way that the SCI protocol minimizes race conditions is by changing the directory structure. In Solihin 11.4, there is a ''Home'' node which keeps track of the cache state and has a certain amount of knowledge about the system as a result. In the SCI protocol, there is still a ''Home'' node of sorts, but this node is only responsible for keeping track of who the current ''Head''. Such a ''Home'' node is usually the node where the memory block physically resides. When a node wants to access a block of memory, it sends a request to the ''Home'' node of that memory block, assuming it isn't already in the sharing list for that block. In the event that the data is already being used, the ''Home'' node will include in its response the address of the ''Head'' node of the sharing list. This requirement necessarily serializes the order of access, as one access request is not serviced at the ''Home'' node until the previous request is finished, and the requests are processed in FIFO order. As well, whichever node is the ''Head'' of the sharing list will be notified by the new ''Head'' when it is demoted. All parties involved in a potential race are thus talking to each other, preventing the race from occurring in the first place.

Because of this requirement, the projected race condition from the previous section, where the ''Head'' node is in a ''FRESH'' state and wants to write while another node also wants to read, cannot occur. This is because both nodes must make their request to the ''Home'' node; the ''Head'' node must request the ability to write, and the other node must request the data from the ''Home'' node. One of these requests will make it to the ''Home'' node first, resulting in the second request being deferred to the first node to reach to ''Home'' node. Such a scenario is pictured below.
[[Image:MemoryAccess.png|center]]

==== Solihin 11.4 Resolution ====
The three previous sections discussed individual parts of how the SCI protocol reduces race conditions. Putting all three of these parts together yields the following scenario which resolves the Early Invalidation race as described in the text.
[[Image:SolihinSCI.png|center]]

# ''A'' sends a request to ''Home'' for access to the memory block. It then goes into a busy state while it waits for a response.
# ''B'' also sends a request to ''Home'' for access to the same memory block. ''A'''s request is received first
# ''Home'' responds to ''A'' with the data, but this response gets caught in network traffic and delayed. This response is sent before ''Home'' processes the request from ''B''.
# ''Home'' responds to the request from ''B'', telling it that ''A'' is the current ''Head'' node.
# ''B'' then sends a request to ''A'' to tell it to demote itself.
# ''A'' still hasn't received the response from ''Home'', so it is still in a transaction, and it tells ''B'' that it is busy. ''B'' will have to retry the request.

It should be noted that ''B'' will be in the ''PENDING''[[#Definitions_and_Terms|[def]]] state, so any requests to demote it from the ''Head'' position will be delayed.. This could lead to a chaining of nodes in the ''PENDING'' state, but as soon as node ''A'' receives its response from ''Home'' or times out, the requests in the chain will be successively accepted and processed.

=== Other Possible Race Conditions ===
The design of the SCI protocol prevents many race conditions from occurring, as has already been shown. Here are two other conditions that may arise, with a discussion of how the protocol resolves the issue.

==== Concurrent List Deletions ====

==== Simultaneous Deletion and Invalidation ====
Even though the ''Head'' node performs most of the coherence actions for a shard list, the individual nodes are able to invalidate themselves, such as when they evict the block from their cache. This is referred to in the SCI standard as a "deletion" from the sharing list. Deletion is accomplished by having the invalidating node "lock" itself and then inform its forward and back nodes that they should now point to each other. This "locking" is essentially another ''PENDING'' state. A problem could arise, then, when two neighboring nodes try to invalidate themselves at the same time, as they would both be locked and not respond to each other's message. In this case, though, the protocol specifies that the node that is closest to the tail takes priority. So, the deadlock and/or race condition is averted by ordering the deletion from the tail towards the head.

== Definitions and Terms ==
* ''Head Node'' - The node at the beginning of the sharing list
* ''Home Node'' - The node responsible for keeping track of the ''Head'' node for a sharing list for a memory block, usually the node at which the memory block resides.
* ''PENDING state'' - The intermediate state for a node while it is attempting to become the ''Head'' node in the sharing list. This includes the transaction involved in querying the ''Home'' node and the transaction involved in demoting the current ''Head'' node.
* ''SCI'' - Scalable Coherent Interface

== References ==
[[#1body|1.]] "IEEE Standard for Scalable Coherent Interface (SCI).," IEEE Std 1596-1992 , vol., no., pp.i, 1993. doi: 10.1109/IEEESTD.1993.120366
URL: http://ieeexplore.ieee.org.proxied.lib.ncsu.edu/stamp/stamp.jsp?tp=&arnumber=347683&isnumber=8049 
[[#2body|2.]] [http://www.scizzl.com/HowSCIcohWorks.html How SCI Coherence Works]

File:Memory state.png

2011-04-19T01:36:37Z

Eapotter: Memory State Diagram

Memory State Diagram

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-19T01:03:04Z

Eapotter: /* Memory States */

__TOC__

== History ==

== Background ==
The Scalable Coherent Interface (SCI) cache coherence protocol defines a set of states for memory, a set of states for cache lines, and a set of actions that transition between these states when processors access memory.

Three sets of these attributes are defined for minimal, typical, and full applications:

* '''Minimal''' - for ‘trivial but correct’ applications that require the presence of the memory in only one cache line. It does not enable read-sharing, and is appropriate for small multi-processors or applications that don’t require significant sharing.

* '''Typical''' - enables read-sharing of a memory location and provisions for efficiency, such as DMA transfers, local caching of data, and error recovery. This set adds an additional stable memory state (FRESH) and multiple cache states. This set will be the focus of this article going forward.

* '''Full''' - adds support for pair-wise sharing, QOLB lock bit synchronization, and cleansing and washing of cache lines.

== State Diagrams ==
=== Memory States ===
The memory states define the state of the memory block from the perspective of the home directory. This 2-bit field is maintained in the memory tag by the home directory along with the pointer to the head of the sharing list (forwId). This simple state model includes three stable states and one semi-stable state.

{|border="1" cellspacing="0" cellpadding="2"
!colspan="5"| Table 1: Memory States
|----
!State
!Description
!Minimal
!Typical
!Full
|----
|HOME
|no sharing list
|Y
|Y
|Y
|----
|FRESH
|sharing-list copy is the same as memory
|
|Y
|Y
|----
|GONE
|sharing-list copy may be different from memory
|Y
|Y
|Y
|----
|WASH*
|transitional state (GONE to FRESH)
|
|
|Y
|----
|}

=== Cache States ===
The cache-line states are maintained by each processors cache-coherency controller. This 7-bit field is stored in each cache line in the sharing list, along with the pointer to the next sharing-list node (forwId) and the previous sharing-list node (backID). Seven bits enable up to 128 possible values, and SCI defines twenty-nine stable-states for use in the minimal, typical, and full sets.

Stable states are those cache states that exist when a memory transaction is not in process. Their names are derived from a combination of
the position of the node in the sharing list, such as ONLY, HEAD, MID, and TAIL, and the state of the data for that node, such as FRESH, CLEAN, DIRTY, VALID, STALE, etc.

====States of the Typical Set====

Following are the states defined for the Typical set. (Note that other states may be supported to provide certain interoperability with nodes implementing the Full set, but are not used when all nodes support the Typical set.)

* '''ONLY_DIRTY''' - only one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and no other processor currently caches the block.
* '''ONLY_FRESH''' - only one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and no other processor currently caches the block.
* '''HEAD_DIRTY''' - more than one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and another processors already caches the block.
* '''HEAD_FRESH''' - more than one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and another processors already caches the block.
* '''MID_VALID''' - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is neither the Head or the Tail of the of the sharing list.
* '''TAIL_VALID''' - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is the Tail of the of the sharing list.

Below is a state diagram that depicts these states:

[[Image:cache_states.png|center]]

The request sub-actions of memory transactions that expect a response from the directory include 17 variations of reads, writes, and locks based on the number of bytes requested and the coherency requirements. For convenience, the notation on the edges of the above graph is derived from the terms used in the SCI specification to describe the memory transactions.

The first value indicates the request to the directory by a node for memory access:
* '''Fetch R''' - indicates a request for a memory block with read privileges.
* '''Fetch RW''' - indicates a request for a memory block with read/write privileges.
* '''Data Modify''' - indicates that a processor (the HEAD or the ONLY processor caching the block) writes to the cache line.

The value in the parenthesis indicates the memory state at the time of the request.

== Race Conditions ==
In a distributed shared memory system with caching, the emergence of race conditions is extremely likely. This is mainly due to the lack of a bus for serialization of actions. It is further compounded by the problem of network errors and congestion.

The early invalidation case from Section 11.4 in Solihin is an excellent example of a race condition that can arise in a distributed system. Recall the diagram from the text, and the cache coherence actions, as shown below.

[[Image:EarlyInValidationRace.png|400px|center]]
The circled actions are as follows:
# ''A'' sends a read request to ''Home''.
# ''Home'' replies with data (but the message gets delayed).
# ''B'' sends a write request to ''Home''.
# ''Home'' sends invalidation to ''A'', and it arrives before the ReplyD

The SCI protocol has a way to handle this race condition, as well as many others, and the following sections will discuss how the SCI protocol design can prevent this race condition.

=== Prevention in the SCI Protocol ===
Race conditions are almost non-existent in the SCI protocol, due primarily to the protocol's design. A brief discussion of how this is accomplished follows, followed by a discussion showing what happens if this condition arises in the SCI protocol.
==== Atomic Transactions ====
SCI's primary method for preventing race conditions is having atomic transactions. A transaction is defined as a set of sub-actions necessary to complete some requested action, such as querying memory for a data or invalidating a node's cache. This usually takes the form a a request-response pair, where the initiating node sends a request to another node and waits for the response. The transaction is not complete until the response returns. Suppose, for example, that node ''A'' is in the middle of a transaction with node ''B''. Node ''C'' then tries to make a request of node ''A''. Node ''A'' will respond to node ''C'' that it is busy, telling node ''C'' to try again, as shown in the following diagram.
[[Image:AtomicBusy.png|center]]
Thus, in the Early Invalidation race, node ''A'' would be in the middle of a transaction, having requested a block of data from memory and not having received a response, which would prevent node ''B'' from invalidating it. Note, however, that nodes will not stay in the busy state indefinitely. Doing so would lead to potential deadlocks, so all transactions have time outs that will eventually cause the transaction to fail, thus moving the node out of the busy state, making it able to respond to requests again.

==== Head Node ====
Another method for preventing race conditions is to have the ''Head'' node of a sharing list perform many of the coherence actions. As a result, only one node is performing actions such as writes and invalidations of other sharers. Since only one node is performing these actions, the possibility of concurrent actions is decreased. If another node wants to write, for instance, it must become the Head Node of the sharing list for the cache line to which it wants to write, displacing the current ''Head'' node as the node that can write. All state changes in the sharing list occur at the behest of the ''Head'' node, with the exception of nodes deleting themselves from the list.

When simply sharing a read-only copy of a node, as in when the memory is a ''FRESH'' state, the ''Head'' node is somewhat irrelevant. All the sharing nodes have their own cached value of the cache line. If any node wants to write, including the ''Head'' node, it must perform an additional action in order to do so. Likewise, when a sharing list is sharing a line of data in the dirty, or ''GONE'', memory state, as long as the ''Head'' node has not written to the line, then all the sharing nodes stay in the list with their cached line. However, at the point the ''Head'' node wants to write, it can write its data immediately, but it then must invalidate all other shared copies, via the forward pointers in the sharing list.

This, however, does not protect against a race condition where the memory is ''FRESH'', the ''Head'' node wants to write, and another node also wants to join the list. The Memory Access mechanism prevents this condition.

==== Memory Access ====
The final way that the SCI protocol minimizes race conditions is by changing the directory structure. In Solihin 11.4, there is a ''Home'' node which keeps track of the cache state and has a certain amount of knowledge about the system as a result. In the SCI protocol, there is still a ''Home'' node of sorts, but this node is only responsible for keeping track of who the current ''Head''. Such a ''Home'' node is usually the node where the memory block physically resides. When a node wants to access a block of memory, it sends a request to the ''Home'' node of that memory block, assuming it isn't already in the sharing list for that block. In the event that the data is already being used, the ''Home'' node will include in its response the address of the ''Head'' node of the sharing list. This requirement necessarily serializes the order of access, as one access request is not serviced at the ''Home'' node until the previous request is finished, and the requests are processed in FIFO order. As well, whichever node is the ''Head'' of the sharing list will be notified by the new ''Head'' when it is demoted. All parties involved in a potential race are thus talking to each other, preventing the race from occurring in the first place.

Because of this requirement, the projected race condition from the previous section, where the ''Head'' node is in a ''FRESH'' state and wants to write while another node also wants to read, cannot occur. This is because both nodes must make their request to the ''Home'' node; the ''Head'' node must request the ability to write, and the other node must request the data from the ''Home'' node. One of these requests will make it to the ''Home'' node first, resulting in the second request being deferred to the first node to reach to ''Home'' node. Such a scenario is pictured below.
[[Image:MemoryAccess.png|center]]

==== Solihin 11.4 Resolution ====
The three previous sections discussed individual parts of how the SCI protocol reduces race conditions. Putting all three of these parts together yields the following scenario which resolves the Early Invalidation race as described in the text.
[[Image:SolihinSCI.png|center]]

# ''A'' sends a request to ''Home'' for access to the memory block. It then goes into a busy state while it waits for a response.
# ''B'' also sends a request to ''Home'' for access to the same memory block. ''A'''s request is received first
# ''Home'' responds to ''A'' with the data, but this response gets caught in network traffic and delayed. This response is sent before ''Home'' processes the request from ''B''.
# ''Home'' responds to the request from ''B'', telling it that ''A'' is the current ''Head'' node.
# ''B'' then sends a request to ''A'' to tell it to demote itself.
# ''A'' still hasn't received the response from ''Home'', so it is still in a transaction, and it tells ''B'' that it is busy. ''B'' will have to retry the request.

It should be noted that ''B'' will be in the ''PENDING''[[#Definitions_and_Terms|[def]]] state, so any requests to demote it from the ''Head'' position will be delayed.. This could lead to a chaining of nodes in the ''PENDING'' state, but as soon as node ''A'' receives its response from ''Home'' or times out, the requests in the chain will be successively accepted and processed.

=== Other Possible Race Conditions ===
The design of the SCI protocol prevents many race conditions from occurring, as has already been shown. Here are two other conditions that may arise, with a discussion of how the protocol resolves the issue.

==== Concurrent List Deletions ====

==== Simultaneous Deletion and Invalidation ====
Even though the ''Head'' node performs most of the coherence actions for a shard list, the individual nodes are able to invalidate themselves, such as when they evict the block from their cache. This is referred to in the SCI standard as a "deletion" from the sharing list. Deletion is accomplished by having the invalidating node "lock" itself and then inform its forward and back nodes that they should now point to each other. This "locking" is essentially another ''PENDING'' state. A problem could arise, then, when two neighboring nodes try to invalidate themselves at the same time, as they would both be locked and not respond to each other's message. In this case, though, the protocol specifies that the node that is closest to the tail takes priority. So, the deadlock and/or race condition is averted by ordering the deletion from the tail towards the head.

== Definitions and Terms ==
* ''Head Node'' - The node at the beginning of the sharing list
* ''Home Node'' - The node responsible for keeping track of the ''Head'' node for a sharing list for a memory block, usually the node at which the memory block resides.
* ''PENDING state'' - The intermediate state for a node while it is attempting to become the ''Head'' node in the sharing list. This includes the transaction involved in querying the ''Home'' node and the transaction involved in demoting the current ''Head'' node.
* ''SCI'' - Scalable Coherent Interface

== References ==
[[#1body|1.]] "IEEE Standard for Scalable Coherent Interface (SCI).," IEEE Std 1596-1992 , vol., no., pp.i, 1993. doi: 10.1109/IEEESTD.1993.120366
URL: http://ieeexplore.ieee.org.proxied.lib.ncsu.edu/stamp/stamp.jsp?tp=&arnumber=347683&isnumber=8049 
[[#2body|2.]] [http://www.scizzl.com/HowSCIcohWorks.html How SCI Coherence Works]

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-19T01:02:48Z

Eapotter: /* Memory States */

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-19T01:01:12Z

Eapotter: /* States of the Typical Set */

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-19T00:58:08Z

Eapotter: /* States of the Typical Set */

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-19T00:23:51Z

Eapotter: /* States of the Typical Set */

__TOC__

== History ==

== Background ==
The Scalable Coherent Interface (SCI) cache coherence protocol defines a set of states for memory, a set of states for cache lines, and a set of actions that transition between these states when processors access memory.

Three sets of these attributes are defined for minimal, typical, and full applications:

* '''Minimal''' - for ‘trivial but correct’ applications that require the presence of the memory in only one cache line. It does not enable read-sharing, and is appropriate for small multi-processors or applications that don’t require significant sharing.

* '''Typical''' - enables read-sharing of a memory location and provisions for efficiency, such as DMA transfers, local caching of data, and error recovery. This set adds an additional stable memory state (FRESH) and multiple cache states. This set will be the focus of this article going forward.

* '''Full''' - adds support for pair-wise sharing, QOLB lock bit synchronization, and cleansing and washing of cache lines.

== State Diagrams ==
=== Memory States ===
The memory states define the state of the memory block from the perspective of the home directory. This 2-bit field is maintained in the memory tag by the home directory along with the pointer to the head of the sharing list (forwId). This simple state model includes three stable states and one semi-stable state.

{|border="1" cellspacing="0" cellpadding="2"
!colspan="5"| Table 1: Memory States
|----
!State
!Description
!Minimal
!Typical
!Full
|----
|HOME
|no sharing list
|Y
|Y
|Y
|----
|FRESH
|sharing-list copy is the same as memory
|
|Y
|Y
|----
|GONE
|sharing-list copy may be different from memory
|Y
|Y
|Y
|----
|WASH*
|transitional state (GONE to FRESH)
|
|
|Y
|----
|}

=== Cache States ===
The cache-line states are maintained by each processors cache-coherency controller. This 7-bit field is stored in each cache line in the sharing list, along with the pointer to the next sharing-list node (forwId) and the previous sharing-list node (backID). Seven bits enable up to 128 possible values, and SCI defines twenty-nine stable-states for use in the minimal, typical, and full sets.

Stable states are those cache states that exist when a memory transaction is not in process. Their names are derived from a combination of
the position of the node in the sharing list, such as ONLY, HEAD, MID, and TAIL, and the state of the data for that node, such as FRESH, CLEAN, DIRTY, VALID, STALE, etc.

====States of the Typical Set====

Following are the states defined for the Typical set. (Note that other states may be supported to provide certain interoperability with nodes implementing the Full set, but are not used when all nodes support the Typical set.)

* '''ONLY_DIRTY''' - only one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and no other processor currently caches the block.
* '''ONLY_FRESH''' - only one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and no other processor currently caches the block.
* '''HEAD_DIRTY''' - more than one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and another processors already caches the block.
* '''HEAD_FRESH''' - more than one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and another processors already caches the block.
* '''MID_VALID''' - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is neither the Head or the Tail of the of the sharing list.
* '''TAIL_VALID''' - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is the Tail of the of the sharing list.

Below is a state diagram that depicts these states:

[[Image:cache_states.png|center]]

The request sub-actions of memory transactions that expect a response from the directory include 17 variations of reads, writes, and locks based on the number of bytes requested and the coherency requirements. For convenience, the notation on the edges of the above graph is derived from the terms used in the SCI specification to describe the memory transactions.

The first value indicates the request to the directory by a node for memory access:
* '''Fetch R''' - indicates a request for a memory block with read privileges.
* '''Fetch RW''' - indicates a request for a memory block with read/write privileges.
* '''DATA_MODIFY''' - indicates that a processor (the HEAD or the ONLY processor caching the block) writes to the cache line.

The value in the parenthesis indicates the memory state at the time of the request.

== Race Conditions ==
In a distributed shared memory system with caching, the emergence of race conditions is extremely likely. This is mainly due to the lack of a bus for serialization of actions. It is further compounded by the problem of network errors and congestion.

The early invalidation case from Section 11.4 in Solihin is an excellent example of a race condition that can arise in a distributed system. Recall the diagram from the text, and the cache coherence actions, as shown below.

[[Image:EarlyInValidationRace.png|400px|center]]
The circled actions are as follows:
# ''A'' sends a read request to ''Home''.
# ''Home'' replies with data (but the message gets delayed).
# ''B'' sends a write request to ''Home''.
# ''Home'' sends invalidation to ''A'', and it arrives before the ReplyD

The SCI protocol has a way to handle this race condition, as well as many others, and the following sections will discuss how the SCI protocol design can prevent this race condition.

=== Prevention in the SCI Protocol ===
Race conditions are almost non-existent in the SCI protocol, due primarily to the protocol's design. A brief discussion of how this is accomplished follows, followed by a discussion showing what happens if this condition arises in the SCI protocol.
==== Atomic Transactions ====
SCI's primary method for preventing race conditions is having atomic transactions. A transaction is defined as a set of sub-actions necessary to complete some requested action, such as querying memory for a data or invalidating a node's cache. This usually takes the form a a request-response pair, where the initiating node sends a request to another node and waits for the response. The transaction is not complete until the response returns. Suppose, for example, that node ''A'' is in the middle of a transaction with node ''B''. Node ''C'' then tries to make a request of node ''A''. Node ''A'' will respond to node ''C'' that it is busy, telling node ''C'' to try again, as shown in the following diagram.
[[Image:AtomicBusy.png|center]]
Thus, in the Early Invalidation race, node ''A'' would be in the middle of a transaction, having requested a block of data from memory and not having received a response, which would prevent node ''B'' from invalidating it. Note, however, that nodes will not stay in the busy state indefinitely. Doing so would lead to potential deadlocks, so all transactions have time outs that will eventually cause the transaction to fail, thus moving the node out of the busy state, making it able to respond to requests again.

==== Head Node ====
Another method for preventing race conditions is to have the ''Head'' node of a sharing list perform many of the coherence actions. As a result, only one node is performing actions such as writes and invalidations of other sharers. Since only one node is performing these actions, the possibility of concurrent actions is decreased. If another node wants to write, for instance, it must become the Head Node of the sharing list for the cache line to which it wants to write, displacing the current ''Head'' node as the node that can write. All state changes in the sharing list occur at the behest of the ''Head'' node, with the exception of nodes deleting themselves from the list.

When simply sharing a read-only copy of a node, as in when the memory is a ''FRESH'' state, the ''Head'' node is somewhat irrelevant. All the sharing nodes have their own cached value of the cache line. If any node wants to write, including the ''Head'' node, it must perform an additional action in order to do so. Likewise, when a sharing list is sharing a line of data in the dirty, or ''GONE'', memory state, as long as the ''Head'' node has not written to the line, then all the sharing nodes stay in the list with their cached line. However, at the point the ''Head'' node wants to write, it can write its data immediately, but it then must invalidate all other shared copies, via the forward pointers in the sharing list.

This, however, does not protect against a race condition where the memory is ''FRESH'', the ''Head'' node wants to write, and another node also wants to join the list. The Memory Access mechanism prevents this condition.

==== Memory Access ====
The final way that the SCI protocol minimizes race conditions is by changing the directory structure. In Solihin 11.4, there is a ''Home'' node which keeps track of the cache state and has a certain amount of knowledge about the system as a result. In the SCI protocol, there is still a ''Home'' node of sorts, but this node is only responsible for keeping track of who the current ''Head''. Such a ''Home'' node is usually the node where the memory block physically resides. When a node wants to access a block of memory, it sends a request to the ''Home'' node of that memory block, assuming it isn't already in the sharing list for that block. In the event that the data is already being used, the ''Home'' node will include in its response the address of the ''Head'' node of the sharing list. This requirement necessarily serializes the order of access, as one access request is not serviced at the ''Home'' node until the previous request is finished, and the requests are processed in FIFO order. As well, whichever node is the ''Head'' of the sharing list will be notified by the new ''Head'' when it is demoted. All parties involved in a potential race are thus talking to each other, preventing the race from occurring in the first place.

Because of this requirement, the projected race condition from the previous section, where the ''Head'' node is in a ''FRESH'' state and wants to write while another node also wants to read, cannot occur. This is because both nodes must make their request to the ''Home'' node; the ''Head'' node must request the ability to write, and the other node must request the data from the ''Home'' node. One of these requests will make it to the ''Home'' node first, resulting in the second request being deferred to the first node to reach to ''Home'' node. Such a scenario is pictured below.
[[Image:MemoryAccess.png|center]]

==== Solihin 11.4 Resolution ====
The three previous sections discussed individual parts of how the SCI protocol reduces race conditions. Putting all three of these parts together yields the following scenario which resolves the Early Invalidation race as described in the text.
[[Image:SolihinSCI.png|center]]

# ''A'' sends a request to ''Home'' for access to the memory block. It then goes into a busy state while it waits for a response.
# ''B'' also sends a request to ''Home'' for access to the same memory block. ''A'''s request is received first
# ''Home'' responds to ''A'' with the data, but this response gets caught in network traffic and delayed. This response is sent before ''Home'' processes the request from ''B''.
# ''Home'' responds to the request from ''B'', telling it that ''A'' is the current ''Head'' node.
# ''B'' then sends a request to ''A'' to tell it to demote itself.
# ''A'' still hasn't received the response from ''Home'', so it is still in a transaction, and it tells ''B'' that it is busy. ''B'' will have to retry the request.

It should be noted that ''B'' will be in the ''PENDING''[[#Definitions_and_Terms|[def]]] state, so any requests to demote it from the ''Head'' position will be delayed.. This could lead to a chaining of nodes in the ''PENDING'' state, but as soon as node ''A'' receives its response from ''Home'' or times out, the requests in the chain will be successively accepted and processed.

=== Other Possible Race Conditions ===
The design of the SCI protocol prevents many race conditions from occurring, as has already been shown. Here are two other conditions that may arise, with a discussion of how the protocol resolves the issue.

==== Concurrent List Deletions ====

==== Simultaneous Deletion and Invalidation ====
Even though the ''Head'' node performs most of the coherence actions for a shard list, the individual nodes are able to invalidate themselves, such as when they evict the block from their cache. This is referred to in the SCI standard as a "deletion" from the sharing list. Deletion is accomplished by having the invalidating node "lock" itself and then inform its forward and back nodes that they should now point to each other. This "locking" is essentially another ''PENDING'' state. A problem could arise, then, when two neighboring nodes try to invalidate themselves at the same time, as they would both be locked and not respond to each other's message. In this case, though, the protocol specifies that the node that is closest to the tail takes priority. So, the deadlock and/or race condition is averted by ordering the deletion from the tail towards the head.

== Definitions and Terms ==
* ''Head Node'' - The node at the beginning of the sharing list
* ''Home Node'' - The node responsible for keeping track of the ''Head'' node for a sharing list for a memory block, usually the node at which the memory block resides.
* ''PENDING state'' - The intermediate state for a node while it is attempting to become the ''Head'' node in the sharing list. This includes the transaction involved in querying the ''Home'' node and the transaction involved in demoting the current ''Head'' node.
* ''SCI'' - Scalable Coherent Interface

== References ==
[[#1body|1.]] "IEEE Standard for Scalable Coherent Interface (SCI).," IEEE Std 1596-1992 , vol., no., pp.i, 1993. doi: 10.1109/IEEESTD.1993.120366
URL: http://ieeexplore.ieee.org.proxied.lib.ncsu.edu/stamp/stamp.jsp?tp=&arnumber=347683&isnumber=8049 
[[#2body|2.]] [http://www.scizzl.com/HowSCIcohWorks.html How SCI Coherence Works]

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-19T00:22:55Z

Eapotter: /* States of the Typical Set */

__TOC__

== History ==

== Background ==
The Scalable Coherent Interface (SCI) cache coherence protocol defines a set of states for memory, a set of states for cache lines, and a set of actions that transition between these states when processors access memory.

Three sets of these attributes are defined for minimal, typical, and full applications:

* '''Minimal''' - for ‘trivial but correct’ applications that require the presence of the memory in only one cache line. It does not enable read-sharing, and is appropriate for small multi-processors or applications that don’t require significant sharing.

* '''Typical''' - enables read-sharing of a memory location and provisions for efficiency, such as DMA transfers, local caching of data, and error recovery. This set adds an additional stable memory state (FRESH) and multiple cache states. This set will be the focus of this article going forward.

* '''Full''' - adds support for pair-wise sharing, QOLB lock bit synchronization, and cleansing and washing of cache lines.

== State Diagrams ==
=== Memory States ===
The memory states define the state of the memory block from the perspective of the home directory. This 2-bit field is maintained in the memory tag by the home directory along with the pointer to the head of the sharing list (forwId). This simple state model includes three stable states and one semi-stable state.

{|border="1" cellspacing="0" cellpadding="2"
!colspan="5"| Table 1: Memory States
|----
!State
!Description
!Minimal
!Typical
!Full
|----
|HOME
|no sharing list
|Y
|Y
|Y
|----
|FRESH
|sharing-list copy is the same as memory
|
|Y
|Y
|----
|GONE
|sharing-list copy may be different from memory
|Y
|Y
|Y
|----
|WASH*
|transitional state (GONE to FRESH)
|
|
|Y
|----
|}

=== Cache States ===
The cache-line states are maintained by each processors cache-coherency controller. This 7-bit field is stored in each cache line in the sharing list, along with the pointer to the next sharing-list node (forwId) and the previous sharing-list node (backID). Seven bits enable up to 128 possible values, and SCI defines twenty-nine stable-states for use in the minimal, typical, and full sets.

Stable states are those cache states that exist when a memory transaction is not in process. Their names are derived from a combination of
the position of the node in the sharing list, such as ONLY, HEAD, MID, and TAIL, and the state of the data for that node, such as FRESH, CLEAN, DIRTY, VALID, STALE, etc.

====States of the Typical Set====

Following are the states defined for the Typical set. (Note that other states may be supported to provide certain interoperability with nodes implementing the Full set, but are not used when all nodes support the Typical set.)

* '''ONLY_DIRTY''' - only one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and no other processor currently caches the block.
* '''ONLY_FRESH''' - only one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and no other processor currently caches the block.
* '''HEAD_DIRTY''' - more than one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and another processors already caches the block.
* '''HEAD_FRESH''' - more than one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and another processors already caches the block.
* '''MID_VALID''' - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is neither the Head or the Tail of the of the sharing list.
* '''TAIL_VALID''' - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is the Tail of the of the sharing list.

Below is a state diagram that depicts these states:

[[Image:cache_states.png|center]]

The request sub-actions of memory transactions that expect a response from the directory include 17 variations of reads, writes, and locks based on the number of bytes requested and the coherency requirements. For convenience, the notation on the edges of the above graph is derived from the terms used in the SCI specification to describe the memory transactions.

The first value indicates the request to the directory by a node for memory access:
*”Fetch R” - indicates a request for a memory block with read privileges.
*”Fetch RW” - indicates a request for a memory block with read/write privileges.
*”DATA_MODIFY” - indicates that a processor (the HEAD or the ONLY processor caching the block) writes to the cache line.

The value in the parenthesis indicates the memory state at the time of the request.

== Race Conditions ==
In a distributed shared memory system with caching, the emergence of race conditions is extremely likely. This is mainly due to the lack of a bus for serialization of actions. It is further compounded by the problem of network errors and congestion.

The early invalidation case from Section 11.4 in Solihin is an excellent example of a race condition that can arise in a distributed system. Recall the diagram from the text, and the cache coherence actions, as shown below.

[[Image:EarlyInValidationRace.png|400px|center]]
The circled actions are as follows:
# ''A'' sends a read request to ''Home''.
# ''Home'' replies with data (but the message gets delayed).
# ''B'' sends a write request to ''Home''.
# ''Home'' sends invalidation to ''A'', and it arrives before the ReplyD

The SCI protocol has a way to handle this race condition, as well as many others, and the following sections will discuss how the SCI protocol design can prevent this race condition.

=== Prevention in the SCI Protocol ===
Race conditions are almost non-existent in the SCI protocol, due primarily to the protocol's design. A brief discussion of how this is accomplished follows, followed by a discussion showing what happens if this condition arises in the SCI protocol.
==== Atomic Transactions ====
SCI's primary method for preventing race conditions is having atomic transactions. A transaction is defined as a set of sub-actions necessary to complete some requested action, such as querying memory for a data or invalidating a node's cache. This usually takes the form a a request-response pair, where the initiating node sends a request to another node and waits for the response. The transaction is not complete until the response returns. Suppose, for example, that node ''A'' is in the middle of a transaction with node ''B''. Node ''C'' then tries to make a request of node ''A''. Node ''A'' will respond to node ''C'' that it is busy, telling node ''C'' to try again, as shown in the following diagram.
[[Image:AtomicBusy.png|center]]
Thus, in the Early Invalidation race, node ''A'' would be in the middle of a transaction, having requested a block of data from memory and not having received a response, which would prevent node ''B'' from invalidating it. Note, however, that nodes will not stay in the busy state indefinitely. Doing so would lead to potential deadlocks, so all transactions have time outs that will eventually cause the transaction to fail, thus moving the node out of the busy state, making it able to respond to requests again.

==== Head Node ====
Another method for preventing race conditions is to have the ''Head'' node of a sharing list perform many of the coherence actions. As a result, only one node is performing actions such as writes and invalidations of other sharers. Since only one node is performing these actions, the possibility of concurrent actions is decreased. If another node wants to write, for instance, it must become the Head Node of the sharing list for the cache line to which it wants to write, displacing the current ''Head'' node as the node that can write. All state changes in the sharing list occur at the behest of the ''Head'' node, with the exception of nodes deleting themselves from the list.

When simply sharing a read-only copy of a node, as in when the memory is a ''FRESH'' state, the ''Head'' node is somewhat irrelevant. All the sharing nodes have their own cached value of the cache line. If any node wants to write, including the ''Head'' node, it must perform an additional action in order to do so. Likewise, when a sharing list is sharing a line of data in the dirty, or ''GONE'', memory state, as long as the ''Head'' node has not written to the line, then all the sharing nodes stay in the list with their cached line. However, at the point the ''Head'' node wants to write, it can write its data immediately, but it then must invalidate all other shared copies, via the forward pointers in the sharing list.

This, however, does not protect against a race condition where the memory is ''FRESH'', the ''Head'' node wants to write, and another node also wants to join the list. The Memory Access mechanism prevents this condition.

==== Memory Access ====
The final way that the SCI protocol minimizes race conditions is by changing the directory structure. In Solihin 11.4, there is a ''Home'' node which keeps track of the cache state and has a certain amount of knowledge about the system as a result. In the SCI protocol, there is still a ''Home'' node of sorts, but this node is only responsible for keeping track of who the current ''Head''. Such a ''Home'' node is usually the node where the memory block physically resides. When a node wants to access a block of memory, it sends a request to the ''Home'' node of that memory block, assuming it isn't already in the sharing list for that block. In the event that the data is already being used, the ''Home'' node will include in its response the address of the ''Head'' node of the sharing list. This requirement necessarily serializes the order of access, as one access request is not serviced at the ''Home'' node until the previous request is finished, and the requests are processed in FIFO order. As well, whichever node is the ''Head'' of the sharing list will be notified by the new ''Head'' when it is demoted. All parties involved in a potential race are thus talking to each other, preventing the race from occurring in the first place.

Because of this requirement, the projected race condition from the previous section, where the ''Head'' node is in a ''FRESH'' state and wants to write while another node also wants to read, cannot occur. This is because both nodes must make their request to the ''Home'' node; the ''Head'' node must request the ability to write, and the other node must request the data from the ''Home'' node. One of these requests will make it to the ''Home'' node first, resulting in the second request being deferred to the first node to reach to ''Home'' node. Such a scenario is pictured below.
[[Image:MemoryAccess.png|center]]

==== Solihin 11.4 Resolution ====
The three previous sections discussed individual parts of how the SCI protocol reduces race conditions. Putting all three of these parts together yields the following scenario which resolves the Early Invalidation race as described in the text.
[[Image:SolihinSCI.png|center]]

# ''A'' sends a request to ''Home'' for access to the memory block. It then goes into a busy state while it waits for a response.
# ''B'' also sends a request to ''Home'' for access to the same memory block. ''A'''s request is received first
# ''Home'' responds to ''A'' with the data, but this response gets caught in network traffic and delayed. This response is sent before ''Home'' processes the request from ''B''.
# ''Home'' responds to the request from ''B'', telling it that ''A'' is the current ''Head'' node.
# ''B'' then sends a request to ''A'' to tell it to demote itself.
# ''A'' still hasn't received the response from ''Home'', so it is still in a transaction, and it tells ''B'' that it is busy. ''B'' will have to retry the request.

It should be noted that ''B'' will be in the ''PENDING''[[#Definitions_and_Terms|[def]]] state, so any requests to demote it from the ''Head'' position will be delayed.. This could lead to a chaining of nodes in the ''PENDING'' state, but as soon as node ''A'' receives its response from ''Home'' or times out, the requests in the chain will be successively accepted and processed.

=== Other Possible Race Conditions ===
The design of the SCI protocol prevents many race conditions from occurring, as has already been shown. Here are two other conditions that may arise, with a discussion of how the protocol resolves the issue.

==== Concurrent List Deletions ====

==== Simultaneous Deletion and Invalidation ====
Even though the ''Head'' node performs most of the coherence actions for a shard list, the individual nodes are able to invalidate themselves, such as when they evict the block from their cache. This is referred to in the SCI standard as a "deletion" from the sharing list. Deletion is accomplished by having the invalidating node "lock" itself and then inform its forward and back nodes that they should now point to each other. This "locking" is essentially another ''PENDING'' state. A problem could arise, then, when two neighboring nodes try to invalidate themselves at the same time, as they would both be locked and not respond to each other's message. In this case, though, the protocol specifies that the node that is closest to the tail takes priority. So, the deadlock and/or race condition is averted by ordering the deletion from the tail towards the head.

== Definitions and Terms ==
* ''Head Node'' - The node at the beginning of the sharing list
* ''Home Node'' - The node responsible for keeping track of the ''Head'' node for a sharing list for a memory block, usually the node at which the memory block resides.
* ''PENDING state'' - The intermediate state for a node while it is attempting to become the ''Head'' node in the sharing list. This includes the transaction involved in querying the ''Home'' node and the transaction involved in demoting the current ''Head'' node.
* ''SCI'' - Scalable Coherent Interface

== References ==
[[#1body|1.]] "IEEE Standard for Scalable Coherent Interface (SCI).," IEEE Std 1596-1992 , vol., no., pp.i, 1993. doi: 10.1109/IEEESTD.1993.120366
URL: http://ieeexplore.ieee.org.proxied.lib.ncsu.edu/stamp/stamp.jsp?tp=&arnumber=347683&isnumber=8049 
[[#2body|2.]] [http://www.scizzl.com/HowSCIcohWorks.html How SCI Coherence Works]

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-19T00:21:53Z

Eapotter: /* States of the Typical Set */

__TOC__

== History ==

== Background ==
The Scalable Coherent Interface (SCI) cache coherence protocol defines a set of states for memory, a set of states for cache lines, and a set of actions that transition between these states when processors access memory.

Three sets of these attributes are defined for minimal, typical, and full applications:

* '''Minimal''' - for ‘trivial but correct’ applications that require the presence of the memory in only one cache line. It does not enable read-sharing, and is appropriate for small multi-processors or applications that don’t require significant sharing.

* '''Typical''' - enables read-sharing of a memory location and provisions for efficiency, such as DMA transfers, local caching of data, and error recovery. This set adds an additional stable memory state (FRESH) and multiple cache states. This set will be the focus of this article going forward.

* '''Full''' - adds support for pair-wise sharing, QOLB lock bit synchronization, and cleansing and washing of cache lines.

== State Diagrams ==
=== Memory States ===
The memory states define the state of the memory block from the perspective of the home directory. This 2-bit field is maintained in the memory tag by the home directory along with the pointer to the head of the sharing list (forwId). This simple state model includes three stable states and one semi-stable state.

{|border="1" cellspacing="0" cellpadding="2"
!colspan="5"| Table 1: Memory States
|----
!State
!Description
!Minimal
!Typical
!Full
|----
|HOME
|no sharing list
|Y
|Y
|Y
|----
|FRESH
|sharing-list copy is the same as memory
|
|Y
|Y
|----
|GONE
|sharing-list copy may be different from memory
|Y
|Y
|Y
|----
|WASH*
|transitional state (GONE to FRESH)
|
|
|Y
|----
|}

=== Cache States ===
The cache-line states are maintained by each processors cache-coherency controller. This 7-bit field is stored in each cache line in the sharing list, along with the pointer to the next sharing-list node (forwId) and the previous sharing-list node (backID). Seven bits enable up to 128 possible values, and SCI defines twenty-nine stable-states for use in the minimal, typical, and full sets.

Stable states are those cache states that exist when a memory transaction is not in process. Their names are derived from a combination of
the position of the node in the sharing list, such as ONLY, HEAD, MID, and TAIL, and the state of the data for that node, such as FRESH, CLEAN, DIRTY, VALID, STALE, etc.

====States of the Typical Set====

Following are the states defined for the Typical set. (Note that other states may be supported to provide certain interoperability with nodes implementing the Full set, but are not used when all nodes support the Typical set.)

* '''ONLY_DIRTY''' - only one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and no other processor currently caches the block.
* '''ONLY_FRESH''' - only one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and no other processor currently caches the block.
* '''HEAD_DIRTY''' - more than one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and another processors already caches the block.
* '''HEAD_FRESH''' - more than one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and another processors already caches the block.
* '''MID_VALID''' - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is neither the Head or the Tail of the of the sharing list.
* '''TAIL_VALID''' - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is the Tail of the of the sharing list.

Below is a state diagram that depicts these states:

[[Image:cache_states.png|center]]

The request sub-actions of transactions that expect a response from the directory include 17 variations of reads, writes, and locks based on the number of blocks requested and the coherency requirements. For convenience, the notation on the edges of the above graph is derived from the terms used in the SCI specification to describe the memory transactions.

The first value indicates the request to the directory by a node for memory access:
*”Fetch R” - indicates a request for a memory block with read privileges.
*”Fetch RW” - indicates a request for a memory block with read/write privileges.
*”DATA_MODIFY” - indicates that a processor (the HEAD or the ONLY processor caching the block) writes to the cache line.

The value in the parenthesis indicates the memory state at the time of the request.

== Race Conditions ==
In a distributed shared memory system with caching, the emergence of race conditions is extremely likely. This is mainly due to the lack of a bus for serialization of actions. It is further compounded by the problem of network errors and congestion.

The early invalidation case from Section 11.4 in Solihin is an excellent example of a race condition that can arise in a distributed system. Recall the diagram from the text, and the cache coherence actions, as shown below.

[[Image:EarlyInValidationRace.png|400px|center]]
The circled actions are as follows:
# ''A'' sends a read request to ''Home''.
# ''Home'' replies with data (but the message gets delayed).
# ''B'' sends a write request to ''Home''.
# ''Home'' sends invalidation to ''A'', and it arrives before the ReplyD

The SCI protocol has a way to handle this race condition, as well as many others, and the following sections will discuss how the SCI protocol design can prevent this race condition.

=== Prevention in the SCI Protocol ===
Race conditions are almost non-existent in the SCI protocol, due primarily to the protocol's design. A brief discussion of how this is accomplished follows, followed by a discussion showing what happens if this condition arises in the SCI protocol.
==== Atomic Transactions ====
SCI's primary method for preventing race conditions is having atomic transactions. A transaction is defined as a set of sub-actions necessary to complete some requested action, such as querying memory for a data or invalidating a node's cache. This usually takes the form a a request-response pair, where the initiating node sends a request to another node and waits for the response. The transaction is not complete until the response returns. Suppose, for example, that node ''A'' is in the middle of a transaction with node ''B''. Node ''C'' then tries to make a request of node ''A''. Node ''A'' will respond to node ''C'' that it is busy, telling node ''C'' to try again, as shown in the following diagram.
[[Image:AtomicBusy.png|center]]
Thus, in the Early Invalidation race, node ''A'' would be in the middle of a transaction, having requested a block of data from memory and not having received a response, which would prevent node ''B'' from invalidating it. Note, however, that nodes will not stay in the busy state indefinitely. Doing so would lead to potential deadlocks, so all transactions have time outs that will eventually cause the transaction to fail, thus moving the node out of the busy state, making it able to respond to requests again.

==== Head Node ====
Another method for preventing race conditions is to have the ''Head'' node of a sharing list perform many of the coherence actions. As a result, only one node is performing actions such as writes and invalidations of other sharers. Since only one node is performing these actions, the possibility of concurrent actions is decreased. If another node wants to write, for instance, it must become the Head Node of the sharing list for the cache line to which it wants to write, displacing the current ''Head'' node as the node that can write. All state changes in the sharing list occur at the behest of the ''Head'' node, with the exception of nodes deleting themselves from the list.

When simply sharing a read-only copy of a node, as in when the memory is a ''FRESH'' state, the ''Head'' node is somewhat irrelevant. All the sharing nodes have their own cached value of the cache line. If any node wants to write, including the ''Head'' node, it must perform an additional action in order to do so. Likewise, when a sharing list is sharing a line of data in the dirty, or ''GONE'', memory state, as long as the ''Head'' node has not written to the line, then all the sharing nodes stay in the list with their cached line. However, at the point the ''Head'' node wants to write, it can write its data immediately, but it then must invalidate all other shared copies, via the forward pointers in the sharing list.

This, however, does not protect against a race condition where the memory is ''FRESH'', the ''Head'' node wants to write, and another node also wants to join the list. The Memory Access mechanism prevents this condition.

==== Memory Access ====
The final way that the SCI protocol minimizes race conditions is by changing the directory structure. In Solihin 11.4, there is a ''Home'' node which keeps track of the cache state and has a certain amount of knowledge about the system as a result. In the SCI protocol, there is still a ''Home'' node of sorts, but this node is only responsible for keeping track of who the current ''Head''. Such a ''Home'' node is usually the node where the memory block physically resides. When a node wants to access a block of memory, it sends a request to the ''Home'' node of that memory block, assuming it isn't already in the sharing list for that block. In the event that the data is already being used, the ''Home'' node will include in its response the address of the ''Head'' node of the sharing list. This requirement necessarily serializes the order of access, as one access request is not serviced at the ''Home'' node until the previous request is finished, and the requests are processed in FIFO order. As well, whichever node is the ''Head'' of the sharing list will be notified by the new ''Head'' when it is demoted. All parties involved in a potential race are thus talking to each other, preventing the race from occurring in the first place.

Because of this requirement, the projected race condition from the previous section, where the ''Head'' node is in a ''FRESH'' state and wants to write while another node also wants to read, cannot occur. This is because both nodes must make their request to the ''Home'' node; the ''Head'' node must request the ability to write, and the other node must request the data from the ''Home'' node. One of these requests will make it to the ''Home'' node first, resulting in the second request being deferred to the first node to reach to ''Home'' node. Such a scenario is pictured below.
[[Image:MemoryAccess.png|center]]

==== Solihin 11.4 Resolution ====
The three previous sections discussed individual parts of how the SCI protocol reduces race conditions. Putting all three of these parts together yields the following scenario which resolves the Early Invalidation race as described in the text.
[[Image:SolihinSCI.png|center]]

# ''A'' sends a request to ''Home'' for access to the memory block. It then goes into a busy state while it waits for a response.
# ''B'' also sends a request to ''Home'' for access to the same memory block. ''A'''s request is received first
# ''Home'' responds to ''A'' with the data, but this response gets caught in network traffic and delayed. This response is sent before ''Home'' processes the request from ''B''.
# ''Home'' responds to the request from ''B'', telling it that ''A'' is the current ''Head'' node.
# ''B'' then sends a request to ''A'' to tell it to demote itself.
# ''A'' still hasn't received the response from ''Home'', so it is still in a transaction, and it tells ''B'' that it is busy. ''B'' will have to retry the request.

It should be noted that ''B'' will be in the ''PENDING''[[#Definitions_and_Terms|[def]]] state, so any requests to demote it from the ''Head'' position will be delayed.. This could lead to a chaining of nodes in the ''PENDING'' state, but as soon as node ''A'' receives its response from ''Home'' or times out, the requests in the chain will be successively accepted and processed.

=== Other Possible Race Conditions ===
The design of the SCI protocol prevents many race conditions from occurring, as has already been shown. Here are two other conditions that may arise, with a discussion of how the protocol resolves the issue.

==== Concurrent List Deletions ====

==== Simultaneous Deletion and Invalidation ====
Even though the ''Head'' node performs most of the coherence actions for a shard list, the individual nodes are able to invalidate themselves, such as when they evict the block from their cache. This is referred to in the SCI standard as a "deletion" from the sharing list. Deletion is accomplished by having the invalidating node "lock" itself and then inform its forward and back nodes that they should now point to each other. This "locking" is essentially another ''PENDING'' state. A problem could arise, then, when two neighboring nodes try to invalidate themselves at the same time, as they would both be locked and not respond to each other's message. In this case, though, the protocol specifies that the node that is closest to the tail takes priority. So, the deadlock and/or race condition is averted by ordering the deletion from the tail towards the head.

== Definitions and Terms ==
* ''Head Node'' - The node at the beginning of the sharing list
* ''Home Node'' - The node responsible for keeping track of the ''Head'' node for a sharing list for a memory block, usually the node at which the memory block resides.
* ''PENDING state'' - The intermediate state for a node while it is attempting to become the ''Head'' node in the sharing list. This includes the transaction involved in querying the ''Home'' node and the transaction involved in demoting the current ''Head'' node.
* ''SCI'' - Scalable Coherent Interface

== References ==
[[#1body|1.]] "IEEE Standard for Scalable Coherent Interface (SCI).," IEEE Std 1596-1992 , vol., no., pp.i, 1993. doi: 10.1109/IEEESTD.1993.120366
URL: http://ieeexplore.ieee.org.proxied.lib.ncsu.edu/stamp/stamp.jsp?tp=&arnumber=347683&isnumber=8049 
[[#2body|2.]] [http://www.scizzl.com/HowSCIcohWorks.html How SCI Coherence Works]

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-19T00:21:38Z

Eapotter: /* States of the Typical Set */

__TOC__

== History ==

== Background ==
The Scalable Coherent Interface (SCI) cache coherence protocol defines a set of states for memory, a set of states for cache lines, and a set of actions that transition between these states when processors access memory.

Three sets of these attributes are defined for minimal, typical, and full applications:

* '''Minimal''' - for ‘trivial but correct’ applications that require the presence of the memory in only one cache line. It does not enable read-sharing, and is appropriate for small multi-processors or applications that don’t require significant sharing.

* '''Typical''' - enables read-sharing of a memory location and provisions for efficiency, such as DMA transfers, local caching of data, and error recovery. This set adds an additional stable memory state (FRESH) and multiple cache states. This set will be the focus of this article going forward.

* '''Full''' - adds support for pair-wise sharing, QOLB lock bit synchronization, and cleansing and washing of cache lines.

== State Diagrams ==
=== Memory States ===
The memory states define the state of the memory block from the perspective of the home directory. This 2-bit field is maintained in the memory tag by the home directory along with the pointer to the head of the sharing list (forwId). This simple state model includes three stable states and one semi-stable state.

{|border="1" cellspacing="0" cellpadding="2"
!colspan="5"| Table 1: Memory States
|----
!State
!Description
!Minimal
!Typical
!Full
|----
|HOME
|no sharing list
|Y
|Y
|Y
|----
|FRESH
|sharing-list copy is the same as memory
|
|Y
|Y
|----
|GONE
|sharing-list copy may be different from memory
|Y
|Y
|Y
|----
|WASH*
|transitional state (GONE to FRESH)
|
|
|Y
|----
|}

=== Cache States ===
The cache-line states are maintained by each processors cache-coherency controller. This 7-bit field is stored in each cache line in the sharing list, along with the pointer to the next sharing-list node (forwId) and the previous sharing-list node (backID). Seven bits enable up to 128 possible values, and SCI defines twenty-nine stable-states for use in the minimal, typical, and full sets.

Stable states are those cache states that exist when a memory transaction is not in process. Their names are derived from a combination of
the position of the node in the sharing list, such as ONLY, HEAD, MID, and TAIL, and the state of the data for that node, such as FRESH, CLEAN, DIRTY, VALID, STALE, etc.

====States of the Typical Set====

Following are the states defined for the Typical set. (Note that other states may be supported to provide certain interoperability with nodes implementing the Full set, but are not used when all nodes support the Typical set.)

* '''ONLY_DIRTY''' - only one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and no other processor currently caches the block.
* '''ONLY_FRESH''' - only one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and no other processor currently caches the block.
* '''HEAD_DIRTY''' - more than one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and another processors already caches the block.
* '''HEAD_FRESH''' - more than one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and another processors already caches the block.
* '''MID_VALID''' - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is neither the Head or the Tail of the of the sharing list.
* '''TAIL_VALID''' - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is the Tail of the of the sharing list.

Below is a state diagram that depicts these states:

[[Image:cache_states.png|center]]

The request sub-actions of transactions that expect a response from the directory include 17 variations of reads, writes, and locks based on the number of blocks requested and the coherency requirements. For convenience, the notation on the edges of the above graph is derived from the terms used in the SCI specification to describe the memory transactions.

The first value indicates the request to the directory by a node for memory access:
*”Fetch R” - indicates a request for a memory block with read privileges.
*”Fetch RW” - indicates a request for a memory block with read/write privileges.
*”DATA_MODIFY” - indicates that a processor (the HEAD or the ONLY processor caching the block) writes to the cache line.

The value in the parenthesis indicates the memory state at the time of the request.

== Race Conditions ==
In a distributed shared memory system with caching, the emergence of race conditions is extremely likely. This is mainly due to the lack of a bus for serialization of actions. It is further compounded by the problem of network errors and congestion.

The early invalidation case from Section 11.4 in Solihin is an excellent example of a race condition that can arise in a distributed system. Recall the diagram from the text, and the cache coherence actions, as shown below.

[[Image:EarlyInValidationRace.png|400px|center]]
The circled actions are as follows:
# ''A'' sends a read request to ''Home''.
# ''Home'' replies with data (but the message gets delayed).
# ''B'' sends a write request to ''Home''.
# ''Home'' sends invalidation to ''A'', and it arrives before the ReplyD

The SCI protocol has a way to handle this race condition, as well as many others, and the following sections will discuss how the SCI protocol design can prevent this race condition.

=== Prevention in the SCI Protocol ===
Race conditions are almost non-existent in the SCI protocol, due primarily to the protocol's design. A brief discussion of how this is accomplished follows, followed by a discussion showing what happens if this condition arises in the SCI protocol.
==== Atomic Transactions ====
SCI's primary method for preventing race conditions is having atomic transactions. A transaction is defined as a set of sub-actions necessary to complete some requested action, such as querying memory for a data or invalidating a node's cache. This usually takes the form a a request-response pair, where the initiating node sends a request to another node and waits for the response. The transaction is not complete until the response returns. Suppose, for example, that node ''A'' is in the middle of a transaction with node ''B''. Node ''C'' then tries to make a request of node ''A''. Node ''A'' will respond to node ''C'' that it is busy, telling node ''C'' to try again, as shown in the following diagram.
[[Image:AtomicBusy.png|center]]
Thus, in the Early Invalidation race, node ''A'' would be in the middle of a transaction, having requested a block of data from memory and not having received a response, which would prevent node ''B'' from invalidating it. Note, however, that nodes will not stay in the busy state indefinitely. Doing so would lead to potential deadlocks, so all transactions have time outs that will eventually cause the transaction to fail, thus moving the node out of the busy state, making it able to respond to requests again.

==== Head Node ====
Another method for preventing race conditions is to have the ''Head'' node of a sharing list perform many of the coherence actions. As a result, only one node is performing actions such as writes and invalidations of other sharers. Since only one node is performing these actions, the possibility of concurrent actions is decreased. If another node wants to write, for instance, it must become the Head Node of the sharing list for the cache line to which it wants to write, displacing the current ''Head'' node as the node that can write. All state changes in the sharing list occur at the behest of the ''Head'' node, with the exception of nodes deleting themselves from the list.

When simply sharing a read-only copy of a node, as in when the memory is a ''FRESH'' state, the ''Head'' node is somewhat irrelevant. All the sharing nodes have their own cached value of the cache line. If any node wants to write, including the ''Head'' node, it must perform an additional action in order to do so. Likewise, when a sharing list is sharing a line of data in the dirty, or ''GONE'', memory state, as long as the ''Head'' node has not written to the line, then all the sharing nodes stay in the list with their cached line. However, at the point the ''Head'' node wants to write, it can write its data immediately, but it then must invalidate all other shared copies, via the forward pointers in the sharing list.

This, however, does not protect against a race condition where the memory is ''FRESH'', the ''Head'' node wants to write, and another node also wants to join the list. The Memory Access mechanism prevents this condition.

==== Memory Access ====
The final way that the SCI protocol minimizes race conditions is by changing the directory structure. In Solihin 11.4, there is a ''Home'' node which keeps track of the cache state and has a certain amount of knowledge about the system as a result. In the SCI protocol, there is still a ''Home'' node of sorts, but this node is only responsible for keeping track of who the current ''Head''. Such a ''Home'' node is usually the node where the memory block physically resides. When a node wants to access a block of memory, it sends a request to the ''Home'' node of that memory block, assuming it isn't already in the sharing list for that block. In the event that the data is already being used, the ''Home'' node will include in its response the address of the ''Head'' node of the sharing list. This requirement necessarily serializes the order of access, as one access request is not serviced at the ''Home'' node until the previous request is finished, and the requests are processed in FIFO order. As well, whichever node is the ''Head'' of the sharing list will be notified by the new ''Head'' when it is demoted. All parties involved in a potential race are thus talking to each other, preventing the race from occurring in the first place.

Because of this requirement, the projected race condition from the previous section, where the ''Head'' node is in a ''FRESH'' state and wants to write while another node also wants to read, cannot occur. This is because both nodes must make their request to the ''Home'' node; the ''Head'' node must request the ability to write, and the other node must request the data from the ''Home'' node. One of these requests will make it to the ''Home'' node first, resulting in the second request being deferred to the first node to reach to ''Home'' node. Such a scenario is pictured below.
[[Image:MemoryAccess.png|center]]

==== Solihin 11.4 Resolution ====
The three previous sections discussed individual parts of how the SCI protocol reduces race conditions. Putting all three of these parts together yields the following scenario which resolves the Early Invalidation race as described in the text.
[[Image:SolihinSCI.png|center]]

# ''A'' sends a request to ''Home'' for access to the memory block. It then goes into a busy state while it waits for a response.
# ''B'' also sends a request to ''Home'' for access to the same memory block. ''A'''s request is received first
# ''Home'' responds to ''A'' with the data, but this response gets caught in network traffic and delayed. This response is sent before ''Home'' processes the request from ''B''.
# ''Home'' responds to the request from ''B'', telling it that ''A'' is the current ''Head'' node.
# ''B'' then sends a request to ''A'' to tell it to demote itself.
# ''A'' still hasn't received the response from ''Home'', so it is still in a transaction, and it tells ''B'' that it is busy. ''B'' will have to retry the request.

It should be noted that ''B'' will be in the ''PENDING''[[#Definitions_and_Terms|[def]]] state, so any requests to demote it from the ''Head'' position will be delayed.. This could lead to a chaining of nodes in the ''PENDING'' state, but as soon as node ''A'' receives its response from ''Home'' or times out, the requests in the chain will be successively accepted and processed.

=== Other Possible Race Conditions ===
The design of the SCI protocol prevents many race conditions from occurring, as has already been shown. Here are two other conditions that may arise, with a discussion of how the protocol resolves the issue.

==== Concurrent List Deletions ====

==== Simultaneous Deletion and Invalidation ====
Even though the ''Head'' node performs most of the coherence actions for a shard list, the individual nodes are able to invalidate themselves, such as when they evict the block from their cache. This is referred to in the SCI standard as a "deletion" from the sharing list. Deletion is accomplished by having the invalidating node "lock" itself and then inform its forward and back nodes that they should now point to each other. This "locking" is essentially another ''PENDING'' state. A problem could arise, then, when two neighboring nodes try to invalidate themselves at the same time, as they would both be locked and not respond to each other's message. In this case, though, the protocol specifies that the node that is closest to the tail takes priority. So, the deadlock and/or race condition is averted by ordering the deletion from the tail towards the head.

== Definitions and Terms ==
* ''Head Node'' - The node at the beginning of the sharing list
* ''Home Node'' - The node responsible for keeping track of the ''Head'' node for a sharing list for a memory block, usually the node at which the memory block resides.
* ''PENDING state'' - The intermediate state for a node while it is attempting to become the ''Head'' node in the sharing list. This includes the transaction involved in querying the ''Home'' node and the transaction involved in demoting the current ''Head'' node.
* ''SCI'' - Scalable Coherent Interface

== References ==
[[#1body|1.]] "IEEE Standard for Scalable Coherent Interface (SCI).," IEEE Std 1596-1992 , vol., no., pp.i, 1993. doi: 10.1109/IEEESTD.1993.120366
URL: http://ieeexplore.ieee.org.proxied.lib.ncsu.edu/stamp/stamp.jsp?tp=&arnumber=347683&isnumber=8049 
[[#2body|2.]] [http://www.scizzl.com/HowSCIcohWorks.html How SCI Coherence Works]

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-19T00:20:50Z

Eapotter: /* States of the Typical Set */

__TOC__

== History ==

== Background ==
The Scalable Coherent Interface (SCI) cache coherence protocol defines a set of states for memory, a set of states for cache lines, and a set of actions that transition between these states when processors access memory.

Three sets of these attributes are defined for minimal, typical, and full applications:

* '''Minimal''' - for ‘trivial but correct’ applications that require the presence of the memory in only one cache line. It does not enable read-sharing, and is appropriate for small multi-processors or applications that don’t require significant sharing.

* '''Typical''' - enables read-sharing of a memory location and provisions for efficiency, such as DMA transfers, local caching of data, and error recovery. This set adds an additional stable memory state (FRESH) and multiple cache states. This set will be the focus of this article going forward.

* '''Full''' - adds support for pair-wise sharing, QOLB lock bit synchronization, and cleansing and washing of cache lines.

== State Diagrams ==
=== Memory States ===
The memory states define the state of the memory block from the perspective of the home directory. This 2-bit field is maintained in the memory tag by the home directory along with the pointer to the head of the sharing list (forwId). This simple state model includes three stable states and one semi-stable state.

{|border="1" cellspacing="0" cellpadding="2"
!colspan="5"| Table 1: Memory States
|----
!State
!Description
!Minimal
!Typical
!Full
|----
|HOME
|no sharing list
|Y
|Y
|Y
|----
|FRESH
|sharing-list copy is the same as memory
|
|Y
|Y
|----
|GONE
|sharing-list copy may be different from memory
|Y
|Y
|Y
|----
|WASH*
|transitional state (GONE to FRESH)
|
|
|Y
|----
|}

=== Cache States ===
The cache-line states are maintained by each processors cache-coherency controller. This 7-bit field is stored in each cache line in the sharing list, along with the pointer to the next sharing-list node (forwId) and the previous sharing-list node (backID). Seven bits enable up to 128 possible values, and SCI defines twenty-nine stable-states for use in the minimal, typical, and full sets.

Stable states are those cache states that exist when a memory transaction is not in process. Their names are derived from a combination of
the position of the node in the sharing list, such as ONLY, HEAD, MID, and TAIL, and the state of the data for that node, such as FRESH, CLEAN, DIRTY, VALID, STALE, etc.

====States of the Typical Set====

Following are the states defined for the Typical set. (Note that other states may be supported to provide certain interoperability with nodes implementing the Full set, but are not used when all nodes support the Typical set.)

* '''ONLY_DIRTY”' - only one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and no other processor currently caches the block.
* '''ONLY_FRESH”' - only one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and no other processor currently caches the block.
* '''HEAD_DIRTY”' - more than one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and another processors already caches the block.
* '''HEAD_FRESH”' - more than one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and another processors already caches the block.
* '''MID_VALID”' - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is neither the Head or the Tail of the of the sharing list.
* '''TAIL_VALID”' - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is the Tail of the of the sharing list.

Below is a state diagram that depicts these states:

[[Image:cache_states.png|center]]

The request sub-actions of transactions that expect a response from the directory include 17 variations of reads, writes, and locks based on the number of blocks requested and the coherency requirements. For convenience, the notation on the edges of the above graph is derived from the terms used in the SCI specification to describe the memory transactions.

The first value indicates the request to the directory by a node for memory access:
*”Fetch R” - indicates a request for a memory block with read privileges.
*”Fetch RW” - indicates a request for a memory block with read/write privileges.
*”DATA_MODIFY” - indicates that a processor (the HEAD or the ONLY processor caching the block) writes to the cache line.

The value in the parenthesis indicates the memory state at the time of the request.

== Race Conditions ==
In a distributed shared memory system with caching, the emergence of race conditions is extremely likely. This is mainly due to the lack of a bus for serialization of actions. It is further compounded by the problem of network errors and congestion.

The early invalidation case from Section 11.4 in Solihin is an excellent example of a race condition that can arise in a distributed system. Recall the diagram from the text, and the cache coherence actions, as shown below.

[[Image:EarlyInValidationRace.png|400px|center]]
The circled actions are as follows:
# ''A'' sends a read request to ''Home''.
# ''Home'' replies with data (but the message gets delayed).
# ''B'' sends a write request to ''Home''.
# ''Home'' sends invalidation to ''A'', and it arrives before the ReplyD

The SCI protocol has a way to handle this race condition, as well as many others, and the following sections will discuss how the SCI protocol design can prevent this race condition.

=== Prevention in the SCI Protocol ===
Race conditions are almost non-existent in the SCI protocol, due primarily to the protocol's design. A brief discussion of how this is accomplished follows, followed by a discussion showing what happens if this condition arises in the SCI protocol.
==== Atomic Transactions ====
SCI's primary method for preventing race conditions is having atomic transactions. A transaction is defined as a set of sub-actions necessary to complete some requested action, such as querying memory for a data or invalidating a node's cache. This usually takes the form a a request-response pair, where the initiating node sends a request to another node and waits for the response. The transaction is not complete until the response returns. Suppose, for example, that node ''A'' is in the middle of a transaction with node ''B''. Node ''C'' then tries to make a request of node ''A''. Node ''A'' will respond to node ''C'' that it is busy, telling node ''C'' to try again, as shown in the following diagram.
[[Image:AtomicBusy.png|center]]
Thus, in the Early Invalidation race, node ''A'' would be in the middle of a transaction, having requested a block of data from memory and not having received a response, which would prevent node ''B'' from invalidating it. Note, however, that nodes will not stay in the busy state indefinitely. Doing so would lead to potential deadlocks, so all transactions have time outs that will eventually cause the transaction to fail, thus moving the node out of the busy state, making it able to respond to requests again.

==== Head Node ====
Another method for preventing race conditions is to have the ''Head'' node of a sharing list perform many of the coherence actions. As a result, only one node is performing actions such as writes and invalidations of other sharers. Since only one node is performing these actions, the possibility of concurrent actions is decreased. If another node wants to write, for instance, it must become the Head Node of the sharing list for the cache line to which it wants to write, displacing the current ''Head'' node as the node that can write. All state changes in the sharing list occur at the behest of the ''Head'' node, with the exception of nodes deleting themselves from the list.

When simply sharing a read-only copy of a node, as in when the memory is a ''FRESH'' state, the ''Head'' node is somewhat irrelevant. All the sharing nodes have their own cached value of the cache line. If any node wants to write, including the ''Head'' node, it must perform an additional action in order to do so. Likewise, when a sharing list is sharing a line of data in the dirty, or ''GONE'', memory state, as long as the ''Head'' node has not written to the line, then all the sharing nodes stay in the list with their cached line. However, at the point the ''Head'' node wants to write, it can write its data immediately, but it then must invalidate all other shared copies, via the forward pointers in the sharing list.

This, however, does not protect against a race condition where the memory is ''FRESH'', the ''Head'' node wants to write, and another node also wants to join the list. The Memory Access mechanism prevents this condition.

==== Memory Access ====
The final way that the SCI protocol minimizes race conditions is by changing the directory structure. In Solihin 11.4, there is a ''Home'' node which keeps track of the cache state and has a certain amount of knowledge about the system as a result. In the SCI protocol, there is still a ''Home'' node of sorts, but this node is only responsible for keeping track of who the current ''Head''. Such a ''Home'' node is usually the node where the memory block physically resides. When a node wants to access a block of memory, it sends a request to the ''Home'' node of that memory block, assuming it isn't already in the sharing list for that block. In the event that the data is already being used, the ''Home'' node will include in its response the address of the ''Head'' node of the sharing list. This requirement necessarily serializes the order of access, as one access request is not serviced at the ''Home'' node until the previous request is finished, and the requests are processed in FIFO order. As well, whichever node is the ''Head'' of the sharing list will be notified by the new ''Head'' when it is demoted. All parties involved in a potential race are thus talking to each other, preventing the race from occurring in the first place.

Because of this requirement, the projected race condition from the previous section, where the ''Head'' node is in a ''FRESH'' state and wants to write while another node also wants to read, cannot occur. This is because both nodes must make their request to the ''Home'' node; the ''Head'' node must request the ability to write, and the other node must request the data from the ''Home'' node. One of these requests will make it to the ''Home'' node first, resulting in the second request being deferred to the first node to reach to ''Home'' node. Such a scenario is pictured below.
[[Image:MemoryAccess.png|center]]

==== Solihin 11.4 Resolution ====
The three previous sections discussed individual parts of how the SCI protocol reduces race conditions. Putting all three of these parts together yields the following scenario which resolves the Early Invalidation race as described in the text.
[[Image:SolihinSCI.png|center]]

# ''A'' sends a request to ''Home'' for access to the memory block. It then goes into a busy state while it waits for a response.
# ''B'' also sends a request to ''Home'' for access to the same memory block. ''A'''s request is received first
# ''Home'' responds to ''A'' with the data, but this response gets caught in network traffic and delayed. This response is sent before ''Home'' processes the request from ''B''.
# ''Home'' responds to the request from ''B'', telling it that ''A'' is the current ''Head'' node.
# ''B'' then sends a request to ''A'' to tell it to demote itself.
# ''A'' still hasn't received the response from ''Home'', so it is still in a transaction, and it tells ''B'' that it is busy. ''B'' will have to retry the request.

It should be noted that ''B'' will be in the ''PENDING''[[#Definitions_and_Terms|[def]]] state, so any requests to demote it from the ''Head'' position will be delayed.. This could lead to a chaining of nodes in the ''PENDING'' state, but as soon as node ''A'' receives its response from ''Home'' or times out, the requests in the chain will be successively accepted and processed.

=== Other Possible Race Conditions ===
The design of the SCI protocol prevents many race conditions from occurring, as has already been shown. Here are two other conditions that may arise, with a discussion of how the protocol resolves the issue.

==== Concurrent List Deletions ====

==== Simultaneous Deletion and Invalidation ====
Even though the ''Head'' node performs most of the coherence actions for a shard list, the individual nodes are able to invalidate themselves, such as when they evict the block from their cache. This is referred to in the SCI standard as a "deletion" from the sharing list. Deletion is accomplished by having the invalidating node "lock" itself and then inform its forward and back nodes that they should now point to each other. This "locking" is essentially another ''PENDING'' state. A problem could arise, then, when two neighboring nodes try to invalidate themselves at the same time, as they would both be locked and not respond to each other's message. In this case, though, the protocol specifies that the node that is closest to the tail takes priority. So, the deadlock and/or race condition is averted by ordering the deletion from the tail towards the head.

== Definitions and Terms ==
* ''Head Node'' - The node at the beginning of the sharing list
* ''Home Node'' - The node responsible for keeping track of the ''Head'' node for a sharing list for a memory block, usually the node at which the memory block resides.
* ''PENDING state'' - The intermediate state for a node while it is attempting to become the ''Head'' node in the sharing list. This includes the transaction involved in querying the ''Home'' node and the transaction involved in demoting the current ''Head'' node.
* ''SCI'' - Scalable Coherent Interface

== References ==
[[#1body|1.]] "IEEE Standard for Scalable Coherent Interface (SCI).," IEEE Std 1596-1992 , vol., no., pp.i, 1993. doi: 10.1109/IEEESTD.1993.120366
URL: http://ieeexplore.ieee.org.proxied.lib.ncsu.edu/stamp/stamp.jsp?tp=&arnumber=347683&isnumber=8049 
[[#2body|2.]] [http://www.scizzl.com/HowSCIcohWorks.html How SCI Coherence Works]

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-19T00:19:58Z

Eapotter: /* States of the Typical Set */

__TOC__

== History ==

== Background ==
The Scalable Coherent Interface (SCI) cache coherence protocol defines a set of states for memory, a set of states for cache lines, and a set of actions that transition between these states when processors access memory.

Three sets of these attributes are defined for minimal, typical, and full applications:

* '''Minimal''' - for ‘trivial but correct’ applications that require the presence of the memory in only one cache line. It does not enable read-sharing, and is appropriate for small multi-processors or applications that don’t require significant sharing.

* '''Typical''' - enables read-sharing of a memory location and provisions for efficiency, such as DMA transfers, local caching of data, and error recovery. This set adds an additional stable memory state (FRESH) and multiple cache states. This set will be the focus of this article going forward.

* '''Full''' - adds support for pair-wise sharing, QOLB lock bit synchronization, and cleansing and washing of cache lines.

== State Diagrams ==
=== Memory States ===
The memory states define the state of the memory block from the perspective of the home directory. This 2-bit field is maintained in the memory tag by the home directory along with the pointer to the head of the sharing list (forwId). This simple state model includes three stable states and one semi-stable state.

{|border="1" cellspacing="0" cellpadding="2"
!colspan="5"| Table 1: Memory States
|----
!State
!Description
!Minimal
!Typical
!Full
|----
|HOME
|no sharing list
|Y
|Y
|Y
|----
|FRESH
|sharing-list copy is the same as memory
|
|Y
|Y
|----
|GONE
|sharing-list copy may be different from memory
|Y
|Y
|Y
|----
|WASH*
|transitional state (GONE to FRESH)
|
|
|Y
|----
|}

=== Cache States ===
The cache-line states are maintained by each processors cache-coherency controller. This 7-bit field is stored in each cache line in the sharing list, along with the pointer to the next sharing-list node (forwId) and the previous sharing-list node (backID). Seven bits enable up to 128 possible values, and SCI defines twenty-nine stable-states for use in the minimal, typical, and full sets.

Stable states are those cache states that exist when a memory transaction is not in process. Their names are derived from a combination of
the position of the node in the sharing list, such as ONLY, HEAD, MID, and TAIL, and the state of the data for that node, such as FRESH, CLEAN, DIRTY, VALID, STALE, etc.

====States of the Typical Set====

Following are the states defined for the Typical set. (Note that other states may be supported to provide certain interoperability with nodes implementing the Full set, but are not used when all nodes support the Typical set.)

* '“ONLY_DIRTY”' - only one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and no other processor currently caches the block.
* '”ONLY_FRESH”' - only one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and no other processor currently caches the block.
* '”HEAD_DIRTY”' - more than one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and another processors already caches the block.
*”HEAD_FRESH”' - more than one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and another processors already caches the block.
* '”MID_VALID”' - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is neither the Head or the Tail of the of the sharing list.
* '”TAIL_VALID”' - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is the Tail of the of the sharing list.

Below is a state diagram that depicts these states:

[[Image:cache_states.png|center]]

The request sub-actions of transactions that expect a response from the directory include 17 variations of reads, writes, and locks based on the number of blocks requested and the coherency requirements. For convenience, the notation on the edges of the above graph is derived from the terms used in the SCI specification to describe the memory transactions.

The first value indicates the request to the directory by a node for memory access:
*”Fetch R” - indicates a request for a memory block with read privileges.
*”Fetch RW” - indicates a request for a memory block with read/write privileges.
*”DATA_MODIFY” - indicates that a processor (the HEAD or the ONLY processor caching the block) writes to the cache line.

The value in the parenthesis indicates the memory state at the time of the request.

== Race Conditions ==
In a distributed shared memory system with caching, the emergence of race conditions is extremely likely. This is mainly due to the lack of a bus for serialization of actions. It is further compounded by the problem of network errors and congestion.

The early invalidation case from Section 11.4 in Solihin is an excellent example of a race condition that can arise in a distributed system. Recall the diagram from the text, and the cache coherence actions, as shown below.

[[Image:EarlyInValidationRace.png|400px|center]]
The circled actions are as follows:
# ''A'' sends a read request to ''Home''.
# ''Home'' replies with data (but the message gets delayed).
# ''B'' sends a write request to ''Home''.
# ''Home'' sends invalidation to ''A'', and it arrives before the ReplyD

The SCI protocol has a way to handle this race condition, as well as many others, and the following sections will discuss how the SCI protocol design can prevent this race condition.

=== Prevention in the SCI Protocol ===
Race conditions are almost non-existent in the SCI protocol, due primarily to the protocol's design. A brief discussion of how this is accomplished follows, followed by a discussion showing what happens if this condition arises in the SCI protocol.
==== Atomic Transactions ====
SCI's primary method for preventing race conditions is having atomic transactions. A transaction is defined as a set of sub-actions necessary to complete some requested action, such as querying memory for a data or invalidating a node's cache. This usually takes the form a a request-response pair, where the initiating node sends a request to another node and waits for the response. The transaction is not complete until the response returns. Suppose, for example, that node ''A'' is in the middle of a transaction with node ''B''. Node ''C'' then tries to make a request of node ''A''. Node ''A'' will respond to node ''C'' that it is busy, telling node ''C'' to try again, as shown in the following diagram.
[[Image:AtomicBusy.png|center]]
Thus, in the Early Invalidation race, node ''A'' would be in the middle of a transaction, having requested a block of data from memory and not having received a response, which would prevent node ''B'' from invalidating it. Note, however, that nodes will not stay in the busy state indefinitely. Doing so would lead to potential deadlocks, so all transactions have time outs that will eventually cause the transaction to fail, thus moving the node out of the busy state, making it able to respond to requests again.

==== Head Node ====
Another method for preventing race conditions is to have the ''Head'' node of a sharing list perform many of the coherence actions. As a result, only one node is performing actions such as writes and invalidations of other sharers. Since only one node is performing these actions, the possibility of concurrent actions is decreased. If another node wants to write, for instance, it must become the Head Node of the sharing list for the cache line to which it wants to write, displacing the current ''Head'' node as the node that can write. All state changes in the sharing list occur at the behest of the ''Head'' node, with the exception of nodes deleting themselves from the list.

When simply sharing a read-only copy of a node, as in when the memory is a ''FRESH'' state, the ''Head'' node is somewhat irrelevant. All the sharing nodes have their own cached value of the cache line. If any node wants to write, including the ''Head'' node, it must perform an additional action in order to do so. Likewise, when a sharing list is sharing a line of data in the dirty, or ''GONE'', memory state, as long as the ''Head'' node has not written to the line, then all the sharing nodes stay in the list with their cached line. However, at the point the ''Head'' node wants to write, it can write its data immediately, but it then must invalidate all other shared copies, via the forward pointers in the sharing list.

This, however, does not protect against a race condition where the memory is ''FRESH'', the ''Head'' node wants to write, and another node also wants to join the list. The Memory Access mechanism prevents this condition.

==== Memory Access ====
The final way that the SCI protocol minimizes race conditions is by changing the directory structure. In Solihin 11.4, there is a ''Home'' node which keeps track of the cache state and has a certain amount of knowledge about the system as a result. In the SCI protocol, there is still a ''Home'' node of sorts, but this node is only responsible for keeping track of who the current ''Head''. Such a ''Home'' node is usually the node where the memory block physically resides. When a node wants to access a block of memory, it sends a request to the ''Home'' node of that memory block, assuming it isn't already in the sharing list for that block. In the event that the data is already being used, the ''Home'' node will include in its response the address of the ''Head'' node of the sharing list. This requirement necessarily serializes the order of access, as one access request is not serviced at the ''Home'' node until the previous request is finished, and the requests are processed in FIFO order. As well, whichever node is the ''Head'' of the sharing list will be notified by the new ''Head'' when it is demoted. All parties involved in a potential race are thus talking to each other, preventing the race from occurring in the first place.

Because of this requirement, the projected race condition from the previous section, where the ''Head'' node is in a ''FRESH'' state and wants to write while another node also wants to read, cannot occur. This is because both nodes must make their request to the ''Home'' node; the ''Head'' node must request the ability to write, and the other node must request the data from the ''Home'' node. One of these requests will make it to the ''Home'' node first, resulting in the second request being deferred to the first node to reach to ''Home'' node. Such a scenario is pictured below.
[[Image:MemoryAccess.png|center]]

==== Solihin 11.4 Resolution ====
The three previous sections discussed individual parts of how the SCI protocol reduces race conditions. Putting all three of these parts together yields the following scenario which resolves the Early Invalidation race as described in the text.
[[Image:SolihinSCI.png|center]]

# ''A'' sends a request to ''Home'' for access to the memory block. It then goes into a busy state while it waits for a response.
# ''B'' also sends a request to ''Home'' for access to the same memory block. ''A'''s request is received first
# ''Home'' responds to ''A'' with the data, but this response gets caught in network traffic and delayed. This response is sent before ''Home'' processes the request from ''B''.
# ''Home'' responds to the request from ''B'', telling it that ''A'' is the current ''Head'' node.
# ''B'' then sends a request to ''A'' to tell it to demote itself.
# ''A'' still hasn't received the response from ''Home'', so it is still in a transaction, and it tells ''B'' that it is busy. ''B'' will have to retry the request.

It should be noted that ''B'' will be in the ''PENDING''[[#Definitions_and_Terms|[def]]] state, so any requests to demote it from the ''Head'' position will be delayed.. This could lead to a chaining of nodes in the ''PENDING'' state, but as soon as node ''A'' receives its response from ''Home'' or times out, the requests in the chain will be successively accepted and processed.

=== Other Possible Race Conditions ===
The design of the SCI protocol prevents many race conditions from occurring, as has already been shown. Here are two other conditions that may arise, with a discussion of how the protocol resolves the issue.

==== Concurrent List Deletions ====

==== Simultaneous Deletion and Invalidation ====
Even though the ''Head'' node performs most of the coherence actions for a shard list, the individual nodes are able to invalidate themselves, such as when they evict the block from their cache. This is referred to in the SCI standard as a "deletion" from the sharing list. Deletion is accomplished by having the invalidating node "lock" itself and then inform its forward and back nodes that they should now point to each other. This "locking" is essentially another ''PENDING'' state. A problem could arise, then, when two neighboring nodes try to invalidate themselves at the same time, as they would both be locked and not respond to each other's message. In this case, though, the protocol specifies that the node that is closest to the tail takes priority. So, the deadlock and/or race condition is averted by ordering the deletion from the tail towards the head.

== Definitions and Terms ==
* ''Head Node'' - The node at the beginning of the sharing list
* ''Home Node'' - The node responsible for keeping track of the ''Head'' node for a sharing list for a memory block, usually the node at which the memory block resides.
* ''PENDING state'' - The intermediate state for a node while it is attempting to become the ''Head'' node in the sharing list. This includes the transaction involved in querying the ''Home'' node and the transaction involved in demoting the current ''Head'' node.
* ''SCI'' - Scalable Coherent Interface

== References ==
[[#1body|1.]] "IEEE Standard for Scalable Coherent Interface (SCI).," IEEE Std 1596-1992 , vol., no., pp.i, 1993. doi: 10.1109/IEEESTD.1993.120366
URL: http://ieeexplore.ieee.org.proxied.lib.ncsu.edu/stamp/stamp.jsp?tp=&arnumber=347683&isnumber=8049 
[[#2body|2.]] [http://www.scizzl.com/HowSCIcohWorks.html How SCI Coherence Works]

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-19T00:18:44Z

Eapotter: /* States of the Typical Set */

__TOC__

== History ==

== Background ==
The Scalable Coherent Interface (SCI) cache coherence protocol defines a set of states for memory, a set of states for cache lines, and a set of actions that transition between these states when processors access memory.

Three sets of these attributes are defined for minimal, typical, and full applications:

* '''Minimal''' - for ‘trivial but correct’ applications that require the presence of the memory in only one cache line. It does not enable read-sharing, and is appropriate for small multi-processors or applications that don’t require significant sharing.

* '''Typical''' - enables read-sharing of a memory location and provisions for efficiency, such as DMA transfers, local caching of data, and error recovery. This set adds an additional stable memory state (FRESH) and multiple cache states. This set will be the focus of this article going forward.

* '''Full''' - adds support for pair-wise sharing, QOLB lock bit synchronization, and cleansing and washing of cache lines.

== State Diagrams ==
=== Memory States ===
The memory states define the state of the memory block from the perspective of the home directory. This 2-bit field is maintained in the memory tag by the home directory along with the pointer to the head of the sharing list (forwId). This simple state model includes three stable states and one semi-stable state.

{|border="1" cellspacing="0" cellpadding="2"
!colspan="5"| Table 1: Memory States
|----
!State
!Description
!Minimal
!Typical
!Full
|----
|HOME
|no sharing list
|Y
|Y
|Y
|----
|FRESH
|sharing-list copy is the same as memory
|
|Y
|Y
|----
|GONE
|sharing-list copy may be different from memory
|Y
|Y
|Y
|----
|WASH*
|transitional state (GONE to FRESH)
|
|
|Y
|----
|}

=== Cache States ===
The cache-line states are maintained by each processors cache-coherency controller. This 7-bit field is stored in each cache line in the sharing list, along with the pointer to the next sharing-list node (forwId) and the previous sharing-list node (backID). Seven bits enable up to 128 possible values, and SCI defines twenty-nine stable-states for use in the minimal, typical, and full sets.

Stable states are those cache states that exist when a memory transaction is not in process. Their names are derived from a combination of
the position of the node in the sharing list, such as ONLY, HEAD, MID, and TAIL, and the state of the data for that node, such as FRESH, CLEAN, DIRTY, VALID, STALE, etc.

====States of the Typical Set====

Following are the states defined for the Typical set. (Note that other states may be supported to provide certain interoperability with nodes implementing the Full set, but are not used when all nodes support the Typical set.)

*“ONLY_DIRTY” - only one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and no other processor currently caches the block.
*”ONLY_FRESH” - only one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and no other processor currently caches the block.
*”HEAD_DIRTY” - more than one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and another processors already caches the block.
*”HEAD_FRESH” - more than one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and another processors already caches the block.
*”MID_VALID” - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is neither the Head or the Tail of the of the sharing list.
*”TAIL_VALID” - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is the Tail of the of the sharing list.

Below is a state diagram that depicts these states:

[[Image:cache_states.png|center]]

The request sub-actions of transactions that expect a response from the directory include 17 variations of reads, writes, and locks based on the number of blocks requested and the coherency requirements. For convenience, the notation on the edges of the above graph is derived from the terms used in the SCI specification to describe the memory transactions.

The first value indicates the request to the directory by a node for memory access:
*”Fetch R” - indicates a request for a memory block with read privileges.
*”Fetch RW” - indicates a request for a memory block with read/write privileges.
*”DATA_MODIFY” - indicates that a processor (the HEAD or the ONLY processor caching the block) writes to the cache line.

The value in the parenthesis indicates the memory state at the time of the request.

== Race Conditions ==
In a distributed shared memory system with caching, the emergence of race conditions is extremely likely. This is mainly due to the lack of a bus for serialization of actions. It is further compounded by the problem of network errors and congestion.

The early invalidation case from Section 11.4 in Solihin is an excellent example of a race condition that can arise in a distributed system. Recall the diagram from the text, and the cache coherence actions, as shown below.

[[Image:EarlyInValidationRace.png|400px|center]]
The circled actions are as follows:
# ''A'' sends a read request to ''Home''.
# ''Home'' replies with data (but the message gets delayed).
# ''B'' sends a write request to ''Home''.
# ''Home'' sends invalidation to ''A'', and it arrives before the ReplyD

The SCI protocol has a way to handle this race condition, as well as many others, and the following sections will discuss how the SCI protocol design can prevent this race condition.

=== Prevention in the SCI Protocol ===
Race conditions are almost non-existent in the SCI protocol, due primarily to the protocol's design. A brief discussion of how this is accomplished follows, followed by a discussion showing what happens if this condition arises in the SCI protocol.
==== Atomic Transactions ====
SCI's primary method for preventing race conditions is having atomic transactions. A transaction is defined as a set of sub-actions necessary to complete some requested action, such as querying memory for a data or invalidating a node's cache. This usually takes the form a a request-response pair, where the initiating node sends a request to another node and waits for the response. The transaction is not complete until the response returns. Suppose, for example, that node ''A'' is in the middle of a transaction with node ''B''. Node ''C'' then tries to make a request of node ''A''. Node ''A'' will respond to node ''C'' that it is busy, telling node ''C'' to try again, as shown in the following diagram.
[[Image:AtomicBusy.png|center]]
Thus, in the Early Invalidation race, node ''A'' would be in the middle of a transaction, having requested a block of data from memory and not having received a response, which would prevent node ''B'' from invalidating it. Note, however, that nodes will not stay in the busy state indefinitely. Doing so would lead to potential deadlocks, so all transactions have time outs that will eventually cause the transaction to fail, thus moving the node out of the busy state, making it able to respond to requests again.

==== Head Node ====
Another method for preventing race conditions is to have the ''Head'' node of a sharing list perform many of the coherence actions. As a result, only one node is performing actions such as writes and invalidations of other sharers. Since only one node is performing these actions, the possibility of concurrent actions is decreased. If another node wants to write, for instance, it must become the Head Node of the sharing list for the cache line to which it wants to write, displacing the current ''Head'' node as the node that can write. All state changes in the sharing list occur at the behest of the ''Head'' node, with the exception of nodes deleting themselves from the list.

When simply sharing a read-only copy of a node, as in when the memory is a ''FRESH'' state, the ''Head'' node is somewhat irrelevant. All the sharing nodes have their own cached value of the cache line. If any node wants to write, including the ''Head'' node, it must perform an additional action in order to do so. Likewise, when a sharing list is sharing a line of data in the dirty, or ''GONE'', memory state, as long as the ''Head'' node has not written to the line, then all the sharing nodes stay in the list with their cached line. However, at the point the ''Head'' node wants to write, it can write its data immediately, but it then must invalidate all other shared copies, via the forward pointers in the sharing list.

This, however, does not protect against a race condition where the memory is ''FRESH'', the ''Head'' node wants to write, and another node also wants to join the list. The Memory Access mechanism prevents this condition.

==== Memory Access ====
The final way that the SCI protocol minimizes race conditions is by changing the directory structure. In Solihin 11.4, there is a ''Home'' node which keeps track of the cache state and has a certain amount of knowledge about the system as a result. In the SCI protocol, there is still a ''Home'' node of sorts, but this node is only responsible for keeping track of who the current ''Head''. Such a ''Home'' node is usually the node where the memory block physically resides. When a node wants to access a block of memory, it sends a request to the ''Home'' node of that memory block, assuming it isn't already in the sharing list for that block. In the event that the data is already being used, the ''Home'' node will include in its response the address of the ''Head'' node of the sharing list. This requirement necessarily serializes the order of access, as one access request is not serviced at the ''Home'' node until the previous request is finished, and the requests are processed in FIFO order. As well, whichever node is the ''Head'' of the sharing list will be notified by the new ''Head'' when it is demoted. All parties involved in a potential race are thus talking to each other, preventing the race from occurring in the first place.

Because of this requirement, the projected race condition from the previous section, where the ''Head'' node is in a ''FRESH'' state and wants to write while another node also wants to read, cannot occur. This is because both nodes must make their request to the ''Home'' node; the ''Head'' node must request the ability to write, and the other node must request the data from the ''Home'' node. One of these requests will make it to the ''Home'' node first, resulting in the second request being deferred to the first node to reach to ''Home'' node. Such a scenario is pictured below.
[[Image:MemoryAccess.png|center]]

==== Solihin 11.4 Resolution ====
The three previous sections discussed individual parts of how the SCI protocol reduces race conditions. Putting all three of these parts together yields the following scenario which resolves the Early Invalidation race as described in the text.
[[Image:SolihinSCI.png|center]]

# ''A'' sends a request to ''Home'' for access to the memory block. It then goes into a busy state while it waits for a response.
# ''B'' also sends a request to ''Home'' for access to the same memory block. ''A'''s request is received first
# ''Home'' responds to ''A'' with the data, but this response gets caught in network traffic and delayed. This response is sent before ''Home'' processes the request from ''B''.
# ''Home'' responds to the request from ''B'', telling it that ''A'' is the current ''Head'' node.
# ''B'' then sends a request to ''A'' to tell it to demote itself.
# ''A'' still hasn't received the response from ''Home'', so it is still in a transaction, and it tells ''B'' that it is busy. ''B'' will have to retry the request.

It should be noted that ''B'' will be in the ''PENDING''[[#Definitions_and_Terms|[def]]] state, so any requests to demote it from the ''Head'' position will be delayed.. This could lead to a chaining of nodes in the ''PENDING'' state, but as soon as node ''A'' receives its response from ''Home'' or times out, the requests in the chain will be successively accepted and processed.

=== Other Possible Race Conditions ===
The design of the SCI protocol prevents many race conditions from occurring, as has already been shown. Here are two other conditions that may arise, with a discussion of how the protocol resolves the issue.

==== Concurrent List Deletions ====

==== Simultaneous Deletion and Invalidation ====
Even though the ''Head'' node performs most of the coherence actions for a shard list, the individual nodes are able to invalidate themselves, such as when they evict the block from their cache. This is referred to in the SCI standard as a "deletion" from the sharing list. Deletion is accomplished by having the invalidating node "lock" itself and then inform its forward and back nodes that they should now point to each other. This "locking" is essentially another ''PENDING'' state. A problem could arise, then, when two neighboring nodes try to invalidate themselves at the same time, as they would both be locked and not respond to each other's message. In this case, though, the protocol specifies that the node that is closest to the tail takes priority. So, the deadlock and/or race condition is averted by ordering the deletion from the tail towards the head.

== Definitions and Terms ==
* ''Head Node'' - The node at the beginning of the sharing list
* ''Home Node'' - The node responsible for keeping track of the ''Head'' node for a sharing list for a memory block, usually the node at which the memory block resides.
* ''PENDING state'' - The intermediate state for a node while it is attempting to become the ''Head'' node in the sharing list. This includes the transaction involved in querying the ''Home'' node and the transaction involved in demoting the current ''Head'' node.
* ''SCI'' - Scalable Coherent Interface

== References ==
[[#1body|1.]] "IEEE Standard for Scalable Coherent Interface (SCI).," IEEE Std 1596-1992 , vol., no., pp.i, 1993. doi: 10.1109/IEEESTD.1993.120366
URL: http://ieeexplore.ieee.org.proxied.lib.ncsu.edu/stamp/stamp.jsp?tp=&arnumber=347683&isnumber=8049 
[[#2body|2.]] [http://www.scizzl.com/HowSCIcohWorks.html How SCI Coherence Works]

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-19T00:18:03Z

Eapotter: /* States of the Typical Set */

__TOC__

== History ==

== Background ==
The Scalable Coherent Interface (SCI) cache coherence protocol defines a set of states for memory, a set of states for cache lines, and a set of actions that transition between these states when processors access memory.

Three sets of these attributes are defined for minimal, typical, and full applications:

* '''Minimal''' - for ‘trivial but correct’ applications that require the presence of the memory in only one cache line. It does not enable read-sharing, and is appropriate for small multi-processors or applications that don’t require significant sharing.

* '''Typical''' - enables read-sharing of a memory location and provisions for efficiency, such as DMA transfers, local caching of data, and error recovery. This set adds an additional stable memory state (FRESH) and multiple cache states. This set will be the focus of this article going forward.

* '''Full''' - adds support for pair-wise sharing, QOLB lock bit synchronization, and cleansing and washing of cache lines.

== State Diagrams ==
=== Memory States ===
The memory states define the state of the memory block from the perspective of the home directory. This 2-bit field is maintained in the memory tag by the home directory along with the pointer to the head of the sharing list (forwId). This simple state model includes three stable states and one semi-stable state.

{|border="1" cellspacing="0" cellpadding="2"
!colspan="5"| Table 1: Memory States
|----
!State
!Description
!Minimal
!Typical
!Full
|----
|HOME
|no sharing list
|Y
|Y
|Y
|----
|FRESH
|sharing-list copy is the same as memory
|
|Y
|Y
|----
|GONE
|sharing-list copy may be different from memory
|Y
|Y
|Y
|----
|WASH*
|transitional state (GONE to FRESH)
|
|
|Y
|----
|}

=== Cache States ===
The cache-line states are maintained by each processors cache-coherency controller. This 7-bit field is stored in each cache line in the sharing list, along with the pointer to the next sharing-list node (forwId) and the previous sharing-list node (backID). Seven bits enable up to 128 possible values, and SCI defines twenty-nine stable-states for use in the minimal, typical, and full sets.

Stable states are those cache states that exist when a memory transaction is not in process. Their names are derived from a combination of
the position of the node in the sharing list, such as ONLY, HEAD, MID, and TAIL, and the state of the data for that node, such as FRESH, CLEAN, DIRTY, VALID, STALE, etc.

====States of the Typical Set====

Following are the states defined for the Typical set. (Note that other states may be supported to provide certain interoperability with nodes implementing the Full set, but are not used when all nodes support the Typical set.)

*“ONLY_DIRTY” - only one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and no other processor currently caches the block.
*”ONLY_FRESH” - only one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and no other processor currently caches the block.
*”HEAD_DIRTY” - more than one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and another processors already caches the block.
*”HEAD_FRESH” - more than one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and another processors already caches the block.
*”MID_VALID” - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is neither the Head or the Tail of the of the sharing list.
*”TAIL_VALID” - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is the Tail of the of the sharing list.

Below is a state diagram that depicts these states:

[[Image:cache_states.png|center]]

The request sub-actions of transactions that expect a response from the directory include 17 variations of reads, writes, and locks based on the number of blocks requested and the coherency requirements. For convenience, the notation on the edges of the above graph is derived from the terms used in the SCI specification to describe the memory transactions.

The first value indicates the request to the directory by a node for memory access:
*”Fetch R” - indicates a request for a memory block with read privileges.
*”Fetch RW” - indicates a request for a memory block with read/write privileges.
*”DATA_MODIFY” - indicates that a processor (the HEAD or the ONLY processor caching the block) writes to the cache line.

The value in the parenthesis indicates the memory state at the time of the request.

The request sub-actions of transactions that expect a response from the directory include 17 variations of reads, writes, and locks based on the number of blocks requested and the coherency requirements. For convenience, the notation on the edges of the above graph is derived from the terms used in the SCI specification to describe the memory transactions.

The first value indicates the request to the directory by a node for memory access:
*”Fetch R” - indicates a request for a memory block with read privileges.
*”Fetch RW” - indicates a request for a memory block with read/write privileges.
*”DATA_MODIFY” - indicates that a processor (the HEAD or the ONLY processor caching the block) writes to the cache line.

The value in the parenthesis indicates the memory state at the time of the request.

== Race Conditions ==
In a distributed shared memory system with caching, the emergence of race conditions is extremely likely. This is mainly due to the lack of a bus for serialization of actions. It is further compounded by the problem of network errors and congestion.

The early invalidation case from Section 11.4 in Solihin is an excellent example of a race condition that can arise in a distributed system. Recall the diagram from the text, and the cache coherence actions, as shown below.

[[Image:EarlyInValidationRace.png|400px|center]]
The circled actions are as follows:
# ''A'' sends a read request to ''Home''.
# ''Home'' replies with data (but the message gets delayed).
# ''B'' sends a write request to ''Home''.
# ''Home'' sends invalidation to ''A'', and it arrives before the ReplyD

The SCI protocol has a way to handle this race condition, as well as many others, and the following sections will discuss how the SCI protocol design can prevent this race condition.

=== Prevention in the SCI Protocol ===
Race conditions are almost non-existent in the SCI protocol, due primarily to the protocol's design. A brief discussion of how this is accomplished follows, followed by a discussion showing what happens if this condition arises in the SCI protocol.
==== Atomic Transactions ====
SCI's primary method for preventing race conditions is having atomic transactions. A transaction is defined as a set of sub-actions necessary to complete some requested action, such as querying memory for a data or invalidating a node's cache. This usually takes the form a a request-response pair, where the initiating node sends a request to another node and waits for the response. The transaction is not complete until the response returns. Suppose, for example, that node ''A'' is in the middle of a transaction with node ''B''. Node ''C'' then tries to make a request of node ''A''. Node ''A'' will respond to node ''C'' that it is busy, telling node ''C'' to try again, as shown in the following diagram.
[[Image:AtomicBusy.png|center]]
Thus, in the Early Invalidation race, node ''A'' would be in the middle of a transaction, having requested a block of data from memory and not having received a response, which would prevent node ''B'' from invalidating it. Note, however, that nodes will not stay in the busy state indefinitely. Doing so would lead to potential deadlocks, so all transactions have time outs that will eventually cause the transaction to fail, thus moving the node out of the busy state, making it able to respond to requests again.

==== Head Node ====
Another method for preventing race conditions is to have the ''Head'' node of a sharing list perform many of the coherence actions. As a result, only one node is performing actions such as writes and invalidations of other sharers. Since only one node is performing these actions, the possibility of concurrent actions is decreased. If another node wants to write, for instance, it must become the Head Node of the sharing list for the cache line to which it wants to write, displacing the current ''Head'' node as the node that can write. All state changes in the sharing list occur at the behest of the ''Head'' node, with the exception of nodes deleting themselves from the list.

When simply sharing a read-only copy of a node, as in when the memory is a ''FRESH'' state, the ''Head'' node is somewhat irrelevant. All the sharing nodes have their own cached value of the cache line. If any node wants to write, including the ''Head'' node, it must perform an additional action in order to do so. Likewise, when a sharing list is sharing a line of data in the dirty, or ''GONE'', memory state, as long as the ''Head'' node has not written to the line, then all the sharing nodes stay in the list with their cached line. However, at the point the ''Head'' node wants to write, it can write its data immediately, but it then must invalidate all other shared copies, via the forward pointers in the sharing list.

This, however, does not protect against a race condition where the memory is ''FRESH'', the ''Head'' node wants to write, and another node also wants to join the list. The Memory Access mechanism prevents this condition.

==== Memory Access ====
The final way that the SCI protocol minimizes race conditions is by changing the directory structure. In Solihin 11.4, there is a ''Home'' node which keeps track of the cache state and has a certain amount of knowledge about the system as a result. In the SCI protocol, there is still a ''Home'' node of sorts, but this node is only responsible for keeping track of who the current ''Head''. Such a ''Home'' node is usually the node where the memory block physically resides. When a node wants to access a block of memory, it sends a request to the ''Home'' node of that memory block, assuming it isn't already in the sharing list for that block. In the event that the data is already being used, the ''Home'' node will include in its response the address of the ''Head'' node of the sharing list. This requirement necessarily serializes the order of access, as one access request is not serviced at the ''Home'' node until the previous request is finished, and the requests are processed in FIFO order. As well, whichever node is the ''Head'' of the sharing list will be notified by the new ''Head'' when it is demoted. All parties involved in a potential race are thus talking to each other, preventing the race from occurring in the first place.

Because of this requirement, the projected race condition from the previous section, where the ''Head'' node is in a ''FRESH'' state and wants to write while another node also wants to read, cannot occur. This is because both nodes must make their request to the ''Home'' node; the ''Head'' node must request the ability to write, and the other node must request the data from the ''Home'' node. One of these requests will make it to the ''Home'' node first, resulting in the second request being deferred to the first node to reach to ''Home'' node. Such a scenario is pictured below.
[[Image:MemoryAccess.png|center]]

==== Solihin 11.4 Resolution ====
The three previous sections discussed individual parts of how the SCI protocol reduces race conditions. Putting all three of these parts together yields the following scenario which resolves the Early Invalidation race as described in the text.
[[Image:SolihinSCI.png|center]]

# ''A'' sends a request to ''Home'' for access to the memory block. It then goes into a busy state while it waits for a response.
# ''B'' also sends a request to ''Home'' for access to the same memory block. ''A'''s request is received first
# ''Home'' responds to ''A'' with the data, but this response gets caught in network traffic and delayed. This response is sent before ''Home'' processes the request from ''B''.
# ''Home'' responds to the request from ''B'', telling it that ''A'' is the current ''Head'' node.
# ''B'' then sends a request to ''A'' to tell it to demote itself.
# ''A'' still hasn't received the response from ''Home'', so it is still in a transaction, and it tells ''B'' that it is busy. ''B'' will have to retry the request.

It should be noted that ''B'' will be in the ''PENDING''[[#Definitions_and_Terms|[def]]] state, so any requests to demote it from the ''Head'' position will be delayed.. This could lead to a chaining of nodes in the ''PENDING'' state, but as soon as node ''A'' receives its response from ''Home'' or times out, the requests in the chain will be successively accepted and processed.

=== Other Possible Race Conditions ===
The design of the SCI protocol prevents many race conditions from occurring, as has already been shown. Here are two other conditions that may arise, with a discussion of how the protocol resolves the issue.

==== Concurrent List Deletions ====

==== Simultaneous Deletion and Invalidation ====
Even though the ''Head'' node performs most of the coherence actions for a shard list, the individual nodes are able to invalidate themselves, such as when they evict the block from their cache. This is referred to in the SCI standard as a "deletion" from the sharing list. Deletion is accomplished by having the invalidating node "lock" itself and then inform its forward and back nodes that they should now point to each other. This "locking" is essentially another ''PENDING'' state. A problem could arise, then, when two neighboring nodes try to invalidate themselves at the same time, as they would both be locked and not respond to each other's message. In this case, though, the protocol specifies that the node that is closest to the tail takes priority. So, the deadlock and/or race condition is averted by ordering the deletion from the tail towards the head.

== Definitions and Terms ==
* ''Head Node'' - The node at the beginning of the sharing list
* ''Home Node'' - The node responsible for keeping track of the ''Head'' node for a sharing list for a memory block, usually the node at which the memory block resides.
* ''PENDING state'' - The intermediate state for a node while it is attempting to become the ''Head'' node in the sharing list. This includes the transaction involved in querying the ''Home'' node and the transaction involved in demoting the current ''Head'' node.
* ''SCI'' - Scalable Coherent Interface

== References ==
[[#1body|1.]] "IEEE Standard for Scalable Coherent Interface (SCI).," IEEE Std 1596-1992 , vol., no., pp.i, 1993. doi: 10.1109/IEEESTD.1993.120366
URL: http://ieeexplore.ieee.org.proxied.lib.ncsu.edu/stamp/stamp.jsp?tp=&arnumber=347683&isnumber=8049 
[[#2body|2.]] [http://www.scizzl.com/HowSCIcohWorks.html How SCI Coherence Works]

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-19T00:16:37Z

Eapotter: /* Cache States */

__TOC__

== History ==

== Background ==
The Scalable Coherent Interface (SCI) cache coherence protocol defines a set of states for memory, a set of states for cache lines, and a set of actions that transition between these states when processors access memory.

Three sets of these attributes are defined for minimal, typical, and full applications:

* '''Minimal''' - for ‘trivial but correct’ applications that require the presence of the memory in only one cache line. It does not enable read-sharing, and is appropriate for small multi-processors or applications that don’t require significant sharing.

* '''Typical''' - enables read-sharing of a memory location and provisions for efficiency, such as DMA transfers, local caching of data, and error recovery. This set adds an additional stable memory state (FRESH) and multiple cache states. This set will be the focus of this article going forward.

* '''Full''' - adds support for pair-wise sharing, QOLB lock bit synchronization, and cleansing and washing of cache lines.

== State Diagrams ==
=== Memory States ===
The memory states define the state of the memory block from the perspective of the home directory. This 2-bit field is maintained in the memory tag by the home directory along with the pointer to the head of the sharing list (forwId). This simple state model includes three stable states and one semi-stable state.

{|border="1" cellspacing="0" cellpadding="2"
!colspan="5"| Table 1: Memory States
|----
!State
!Description
!Minimal
!Typical
!Full
|----
|HOME
|no sharing list
|Y
|Y
|Y
|----
|FRESH
|sharing-list copy is the same as memory
|
|Y
|Y
|----
|GONE
|sharing-list copy may be different from memory
|Y
|Y
|Y
|----
|WASH*
|transitional state (GONE to FRESH)
|
|
|Y
|----
|}

=== Cache States ===
The cache-line states are maintained by each processors cache-coherency controller. This 7-bit field is stored in each cache line in the sharing list, along with the pointer to the next sharing-list node (forwId) and the previous sharing-list node (backID). Seven bits enable up to 128 possible values, and SCI defines twenty-nine stable-states for use in the minimal, typical, and full sets.

Stable states are those cache states that exist when a memory transaction is not in process. Their names are derived from a combination of
the position of the node in the sharing list, such as ONLY, HEAD, MID, and TAIL, and the state of the data for that node, such as FRESH, CLEAN, DIRTY, VALID, STALE, etc.

====States of the Typical Set====

Following are the states defined for the Typical set. (Note that other states may be supported to provide certain interoperability with nodes implementing the Full set, but are not used when all nodes support the Typical set.)

*“ONLY_DIRTY” - only one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and no other processor currently caches the block.
*”ONLY_FRESH” - only one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and no other processor currently caches the block.
*”HEAD_DIRTY” - more than one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and another processors already caches the block.
*”HEAD_FRESH” - more than one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and another processors already caches the block.
*”MID_VALID” - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is neither the Head or the Tail of the of the sharing list.
*”TAIL_VALID” - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is the Tail of the of the sharing list.

[[Image:cache_states.png|center]]

== Race Conditions ==
In a distributed shared memory system with caching, the emergence of race conditions is extremely likely. This is mainly due to the lack of a bus for serialization of actions. It is further compounded by the problem of network errors and congestion.

The early invalidation case from Section 11.4 in Solihin is an excellent example of a race condition that can arise in a distributed system. Recall the diagram from the text, and the cache coherence actions, as shown below.

[[Image:EarlyInValidationRace.png|400px|center]]
The circled actions are as follows:
# ''A'' sends a read request to ''Home''.
# ''Home'' replies with data (but the message gets delayed).
# ''B'' sends a write request to ''Home''.
# ''Home'' sends invalidation to ''A'', and it arrives before the ReplyD

The SCI protocol has a way to handle this race condition, as well as many others, and the following sections will discuss how the SCI protocol design can prevent this race condition.

=== Prevention in the SCI Protocol ===
Race conditions are almost non-existent in the SCI protocol, due primarily to the protocol's design. A brief discussion of how this is accomplished follows, followed by a discussion showing what happens if this condition arises in the SCI protocol.
==== Atomic Transactions ====
SCI's primary method for preventing race conditions is having atomic transactions. A transaction is defined as a set of sub-actions necessary to complete some requested action, such as querying memory for a data or invalidating a node's cache. This usually takes the form a a request-response pair, where the initiating node sends a request to another node and waits for the response. The transaction is not complete until the response returns. Suppose, for example, that node ''A'' is in the middle of a transaction with node ''B''. Node ''C'' then tries to make a request of node ''A''. Node ''A'' will respond to node ''C'' that it is busy, telling node ''C'' to try again, as shown in the following diagram.
[[Image:AtomicBusy.png|center]]
Thus, in the Early Invalidation race, node ''A'' would be in the middle of a transaction, having requested a block of data from memory and not having received a response, which would prevent node ''B'' from invalidating it. Note, however, that nodes will not stay in the busy state indefinitely. Doing so would lead to potential deadlocks, so all transactions have time outs that will eventually cause the transaction to fail, thus moving the node out of the busy state, making it able to respond to requests again.

==== Head Node ====
Another method for preventing race conditions is to have the ''Head'' node of a sharing list perform many of the coherence actions. As a result, only one node is performing actions such as writes and invalidations of other sharers. Since only one node is performing these actions, the possibility of concurrent actions is decreased. If another node wants to write, for instance, it must become the Head Node of the sharing list for the cache line to which it wants to write, displacing the current ''Head'' node as the node that can write. All state changes in the sharing list occur at the behest of the ''Head'' node, with the exception of nodes deleting themselves from the list.

When simply sharing a read-only copy of a node, as in when the memory is a ''FRESH'' state, the ''Head'' node is somewhat irrelevant. All the sharing nodes have their own cached value of the cache line. If any node wants to write, including the ''Head'' node, it must perform an additional action in order to do so. Likewise, when a sharing list is sharing a line of data in the dirty, or ''GONE'', memory state, as long as the ''Head'' node has not written to the line, then all the sharing nodes stay in the list with their cached line. However, at the point the ''Head'' node wants to write, it can write its data immediately, but it then must invalidate all other shared copies, via the forward pointers in the sharing list.

This, however, does not protect against a race condition where the memory is ''FRESH'', the ''Head'' node wants to write, and another node also wants to join the list. The Memory Access mechanism prevents this condition.

==== Memory Access ====
The final way that the SCI protocol minimizes race conditions is by changing the directory structure. In Solihin 11.4, there is a ''Home'' node which keeps track of the cache state and has a certain amount of knowledge about the system as a result. In the SCI protocol, there is still a ''Home'' node of sorts, but this node is only responsible for keeping track of who the current ''Head''. Such a ''Home'' node is usually the node where the memory block physically resides. When a node wants to access a block of memory, it sends a request to the ''Home'' node of that memory block, assuming it isn't already in the sharing list for that block. In the event that the data is already being used, the ''Home'' node will include in its response the address of the ''Head'' node of the sharing list. This requirement necessarily serializes the order of access, as one access request is not serviced at the ''Home'' node until the previous request is finished, and the requests are processed in FIFO order. As well, whichever node is the ''Head'' of the sharing list will be notified by the new ''Head'' when it is demoted. All parties involved in a potential race are thus talking to each other, preventing the race from occurring in the first place.

Because of this requirement, the projected race condition from the previous section, where the ''Head'' node is in a ''FRESH'' state and wants to write while another node also wants to read, cannot occur. This is because both nodes must make their request to the ''Home'' node; the ''Head'' node must request the ability to write, and the other node must request the data from the ''Home'' node. One of these requests will make it to the ''Home'' node first, resulting in the second request being deferred to the first node to reach to ''Home'' node. Such a scenario is pictured below.
[[Image:MemoryAccess.png|center]]

==== Solihin 11.4 Resolution ====
The three previous sections discussed individual parts of how the SCI protocol reduces race conditions. Putting all three of these parts together yields the following scenario which resolves the Early Invalidation race as described in the text.
[[Image:SolihinSCI.png|center]]

# ''A'' sends a request to ''Home'' for access to the memory block. It then goes into a busy state while it waits for a response.
# ''B'' also sends a request to ''Home'' for access to the same memory block. ''A'''s request is received first
# ''Home'' responds to ''A'' with the data, but this response gets caught in network traffic and delayed. This response is sent before ''Home'' processes the request from ''B''.
# ''Home'' responds to the request from ''B'', telling it that ''A'' is the current ''Head'' node.
# ''B'' then sends a request to ''A'' to tell it to demote itself.
# ''A'' still hasn't received the response from ''Home'', so it is still in a transaction, and it tells ''B'' that it is busy. ''B'' will have to retry the request.

It should be noted that ''B'' will be in the ''PENDING''[[#Definitions_and_Terms|[def]]] state, so any requests to demote it from the ''Head'' position will be delayed.. This could lead to a chaining of nodes in the ''PENDING'' state, but as soon as node ''A'' receives its response from ''Home'' or times out, the requests in the chain will be successively accepted and processed.

=== Other Possible Race Conditions ===
The design of the SCI protocol prevents many race conditions from occurring, as has already been shown. Here are two other conditions that may arise, with a discussion of how the protocol resolves the issue.

==== Concurrent List Deletions ====

==== Simultaneous Deletion and Invalidation ====
Even though the ''Head'' node performs most of the coherence actions for a shard list, the individual nodes are able to invalidate themselves, such as when they evict the block from their cache. This is referred to in the SCI standard as a "deletion" from the sharing list. Deletion is accomplished by having the invalidating node "lock" itself and then inform its forward and back nodes that they should now point to each other. This "locking" is essentially another ''PENDING'' state. A problem could arise, then, when two neighboring nodes try to invalidate themselves at the same time, as they would both be locked and not respond to each other's message. In this case, though, the protocol specifies that the node that is closest to the tail takes priority. So, the deadlock and/or race condition is averted by ordering the deletion from the tail towards the head.

== Definitions and Terms ==
* ''Head Node'' - The node at the beginning of the sharing list
* ''Home Node'' - The node responsible for keeping track of the ''Head'' node for a sharing list for a memory block, usually the node at which the memory block resides.
* ''PENDING state'' - The intermediate state for a node while it is attempting to become the ''Head'' node in the sharing list. This includes the transaction involved in querying the ''Home'' node and the transaction involved in demoting the current ''Head'' node.
* ''SCI'' - Scalable Coherent Interface

== References ==
[[#1body|1.]] "IEEE Standard for Scalable Coherent Interface (SCI).," IEEE Std 1596-1992 , vol., no., pp.i, 1993. doi: 10.1109/IEEESTD.1993.120366
URL: http://ieeexplore.ieee.org.proxied.lib.ncsu.edu/stamp/stamp.jsp?tp=&arnumber=347683&isnumber=8049 
[[#2body|2.]] [http://www.scizzl.com/HowSCIcohWorks.html How SCI Coherence Works]

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-19T00:16:05Z

Eapotter: /* Cache States */

__TOC__

== History ==

== Background ==
The Scalable Coherent Interface (SCI) cache coherence protocol defines a set of states for memory, a set of states for cache lines, and a set of actions that transition between these states when processors access memory.

Three sets of these attributes are defined for minimal, typical, and full applications:

* '''Minimal''' - for ‘trivial but correct’ applications that require the presence of the memory in only one cache line. It does not enable read-sharing, and is appropriate for small multi-processors or applications that don’t require significant sharing.

* '''Typical''' - enables read-sharing of a memory location and provisions for efficiency, such as DMA transfers, local caching of data, and error recovery. This set adds an additional stable memory state (FRESH) and multiple cache states. This set will be the focus of this article going forward.

* '''Full''' - adds support for pair-wise sharing, QOLB lock bit synchronization, and cleansing and washing of cache lines.

== State Diagrams ==
=== Memory States ===
The memory states define the state of the memory block from the perspective of the home directory. This 2-bit field is maintained in the memory tag by the home directory along with the pointer to the head of the sharing list (forwId). This simple state model includes three stable states and one semi-stable state.

{|border="1" cellspacing="0" cellpadding="2"
!colspan="5"| Table 1: Memory States
|----
!State
!Description
!Minimal
!Typical
!Full
|----
|HOME
|no sharing list
|Y
|Y
|Y
|----
|FRESH
|sharing-list copy is the same as memory
|
|Y
|Y
|----
|GONE
|sharing-list copy may be different from memory
|Y
|Y
|Y
|----
|WASH*
|transitional state (GONE to FRESH)
|
|
|Y
|----
|}

=== Cache States ===
The cache-line states are maintained by each processors cache-coherency controller. This 7-bit field is stored in each cache line in the sharing list, along with the pointer to the next sharing-list node (forwId) and the previous sharing-list node (backID). Seven bits enable up to 128 possible values, and SCI defines twenty-nine stable-states for use in the minimal, typical, and full sets.

Stable states are those cache states that exist when a memory transaction is not in process. Their names are derived from a combination of
the position of the node in the sharing list, such as ONLY, HEAD, MID, and TAIL, and the state of the data for that node, such as FRESH, CLEAN, DIRTY, VALID, STALE, etc.

States of the Typical Set

Following are the states defined for the Typical set. (Note that other states may be supported to provide certain interoperability with nodes implementing the Full set, but are not used when all nodes support the Typical set.)

*“ONLY_DIRTY” - only one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and no other processor currently caches the block.
*”ONLY_FRESH” - only one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and no other processor currently caches the block.
*”HEAD_DIRTY” - more than one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and another processors already caches the block.
*”HEAD_FRESH” - more than one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and another processors already caches the block.
*”MID_VALID” - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is neither the Head or the Tail of the of the sharing list.
*”TAIL_VALID” - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is the Tail of the of the sharing list.

[[Image:cache_states.png|center]]

== Race Conditions ==
In a distributed shared memory system with caching, the emergence of race conditions is extremely likely. This is mainly due to the lack of a bus for serialization of actions. It is further compounded by the problem of network errors and congestion.

The early invalidation case from Section 11.4 in Solihin is an excellent example of a race condition that can arise in a distributed system. Recall the diagram from the text, and the cache coherence actions, as shown below.

[[Image:EarlyInValidationRace.png|400px|center]]
The circled actions are as follows:
# ''A'' sends a read request to ''Home''.
# ''Home'' replies with data (but the message gets delayed).
# ''B'' sends a write request to ''Home''.
# ''Home'' sends invalidation to ''A'', and it arrives before the ReplyD

The SCI protocol has a way to handle this race condition, as well as many others, and the following sections will discuss how the SCI protocol design can prevent this race condition.

=== Prevention in the SCI Protocol ===
Race conditions are almost non-existent in the SCI protocol, due primarily to the protocol's design. A brief discussion of how this is accomplished follows, followed by a discussion showing what happens if this condition arises in the SCI protocol.
==== Atomic Transactions ====
SCI's primary method for preventing race conditions is having atomic transactions. A transaction is defined as a set of sub-actions necessary to complete some requested action, such as querying memory for a data or invalidating a node's cache. This usually takes the form a a request-response pair, where the initiating node sends a request to another node and waits for the response. The transaction is not complete until the response returns. Suppose, for example, that node ''A'' is in the middle of a transaction with node ''B''. Node ''C'' then tries to make a request of node ''A''. Node ''A'' will respond to node ''C'' that it is busy, telling node ''C'' to try again, as shown in the following diagram.
[[Image:AtomicBusy.png|center]]
Thus, in the Early Invalidation race, node ''A'' would be in the middle of a transaction, having requested a block of data from memory and not having received a response, which would prevent node ''B'' from invalidating it. Note, however, that nodes will not stay in the busy state indefinitely. Doing so would lead to potential deadlocks, so all transactions have time outs that will eventually cause the transaction to fail, thus moving the node out of the busy state, making it able to respond to requests again.

==== Head Node ====
Another method for preventing race conditions is to have the ''Head'' node of a sharing list perform many of the coherence actions. As a result, only one node is performing actions such as writes and invalidations of other sharers. Since only one node is performing these actions, the possibility of concurrent actions is decreased. If another node wants to write, for instance, it must become the Head Node of the sharing list for the cache line to which it wants to write, displacing the current ''Head'' node as the node that can write. All state changes in the sharing list occur at the behest of the ''Head'' node, with the exception of nodes deleting themselves from the list.

When simply sharing a read-only copy of a node, as in when the memory is a ''FRESH'' state, the ''Head'' node is somewhat irrelevant. All the sharing nodes have their own cached value of the cache line. If any node wants to write, including the ''Head'' node, it must perform an additional action in order to do so. Likewise, when a sharing list is sharing a line of data in the dirty, or ''GONE'', memory state, as long as the ''Head'' node has not written to the line, then all the sharing nodes stay in the list with their cached line. However, at the point the ''Head'' node wants to write, it can write its data immediately, but it then must invalidate all other shared copies, via the forward pointers in the sharing list.

This, however, does not protect against a race condition where the memory is ''FRESH'', the ''Head'' node wants to write, and another node also wants to join the list. The Memory Access mechanism prevents this condition.

==== Memory Access ====
The final way that the SCI protocol minimizes race conditions is by changing the directory structure. In Solihin 11.4, there is a ''Home'' node which keeps track of the cache state and has a certain amount of knowledge about the system as a result. In the SCI protocol, there is still a ''Home'' node of sorts, but this node is only responsible for keeping track of who the current ''Head''. Such a ''Home'' node is usually the node where the memory block physically resides. When a node wants to access a block of memory, it sends a request to the ''Home'' node of that memory block, assuming it isn't already in the sharing list for that block. In the event that the data is already being used, the ''Home'' node will include in its response the address of the ''Head'' node of the sharing list. This requirement necessarily serializes the order of access, as one access request is not serviced at the ''Home'' node until the previous request is finished, and the requests are processed in FIFO order. As well, whichever node is the ''Head'' of the sharing list will be notified by the new ''Head'' when it is demoted. All parties involved in a potential race are thus talking to each other, preventing the race from occurring in the first place.

Because of this requirement, the projected race condition from the previous section, where the ''Head'' node is in a ''FRESH'' state and wants to write while another node also wants to read, cannot occur. This is because both nodes must make their request to the ''Home'' node; the ''Head'' node must request the ability to write, and the other node must request the data from the ''Home'' node. One of these requests will make it to the ''Home'' node first, resulting in the second request being deferred to the first node to reach to ''Home'' node. Such a scenario is pictured below.
[[Image:MemoryAccess.png|center]]

==== Solihin 11.4 Resolution ====
The three previous sections discussed individual parts of how the SCI protocol reduces race conditions. Putting all three of these parts together yields the following scenario which resolves the Early Invalidation race as described in the text.
[[Image:SolihinSCI.png|center]]

# ''A'' sends a request to ''Home'' for access to the memory block. It then goes into a busy state while it waits for a response.
# ''B'' also sends a request to ''Home'' for access to the same memory block. ''A'''s request is received first
# ''Home'' responds to ''A'' with the data, but this response gets caught in network traffic and delayed. This response is sent before ''Home'' processes the request from ''B''.
# ''Home'' responds to the request from ''B'', telling it that ''A'' is the current ''Head'' node.
# ''B'' then sends a request to ''A'' to tell it to demote itself.
# ''A'' still hasn't received the response from ''Home'', so it is still in a transaction, and it tells ''B'' that it is busy. ''B'' will have to retry the request.

It should be noted that ''B'' will be in the ''PENDING''[[#Definitions_and_Terms|[def]]] state, so any requests to demote it from the ''Head'' position will be delayed.. This could lead to a chaining of nodes in the ''PENDING'' state, but as soon as node ''A'' receives its response from ''Home'' or times out, the requests in the chain will be successively accepted and processed.

=== Other Possible Race Conditions ===
The design of the SCI protocol prevents many race conditions from occurring, as has already been shown. Here are two other conditions that may arise, with a discussion of how the protocol resolves the issue.

==== Concurrent List Deletions ====

==== Simultaneous Deletion and Invalidation ====
Even though the ''Head'' node performs most of the coherence actions for a shard list, the individual nodes are able to invalidate themselves, such as when they evict the block from their cache. This is referred to in the SCI standard as a "deletion" from the sharing list. Deletion is accomplished by having the invalidating node "lock" itself and then inform its forward and back nodes that they should now point to each other. This "locking" is essentially another ''PENDING'' state. A problem could arise, then, when two neighboring nodes try to invalidate themselves at the same time, as they would both be locked and not respond to each other's message. In this case, though, the protocol specifies that the node that is closest to the tail takes priority. So, the deadlock and/or race condition is averted by ordering the deletion from the tail towards the head.

== Definitions and Terms ==
* ''Head Node'' - The node at the beginning of the sharing list
* ''Home Node'' - The node responsible for keeping track of the ''Head'' node for a sharing list for a memory block, usually the node at which the memory block resides.
* ''PENDING state'' - The intermediate state for a node while it is attempting to become the ''Head'' node in the sharing list. This includes the transaction involved in querying the ''Home'' node and the transaction involved in demoting the current ''Head'' node.
* ''SCI'' - Scalable Coherent Interface

== References ==
[[#1body|1.]] "IEEE Standard for Scalable Coherent Interface (SCI).," IEEE Std 1596-1992 , vol., no., pp.i, 1993. doi: 10.1109/IEEESTD.1993.120366
URL: http://ieeexplore.ieee.org.proxied.lib.ncsu.edu/stamp/stamp.jsp?tp=&arnumber=347683&isnumber=8049 
[[#2body|2.]] [http://www.scizzl.com/HowSCIcohWorks.html How SCI Coherence Works]

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-19T00:15:24Z

Eapotter: /* Cache States */

__TOC__

== History ==

== Background ==
The Scalable Coherent Interface (SCI) cache coherence protocol defines a set of states for memory, a set of states for cache lines, and a set of actions that transition between these states when processors access memory.

Three sets of these attributes are defined for minimal, typical, and full applications:

* '''Minimal''' - for ‘trivial but correct’ applications that require the presence of the memory in only one cache line. It does not enable read-sharing, and is appropriate for small multi-processors or applications that don’t require significant sharing.

* '''Typical''' - enables read-sharing of a memory location and provisions for efficiency, such as DMA transfers, local caching of data, and error recovery. This set adds an additional stable memory state (FRESH) and multiple cache states. This set will be the focus of this article going forward.

* '''Full''' - adds support for pair-wise sharing, QOLB lock bit synchronization, and cleansing and washing of cache lines.

== State Diagrams ==
=== Memory States ===
The memory states define the state of the memory block from the perspective of the home directory. This 2-bit field is maintained in the memory tag by the home directory along with the pointer to the head of the sharing list (forwId). This simple state model includes three stable states and one semi-stable state.

{|border="1" cellspacing="0" cellpadding="2"
!colspan="5"| Table 1: Memory States
|----
!State
!Description
!Minimal
!Typical
!Full
|----
|HOME
|no sharing list
|Y
|Y
|Y
|----
|FRESH
|sharing-list copy is the same as memory
|
|Y
|Y
|----
|GONE
|sharing-list copy may be different from memory
|Y
|Y
|Y
|----
|WASH*
|transitional state (GONE to FRESH)
|
|
|Y
|----
|}

=== Cache States ===
The cache-line states are maintained by each processors cache-coherency controller. This 7-bit field is stored in each cache line in the sharing list, along with the pointer to the next sharing-list node (forwId) and the previous sharing-list node (backID). Seven bits enable up to 128 possible values, and SCI defines twenty-nine stable-states for use in the minimal, typical, and full sets.

Stable states are those cache states that exist when a memory transaction is not in process. Their names are derived from a combination of
the position of the node in the sharing list, such as ONLY, HEAD, MID, and TAIL, and the state of the data for that node, such as FRESH, CLEAN, DIRTY, VALID, STALE, etc.

States of the Typical Set
Following are the states defined for the Typical set. (Note that other states may be supported to provide certain interoperability with nodes implementing the Full set, but are not used when all nodes support the Typical set.)

*“ONLY_DIRTY” - only one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and no other processor currently caches the block.
*”ONLY_FRESH” - only one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and no other processor currently caches the block.
*”HEAD_DIRTY” - more than one processor has the memory block in its cache. This block is writable, and the processor has written (or intends) to write to it. This state is set when the processor requests the block with read/write privileges, and another processors already caches the block.
*”HEAD_FRESH” - more than one processor had the memory block in its cache. This block is writeable, but processor has not written to it. This state is set when the processor requests the block with read privileges, and another processors already caches the block.
*”MID_VALID” - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is neither the Head or the Tail of the of the sharing list.
*”TAIL_VALID” - more than two processors have the memory in its cache, and it is readable. The processor cache with this state is the Tail of the of the sharing list.

[[Image:cache_states.png|center]]

== Race Conditions ==
In a distributed shared memory system with caching, the emergence of race conditions is extremely likely. This is mainly due to the lack of a bus for serialization of actions. It is further compounded by the problem of network errors and congestion.

The early invalidation case from Section 11.4 in Solihin is an excellent example of a race condition that can arise in a distributed system. Recall the diagram from the text, and the cache coherence actions, as shown below.

[[Image:EarlyInValidationRace.png|400px|center]]
The circled actions are as follows:
# ''A'' sends a read request to ''Home''.
# ''Home'' replies with data (but the message gets delayed).
# ''B'' sends a write request to ''Home''.
# ''Home'' sends invalidation to ''A'', and it arrives before the ReplyD

The SCI protocol has a way to handle this race condition, as well as many others, and the following sections will discuss how the SCI protocol design can prevent this race condition.

=== Prevention in the SCI Protocol ===
Race conditions are almost non-existent in the SCI protocol, due primarily to the protocol's design. A brief discussion of how this is accomplished follows, followed by a discussion showing what happens if this condition arises in the SCI protocol.
==== Atomic Transactions ====
SCI's primary method for preventing race conditions is having atomic transactions. A transaction is defined as a set of sub-actions necessary to complete some requested action, such as querying memory for a data or invalidating a node's cache. This usually takes the form a a request-response pair, where the initiating node sends a request to another node and waits for the response. The transaction is not complete until the response returns. Suppose, for example, that node ''A'' is in the middle of a transaction with node ''B''. Node ''C'' then tries to make a request of node ''A''. Node ''A'' will respond to node ''C'' that it is busy, telling node ''C'' to try again, as shown in the following diagram.
[[Image:AtomicBusy.png|center]]
Thus, in the Early Invalidation race, node ''A'' would be in the middle of a transaction, having requested a block of data from memory and not having received a response, which would prevent node ''B'' from invalidating it. Note, however, that nodes will not stay in the busy state indefinitely. Doing so would lead to potential deadlocks, so all transactions have time outs that will eventually cause the transaction to fail, thus moving the node out of the busy state, making it able to respond to requests again.

==== Head Node ====
Another method for preventing race conditions is to have the ''Head'' node of a sharing list perform many of the coherence actions. As a result, only one node is performing actions such as writes and invalidations of other sharers. Since only one node is performing these actions, the possibility of concurrent actions is decreased. If another node wants to write, for instance, it must become the Head Node of the sharing list for the cache line to which it wants to write, displacing the current ''Head'' node as the node that can write. All state changes in the sharing list occur at the behest of the ''Head'' node, with the exception of nodes deleting themselves from the list.

When simply sharing a read-only copy of a node, as in when the memory is a ''FRESH'' state, the ''Head'' node is somewhat irrelevant. All the sharing nodes have their own cached value of the cache line. If any node wants to write, including the ''Head'' node, it must perform an additional action in order to do so. Likewise, when a sharing list is sharing a line of data in the dirty, or ''GONE'', memory state, as long as the ''Head'' node has not written to the line, then all the sharing nodes stay in the list with their cached line. However, at the point the ''Head'' node wants to write, it can write its data immediately, but it then must invalidate all other shared copies, via the forward pointers in the sharing list.

This, however, does not protect against a race condition where the memory is ''FRESH'', the ''Head'' node wants to write, and another node also wants to join the list. The Memory Access mechanism prevents this condition.

==== Memory Access ====
The final way that the SCI protocol minimizes race conditions is by changing the directory structure. In Solihin 11.4, there is a ''Home'' node which keeps track of the cache state and has a certain amount of knowledge about the system as a result. In the SCI protocol, there is still a ''Home'' node of sorts, but this node is only responsible for keeping track of who the current ''Head''. Such a ''Home'' node is usually the node where the memory block physically resides. When a node wants to access a block of memory, it sends a request to the ''Home'' node of that memory block, assuming it isn't already in the sharing list for that block. In the event that the data is already being used, the ''Home'' node will include in its response the address of the ''Head'' node of the sharing list. This requirement necessarily serializes the order of access, as one access request is not serviced at the ''Home'' node until the previous request is finished, and the requests are processed in FIFO order. As well, whichever node is the ''Head'' of the sharing list will be notified by the new ''Head'' when it is demoted. All parties involved in a potential race are thus talking to each other, preventing the race from occurring in the first place.

Because of this requirement, the projected race condition from the previous section, where the ''Head'' node is in a ''FRESH'' state and wants to write while another node also wants to read, cannot occur. This is because both nodes must make their request to the ''Home'' node; the ''Head'' node must request the ability to write, and the other node must request the data from the ''Home'' node. One of these requests will make it to the ''Home'' node first, resulting in the second request being deferred to the first node to reach to ''Home'' node. Such a scenario is pictured below.
[[Image:MemoryAccess.png|center]]

==== Solihin 11.4 Resolution ====
The three previous sections discussed individual parts of how the SCI protocol reduces race conditions. Putting all three of these parts together yields the following scenario which resolves the Early Invalidation race as described in the text.
[[Image:SolihinSCI.png|center]]

# ''A'' sends a request to ''Home'' for access to the memory block. It then goes into a busy state while it waits for a response.
# ''B'' also sends a request to ''Home'' for access to the same memory block. ''A'''s request is received first
# ''Home'' responds to ''A'' with the data, but this response gets caught in network traffic and delayed. This response is sent before ''Home'' processes the request from ''B''.
# ''Home'' responds to the request from ''B'', telling it that ''A'' is the current ''Head'' node.
# ''B'' then sends a request to ''A'' to tell it to demote itself.
# ''A'' still hasn't received the response from ''Home'', so it is still in a transaction, and it tells ''B'' that it is busy. ''B'' will have to retry the request.

It should be noted that ''B'' will be in the ''PENDING''[[#Definitions_and_Terms|[def]]] state, so any requests to demote it from the ''Head'' position will be delayed.. This could lead to a chaining of nodes in the ''PENDING'' state, but as soon as node ''A'' receives its response from ''Home'' or times out, the requests in the chain will be successively accepted and processed.

=== Other Possible Race Conditions ===
The design of the SCI protocol prevents many race conditions from occurring, as has already been shown. Here are two other conditions that may arise, with a discussion of how the protocol resolves the issue.

==== Concurrent List Deletions ====

==== Simultaneous Deletion and Invalidation ====
Even though the ''Head'' node performs most of the coherence actions for a shard list, the individual nodes are able to invalidate themselves, such as when they evict the block from their cache. This is referred to in the SCI standard as a "deletion" from the sharing list. Deletion is accomplished by having the invalidating node "lock" itself and then inform its forward and back nodes that they should now point to each other. This "locking" is essentially another ''PENDING'' state. A problem could arise, then, when two neighboring nodes try to invalidate themselves at the same time, as they would both be locked and not respond to each other's message. In this case, though, the protocol specifies that the node that is closest to the tail takes priority. So, the deadlock and/or race condition is averted by ordering the deletion from the tail towards the head.

== Definitions and Terms ==
* ''Head Node'' - The node at the beginning of the sharing list
* ''Home Node'' - The node responsible for keeping track of the ''Head'' node for a sharing list for a memory block, usually the node at which the memory block resides.
* ''PENDING state'' - The intermediate state for a node while it is attempting to become the ''Head'' node in the sharing list. This includes the transaction involved in querying the ''Home'' node and the transaction involved in demoting the current ''Head'' node.
* ''SCI'' - Scalable Coherent Interface

== References ==
[[#1body|1.]] "IEEE Standard for Scalable Coherent Interface (SCI).," IEEE Std 1596-1992 , vol., no., pp.i, 1993. doi: 10.1109/IEEESTD.1993.120366
URL: http://ieeexplore.ieee.org.proxied.lib.ncsu.edu/stamp/stamp.jsp?tp=&arnumber=347683&isnumber=8049 
[[#2body|2.]] [http://www.scizzl.com/HowSCIcohWorks.html How SCI Coherence Works]

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-18T22:32:01Z

Eapotter: /* Cache States */

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-18T15:16:01Z

Eapotter: /* Memory States */

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-18T04:07:44Z

Eapotter: /* Memory States */

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-18T04:02:18Z

Eapotter: /* Memory States */

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-18T03:55:18Z

Eapotter: /* Memory States */

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-18T03:53:08Z

Eapotter: /* Memory States */

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-18T03:44:53Z

Eapotter: /* Background */

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-18T03:42:55Z

Eapotter: /* References */

__TOC__

== History ==

== Background ==

== State Diagrams ==
=== Memory States ===
The memory states define the state of the memory block from the perspective of the home directory. This 2-bit field is maintained in the memory tag by the home directory along with the pointer to the head of the sharing list (forwId). This simple state model includes three stable states and one semi-stable state.

=== Cache States ===
The cache-line states are maintained by each processors cache-coherency controller. This 7-bit field is stored in each cache line in the sharing list, along with the pointer to the next sharing-list node (forwId) and the previous sharing-list node (backID). Seven bits enable up to 128 possible values, and SCI defines twenty-nine stable-states for use in the minimal, typical, and full sets.

Stable states are those cache states that exist when a memory transaction is not in process. Their names are derived from a combination of
the position of the node in the sharing list, such as ONLY, HEAD, MID, and TAIL
the state of the data for that node, such as FRESH, CLEAN, DIRTY, VALID, STALE, etc.

[[Image:cache_states.png|center]]

== Race Conditions ==
In a distributed shared memory system with caching, the emergence of race conditions is extremely likely. This is mainly due to the lack of a bus for serialization of actions. It is further compounded by the problem of network errors and congestion.

The early invalidation case from Section 11.4 in Solihin is an excellent example of a race condition that can arise in a distributed system. Recall the diagram from the text, and the cache coherence actions, as shown below.

[[Image:EarlyInValidationRace.png|400px|center]]
The circled actions are as follows:
# ''A'' sends a read request to ''Home''.
# ''Home'' replies with data (but the message gets delayed).
# ''B'' sends a write request to ''Home''.
# ''Home'' sends invalidation to ''A'', and it arrives before the ReplyD

The SCI protocol has a way to handle this race condition, as well as many others, and the following sections will discuss how the SCI protocol design can prevent this race condition.

=== Prevention in the SCI Protocol ===
Race conditions are almost non-existent in the SCI protocol, due primarily to the protocol's design. A brief discussion of how this is accomplished follows, followed by a discussion showing what happens if this condition arises in the SCI protocol.
==== Atomic Transactions ====
SCI's primary method for preventing race conditions is having atomic transactions. A transaction is defined as a set of sub-actions necessary to complete some requested action, such as reading from memory or writing to a variable. Suppose, for example, that node ''A'' is in the middle of a transaction with node ''B''. Node ''C'' then tries to make a request of node ''A''. Node ''A'' will respond to node ''C'' that it is busy, telling node ''C'' to try again, as shown in the following diagram.
[[Image:AtomicBusy.png|center]]
Thus, in the Early Invalidation race, node ''A'' would be in the middle of a transaction, which would prevent node ''B'' from invalidating it.

==== Head Node ====
Another method for preventing race conditions is to have the Head Node of a sharing list perform many of the coherence actions. As a result, only one node is performing actions such as writes and invalidations of other sharers. Since only one node is performing these actions, the possibility of concurrent actions is decreased. If another node wants to write, for instance, it must become the Head Node of the sharing list for the cache line to which it wants to write, displacing the current Head Node as the node that can write.

When simply sharing a read-only copy of a node, as in when the memory is a ''FRESH'' state, the Head Node is somewhat irrelevant. All the sharing nodes have their own cached value of the cache line. If any node wants to write, including the Head Node, it must perform an additional action in order to do so. Likewise, when a sharing list is sharing a line of data in the dirty, or ''GONE'', memory state, as long as the Head Node has not written to the line, then all the sharing nodes stay in the list with their cached line. However, at the point the Head Node wants to write, it can write its data immediately, but it then must invalidate all other shared copies, via the forward pointers in the sharing list.

This, however, does not protect against a race condition where the memory is ''FRESH'', the Head Node wants to write, and another node also wants to join the list. The Memory Access mechanism prevents this condition.

==== Memory Access ====
The final way that the SCI protocol minimizes race conditions is by changing the directory structure. In Solihin 11.4, there is a ''Home'' node which keeps track of the cache state and has a certain amount of knowledge about the system as a result. In the SCI protocol, there is still a ''Home'' node of sorts, but this node is only responsible for keeping track of who the current ''Head'' node is. Such a ''Home'' node is usually the node where the memory block physically resides. When a node wants to access a block of memory, it sends a request to the ''Home'' node of that memory block, assuming it isn't already in the sharing list for that block. This requirement necessarily serializes the order of access, as one access request is not serviced at the ''Home'' node until the previous request is finished, and the requests are processed in FIFO order.

=== Possible Race Conditions ===
==== Communication Delays ====

==== Concurrent List Deletions ====

==== Simultaneous Deletion and Invalidation ====

== Summary ==

== Definitions and Terms ==

== References ==
[1] "IEEE Standard for Scalable Coherent Interface (SCI).," IEEE Std 1596-1992 , vol., no., pp.i, 1993. doi: 10.1109/IEEESTD.1993.120366
URL: http://ieeexplore.ieee.org.proxied.lib.ncsu.edu/stamp/stamp.jsp?tp=&arnumber=347683&isnumber=8049

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-18T03:40:41Z

Eapotter: /* Cache States */

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-18T03:40:07Z

Eapotter: /* Memory States */

__TOC__

== History ==

== Background ==

== State Diagrams ==
=== Memory States ===
The memory states define the state of the memory block from the perspective of the home directory. This 2-bit field is maintained in the memory tag by the home directory along with the pointer to the head of the sharing list (forwId). This simple state model includes three stable states and one semi-stable state.

=== Cache States ===
[[Image:cache_states.png|center]]

== Race Conditions ==
In a distributed shared memory system with caching, the emergence of race conditions is extremely likely. This is mainly due to the lack of a bus for serialization of actions. It is further compounded by the problem of network errors and congestion.

The early invalidation case from Section 11.4 in Solihin is an excellent example of a race condition that can arise in a distributed system. Recall the diagram from the text, and the cache coherence actions, as shown below.

[[Image:EarlyInValidationRace.png|400px|center]]
The circled actions are as follows:
# ''A'' sends a read request to ''Home''.
# ''Home'' replies with data (but the message gets delayed).
# ''B'' sends a write request to ''Home''.
# ''Home'' sends invalidation to ''A'', and it arrives before the ReplyD

The SCI protocol has a way to handle this race condition, as well as many others, and the following sections will discuss how the SCI protocol design can prevent this race condition.

=== Prevention in the SCI Protocol ===
Race conditions are almost non-existent in the SCI protocol, due primarily to the protocol's design. A brief discussion of how this is accomplished follows, followed by a discussion showing what happens if this condition arises in the SCI protocol.
==== Atomic Transactions ====
SCI's primary method for preventing race conditions is having atomic transactions. A transaction is defined as a set of sub-actions necessary to complete some requested action, such as reading from memory or writing to a variable. Suppose, for example, that node ''A'' is in the middle of a transaction with node ''B''. Node ''C'' then tries to make a request of node ''A''. Node ''A'' will respond to node ''C'' that it is busy, telling node ''C'' to try again, as shown in the following diagram.
[[Image:AtomicBusy.png|center]]
Thus, in the Early Invalidation race, node ''A'' would be in the middle of a transaction, which would prevent node ''B'' from invalidating it.

==== Head Node ====
Another method for preventing race conditions is to have the Head Node of a sharing list perform many of the coherence actions. As a result, only one node is performing actions such as writes and invalidations of other sharers. Since only one node is performing these actions, the possibility of concurrent actions is decreased. If another node wants to write, for instance, it must become the Head Node of the sharing list for the cache line to which it wants to write, displacing the current Head Node as the node that can write.

When simply sharing a read-only copy of a node, as in when the memory is a ''FRESH'' state, the Head Node is somewhat irrelevant. All the sharing nodes have their own cached value of the cache line. If any node wants to write, including the Head Node, it must perform an additional action in order to do so. Likewise, when a sharing list is sharing a line of data in the dirty, or ''GONE'', memory state, as long as the Head Node has not written to the line, then all the sharing nodes stay in the list with their cached line. However, at the point the Head Node wants to write, it can write its data immediately, but it then must invalidate all other shared copies, via the forward pointers in the sharing list.

This, however, does not protect against a race condition where the memory is ''FRESH'', the Head Node wants to write, and another node also wants to join the list. The Memory Access mechanism prevents this condition.

==== Memory Access ====
The final way that the SCI protocol minimizes race conditions is by changing the directory structure. In Solihin 11.4, there is a ''Home'' node which keeps track of the cache state and has a certain amount of knowledge about the system as a result. In the SCI protocol, there is still a ''Home'' node of sorts, but this node is only responsible for keeping track of who the current ''Head'' node is. Such a ''Home'' node is usually the node where the memory block physically resides. When a node wants to access a block of memory, it sends a request to the ''Home'' node of that memory block, assuming it isn't already in the sharing list for that block. This requirement necessarily serializes the order of access, as one access request is not serviced at the ''Home'' node until the previous request is finished, and the requests are processed in FIFO order.

=== Possible Race Conditions ===
==== Communication Delays ====

==== Concurrent List Deletions ====

==== Simultaneous Deletion and Invalidation ====

== Summary ==

== Definitions and Terms ==

== References ==

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-18T03:38:33Z

Eapotter: /* Memory */

__TOC__

== History ==

== Background ==

== State Diagrams ==
=== Memory States ===

=== Cache States ===
[[Image:cache_states.png|center]]

== Race Conditions ==
In a distributed shared memory system with caching, the emergence of race conditions is extremely likely. This is mainly due to the lack of a bus for serialization of actions. It is further compounded by the problem of network errors and congestion.

The early invalidation case from Section 11.4 in Solihin is an excellent example of a race condition that can arise in a distributed system. Recall the diagram from the text, and the cache coherence actions, as shown below.

[[Image:EarlyInValidationRace.png|400px|center]]
The circled actions are as follows:
# ''A'' sends a read request to ''Home''.
# ''Home'' replies with data (but the message gets delayed).
# ''B'' sends a write request to ''Home''.
# ''Home'' sends invalidation to ''A'', and it arrives before the ReplyD

The SCI protocol has a way to handle this race condition, as well as many others, and the following sections will discuss how the SCI protocol design can prevent this race condition.

=== Prevention in the SCI Protocol ===
Race conditions are almost non-existent in the SCI protocol, due primarily to the protocol's design. A brief discussion of how this is accomplished follows, followed by a discussion showing what happens if this condition arises in the SCI protocol.
==== Atomic Transactions ====
SCI's primary method for preventing race conditions is having atomic transactions. A transaction is defined as a set of sub-actions necessary to complete some requested action, such as reading from memory or writing to a variable. Suppose, for example, that node ''A'' is in the middle of a transaction with node ''B''. Node ''C'' then tries to make a request of node ''A''. Node ''A'' will respond to node ''C'' that it is busy, telling node ''C'' to try again, as shown in the following diagram.
[[Image:AtomicBusy.png|center]]
Thus, in the Early Invalidation race, node ''A'' would be in the middle of a transaction, which would prevent node ''B'' from invalidating it.

==== Head Node ====
Another method for preventing race conditions is to have the Head Node of a sharing list perform many of the coherence actions. As a result, only one node is performing actions such as writes and invalidations of other sharers. Since only one node is performing these actions, the possibility of concurrent actions is decreased. If another node wants to write, for instance, it must become the Head Node of the sharing list for the cache line to which it wants to write, displacing the current Head Node as the node that can write.

When simply sharing a read-only copy of a node, as in when the memory is a ''FRESH'' state, the Head Node is somewhat irrelevant. All the sharing nodes have their own cached value of the cache line. If any node wants to write, including the Head Node, it must perform an additional action in order to do so. Likewise, when a sharing list is sharing a line of data in the dirty, or ''GONE'', memory state, as long as the Head Node has not written to the line, then all the sharing nodes stay in the list with their cached line. However, at the point the Head Node wants to write, it can write its data immediately, but it then must invalidate all other shared copies, via the forward pointers in the sharing list.

This, however, does not protect against a race condition where the memory is ''FRESH'', the Head Node wants to write, and another node also wants to join the list. The Memory Access mechanism prevents this condition.

==== Memory Access ====

=== Possible Race Conditions ===
==== Communication Delays ====

==== Concurrent List Deletions ====

==== Simultaneous Deletion and Invalidation ====

== Summary ==

== Definitions and Terms ==

== References ==

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-18T03:38:21Z

Eapotter: /* Processors */

__TOC__

== History ==

== Background ==

== State Diagrams ==
=== Memory ===

=== Cache States ===
[[Image:cache_states.png|center]]

== Race Conditions ==
In a distributed shared memory system with caching, the emergence of race conditions is extremely likely. This is mainly due to the lack of a bus for serialization of actions. It is further compounded by the problem of network errors and congestion.

The early invalidation case from Section 11.4 in Solihin is an excellent example of a race condition that can arise in a distributed system. Recall the diagram from the text, and the cache coherence actions, as shown below.

[[Image:EarlyInValidationRace.png|400px|center]]
The circled actions are as follows:
# ''A'' sends a read request to ''Home''.
# ''Home'' replies with data (but the message gets delayed).
# ''B'' sends a write request to ''Home''.
# ''Home'' sends invalidation to ''A'', and it arrives before the ReplyD

The SCI protocol has a way to handle this race condition, as well as many others, and the following sections will discuss how the SCI protocol design can prevent this race condition.

=== Prevention in the SCI Protocol ===
Race conditions are almost non-existent in the SCI protocol, due primarily to the protocol's design. A brief discussion of how this is accomplished follows, followed by a discussion showing what happens if this condition arises in the SCI protocol.
==== Atomic Transactions ====
SCI's primary method for preventing race conditions is having atomic transactions. A transaction is defined as a set of sub-actions necessary to complete some requested action, such as reading from memory or writing to a variable. Suppose, for example, that node ''A'' is in the middle of a transaction with node ''B''. Node ''C'' then tries to make a request of node ''A''. Node ''A'' will respond to node ''C'' that it is busy, telling node ''C'' to try again, as shown in the following diagram.
[[Image:AtomicBusy.png|center]]
Thus, in the Early Invalidation race, node ''A'' would be in the middle of a transaction, which would prevent node ''B'' from invalidating it.

==== Head Node ====
Another method for preventing race conditions is to have the Head Node of a sharing list perform many of the coherence actions. As a result, only one node is performing actions such as writes and invalidations of other sharers. Since only one node is performing these actions, the possibility of concurrent actions is decreased. If another node wants to write, for instance, it must become the Head Node of the sharing list for the cache line to which it wants to write, displacing the current Head Node as the node that can write.

When simply sharing a read-only copy of a node, as in when the memory is a ''FRESH'' state, the Head Node is somewhat irrelevant. All the sharing nodes have their own cached value of the cache line. If any node wants to write, including the Head Node, it must perform an additional action in order to do so. Likewise, when a sharing list is sharing a line of data in the dirty, or ''GONE'', memory state, as long as the Head Node has not written to the line, then all the sharing nodes stay in the list with their cached line. However, at the point the Head Node wants to write, it can write its data immediately, but it then must invalidate all other shared copies, via the forward pointers in the sharing list.

This, however, does not protect against a race condition where the memory is ''FRESH'', the Head Node wants to write, and another node also wants to join the list. The Memory Access mechanism prevents this condition.

==== Memory Access ====

=== Possible Race Conditions ===
==== Communication Delays ====

==== Concurrent List Deletions ====

==== Simultaneous Deletion and Invalidation ====

== Summary ==

== Definitions and Terms ==

== References ==

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-18T03:37:44Z

Eapotter: /* Processors */

__TOC__

== History ==

== Background ==

== State Diagrams ==
=== Memory ===

=== Processors ===
[[Image:cache_states.png|center]]

== Race Conditions ==
In a distributed shared memory system with caching, the emergence of race conditions is extremely likely. This is mainly due to the lack of a bus for serialization of actions. It is further compounded by the problem of network errors and congestion.

The early invalidation case from Section 11.4 in Solihin is an excellent example of a race condition that can arise in a distributed system. Recall the diagram from the text, and the cache coherence actions, as shown below.

[[Image:EarlyInValidationRace.png|400px|center]]
The circled actions are as follows:
# ''A'' sends a read request to ''Home''.
# ''Home'' replies with data (but the message gets delayed).
# ''B'' sends a write request to ''Home''.
# ''Home'' sends invalidation to ''A'', and it arrives before the ReplyD

The SCI protocol has a way to handle this race condition, as well as many others, and the following sections will discuss how the SCI protocol design can prevent this race condition.

=== Prevention in the SCI Protocol ===
Race conditions are almost non-existent in the SCI protocol, due primarily to the protocol's design. A brief discussion of how this is accomplished follows, followed by a discussion showing what happens if this condition arises in the SCI protocol.
==== Atomic Transactions ====
SCI's primary method for preventing race conditions is having atomic transactions. A transaction is defined as a set of sub-actions necessary to complete some requested action, such as reading from memory or writing to a variable. Suppose, for example, that node ''A'' is in the middle of a transaction with node ''B''. Node ''C'' then tries to make a request of node ''A''. Node ''A'' will respond to node ''C'' that it is busy, telling node ''C'' to try again, as shown in the following diagram.
[[Image:AtomicBusy.png|center]]
Thus, in the Early Invalidation race, node ''A'' would be in the middle of a transaction, which would prevent node ''B'' from invalidating it.

==== Head Node ====
Another method for preventing race conditions is to have the Head Node of a sharing list perform many of the coherence actions. As a result, only one node is performing actions such as writes and invalidations of other sharers. Since only one node is performing these actions, the possibility of concurrent actions is decreased. If another node wants to write, for instance, it must become the Head Node of the sharing list for the cache line to which it wants to write, displacing the current Head Node as the node that can write.

When simply sharing a read-only copy of a node, as in when the memory is a ''FRESH'' state, the Head Node is somewhat irrelevant. All the sharing nodes have their own cached value of the cache line. If any node wants to write, including the Head Node, it must perform an additional action in order to do so. Likewise, when a sharing list is sharing a line of data in the dirty, or ''GONE'', memory state, as long as the Head Node has not written to the line, then all the sharing nodes stay in the list with their cached line. However, at the point the Head Node wants to write, it can write its data immediately, but it then must invalidate all other shared copies, via the forward pointers in the sharing list.

This, however, does not protect against a race condition where the memory is ''FRESH'', the Head Node wants to write, and another node also wants to join the list. The Memory Access mechanism prevents this condition.

==== Memory Access ====

=== Possible Race Conditions ===
==== Communication Delays ====

==== Concurrent List Deletions ====

==== Simultaneous Deletion and Invalidation ====

== Summary ==

== Definitions and Terms ==

== References ==

File:Cache states.png

2011-04-18T03:34:44Z

Eapotter: Cache State Diagram

Cache State Diagram

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-16T19:42:09Z

Eapotter: /* References */

__TOC__

= Overview =
= State Diagram =
= Race Conditions =
= References =
[1]:

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-16T19:02:42Z

Eapotter: /* References */

__TOC__

= Overview =
= State Diagram =
= Race Conditions =
= References =
[1]: "IEEE Standard for Scalable Coherent Interface (SCI).," IEEE Std 1596-1992 , vol., no., pp.i, 1993. doi: 10.1109/IEEESTD.1993.120366
URL: http://ieeexplore.ieee.org.proxied.lib.ncsu.edu/stamp/stamp.jsp?tp=&arnumber=347683&isnumber=8049

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-15T15:02:53Z

Eapotter:

__TOC__

= Overview =
= State Diagram =
= Race Conditions =
= References =
[1]: http://www.techarp.com/showarticle.aspx?artno=337

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-15T15:02:15Z

Eapotter:

__TOC__

== Overview ==
== State Diagram ==
== Race Conditions ==
= References =
[1]: http://www.techarp.com/showarticle.aspx?artno=337

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-15T15:00:30Z

Eapotter:

__TOC__

== Overview ==
== State Diagram ==
== Race Conditions ==

CSC/ECE 506 Spring 2011/ch11 BB EP

2011-04-15T15:00:08Z

Eapotter:

== Overview ==
== State Diagram ==
== Race Conditions ==

ECE506 Main Page

2011-04-15T14:51:26Z

Eapotter:

This page serves as a portal for all wiki material related to CSC506 and ECE506. Link to any new wiki pages from this page, and add links to any current pages.

=Supplements to Solihin Text=

Post links to the textbook supplements in this section.
*Chapter 2 [[CSC/ECE 506 Spring 2011/ch2 dm | CSC/ECE 506 Spring 2011/ch2 dm]]
*Chapter 2 [[Parallel_Programming_Models | Parallel Programming Models]]
*Chapter 2 (Still being revised) [[CSC/ECE 506 Spring 2011/ch2 cl | CSC/ECE 506 Spring 2011/ch2 cl]]
*Chapter 2a [[ CSC/ECE 506 Spring 2011/ch2a mc | Current Data-Parallel Architectures ]]
*Chapter 3 (Final Revision) [[ CSC/ECE 506 Spring 2011/ch3 ab | Parallel Architecture Mechanisms and Programming Models ]]
*Chapter 4a[[ CSC/ECE 506 Spring 2011/ch4a ob | Parallelization of Nelder Mead Algorithm ]]
*Chapter 4a (Under Construction) [[ CSC/ECE_506_Spring_2011/ch4a_bm | Parallelization of Algorithms ]]
*Chapter 4a [[ CSC/ECE 506 Spring 2011/ch4a zz | CSC/ECE 506 Spring 2011/ch4a zz ]]
*Chapter 6a (Under Construction) [[ CSC/ECE 506 Spring 2011/ch6a jp | CSC/ECE 506 Spring 2011/ch6a jp ]]
*Chapter 6a (Under Construction) [[ CSC/ECE 506 Spring 2011/ch6a ep | CSC/ECE 506 Spring 2011/ch6a ep ]]
*Chapter 6b (Ready for First Review) [[CSC/ECE 506 Spring 2011/ch6b ab | CSC/ECE 506 Spring 2011/ch6b ab]]
*Chapter 7 (Under Construction) [[CSC/ECE 506 Spring 2011/ch7 jp | CSC/ECE 506 Spring 2011/ch7 jp]]
*Chapter 8 (Under Construction) [[CSC/ECE 506 Spring 2011/ch8 mc | CSC/ECE 506 Spring 2011/ch8 mc]]
*Chapter 10 (Under Construction) [[CSC/ECE 506 Spring 2011/ch10 sb | CSC/ECE 506 Spring 2011/ch10 sb]]
*Chapter 10a [[CSC/ECE_506_Spring_2011/ch10a_dc | CSC/ECE_506_Spring_2011/ch10a_dc]]
*Chapter 11 [[CSC/ECE_506_Spring_2011/ch11_BB_EP | Chapter 11 Supplement]]

CSC/ECE 506 Spring 2011/ch6a ep

2011-03-08T04:23:37Z

Eapotter: /* Recent Architectures and their Cache Characteristics */

INTRODUCTION TO MEMORY HIERARCHY ORGANIZATION 
Write-Miss Policies and Prefetching

__TOC__

Write miss policies and prefetching are two strategies that are used by multiprocessors to achieve optimal performance for memory accesses. Write miss policies can help eliminate write misses and therefore reduce bus traffic. This reduces the wait-time of all processors sharing the bus. Prefetching assures that a CPU has data blocks in cache to be read when processing large data files and streaming data. When CPUs or cores share a cache, prefetched data in a shared cache is available to all processes on those cores processing the data.

This article begins by highlighting the variety of multicore processors on the market today that have hierarchal memory structures and shared caches. It then explores the write miss policies and prefetching techniques that these multiprocessors can use to take advantage of these architectures.

= Recent Architectures and their Cache Characteristics =

Cache management is impacted by three characteristics of modern processor architectures: multilevel cache hierarchies, multi-core and multiprocessor chip and systems designs, and shared vs. private caches.

Multi-level cache hierarchies are used by most contemporary chip designs to enable memory access to keep pace with CPU speeds that are advancing at a more rapid pace. Two-level and three-level cache hierarchies are common. L1 typically ranges from 16-64KB and provide access in 2-4 cycles. L2 typically ranges from 512KB to as much as 8MB and provides access in 6-15 cycles. L3 typically ranges from 4MB to 32MB and provides access to the data it contains within 50-30 cycles.[[#References|[8]]]

Multi-core and multprocessor designs place additional constraints on cache management by requiring them to consider all processes running concurrently in the cache coherency, replenishment, fetching, and other management functions.

In multiprocessors with multiple levels of cache, certain cache levels may be shared by processors and others private to each processor. Typically, lower-level caches are private and high-level caches may or may not be shared. Notice that none of the examples in the below have a shared L1 cache. Cache management functions must consider both shared and private caches when reading and writing data from memory.

Below is a table that illustrates the variety with which these characteristics have been combined in processors from four manufacturers over the past 6 years.

{| border="1" cellspacing="0" cellpadding="2"
!colspan="7"| Table 1: Recent Architectures and their Cache Characteristics [[#References|[1]]][[#References|[2]]][[#References|[3]]][[#References|[4]]][[#References|[9]]][[#References|[10]]][[#References|[11]]][[#References|[12]]][[#References|[13]]][[#References|[14]]][[#References|[15]]][[#References|[16]]]
|----
!Company
!Processor
!Cores
!L1 Cache
!L2 Cache
!L3 Cache
!Released
|----
|AMD
|Athlon 64 X2
|2
|128 KB x 2
|512 KB x 2 1MB x 2
|
|2005
|----
|AMD
|Athlon 64 FX
|2
|128 KB x 2
|1MB x 2
|
|2006
|----
|AMD
|Athlon X2
|2
|128 KB x 2
|512 KB x 2
|
|2007
|----
|AMD
|Phenom X3
|3
|128 KB x 3
|512 KB x 3
|2 MB
|2008
|----
|AMD
|Phenom X4
|4
|128 KB x 4
|512 KB x 4
|2 MB
|2007
|----
|AMD
|Athlon II X2
|2
|128 KB x 2
|512 KB x 2 1MB x 2
|
|2009
|----
|AMD
|Athlon II X3
|3
|128 KB x 3
|512 KB x 3
|
|2009
|----
|AMD
|Athlon II X4
|4
|128 KB x 4
|512 KB x 4
|
|2009
|----
|AMD
|Phenom II X2
|2
|128 KB x 2
|512 KB x 2
|6 MB
|2009
|----
|AMD
|Phenom II X3
|3
|128 KB x 3
|512 KB x 3
|6 MB
|2009
|----
|AMD
|Phenom II X4
|4
|128 KB x 4
|512 KB x 4
|4-6 MB
|2009
|----
|AMD
|Phenom II X6
|6
|128 KB x 6
|512 KB x 4
|6 MB
|2010
|----
|Intel
|Pentium D
|2
|12+16KB x 2
|2 MB
|
|2005
|----
|Intel
|Celeron E
|2
|32 KB x2
|512 -1 MB
|
|2006
|----
|Intel
|Pentium D
|2
|32 KB x2
|1 MB
|
|2006
|----
|Intel
|Core 2 Duo
|2
|32 KB x2
|2 - 4 MB
|
|2006
|----
|Intel
|Core 2 Quad
|4
|32 KB x2
|8 MB
|
|2007
|----
|Intel
|Atom 330
|2
|32+24KB x 2
|512 MB x 2
|
|2008
|----
|Intel
|Pentium E
|2
|32 KB x 2
|2 MB
|
|2007
|----
|Intel
|Core 2 Duo E
|2
|32 KB x 2
|3-6 MB
|
|2008
|----
|Intel
|Core 2 Quad
|4
|32 KB x 4
|2-6 MB x 2
|
|2008
|----
|Intel
|Core i3
|2
|32+32 KB x 2
|256 KB x2
|4 MB
|2010
|----
|Intel
|Core i5 - 6 Series
|2
|32+32 KB x 2
|256 KB x2
|4 MB
|2010
|----
|Intel
|Core i5 - 7 Series
|4
|32+32 KB x 4
|256 KB x 4
|8 MB
|2010
|----
|Intel
|Core i5 - 2400 Series Core i5 - 2500 Series
|4
|32+32 KB x 4
|256 KB x 4
|6 MB
|2010
|----
|Intel
|Core i7 - 8 Series
|4
|32+32 KB x 4
|256 KB x 4
|8 MB
|2010
|----
|Intel
|Core i7 - 9 Series
|4
|32+32 KB x 4
|256 KB x 4
|8 MB
|2010
|----
|Intel
|Core i7 - 970
|6
|32+32 KB x 6
|256 KB x 6
|12 MB
|2010
|----
|Sun
|UltraSPARC T1
|8
|8 K x 8 Inst. 16 K x 8 Data
|3 MB
|
|2005
|----
|Sun
|SPARC64 VI
|2
|128 K x 2 Inst. 128 K x 2 Data
|5 MB
|
|2007
|----
|Sun
|UltraSPARC T2
|8
|8 K x 4 Inst. 16 K x 4 Data
|4 MB
|
|2007
|----
|Sun
|SPARC64 VII
|4
|64 K x 4 Inst. 64 K x 4 Data
|6 MB
|
|2008
|----
|Sun
|SPARC T3
|8
|8 K x 8 Inst. 16 K x 8 Data
|6 MB
|
|2010
|----
|Sun
|SPARC64 VII+
|2
|128 K x 2 Inst. 128 K x 2 Data
|12 MB
|
|2010
|----
|IBM
|Power5
|2
|64 K x 2 Inst. 64 K x 2 Data
|4 MB x 2
|32 MB
|2004
|----
|IBM
|Power7
|4, 6, or 8
|32+32 KB x C
|256 kB x C
|4 - 32 MB x C
|2010
|----
|}

Note: The 'x #' in the L1 and L2 columns indicates that this cache is for each core.

The next two sections will discuss cache write policies and cache prefetching as techniques to improve the performance of these complex caching architectures by reducing write-miss and read-miss rates.

=Cache Write Policies[[#References|[5]]]=
In section 6.2.3[[#References|[8]]], cache write hit policies and write miss policies were explored. The write hit policies, write-through and write-back, determine when the data written in the local cache is propagated to lower-level memory. As review, write-through writes data to the cache and memory on a write. Write-back writes to cache first and to memory only when a flush is required.

The write miss policies covered in the text[[#References|[8]]], write-allocate and no-write-allocate, determine if a memory block is stored in a cache line after the write occurs. Note that since a miss occurred on write, the block is not already in a cache line.

Write-miss policies can also be expressed in terms of write-allocate, fetch-on-write, and write-before-hit. These can be used in combination with the same write-through and write-back hit policies to define the complete cache write policy. Investigating alternative policies such as these is important since write-misses produce on average one-third of all cache misses.

The following discusses each of these policies:

==Fetch-on-Write==
On a write miss, the memory block containing the address to be written is fetched from the lower level memory hierarchy before the write proceeds. Note that this is different from write-allocate. Fetch-on-write determines if the block is fetched from memory, and write-allocate determines if the memory block is stored in a cache line. Certain write policies may allocate a line in cache for data that is written by the CPU without retrieving it from memory first.

==No-Fetch-on-Write==
On a write miss, the memory block is not fetched first from the lower level memory hierarchy. Therefore, the write can proceed with out having to wait for the memory block to be returned.

==Write-Before-Hit==
On a write, the write proceeds before the cache determines if a hit or miss occurred. In this scenario, the tag and the data can be written simultaneously, but it incurs an immediate bus transaction for each write by the processor.

==No-Write-Before-Hit==
On a write, the write waits until the cache determines if the block being written to is in the cache or not. This may avoid a bus transaction by allowing the processor to write to the cache multiple times before the cache line is flushed to memory.

==Write-Miss Policy Combinations==

In practice, these policies are used in combination to provide an over-all write policy. Four combinations of these three write miss policies are relevant, as illustrated in the table below:

<center>[[image:policies.jpg]]</center>

The two combinations of fetch-on-write and no-write-allocate are blocked out in the above diagram. They are considered unproductive since the data is fetched from lower level memory, but is not stored in the cache. Temporal and spacial locality suggest that this data will be used again, and will therefore need to be retrieved a second time.

Three combinations named write-validate, write-around, and write-invalidate all use no-fetch-on-write. They result in 'eliminated misses' when compared to a fetch-on-write policy. In general, this will yield better cache performance if the overhead to manage the policy remains low.

The following discusses each of these combinations:

===Write-Validate===
The combination of no-fetch-on-write and write-allocation is referred to as 'write-validate'. It writes the data into the cache line without fetching the corresponding block from memory first. The assumption is that the block will be written to memory at a later time. It requires additional overhead, or dirty bits, to track what bytes have been written into that cache line and which bytes were not written. Lower level memories also must be able can process only the changed portions of these lines. Otherwise, when the line is flushed to memory, the unwritten bytes may overwrite valid data.

The no-fetch-on-write has the advantage of avoiding an immediate bus transaction to retrieve the block from memory. For a multiprocessor with a cache coherency model, though, a bus transaction will be required to get exclusive ownership of the block. While this appears to negate the advantages of no-fetch-on-write, the process is able to continue other tasks while this transaction is occurring, including rewriting to the same cache location.

Tests by the author demonstrated more than a 90% reduction in write misses when compared to fetch-on-write policies.

===Write-Around===
The combination of no-fetch-on-write, no-write-allocate, and no-write-before-hit is referred to as a 'write-around'. It bypasses the higher level cache, writes directly to the lower level memory, and leaves the existing block line in the cache unaltered on a hit or a miss. This strategy shows performance improvements when the data that is written will not be reread in the near future. Since we are writing before a hit is detected, the cache is written around for both hits and misses.

The author notes that in only but a few cases write-around performs worse than write-validate policies. Most applications tend to reread what they have recently written. Using a write-around policy, this would result in a cache miss and a read from lower-level memory. With write-validate, the data would be in cache.

Tests by the author demonstrated a 40-70% reduction in write misses when compared to fetch-on-write policies.

===Write-Invalidate===
The combination of write-before-hit, no-fetch-on-write, and no-write-allocate is referred to as 'write-invalidate' because the line is invalidated on the miss. The copy that exists in lower level memory after the write miss differs from the one in the cache. For write hits, though, the data is simply written into the cache using the cache hit policy. Thus, for hits, the cache is not written around.

Of these three combinations, write validate performed the worse. The author notes, though, that it does perform better than fetch-on-write and is easy to implement. Tests by the author demonstrated a 35-50% reduction in write misses when compared to fetch-on-write policies.

===Fetch-on-write===
When used with write-allocate, fetch-on-write stores the block and processes the write to cache as expected. This policy was used as the base-line for determining the performance of the other three policy combinations. In general, all three previous combinations exhibited fewer misses than this policy.

=Prefetching in Contemporary Parallel Processors=
Prefetching is a technique that can be used in addition to write-hit and write-miss policies to improved the utilization of the cache. Microprocessors, such as those listed in Table 1, support prefetching logic directly in the chip.

Prefetching is the process of retrieving instructions or data from memory before the process explicitly requests it. Instruction prefetching is commonly used by single and multiprocessors to reduce process wait states.[[#References|[17]]] Prefetching of data may also be used, though, to pre-populate caches with data that is likely going to be required by the processors in the near term. If the data requirements are anticipated correctly, the requests to memory will result in a greater cache hit rates and, therefore, reduce overall memory access time. If the prefetcher guesses wrong, bus traffic can increase unnecessarily, more relevant data can be flushed from caches, and miss rates can increase.

Prefetching algorithms can leverage both temporal and spacial locality in making these decisions. For example, streaming and sequential access applications often process adjacent memory locations in subsequent tasks.

Following are two examples of processors that support prefetching directly in the chip design:

== Intel Core i7 [[#References|[6]]]==
The Intel 64 Architecture, including the Intel Core i7, includes both instruction and data prefetching directly on the chip.

The Data Cache Unit prefetcher is a streaming prefetcher for L1 caches that detects ascending access to data that has been loaded very recently. The processor assumes that this ascending access will continue, and prefetches the next line.

The data prefetch logic (DPL) maintains two arrays to track the recent accesses to memory: one for the upstreams that has 12 entries, and one for downstreams that has 4 entires. As pages are accessed, their addresses are tracked in these arrays. When the DPL detects an access to a page that is sequential to an existing entry, it assumes this sequential access will continue, and prefetches the next cache line from memory.

== AMD [[#References|[7]]]==

The AMD Family 10h processors use a stream-detection strategy similar to the Intel process described above to trigger prefetching of the next sequential memory location into the L1 cache. (Previous AMD processors fetched into the L2 cache, which introduced the access latency of the L2 cache and hindered performance.)

The AMD Family 10h can prefetch more than just the next sequential block when a stream is detected, though. When this 'unit-stride' prefetcher detects misses to sequential blocks, it can trigger a preset number of prefetch requests from memory. For example, if the preset value is two, the next two blocks of memory will be prefetched when a sequential access is detected. Subsequent detection of misses of sequential blocks may only prefetch a single block. This maintains the 'unit-stride' of two blocks in the cache ahead of the next anticipated read.

AMD contends that this is beneficial when processing large data sets which often process sequential data and process all the data in the stream. In these cases, a larger unit-stride populates the cache with blocks that will ultimately be processed by the CPU. AMD suggests that the optimal number of blocks to prefetch is between 4 and 8, and that fetching too many or fetching too soon has impaired performance in empirical testing.

The AMD Family 10h also includes 'Adaptive Prefetching', a hardware optimization that triggers prefetching when the demand stream catches up to the prefetch stream. In this case, the unit-stride is increased to assure that the prefetcher is fetching at a rate sufficient enough to continuously provide data to the processor.

= References =
[1]: http://www.techarp.com/showarticle.aspx?artno=337 
[2]: http://en.wikipedia.org/wiki/SPARC 
[3]: http://en.wikipedia.org/wiki/POWER7 
[4]: http://en.wikipedia.org/wiki/POWER5 
[5]: “Cache write policies and performance,” Norman Jouppi, Proc. 20th International Symposium on Computer Architecture (ACM Computer Architecture News 21:2), May 1993, pp. 191–201. http://www.hpl.hp.com/techreports/Compaq-DEC/WRL-91-12.pdf 
[6]: "Intel® 64 and IA-32 Architectures Optimization Reference Manual", Intel Corporation, 1997-2011. http://www.intel.com/Assets/PDF/manual/248966.pdf 
[7]: "Software Optimization Guide for AMD Family 10h and 12h Processors", Advanced Micro Devices, Inc., 2006–2010. http://support.amd.com/us/Processor_TechDocs/40546.pdf 
[8]: Yan Solihin, ''Fundamentals of Parallel Computer Architecture'' (Solihin Publishing and Consulting, LLC, 2008-2009), p. 151 
[9]: http://en.wikipedia.org/wiki/AMD_Phenom 
[10]: http://en.wikipedia.org/wiki/Phenom_II 
[11]: http://en.wikipedia.org/wiki/Athlon_II 
[12]: http://en.wikipedia.org/wiki/List_of_AMD_Athlon_64_microprocessors 
[13]: http://en.wikipedia.org/wiki/List_of_AMD_Athlon_X2_microprocessors 
[14]: http://en.wikipedia.org/wiki/List_of_Intel_Core_i5_microprocessors 
[15]: http://en.wikipedia.org/wiki/List_of_Intel_Core_i3_microprocessors 
[16]: http://www.intel.com/pressroom/kits/quickrefyr.htm 
[17]: http://en.wikipedia.org/wiki/Instruction_prefetch

CSC/ECE 506 Spring 2011/ch6a ep

2011-03-08T04:22:02Z

Eapotter: /* Recent Architectures and their Cache Characteristics */

INTRODUCTION TO MEMORY HIERARCHY ORGANIZATION 
Write-Miss Policies and Prefetching

__TOC__

Write miss policies and prefetching are two strategies that are used by multiprocessors to achieve optimal performance for memory accesses. Write miss policies can help eliminate write misses and therefore reduce bus traffic. This reduces the wait-time of all processors sharing the bus. Prefetching assures that a CPU has data blocks in cache to be read when processing large data files and streaming data. When CPUs or cores share a cache, prefetched data in a shared cache is available to all processes on those cores processing the data.

This article begins by highlighting the variety of multicore processors on the market today that have hierarchal memory structures and shared caches. It then explores the write miss policies and prefetching techniques that these multiprocessors can use to take advantage of these architectures.

= Recent Architectures and their Cache Characteristics =

Cache management is impacted by three characteristics of modern processor architectures: multilevel cache hierarchies, multi-core and multiprocessor chip and systems designs, and shared vs. private caches.

Multi-level cache hierarchies are used by most contemporary chip designs to enable memory access to keep pace with CPU speeds that are advancing at a more rapid pace. Two-level and three-level cache hierarchies are common. L1 typically ranges from 16-64KB and provide access in 2-4 cycles. L2 typically ranges from 512KB to as much as 8MB and provides access in 6-15 cycles. L3 typically ranges from 4MB to 32MB and provides access to the data it contains within 50-30 cycles.[[#References|[8]]]

Multi-core and multprocessor designs place additional constraints on cache management by requiring them to consider all processes running concurrently in the cache coherency, replenishment, fetching, and other management functions.

In multiprocessors with multiple levels of cache, certain cache levels may be shared by processors and others private to each processor. Typically, lower-level caches are private and high-level caches may or may not be shared. Notice that none of the examples in the below have a shared L1 cache. Cache management functions must consider both shared and private caches when reading and writing data from memory.

Below is a table that illustrates the variety with which these characteristics have been combined in processors from four manufacturers over the past 6 years.

{| border="1" cellspacing="0" cellpadding="2"
!colspan="7"| Table 1: Recent Architectures and their Cache Characteristics [[#References|[1]]][[#References|[2]]][[#References|[3]]][[#References|[4]]][[#References|[9]]][[#References|[10]]][[#References|[11]]][[#References|[12]]][[#References|[13]]][[#References|[14]]][[#References|[15]]][[#References|[16]]]
|----
!Company
!Processor
!Cores
!L1 Cache
!L2 Cache
!L3 Cache
!Released
|----
|AMD
|Athlon 64 X2
|2
|128 KB x 2
|512 KB x 2 1MB x 2
|
|2005
|----
|AMD
|Athlon 64 FX
|2
|128 KB x 2
|1MB x 2
|
|2006
|----
|AMD
|Athlon X2
|2
|128 KB x 2
|512 KB x 2
|
|2007
|----
|AMD
|Phenom X3
|3
|128 KB x 3
|512 KB x 3
|2 MB
|2008
|----
|AMD
|Phenom X4
|4
|128 KB x 4
|512 KB x 4
|2 MB
|2007
|----
|AMD
|Athlon II X2
|2
|128 KB x 2
|512 KB x 2 1MB x 2
|
|2009
|----
|AMD
|Athlon II X3
|3
|128 KB x 3
|512 KB x 3
|
|2009
|----
|AMD
|Athlon II X4
|4
|128 KB x 4
|512 KB x 4
|
|2009
|----
|AMD
|Phenom II X2
|2
|128 KB x 2
|512 KB x 2
|6 MB
|2009
|----
|AMD
|Phenom II X3
|3
|128 KB x 3
|512 KB x 3
|6 MB
|2009
|----
|AMD
|Phenom II X4
|4
|128 KB x 4
|512 KB x 4
|4-6 MB
|2009
|----
|AMD
|Phenom II X6
|6
|128 KB x 6
|512 KB x 4
|6 MB
|2010
|----
|Intel
|Pentium D
|2
|12+16KB x 2
|2 MB
|
|2005
|----
|Intel
|Celeron E
|2
|32 KB x2
|512 -1 MB
|
|2006
|----
|Intel
|Pentium D
|2
|32 KB x2
|1 MB
|
|2006
|----
|Intel
|Core 2 Duo
|2
|32 KB x2
|2 - 4 MB
|
|2006
|----
|Intel
|Core 2 Quad
|4
|32 KB x2
|8 MB
|
|2007
|----
|Intel
|Atom 330
|2
|32+24KB x 2
|512 MB x 2
|
|2008
|----
|Intel
|Pentium E
|2
|32 KB x 2
|2 MB
|
|2007
|----
|Intel
|Core 2 Duo E
|2
|32 KB x 2
|3-6 MB
|
|2008
|----
|Intel
|Core 2 Quad
|4
|32 KB x 4
|2-6 MB x 2
|
|2008
|----
|Intel
|Core i3
|2
|32+32 KB x 2
|256 KB x2
|4 MB
|2010
|----
|Intel
|Core i5 - 6 Series
|2
|32+32 KB x 2
|256 KB x2
|4 MB
|2010
|----
|Intel
|Core i5 - 7 Series
|4
|32+32 KB x 4
|256 KB x 4
|8 MB
|2010
|----
|Intel
|Core i5 - 2400 Series Core i5 - 2500 Series
|4
|32+32 KB x 4
|256 KB x 4
|6 MB
|2010
|----
|Intel
|Core i7 - 8 Series
|4
|32+32 KB x 4
|256 KB x 4
|8 MB
|2010
|----
|Intel
|Core i7 - 9 Series
|4
|32+32 KB x 4
|256 KB x 4
|8 MB
|2010
|----
|Intel
|Core i7 - 970
|6
|32+32 KB x 6
|256 KB x 6
|12 MB
|2010
|----
|Sun
|UltraSPARC T1
|8
|8 K x 8 Inst. 16 K x 8 Data
|3 MB
|
|2005
|----
|Sun
|SPARC64 VI
|2
|128 K x 2 Inst. 128 K x 2 Data
|5 MB
|
|2007
|----
|Sun
|UltraSPARC T2
|8
|8 K x 4 Inst. 16 K x 4 Data
|4 MB
|
|2007
|----
|Sun
|SPARC64 VII
|4
|64 K x 4 Inst. 64 K x 4 Data
|6 MB
|
|2008
|----
|Sun
|SPARC T3
|8
|8 K x 8 Inst. 16 K x 8 Data
|6 MB
|
|2010
|----
|Sun
|SPARC64 VII+
|2
|128 K x 2 Inst. 128 K x 2 Data
|12 MB
|
|2010
|----
|IBM
|Power5
|2
|64 K x 2 Inst. 64 K x 2 Data
|4 MB x 2
|32 MB
|2004
|----
|IBM
|Power7
|4, 6, or 8
|32+32 KB x C
|256 kB x C
|4 - 32 MB x C
|2010
|----
|}

Note: The 'x #' in the L1 and L2 columns indicates that this cache is for each core.

The next two sections will discuss cache write policies and cache prefetching as techniques to improve cache performance by reducing write-miss and read-miss rates.

=Cache Write Policies[[#References|[5]]]=
In section 6.2.3[[#References|[8]]], cache write hit policies and write miss policies were explored. The write hit policies, write-through and write-back, determine when the data written in the local cache is propagated to lower-level memory. As review, write-through writes data to the cache and memory on a write. Write-back writes to cache first and to memory only when a flush is required.

The write miss policies covered in the text[[#References|[8]]], write-allocate and no-write-allocate, determine if a memory block is stored in a cache line after the write occurs. Note that since a miss occurred on write, the block is not already in a cache line.

Write-miss policies can also be expressed in terms of write-allocate, fetch-on-write, and write-before-hit. These can be used in combination with the same write-through and write-back hit policies to define the complete cache write policy. Investigating alternative policies such as these is important since write-misses produce on average one-third of all cache misses.

The following discusses each of these policies:

==Fetch-on-Write==
On a write miss, the memory block containing the address to be written is fetched from the lower level memory hierarchy before the write proceeds. Note that this is different from write-allocate. Fetch-on-write determines if the block is fetched from memory, and write-allocate determines if the memory block is stored in a cache line. Certain write policies may allocate a line in cache for data that is written by the CPU without retrieving it from memory first.

==No-Fetch-on-Write==
On a write miss, the memory block is not fetched first from the lower level memory hierarchy. Therefore, the write can proceed with out having to wait for the memory block to be returned.

==Write-Before-Hit==
On a write, the write proceeds before the cache determines if a hit or miss occurred. In this scenario, the tag and the data can be written simultaneously, but it incurs an immediate bus transaction for each write by the processor.

==No-Write-Before-Hit==
On a write, the write waits until the cache determines if the block being written to is in the cache or not. This may avoid a bus transaction by allowing the processor to write to the cache multiple times before the cache line is flushed to memory.

==Write-Miss Policy Combinations==

In practice, these policies are used in combination to provide an over-all write policy. Four combinations of these three write miss policies are relevant, as illustrated in the table below:

<center>[[image:policies.jpg]]</center>

The two combinations of fetch-on-write and no-write-allocate are blocked out in the above diagram. They are considered unproductive since the data is fetched from lower level memory, but is not stored in the cache. Temporal and spacial locality suggest that this data will be used again, and will therefore need to be retrieved a second time.

Three combinations named write-validate, write-around, and write-invalidate all use no-fetch-on-write. They result in 'eliminated misses' when compared to a fetch-on-write policy. In general, this will yield better cache performance if the overhead to manage the policy remains low.

The following discusses each of these combinations:

===Write-Validate===
The combination of no-fetch-on-write and write-allocation is referred to as 'write-validate'. It writes the data into the cache line without fetching the corresponding block from memory first. The assumption is that the block will be written to memory at a later time. It requires additional overhead, or dirty bits, to track what bytes have been written into that cache line and which bytes were not written. Lower level memories also must be able can process only the changed portions of these lines. Otherwise, when the line is flushed to memory, the unwritten bytes may overwrite valid data.

The no-fetch-on-write has the advantage of avoiding an immediate bus transaction to retrieve the block from memory. For a multiprocessor with a cache coherency model, though, a bus transaction will be required to get exclusive ownership of the block. While this appears to negate the advantages of no-fetch-on-write, the process is able to continue other tasks while this transaction is occurring, including rewriting to the same cache location.

Tests by the author demonstrated more than a 90% reduction in write misses when compared to fetch-on-write policies.

===Write-Around===
The combination of no-fetch-on-write, no-write-allocate, and no-write-before-hit is referred to as a 'write-around'. It bypasses the higher level cache, writes directly to the lower level memory, and leaves the existing block line in the cache unaltered on a hit or a miss. This strategy shows performance improvements when the data that is written will not be reread in the near future. Since we are writing before a hit is detected, the cache is written around for both hits and misses.

The author notes that in only but a few cases write-around performs worse than write-validate policies. Most applications tend to reread what they have recently written. Using a write-around policy, this would result in a cache miss and a read from lower-level memory. With write-validate, the data would be in cache.

Tests by the author demonstrated a 40-70% reduction in write misses when compared to fetch-on-write policies.

===Write-Invalidate===
The combination of write-before-hit, no-fetch-on-write, and no-write-allocate is referred to as 'write-invalidate' because the line is invalidated on the miss. The copy that exists in lower level memory after the write miss differs from the one in the cache. For write hits, though, the data is simply written into the cache using the cache hit policy. Thus, for hits, the cache is not written around.

Of these three combinations, write validate performed the worse. The author notes, though, that it does perform better than fetch-on-write and is easy to implement. Tests by the author demonstrated a 35-50% reduction in write misses when compared to fetch-on-write policies.

===Fetch-on-write===
When used with write-allocate, fetch-on-write stores the block and processes the write to cache as expected. This policy was used as the base-line for determining the performance of the other three policy combinations. In general, all three previous combinations exhibited fewer misses than this policy.

=Prefetching in Contemporary Parallel Processors=
Prefetching is a technique that can be used in addition to write-hit and write-miss policies to improved the utilization of the cache. Microprocessors, such as those listed in Table 1, support prefetching logic directly in the chip.

Prefetching is the process of retrieving instructions or data from memory before the process explicitly requests it. Instruction prefetching is commonly used by single and multiprocessors to reduce process wait states.[[#References|[17]]] Prefetching of data may also be used, though, to pre-populate caches with data that is likely going to be required by the processors in the near term. If the data requirements are anticipated correctly, the requests to memory will result in a greater cache hit rates and, therefore, reduce overall memory access time. If the prefetcher guesses wrong, bus traffic can increase unnecessarily, more relevant data can be flushed from caches, and miss rates can increase.

Prefetching algorithms can leverage both temporal and spacial locality in making these decisions. For example, streaming and sequential access applications often process adjacent memory locations in subsequent tasks.

Following are two examples of processors that support prefetching directly in the chip design:

== Intel Core i7 [[#References|[6]]]==
The Intel 64 Architecture, including the Intel Core i7, includes both instruction and data prefetching directly on the chip.

The Data Cache Unit prefetcher is a streaming prefetcher for L1 caches that detects ascending access to data that has been loaded very recently. The processor assumes that this ascending access will continue, and prefetches the next line.

The data prefetch logic (DPL) maintains two arrays to track the recent accesses to memory: one for the upstreams that has 12 entries, and one for downstreams that has 4 entires. As pages are accessed, their addresses are tracked in these arrays. When the DPL detects an access to a page that is sequential to an existing entry, it assumes this sequential access will continue, and prefetches the next cache line from memory.

== AMD [[#References|[7]]]==

The AMD Family 10h processors use a stream-detection strategy similar to the Intel process described above to trigger prefetching of the next sequential memory location into the L1 cache. (Previous AMD processors fetched into the L2 cache, which introduced the access latency of the L2 cache and hindered performance.)

The AMD Family 10h can prefetch more than just the next sequential block when a stream is detected, though. When this 'unit-stride' prefetcher detects misses to sequential blocks, it can trigger a preset number of prefetch requests from memory. For example, if the preset value is two, the next two blocks of memory will be prefetched when a sequential access is detected. Subsequent detection of misses of sequential blocks may only prefetch a single block. This maintains the 'unit-stride' of two blocks in the cache ahead of the next anticipated read.

AMD contends that this is beneficial when processing large data sets which often process sequential data and process all the data in the stream. In these cases, a larger unit-stride populates the cache with blocks that will ultimately be processed by the CPU. AMD suggests that the optimal number of blocks to prefetch is between 4 and 8, and that fetching too many or fetching too soon has impaired performance in empirical testing.

The AMD Family 10h also includes 'Adaptive Prefetching', a hardware optimization that triggers prefetching when the demand stream catches up to the prefetch stream. In this case, the unit-stride is increased to assure that the prefetcher is fetching at a rate sufficient enough to continuously provide data to the processor.

= References =
[1]: http://www.techarp.com/showarticle.aspx?artno=337 
[2]: http://en.wikipedia.org/wiki/SPARC 
[3]: http://en.wikipedia.org/wiki/POWER7 
[4]: http://en.wikipedia.org/wiki/POWER5 
[5]: “Cache write policies and performance,” Norman Jouppi, Proc. 20th International Symposium on Computer Architecture (ACM Computer Architecture News 21:2), May 1993, pp. 191–201. http://www.hpl.hp.com/techreports/Compaq-DEC/WRL-91-12.pdf 
[6]: "Intel® 64 and IA-32 Architectures Optimization Reference Manual", Intel Corporation, 1997-2011. http://www.intel.com/Assets/PDF/manual/248966.pdf 
[7]: "Software Optimization Guide for AMD Family 10h and 12h Processors", Advanced Micro Devices, Inc., 2006–2010. http://support.amd.com/us/Processor_TechDocs/40546.pdf 
[8]: Yan Solihin, ''Fundamentals of Parallel Computer Architecture'' (Solihin Publishing and Consulting, LLC, 2008-2009), p. 151 
[9]: http://en.wikipedia.org/wiki/AMD_Phenom 
[10]: http://en.wikipedia.org/wiki/Phenom_II 
[11]: http://en.wikipedia.org/wiki/Athlon_II 
[12]: http://en.wikipedia.org/wiki/List_of_AMD_Athlon_64_microprocessors 
[13]: http://en.wikipedia.org/wiki/List_of_AMD_Athlon_X2_microprocessors 
[14]: http://en.wikipedia.org/wiki/List_of_Intel_Core_i5_microprocessors 
[15]: http://en.wikipedia.org/wiki/List_of_Intel_Core_i3_microprocessors 
[16]: http://www.intel.com/pressroom/kits/quickrefyr.htm 
[17]: http://en.wikipedia.org/wiki/Instruction_prefetch

CSC/ECE 506 Spring 2011/ch6a ep

2011-03-08T04:20:40Z

Eapotter: /* Recent Architectures and their Cache Characteristics */

INTRODUCTION TO MEMORY HIERARCHY ORGANIZATION 
Write-Miss Policies and Prefetching

__TOC__

Write miss policies and prefetching are two strategies that are used by multiprocessors to achieve optimal performance for memory accesses. Write miss policies can help eliminate write misses and therefore reduce bus traffic. This reduces the wait-time of all processors sharing the bus. Prefetching assures that a CPU has data blocks in cache to be read when processing large data files and streaming data. When CPUs or cores share a cache, prefetched data in a shared cache is available to all processes on those cores processing the data.

This article begins by highlighting the variety of multicore processors on the market today that have hierarchal memory structures and shared caches. It then explores the write miss policies and prefetching techniques that these multiprocessors can use to take advantage of these architectures.

= Recent Architectures and their Cache Characteristics =

Cache management is impacted by three characteristics of modern processor architectures: multilevel cache hierarchies, multi-core and multiprocessor chip and systems designs, and shared vs. private caches.

Multi-level cache hierarchies are used by most contemporary chip designs to enable memory access to keep pace with CPU speeds that are advancing at a more rapid pace. Two-level and three-level cache hierarchies are common. L1 typically ranges from 16-64KB and provide access in 2-4 cycles. L2 typically ranges from 512KB to as much as 8MB and provides access in 6-15 cycles. L3 typically ranges from 4MB to 32MB and provides access to the data it contains within 50-30 cycles.[[#References|[8]]]

Multi-core and multprocessor designs place additional constraints on cache management by requiring them to consider all processes running concurrently in the cache coherency, replenishment, fetching, and other management functions.

In multiprocessors with multiple levels of cache, certain cache levels may be shared by processors and others private to each processor. Typically, lower-level caches are private and high-level caches may or may not be shared. Notice that none of the examples in the below have a shared L1 cache. Cache management functions must consider both shared and private caches when reading and writing data from memory.

Below is a table that illustrates the variety with which these characteristics have been combined in processors from four manufacturers over the past 6 years.

{| border="1" cellspacing="0" cellpadding="2"
!colspan="7"| Table 1: Recent Architectures and their Cache Characteristics [[#References|[1]]][[#References|[2]]][[#References|[3]]][[#References|[4]]][[#References|[9]]][[#References|[10]]][[#References|[11]]][[#References|[12]]][[#References|[13]]][[#References|[14]]][[#References|[15]]][[#References|[16]]]
|----
!Company
!Processor
!Cores
!L1 Cache
!L2 Cache
!L3 Cache
!Released
|----
|AMD
|Athlon 64 X2
|2
|128 KB x 2
|512 KB x 2 1MB x 2
|
|2005
|----
|AMD
|Athlon 64 FX
|2
|128 KB x 2
|1MB x 2
|
|2006
|----
|AMD
|Athlon X2
|2
|128 KB x 2
|512 KB x 2
|
|2007
|----
|AMD
|Phenom X3
|3
|128 KB x 3
|512 KB x 3
|2 MB
|2008
|----
|AMD
|Phenom X4
|4
|128 KB x 4
|512 KB x 4
|2 MB
|2007
|----
|AMD
|Athlon II X2
|2
|128 KB x 2
|512 KB x 2 1MB x 2
|
|2009
|----
|AMD
|Athlon II X3
|3
|128 KB x 3
|512 KB x 3
|
|2009
|----
|AMD
|Athlon II X4
|4
|128 KB x 4
|512 KB x 4
|
|2009
|----
|AMD
|Phenom II X2
|2
|128 KB x 2
|512 KB x 2
|6 MB
|2009
|----
|AMD
|Phenom II X3
|3
|128 KB x 3
|512 KB x 3
|6 MB
|2009
|----
|AMD
|Phenom II X4
|4
|128 KB x 4
|512 KB x 4
|4-6 MB
|2009
|----
|AMD
|Phenom II X6
|6
|128 KB x 6
|512 KB x 4
|6 MB
|2010
|----
|Intel
|Pentium D
|2
|12+16KB x 2
|2 MB
|
|2005
|----
|Intel
|Celeron E
|2
|32 KB x2
|512 -1 MB
|
|2006
|----
|Intel
|Pentium D
|2
|32 KB x2
|1 MB
|
|2006
|----
|Intel
|Core 2 Duo
|2
|32 KB x2
|2 - 4 MB
|
|2006
|----
|Intel
|Core 2 Quad
|4
|32 KB x2
|8 MB
|
|2007
|----
|Intel
|Atom 330
|2
|32+24KB x 2
|512 MB x 2
|
|2008
|----
|Intel
|Pentium E
|2
|32 KB x 2
|2 MB
|
|2007
|----
|Intel
|Core 2 Duo E
|2
|32 KB x 2
|3-6 MB
|
|2008
|----
|Intel
|Core 2 Quad
|4
|32 KB x 4
|2-6 MB x 2
|
|2008
|----
|Intel
|Core i3
|2
|32+32 KB x 2
|256 KB x2
|4 MB
|2010
|----
|Intel
|Core i5 - 6 Series
|2
|32+32 KB x 2
|256 KB x2
|4 MB
|2010
|----
|Intel
|Core i5 - 7 Series
|4
|32+32 KB x 4
|256 KB x 4
|8 MB
|2010
|----
|Intel
|Core i5 - 2400 Series Core i5 - 2500 Series
|4
|32+32 KB x 4
|256 KB x 4
|6 MB
|2010
|----
|Intel
|Core i7 - 8 Series
|4
|32+32 KB x 4
|256 KB x 4
|8 MB
|2010
|----
|Intel
|Core i7 - 9 Series
|4
|32+32 KB x 4
|256 KB x 4
|8 MB
|2010
|----
|Intel
|Core i7 - 970
|6
|32+32 KB x 6
|256 KB x 6
|12 MB
|2010
|----
|Sun
|UltraSPARC T1
|8
|8 K x 8 Inst. 16 K x 8 Data
|3 MB
|
|2005
|----
|Sun
|SPARC64 VI
|2
|128 K x 2 Inst. 128 K x 2 Data
|5 MB
|
|2007
|----
|Sun
|UltraSPARC T2
|8
|8 K x 4 Inst. 16 K x 4 Data
|4 MB
|
|2007
|----
|Sun
|SPARC64 VII
|4
|64 K x 4 Inst. 64 K x 4 Data
|6 MB
|
|2008
|----
|Sun
|SPARC T3
|8
|8 K x 8 Inst. 16 K x 8 Data
|6 MB
|
|2010
|----
|Sun
|SPARC64 VII+
|2
|128 K x 2 Inst. 128 K x 2 Data
|12 MB
|
|2010
|----
|IBM
|Power5
|2
|64 K x 2 Inst. 64 K x 2 Data
|4 MB x 2
|32 MB
|2004
|----
|IBM
|Power7
|4, 6, or 8
|32+32 KB x C
|256 kB x C
|4 - 32 MB x C
|2010
|----
|}

Note: The 'x #' in the L1 and L2 columns indicates that this cache is for each core.

The next two sections will discuss cache write policies and cache prefetching as two techniques improve cache performance by reducing write-miss and read-miss rates.

=Cache Write Policies[[#References|[5]]]=
In section 6.2.3[[#References|[8]]], cache write hit policies and write miss policies were explored. The write hit policies, write-through and write-back, determine when the data written in the local cache is propagated to lower-level memory. As review, write-through writes data to the cache and memory on a write. Write-back writes to cache first and to memory only when a flush is required.

The write miss policies covered in the text[[#References|[8]]], write-allocate and no-write-allocate, determine if a memory block is stored in a cache line after the write occurs. Note that since a miss occurred on write, the block is not already in a cache line.

Write-miss policies can also be expressed in terms of write-allocate, fetch-on-write, and write-before-hit. These can be used in combination with the same write-through and write-back hit policies to define the complete cache write policy. Investigating alternative policies such as these is important since write-misses produce on average one-third of all cache misses.

The following discusses each of these policies:

==Fetch-on-Write==
On a write miss, the memory block containing the address to be written is fetched from the lower level memory hierarchy before the write proceeds. Note that this is different from write-allocate. Fetch-on-write determines if the block is fetched from memory, and write-allocate determines if the memory block is stored in a cache line. Certain write policies may allocate a line in cache for data that is written by the CPU without retrieving it from memory first.

==No-Fetch-on-Write==
On a write miss, the memory block is not fetched first from the lower level memory hierarchy. Therefore, the write can proceed with out having to wait for the memory block to be returned.

==Write-Before-Hit==
On a write, the write proceeds before the cache determines if a hit or miss occurred. In this scenario, the tag and the data can be written simultaneously, but it incurs an immediate bus transaction for each write by the processor.

==No-Write-Before-Hit==
On a write, the write waits until the cache determines if the block being written to is in the cache or not. This may avoid a bus transaction by allowing the processor to write to the cache multiple times before the cache line is flushed to memory.

==Write-Miss Policy Combinations==

In practice, these policies are used in combination to provide an over-all write policy. Four combinations of these three write miss policies are relevant, as illustrated in the table below:

<center>[[image:policies.jpg]]</center>

The two combinations of fetch-on-write and no-write-allocate are blocked out in the above diagram. They are considered unproductive since the data is fetched from lower level memory, but is not stored in the cache. Temporal and spacial locality suggest that this data will be used again, and will therefore need to be retrieved a second time.

Three combinations named write-validate, write-around, and write-invalidate all use no-fetch-on-write. They result in 'eliminated misses' when compared to a fetch-on-write policy. In general, this will yield better cache performance if the overhead to manage the policy remains low.

The following discusses each of these combinations:

===Write-Validate===
The combination of no-fetch-on-write and write-allocation is referred to as 'write-validate'. It writes the data into the cache line without fetching the corresponding block from memory first. The assumption is that the block will be written to memory at a later time. It requires additional overhead, or dirty bits, to track what bytes have been written into that cache line and which bytes were not written. Lower level memories also must be able can process only the changed portions of these lines. Otherwise, when the line is flushed to memory, the unwritten bytes may overwrite valid data.

The no-fetch-on-write has the advantage of avoiding an immediate bus transaction to retrieve the block from memory. For a multiprocessor with a cache coherency model, though, a bus transaction will be required to get exclusive ownership of the block. While this appears to negate the advantages of no-fetch-on-write, the process is able to continue other tasks while this transaction is occurring, including rewriting to the same cache location.

Tests by the author demonstrated more than a 90% reduction in write misses when compared to fetch-on-write policies.

===Write-Around===
The combination of no-fetch-on-write, no-write-allocate, and no-write-before-hit is referred to as a 'write-around'. It bypasses the higher level cache, writes directly to the lower level memory, and leaves the existing block line in the cache unaltered on a hit or a miss. This strategy shows performance improvements when the data that is written will not be reread in the near future. Since we are writing before a hit is detected, the cache is written around for both hits and misses.

The author notes that in only but a few cases write-around performs worse than write-validate policies. Most applications tend to reread what they have recently written. Using a write-around policy, this would result in a cache miss and a read from lower-level memory. With write-validate, the data would be in cache.

Tests by the author demonstrated a 40-70% reduction in write misses when compared to fetch-on-write policies.

===Write-Invalidate===
The combination of write-before-hit, no-fetch-on-write, and no-write-allocate is referred to as 'write-invalidate' because the line is invalidated on the miss. The copy that exists in lower level memory after the write miss differs from the one in the cache. For write hits, though, the data is simply written into the cache using the cache hit policy. Thus, for hits, the cache is not written around.

Of these three combinations, write validate performed the worse. The author notes, though, that it does perform better than fetch-on-write and is easy to implement. Tests by the author demonstrated a 35-50% reduction in write misses when compared to fetch-on-write policies.

===Fetch-on-write===
When used with write-allocate, fetch-on-write stores the block and processes the write to cache as expected. This policy was used as the base-line for determining the performance of the other three policy combinations. In general, all three previous combinations exhibited fewer misses than this policy.

=Prefetching in Contemporary Parallel Processors=
Prefetching is a technique that can be used in addition to write-hit and write-miss policies to improved the utilization of the cache. Microprocessors, such as those listed in Table 1, support prefetching logic directly in the chip.

Prefetching is the process of retrieving instructions or data from memory before the process explicitly requests it. Instruction prefetching is commonly used by single and multiprocessors to reduce process wait states.[[#References|[17]]] Prefetching of data may also be used, though, to pre-populate caches with data that is likely going to be required by the processors in the near term. If the data requirements are anticipated correctly, the requests to memory will result in a greater cache hit rates and, therefore, reduce overall memory access time. If the prefetcher guesses wrong, bus traffic can increase unnecessarily, more relevant data can be flushed from caches, and miss rates can increase.

Prefetching algorithms can leverage both temporal and spacial locality in making these decisions. For example, streaming and sequential access applications often process adjacent memory locations in subsequent tasks.

Following are two examples of processors that support prefetching directly in the chip design:

== Intel Core i7 [[#References|[6]]]==
The Intel 64 Architecture, including the Intel Core i7, includes both instruction and data prefetching directly on the chip.

The Data Cache Unit prefetcher is a streaming prefetcher for L1 caches that detects ascending access to data that has been loaded very recently. The processor assumes that this ascending access will continue, and prefetches the next line.

The data prefetch logic (DPL) maintains two arrays to track the recent accesses to memory: one for the upstreams that has 12 entries, and one for downstreams that has 4 entires. As pages are accessed, their addresses are tracked in these arrays. When the DPL detects an access to a page that is sequential to an existing entry, it assumes this sequential access will continue, and prefetches the next cache line from memory.

== AMD [[#References|[7]]]==

The AMD Family 10h processors use a stream-detection strategy similar to the Intel process described above to trigger prefetching of the next sequential memory location into the L1 cache. (Previous AMD processors fetched into the L2 cache, which introduced the access latency of the L2 cache and hindered performance.)

The AMD Family 10h can prefetch more than just the next sequential block when a stream is detected, though. When this 'unit-stride' prefetcher detects misses to sequential blocks, it can trigger a preset number of prefetch requests from memory. For example, if the preset value is two, the next two blocks of memory will be prefetched when a sequential access is detected. Subsequent detection of misses of sequential blocks may only prefetch a single block. This maintains the 'unit-stride' of two blocks in the cache ahead of the next anticipated read.

AMD contends that this is beneficial when processing large data sets which often process sequential data and process all the data in the stream. In these cases, a larger unit-stride populates the cache with blocks that will ultimately be processed by the CPU. AMD suggests that the optimal number of blocks to prefetch is between 4 and 8, and that fetching too many or fetching too soon has impaired performance in empirical testing.

The AMD Family 10h also includes 'Adaptive Prefetching', a hardware optimization that triggers prefetching when the demand stream catches up to the prefetch stream. In this case, the unit-stride is increased to assure that the prefetcher is fetching at a rate sufficient enough to continuously provide data to the processor.

= References =
[1]: http://www.techarp.com/showarticle.aspx?artno=337 
[2]: http://en.wikipedia.org/wiki/SPARC 
[3]: http://en.wikipedia.org/wiki/POWER7 
[4]: http://en.wikipedia.org/wiki/POWER5 
[5]: “Cache write policies and performance,” Norman Jouppi, Proc. 20th International Symposium on Computer Architecture (ACM Computer Architecture News 21:2), May 1993, pp. 191–201. http://www.hpl.hp.com/techreports/Compaq-DEC/WRL-91-12.pdf 
[6]: "Intel® 64 and IA-32 Architectures Optimization Reference Manual", Intel Corporation, 1997-2011. http://www.intel.com/Assets/PDF/manual/248966.pdf 
[7]: "Software Optimization Guide for AMD Family 10h and 12h Processors", Advanced Micro Devices, Inc., 2006–2010. http://support.amd.com/us/Processor_TechDocs/40546.pdf 
[8]: Yan Solihin, ''Fundamentals of Parallel Computer Architecture'' (Solihin Publishing and Consulting, LLC, 2008-2009), p. 151 
[9]: http://en.wikipedia.org/wiki/AMD_Phenom 
[10]: http://en.wikipedia.org/wiki/Phenom_II 
[11]: http://en.wikipedia.org/wiki/Athlon_II 
[12]: http://en.wikipedia.org/wiki/List_of_AMD_Athlon_64_microprocessors 
[13]: http://en.wikipedia.org/wiki/List_of_AMD_Athlon_X2_microprocessors 
[14]: http://en.wikipedia.org/wiki/List_of_Intel_Core_i5_microprocessors 
[15]: http://en.wikipedia.org/wiki/List_of_Intel_Core_i3_microprocessors 
[16]: http://www.intel.com/pressroom/kits/quickrefyr.htm 
[17]: http://en.wikipedia.org/wiki/Instruction_prefetch

CSC/ECE 506 Spring 2011/ch6a ep

2011-03-08T04:15:34Z

Eapotter: /* Prefetching in Contemporary Parallel Processors */

INTRODUCTION TO MEMORY HIERARCHY ORGANIZATION 
Write-Miss Policies and Prefetching

__TOC__

Write miss policies and prefetching are two strategies that are used by multiprocessors to achieve optimal performance for memory accesses. Write miss policies can help eliminate write misses and therefore reduce bus traffic. This reduces the wait-time of all processors sharing the bus. Prefetching assures that a CPU has data blocks in cache to be read when processing large data files and streaming data. When CPUs or cores share a cache, prefetched data in a shared cache is available to all processes on those cores processing the data.

This article begins by highlighting the variety of multicore processors on the market today that have hierarchal memory structures and shared caches. It then explores the write miss policies and prefetching techniques that these multiprocessors can use to take advantage of these architectures.

= Recent Architectures and their Cache Characteristics =

Cache management is impacted by three characteristics of modern processor architectures: multilevel cache hierarchies, multi-core and multiprocessor chip and systems designs, and shared vs. private caches.

Multi-level cache hierarchies are used by most contemporary chip designs to enable memory access to keep pace with CPU speeds that are advancing at a more rapid pace. Two-level and three-level cache hierarchies are common. L1 typically ranges from 16-64KB and provide access in 2-4 cycles. L2 typically ranges from 512KB to as much as 8MB and provides access in 6-15 cycles. L3 typically ranges from 4MB to 32MB and provides access to the data it contains within 50-30 cycles.[[#References|[8]]]

Multi-core and multprocessor designs place additional constraints on cache management by requiring them to consider all processes running concurrently in the cache coherency, replenishment, fetching, and other management functions.

In multiprocessors with multiple levels of cache, certain cache levels may be shared by processors and others private to each processor. Typically, lower-level caches are private and high-level caches may or may not be shared. Notice that none of the examples in the below have a shared L1 cache. Cache management functions must consider both shared and private caches when reading and writing data from memory.

Below is a table that illustrates the variety with which these characteristics have been combined in processors from four manufacturers over the past 6 years.

{| border="1" cellspacing="0" cellpadding="2"
!colspan="7"| Table 1: Recent Architectures and their Cache Characteristics [[#References|[1]]][[#References|[2]]][[#References|[3]]][[#References|[4]]][[#References|[9]]][[#References|[10]]][[#References|[11]]][[#References|[12]]][[#References|[13]]][[#References|[14]]][[#References|[15]]][[#References|[16]]]
|----
!Company
!Processor
!Cores
!L1 Cache
!L2 Cache
!L3 Cache
!Released
|----
|AMD
|Athlon 64 X2
|2
|128 KB x 2
|512 KB x 2 1MB x 2
|
|2005
|----
|AMD
|Athlon 64 FX
|2
|128 KB x 2
|1MB x 2
|
|2006
|----
|AMD
|Athlon X2
|2
|128 KB x 2
|512 KB x 2
|
|2007
|----
|AMD
|Phenom X3
|3
|128 KB x 3
|512 KB x 3
|2 MB
|2008
|----
|AMD
|Phenom X4
|4
|128 KB x 4
|512 KB x 4
|2 MB
|2007
|----
|AMD
|Athlon II X2
|2
|128 KB x 2
|512 KB x 2 1MB x 2
|
|2009
|----
|AMD
|Athlon II X3
|3
|128 KB x 3
|512 KB x 3
|
|2009
|----
|AMD
|Athlon II X4
|4
|128 KB x 4
|512 KB x 4
|
|2009
|----
|AMD
|Phenom II X2
|2
|128 KB x 2
|512 KB x 2
|6 MB
|2009
|----
|AMD
|Phenom II X3
|3
|128 KB x 3
|512 KB x 3
|6 MB
|2009
|----
|AMD
|Phenom II X4
|4
|128 KB x 4
|512 KB x 4
|4-6 MB
|2009
|----
|AMD
|Phenom II X6
|6
|128 KB x 6
|512 KB x 4
|6 MB
|2010
|----
|Intel
|Pentium D
|2
|12+16KB x 2
|2 MB
|
|2005
|----
|Intel
|Celeron E
|2
|32 KB x2
|512 -1 MB
|
|2006
|----
|Intel
|Pentium D
|2
|32 KB x2
|1 MB
|
|2006
|----
|Intel
|Core 2 Duo
|2
|32 KB x2
|2 - 4 MB
|
|2006
|----
|Intel
|Core 2 Quad
|4
|32 KB x2
|8 MB
|
|2007
|----
|Intel
|Atom 330
|2
|32+24KB x 2
|512 MB x 2
|
|2008
|----
|Intel
|Pentium E
|2
|32 KB x 2
|2 MB
|
|2007
|----
|Intel
|Core 2 Duo E
|2
|32 KB x 2
|3-6 MB
|
|2008
|----
|Intel
|Core 2 Quad
|4
|32 KB x 4
|2-6 MB x 2
|
|2008
|----
|Intel
|Core i3
|2
|32+32 KB x 2
|256 KB x2
|4 MB
|2010
|----
|Intel
|Core i5 - 6 Series
|2
|32+32 KB x 2
|256 KB x2
|4 MB
|2010
|----
|Intel
|Core i5 - 7 Series
|4
|32+32 KB x 4
|256 KB x 4
|8 MB
|2010
|----
|Intel
|Core i5 - 2400 Series Core i5 - 2500 Series
|4
|32+32 KB x 4
|256 KB x 4
|6 MB
|2010
|----
|Intel
|Core i7 - 8 Series
|4
|32+32 KB x 4
|256 KB x 4
|8 MB
|2010
|----
|Intel
|Core i7 - 9 Series
|4
|32+32 KB x 4
|256 KB x 4
|8 MB
|2010
|----
|Intel
|Core i7 - 970
|6
|32+32 KB x 6
|256 KB x 6
|12 MB
|2010
|----
|Sun
|UltraSPARC T1
|8
|8 K x 8 Inst. 16 K x 8 Data
|3 MB
|
|2005
|----
|Sun
|SPARC64 VI
|2
|128 K x 2 Inst. 128 K x 2 Data
|5 MB
|
|2007
|----
|Sun
|UltraSPARC T2
|8
|8 K x 4 Inst. 16 K x 4 Data
|4 MB
|
|2007
|----
|Sun
|SPARC64 VII
|4
|64 K x 4 Inst. 64 K x 4 Data
|6 MB
|
|2008
|----
|Sun
|SPARC T3
|8
|8 K x 8 Inst. 16 K x 8 Data
|6 MB
|
|2010
|----
|Sun
|SPARC64 VII+
|2
|128 K x 2 Inst. 128 K x 2 Data
|12 MB
|
|2010
|----
|IBM
|Power5
|2
|64 K x 2 Inst. 64 K x 2 Data
|4 MB x 2
|32 MB
|2004
|----
|IBM
|Power7
|4, 6, or 8
|32+32 KB x C
|256 kB x C
|4 - 32 MB x C
|2010
|----
|}

Note: The 'x #' in the L1 and L2 columns indicates that this cache is for each core.

=Cache Write Policies[[#References|[5]]]=
In section 6.2.3[[#References|[8]]], cache write hit policies and write miss policies were explored. The write hit policies, write-through and write-back, determine when the data written in the local cache is propagated to lower-level memory. As review, write-through writes data to the cache and memory on a write. Write-back writes to cache first and to memory only when a flush is required.

The write miss policies covered in the text[[#References|[8]]], write-allocate and no-write-allocate, determine if a memory block is stored in a cache line after the write occurs. Note that since a miss occurred on write, the block is not already in a cache line.

Write-miss policies can also be expressed in terms of write-allocate, fetch-on-write, and write-before-hit. These can be used in combination with the same write-through and write-back hit policies to define the complete cache write policy. Investigating alternative policies such as these is important since write-misses produce on average one-third of all cache misses.

The following discusses each of these policies:

==Fetch-on-Write==
On a write miss, the memory block containing the address to be written is fetched from the lower level memory hierarchy before the write proceeds. Note that this is different from write-allocate. Fetch-on-write determines if the block is fetched from memory, and write-allocate determines if the memory block is stored in a cache line. Certain write policies may allocate a line in cache for data that is written by the CPU without retrieving it from memory first.

==No-Fetch-on-Write==
On a write miss, the memory block is not fetched first from the lower level memory hierarchy. Therefore, the write can proceed with out having to wait for the memory block to be returned.

==Write-Before-Hit==
On a write, the write proceeds before the cache determines if a hit or miss occurred. In this scenario, the tag and the data can be written simultaneously, but it incurs an immediate bus transaction for each write by the processor.

==No-Write-Before-Hit==
On a write, the write waits until the cache determines if the block being written to is in the cache or not. This may avoid a bus transaction by allowing the processor to write to the cache multiple times before the cache line is flushed to memory.

==Write-Miss Policy Combinations==

In practice, these policies are used in combination to provide an over-all write policy. Four combinations of these three write miss policies are relevant, as illustrated in the table below:

<center>[[image:policies.jpg]]</center>

The two combinations of fetch-on-write and no-write-allocate are blocked out in the above diagram. They are considered unproductive since the data is fetched from lower level memory, but is not stored in the cache. Temporal and spacial locality suggest that this data will be used again, and will therefore need to be retrieved a second time.

Three combinations named write-validate, write-around, and write-invalidate all use no-fetch-on-write. They result in 'eliminated misses' when compared to a fetch-on-write policy. In general, this will yield better cache performance if the overhead to manage the policy remains low.

The following discusses each of these combinations:

===Write-Validate===
The combination of no-fetch-on-write and write-allocation is referred to as 'write-validate'. It writes the data into the cache line without fetching the corresponding block from memory first. The assumption is that the block will be written to memory at a later time. It requires additional overhead, or dirty bits, to track what bytes have been written into that cache line and which bytes were not written. Lower level memories also must be able can process only the changed portions of these lines. Otherwise, when the line is flushed to memory, the unwritten bytes may overwrite valid data.

The no-fetch-on-write has the advantage of avoiding an immediate bus transaction to retrieve the block from memory. For a multiprocessor with a cache coherency model, though, a bus transaction will be required to get exclusive ownership of the block. While this appears to negate the advantages of no-fetch-on-write, the process is able to continue other tasks while this transaction is occurring, including rewriting to the same cache location.

Tests by the author demonstrated more than a 90% reduction in write misses when compared to fetch-on-write policies.

===Write-Around===
The combination of no-fetch-on-write, no-write-allocate, and no-write-before-hit is referred to as a 'write-around'. It bypasses the higher level cache, writes directly to the lower level memory, and leaves the existing block line in the cache unaltered on a hit or a miss. This strategy shows performance improvements when the data that is written will not be reread in the near future. Since we are writing before a hit is detected, the cache is written around for both hits and misses.

The author notes that in only but a few cases write-around performs worse than write-validate policies. Most applications tend to reread what they have recently written. Using a write-around policy, this would result in a cache miss and a read from lower-level memory. With write-validate, the data would be in cache.

Tests by the author demonstrated a 40-70% reduction in write misses when compared to fetch-on-write policies.

===Write-Invalidate===
The combination of write-before-hit, no-fetch-on-write, and no-write-allocate is referred to as 'write-invalidate' because the line is invalidated on the miss. The copy that exists in lower level memory after the write miss differs from the one in the cache. For write hits, though, the data is simply written into the cache using the cache hit policy. Thus, for hits, the cache is not written around.

Of these three combinations, write validate performed the worse. The author notes, though, that it does perform better than fetch-on-write and is easy to implement. Tests by the author demonstrated a 35-50% reduction in write misses when compared to fetch-on-write policies.

===Fetch-on-write===
When used with write-allocate, fetch-on-write stores the block and processes the write to cache as expected. This policy was used as the base-line for determining the performance of the other three policy combinations. In general, all three previous combinations exhibited fewer misses than this policy.

=Prefetching in Contemporary Parallel Processors=
Prefetching is a technique that can be used in addition to write-hit and write-miss policies to improved the utilization of the cache. Microprocessors, such as those listed in Table 1, support prefetching logic directly in the chip.

Prefetching is the process of retrieving instructions or data from memory before the process explicitly requests it. Instruction prefetching is commonly used by single and multiprocessors to reduce process wait states.[[#References|[17]]] Prefetching of data may also be used, though, to pre-populate caches with data that is likely going to be required by the processors in the near term. If the data requirements are anticipated correctly, the requests to memory will result in a greater cache hit rates and, therefore, reduce overall memory access time. If the prefetcher guesses wrong, bus traffic can increase unnecessarily, more relevant data can be flushed from caches, and miss rates can increase.

Prefetching algorithms can leverage both temporal and spacial locality in making these decisions. For example, streaming and sequential access applications often process adjacent memory locations in subsequent tasks.

Following are two examples of processors that support prefetching directly in the chip design:

== Intel Core i7 [[#References|[6]]]==
The Intel 64 Architecture, including the Intel Core i7, includes both instruction and data prefetching directly on the chip.

The Data Cache Unit prefetcher is a streaming prefetcher for L1 caches that detects ascending access to data that has been loaded very recently. The processor assumes that this ascending access will continue, and prefetches the next line.

The data prefetch logic (DPL) maintains two arrays to track the recent accesses to memory: one for the upstreams that has 12 entries, and one for downstreams that has 4 entires. As pages are accessed, their addresses are tracked in these arrays. When the DPL detects an access to a page that is sequential to an existing entry, it assumes this sequential access will continue, and prefetches the next cache line from memory.

== AMD [[#References|[7]]]==

The AMD Family 10h processors use a stream-detection strategy similar to the Intel process described above to trigger prefetching of the next sequential memory location into the L1 cache. (Previous AMD processors fetched into the L2 cache, which introduced the access latency of the L2 cache and hindered performance.)

The AMD Family 10h can prefetch more than just the next sequential block when a stream is detected, though. When this 'unit-stride' prefetcher detects misses to sequential blocks, it can trigger a preset number of prefetch requests from memory. For example, if the preset value is two, the next two blocks of memory will be prefetched when a sequential access is detected. Subsequent detection of misses of sequential blocks may only prefetch a single block. This maintains the 'unit-stride' of two blocks in the cache ahead of the next anticipated read.

AMD contends that this is beneficial when processing large data sets which often process sequential data and process all the data in the stream. In these cases, a larger unit-stride populates the cache with blocks that will ultimately be processed by the CPU. AMD suggests that the optimal number of blocks to prefetch is between 4 and 8, and that fetching too many or fetching too soon has impaired performance in empirical testing.

The AMD Family 10h also includes 'Adaptive Prefetching', a hardware optimization that triggers prefetching when the demand stream catches up to the prefetch stream. In this case, the unit-stride is increased to assure that the prefetcher is fetching at a rate sufficient enough to continuously provide data to the processor.

= References =
[1]: http://www.techarp.com/showarticle.aspx?artno=337 
[2]: http://en.wikipedia.org/wiki/SPARC 
[3]: http://en.wikipedia.org/wiki/POWER7 
[4]: http://en.wikipedia.org/wiki/POWER5 
[5]: “Cache write policies and performance,” Norman Jouppi, Proc. 20th International Symposium on Computer Architecture (ACM Computer Architecture News 21:2), May 1993, pp. 191–201. http://www.hpl.hp.com/techreports/Compaq-DEC/WRL-91-12.pdf 
[6]: "Intel® 64 and IA-32 Architectures Optimization Reference Manual", Intel Corporation, 1997-2011. http://www.intel.com/Assets/PDF/manual/248966.pdf 
[7]: "Software Optimization Guide for AMD Family 10h and 12h Processors", Advanced Micro Devices, Inc., 2006–2010. http://support.amd.com/us/Processor_TechDocs/40546.pdf 
[8]: Yan Solihin, ''Fundamentals of Parallel Computer Architecture'' (Solihin Publishing and Consulting, LLC, 2008-2009), p. 151 
[9]: http://en.wikipedia.org/wiki/AMD_Phenom 
[10]: http://en.wikipedia.org/wiki/Phenom_II 
[11]: http://en.wikipedia.org/wiki/Athlon_II 
[12]: http://en.wikipedia.org/wiki/List_of_AMD_Athlon_64_microprocessors 
[13]: http://en.wikipedia.org/wiki/List_of_AMD_Athlon_X2_microprocessors 
[14]: http://en.wikipedia.org/wiki/List_of_Intel_Core_i5_microprocessors 
[15]: http://en.wikipedia.org/wiki/List_of_Intel_Core_i3_microprocessors 
[16]: http://www.intel.com/pressroom/kits/quickrefyr.htm 
[17]: http://en.wikipedia.org/wiki/Instruction_prefetch

CSC/ECE 506 Spring 2011/ch6a ep

2011-03-08T04:14:12Z

Eapotter: /* Cache Write Policies[5] */

INTRODUCTION TO MEMORY HIERARCHY ORGANIZATION 
Write-Miss Policies and Prefetching

__TOC__

Write miss policies and prefetching are two strategies that are used by multiprocessors to achieve optimal performance for memory accesses. Write miss policies can help eliminate write misses and therefore reduce bus traffic. This reduces the wait-time of all processors sharing the bus. Prefetching assures that a CPU has data blocks in cache to be read when processing large data files and streaming data. When CPUs or cores share a cache, prefetched data in a shared cache is available to all processes on those cores processing the data.

This article begins by highlighting the variety of multicore processors on the market today that have hierarchal memory structures and shared caches. It then explores the write miss policies and prefetching techniques that these multiprocessors can use to take advantage of these architectures.

= Recent Architectures and their Cache Characteristics =

Cache management is impacted by three characteristics of modern processor architectures: multilevel cache hierarchies, multi-core and multiprocessor chip and systems designs, and shared vs. private caches.

Multi-level cache hierarchies are used by most contemporary chip designs to enable memory access to keep pace with CPU speeds that are advancing at a more rapid pace. Two-level and three-level cache hierarchies are common. L1 typically ranges from 16-64KB and provide access in 2-4 cycles. L2 typically ranges from 512KB to as much as 8MB and provides access in 6-15 cycles. L3 typically ranges from 4MB to 32MB and provides access to the data it contains within 50-30 cycles.[[#References|[8]]]

Multi-core and multprocessor designs place additional constraints on cache management by requiring them to consider all processes running concurrently in the cache coherency, replenishment, fetching, and other management functions.

In multiprocessors with multiple levels of cache, certain cache levels may be shared by processors and others private to each processor. Typically, lower-level caches are private and high-level caches may or may not be shared. Notice that none of the examples in the below have a shared L1 cache. Cache management functions must consider both shared and private caches when reading and writing data from memory.

Below is a table that illustrates the variety with which these characteristics have been combined in processors from four manufacturers over the past 6 years.

{| border="1" cellspacing="0" cellpadding="2"
!colspan="7"| Table 1: Recent Architectures and their Cache Characteristics [[#References|[1]]][[#References|[2]]][[#References|[3]]][[#References|[4]]][[#References|[9]]][[#References|[10]]][[#References|[11]]][[#References|[12]]][[#References|[13]]][[#References|[14]]][[#References|[15]]][[#References|[16]]]
|----
!Company
!Processor
!Cores
!L1 Cache
!L2 Cache
!L3 Cache
!Released
|----
|AMD
|Athlon 64 X2
|2
|128 KB x 2
|512 KB x 2 1MB x 2
|
|2005
|----
|AMD
|Athlon 64 FX
|2
|128 KB x 2
|1MB x 2
|
|2006
|----
|AMD
|Athlon X2
|2
|128 KB x 2
|512 KB x 2
|
|2007
|----
|AMD
|Phenom X3
|3
|128 KB x 3
|512 KB x 3
|2 MB
|2008
|----
|AMD
|Phenom X4
|4
|128 KB x 4
|512 KB x 4
|2 MB
|2007
|----
|AMD
|Athlon II X2
|2
|128 KB x 2
|512 KB x 2 1MB x 2
|
|2009
|----
|AMD
|Athlon II X3
|3
|128 KB x 3
|512 KB x 3
|
|2009
|----
|AMD
|Athlon II X4
|4
|128 KB x 4
|512 KB x 4
|
|2009
|----
|AMD
|Phenom II X2
|2
|128 KB x 2
|512 KB x 2
|6 MB
|2009
|----
|AMD
|Phenom II X3
|3
|128 KB x 3
|512 KB x 3
|6 MB
|2009
|----
|AMD
|Phenom II X4
|4
|128 KB x 4
|512 KB x 4
|4-6 MB
|2009
|----
|AMD
|Phenom II X6
|6
|128 KB x 6
|512 KB x 4
|6 MB
|2010
|----
|Intel
|Pentium D
|2
|12+16KB x 2
|2 MB
|
|2005
|----
|Intel
|Celeron E
|2
|32 KB x2
|512 -1 MB
|
|2006
|----
|Intel
|Pentium D
|2
|32 KB x2
|1 MB
|
|2006
|----
|Intel
|Core 2 Duo
|2
|32 KB x2
|2 - 4 MB
|
|2006
|----
|Intel
|Core 2 Quad
|4
|32 KB x2
|8 MB
|
|2007
|----
|Intel
|Atom 330
|2
|32+24KB x 2
|512 MB x 2
|
|2008
|----
|Intel
|Pentium E
|2
|32 KB x 2
|2 MB
|
|2007
|----
|Intel
|Core 2 Duo E
|2
|32 KB x 2
|3-6 MB
|
|2008
|----
|Intel
|Core 2 Quad
|4
|32 KB x 4
|2-6 MB x 2
|
|2008
|----
|Intel
|Core i3
|2
|32+32 KB x 2
|256 KB x2
|4 MB
|2010
|----
|Intel
|Core i5 - 6 Series
|2
|32+32 KB x 2
|256 KB x2
|4 MB
|2010
|----
|Intel
|Core i5 - 7 Series
|4
|32+32 KB x 4
|256 KB x 4
|8 MB
|2010
|----
|Intel
|Core i5 - 2400 Series Core i5 - 2500 Series
|4
|32+32 KB x 4
|256 KB x 4
|6 MB
|2010
|----
|Intel
|Core i7 - 8 Series
|4
|32+32 KB x 4
|256 KB x 4
|8 MB
|2010
|----
|Intel
|Core i7 - 9 Series
|4
|32+32 KB x 4
|256 KB x 4
|8 MB
|2010
|----
|Intel
|Core i7 - 970
|6
|32+32 KB x 6
|256 KB x 6
|12 MB
|2010
|----
|Sun
|UltraSPARC T1
|8
|8 K x 8 Inst. 16 K x 8 Data
|3 MB
|
|2005
|----
|Sun
|SPARC64 VI
|2
|128 K x 2 Inst. 128 K x 2 Data
|5 MB
|
|2007
|----
|Sun
|UltraSPARC T2
|8
|8 K x 4 Inst. 16 K x 4 Data
|4 MB
|
|2007
|----
|Sun
|SPARC64 VII
|4
|64 K x 4 Inst. 64 K x 4 Data
|6 MB
|
|2008
|----
|Sun
|SPARC T3
|8
|8 K x 8 Inst. 16 K x 8 Data
|6 MB
|
|2010
|----
|Sun
|SPARC64 VII+
|2
|128 K x 2 Inst. 128 K x 2 Data
|12 MB
|
|2010
|----
|IBM
|Power5
|2
|64 K x 2 Inst. 64 K x 2 Data
|4 MB x 2
|32 MB
|2004
|----
|IBM
|Power7
|4, 6, or 8
|32+32 KB x C
|256 kB x C
|4 - 32 MB x C
|2010
|----
|}

Note: The 'x #' in the L1 and L2 columns indicates that this cache is for each core.

=Cache Write Policies[[#References|[5]]]=
In section 6.2.3[[#References|[8]]], cache write hit policies and write miss policies were explored. The write hit policies, write-through and write-back, determine when the data written in the local cache is propagated to lower-level memory. As review, write-through writes data to the cache and memory on a write. Write-back writes to cache first and to memory only when a flush is required.

The write miss policies covered in the text[[#References|[8]]], write-allocate and no-write-allocate, determine if a memory block is stored in a cache line after the write occurs. Note that since a miss occurred on write, the block is not already in a cache line.

Write-miss policies can also be expressed in terms of write-allocate, fetch-on-write, and write-before-hit. These can be used in combination with the same write-through and write-back hit policies to define the complete cache write policy. Investigating alternative policies such as these is important since write-misses produce on average one-third of all cache misses.

The following discusses each of these policies:

==Fetch-on-Write==
On a write miss, the memory block containing the address to be written is fetched from the lower level memory hierarchy before the write proceeds. Note that this is different from write-allocate. Fetch-on-write determines if the block is fetched from memory, and write-allocate determines if the memory block is stored in a cache line. Certain write policies may allocate a line in cache for data that is written by the CPU without retrieving it from memory first.

==No-Fetch-on-Write==
On a write miss, the memory block is not fetched first from the lower level memory hierarchy. Therefore, the write can proceed with out having to wait for the memory block to be returned.

==Write-Before-Hit==
On a write, the write proceeds before the cache determines if a hit or miss occurred. In this scenario, the tag and the data can be written simultaneously, but it incurs an immediate bus transaction for each write by the processor.

==No-Write-Before-Hit==
On a write, the write waits until the cache determines if the block being written to is in the cache or not. This may avoid a bus transaction by allowing the processor to write to the cache multiple times before the cache line is flushed to memory.

==Write-Miss Policy Combinations==

In practice, these policies are used in combination to provide an over-all write policy. Four combinations of these three write miss policies are relevant, as illustrated in the table below:

<center>[[image:policies.jpg]]</center>

The two combinations of fetch-on-write and no-write-allocate are blocked out in the above diagram. They are considered unproductive since the data is fetched from lower level memory, but is not stored in the cache. Temporal and spacial locality suggest that this data will be used again, and will therefore need to be retrieved a second time.

Three combinations named write-validate, write-around, and write-invalidate all use no-fetch-on-write. They result in 'eliminated misses' when compared to a fetch-on-write policy. In general, this will yield better cache performance if the overhead to manage the policy remains low.

The following discusses each of these combinations:

===Write-Validate===
The combination of no-fetch-on-write and write-allocation is referred to as 'write-validate'. It writes the data into the cache line without fetching the corresponding block from memory first. The assumption is that the block will be written to memory at a later time. It requires additional overhead, or dirty bits, to track what bytes have been written into that cache line and which bytes were not written. Lower level memories also must be able can process only the changed portions of these lines. Otherwise, when the line is flushed to memory, the unwritten bytes may overwrite valid data.

The no-fetch-on-write has the advantage of avoiding an immediate bus transaction to retrieve the block from memory. For a multiprocessor with a cache coherency model, though, a bus transaction will be required to get exclusive ownership of the block. While this appears to negate the advantages of no-fetch-on-write, the process is able to continue other tasks while this transaction is occurring, including rewriting to the same cache location.

Tests by the author demonstrated more than a 90% reduction in write misses when compared to fetch-on-write policies.

===Write-Around===
The combination of no-fetch-on-write, no-write-allocate, and no-write-before-hit is referred to as a 'write-around'. It bypasses the higher level cache, writes directly to the lower level memory, and leaves the existing block line in the cache unaltered on a hit or a miss. This strategy shows performance improvements when the data that is written will not be reread in the near future. Since we are writing before a hit is detected, the cache is written around for both hits and misses.

The author notes that in only but a few cases write-around performs worse than write-validate policies. Most applications tend to reread what they have recently written. Using a write-around policy, this would result in a cache miss and a read from lower-level memory. With write-validate, the data would be in cache.

Tests by the author demonstrated a 40-70% reduction in write misses when compared to fetch-on-write policies.

===Write-Invalidate===
The combination of write-before-hit, no-fetch-on-write, and no-write-allocate is referred to as 'write-invalidate' because the line is invalidated on the miss. The copy that exists in lower level memory after the write miss differs from the one in the cache. For write hits, though, the data is simply written into the cache using the cache hit policy. Thus, for hits, the cache is not written around.

Of these three combinations, write validate performed the worse. The author notes, though, that it does perform better than fetch-on-write and is easy to implement. Tests by the author demonstrated a 35-50% reduction in write misses when compared to fetch-on-write policies.

===Fetch-on-write===
When used with write-allocate, fetch-on-write stores the block and processes the write to cache as expected. This policy was used as the base-line for determining the performance of the other three policy combinations. In general, all three previous combinations exhibited fewer misses than this policy.

=Prefetching in Contemporary Parallel Processors=
Prefetching is a technique that can be used in addition to write-hit and write-miss policies to improved the utilization of the cache. Microprocessors, such as those listed in Table 1, support prefetching logic directly in the chip.

Prefetching is the process of retrieving instructions or data from memory before the process explicitly requests it. Instruction prefetching is commonly used by single and multiprocessors to reduce process wait states.[[#References|[17]]] Prefetching of data may also be used, though, to pre-populate caches with data that is likely going to be required by the processors in the near term. If the data requirements are anticipated correctly, the requests to memory will result in a greater percentage of cache hit rates and, therefore, reduce overall memory access time. If the prefetcher guesses wrong, bus traffic can increase unnecessarily, more relevant data can be flushed from caches, and miss rates can increase.

Prefetching algorithms can leverage both temporal and spacial locality in making these decisions. For example, streaming and sequential access applications often process adjacent memory locations in subsequent tasks.

Following are two examples of processors that support prefetching directly in the chip design:

== Intel Core i7 [[#References|[6]]]==
The Intel 64 Architecture, including the Intel Core i7, includes both instruction and data prefetching directly on the chip.

The Data Cache Unit prefetcher is a streaming prefetcher for L1 caches that detects ascending access to data that has been loaded very recently. The processor assumes that this ascending access will continue, and prefetches the next line.

The data prefetch logic (DPL) maintains two arrays to track the recent accesses to memory: one for the upstreams that has 12 entries, and one for downstreams that has 4 entires. As pages are accessed, their addresses are tracked in these arrays. When the DPL detects an access to a page that is sequential to an existing entry, it assumes this sequential access will continue, and prefetches the next cache line from memory.

== AMD [[#References|[7]]]==

The AMD Family 10h processors use a stream-detection strategy similar to the Intel process described above to trigger prefetching of the next sequential memory location into the L1 cache. (Previous AMD processors fetched into the L2 cache, which introduced the access latency of the L2 cache and hindered performance.)

The AMD Family 10h can prefetch more than just the next sequential block when a stream is detected, though. When this 'unit-stride' prefetcher detects misses to sequential blocks, it can trigger a preset number of prefetch requests from memory. For example, if the preset value is two, the next two blocks of memory will be prefetched when a sequential access is detected. Subsequent detection of misses of sequential blocks may only prefetch a single block. This maintains the 'unit-stride' of two blocks in the cache ahead of the next anticipated read.

AMD contends that this is beneficial when processing large data sets which often process sequential data and process all the data in the stream. In these cases, a larger unit-stride populates the cache with blocks that will ultimately be processed by the CPU. AMD suggests that the optimal number of blocks to prefetch is between 4 and 8, and that fetching too many or fetching too soon has impaired performance in empirical testing.

The AMD Family 10h also includes 'Adaptive Prefetching', a hardware optimization that triggers prefetching when the demand stream catches up to the prefetch stream. In this case, the unit-stride is increased to assure that the prefetcher is fetching at a rate sufficient enough to continuously provide data to the processor.

= References =
[1]: http://www.techarp.com/showarticle.aspx?artno=337 
[2]: http://en.wikipedia.org/wiki/SPARC 
[3]: http://en.wikipedia.org/wiki/POWER7 
[4]: http://en.wikipedia.org/wiki/POWER5 
[5]: “Cache write policies and performance,” Norman Jouppi, Proc. 20th International Symposium on Computer Architecture (ACM Computer Architecture News 21:2), May 1993, pp. 191–201. http://www.hpl.hp.com/techreports/Compaq-DEC/WRL-91-12.pdf 
[6]: "Intel® 64 and IA-32 Architectures Optimization Reference Manual", Intel Corporation, 1997-2011. http://www.intel.com/Assets/PDF/manual/248966.pdf 
[7]: "Software Optimization Guide for AMD Family 10h and 12h Processors", Advanced Micro Devices, Inc., 2006–2010. http://support.amd.com/us/Processor_TechDocs/40546.pdf 
[8]: Yan Solihin, ''Fundamentals of Parallel Computer Architecture'' (Solihin Publishing and Consulting, LLC, 2008-2009), p. 151 
[9]: http://en.wikipedia.org/wiki/AMD_Phenom 
[10]: http://en.wikipedia.org/wiki/Phenom_II 
[11]: http://en.wikipedia.org/wiki/Athlon_II 
[12]: http://en.wikipedia.org/wiki/List_of_AMD_Athlon_64_microprocessors 
[13]: http://en.wikipedia.org/wiki/List_of_AMD_Athlon_X2_microprocessors 
[14]: http://en.wikipedia.org/wiki/List_of_Intel_Core_i5_microprocessors 
[15]: http://en.wikipedia.org/wiki/List_of_Intel_Core_i3_microprocessors 
[16]: http://www.intel.com/pressroom/kits/quickrefyr.htm 
[17]: http://en.wikipedia.org/wiki/Instruction_prefetch

CSC/ECE 506 Spring 2011/ch6a ep

2011-03-08T04:10:40Z

Eapotter: /* Write-Invalidate */

INTRODUCTION TO MEMORY HIERARCHY ORGANIZATION 
Write-Miss Policies and Prefetching

__TOC__

Write miss policies and prefetching are two strategies that are used by multiprocessors to achieve optimal performance for memory accesses. Write miss policies can help eliminate write misses and therefore reduce bus traffic. This reduces the wait-time of all processors sharing the bus. Prefetching assures that a CPU has data blocks in cache to be read when processing large data files and streaming data. When CPUs or cores share a cache, prefetched data in a shared cache is available to all processes on those cores processing the data.

This article begins by highlighting the variety of multicore processors on the market today that have hierarchal memory structures and shared caches. It then explores the write miss policies and prefetching techniques that these multiprocessors can use to take advantage of these architectures.

= Recent Architectures and their Cache Characteristics =

Cache management is impacted by three characteristics of modern processor architectures: multilevel cache hierarchies, multi-core and multiprocessor chip and systems designs, and shared vs. private caches.

Multi-level cache hierarchies are used by most contemporary chip designs to enable memory access to keep pace with CPU speeds that are advancing at a more rapid pace. Two-level and three-level cache hierarchies are common. L1 typically ranges from 16-64KB and provide access in 2-4 cycles. L2 typically ranges from 512KB to as much as 8MB and provides access in 6-15 cycles. L3 typically ranges from 4MB to 32MB and provides access to the data it contains within 50-30 cycles.[[#References|[8]]]

Multi-core and multprocessor designs place additional constraints on cache management by requiring them to consider all processes running concurrently in the cache coherency, replenishment, fetching, and other management functions.

In multiprocessors with multiple levels of cache, certain cache levels may be shared by processors and others private to each processor. Typically, lower-level caches are private and high-level caches may or may not be shared. Notice that none of the examples in the below have a shared L1 cache. Cache management functions must consider both shared and private caches when reading and writing data from memory.

Below is a table that illustrates the variety with which these characteristics have been combined in processors from four manufacturers over the past 6 years.

{| border="1" cellspacing="0" cellpadding="2"
!colspan="7"| Table 1: Recent Architectures and their Cache Characteristics [[#References|[1]]][[#References|[2]]][[#References|[3]]][[#References|[4]]][[#References|[9]]][[#References|[10]]][[#References|[11]]][[#References|[12]]][[#References|[13]]][[#References|[14]]][[#References|[15]]][[#References|[16]]]
|----
!Company
!Processor
!Cores
!L1 Cache
!L2 Cache
!L3 Cache
!Released
|----
|AMD
|Athlon 64 X2
|2
|128 KB x 2
|512 KB x 2 1MB x 2
|
|2005
|----
|AMD
|Athlon 64 FX
|2
|128 KB x 2
|1MB x 2
|
|2006
|----
|AMD
|Athlon X2
|2
|128 KB x 2
|512 KB x 2
|
|2007
|----
|AMD
|Phenom X3
|3
|128 KB x 3
|512 KB x 3
|2 MB
|2008
|----
|AMD
|Phenom X4
|4
|128 KB x 4
|512 KB x 4
|2 MB
|2007
|----
|AMD
|Athlon II X2
|2
|128 KB x 2
|512 KB x 2 1MB x 2
|
|2009
|----
|AMD
|Athlon II X3
|3
|128 KB x 3
|512 KB x 3
|
|2009
|----
|AMD
|Athlon II X4
|4
|128 KB x 4
|512 KB x 4
|
|2009
|----
|AMD
|Phenom II X2
|2
|128 KB x 2
|512 KB x 2
|6 MB
|2009
|----
|AMD
|Phenom II X3
|3
|128 KB x 3
|512 KB x 3
|6 MB
|2009
|----
|AMD
|Phenom II X4
|4
|128 KB x 4
|512 KB x 4
|4-6 MB
|2009
|----
|AMD
|Phenom II X6
|6
|128 KB x 6
|512 KB x 4
|6 MB
|2010
|----
|Intel
|Pentium D
|2
|12+16KB x 2
|2 MB
|
|2005
|----
|Intel
|Celeron E
|2
|32 KB x2
|512 -1 MB
|
|2006
|----
|Intel
|Pentium D
|2
|32 KB x2
|1 MB
|
|2006
|----
|Intel
|Core 2 Duo
|2
|32 KB x2
|2 - 4 MB
|
|2006
|----
|Intel
|Core 2 Quad
|4
|32 KB x2
|8 MB
|
|2007
|----
|Intel
|Atom 330
|2
|32+24KB x 2
|512 MB x 2
|
|2008
|----
|Intel
|Pentium E
|2
|32 KB x 2
|2 MB
|
|2007
|----
|Intel
|Core 2 Duo E
|2
|32 KB x 2
|3-6 MB
|
|2008
|----
|Intel
|Core 2 Quad
|4
|32 KB x 4
|2-6 MB x 2
|
|2008
|----
|Intel
|Core i3
|2
|32+32 KB x 2
|256 KB x2
|4 MB
|2010
|----
|Intel
|Core i5 - 6 Series
|2
|32+32 KB x 2
|256 KB x2
|4 MB
|2010
|----
|Intel
|Core i5 - 7 Series
|4
|32+32 KB x 4
|256 KB x 4
|8 MB
|2010
|----
|Intel
|Core i5 - 2400 Series Core i5 - 2500 Series
|4
|32+32 KB x 4
|256 KB x 4
|6 MB
|2010
|----
|Intel
|Core i7 - 8 Series
|4
|32+32 KB x 4
|256 KB x 4
|8 MB
|2010
|----
|Intel
|Core i7 - 9 Series
|4
|32+32 KB x 4
|256 KB x 4
|8 MB
|2010
|----
|Intel
|Core i7 - 970
|6
|32+32 KB x 6
|256 KB x 6
|12 MB
|2010
|----
|Sun
|UltraSPARC T1
|8
|8 K x 8 Inst. 16 K x 8 Data
|3 MB
|
|2005
|----
|Sun
|SPARC64 VI
|2
|128 K x 2 Inst. 128 K x 2 Data
|5 MB
|
|2007
|----
|Sun
|UltraSPARC T2
|8
|8 K x 4 Inst. 16 K x 4 Data
|4 MB
|
|2007
|----
|Sun
|SPARC64 VII
|4
|64 K x 4 Inst. 64 K x 4 Data
|6 MB
|
|2008
|----
|Sun
|SPARC T3
|8
|8 K x 8 Inst. 16 K x 8 Data
|6 MB
|
|2010
|----
|Sun
|SPARC64 VII+
|2
|128 K x 2 Inst. 128 K x 2 Data
|12 MB
|
|2010
|----
|IBM
|Power5
|2
|64 K x 2 Inst. 64 K x 2 Data
|4 MB x 2
|32 MB
|2004
|----
|IBM
|Power7
|4, 6, or 8
|32+32 KB x C
|256 kB x C
|4 - 32 MB x C
|2010
|----
|}

Note: The 'x #' in the L1 and L2 columns indicates that this cache is for each core.

=Cache Write Policies[[#References|[5]]]=
In section 6.2.3[[#References|[8]]], cache write hit policies and write miss policies were explored. The write hit policies, write-through and write-back, determine when the data written in the local cache is propagated to lower-level memory. As review, write-through writes data to the cache and memory on a write. Write-back writes to cache first and to memory only when a flush is required.

The write miss policies covered in the text[[#References|[8]]] write-allocate and no-write-allocate, determine if a memory block is stored in a cache line after the write occurs. Note that since a miss occurred on write, the block is not already in a cache line.

Write-miss policies can also be expressed in terms of write-allocate, fetch-on-write, and write-before-hit. These can be used in combination with the same write-through and write-back hit policies to define the complete cache write policy. Investigating alternative policies such as these is important since write-misses produce on average one-third of all cache misses.

The following discusses each of these policies:

==Fetch-on-Write==
On a write miss, the memory block containing the address to be written is fetched from the lower level memory hierarchy before the write proceeds. Note that this is different from write-allocate. Fetch-on-write determines if the block is fetched from memory, and write-allocate determines if the memory block is stored in a cache line. Certain write policies may allocate a line in cache for data that is written by the CPU without retrieving it from memory first.

==No-Fetch-on-Write==
On a write miss, the memory block is not fetched first from the lower level memory hierarchy. Therefore, the write can proceed with out having to wait for the memory block to be returned.

==Write-Before-Hit==
On a write, the write proceeds before the cache determines if a hit or miss occurred. In this scenario, the tag and the data can be written simultaneously, but it incurs an immediate bus transaction for each write by the processor.

==No-Write-Before-Hit==
On a write, the write waits until the cache determines if the block being written to is in the cache or not. This may avoid a bus transaction by allowing the processor to write to the cache multiple times before the cache line is flushed to memory.

==Write-Miss Policy Combinations==

In practice, these policies are used in combination to provide an over-all write policy. Four combinations of these three write miss policies are relevant, as illustrated in the table below:

<center>[[image:policies.jpg]]</center>

The two combinations of fetch-on-write and no-write-allocate are blocked out in the above diagram. They are considered unproductive since the data is fetched from lower level memory, but is not stored in the cache. Temporal and spacial locality suggest that this data will be used again, and will therefore need to be retrieved a second time.

Three combinations named write-validate, write-around, and write-invalidate all use no-fetch-on-write. They result in 'eliminated misses' when compared to a fetch-on-write policy. In general, this will yield better cache performance if the overhead to manage the policy remains low.

The following discusses each of these combinations:

===Write-Validate===
The combination of no-fetch-on-write and write-allocation is referred to as 'write-validate'. It writes the data into the cache line without fetching the corresponding block from memory first. The assumption is that the block will be written to memory at a later time. It requires additional overhead, or dirty bits, to track what bytes have been written into that cache line and which bytes were not written. Lower level memories also must be able can process only the changed portions of these lines. Otherwise, when the line is flushed to memory, the unwritten bytes may overwrite valid data.

The no-fetch-on-write has the advantage of avoiding an immediate bus transaction to retrieve the block from memory. For a multiprocessor with a cache coherency model, though, a bus transaction will be required to get exclusive ownership of the block. While this appears to negate the advantages of no-fetch-on-write, the process is able to continue other tasks while this transaction is occurring, including rewriting to the same cache location.

Tests by the author demonstrated more than a 90% reduction in write misses when compared to fetch-on-write policies.

===Write-Around===
The combination of no-fetch-on-write, no-write-allocate, and no-write-before-hit is referred to as a 'write-around'. It bypasses the higher level cache, writes directly to the lower level memory, and leaves the existing block line in the cache unaltered on a hit or a miss. This strategy shows performance improvements when the data that is written will not be reread in the near future. Since we are writing before a hit is detected, the cache is written around for both hits and misses.

The author notes that in only but a few cases write-around performs worse than write-validate policies. Most applications tend to reread what they have recently written. Using a write-around policy, this would result in a cache miss and a read from lower-level memory. With write-validate, the data would be in cache.

Tests by the author demonstrated a 40-70% reduction in write misses when compared to fetch-on-write policies.

===Write-Invalidate===
The combination of write-before-hit, no-fetch-on-write, and no-write-allocate is referred to as 'write-invalidate' because the line is invalidated on the miss. The copy that exists in lower level memory after the write miss differs from the one in the cache. For write hits, though, the data is simply written into the cache using the cache hit policy. Thus, for hits, the cache is not written around.

Of these three combinations, write validate performed the worse. The author notes, though, that it does perform better than fetch-on-write and is easy to implement. Tests by the author demonstrated a 35-50% reduction in write misses when compared to fetch-on-write policies.

===Fetch-on-write===
When used with write-allocate, fetch-on-write stores the block and processes the write to cache as expected. This policy was used as the base-line for determining the performance of the other three policy combinations. In general, all three previous combinations exhibited fewer misses than this policy.

=Prefetching in Contemporary Parallel Processors=
Prefetching is a technique that can be used in addition to write-hit and write-miss policies to improved the utilization of the cache. Microprocessors, such as those listed in Table 1, support prefetching logic directly in the chip.

Prefetching is the process of retrieving instructions or data from memory before the process explicitly requests it. Instruction prefetching is commonly used by single and multiprocessors to reduce process wait states.[[#References|[17]]] Prefetching of data may also be used, though, to pre-populate caches with data that is likely going to be required by the processors in the near term. If the data requirements are anticipated correctly, the requests to memory will result in a greater percentage of cache hit rates and, therefore, reduce overall memory access time. If the prefetcher guesses wrong, bus traffic can increase unnecessarily, more relevant data can be flushed from caches, and miss rates can increase.

Prefetching algorithms can leverage both temporal and spacial locality in making these decisions. For example, streaming and sequential access applications often process adjacent memory locations in subsequent tasks.

Following are two examples of processors that support prefetching directly in the chip design:

== Intel Core i7 [[#References|[6]]]==
The Intel 64 Architecture, including the Intel Core i7, includes both instruction and data prefetching directly on the chip.

The Data Cache Unit prefetcher is a streaming prefetcher for L1 caches that detects ascending access to data that has been loaded very recently. The processor assumes that this ascending access will continue, and prefetches the next line.

The data prefetch logic (DPL) maintains two arrays to track the recent accesses to memory: one for the upstreams that has 12 entries, and one for downstreams that has 4 entires. As pages are accessed, their addresses are tracked in these arrays. When the DPL detects an access to a page that is sequential to an existing entry, it assumes this sequential access will continue, and prefetches the next cache line from memory.

== AMD [[#References|[7]]]==

The AMD Family 10h processors use a stream-detection strategy similar to the Intel process described above to trigger prefetching of the next sequential memory location into the L1 cache. (Previous AMD processors fetched into the L2 cache, which introduced the access latency of the L2 cache and hindered performance.)

The AMD Family 10h can prefetch more than just the next sequential block when a stream is detected, though. When this 'unit-stride' prefetcher detects misses to sequential blocks, it can trigger a preset number of prefetch requests from memory. For example, if the preset value is two, the next two blocks of memory will be prefetched when a sequential access is detected. Subsequent detection of misses of sequential blocks may only prefetch a single block. This maintains the 'unit-stride' of two blocks in the cache ahead of the next anticipated read.

AMD contends that this is beneficial when processing large data sets which often process sequential data and process all the data in the stream. In these cases, a larger unit-stride populates the cache with blocks that will ultimately be processed by the CPU. AMD suggests that the optimal number of blocks to prefetch is between 4 and 8, and that fetching too many or fetching too soon has impaired performance in empirical testing.

The AMD Family 10h also includes 'Adaptive Prefetching', a hardware optimization that triggers prefetching when the demand stream catches up to the prefetch stream. In this case, the unit-stride is increased to assure that the prefetcher is fetching at a rate sufficient enough to continuously provide data to the processor.

= References =
[1]: http://www.techarp.com/showarticle.aspx?artno=337 
[2]: http://en.wikipedia.org/wiki/SPARC 
[3]: http://en.wikipedia.org/wiki/POWER7 
[4]: http://en.wikipedia.org/wiki/POWER5 
[5]: “Cache write policies and performance,” Norman Jouppi, Proc. 20th International Symposium on Computer Architecture (ACM Computer Architecture News 21:2), May 1993, pp. 191–201. http://www.hpl.hp.com/techreports/Compaq-DEC/WRL-91-12.pdf 
[6]: "Intel® 64 and IA-32 Architectures Optimization Reference Manual", Intel Corporation, 1997-2011. http://www.intel.com/Assets/PDF/manual/248966.pdf 
[7]: "Software Optimization Guide for AMD Family 10h and 12h Processors", Advanced Micro Devices, Inc., 2006–2010. http://support.amd.com/us/Processor_TechDocs/40546.pdf 
[8]: Yan Solihin, ''Fundamentals of Parallel Computer Architecture'' (Solihin Publishing and Consulting, LLC, 2008-2009), p. 151 
[9]: http://en.wikipedia.org/wiki/AMD_Phenom 
[10]: http://en.wikipedia.org/wiki/Phenom_II 
[11]: http://en.wikipedia.org/wiki/Athlon_II 
[12]: http://en.wikipedia.org/wiki/List_of_AMD_Athlon_64_microprocessors 
[13]: http://en.wikipedia.org/wiki/List_of_AMD_Athlon_X2_microprocessors 
[14]: http://en.wikipedia.org/wiki/List_of_Intel_Core_i5_microprocessors 
[15]: http://en.wikipedia.org/wiki/List_of_Intel_Core_i3_microprocessors 
[16]: http://www.intel.com/pressroom/kits/quickrefyr.htm 
[17]: http://en.wikipedia.org/wiki/Instruction_prefetch