Expertiza_Wiki - User contributions [en]

CSC/ECE 506 Fall 2007/wiki3 2 1r

2007-10-20T03:39:53Z

Arangan: /* External links */

''Wiki: SCI. The IEEE Scalable Coherent Interface is a superset of the SSCI protocol we have been considering in class. A lot has been written about it, but it is still difficult to comprehend. Using SSCI as a starting point, explain why additional states are necessary, and give (or cite) examples that demonstrate how they work. Ideally, this would still be an overview of the working of the protocol, referencing more detailed documentation on the Web.''

__TOC__

= Introduction =

In parallel systems, data can be obtained from a variety of sources, which includes local or remote memory modules, secondary storage devices and network interfaces. It would be beneficial if certain aspects of mechanisms that are required to gain access to data are standardized such that it allows easily expandable and flexible designs. Various portions of a parallel system may be modified or even upgraded without any effect on the rest of the design if the system adheres to such standards. Also, utilization of standard interfaces forwards the integration of components and subsystems from several different vendors into efficient and usable parallel systems. The Scalable Coherent Interface is one such standard which facilitates the implementation of large-scale cache-coherent parallel systems.

= Scalable Coherent Interface (SCI) =

The scalable coherent interface has now become a chief hardware based approach to the cache coherence problem in shared memory multiprocessors. SCI is a directory based invalidate coherence protocol. The state of a cache block is distributed to the sharers of that block. Limits that are inherent in bus technology are easily avoided by SCI. The SCI protocol is to provide scalability, coherence and an interface. Scalability is to guarantee that the same mechanisms can be used in single processor systems and large highly parallel multiprocessors. Coherence is to guarantee efficient and integral use of cache memories in distributed shared memory. An interface provides a communication architecture that has multiple values to be brought into a single system and provide smooth inter-operation.

===Goals of SCI===

*High Performance – For distributed or parallel applications, high communication performance must be obtained when using SCI. Usual aspects of such performance would be a low latency, low CPU overhead for operations dealing with communication and high sustained throughput.

*Scalability – This is addressed in many ways. Some are
**Performance scalability – as the number of nodes being added to the system increases.
**Interconnect Distance scalability
**Memory System scalability – chiefly of cache coherence protocols, which should not have limitations to the number of processors or modules in memory that it can handle.
**Technological scalability – utilization of similar mechanisms in large, small-scale, loosely-coupled or tightly-coupled systems. Also, when new advances are made in technology, it must have the ability to readily make use of such advances.
**Economic scalability – use of similar components and mechanisms in high-end, low-volume systems as well as low-end, high-volume systems.
**Addressing capability has no short term practical limits.

*Coherent Memory System – For the purpose of reduction of Average Access Time to data, caches are becoming more important each day for microprocessors.

*Interface Characteristics – This specifies a standard interface to an interconnect that makes several devices to be connected or attached together and to interoperate.

===Operation===

In SCI, every interface does not wait for the signal to propagate before it begins to send the next signal. Also, SCI utilizes multiple links so that, concurrently, several transfers can take place.

Usually, a directory entry is in either of the two states : ‘home’ or ‘gone’. If the state is in ‘home’, then memory can immediately satisfy requests to a block as the block has not been cached by any processor. If the state is ‘gone’, then the block has been cached by a processor and might even be modified. Now, the directory contains a pointer to the first processor in the sharing list for this particular block. Hence, on requesting the data, memory returns the pointer to the first processor on the shared list rather than the data itself. The processor asking for the block now forwards its request to the processor on the top of the shared list. Now, the requesting processor adds itself into the shared list as the new head of the list.

In the SCI protocol, any coherent transaction has three phases.

Memory read – When a processor misses in its cache, It asks for the block in the home directory in memory. If the state of the memory is ‘home’, then the main memory replies with the block to the requesting processor. If the state is ‘gone’, then the main memory returns the head of the shared list of processors for that particular block. Then, memory updates its pointer and puts the requesting processor as the new head.

Cache read – When memory returns a pointer toe the requesting processor instead of data, the processor forwards its request for the block to the head of the doubly linked list. When the cache receives, the head of the list returns the data which might have been modified. The head of the list changes its backward pointer to the requesting processor’s node. Now, the requesting processor becomes the head of the list and it has the cache block.

Cleanup – If the cache miss from the requesting processor is a store, the processor has to first invalidate all other cached copies and then only proceed with the store. The new head gives an invalidate request to the next address on the list i.e. to its next processor. This processor invalidates and gives back a pointer to the next processor on its list. The head of the list uses this new pointer and sends it an invalidate request. This goes on until a NULL pointer is returned. This is the cleanup process for invalidation of other cache copies of the block.

===Applications===

1. SCI-based CC-NUMA machines

SCI is attached to the memory bus of a node. Here, SCI can take part in and "export" cache coherence and memory traffic on the bus and make the memory of the node visible to other nodes. Each node's memory address range can be scaled to span a global address space, giving coherent and transparent access to anywhere in the system to requesting processors. This method is used to form a large-scale cache-coherent shared-memory system, known as a CC-NUMA machine.

2. SGI Origin 2000

This would seem to be the only commercial alternative to CC-Numa machines. The Origin 2000 is an advanced version of the SCI-based CC-NUMA machines but has its origins in the Stanford DASH project. Cache coherence is maintained here with a bit-vector directory scheme.

3. S3.mp

This was a project by SUN to connect off-the-shelf workstations and make this interconnection into a large-scale cache-coherent multiprocessor. This has an invalidation based protocol which notes the nodes that share a memory block and stores such information in linked lists. The nodes in this project were spatially distributed and were connected together using arbitrary topologies.

= Simple Scalable Coherent Interface (SSCI) =

In directory-based approach, Every memory block has associated directory information; it keeps track of copies of cached blocks and their states. On a miss, it finds the directory entry, looks it up, and communicates only with the nodes that have copies (if necessary).

There are mainly two approaches: Full-bit vector: For k processors, it maintains k presence bit and 1 dirty bit at the home node. Cache state is represented the same way as in bus-based designs (MSI, MESI, etc.). It has three cache states: EM (exclusive or modified), S (shared), U (unowned). Limitation is - Number of presence bits needed grows as the number of processors.

Memory-based schemes store the information about all cached copies at the home node of the block. Cache-based schemes distribute information about copies among the copies themselves. The home contains a pointer to one cached copy of the block. Each copy contains the identity of the next node that has a copy of the block. The location of the copies is therefore determined through network transactions.

Simple SCI (SSCI) retains similarity with full-bit vector protocol: MESI states in the cache; U, S, EM states in the memory directory; It replaces the presence bits with a pointer.

== Why additional states are necessary? ==

Performance & Correctness (Coherence & Consistency) requirement for the protocol necessitates the additional states in the protocol.

=== Need for Busy state ===

On a scalable multiprocessors without coherent caches, the main-memory module determined the ordering of writes. The order that writes become visible to all processors is the order in which they reached memory.

e.g. If two processors issue read-exclusive requests for a particular word, the home will provide the requestors with the location of
the dirty node. But which request will reach the dirty node first cannot be guaranteed. This creates the need for additional busy state in the directory.

This can be solved by
# holding requests at home or requestor node and serve them in the order of their arrival
# If the block is busy, reject any further request to it & that request will be retried later
# If directory is busy forward request to dirty node & dirty node will serialize the request execution

In non-coherent scalable multiprocessors,
# For write atomicity in invalidation based protocol, current owner of block has to wait until it recieve all invalidation acks and then only it can read/write new value.
# For write completion, current owner of the block need to wait for ack from memory

In Origin, there are 3 busy states indicates that the directory has received the request but is waiting for completion hence another request for the same block won't be accepted.
# Busy read
# Busy read exclusive or upgrade
# Busy uncached read: In case of DMA, if the memory block is being read, no other processor should be allowed to get write access to it.

=== Poison state ===

To improve locality, origin relies on page migration. Hardware keeps reference counts on each page. On every access of memory count is incremented and compared with home node. If count is more than programmable threshold, hardware interrupts one of the local processors to copy the page. Block transfer engine “poison” the source page. Subsequent accesses by other processors receive a bus error and that removes the TLB entry for that location, read the location from the page table & update the TLB with new location. This approach is called - “Lazy TLB shootdown” which reduces the overall cost of migrating memory and changing the virtual-to-physical address mappings.

=== Additional states ===

Additional states can be introduced into the sharing list. The SSCI protocol has only one kind of sharing list state : dirty. More options can be introduced into the list to improve performance of the overall shared multiprocessors. Although additional states do increment the complexity of the protocol being utilized and implemented, it would be very much beneficial to the system when the overall performance of the system is increased. The question “Why Additional States are necessary” is answered alongside the introduction of each new state i.e. explanation for the necessity of each and every state has been provided.

Having the following states implemented in the protocol can improve performance.

1. “fresh” and “gone” – Main memory may still be able to respond to a requesting processor if the memory can differentiate between ‘cached and unmodified’ and ‘cached and modified’. Even if the cached value is elsewhere, memory may still be able to respond to the request if the data remains unmodified.

*The “fresh” state shows that memory still has an up-to-date copy of the block, even if the block has been cached and is being shared between processors. As long as the value remains unmodified by any of the processors sharing the data, the memory can keep responding by returning the requested data to the requesting processor.

*The “gone” state indicates that the block has been cached and has been modified. This shows that main memory cant return the data anymore, but has to return the pointer that points to the head of the list.

2. “clean lists” – Although the introduction of these states increase the complexity of handling each access, it does help with the overall performance. In this case, it may be helpful to differentiate the types of read accesses to the cache. One type of read access would be a ‘normal read-only’. Another type of read would be a ‘read with intent to modify later’. The memory in the state is ‘fresh’ when a read-fresh takes place. A read-clean leaves the memory in the ‘gone’ state. Just as with the dirty state, the read-clean allows a write to the block to proceed immediately. When a block is replaced in the cache and the cache is in the ‘clean’ state, no data needs to be written to memory. But with the dirty state, a write-back would be required even if the data had not been modified.

== Examples ==
=== DASH Cache Coherence Protocol ===

DASH (Directory Architecture for SHared memory) is a scalable shared-memory multiprocessor currently being developed at Stanford’s Computer Systems Laboratory. DASH protocol uses point-to-point messages sent between the processors and memories to keep caches consistent.

The DASH coherence protocol is an invalidation-based ownership protocol. A memory block can be in one of three states as indicated by the associated directory entry: (i) uncached-remote, that is not cached by any remote cluster; (ii) shared-remote, that is cached in an unmodified state by one or more remote clusters; or (iii) dirty-remote, that is cached in a modified state by a single remote cluster.

Please see in the figures below: Left - Flow of Read Request to remote memory with directory in dirty-remote state. Right - Flow of Read-Exclusive Request to remote memory with directory in shared-remote state.

[[Image:DASH.jpg]]

Write back request: A dirty cache line that is replaced must be written back to memory. If the home of the memory block is the local cluster, then the data is simply written back to main memory. If the home cluster is remote, then a message is sent to the remote
home which updates the main memory and marks the block uncached-remote.

= References =

1 The DASH Cache Coherence Protocol
Daniel Lenoski, James Laudon, Kourosh Gharachorloo, Anoop Gupta, John Hennessy
May 1990
ACM SIGARCH Computer Architecture News , Proceedings of the 17th annual international symposium on Computer Architecture ISCA '90,
Volume 18 Issue 3a
Publisher: ACM Press

2. 1596-1992 IEEE standard for scalable coherent interface (SCI).
E-ISBN: 0-7381-1204-6
Year: 1993
Sponsored by: IEEE Computer Society

3. The SGI Origin: A ccnuma Highly Scalable Server
Laudon, J.; Lenoski, D.;
Computer Architecture, 1997. Conference Proceedings. The 24th Annual International Symposium on
June 2-4, 1997 Page(s):241 - 251

4. The Scalable Coherence Interface
David.B.Gustavson
September 1991
"The Scalable Coherence Interface and related standard projects"

5. An analysis of the Scalable Coherent Interface
Eric Rotenberg
June 1995
An Analytical model of the SCI Coherence protocol

6. Interfaces and Standards : SCI
Behrooz. Parhami
1999
Introduction to Parallel Processing: Algorithms and Architectures

7. SCI: Scalable Coherent Interface
Hermann Hellwagner
1999
SCI: Scalable Coherent Interface - Architecture and software for high-performance Compute Clusters

= Further reading =

1. The Stanford Dash multiprocessor
Lenoski, D.; Laudon, J.; Gharachorloo, K.; Weber, W.-D.; Gupta, A.; Hennessy, J.; Horowitz, M.; Lam, M.S.;
Computer
IEEE Volume 25, Issue 3, March 1992 Page(s):63 - 79
Digital Object Identifier 10.1109/2.121510

2. SCI: Scalable Coherent Interface
Hermann Hellwagner
1999
SCI: Scalable Coherent Interface - Architecture and software for high-performance Compute Clusters

= External links =

[http://www.scizzl.com/WhatIsSCI.html www.scizzl.com/WhatIsSCI.html] - What is SCI ?

[http://www.dolphinics.com/corporate/scitech.html www.dolphinics.com/corporate/scitech.html] - The SCI technology provided by Dolphin Interconnect Solutions Inc

CSC/ECE 506 Fall 2007/wiki3 2 1r

2007-10-20T03:38:41Z

Arangan: /* External links */

''Wiki: SCI. The IEEE Scalable Coherent Interface is a superset of the SSCI protocol we have been considering in class. A lot has been written about it, but it is still difficult to comprehend. Using SSCI as a starting point, explain why additional states are necessary, and give (or cite) examples that demonstrate how they work. Ideally, this would still be an overview of the working of the protocol, referencing more detailed documentation on the Web.''

__TOC__

= Introduction =

In parallel systems, data can be obtained from a variety of sources, which includes local or remote memory modules, secondary storage devices and network interfaces. It would be beneficial if certain aspects of mechanisms that are required to gain access to data are standardized such that it allows easily expandable and flexible designs. Various portions of a parallel system may be modified or even upgraded without any effect on the rest of the design if the system adheres to such standards. Also, utilization of standard interfaces forwards the integration of components and subsystems from several different vendors into efficient and usable parallel systems. The Scalable Coherent Interface is one such standard which facilitates the implementation of large-scale cache-coherent parallel systems.

= Scalable Coherent Interface (SCI) =

The scalable coherent interface has now become a chief hardware based approach to the cache coherence problem in shared memory multiprocessors. SCI is a directory based invalidate coherence protocol. The state of a cache block is distributed to the sharers of that block. Limits that are inherent in bus technology are easily avoided by SCI. The SCI protocol is to provide scalability, coherence and an interface. Scalability is to guarantee that the same mechanisms can be used in single processor systems and large highly parallel multiprocessors. Coherence is to guarantee efficient and integral use of cache memories in distributed shared memory. An interface provides a communication architecture that has multiple values to be brought into a single system and provide smooth inter-operation.

===Goals of SCI===

*High Performance – For distributed or parallel applications, high communication performance must be obtained when using SCI. Usual aspects of such performance would be a low latency, low CPU overhead for operations dealing with communication and high sustained throughput.

*Scalability – This is addressed in many ways. Some are
**Performance scalability – as the number of nodes being added to the system increases.
**Interconnect Distance scalability
**Memory System scalability – chiefly of cache coherence protocols, which should not have limitations to the number of processors or modules in memory that it can handle.
**Technological scalability – utilization of similar mechanisms in large, small-scale, loosely-coupled or tightly-coupled systems. Also, when new advances are made in technology, it must have the ability to readily make use of such advances.
**Economic scalability – use of similar components and mechanisms in high-end, low-volume systems as well as low-end, high-volume systems.
**Addressing capability has no short term practical limits.

*Coherent Memory System – For the purpose of reduction of Average Access Time to data, caches are becoming more important each day for microprocessors.

*Interface Characteristics – This specifies a standard interface to an interconnect that makes several devices to be connected or attached together and to interoperate.

===Operation===

In SCI, every interface does not wait for the signal to propagate before it begins to send the next signal. Also, SCI utilizes multiple links so that, concurrently, several transfers can take place.

Usually, a directory entry is in either of the two states : ‘home’ or ‘gone’. If the state is in ‘home’, then memory can immediately satisfy requests to a block as the block has not been cached by any processor. If the state is ‘gone’, then the block has been cached by a processor and might even be modified. Now, the directory contains a pointer to the first processor in the sharing list for this particular block. Hence, on requesting the data, memory returns the pointer to the first processor on the shared list rather than the data itself. The processor asking for the block now forwards its request to the processor on the top of the shared list. Now, the requesting processor adds itself into the shared list as the new head of the list.

In the SCI protocol, any coherent transaction has three phases.

Memory read – When a processor misses in its cache, It asks for the block in the home directory in memory. If the state of the memory is ‘home’, then the main memory replies with the block to the requesting processor. If the state is ‘gone’, then the main memory returns the head of the shared list of processors for that particular block. Then, memory updates its pointer and puts the requesting processor as the new head.

Cache read – When memory returns a pointer toe the requesting processor instead of data, the processor forwards its request for the block to the head of the doubly linked list. When the cache receives, the head of the list returns the data which might have been modified. The head of the list changes its backward pointer to the requesting processor’s node. Now, the requesting processor becomes the head of the list and it has the cache block.

Cleanup – If the cache miss from the requesting processor is a store, the processor has to first invalidate all other cached copies and then only proceed with the store. The new head gives an invalidate request to the next address on the list i.e. to its next processor. This processor invalidates and gives back a pointer to the next processor on its list. The head of the list uses this new pointer and sends it an invalidate request. This goes on until a NULL pointer is returned. This is the cleanup process for invalidation of other cache copies of the block.

===Applications===

1. SCI-based CC-NUMA machines

SCI is attached to the memory bus of a node. Here, SCI can take part in and "export" cache coherence and memory traffic on the bus and make the memory of the node visible to other nodes. Each node's memory address range can be scaled to span a global address space, giving coherent and transparent access to anywhere in the system to requesting processors. This method is used to form a large-scale cache-coherent shared-memory system, known as a CC-NUMA machine.

2. SGI Origin 2000

This would seem to be the only commercial alternative to CC-Numa machines. The Origin 2000 is an advanced version of the SCI-based CC-NUMA machines but has its origins in the Stanford DASH project. Cache coherence is maintained here with a bit-vector directory scheme.

3. S3.mp

This was a project by SUN to connect off-the-shelf workstations and make this interconnection into a large-scale cache-coherent multiprocessor. This has an invalidation based protocol which notes the nodes that share a memory block and stores such information in linked lists. The nodes in this project were spatially distributed and were connected together using arbitrary topologies.

= Simple Scalable Coherent Interface (SSCI) =

In directory-based approach, Every memory block has associated directory information; it keeps track of copies of cached blocks and their states. On a miss, it finds the directory entry, looks it up, and communicates only with the nodes that have copies (if necessary).

There are mainly two approaches: Full-bit vector: For k processors, it maintains k presence bit and 1 dirty bit at the home node. Cache state is represented the same way as in bus-based designs (MSI, MESI, etc.). It has three cache states: EM (exclusive or modified), S (shared), U (unowned). Limitation is - Number of presence bits needed grows as the number of processors.

Memory-based schemes store the information about all cached copies at the home node of the block. Cache-based schemes distribute information about copies among the copies themselves. The home contains a pointer to one cached copy of the block. Each copy contains the identity of the next node that has a copy of the block. The location of the copies is therefore determined through network transactions.

Simple SCI (SSCI) retains similarity with full-bit vector protocol: MESI states in the cache; U, S, EM states in the memory directory; It replaces the presence bits with a pointer.

== Why additional states are necessary? ==

Performance & Correctness (Coherence & Consistency) requirement for the protocol necessitates the additional states in the protocol.

=== Need for Busy state ===

On a scalable multiprocessors without coherent caches, the main-memory module determined the ordering of writes. The order that writes become visible to all processors is the order in which they reached memory.

e.g. If two processors issue read-exclusive requests for a particular word, the home will provide the requestors with the location of
the dirty node. But which request will reach the dirty node first cannot be guaranteed. This creates the need for additional busy state in the directory.

This can be solved by
# holding requests at home or requestor node and serve them in the order of their arrival
# If the block is busy, reject any further request to it & that request will be retried later
# If directory is busy forward request to dirty node & dirty node will serialize the request execution

In non-coherent scalable multiprocessors,
# For write atomicity in invalidation based protocol, current owner of block has to wait until it recieve all invalidation acks and then only it can read/write new value.
# For write completion, current owner of the block need to wait for ack from memory

In Origin, there are 3 busy states indicates that the directory has received the request but is waiting for completion hence another request for the same block won't be accepted.
# Busy read
# Busy read exclusive or upgrade
# Busy uncached read: In case of DMA, if the memory block is being read, no other processor should be allowed to get write access to it.

=== Poison state ===

To improve locality, origin relies on page migration. Hardware keeps reference counts on each page. On every access of memory count is incremented and compared with home node. If count is more than programmable threshold, hardware interrupts one of the local processors to copy the page. Block transfer engine “poison” the source page. Subsequent accesses by other processors receive a bus error and that removes the TLB entry for that location, read the location from the page table & update the TLB with new location. This approach is called - “Lazy TLB shootdown” which reduces the overall cost of migrating memory and changing the virtual-to-physical address mappings.

=== Additional states ===

Additional states can be introduced into the sharing list. The SSCI protocol has only one kind of sharing list state : dirty. More options can be introduced into the list to improve performance of the overall shared multiprocessors. Although additional states do increment the complexity of the protocol being utilized and implemented, it would be very much beneficial to the system when the overall performance of the system is increased. The question “Why Additional States are necessary” is answered alongside the introduction of each new state i.e. explanation for the necessity of each and every state has been provided.

Having the following states implemented in the protocol can improve performance.

1. “fresh” and “gone” – Main memory may still be able to respond to a requesting processor if the memory can differentiate between ‘cached and unmodified’ and ‘cached and modified’. Even if the cached value is elsewhere, memory may still be able to respond to the request if the data remains unmodified.

*The “fresh” state shows that memory still has an up-to-date copy of the block, even if the block has been cached and is being shared between processors. As long as the value remains unmodified by any of the processors sharing the data, the memory can keep responding by returning the requested data to the requesting processor.

*The “gone” state indicates that the block has been cached and has been modified. This shows that main memory cant return the data anymore, but has to return the pointer that points to the head of the list.

2. “clean lists” – Although the introduction of these states increase the complexity of handling each access, it does help with the overall performance. In this case, it may be helpful to differentiate the types of read accesses to the cache. One type of read access would be a ‘normal read-only’. Another type of read would be a ‘read with intent to modify later’. The memory in the state is ‘fresh’ when a read-fresh takes place. A read-clean leaves the memory in the ‘gone’ state. Just as with the dirty state, the read-clean allows a write to the block to proceed immediately. When a block is replaced in the cache and the cache is in the ‘clean’ state, no data needs to be written to memory. But with the dirty state, a write-back would be required even if the data had not been modified.

== Examples ==
=== DASH Cache Coherence Protocol ===

DASH (Directory Architecture for SHared memory) is a scalable shared-memory multiprocessor currently being developed at Stanford’s Computer Systems Laboratory. DASH protocol uses point-to-point messages sent between the processors and memories to keep caches consistent.

The DASH coherence protocol is an invalidation-based ownership protocol. A memory block can be in one of three states as indicated by the associated directory entry: (i) uncached-remote, that is not cached by any remote cluster; (ii) shared-remote, that is cached in an unmodified state by one or more remote clusters; or (iii) dirty-remote, that is cached in a modified state by a single remote cluster.

Please see in the figures below: Left - Flow of Read Request to remote memory with directory in dirty-remote state. Right - Flow of Read-Exclusive Request to remote memory with directory in shared-remote state.

[[Image:DASH.jpg]]

Write back request: A dirty cache line that is replaced must be written back to memory. If the home of the memory block is the local cluster, then the data is simply written back to main memory. If the home cluster is remote, then a message is sent to the remote
home which updates the main memory and marks the block uncached-remote.

= References =

1 The DASH Cache Coherence Protocol
Daniel Lenoski, James Laudon, Kourosh Gharachorloo, Anoop Gupta, John Hennessy
May 1990
ACM SIGARCH Computer Architecture News , Proceedings of the 17th annual international symposium on Computer Architecture ISCA '90,
Volume 18 Issue 3a
Publisher: ACM Press

2. 1596-1992 IEEE standard for scalable coherent interface (SCI).
E-ISBN: 0-7381-1204-6
Year: 1993
Sponsored by: IEEE Computer Society

3. The SGI Origin: A ccnuma Highly Scalable Server
Laudon, J.; Lenoski, D.;
Computer Architecture, 1997. Conference Proceedings. The 24th Annual International Symposium on
June 2-4, 1997 Page(s):241 - 251

4. The Scalable Coherence Interface
David.B.Gustavson
September 1991
"The Scalable Coherence Interface and related standard projects"

5. An analysis of the Scalable Coherent Interface
Eric Rotenberg
June 1995
An Analytical model of the SCI Coherence protocol

6. Interfaces and Standards : SCI
Behrooz. Parhami
1999
Introduction to Parallel Processing: Algorithms and Architectures

7. SCI: Scalable Coherent Interface
Hermann Hellwagner
1999
SCI: Scalable Coherent Interface - Architecture and software for high-performance Compute Clusters

= Further reading =

1. The Stanford Dash multiprocessor
Lenoski, D.; Laudon, J.; Gharachorloo, K.; Weber, W.-D.; Gupta, A.; Hennessy, J.; Horowitz, M.; Lam, M.S.;
Computer
IEEE Volume 25, Issue 3, March 1992 Page(s):63 - 79
Digital Object Identifier 10.1109/2.121510

2. SCI: Scalable Coherent Interface
Hermann Hellwagner
1999
SCI: Scalable Coherent Interface - Architecture and software for high-performance Compute Clusters

= External links =

[http://www.scizzl.com/WhatIsSCI.html] - What is SCI ?

[http://www.dolphinics.com/corporate/scitech.html] - The SCI technology provided by Dolphin Interconnect Solutions Inc

CSC/ECE 506 Fall 2007/wiki3 2 1r

2007-10-20T03:35:12Z

Arangan: /* External links */

''Wiki: SCI. The IEEE Scalable Coherent Interface is a superset of the SSCI protocol we have been considering in class. A lot has been written about it, but it is still difficult to comprehend. Using SSCI as a starting point, explain why additional states are necessary, and give (or cite) examples that demonstrate how they work. Ideally, this would still be an overview of the working of the protocol, referencing more detailed documentation on the Web.''

__TOC__

= Introduction =

In parallel systems, data can be obtained from a variety of sources, which includes local or remote memory modules, secondary storage devices and network interfaces. It would be beneficial if certain aspects of mechanisms that are required to gain access to data are standardized such that it allows easily expandable and flexible designs. Various portions of a parallel system may be modified or even upgraded without any effect on the rest of the design if the system adheres to such standards. Also, utilization of standard interfaces forwards the integration of components and subsystems from several different vendors into efficient and usable parallel systems. The Scalable Coherent Interface is one such standard which facilitates the implementation of large-scale cache-coherent parallel systems.

= Scalable Coherent Interface (SCI) =

The scalable coherent interface has now become a chief hardware based approach to the cache coherence problem in shared memory multiprocessors. SCI is a directory based invalidate coherence protocol. The state of a cache block is distributed to the sharers of that block. Limits that are inherent in bus technology are easily avoided by SCI. The SCI protocol is to provide scalability, coherence and an interface. Scalability is to guarantee that the same mechanisms can be used in single processor systems and large highly parallel multiprocessors. Coherence is to guarantee efficient and integral use of cache memories in distributed shared memory. An interface provides a communication architecture that has multiple values to be brought into a single system and provide smooth inter-operation.

===Goals of SCI===

*High Performance – For distributed or parallel applications, high communication performance must be obtained when using SCI. Usual aspects of such performance would be a low latency, low CPU overhead for operations dealing with communication and high sustained throughput.

*Scalability – This is addressed in many ways. Some are
**Performance scalability – as the number of nodes being added to the system increases.
**Interconnect Distance scalability
**Memory System scalability – chiefly of cache coherence protocols, which should not have limitations to the number of processors or modules in memory that it can handle.
**Technological scalability – utilization of similar mechanisms in large, small-scale, loosely-coupled or tightly-coupled systems. Also, when new advances are made in technology, it must have the ability to readily make use of such advances.
**Economic scalability – use of similar components and mechanisms in high-end, low-volume systems as well as low-end, high-volume systems.
**Addressing capability has no short term practical limits.

*Coherent Memory System – For the purpose of reduction of Average Access Time to data, caches are becoming more important each day for microprocessors.

*Interface Characteristics – This specifies a standard interface to an interconnect that makes several devices to be connected or attached together and to interoperate.

===Operation===

In SCI, every interface does not wait for the signal to propagate before it begins to send the next signal. Also, SCI utilizes multiple links so that, concurrently, several transfers can take place.

Usually, a directory entry is in either of the two states : ‘home’ or ‘gone’. If the state is in ‘home’, then memory can immediately satisfy requests to a block as the block has not been cached by any processor. If the state is ‘gone’, then the block has been cached by a processor and might even be modified. Now, the directory contains a pointer to the first processor in the sharing list for this particular block. Hence, on requesting the data, memory returns the pointer to the first processor on the shared list rather than the data itself. The processor asking for the block now forwards its request to the processor on the top of the shared list. Now, the requesting processor adds itself into the shared list as the new head of the list.

In the SCI protocol, any coherent transaction has three phases.

Memory read – When a processor misses in its cache, It asks for the block in the home directory in memory. If the state of the memory is ‘home’, then the main memory replies with the block to the requesting processor. If the state is ‘gone’, then the main memory returns the head of the shared list of processors for that particular block. Then, memory updates its pointer and puts the requesting processor as the new head.

Cache read – When memory returns a pointer toe the requesting processor instead of data, the processor forwards its request for the block to the head of the doubly linked list. When the cache receives, the head of the list returns the data which might have been modified. The head of the list changes its backward pointer to the requesting processor’s node. Now, the requesting processor becomes the head of the list and it has the cache block.

Cleanup – If the cache miss from the requesting processor is a store, the processor has to first invalidate all other cached copies and then only proceed with the store. The new head gives an invalidate request to the next address on the list i.e. to its next processor. This processor invalidates and gives back a pointer to the next processor on its list. The head of the list uses this new pointer and sends it an invalidate request. This goes on until a NULL pointer is returned. This is the cleanup process for invalidation of other cache copies of the block.

===Applications===

1. SCI-based CC-NUMA machines

SCI is attached to the memory bus of a node. Here, SCI can take part in and "export" cache coherence and memory traffic on the bus and make the memory of the node visible to other nodes. Each node's memory address range can be scaled to span a global address space, giving coherent and transparent access to anywhere in the system to requesting processors. This method is used to form a large-scale cache-coherent shared-memory system, known as a CC-NUMA machine.

2. SGI Origin 2000

This would seem to be the only commercial alternative to CC-Numa machines. The Origin 2000 is an advanced version of the SCI-based CC-NUMA machines but has its origins in the Stanford DASH project. Cache coherence is maintained here with a bit-vector directory scheme.

3. S3.mp

This was a project by SUN to connect off-the-shelf workstations and make this interconnection into a large-scale cache-coherent multiprocessor. This has an invalidation based protocol which notes the nodes that share a memory block and stores such information in linked lists. The nodes in this project were spatially distributed and were connected together using arbitrary topologies.

= Simple Scalable Coherent Interface (SSCI) =

In directory-based approach, Every memory block has associated directory information; it keeps track of copies of cached blocks and their states. On a miss, it finds the directory entry, looks it up, and communicates only with the nodes that have copies (if necessary).

There are mainly two approaches: Full-bit vector: For k processors, it maintains k presence bit and 1 dirty bit at the home node. Cache state is represented the same way as in bus-based designs (MSI, MESI, etc.). It has three cache states: EM (exclusive or modified), S (shared), U (unowned). Limitation is - Number of presence bits needed grows as the number of processors.

Memory-based schemes store the information about all cached copies at the home node of the block. Cache-based schemes distribute information about copies among the copies themselves. The home contains a pointer to one cached copy of the block. Each copy contains the identity of the next node that has a copy of the block. The location of the copies is therefore determined through network transactions.

Simple SCI (SSCI) retains similarity with full-bit vector protocol: MESI states in the cache; U, S, EM states in the memory directory; It replaces the presence bits with a pointer.

== Why additional states are necessary? ==

Performance & Correctness (Coherence & Consistency) requirement for the protocol necessitates the additional states in the protocol.

=== Need for Busy state ===

On a scalable multiprocessors without coherent caches, the main-memory module determined the ordering of writes. The order that writes become visible to all processors is the order in which they reached memory.

e.g. If two processors issue read-exclusive requests for a particular word, the home will provide the requestors with the location of
the dirty node. But which request will reach the dirty node first cannot be guaranteed. This creates the need for additional busy state in the directory.

This can be solved by
# holding requests at home or requestor node and serve them in the order of their arrival
# If the block is busy, reject any further request to it & that request will be retried later
# If directory is busy forward request to dirty node & dirty node will serialize the request execution

In non-coherent scalable multiprocessors,
# For write atomicity in invalidation based protocol, current owner of block has to wait until it recieve all invalidation acks and then only it can read/write new value.
# For write completion, current owner of the block need to wait for ack from memory

In Origin, there are 3 busy states indicates that the directory has received the request but is waiting for completion hence another request for the same block won't be accepted.
# Busy read
# Busy read exclusive or upgrade
# Busy uncached read: In case of DMA, if the memory block is being read, no other processor should be allowed to get write access to it.

=== Poison state ===

To improve locality, origin relies on page migration. Hardware keeps reference counts on each page. On every access of memory count is incremented and compared with home node. If count is more than programmable threshold, hardware interrupts one of the local processors to copy the page. Block transfer engine “poison” the source page. Subsequent accesses by other processors receive a bus error and that removes the TLB entry for that location, read the location from the page table & update the TLB with new location. This approach is called - “Lazy TLB shootdown” which reduces the overall cost of migrating memory and changing the virtual-to-physical address mappings.

=== Additional states ===

Additional states can be introduced into the sharing list. The SSCI protocol has only one kind of sharing list state : dirty. More options can be introduced into the list to improve performance of the overall shared multiprocessors. Although additional states do increment the complexity of the protocol being utilized and implemented, it would be very much beneficial to the system when the overall performance of the system is increased. The question “Why Additional States are necessary” is answered alongside the introduction of each new state i.e. explanation for the necessity of each and every state has been provided.

Having the following states implemented in the protocol can improve performance.

1. “fresh” and “gone” – Main memory may still be able to respond to a requesting processor if the memory can differentiate between ‘cached and unmodified’ and ‘cached and modified’. Even if the cached value is elsewhere, memory may still be able to respond to the request if the data remains unmodified.

*The “fresh” state shows that memory still has an up-to-date copy of the block, even if the block has been cached and is being shared between processors. As long as the value remains unmodified by any of the processors sharing the data, the memory can keep responding by returning the requested data to the requesting processor.

*The “gone” state indicates that the block has been cached and has been modified. This shows that main memory cant return the data anymore, but has to return the pointer that points to the head of the list.

2. “clean lists” – Although the introduction of these states increase the complexity of handling each access, it does help with the overall performance. In this case, it may be helpful to differentiate the types of read accesses to the cache. One type of read access would be a ‘normal read-only’. Another type of read would be a ‘read with intent to modify later’. The memory in the state is ‘fresh’ when a read-fresh takes place. A read-clean leaves the memory in the ‘gone’ state. Just as with the dirty state, the read-clean allows a write to the block to proceed immediately. When a block is replaced in the cache and the cache is in the ‘clean’ state, no data needs to be written to memory. But with the dirty state, a write-back would be required even if the data had not been modified.

== Examples ==
=== DASH Cache Coherence Protocol ===

DASH (Directory Architecture for SHared memory) is a scalable shared-memory multiprocessor currently being developed at Stanford’s Computer Systems Laboratory. DASH protocol uses point-to-point messages sent between the processors and memories to keep caches consistent.

The DASH coherence protocol is an invalidation-based ownership protocol. A memory block can be in one of three states as indicated by the associated directory entry: (i) uncached-remote, that is not cached by any remote cluster; (ii) shared-remote, that is cached in an unmodified state by one or more remote clusters; or (iii) dirty-remote, that is cached in a modified state by a single remote cluster.

Please see in the figures below: Left - Flow of Read Request to remote memory with directory in dirty-remote state. Right - Flow of Read-Exclusive Request to remote memory with directory in shared-remote state.

[[Image:DASH.jpg]]

Write back request: A dirty cache line that is replaced must be written back to memory. If the home of the memory block is the local cluster, then the data is simply written back to main memory. If the home cluster is remote, then a message is sent to the remote
home which updates the main memory and marks the block uncached-remote.

= References =

1 The DASH Cache Coherence Protocol
Daniel Lenoski, James Laudon, Kourosh Gharachorloo, Anoop Gupta, John Hennessy
May 1990
ACM SIGARCH Computer Architecture News , Proceedings of the 17th annual international symposium on Computer Architecture ISCA '90,
Volume 18 Issue 3a
Publisher: ACM Press

2. 1596-1992 IEEE standard for scalable coherent interface (SCI).
E-ISBN: 0-7381-1204-6
Year: 1993
Sponsored by: IEEE Computer Society

3. The SGI Origin: A ccnuma Highly Scalable Server
Laudon, J.; Lenoski, D.;
Computer Architecture, 1997. Conference Proceedings. The 24th Annual International Symposium on
June 2-4, 1997 Page(s):241 - 251

4. The Scalable Coherence Interface
David.B.Gustavson
September 1991
"The Scalable Coherence Interface and related standard projects"

5. An analysis of the Scalable Coherent Interface
Eric Rotenberg
June 1995
An Analytical model of the SCI Coherence protocol

6. Interfaces and Standards : SCI
Behrooz. Parhami
1999
Introduction to Parallel Processing: Algorithms and Architectures

7. SCI: Scalable Coherent Interface
Hermann Hellwagner
1999
SCI: Scalable Coherent Interface - Architecture and software for high-performance Compute Clusters

= Further reading =

1. The Stanford Dash multiprocessor
Lenoski, D.; Laudon, J.; Gharachorloo, K.; Weber, W.-D.; Gupta, A.; Hennessy, J.; Horowitz, M.; Lam, M.S.;
Computer
IEEE Volume 25, Issue 3, March 1992 Page(s):63 - 79
Digital Object Identifier 10.1109/2.121510

2. SCI: Scalable Coherent Interface
Hermann Hellwagner
1999
SCI: Scalable Coherent Interface - Architecture and software for high-performance Compute Clusters

= External links =

[www.scizzl.com/WhatIsSCI.html] - What is SCI ?

[www.dolphinics.com/corporate/scitech.html] - The SCI technology provided by Dolphin Interconnect Solutions Inc

CSC/ECE 506 Fall 2007/wiki3 2 1r

2007-10-20T03:34:22Z

Arangan: /* External links */

''Wiki: SCI. The IEEE Scalable Coherent Interface is a superset of the SSCI protocol we have been considering in class. A lot has been written about it, but it is still difficult to comprehend. Using SSCI as a starting point, explain why additional states are necessary, and give (or cite) examples that demonstrate how they work. Ideally, this would still be an overview of the working of the protocol, referencing more detailed documentation on the Web.''

__TOC__

= Introduction =

In parallel systems, data can be obtained from a variety of sources, which includes local or remote memory modules, secondary storage devices and network interfaces. It would be beneficial if certain aspects of mechanisms that are required to gain access to data are standardized such that it allows easily expandable and flexible designs. Various portions of a parallel system may be modified or even upgraded without any effect on the rest of the design if the system adheres to such standards. Also, utilization of standard interfaces forwards the integration of components and subsystems from several different vendors into efficient and usable parallel systems. The Scalable Coherent Interface is one such standard which facilitates the implementation of large-scale cache-coherent parallel systems.

= Scalable Coherent Interface (SCI) =

The scalable coherent interface has now become a chief hardware based approach to the cache coherence problem in shared memory multiprocessors. SCI is a directory based invalidate coherence protocol. The state of a cache block is distributed to the sharers of that block. Limits that are inherent in bus technology are easily avoided by SCI. The SCI protocol is to provide scalability, coherence and an interface. Scalability is to guarantee that the same mechanisms can be used in single processor systems and large highly parallel multiprocessors. Coherence is to guarantee efficient and integral use of cache memories in distributed shared memory. An interface provides a communication architecture that has multiple values to be brought into a single system and provide smooth inter-operation.

===Goals of SCI===

*High Performance – For distributed or parallel applications, high communication performance must be obtained when using SCI. Usual aspects of such performance would be a low latency, low CPU overhead for operations dealing with communication and high sustained throughput.

*Scalability – This is addressed in many ways. Some are
**Performance scalability – as the number of nodes being added to the system increases.
**Interconnect Distance scalability
**Memory System scalability – chiefly of cache coherence protocols, which should not have limitations to the number of processors or modules in memory that it can handle.
**Technological scalability – utilization of similar mechanisms in large, small-scale, loosely-coupled or tightly-coupled systems. Also, when new advances are made in technology, it must have the ability to readily make use of such advances.
**Economic scalability – use of similar components and mechanisms in high-end, low-volume systems as well as low-end, high-volume systems.
**Addressing capability has no short term practical limits.

*Coherent Memory System – For the purpose of reduction of Average Access Time to data, caches are becoming more important each day for microprocessors.

*Interface Characteristics – This specifies a standard interface to an interconnect that makes several devices to be connected or attached together and to interoperate.

===Operation===

In SCI, every interface does not wait for the signal to propagate before it begins to send the next signal. Also, SCI utilizes multiple links so that, concurrently, several transfers can take place.

Usually, a directory entry is in either of the two states : ‘home’ or ‘gone’. If the state is in ‘home’, then memory can immediately satisfy requests to a block as the block has not been cached by any processor. If the state is ‘gone’, then the block has been cached by a processor and might even be modified. Now, the directory contains a pointer to the first processor in the sharing list for this particular block. Hence, on requesting the data, memory returns the pointer to the first processor on the shared list rather than the data itself. The processor asking for the block now forwards its request to the processor on the top of the shared list. Now, the requesting processor adds itself into the shared list as the new head of the list.

In the SCI protocol, any coherent transaction has three phases.

Memory read – When a processor misses in its cache, It asks for the block in the home directory in memory. If the state of the memory is ‘home’, then the main memory replies with the block to the requesting processor. If the state is ‘gone’, then the main memory returns the head of the shared list of processors for that particular block. Then, memory updates its pointer and puts the requesting processor as the new head.

Cache read – When memory returns a pointer toe the requesting processor instead of data, the processor forwards its request for the block to the head of the doubly linked list. When the cache receives, the head of the list returns the data which might have been modified. The head of the list changes its backward pointer to the requesting processor’s node. Now, the requesting processor becomes the head of the list and it has the cache block.

Cleanup – If the cache miss from the requesting processor is a store, the processor has to first invalidate all other cached copies and then only proceed with the store. The new head gives an invalidate request to the next address on the list i.e. to its next processor. This processor invalidates and gives back a pointer to the next processor on its list. The head of the list uses this new pointer and sends it an invalidate request. This goes on until a NULL pointer is returned. This is the cleanup process for invalidation of other cache copies of the block.

===Applications===

1. SCI-based CC-NUMA machines

SCI is attached to the memory bus of a node. Here, SCI can take part in and "export" cache coherence and memory traffic on the bus and make the memory of the node visible to other nodes. Each node's memory address range can be scaled to span a global address space, giving coherent and transparent access to anywhere in the system to requesting processors. This method is used to form a large-scale cache-coherent shared-memory system, known as a CC-NUMA machine.

2. SGI Origin 2000

This would seem to be the only commercial alternative to CC-Numa machines. The Origin 2000 is an advanced version of the SCI-based CC-NUMA machines but has its origins in the Stanford DASH project. Cache coherence is maintained here with a bit-vector directory scheme.

3. S3.mp

This was a project by SUN to connect off-the-shelf workstations and make this interconnection into a large-scale cache-coherent multiprocessor. This has an invalidation based protocol which notes the nodes that share a memory block and stores such information in linked lists. The nodes in this project were spatially distributed and were connected together using arbitrary topologies.

= Simple Scalable Coherent Interface (SSCI) =

In directory-based approach, Every memory block has associated directory information; it keeps track of copies of cached blocks and their states. On a miss, it finds the directory entry, looks it up, and communicates only with the nodes that have copies (if necessary).

There are mainly two approaches: Full-bit vector: For k processors, it maintains k presence bit and 1 dirty bit at the home node. Cache state is represented the same way as in bus-based designs (MSI, MESI, etc.). It has three cache states: EM (exclusive or modified), S (shared), U (unowned). Limitation is - Number of presence bits needed grows as the number of processors.

Memory-based schemes store the information about all cached copies at the home node of the block. Cache-based schemes distribute information about copies among the copies themselves. The home contains a pointer to one cached copy of the block. Each copy contains the identity of the next node that has a copy of the block. The location of the copies is therefore determined through network transactions.

Simple SCI (SSCI) retains similarity with full-bit vector protocol: MESI states in the cache; U, S, EM states in the memory directory; It replaces the presence bits with a pointer.

== Why additional states are necessary? ==

Performance & Correctness (Coherence & Consistency) requirement for the protocol necessitates the additional states in the protocol.

=== Need for Busy state ===

On a scalable multiprocessors without coherent caches, the main-memory module determined the ordering of writes. The order that writes become visible to all processors is the order in which they reached memory.

e.g. If two processors issue read-exclusive requests for a particular word, the home will provide the requestors with the location of
the dirty node. But which request will reach the dirty node first cannot be guaranteed. This creates the need for additional busy state in the directory.

This can be solved by
# holding requests at home or requestor node and serve them in the order of their arrival
# If the block is busy, reject any further request to it & that request will be retried later
# If directory is busy forward request to dirty node & dirty node will serialize the request execution

In non-coherent scalable multiprocessors,
# For write atomicity in invalidation based protocol, current owner of block has to wait until it recieve all invalidation acks and then only it can read/write new value.
# For write completion, current owner of the block need to wait for ack from memory

In Origin, there are 3 busy states indicates that the directory has received the request but is waiting for completion hence another request for the same block won't be accepted.
# Busy read
# Busy read exclusive or upgrade
# Busy uncached read: In case of DMA, if the memory block is being read, no other processor should be allowed to get write access to it.

=== Poison state ===

To improve locality, origin relies on page migration. Hardware keeps reference counts on each page. On every access of memory count is incremented and compared with home node. If count is more than programmable threshold, hardware interrupts one of the local processors to copy the page. Block transfer engine “poison” the source page. Subsequent accesses by other processors receive a bus error and that removes the TLB entry for that location, read the location from the page table & update the TLB with new location. This approach is called - “Lazy TLB shootdown” which reduces the overall cost of migrating memory and changing the virtual-to-physical address mappings.

=== Additional states ===

Additional states can be introduced into the sharing list. The SSCI protocol has only one kind of sharing list state : dirty. More options can be introduced into the list to improve performance of the overall shared multiprocessors. Although additional states do increment the complexity of the protocol being utilized and implemented, it would be very much beneficial to the system when the overall performance of the system is increased. The question “Why Additional States are necessary” is answered alongside the introduction of each new state i.e. explanation for the necessity of each and every state has been provided.

Having the following states implemented in the protocol can improve performance.

1. “fresh” and “gone” – Main memory may still be able to respond to a requesting processor if the memory can differentiate between ‘cached and unmodified’ and ‘cached and modified’. Even if the cached value is elsewhere, memory may still be able to respond to the request if the data remains unmodified.

*The “fresh” state shows that memory still has an up-to-date copy of the block, even if the block has been cached and is being shared between processors. As long as the value remains unmodified by any of the processors sharing the data, the memory can keep responding by returning the requested data to the requesting processor.

*The “gone” state indicates that the block has been cached and has been modified. This shows that main memory cant return the data anymore, but has to return the pointer that points to the head of the list.

2. “clean lists” – Although the introduction of these states increase the complexity of handling each access, it does help with the overall performance. In this case, it may be helpful to differentiate the types of read accesses to the cache. One type of read access would be a ‘normal read-only’. Another type of read would be a ‘read with intent to modify later’. The memory in the state is ‘fresh’ when a read-fresh takes place. A read-clean leaves the memory in the ‘gone’ state. Just as with the dirty state, the read-clean allows a write to the block to proceed immediately. When a block is replaced in the cache and the cache is in the ‘clean’ state, no data needs to be written to memory. But with the dirty state, a write-back would be required even if the data had not been modified.

== Examples ==
=== DASH Cache Coherence Protocol ===

DASH (Directory Architecture for SHared memory) is a scalable shared-memory multiprocessor currently being developed at Stanford’s Computer Systems Laboratory. DASH protocol uses point-to-point messages sent between the processors and memories to keep caches consistent.

The DASH coherence protocol is an invalidation-based ownership protocol. A memory block can be in one of three states as indicated by the associated directory entry: (i) uncached-remote, that is not cached by any remote cluster; (ii) shared-remote, that is cached in an unmodified state by one or more remote clusters; or (iii) dirty-remote, that is cached in a modified state by a single remote cluster.

Please see in the figures below: Left - Flow of Read Request to remote memory with directory in dirty-remote state. Right - Flow of Read-Exclusive Request to remote memory with directory in shared-remote state.

[[Image:DASH.jpg]]

Write back request: A dirty cache line that is replaced must be written back to memory. If the home of the memory block is the local cluster, then the data is simply written back to main memory. If the home cluster is remote, then a message is sent to the remote
home which updates the main memory and marks the block uncached-remote.

= References =

1 The DASH Cache Coherence Protocol
Daniel Lenoski, James Laudon, Kourosh Gharachorloo, Anoop Gupta, John Hennessy
May 1990
ACM SIGARCH Computer Architecture News , Proceedings of the 17th annual international symposium on Computer Architecture ISCA '90,
Volume 18 Issue 3a
Publisher: ACM Press

2. 1596-1992 IEEE standard for scalable coherent interface (SCI).
E-ISBN: 0-7381-1204-6
Year: 1993
Sponsored by: IEEE Computer Society

3. The SGI Origin: A ccnuma Highly Scalable Server
Laudon, J.; Lenoski, D.;
Computer Architecture, 1997. Conference Proceedings. The 24th Annual International Symposium on
June 2-4, 1997 Page(s):241 - 251

4. The Scalable Coherence Interface
David.B.Gustavson
September 1991
"The Scalable Coherence Interface and related standard projects"

5. An analysis of the Scalable Coherent Interface
Eric Rotenberg
June 1995
An Analytical model of the SCI Coherence protocol

6. Interfaces and Standards : SCI
Behrooz. Parhami
1999
Introduction to Parallel Processing: Algorithms and Architectures

7. SCI: Scalable Coherent Interface
Hermann Hellwagner
1999
SCI: Scalable Coherent Interface - Architecture and software for high-performance Compute Clusters

= Further reading =

1. The Stanford Dash multiprocessor
Lenoski, D.; Laudon, J.; Gharachorloo, K.; Weber, W.-D.; Gupta, A.; Hennessy, J.; Horowitz, M.; Lam, M.S.;
Computer
IEEE Volume 25, Issue 3, March 1992 Page(s):63 - 79
Digital Object Identifier 10.1109/2.121510

2. SCI: Scalable Coherent Interface
Hermann Hellwagner
1999
SCI: Scalable Coherent Interface - Architecture and software for high-performance Compute Clusters

= External links =

www.scizzl.com/WhatIsSCI.html - What is SCI ?

www.dolphinics.com/corporate/scitech.html - The SCI technology provided by Dolphin Interconnect Solutions Inc

CSC/ECE 506 Fall 2007/wiki3 2 1r

2007-10-20T03:31:22Z

Arangan: /* Applications */

CSC/ECE 506 Fall 2007/wiki3 2 1r

2007-10-20T03:30:53Z

Arangan: /* Scalable Coherent Interface (SCI) */

''Wiki: SCI. The IEEE Scalable Coherent Interface is a superset of the SSCI protocol we have been considering in class. A lot has been written about it, but it is still difficult to comprehend. Using SSCI as a starting point, explain why additional states are necessary, and give (or cite) examples that demonstrate how they work. Ideally, this would still be an overview of the working of the protocol, referencing more detailed documentation on the Web.''

__TOC__

= Introduction =

In parallel systems, data can be obtained from a variety of sources, which includes local or remote memory modules, secondary storage devices and network interfaces. It would be beneficial if certain aspects of mechanisms that are required to gain access to data are standardized such that it allows easily expandable and flexible designs. Various portions of a parallel system may be modified or even upgraded without any effect on the rest of the design if the system adheres to such standards. Also, utilization of standard interfaces forwards the integration of components and subsystems from several different vendors into efficient and usable parallel systems. The Scalable Coherent Interface is one such standard which facilitates the implementation of large-scale cache-coherent parallel systems.

= Scalable Coherent Interface (SCI) =

The scalable coherent interface has now become a chief hardware based approach to the cache coherence problem in shared memory multiprocessors. SCI is a directory based invalidate coherence protocol. The state of a cache block is distributed to the sharers of that block. Limits that are inherent in bus technology are easily avoided by SCI. The SCI protocol is to provide scalability, coherence and an interface. Scalability is to guarantee that the same mechanisms can be used in single processor systems and large highly parallel multiprocessors. Coherence is to guarantee efficient and integral use of cache memories in distributed shared memory. An interface provides a communication architecture that has multiple values to be brought into a single system and provide smooth inter-operation.

===Goals of SCI===

*High Performance – For distributed or parallel applications, high communication performance must be obtained when using SCI. Usual aspects of such performance would be a low latency, low CPU overhead for operations dealing with communication and high sustained throughput.

*Scalability – This is addressed in many ways. Some are
**Performance scalability – as the number of nodes being added to the system increases.
**Interconnect Distance scalability
**Memory System scalability – chiefly of cache coherence protocols, which should not have limitations to the number of processors or modules in memory that it can handle.
**Technological scalability – utilization of similar mechanisms in large, small-scale, loosely-coupled or tightly-coupled systems. Also, when new advances are made in technology, it must have the ability to readily make use of such advances.
**Economic scalability – use of similar components and mechanisms in high-end, low-volume systems as well as low-end, high-volume systems.
**Addressing capability has no short term practical limits.

*Coherent Memory System – For the purpose of reduction of Average Access Time to data, caches are becoming more important each day for microprocessors.

*Interface Characteristics – This specifies a standard interface to an interconnect that makes several devices to be connected or attached together and to interoperate.

===Operation===

In SCI, every interface does not wait for the signal to propagate before it begins to send the next signal. Also, SCI utilizes multiple links so that, concurrently, several transfers can take place.

Usually, a directory entry is in either of the two states : ‘home’ or ‘gone’. If the state is in ‘home’, then memory can immediately satisfy requests to a block as the block has not been cached by any processor. If the state is ‘gone’, then the block has been cached by a processor and might even be modified. Now, the directory contains a pointer to the first processor in the sharing list for this particular block. Hence, on requesting the data, memory returns the pointer to the first processor on the shared list rather than the data itself. The processor asking for the block now forwards its request to the processor on the top of the shared list. Now, the requesting processor adds itself into the shared list as the new head of the list.

In the SCI protocol, any coherent transaction has three phases.

Memory read – When a processor misses in its cache, It asks for the block in the home directory in memory. If the state of the memory is ‘home’, then the main memory replies with the block to the requesting processor. If the state is ‘gone’, then the main memory returns the head of the shared list of processors for that particular block. Then, memory updates its pointer and puts the requesting processor as the new head.

Cache read – When memory returns a pointer toe the requesting processor instead of data, the processor forwards its request for the block to the head of the doubly linked list. When the cache receives, the head of the list returns the data which might have been modified. The head of the list changes its backward pointer to the requesting processor’s node. Now, the requesting processor becomes the head of the list and it has the cache block.

Cleanup – If the cache miss from the requesting processor is a store, the processor has to first invalidate all other cached copies and then only proceed with the store. The new head gives an invalidate request to the next address on the list i.e. to its next processor. This processor invalidates and gives back a pointer to the next processor on its list. The head of the list uses this new pointer and sends it an invalidate request. This goes on until a NULL pointer is returned. This is the cleanup process for invalidation of other cache copies of the block.

===Applications===

1. SCI-based CC-NUMA machines ===

SCI is attached to the memory bus of a node. Here, SCI can take part in and "export" cache coherence and memory traffic on the bus and make the memory of the node visible to other nodes. Each node's memory address range can be scaled to span a global address space, giving coherent and transparent access to anywhere in the system to requesting processors. This method is used to form a large-scale cache-coherent shared-memory system, known as a CC-NUMA machine.

2. SGI Origin 2000 ===

This would seem to be the only commercial alternative to CC-Numa machines. The Origin 2000 is an advanced version of the SCI-based CC-NUMA machines but has its origins in the Stanford DASH project. Cache coherence is maintained here with a bit-vector directory scheme.

3. S3.mp ===

This was a project by SUN to connect off-the-shelf workstations and make this interconnection into a large-scale cache-coherent multiprocessor. This has an invalidation based protocol which notes the nodes that share a memory block and stores such information in linked lists. The nodes in this project were spatially distributed and were connected together using arbitrary topologies.

= Simple Scalable Coherent Interface (SSCI) =

In directory-based approach, Every memory block has associated directory information; it keeps track of copies of cached blocks and their states. On a miss, it finds the directory entry, looks it up, and communicates only with the nodes that have copies (if necessary).

There are mainly two approaches: Full-bit vector: For k processors, it maintains k presence bit and 1 dirty bit at the home node. Cache state is represented the same way as in bus-based designs (MSI, MESI, etc.). It has three cache states: EM (exclusive or modified), S (shared), U (unowned). Limitation is - Number of presence bits needed grows as the number of processors.

Memory-based schemes store the information about all cached copies at the home node of the block. Cache-based schemes distribute information about copies among the copies themselves. The home contains a pointer to one cached copy of the block. Each copy contains the identity of the next node that has a copy of the block. The location of the copies is therefore determined through network transactions.

Simple SCI (SSCI) retains similarity with full-bit vector protocol: MESI states in the cache; U, S, EM states in the memory directory; It replaces the presence bits with a pointer.

== Why additional states are necessary? ==

Performance & Correctness (Coherence & Consistency) requirement for the protocol necessitates the additional states in the protocol.

=== Need for Busy state ===

On a scalable multiprocessors without coherent caches, the main-memory module determined the ordering of writes. The order that writes become visible to all processors is the order in which they reached memory.

e.g. If two processors issue read-exclusive requests for a particular word, the home will provide the requestors with the location of
the dirty node. But which request will reach the dirty node first cannot be guaranteed. This creates the need for additional busy state in the directory.

This can be solved by
# holding requests at home or requestor node and serve them in the order of their arrival
# If the block is busy, reject any further request to it & that request will be retried later
# If directory is busy forward request to dirty node & dirty node will serialize the request execution

In non-coherent scalable multiprocessors,
# For write atomicity in invalidation based protocol, current owner of block has to wait until it recieve all invalidation acks and then only it can read/write new value.
# For write completion, current owner of the block need to wait for ack from memory

In Origin, there are 3 busy states indicates that the directory has received the request but is waiting for completion hence another request for the same block won't be accepted.
# Busy read
# Busy read exclusive or upgrade
# Busy uncached read: In case of DMA, if the memory block is being read, no other processor should be allowed to get write access to it.

=== Poison state ===

To improve locality, origin relies on page migration. Hardware keeps reference counts on each page. On every access of memory count is incremented and compared with home node. If count is more than programmable threshold, hardware interrupts one of the local processors to copy the page. Block transfer engine “poison” the source page. Subsequent accesses by other processors receive a bus error and that removes the TLB entry for that location, read the location from the page table & update the TLB with new location. This approach is called - “Lazy TLB shootdown” which reduces the overall cost of migrating memory and changing the virtual-to-physical address mappings.

=== Additional states ===

Additional states can be introduced into the sharing list. The SSCI protocol has only one kind of sharing list state : dirty. More options can be introduced into the list to improve performance of the overall shared multiprocessors. Although additional states do increment the complexity of the protocol being utilized and implemented, it would be very much beneficial to the system when the overall performance of the system is increased. The question “Why Additional States are necessary” is answered alongside the introduction of each new state i.e. explanation for the necessity of each and every state has been provided.

Having the following states implemented in the protocol can improve performance.

1. “fresh” and “gone” – Main memory may still be able to respond to a requesting processor if the memory can differentiate between ‘cached and unmodified’ and ‘cached and modified’. Even if the cached value is elsewhere, memory may still be able to respond to the request if the data remains unmodified.

*The “fresh” state shows that memory still has an up-to-date copy of the block, even if the block has been cached and is being shared between processors. As long as the value remains unmodified by any of the processors sharing the data, the memory can keep responding by returning the requested data to the requesting processor.

*The “gone” state indicates that the block has been cached and has been modified. This shows that main memory cant return the data anymore, but has to return the pointer that points to the head of the list.

2. “clean lists” – Although the introduction of these states increase the complexity of handling each access, it does help with the overall performance. In this case, it may be helpful to differentiate the types of read accesses to the cache. One type of read access would be a ‘normal read-only’. Another type of read would be a ‘read with intent to modify later’. The memory in the state is ‘fresh’ when a read-fresh takes place. A read-clean leaves the memory in the ‘gone’ state. Just as with the dirty state, the read-clean allows a write to the block to proceed immediately. When a block is replaced in the cache and the cache is in the ‘clean’ state, no data needs to be written to memory. But with the dirty state, a write-back would be required even if the data had not been modified.

== Examples ==
=== DASH Cache Coherence Protocol ===

DASH (Directory Architecture for SHared memory) is a scalable shared-memory multiprocessor currently being developed at Stanford’s Computer Systems Laboratory. DASH protocol uses point-to-point messages sent between the processors and memories to keep caches consistent.

The DASH coherence protocol is an invalidation-based ownership protocol. A memory block can be in one of three states as indicated by the associated directory entry: (i) uncached-remote, that is not cached by any remote cluster; (ii) shared-remote, that is cached in an unmodified state by one or more remote clusters; or (iii) dirty-remote, that is cached in a modified state by a single remote cluster.

Please see in the figures below: Left - Flow of Read Request to remote memory with directory in dirty-remote state. Right - Flow of Read-Exclusive Request to remote memory with directory in shared-remote state.

[[Image:DASH.jpg]]

Write back request: A dirty cache line that is replaced must be written back to memory. If the home of the memory block is the local cluster, then the data is simply written back to main memory. If the home cluster is remote, then a message is sent to the remote
home which updates the main memory and marks the block uncached-remote.

= References =

1 The DASH Cache Coherence Protocol
Daniel Lenoski, James Laudon, Kourosh Gharachorloo, Anoop Gupta, John Hennessy
May 1990
ACM SIGARCH Computer Architecture News , Proceedings of the 17th annual international symposium on Computer Architecture ISCA '90,
Volume 18 Issue 3a
Publisher: ACM Press

2. 1596-1992 IEEE standard for scalable coherent interface (SCI).
E-ISBN: 0-7381-1204-6
Year: 1993
Sponsored by: IEEE Computer Society

3. The SGI Origin: A ccnuma Highly Scalable Server
Laudon, J.; Lenoski, D.;
Computer Architecture, 1997. Conference Proceedings. The 24th Annual International Symposium on
June 2-4, 1997 Page(s):241 - 251

4. The Scalable Coherence Interface
David.B.Gustavson
September 1991
"The Scalable Coherence Interface and related standard projects"

5. An analysis of the Scalable Coherent Interface
Eric Rotenberg
June 1995
An Analytical model of the SCI Coherence protocol

6. Interfaces and Standards : SCI
Behrooz. Parhami
1999
Introduction to Parallel Processing: Algorithms and Architectures

7. SCI: Scalable Coherent Interface
Hermann Hellwagner
1999
SCI: Scalable Coherent Interface - Architecture and software for high-performance Compute Clusters

= Further reading =

1. The Stanford Dash multiprocessor
Lenoski, D.; Laudon, J.; Gharachorloo, K.; Weber, W.-D.; Gupta, A.; Hennessy, J.; Horowitz, M.; Lam, M.S.;
Computer
IEEE Volume 25, Issue 3, March 1992 Page(s):63 - 79
Digital Object Identifier 10.1109/2.121510

2. SCI: Scalable Coherent Interface
Hermann Hellwagner
1999
SCI: Scalable Coherent Interface - Architecture and software for high-performance Compute Clusters

= External links =

CSC/ECE 506 Fall 2007/wiki3 2 1r

2007-10-20T03:30:05Z

Arangan: /* Examples */

''Wiki: SCI. The IEEE Scalable Coherent Interface is a superset of the SSCI protocol we have been considering in class. A lot has been written about it, but it is still difficult to comprehend. Using SSCI as a starting point, explain why additional states are necessary, and give (or cite) examples that demonstrate how they work. Ideally, this would still be an overview of the working of the protocol, referencing more detailed documentation on the Web.''

__TOC__

= Introduction =

In parallel systems, data can be obtained from a variety of sources, which includes local or remote memory modules, secondary storage devices and network interfaces. It would be beneficial if certain aspects of mechanisms that are required to gain access to data are standardized such that it allows easily expandable and flexible designs. Various portions of a parallel system may be modified or even upgraded without any effect on the rest of the design if the system adheres to such standards. Also, utilization of standard interfaces forwards the integration of components and subsystems from several different vendors into efficient and usable parallel systems. The Scalable Coherent Interface is one such standard which facilitates the implementation of large-scale cache-coherent parallel systems.

= Scalable Coherent Interface (SCI) =

The scalable coherent interface has now become a chief hardware based approach to the cache coherence problem in shared memory multiprocessors. SCI is a directory based invalidate coherence protocol. The state of a cache block is distributed to the sharers of that block. Limits that are inherent in bus technology are easily avoided by SCI. The SCI protocol is to provide scalability, coherence and an interface. Scalability is to guarantee that the same mechanisms can be used in single processor systems and large highly parallel multiprocessors. Coherence is to guarantee efficient and integral use of cache memories in distributed shared memory. An interface provides a communication architecture that has multiple values to be brought into a single system and provide smooth inter-operation.

===Goals of SCI===

*High Performance – For distributed or parallel applications, high communication performance must be obtained when using SCI. Usual aspects of such performance would be a low latency, low CPU overhead for operations dealing with communication and high sustained throughput.

*Scalability – This is addressed in many ways. Some are
**Performance scalability – as the number of nodes being added to the system increases.
**Interconnect Distance scalability
**Memory System scalability – chiefly of cache coherence protocols, which should not have limitations to the number of processors or modules in memory that it can handle.
**Technological scalability – utilization of similar mechanisms in large, small-scale, loosely-coupled or tightly-coupled systems. Also, when new advances are made in technology, it must have the ability to readily make use of such advances.
**Economic scalability – use of similar components and mechanisms in high-end, low-volume systems as well as low-end, high-volume systems.
**Addressing capability has no short term practical limits.

*Coherent Memory System – For the purpose of reduction of Average Access Time to data, caches are becoming more important each day for microprocessors.

*Interface Characteristics – This specifies a standard interface to an interconnect that makes several devices to be connected or attached together and to interoperate.

===Operation===

In SCI, every interface does not wait for the signal to propagate before it begins to send the next signal. Also, SCI utilizes multiple links so that, concurrently, several transfers can take place.

Usually, a directory entry is in either of the two states : ‘home’ or ‘gone’. If the state is in ‘home’, then memory can immediately satisfy requests to a block as the block has not been cached by any processor. If the state is ‘gone’, then the block has been cached by a processor and might even be modified. Now, the directory contains a pointer to the first processor in the sharing list for this particular block. Hence, on requesting the data, memory returns the pointer to the first processor on the shared list rather than the data itself. The processor asking for the block now forwards its request to the processor on the top of the shared list. Now, the requesting processor adds itself into the shared list as the new head of the list.

In the SCI protocol, any coherent transaction has three phases.

Memory read – When a processor misses in its cache, It asks for the block in the home directory in memory. If the state of the memory is ‘home’, then the main memory replies with the block to the requesting processor. If the state is ‘gone’, then the main memory returns the head of the shared list of processors for that particular block. Then, memory updates its pointer and puts the requesting processor as the new head.

Cache read – When memory returns a pointer toe the requesting processor instead of data, the processor forwards its request for the block to the head of the doubly linked list. When the cache receives, the head of the list returns the data which might have been modified. The head of the list changes its backward pointer to the requesting processor’s node. Now, the requesting processor becomes the head of the list and it has the cache block.

Cleanup – If the cache miss from the requesting processor is a store, the processor has to first invalidate all other cached copies and then only proceed with the store. The new head gives an invalidate request to the next address on the list i.e. to its next processor. This processor invalidates and gives back a pointer to the next processor on its list. The head of the list uses this new pointer and sends it an invalidate request. This goes on until a NULL pointer is returned. This is the cleanup process for invalidation of other cache copies of the block.

= Simple Scalable Coherent Interface (SSCI) =

In directory-based approach, Every memory block has associated directory information; it keeps track of copies of cached blocks and their states. On a miss, it finds the directory entry, looks it up, and communicates only with the nodes that have copies (if necessary).

There are mainly two approaches: Full-bit vector: For k processors, it maintains k presence bit and 1 dirty bit at the home node. Cache state is represented the same way as in bus-based designs (MSI, MESI, etc.). It has three cache states: EM (exclusive or modified), S (shared), U (unowned). Limitation is - Number of presence bits needed grows as the number of processors.

Memory-based schemes store the information about all cached copies at the home node of the block. Cache-based schemes distribute information about copies among the copies themselves. The home contains a pointer to one cached copy of the block. Each copy contains the identity of the next node that has a copy of the block. The location of the copies is therefore determined through network transactions.

Simple SCI (SSCI) retains similarity with full-bit vector protocol: MESI states in the cache; U, S, EM states in the memory directory; It replaces the presence bits with a pointer.

== Why additional states are necessary? ==

Performance & Correctness (Coherence & Consistency) requirement for the protocol necessitates the additional states in the protocol.

=== Need for Busy state ===

On a scalable multiprocessors without coherent caches, the main-memory module determined the ordering of writes. The order that writes become visible to all processors is the order in which they reached memory.

e.g. If two processors issue read-exclusive requests for a particular word, the home will provide the requestors with the location of
the dirty node. But which request will reach the dirty node first cannot be guaranteed. This creates the need for additional busy state in the directory.

This can be solved by
# holding requests at home or requestor node and serve them in the order of their arrival
# If the block is busy, reject any further request to it & that request will be retried later
# If directory is busy forward request to dirty node & dirty node will serialize the request execution

In non-coherent scalable multiprocessors,
# For write atomicity in invalidation based protocol, current owner of block has to wait until it recieve all invalidation acks and then only it can read/write new value.
# For write completion, current owner of the block need to wait for ack from memory

In Origin, there are 3 busy states indicates that the directory has received the request but is waiting for completion hence another request for the same block won't be accepted.
# Busy read
# Busy read exclusive or upgrade
# Busy uncached read: In case of DMA, if the memory block is being read, no other processor should be allowed to get write access to it.

=== Poison state ===

To improve locality, origin relies on page migration. Hardware keeps reference counts on each page. On every access of memory count is incremented and compared with home node. If count is more than programmable threshold, hardware interrupts one of the local processors to copy the page. Block transfer engine “poison” the source page. Subsequent accesses by other processors receive a bus error and that removes the TLB entry for that location, read the location from the page table & update the TLB with new location. This approach is called - “Lazy TLB shootdown” which reduces the overall cost of migrating memory and changing the virtual-to-physical address mappings.

=== Additional states ===

Additional states can be introduced into the sharing list. The SSCI protocol has only one kind of sharing list state : dirty. More options can be introduced into the list to improve performance of the overall shared multiprocessors. Although additional states do increment the complexity of the protocol being utilized and implemented, it would be very much beneficial to the system when the overall performance of the system is increased. The question “Why Additional States are necessary” is answered alongside the introduction of each new state i.e. explanation for the necessity of each and every state has been provided.

Having the following states implemented in the protocol can improve performance.

1. “fresh” and “gone” – Main memory may still be able to respond to a requesting processor if the memory can differentiate between ‘cached and unmodified’ and ‘cached and modified’. Even if the cached value is elsewhere, memory may still be able to respond to the request if the data remains unmodified.

*The “fresh” state shows that memory still has an up-to-date copy of the block, even if the block has been cached and is being shared between processors. As long as the value remains unmodified by any of the processors sharing the data, the memory can keep responding by returning the requested data to the requesting processor.

*The “gone” state indicates that the block has been cached and has been modified. This shows that main memory cant return the data anymore, but has to return the pointer that points to the head of the list.

2. “clean lists” – Although the introduction of these states increase the complexity of handling each access, it does help with the overall performance. In this case, it may be helpful to differentiate the types of read accesses to the cache. One type of read access would be a ‘normal read-only’. Another type of read would be a ‘read with intent to modify later’. The memory in the state is ‘fresh’ when a read-fresh takes place. A read-clean leaves the memory in the ‘gone’ state. Just as with the dirty state, the read-clean allows a write to the block to proceed immediately. When a block is replaced in the cache and the cache is in the ‘clean’ state, no data needs to be written to memory. But with the dirty state, a write-back would be required even if the data had not been modified.

== Examples ==
=== DASH Cache Coherence Protocol ===

DASH (Directory Architecture for SHared memory) is a scalable shared-memory multiprocessor currently being developed at Stanford’s Computer Systems Laboratory. DASH protocol uses point-to-point messages sent between the processors and memories to keep caches consistent.

The DASH coherence protocol is an invalidation-based ownership protocol. A memory block can be in one of three states as indicated by the associated directory entry: (i) uncached-remote, that is not cached by any remote cluster; (ii) shared-remote, that is cached in an unmodified state by one or more remote clusters; or (iii) dirty-remote, that is cached in a modified state by a single remote cluster.

Please see in the figures below: Left - Flow of Read Request to remote memory with directory in dirty-remote state. Right - Flow of Read-Exclusive Request to remote memory with directory in shared-remote state.

[[Image:DASH.jpg]]

Write back request: A dirty cache line that is replaced must be written back to memory. If the home of the memory block is the local cluster, then the data is simply written back to main memory. If the home cluster is remote, then a message is sent to the remote
home which updates the main memory and marks the block uncached-remote.

= References =

1 The DASH Cache Coherence Protocol
Daniel Lenoski, James Laudon, Kourosh Gharachorloo, Anoop Gupta, John Hennessy
May 1990
ACM SIGARCH Computer Architecture News , Proceedings of the 17th annual international symposium on Computer Architecture ISCA '90,
Volume 18 Issue 3a
Publisher: ACM Press

2. 1596-1992 IEEE standard for scalable coherent interface (SCI).
E-ISBN: 0-7381-1204-6
Year: 1993
Sponsored by: IEEE Computer Society

3. The SGI Origin: A ccnuma Highly Scalable Server
Laudon, J.; Lenoski, D.;
Computer Architecture, 1997. Conference Proceedings. The 24th Annual International Symposium on
June 2-4, 1997 Page(s):241 - 251

4. The Scalable Coherence Interface
David.B.Gustavson
September 1991
"The Scalable Coherence Interface and related standard projects"

5. An analysis of the Scalable Coherent Interface
Eric Rotenberg
June 1995
An Analytical model of the SCI Coherence protocol

6. Interfaces and Standards : SCI
Behrooz. Parhami
1999
Introduction to Parallel Processing: Algorithms and Architectures

7. SCI: Scalable Coherent Interface
Hermann Hellwagner
1999
SCI: Scalable Coherent Interface - Architecture and software for high-performance Compute Clusters

= Further reading =

1. The Stanford Dash multiprocessor
Lenoski, D.; Laudon, J.; Gharachorloo, K.; Weber, W.-D.; Gupta, A.; Hennessy, J.; Horowitz, M.; Lam, M.S.;
Computer
IEEE Volume 25, Issue 3, March 1992 Page(s):63 - 79
Digital Object Identifier 10.1109/2.121510

2. SCI: Scalable Coherent Interface
Hermann Hellwagner
1999
SCI: Scalable Coherent Interface - Architecture and software for high-performance Compute Clusters

= External links =

CSC/ECE 506 Fall 2007/wiki3 2 1r

2007-10-20T03:28:25Z

Arangan: /* 4. S3.mp */

''Wiki: SCI. The IEEE Scalable Coherent Interface is a superset of the SSCI protocol we have been considering in class. A lot has been written about it, but it is still difficult to comprehend. Using SSCI as a starting point, explain why additional states are necessary, and give (or cite) examples that demonstrate how they work. Ideally, this would still be an overview of the working of the protocol, referencing more detailed documentation on the Web.''

__TOC__

= Introduction =

In parallel systems, data can be obtained from a variety of sources, which includes local or remote memory modules, secondary storage devices and network interfaces. It would be beneficial if certain aspects of mechanisms that are required to gain access to data are standardized such that it allows easily expandable and flexible designs. Various portions of a parallel system may be modified or even upgraded without any effect on the rest of the design if the system adheres to such standards. Also, utilization of standard interfaces forwards the integration of components and subsystems from several different vendors into efficient and usable parallel systems. The Scalable Coherent Interface is one such standard which facilitates the implementation of large-scale cache-coherent parallel systems.

= Scalable Coherent Interface (SCI) =

The scalable coherent interface has now become a chief hardware based approach to the cache coherence problem in shared memory multiprocessors. SCI is a directory based invalidate coherence protocol. The state of a cache block is distributed to the sharers of that block. Limits that are inherent in bus technology are easily avoided by SCI. The SCI protocol is to provide scalability, coherence and an interface. Scalability is to guarantee that the same mechanisms can be used in single processor systems and large highly parallel multiprocessors. Coherence is to guarantee efficient and integral use of cache memories in distributed shared memory. An interface provides a communication architecture that has multiple values to be brought into a single system and provide smooth inter-operation.

===Goals of SCI===

*High Performance – For distributed or parallel applications, high communication performance must be obtained when using SCI. Usual aspects of such performance would be a low latency, low CPU overhead for operations dealing with communication and high sustained throughput.

*Scalability – This is addressed in many ways. Some are
**Performance scalability – as the number of nodes being added to the system increases.
**Interconnect Distance scalability
**Memory System scalability – chiefly of cache coherence protocols, which should not have limitations to the number of processors or modules in memory that it can handle.
**Technological scalability – utilization of similar mechanisms in large, small-scale, loosely-coupled or tightly-coupled systems. Also, when new advances are made in technology, it must have the ability to readily make use of such advances.
**Economic scalability – use of similar components and mechanisms in high-end, low-volume systems as well as low-end, high-volume systems.
**Addressing capability has no short term practical limits.

*Coherent Memory System – For the purpose of reduction of Average Access Time to data, caches are becoming more important each day for microprocessors.

*Interface Characteristics – This specifies a standard interface to an interconnect that makes several devices to be connected or attached together and to interoperate.

===Operation===

In SCI, every interface does not wait for the signal to propagate before it begins to send the next signal. Also, SCI utilizes multiple links so that, concurrently, several transfers can take place.

Usually, a directory entry is in either of the two states : ‘home’ or ‘gone’. If the state is in ‘home’, then memory can immediately satisfy requests to a block as the block has not been cached by any processor. If the state is ‘gone’, then the block has been cached by a processor and might even be modified. Now, the directory contains a pointer to the first processor in the sharing list for this particular block. Hence, on requesting the data, memory returns the pointer to the first processor on the shared list rather than the data itself. The processor asking for the block now forwards its request to the processor on the top of the shared list. Now, the requesting processor adds itself into the shared list as the new head of the list.

In the SCI protocol, any coherent transaction has three phases.

Memory read – When a processor misses in its cache, It asks for the block in the home directory in memory. If the state of the memory is ‘home’, then the main memory replies with the block to the requesting processor. If the state is ‘gone’, then the main memory returns the head of the shared list of processors for that particular block. Then, memory updates its pointer and puts the requesting processor as the new head.

Cache read – When memory returns a pointer toe the requesting processor instead of data, the processor forwards its request for the block to the head of the doubly linked list. When the cache receives, the head of the list returns the data which might have been modified. The head of the list changes its backward pointer to the requesting processor’s node. Now, the requesting processor becomes the head of the list and it has the cache block.

Cleanup – If the cache miss from the requesting processor is a store, the processor has to first invalidate all other cached copies and then only proceed with the store. The new head gives an invalidate request to the next address on the list i.e. to its next processor. This processor invalidates and gives back a pointer to the next processor on its list. The head of the list uses this new pointer and sends it an invalidate request. This goes on until a NULL pointer is returned. This is the cleanup process for invalidation of other cache copies of the block.

= Simple Scalable Coherent Interface (SSCI) =

In directory-based approach, Every memory block has associated directory information; it keeps track of copies of cached blocks and their states. On a miss, it finds the directory entry, looks it up, and communicates only with the nodes that have copies (if necessary).

There are mainly two approaches: Full-bit vector: For k processors, it maintains k presence bit and 1 dirty bit at the home node. Cache state is represented the same way as in bus-based designs (MSI, MESI, etc.). It has three cache states: EM (exclusive or modified), S (shared), U (unowned). Limitation is - Number of presence bits needed grows as the number of processors.

Memory-based schemes store the information about all cached copies at the home node of the block. Cache-based schemes distribute information about copies among the copies themselves. The home contains a pointer to one cached copy of the block. Each copy contains the identity of the next node that has a copy of the block. The location of the copies is therefore determined through network transactions.

Simple SCI (SSCI) retains similarity with full-bit vector protocol: MESI states in the cache; U, S, EM states in the memory directory; It replaces the presence bits with a pointer.

== Why additional states are necessary? ==

Performance & Correctness (Coherence & Consistency) requirement for the protocol necessitates the additional states in the protocol.

=== Need for Busy state ===

On a scalable multiprocessors without coherent caches, the main-memory module determined the ordering of writes. The order that writes become visible to all processors is the order in which they reached memory.

e.g. If two processors issue read-exclusive requests for a particular word, the home will provide the requestors with the location of
the dirty node. But which request will reach the dirty node first cannot be guaranteed. This creates the need for additional busy state in the directory.

This can be solved by
# holding requests at home or requestor node and serve them in the order of their arrival
# If the block is busy, reject any further request to it & that request will be retried later
# If directory is busy forward request to dirty node & dirty node will serialize the request execution

In non-coherent scalable multiprocessors,
# For write atomicity in invalidation based protocol, current owner of block has to wait until it recieve all invalidation acks and then only it can read/write new value.
# For write completion, current owner of the block need to wait for ack from memory

In Origin, there are 3 busy states indicates that the directory has received the request but is waiting for completion hence another request for the same block won't be accepted.
# Busy read
# Busy read exclusive or upgrade
# Busy uncached read: In case of DMA, if the memory block is being read, no other processor should be allowed to get write access to it.

=== Poison state ===

To improve locality, origin relies on page migration. Hardware keeps reference counts on each page. On every access of memory count is incremented and compared with home node. If count is more than programmable threshold, hardware interrupts one of the local processors to copy the page. Block transfer engine “poison” the source page. Subsequent accesses by other processors receive a bus error and that removes the TLB entry for that location, read the location from the page table & update the TLB with new location. This approach is called - “Lazy TLB shootdown” which reduces the overall cost of migrating memory and changing the virtual-to-physical address mappings.

=== Additional states ===

Additional states can be introduced into the sharing list. The SSCI protocol has only one kind of sharing list state : dirty. More options can be introduced into the list to improve performance of the overall shared multiprocessors. Although additional states do increment the complexity of the protocol being utilized and implemented, it would be very much beneficial to the system when the overall performance of the system is increased. The question “Why Additional States are necessary” is answered alongside the introduction of each new state i.e. explanation for the necessity of each and every state has been provided.

Having the following states implemented in the protocol can improve performance.

1. “fresh” and “gone” – Main memory may still be able to respond to a requesting processor if the memory can differentiate between ‘cached and unmodified’ and ‘cached and modified’. Even if the cached value is elsewhere, memory may still be able to respond to the request if the data remains unmodified.

*The “fresh” state shows that memory still has an up-to-date copy of the block, even if the block has been cached and is being shared between processors. As long as the value remains unmodified by any of the processors sharing the data, the memory can keep responding by returning the requested data to the requesting processor.

*The “gone” state indicates that the block has been cached and has been modified. This shows that main memory cant return the data anymore, but has to return the pointer that points to the head of the list.

2. “clean lists” – Although the introduction of these states increase the complexity of handling each access, it does help with the overall performance. In this case, it may be helpful to differentiate the types of read accesses to the cache. One type of read access would be a ‘normal read-only’. Another type of read would be a ‘read with intent to modify later’. The memory in the state is ‘fresh’ when a read-fresh takes place. A read-clean leaves the memory in the ‘gone’ state. Just as with the dirty state, the read-clean allows a write to the block to proceed immediately. When a block is replaced in the cache and the cache is in the ‘clean’ state, no data needs to be written to memory. But with the dirty state, a write-back would be required even if the data had not been modified.

== Examples ==
=== 1. DASH Cache Coherence Protocol ===

DASH (Directory Architecture for SHared memory) is a scalable shared-memory multiprocessor currently being developed at Stanford’s Computer Systems Laboratory. DASH protocol uses point-to-point messages sent between the processors and memories to keep caches consistent.

The DASH coherence protocol is an invalidation-based ownership protocol. A memory block can be in one of three states as indicated by the associated directory entry: (i) uncached-remote, that is not cached by any remote cluster; (ii) shared-remote, that is cached in an unmodified state by one or more remote clusters; or (iii) dirty-remote, that is cached in a modified state by a single remote cluster.

Please see in the figures below: Left - Flow of Read Request to remote memory with directory in dirty-remote state. Right - Flow of Read-Exclusive Request to remote memory with directory in shared-remote state.

[[Image:DASH.jpg]]

Write back request: A dirty cache line that is replaced must be written back to memory. If the home of the memory block is the local cluster, then the data is simply written back to main memory. If the home cluster is remote, then a message is sent to the remote
home which updates the main memory and marks the block uncached-remote.

=== 2. SCI-based CC-NUMA machines ===

SCI is attached to the memory bus of a node. Here, SCI can take part in and "export" cache coherence and memory traffic on the bus and make the memory of the node visible to other nodes. Each node's memory address range can be scaled to span a global address space, giving coherent and transparent access to anywhere in the system to requesting processors. This method is used to form a large-scale cache-coherent shared-memory system, known as a CC-NUMA machine.

=== 3. SGI Origin 2000 ===

This would seem to be the only commercial alternative to CC-Numa machines. The Origin 2000 is an advanced version of the SCI-based CC-NUMA machines but has its origins in the Stanford DASH project. Cache coherence is maintained here with a bit-vector directory scheme.

=== 4. S3.mp ===

This was a project by SUN to connect off-the-shelf workstations and make this interconnection into a large-scale cache-coherent multiprocessor. This has an invalidation based protocol which notes the nodes that share a memory block and stores such information in linked lists. The nodes in this project were spatially distributed and were connected together using arbitrary topologies.

= References =

1 The DASH Cache Coherence Protocol
Daniel Lenoski, James Laudon, Kourosh Gharachorloo, Anoop Gupta, John Hennessy
May 1990
ACM SIGARCH Computer Architecture News , Proceedings of the 17th annual international symposium on Computer Architecture ISCA '90,
Volume 18 Issue 3a
Publisher: ACM Press

2. 1596-1992 IEEE standard for scalable coherent interface (SCI).
E-ISBN: 0-7381-1204-6
Year: 1993
Sponsored by: IEEE Computer Society

3. The SGI Origin: A ccnuma Highly Scalable Server
Laudon, J.; Lenoski, D.;
Computer Architecture, 1997. Conference Proceedings. The 24th Annual International Symposium on
June 2-4, 1997 Page(s):241 - 251

4. The Scalable Coherence Interface
David.B.Gustavson
September 1991
"The Scalable Coherence Interface and related standard projects"

5. An analysis of the Scalable Coherent Interface
Eric Rotenberg
June 1995
An Analytical model of the SCI Coherence protocol

6. Interfaces and Standards : SCI
Behrooz. Parhami
1999
Introduction to Parallel Processing: Algorithms and Architectures

7. SCI: Scalable Coherent Interface
Hermann Hellwagner
1999
SCI: Scalable Coherent Interface - Architecture and software for high-performance Compute Clusters

= Further reading =

1. The Stanford Dash multiprocessor
Lenoski, D.; Laudon, J.; Gharachorloo, K.; Weber, W.-D.; Gupta, A.; Hennessy, J.; Horowitz, M.; Lam, M.S.;
Computer
IEEE Volume 25, Issue 3, March 1992 Page(s):63 - 79
Digital Object Identifier 10.1109/2.121510

2. SCI: Scalable Coherent Interface
Hermann Hellwagner
1999
SCI: Scalable Coherent Interface - Architecture and software for high-performance Compute Clusters

= External links =

CSC/ECE 506 Fall 2007/wiki3 2 1r

2007-10-20T03:27:46Z

Arangan: /* 3. SGI Origin 2000 */

''Wiki: SCI. The IEEE Scalable Coherent Interface is a superset of the SSCI protocol we have been considering in class. A lot has been written about it, but it is still difficult to comprehend. Using SSCI as a starting point, explain why additional states are necessary, and give (or cite) examples that demonstrate how they work. Ideally, this would still be an overview of the working of the protocol, referencing more detailed documentation on the Web.''

__TOC__

= Introduction =

In parallel systems, data can be obtained from a variety of sources, which includes local or remote memory modules, secondary storage devices and network interfaces. It would be beneficial if certain aspects of mechanisms that are required to gain access to data are standardized such that it allows easily expandable and flexible designs. Various portions of a parallel system may be modified or even upgraded without any effect on the rest of the design if the system adheres to such standards. Also, utilization of standard interfaces forwards the integration of components and subsystems from several different vendors into efficient and usable parallel systems. The Scalable Coherent Interface is one such standard which facilitates the implementation of large-scale cache-coherent parallel systems.

= Scalable Coherent Interface (SCI) =

The scalable coherent interface has now become a chief hardware based approach to the cache coherence problem in shared memory multiprocessors. SCI is a directory based invalidate coherence protocol. The state of a cache block is distributed to the sharers of that block. Limits that are inherent in bus technology are easily avoided by SCI. The SCI protocol is to provide scalability, coherence and an interface. Scalability is to guarantee that the same mechanisms can be used in single processor systems and large highly parallel multiprocessors. Coherence is to guarantee efficient and integral use of cache memories in distributed shared memory. An interface provides a communication architecture that has multiple values to be brought into a single system and provide smooth inter-operation.

===Goals of SCI===

*High Performance – For distributed or parallel applications, high communication performance must be obtained when using SCI. Usual aspects of such performance would be a low latency, low CPU overhead for operations dealing with communication and high sustained throughput.

*Scalability – This is addressed in many ways. Some are
**Performance scalability – as the number of nodes being added to the system increases.
**Interconnect Distance scalability
**Memory System scalability – chiefly of cache coherence protocols, which should not have limitations to the number of processors or modules in memory that it can handle.
**Technological scalability – utilization of similar mechanisms in large, small-scale, loosely-coupled or tightly-coupled systems. Also, when new advances are made in technology, it must have the ability to readily make use of such advances.
**Economic scalability – use of similar components and mechanisms in high-end, low-volume systems as well as low-end, high-volume systems.
**Addressing capability has no short term practical limits.

*Coherent Memory System – For the purpose of reduction of Average Access Time to data, caches are becoming more important each day for microprocessors.

*Interface Characteristics – This specifies a standard interface to an interconnect that makes several devices to be connected or attached together and to interoperate.

===Operation===

In SCI, every interface does not wait for the signal to propagate before it begins to send the next signal. Also, SCI utilizes multiple links so that, concurrently, several transfers can take place.

Usually, a directory entry is in either of the two states : ‘home’ or ‘gone’. If the state is in ‘home’, then memory can immediately satisfy requests to a block as the block has not been cached by any processor. If the state is ‘gone’, then the block has been cached by a processor and might even be modified. Now, the directory contains a pointer to the first processor in the sharing list for this particular block. Hence, on requesting the data, memory returns the pointer to the first processor on the shared list rather than the data itself. The processor asking for the block now forwards its request to the processor on the top of the shared list. Now, the requesting processor adds itself into the shared list as the new head of the list.

In the SCI protocol, any coherent transaction has three phases.

Memory read – When a processor misses in its cache, It asks for the block in the home directory in memory. If the state of the memory is ‘home’, then the main memory replies with the block to the requesting processor. If the state is ‘gone’, then the main memory returns the head of the shared list of processors for that particular block. Then, memory updates its pointer and puts the requesting processor as the new head.

Cache read – When memory returns a pointer toe the requesting processor instead of data, the processor forwards its request for the block to the head of the doubly linked list. When the cache receives, the head of the list returns the data which might have been modified. The head of the list changes its backward pointer to the requesting processor’s node. Now, the requesting processor becomes the head of the list and it has the cache block.

Cleanup – If the cache miss from the requesting processor is a store, the processor has to first invalidate all other cached copies and then only proceed with the store. The new head gives an invalidate request to the next address on the list i.e. to its next processor. This processor invalidates and gives back a pointer to the next processor on its list. The head of the list uses this new pointer and sends it an invalidate request. This goes on until a NULL pointer is returned. This is the cleanup process for invalidation of other cache copies of the block.

= Simple Scalable Coherent Interface (SSCI) =

In directory-based approach, Every memory block has associated directory information; it keeps track of copies of cached blocks and their states. On a miss, it finds the directory entry, looks it up, and communicates only with the nodes that have copies (if necessary).

There are mainly two approaches: Full-bit vector: For k processors, it maintains k presence bit and 1 dirty bit at the home node. Cache state is represented the same way as in bus-based designs (MSI, MESI, etc.). It has three cache states: EM (exclusive or modified), S (shared), U (unowned). Limitation is - Number of presence bits needed grows as the number of processors.

Memory-based schemes store the information about all cached copies at the home node of the block. Cache-based schemes distribute information about copies among the copies themselves. The home contains a pointer to one cached copy of the block. Each copy contains the identity of the next node that has a copy of the block. The location of the copies is therefore determined through network transactions.

Simple SCI (SSCI) retains similarity with full-bit vector protocol: MESI states in the cache; U, S, EM states in the memory directory; It replaces the presence bits with a pointer.

== Why additional states are necessary? ==

Performance & Correctness (Coherence & Consistency) requirement for the protocol necessitates the additional states in the protocol.

=== Need for Busy state ===

On a scalable multiprocessors without coherent caches, the main-memory module determined the ordering of writes. The order that writes become visible to all processors is the order in which they reached memory.

e.g. If two processors issue read-exclusive requests for a particular word, the home will provide the requestors with the location of
the dirty node. But which request will reach the dirty node first cannot be guaranteed. This creates the need for additional busy state in the directory.

This can be solved by
# holding requests at home or requestor node and serve them in the order of their arrival
# If the block is busy, reject any further request to it & that request will be retried later
# If directory is busy forward request to dirty node & dirty node will serialize the request execution

In non-coherent scalable multiprocessors,
# For write atomicity in invalidation based protocol, current owner of block has to wait until it recieve all invalidation acks and then only it can read/write new value.
# For write completion, current owner of the block need to wait for ack from memory

In Origin, there are 3 busy states indicates that the directory has received the request but is waiting for completion hence another request for the same block won't be accepted.
# Busy read
# Busy read exclusive or upgrade
# Busy uncached read: In case of DMA, if the memory block is being read, no other processor should be allowed to get write access to it.

=== Poison state ===

To improve locality, origin relies on page migration. Hardware keeps reference counts on each page. On every access of memory count is incremented and compared with home node. If count is more than programmable threshold, hardware interrupts one of the local processors to copy the page. Block transfer engine “poison” the source page. Subsequent accesses by other processors receive a bus error and that removes the TLB entry for that location, read the location from the page table & update the TLB with new location. This approach is called - “Lazy TLB shootdown” which reduces the overall cost of migrating memory and changing the virtual-to-physical address mappings.

=== Additional states ===

Additional states can be introduced into the sharing list. The SSCI protocol has only one kind of sharing list state : dirty. More options can be introduced into the list to improve performance of the overall shared multiprocessors. Although additional states do increment the complexity of the protocol being utilized and implemented, it would be very much beneficial to the system when the overall performance of the system is increased. The question “Why Additional States are necessary” is answered alongside the introduction of each new state i.e. explanation for the necessity of each and every state has been provided.

Having the following states implemented in the protocol can improve performance.

1. “fresh” and “gone” – Main memory may still be able to respond to a requesting processor if the memory can differentiate between ‘cached and unmodified’ and ‘cached and modified’. Even if the cached value is elsewhere, memory may still be able to respond to the request if the data remains unmodified.

*The “fresh” state shows that memory still has an up-to-date copy of the block, even if the block has been cached and is being shared between processors. As long as the value remains unmodified by any of the processors sharing the data, the memory can keep responding by returning the requested data to the requesting processor.

*The “gone” state indicates that the block has been cached and has been modified. This shows that main memory cant return the data anymore, but has to return the pointer that points to the head of the list.

2. “clean lists” – Although the introduction of these states increase the complexity of handling each access, it does help with the overall performance. In this case, it may be helpful to differentiate the types of read accesses to the cache. One type of read access would be a ‘normal read-only’. Another type of read would be a ‘read with intent to modify later’. The memory in the state is ‘fresh’ when a read-fresh takes place. A read-clean leaves the memory in the ‘gone’ state. Just as with the dirty state, the read-clean allows a write to the block to proceed immediately. When a block is replaced in the cache and the cache is in the ‘clean’ state, no data needs to be written to memory. But with the dirty state, a write-back would be required even if the data had not been modified.

== Examples ==
=== 1. DASH Cache Coherence Protocol ===

DASH (Directory Architecture for SHared memory) is a scalable shared-memory multiprocessor currently being developed at Stanford’s Computer Systems Laboratory. DASH protocol uses point-to-point messages sent between the processors and memories to keep caches consistent.

The DASH coherence protocol is an invalidation-based ownership protocol. A memory block can be in one of three states as indicated by the associated directory entry: (i) uncached-remote, that is not cached by any remote cluster; (ii) shared-remote, that is cached in an unmodified state by one or more remote clusters; or (iii) dirty-remote, that is cached in a modified state by a single remote cluster.

Please see in the figures below: Left - Flow of Read Request to remote memory with directory in dirty-remote state. Right - Flow of Read-Exclusive Request to remote memory with directory in shared-remote state.

[[Image:DASH.jpg]]

Write back request: A dirty cache line that is replaced must be written back to memory. If the home of the memory block is the local cluster, then the data is simply written back to main memory. If the home cluster is remote, then a message is sent to the remote
home which updates the main memory and marks the block uncached-remote.

=== 2. SCI-based CC-NUMA machines ===

SCI is attached to the memory bus of a node. Here, SCI can take part in and "export" cache coherence and memory traffic on the bus and make the memory of the node visible to other nodes. Each node's memory address range can be scaled to span a global address space, giving coherent and transparent access to anywhere in the system to requesting processors. This method is used to form a large-scale cache-coherent shared-memory system, known as a CC-NUMA machine.

=== 3. SGI Origin 2000 ===

This would seem to be the only commercial alternative to CC-Numa machines. The Origin 2000 is an advanced version of the SCI-based CC-NUMA machines but has its origins in the Stanford DASH project. Cache coherence is maintained here with a bit-vector directory scheme.

=== 4. S3.mp ===

This was a project by SUN to connect off-the-shelf workstations and make this interconnection into a large-scale cache-coherent multiprocessor. This has an invalidation based protocol which notes the nodes that ahre a memory block and stores such information in linked lists. The nodes in this project were spatially distributed and were connected together using arbitrary topologies.

= References =

1 The DASH Cache Coherence Protocol
Daniel Lenoski, James Laudon, Kourosh Gharachorloo, Anoop Gupta, John Hennessy
May 1990
ACM SIGARCH Computer Architecture News , Proceedings of the 17th annual international symposium on Computer Architecture ISCA '90,
Volume 18 Issue 3a
Publisher: ACM Press

2. 1596-1992 IEEE standard for scalable coherent interface (SCI).
E-ISBN: 0-7381-1204-6
Year: 1993
Sponsored by: IEEE Computer Society

3. The SGI Origin: A ccnuma Highly Scalable Server
Laudon, J.; Lenoski, D.;
Computer Architecture, 1997. Conference Proceedings. The 24th Annual International Symposium on
June 2-4, 1997 Page(s):241 - 251

4. The Scalable Coherence Interface
David.B.Gustavson
September 1991
"The Scalable Coherence Interface and related standard projects"

5. An analysis of the Scalable Coherent Interface
Eric Rotenberg
June 1995
An Analytical model of the SCI Coherence protocol

6. Interfaces and Standards : SCI
Behrooz. Parhami
1999
Introduction to Parallel Processing: Algorithms and Architectures

7. SCI: Scalable Coherent Interface
Hermann Hellwagner
1999
SCI: Scalable Coherent Interface - Architecture and software for high-performance Compute Clusters

= Further reading =

1. The Stanford Dash multiprocessor
Lenoski, D.; Laudon, J.; Gharachorloo, K.; Weber, W.-D.; Gupta, A.; Hennessy, J.; Horowitz, M.; Lam, M.S.;
Computer
IEEE Volume 25, Issue 3, March 1992 Page(s):63 - 79
Digital Object Identifier 10.1109/2.121510

2. SCI: Scalable Coherent Interface
Hermann Hellwagner
1999
SCI: Scalable Coherent Interface - Architecture and software for high-performance Compute Clusters

= External links =

CSC/ECE 506 Fall 2007/wiki3 2 1r

2007-10-20T03:27:04Z

Arangan: /* Examples */

''Wiki: SCI. The IEEE Scalable Coherent Interface is a superset of the SSCI protocol we have been considering in class. A lot has been written about it, but it is still difficult to comprehend. Using SSCI as a starting point, explain why additional states are necessary, and give (or cite) examples that demonstrate how they work. Ideally, this would still be an overview of the working of the protocol, referencing more detailed documentation on the Web.''

__TOC__

= Introduction =

In parallel systems, data can be obtained from a variety of sources, which includes local or remote memory modules, secondary storage devices and network interfaces. It would be beneficial if certain aspects of mechanisms that are required to gain access to data are standardized such that it allows easily expandable and flexible designs. Various portions of a parallel system may be modified or even upgraded without any effect on the rest of the design if the system adheres to such standards. Also, utilization of standard interfaces forwards the integration of components and subsystems from several different vendors into efficient and usable parallel systems. The Scalable Coherent Interface is one such standard which facilitates the implementation of large-scale cache-coherent parallel systems.

= Scalable Coherent Interface (SCI) =

The scalable coherent interface has now become a chief hardware based approach to the cache coherence problem in shared memory multiprocessors. SCI is a directory based invalidate coherence protocol. The state of a cache block is distributed to the sharers of that block. Limits that are inherent in bus technology are easily avoided by SCI. The SCI protocol is to provide scalability, coherence and an interface. Scalability is to guarantee that the same mechanisms can be used in single processor systems and large highly parallel multiprocessors. Coherence is to guarantee efficient and integral use of cache memories in distributed shared memory. An interface provides a communication architecture that has multiple values to be brought into a single system and provide smooth inter-operation.

===Goals of SCI===

*High Performance – For distributed or parallel applications, high communication performance must be obtained when using SCI. Usual aspects of such performance would be a low latency, low CPU overhead for operations dealing with communication and high sustained throughput.

*Scalability – This is addressed in many ways. Some are
**Performance scalability – as the number of nodes being added to the system increases.
**Interconnect Distance scalability
**Memory System scalability – chiefly of cache coherence protocols, which should not have limitations to the number of processors or modules in memory that it can handle.
**Technological scalability – utilization of similar mechanisms in large, small-scale, loosely-coupled or tightly-coupled systems. Also, when new advances are made in technology, it must have the ability to readily make use of such advances.
**Economic scalability – use of similar components and mechanisms in high-end, low-volume systems as well as low-end, high-volume systems.
**Addressing capability has no short term practical limits.

*Coherent Memory System – For the purpose of reduction of Average Access Time to data, caches are becoming more important each day for microprocessors.

*Interface Characteristics – This specifies a standard interface to an interconnect that makes several devices to be connected or attached together and to interoperate.

===Operation===

In SCI, every interface does not wait for the signal to propagate before it begins to send the next signal. Also, SCI utilizes multiple links so that, concurrently, several transfers can take place.

Usually, a directory entry is in either of the two states : ‘home’ or ‘gone’. If the state is in ‘home’, then memory can immediately satisfy requests to a block as the block has not been cached by any processor. If the state is ‘gone’, then the block has been cached by a processor and might even be modified. Now, the directory contains a pointer to the first processor in the sharing list for this particular block. Hence, on requesting the data, memory returns the pointer to the first processor on the shared list rather than the data itself. The processor asking for the block now forwards its request to the processor on the top of the shared list. Now, the requesting processor adds itself into the shared list as the new head of the list.

In the SCI protocol, any coherent transaction has three phases.

Memory read – When a processor misses in its cache, It asks for the block in the home directory in memory. If the state of the memory is ‘home’, then the main memory replies with the block to the requesting processor. If the state is ‘gone’, then the main memory returns the head of the shared list of processors for that particular block. Then, memory updates its pointer and puts the requesting processor as the new head.

Cache read – When memory returns a pointer toe the requesting processor instead of data, the processor forwards its request for the block to the head of the doubly linked list. When the cache receives, the head of the list returns the data which might have been modified. The head of the list changes its backward pointer to the requesting processor’s node. Now, the requesting processor becomes the head of the list and it has the cache block.

Cleanup – If the cache miss from the requesting processor is a store, the processor has to first invalidate all other cached copies and then only proceed with the store. The new head gives an invalidate request to the next address on the list i.e. to its next processor. This processor invalidates and gives back a pointer to the next processor on its list. The head of the list uses this new pointer and sends it an invalidate request. This goes on until a NULL pointer is returned. This is the cleanup process for invalidation of other cache copies of the block.

= Simple Scalable Coherent Interface (SSCI) =

In directory-based approach, Every memory block has associated directory information; it keeps track of copies of cached blocks and their states. On a miss, it finds the directory entry, looks it up, and communicates only with the nodes that have copies (if necessary).

There are mainly two approaches: Full-bit vector: For k processors, it maintains k presence bit and 1 dirty bit at the home node. Cache state is represented the same way as in bus-based designs (MSI, MESI, etc.). It has three cache states: EM (exclusive or modified), S (shared), U (unowned). Limitation is - Number of presence bits needed grows as the number of processors.

Memory-based schemes store the information about all cached copies at the home node of the block. Cache-based schemes distribute information about copies among the copies themselves. The home contains a pointer to one cached copy of the block. Each copy contains the identity of the next node that has a copy of the block. The location of the copies is therefore determined through network transactions.

Simple SCI (SSCI) retains similarity with full-bit vector protocol: MESI states in the cache; U, S, EM states in the memory directory; It replaces the presence bits with a pointer.

== Why additional states are necessary? ==

Performance & Correctness (Coherence & Consistency) requirement for the protocol necessitates the additional states in the protocol.

=== Need for Busy state ===

On a scalable multiprocessors without coherent caches, the main-memory module determined the ordering of writes. The order that writes become visible to all processors is the order in which they reached memory.

e.g. If two processors issue read-exclusive requests for a particular word, the home will provide the requestors with the location of
the dirty node. But which request will reach the dirty node first cannot be guaranteed. This creates the need for additional busy state in the directory.

This can be solved by
# holding requests at home or requestor node and serve them in the order of their arrival
# If the block is busy, reject any further request to it & that request will be retried later
# If directory is busy forward request to dirty node & dirty node will serialize the request execution

In non-coherent scalable multiprocessors,
# For write atomicity in invalidation based protocol, current owner of block has to wait until it recieve all invalidation acks and then only it can read/write new value.
# For write completion, current owner of the block need to wait for ack from memory

In Origin, there are 3 busy states indicates that the directory has received the request but is waiting for completion hence another request for the same block won't be accepted.
# Busy read
# Busy read exclusive or upgrade
# Busy uncached read: In case of DMA, if the memory block is being read, no other processor should be allowed to get write access to it.

=== Poison state ===

To improve locality, origin relies on page migration. Hardware keeps reference counts on each page. On every access of memory count is incremented and compared with home node. If count is more than programmable threshold, hardware interrupts one of the local processors to copy the page. Block transfer engine “poison” the source page. Subsequent accesses by other processors receive a bus error and that removes the TLB entry for that location, read the location from the page table & update the TLB with new location. This approach is called - “Lazy TLB shootdown” which reduces the overall cost of migrating memory and changing the virtual-to-physical address mappings.

=== Additional states ===

Additional states can be introduced into the sharing list. The SSCI protocol has only one kind of sharing list state : dirty. More options can be introduced into the list to improve performance of the overall shared multiprocessors. Although additional states do increment the complexity of the protocol being utilized and implemented, it would be very much beneficial to the system when the overall performance of the system is increased. The question “Why Additional States are necessary” is answered alongside the introduction of each new state i.e. explanation for the necessity of each and every state has been provided.

Having the following states implemented in the protocol can improve performance.

1. “fresh” and “gone” – Main memory may still be able to respond to a requesting processor if the memory can differentiate between ‘cached and unmodified’ and ‘cached and modified’. Even if the cached value is elsewhere, memory may still be able to respond to the request if the data remains unmodified.

*The “fresh” state shows that memory still has an up-to-date copy of the block, even if the block has been cached and is being shared between processors. As long as the value remains unmodified by any of the processors sharing the data, the memory can keep responding by returning the requested data to the requesting processor.

*The “gone” state indicates that the block has been cached and has been modified. This shows that main memory cant return the data anymore, but has to return the pointer that points to the head of the list.

2. “clean lists” – Although the introduction of these states increase the complexity of handling each access, it does help with the overall performance. In this case, it may be helpful to differentiate the types of read accesses to the cache. One type of read access would be a ‘normal read-only’. Another type of read would be a ‘read with intent to modify later’. The memory in the state is ‘fresh’ when a read-fresh takes place. A read-clean leaves the memory in the ‘gone’ state. Just as with the dirty state, the read-clean allows a write to the block to proceed immediately. When a block is replaced in the cache and the cache is in the ‘clean’ state, no data needs to be written to memory. But with the dirty state, a write-back would be required even if the data had not been modified.

== Examples ==
=== 1. DASH Cache Coherence Protocol ===

DASH (Directory Architecture for SHared memory) is a scalable shared-memory multiprocessor currently being developed at Stanford’s Computer Systems Laboratory. DASH protocol uses point-to-point messages sent between the processors and memories to keep caches consistent.

The DASH coherence protocol is an invalidation-based ownership protocol. A memory block can be in one of three states as indicated by the associated directory entry: (i) uncached-remote, that is not cached by any remote cluster; (ii) shared-remote, that is cached in an unmodified state by one or more remote clusters; or (iii) dirty-remote, that is cached in a modified state by a single remote cluster.

Please see in the figures below: Left - Flow of Read Request to remote memory with directory in dirty-remote state. Right - Flow of Read-Exclusive Request to remote memory with directory in shared-remote state.

[[Image:DASH.jpg]]

Write back request: A dirty cache line that is replaced must be written back to memory. If the home of the memory block is the local cluster, then the data is simply written back to main memory. If the home cluster is remote, then a message is sent to the remote
home which updates the main memory and marks the block uncached-remote.

=== 2. SCI-based CC-NUMA machines ===

SCI is attached to the memory bus of a node. Here, SCI can take part in and "export" cache coherence and memory traffic on the bus and make the memory of the node visible to other nodes. Each node's memory address range can be scaled to span a global address space, giving coherent and transparent access to anywhere in the system to requesting processors. This method is used to form a large-scale cache-coherent shared-memory system, known as a CC-NUMA machine.

=== 3. SGI Origin 2000 ===

This would seem to be the only commercial alternative to CC-Numa machines. The Origin 2000 is an advanced version of the SCI-ased CC-NUMA machines but has its origins in the Stanford DASH project. Cache coherence is maintained here with a bit-vector directory scheme.

=== 4. S3.mp ===

This was a project by SUN to connect off-the-shelf workstations and make this interconnection into a large-scale cache-coherent multiprocessor. This has an invalidation based protocol which notes the nodes that ahre a memory block and stores such information in linked lists. The nodes in this project were spatially distributed and were connected together using arbitrary topologies.

= References =

1 The DASH Cache Coherence Protocol
Daniel Lenoski, James Laudon, Kourosh Gharachorloo, Anoop Gupta, John Hennessy
May 1990
ACM SIGARCH Computer Architecture News , Proceedings of the 17th annual international symposium on Computer Architecture ISCA '90,
Volume 18 Issue 3a
Publisher: ACM Press

2. 1596-1992 IEEE standard for scalable coherent interface (SCI).
E-ISBN: 0-7381-1204-6
Year: 1993
Sponsored by: IEEE Computer Society

3. The SGI Origin: A ccnuma Highly Scalable Server
Laudon, J.; Lenoski, D.;
Computer Architecture, 1997. Conference Proceedings. The 24th Annual International Symposium on
June 2-4, 1997 Page(s):241 - 251

4. The Scalable Coherence Interface
David.B.Gustavson
September 1991
"The Scalable Coherence Interface and related standard projects"

5. An analysis of the Scalable Coherent Interface
Eric Rotenberg
June 1995
An Analytical model of the SCI Coherence protocol

6. Interfaces and Standards : SCI
Behrooz. Parhami
1999
Introduction to Parallel Processing: Algorithms and Architectures

7. SCI: Scalable Coherent Interface
Hermann Hellwagner
1999
SCI: Scalable Coherent Interface - Architecture and software for high-performance Compute Clusters

= Further reading =

1. The Stanford Dash multiprocessor
Lenoski, D.; Laudon, J.; Gharachorloo, K.; Weber, W.-D.; Gupta, A.; Hennessy, J.; Horowitz, M.; Lam, M.S.;
Computer
IEEE Volume 25, Issue 3, March 1992 Page(s):63 - 79
Digital Object Identifier 10.1109/2.121510

2. SCI: Scalable Coherent Interface
Hermann Hellwagner
1999
SCI: Scalable Coherent Interface - Architecture and software for high-performance Compute Clusters

= External links =

CSC/ECE 506 Fall 2007/wiki3 2 1r

2007-10-20T03:16:51Z

Arangan: /* 1 DASH Cache Coherence Protocol */

''Wiki: SCI. The IEEE Scalable Coherent Interface is a superset of the SSCI protocol we have been considering in class. A lot has been written about it, but it is still difficult to comprehend. Using SSCI as a starting point, explain why additional states are necessary, and give (or cite) examples that demonstrate how they work. Ideally, this would still be an overview of the working of the protocol, referencing more detailed documentation on the Web.''

__TOC__

= Introduction =

In parallel systems, data can be obtained from a variety of sources, which includes local or remote memory modules, secondary storage devices and network interfaces. It would be beneficial if certain aspects of mechanisms that are required to gain access to data are standardized such that it allows easily expandable and flexible designs. Various portions of a parallel system may be modified or even upgraded without any effect on the rest of the design if the system adheres to such standards. Also, utilization of standard interfaces forwards the integration of components and subsystems from several different vendors into efficient and usable parallel systems. The Scalable Coherent Interface is one such standard which facilitates the implementation of large-scale cache-coherent parallel systems.

= Scalable Coherent Interface (SCI) =

The scalable coherent interface has now become a chief hardware based approach to the cache coherence problem in shared memory multiprocessors. SCI is a directory based invalidate coherence protocol. The state of a cache block is distributed to the sharers of that block. Limits that are inherent in bus technology are easily avoided by SCI. The SCI protocol is to provide scalability, coherence and an interface. Scalability is to guarantee that the same mechanisms can be used in single processor systems and large highly parallel multiprocessors. Coherence is to guarantee efficient and integral use of cache memories in distributed shared memory. An interface provides a communication architecture that has multiple values to be brought into a single system and provide smooth inter-operation.

===Goals of SCI===

*High Performance – For distributed or parallel applications, high communication performance must be obtained when using SCI. Usual aspects of such performance would be a low latency, low CPU overhead for operations dealing with communication and high sustained throughput.

*Scalability – This is addressed in many ways. Some are
**Performance scalability – as the number of nodes being added to the system increases.
**Interconnect Distance scalability
**Memory System scalability – chiefly of cache coherence protocols, which should not have limitations to the number of processors or modules in memory that it can handle.
**Technological scalability – utilization of similar mechanisms in large, small-scale, loosely-coupled or tightly-coupled systems. Also, when new advances are made in technology, it must have the ability to readily make use of such advances.
**Economic scalability – use of similar components and mechanisms in high-end, low-volume systems as well as low-end, high-volume systems.
**Addressing capability has no short term practical limits.

*Coherent Memory System – For the purpose of reduction of Average Access Time to data, caches are becoming more important each day for microprocessors.

*Interface Characteristics – This specifies a standard interface to an interconnect that makes several devices to be connected or attached together and to interoperate.

===Operation===

In SCI, every interface does not wait for the signal to propagate before it begins to send the next signal. Also, SCI utilizes multiple links so that, concurrently, several transfers can take place.

Usually, a directory entry is in either of the two states : ‘home’ or ‘gone’. If the state is in ‘home’, then memory can immediately satisfy requests to a block as the block has not been cached by any processor. If the state is ‘gone’, then the block has been cached by a processor and might even be modified. Now, the directory contains a pointer to the first processor in the sharing list for this particular block. Hence, on requesting the data, memory returns the pointer to the first processor on the shared list rather than the data itself. The processor asking for the block now forwards its request to the processor on the top of the shared list. Now, the requesting processor adds itself into the shared list as the new head of the list.

In the SCI protocol, any coherent transaction has three phases.

Memory read – When a processor misses in its cache, It asks for the block in the home directory in memory. If the state of the memory is ‘home’, then the main memory replies with the block to the requesting processor. If the state is ‘gone’, then the main memory returns the head of the shared list of processors for that particular block. Then, memory updates its pointer and puts the requesting processor as the new head.

Cache read – When memory returns a pointer toe the requesting processor instead of data, the processor forwards its request for the block to the head of the doubly linked list. When the cache receives, the head of the list returns the data which might have been modified. The head of the list changes its backward pointer to the requesting processor’s node. Now, the requesting processor becomes the head of the list and it has the cache block.

Cleanup – If the cache miss from the requesting processor is a store, the processor has to first invalidate all other cached copies and then only proceed with the store. The new head gives an invalidate request to the next address on the list i.e. to its next processor. This processor invalidates and gives back a pointer to the next processor on its list. The head of the list uses this new pointer and sends it an invalidate request. This goes on until a NULL pointer is returned. This is the cleanup process for invalidation of other cache copies of the block.

= Simple Scalable Coherent Interface (SSCI) =

In directory-based approach, Every memory block has associated directory information; it keeps track of copies of cached blocks and their states. On a miss, it finds the directory entry, looks it up, and communicates only with the nodes that have copies (if necessary).

There are mainly two approaches: Full-bit vector: For k processors, it maintains k presence bit and 1 dirty bit at the home node. Cache state is represented the same way as in bus-based designs (MSI, MESI, etc.). It has three cache states: EM (exclusive or modified), S (shared), U (unowned). Limitation is - Number of presence bits needed grows as the number of processors.

Memory-based schemes store the information about all cached copies at the home node of the block. Cache-based schemes distribute information about copies among the copies themselves. The home contains a pointer to one cached copy of the block. Each copy contains the identity of the next node that has a copy of the block. The location of the copies is therefore determined through network transactions.

Simple SCI (SSCI) retains similarity with full-bit vector protocol: MESI states in the cache; U, S, EM states in the memory directory; It replaces the presence bits with a pointer.

== Why additional states are necessary? ==

Performance & Correctness (Coherence & Consistency) requirement for the protocol necessitates the additional states in the protocol.

=== Need for Busy state ===

On a scalable multiprocessors without coherent caches, the main-memory module determined the ordering of writes. The order that writes become visible to all processors is the order in which they reached memory.

e.g. If two processors issue read-exclusive requests for a particular word, the home will provide the requestors with the location of
the dirty node. But which request will reach the dirty node first cannot be guaranteed. This creates the need for additional busy state in the directory.

This can be solved by
# holding requests at home or requestor node and serve them in the order of their arrival
# If the block is busy, reject any further request to it & that request will be retried later
# If directory is busy forward request to dirty node & dirty node will serialize the request execution

In non-coherent scalable multiprocessors,
# For write atomicity in invalidation based protocol, current owner of block has to wait until it recieve all invalidation acks and then only it can read/write new value.
# For write completion, current owner of the block need to wait for ack from memory

In Origin, there are 3 busy states indicates that the directory has received the request but is waiting for completion hence another request for the same block won't be accepted.
# Busy read
# Busy read exclusive or upgrade
# Busy uncached read: In case of DMA, if the memory block is being read, no other processor should be allowed to get write access to it.

=== Poison state ===

To improve locality, origin relies on page migration. Hardware keeps reference counts on each page. On every access of memory count is incremented and compared with home node. If count is more than programmable threshold, hardware interrupts one of the local processors to copy the page. Block transfer engine “poison” the source page. Subsequent accesses by other processors receive a bus error and that removes the TLB entry for that location, read the location from the page table & update the TLB with new location. This approach is called - “Lazy TLB shootdown” which reduces the overall cost of migrating memory and changing the virtual-to-physical address mappings.

=== Additional states ===

Additional states can be introduced into the sharing list. The SSCI protocol has only one kind of sharing list state : dirty. More options can be introduced into the list to improve performance of the overall shared multiprocessors. Although additional states do increment the complexity of the protocol being utilized and implemented, it would be very much beneficial to the system when the overall performance of the system is increased. The question “Why Additional States are necessary” is answered alongside the introduction of each new state i.e. explanation for the necessity of each and every state has been provided.

Having the following states implemented in the protocol can improve performance.

1. “fresh” and “gone” – Main memory may still be able to respond to a requesting processor if the memory can differentiate between ‘cached and unmodified’ and ‘cached and modified’. Even if the cached value is elsewhere, memory may still be able to respond to the request if the data remains unmodified.

*The “fresh” state shows that memory still has an up-to-date copy of the block, even if the block has been cached and is being shared between processors. As long as the value remains unmodified by any of the processors sharing the data, the memory can keep responding by returning the requested data to the requesting processor.

*The “gone” state indicates that the block has been cached and has been modified. This shows that main memory cant return the data anymore, but has to return the pointer that points to the head of the list.

2. “clean lists” – Although the introduction of these states increase the complexity of handling each access, it does help with the overall performance. In this case, it may be helpful to differentiate the types of read accesses to the cache. One type of read access would be a ‘normal read-only’. Another type of read would be a ‘read with intent to modify later’. The memory in the state is ‘fresh’ when a read-fresh takes place. A read-clean leaves the memory in the ‘gone’ state. Just as with the dirty state, the read-clean allows a write to the block to proceed immediately. When a block is replaced in the cache and the cache is in the ‘clean’ state, no data needs to be written to memory. But with the dirty state, a write-back would be required even if the data had not been modified.

== Examples ==
=== 1. DASH Cache Coherence Protocol ===

DASH (Directory Architecture for SHared memory) is a scalable shared-memory multiprocessor currently being developed at Stanford’s Computer Systems Laboratory. DASH protocol uses point-to-point messages sent between the processors and memories to keep caches consistent.

The DASH coherence protocol is an invalidation-based ownership protocol. A memory block can be in one of three states as indicated by the associated directory entry: (i) uncached-remote, that is not cached by any remote cluster; (ii) shared-remote, that is cached in an unmodified state by one or more remote clusters; or (iii) dirty-remote, that is cached in a modified state by a single remote cluster.

Please see in the figures below: Left - Flow of Read Request to remote memory with directory in dirty-remote state. Right - Flow of Read-Exclusive Request to remote memory with directory in shared-remote state.

[[Image:DASH.jpg]]

Write back request: A dirty cache line that is replaced must be written back to memory. If the home of the memory block is the local cluster, then the data is simply written back to main memory. If the home cluster is remote, then a message is sent to the remote
home which updates the main memory and marks the block uncached-remote.

=== 2. SCI-based CC-NUMA machines ===

SCI is attached to the memory bus of a node. Here, SCI can take part in and "export" cache coherence and memory traffic on the bus and make the memory of the node visible to other nodes. Each node's memory address range can be scaled to span a global address space, giving coherent and transparent access to anywhere in the system to requesting processors. This method is used to form a large-scale cache-coherent shared-memory system, known as a CC-NUMA machine.

= References =

1 The DASH Cache Coherence Protocol
Daniel Lenoski, James Laudon, Kourosh Gharachorloo, Anoop Gupta, John Hennessy
May 1990
ACM SIGARCH Computer Architecture News , Proceedings of the 17th annual international symposium on Computer Architecture ISCA '90,
Volume 18 Issue 3a
Publisher: ACM Press

2. 1596-1992 IEEE standard for scalable coherent interface (SCI).
E-ISBN: 0-7381-1204-6
Year: 1993
Sponsored by: IEEE Computer Society

3. The SGI Origin: A ccnuma Highly Scalable Server
Laudon, J.; Lenoski, D.;
Computer Architecture, 1997. Conference Proceedings. The 24th Annual International Symposium on
June 2-4, 1997 Page(s):241 - 251

4. The Scalable Coherence Interface
David.B.Gustavson
September 1991
"The Scalable Coherence Interface and related standard projects"

5. An analysis of the Scalable Coherent Interface
Eric Rotenberg
June 1995
An Analytical model of the SCI Coherence protocol

6. Interfaces and Standards : SCI
Behrooz. Parhami
1999
Introduction to Parallel Processing: Algorithms and Architectures

7. SCI: Scalable Coherent Interface
Hermann Hellwagner
1999
SCI: Scalable Coherent Interface - Architecture and software for high-performance Compute Clusters

= Further reading =

1. The Stanford Dash multiprocessor
Lenoski, D.; Laudon, J.; Gharachorloo, K.; Weber, W.-D.; Gupta, A.; Hennessy, J.; Horowitz, M.; Lam, M.S.;
Computer
IEEE Volume 25, Issue 3, March 1992 Page(s):63 - 79
Digital Object Identifier 10.1109/2.121510

2. SCI: Scalable Coherent Interface
Hermann Hellwagner
1999
SCI: Scalable Coherent Interface - Architecture and software for high-performance Compute Clusters

= External links =

CSC/ECE 506 Fall 2007/wiki3 2 1r

2007-10-20T03:16:22Z

Arangan: /* SCI-based CC-NUMA machines */

''Wiki: SCI. The IEEE Scalable Coherent Interface is a superset of the SSCI protocol we have been considering in class. A lot has been written about it, but it is still difficult to comprehend. Using SSCI as a starting point, explain why additional states are necessary, and give (or cite) examples that demonstrate how they work. Ideally, this would still be an overview of the working of the protocol, referencing more detailed documentation on the Web.''

__TOC__

= Introduction =

In parallel systems, data can be obtained from a variety of sources, which includes local or remote memory modules, secondary storage devices and network interfaces. It would be beneficial if certain aspects of mechanisms that are required to gain access to data are standardized such that it allows easily expandable and flexible designs. Various portions of a parallel system may be modified or even upgraded without any effect on the rest of the design if the system adheres to such standards. Also, utilization of standard interfaces forwards the integration of components and subsystems from several different vendors into efficient and usable parallel systems. The Scalable Coherent Interface is one such standard which facilitates the implementation of large-scale cache-coherent parallel systems.

= Scalable Coherent Interface (SCI) =

The scalable coherent interface has now become a chief hardware based approach to the cache coherence problem in shared memory multiprocessors. SCI is a directory based invalidate coherence protocol. The state of a cache block is distributed to the sharers of that block. Limits that are inherent in bus technology are easily avoided by SCI. The SCI protocol is to provide scalability, coherence and an interface. Scalability is to guarantee that the same mechanisms can be used in single processor systems and large highly parallel multiprocessors. Coherence is to guarantee efficient and integral use of cache memories in distributed shared memory. An interface provides a communication architecture that has multiple values to be brought into a single system and provide smooth inter-operation.

===Goals of SCI===

*High Performance – For distributed or parallel applications, high communication performance must be obtained when using SCI. Usual aspects of such performance would be a low latency, low CPU overhead for operations dealing with communication and high sustained throughput.

*Scalability – This is addressed in many ways. Some are
**Performance scalability – as the number of nodes being added to the system increases.
**Interconnect Distance scalability
**Memory System scalability – chiefly of cache coherence protocols, which should not have limitations to the number of processors or modules in memory that it can handle.
**Technological scalability – utilization of similar mechanisms in large, small-scale, loosely-coupled or tightly-coupled systems. Also, when new advances are made in technology, it must have the ability to readily make use of such advances.
**Economic scalability – use of similar components and mechanisms in high-end, low-volume systems as well as low-end, high-volume systems.
**Addressing capability has no short term practical limits.

*Coherent Memory System – For the purpose of reduction of Average Access Time to data, caches are becoming more important each day for microprocessors.

*Interface Characteristics – This specifies a standard interface to an interconnect that makes several devices to be connected or attached together and to interoperate.

===Operation===

In SCI, every interface does not wait for the signal to propagate before it begins to send the next signal. Also, SCI utilizes multiple links so that, concurrently, several transfers can take place.

Usually, a directory entry is in either of the two states : ‘home’ or ‘gone’. If the state is in ‘home’, then memory can immediately satisfy requests to a block as the block has not been cached by any processor. If the state is ‘gone’, then the block has been cached by a processor and might even be modified. Now, the directory contains a pointer to the first processor in the sharing list for this particular block. Hence, on requesting the data, memory returns the pointer to the first processor on the shared list rather than the data itself. The processor asking for the block now forwards its request to the processor on the top of the shared list. Now, the requesting processor adds itself into the shared list as the new head of the list.

In the SCI protocol, any coherent transaction has three phases.

Memory read – When a processor misses in its cache, It asks for the block in the home directory in memory. If the state of the memory is ‘home’, then the main memory replies with the block to the requesting processor. If the state is ‘gone’, then the main memory returns the head of the shared list of processors for that particular block. Then, memory updates its pointer and puts the requesting processor as the new head.

Cache read – When memory returns a pointer toe the requesting processor instead of data, the processor forwards its request for the block to the head of the doubly linked list. When the cache receives, the head of the list returns the data which might have been modified. The head of the list changes its backward pointer to the requesting processor’s node. Now, the requesting processor becomes the head of the list and it has the cache block.

Cleanup – If the cache miss from the requesting processor is a store, the processor has to first invalidate all other cached copies and then only proceed with the store. The new head gives an invalidate request to the next address on the list i.e. to its next processor. This processor invalidates and gives back a pointer to the next processor on its list. The head of the list uses this new pointer and sends it an invalidate request. This goes on until a NULL pointer is returned. This is the cleanup process for invalidation of other cache copies of the block.

= Simple Scalable Coherent Interface (SSCI) =

In directory-based approach, Every memory block has associated directory information; it keeps track of copies of cached blocks and their states. On a miss, it finds the directory entry, looks it up, and communicates only with the nodes that have copies (if necessary).

There are mainly two approaches: Full-bit vector: For k processors, it maintains k presence bit and 1 dirty bit at the home node. Cache state is represented the same way as in bus-based designs (MSI, MESI, etc.). It has three cache states: EM (exclusive or modified), S (shared), U (unowned). Limitation is - Number of presence bits needed grows as the number of processors.

Memory-based schemes store the information about all cached copies at the home node of the block. Cache-based schemes distribute information about copies among the copies themselves. The home contains a pointer to one cached copy of the block. Each copy contains the identity of the next node that has a copy of the block. The location of the copies is therefore determined through network transactions.

Simple SCI (SSCI) retains similarity with full-bit vector protocol: MESI states in the cache; U, S, EM states in the memory directory; It replaces the presence bits with a pointer.

== Why additional states are necessary? ==

Performance & Correctness (Coherence & Consistency) requirement for the protocol necessitates the additional states in the protocol.

=== Need for Busy state ===

On a scalable multiprocessors without coherent caches, the main-memory module determined the ordering of writes. The order that writes become visible to all processors is the order in which they reached memory.

e.g. If two processors issue read-exclusive requests for a particular word, the home will provide the requestors with the location of
the dirty node. But which request will reach the dirty node first cannot be guaranteed. This creates the need for additional busy state in the directory.

This can be solved by
# holding requests at home or requestor node and serve them in the order of their arrival
# If the block is busy, reject any further request to it & that request will be retried later
# If directory is busy forward request to dirty node & dirty node will serialize the request execution

In non-coherent scalable multiprocessors,
# For write atomicity in invalidation based protocol, current owner of block has to wait until it recieve all invalidation acks and then only it can read/write new value.
# For write completion, current owner of the block need to wait for ack from memory

In Origin, there are 3 busy states indicates that the directory has received the request but is waiting for completion hence another request for the same block won't be accepted.
# Busy read
# Busy read exclusive or upgrade
# Busy uncached read: In case of DMA, if the memory block is being read, no other processor should be allowed to get write access to it.

=== Poison state ===

To improve locality, origin relies on page migration. Hardware keeps reference counts on each page. On every access of memory count is incremented and compared with home node. If count is more than programmable threshold, hardware interrupts one of the local processors to copy the page. Block transfer engine “poison” the source page. Subsequent accesses by other processors receive a bus error and that removes the TLB entry for that location, read the location from the page table & update the TLB with new location. This approach is called - “Lazy TLB shootdown” which reduces the overall cost of migrating memory and changing the virtual-to-physical address mappings.

=== Additional states ===

Additional states can be introduced into the sharing list. The SSCI protocol has only one kind of sharing list state : dirty. More options can be introduced into the list to improve performance of the overall shared multiprocessors. Although additional states do increment the complexity of the protocol being utilized and implemented, it would be very much beneficial to the system when the overall performance of the system is increased. The question “Why Additional States are necessary” is answered alongside the introduction of each new state i.e. explanation for the necessity of each and every state has been provided.

Having the following states implemented in the protocol can improve performance.

1. “fresh” and “gone” – Main memory may still be able to respond to a requesting processor if the memory can differentiate between ‘cached and unmodified’ and ‘cached and modified’. Even if the cached value is elsewhere, memory may still be able to respond to the request if the data remains unmodified.

*The “fresh” state shows that memory still has an up-to-date copy of the block, even if the block has been cached and is being shared between processors. As long as the value remains unmodified by any of the processors sharing the data, the memory can keep responding by returning the requested data to the requesting processor.

*The “gone” state indicates that the block has been cached and has been modified. This shows that main memory cant return the data anymore, but has to return the pointer that points to the head of the list.

2. “clean lists” – Although the introduction of these states increase the complexity of handling each access, it does help with the overall performance. In this case, it may be helpful to differentiate the types of read accesses to the cache. One type of read access would be a ‘normal read-only’. Another type of read would be a ‘read with intent to modify later’. The memory in the state is ‘fresh’ when a read-fresh takes place. A read-clean leaves the memory in the ‘gone’ state. Just as with the dirty state, the read-clean allows a write to the block to proceed immediately. When a block is replaced in the cache and the cache is in the ‘clean’ state, no data needs to be written to memory. But with the dirty state, a write-back would be required even if the data had not been modified.

== Examples ==
=== 1 DASH Cache Coherence Protocol ===

DASH (Directory Architecture for SHared memory) is a scalable shared-memory multiprocessor currently being developed at Stanford’s Computer Systems Laboratory. DASH protocol uses point-to-point messages sent between the processors and memories to keep caches consistent.

The DASH coherence protocol is an invalidation-based ownership protocol. A memory block can be in one of three states as indicated by the associated directory entry: (i) uncached-remote, that is not cached by any remote cluster; (ii) shared-remote, that is cached in an unmodified state by one or more remote clusters; or (iii) dirty-remote, that is cached in a modified state by a single remote cluster.

Please see in the figures below: Left - Flow of Read Request to remote memory with directory in dirty-remote state. Right - Flow of Read-Exclusive Request to remote memory with directory in shared-remote state.

[[Image:DASH.jpg]]

Write back request: A dirty cache line that is replaced must be written back to memory. If the home of the memory block is the local cluster, then the data is simply written back to main memory. If the home cluster is remote, then a message is sent to the remote
home which updates the main memory and marks the block uncached-remote.

=== 2. SCI-based CC-NUMA machines ===

SCI is attached to the memory bus of a node. Here, SCI can take part in and "export" cache coherence and memory traffic on the bus and make the memory of the node visible to other nodes. Each node's memory address range can be scaled to span a global address space, giving coherent and transparent access to anywhere in the system to requesting processors. This method is used to form a large-scale cache-coherent shared-memory system, known as a CC-NUMA machine.

= References =

1 The DASH Cache Coherence Protocol
Daniel Lenoski, James Laudon, Kourosh Gharachorloo, Anoop Gupta, John Hennessy
May 1990
ACM SIGARCH Computer Architecture News , Proceedings of the 17th annual international symposium on Computer Architecture ISCA '90,
Volume 18 Issue 3a
Publisher: ACM Press

2. 1596-1992 IEEE standard for scalable coherent interface (SCI).
E-ISBN: 0-7381-1204-6
Year: 1993
Sponsored by: IEEE Computer Society

3. The SGI Origin: A ccnuma Highly Scalable Server
Laudon, J.; Lenoski, D.;
Computer Architecture, 1997. Conference Proceedings. The 24th Annual International Symposium on
June 2-4, 1997 Page(s):241 - 251

4. The Scalable Coherence Interface
David.B.Gustavson
September 1991
"The Scalable Coherence Interface and related standard projects"

5. An analysis of the Scalable Coherent Interface
Eric Rotenberg
June 1995
An Analytical model of the SCI Coherence protocol

6. Interfaces and Standards : SCI
Behrooz. Parhami
1999
Introduction to Parallel Processing: Algorithms and Architectures

7. SCI: Scalable Coherent Interface
Hermann Hellwagner
1999
SCI: Scalable Coherent Interface - Architecture and software for high-performance Compute Clusters

= Further reading =

1. The Stanford Dash multiprocessor
Lenoski, D.; Laudon, J.; Gharachorloo, K.; Weber, W.-D.; Gupta, A.; Hennessy, J.; Horowitz, M.; Lam, M.S.;
Computer
IEEE Volume 25, Issue 3, March 1992 Page(s):63 - 79
Digital Object Identifier 10.1109/2.121510

2. SCI: Scalable Coherent Interface
Hermann Hellwagner
1999
SCI: Scalable Coherent Interface - Architecture and software for high-performance Compute Clusters

= External links =

CSC/ECE 506 Fall 2007/wiki3 2 1r

2007-10-20T03:16:05Z

Arangan: /* Examples */

''Wiki: SCI. The IEEE Scalable Coherent Interface is a superset of the SSCI protocol we have been considering in class. A lot has been written about it, but it is still difficult to comprehend. Using SSCI as a starting point, explain why additional states are necessary, and give (or cite) examples that demonstrate how they work. Ideally, this would still be an overview of the working of the protocol, referencing more detailed documentation on the Web.''

__TOC__

= Introduction =

In parallel systems, data can be obtained from a variety of sources, which includes local or remote memory modules, secondary storage devices and network interfaces. It would be beneficial if certain aspects of mechanisms that are required to gain access to data are standardized such that it allows easily expandable and flexible designs. Various portions of a parallel system may be modified or even upgraded without any effect on the rest of the design if the system adheres to such standards. Also, utilization of standard interfaces forwards the integration of components and subsystems from several different vendors into efficient and usable parallel systems. The Scalable Coherent Interface is one such standard which facilitates the implementation of large-scale cache-coherent parallel systems.

= Scalable Coherent Interface (SCI) =

The scalable coherent interface has now become a chief hardware based approach to the cache coherence problem in shared memory multiprocessors. SCI is a directory based invalidate coherence protocol. The state of a cache block is distributed to the sharers of that block. Limits that are inherent in bus technology are easily avoided by SCI. The SCI protocol is to provide scalability, coherence and an interface. Scalability is to guarantee that the same mechanisms can be used in single processor systems and large highly parallel multiprocessors. Coherence is to guarantee efficient and integral use of cache memories in distributed shared memory. An interface provides a communication architecture that has multiple values to be brought into a single system and provide smooth inter-operation.

===Goals of SCI===

*High Performance – For distributed or parallel applications, high communication performance must be obtained when using SCI. Usual aspects of such performance would be a low latency, low CPU overhead for operations dealing with communication and high sustained throughput.

*Scalability – This is addressed in many ways. Some are
**Performance scalability – as the number of nodes being added to the system increases.
**Interconnect Distance scalability
**Memory System scalability – chiefly of cache coherence protocols, which should not have limitations to the number of processors or modules in memory that it can handle.
**Technological scalability – utilization of similar mechanisms in large, small-scale, loosely-coupled or tightly-coupled systems. Also, when new advances are made in technology, it must have the ability to readily make use of such advances.
**Economic scalability – use of similar components and mechanisms in high-end, low-volume systems as well as low-end, high-volume systems.
**Addressing capability has no short term practical limits.

*Coherent Memory System – For the purpose of reduction of Average Access Time to data, caches are becoming more important each day for microprocessors.

*Interface Characteristics – This specifies a standard interface to an interconnect that makes several devices to be connected or attached together and to interoperate.

===Operation===

In SCI, every interface does not wait for the signal to propagate before it begins to send the next signal. Also, SCI utilizes multiple links so that, concurrently, several transfers can take place.

Usually, a directory entry is in either of the two states : ‘home’ or ‘gone’. If the state is in ‘home’, then memory can immediately satisfy requests to a block as the block has not been cached by any processor. If the state is ‘gone’, then the block has been cached by a processor and might even be modified. Now, the directory contains a pointer to the first processor in the sharing list for this particular block. Hence, on requesting the data, memory returns the pointer to the first processor on the shared list rather than the data itself. The processor asking for the block now forwards its request to the processor on the top of the shared list. Now, the requesting processor adds itself into the shared list as the new head of the list.

In the SCI protocol, any coherent transaction has three phases.

Memory read – When a processor misses in its cache, It asks for the block in the home directory in memory. If the state of the memory is ‘home’, then the main memory replies with the block to the requesting processor. If the state is ‘gone’, then the main memory returns the head of the shared list of processors for that particular block. Then, memory updates its pointer and puts the requesting processor as the new head.

Cache read – When memory returns a pointer toe the requesting processor instead of data, the processor forwards its request for the block to the head of the doubly linked list. When the cache receives, the head of the list returns the data which might have been modified. The head of the list changes its backward pointer to the requesting processor’s node. Now, the requesting processor becomes the head of the list and it has the cache block.

Cleanup – If the cache miss from the requesting processor is a store, the processor has to first invalidate all other cached copies and then only proceed with the store. The new head gives an invalidate request to the next address on the list i.e. to its next processor. This processor invalidates and gives back a pointer to the next processor on its list. The head of the list uses this new pointer and sends it an invalidate request. This goes on until a NULL pointer is returned. This is the cleanup process for invalidation of other cache copies of the block.

= Simple Scalable Coherent Interface (SSCI) =

In directory-based approach, Every memory block has associated directory information; it keeps track of copies of cached blocks and their states. On a miss, it finds the directory entry, looks it up, and communicates only with the nodes that have copies (if necessary).

There are mainly two approaches: Full-bit vector: For k processors, it maintains k presence bit and 1 dirty bit at the home node. Cache state is represented the same way as in bus-based designs (MSI, MESI, etc.). It has three cache states: EM (exclusive or modified), S (shared), U (unowned). Limitation is - Number of presence bits needed grows as the number of processors.

Memory-based schemes store the information about all cached copies at the home node of the block. Cache-based schemes distribute information about copies among the copies themselves. The home contains a pointer to one cached copy of the block. Each copy contains the identity of the next node that has a copy of the block. The location of the copies is therefore determined through network transactions.

Simple SCI (SSCI) retains similarity with full-bit vector protocol: MESI states in the cache; U, S, EM states in the memory directory; It replaces the presence bits with a pointer.

== Why additional states are necessary? ==

Performance & Correctness (Coherence & Consistency) requirement for the protocol necessitates the additional states in the protocol.

=== Need for Busy state ===

On a scalable multiprocessors without coherent caches, the main-memory module determined the ordering of writes. The order that writes become visible to all processors is the order in which they reached memory.

e.g. If two processors issue read-exclusive requests for a particular word, the home will provide the requestors with the location of
the dirty node. But which request will reach the dirty node first cannot be guaranteed. This creates the need for additional busy state in the directory.

This can be solved by
# holding requests at home or requestor node and serve them in the order of their arrival
# If the block is busy, reject any further request to it & that request will be retried later
# If directory is busy forward request to dirty node & dirty node will serialize the request execution

In non-coherent scalable multiprocessors,
# For write atomicity in invalidation based protocol, current owner of block has to wait until it recieve all invalidation acks and then only it can read/write new value.
# For write completion, current owner of the block need to wait for ack from memory

In Origin, there are 3 busy states indicates that the directory has received the request but is waiting for completion hence another request for the same block won't be accepted.
# Busy read
# Busy read exclusive or upgrade
# Busy uncached read: In case of DMA, if the memory block is being read, no other processor should be allowed to get write access to it.

=== Poison state ===

To improve locality, origin relies on page migration. Hardware keeps reference counts on each page. On every access of memory count is incremented and compared with home node. If count is more than programmable threshold, hardware interrupts one of the local processors to copy the page. Block transfer engine “poison” the source page. Subsequent accesses by other processors receive a bus error and that removes the TLB entry for that location, read the location from the page table & update the TLB with new location. This approach is called - “Lazy TLB shootdown” which reduces the overall cost of migrating memory and changing the virtual-to-physical address mappings.

=== Additional states ===

Additional states can be introduced into the sharing list. The SSCI protocol has only one kind of sharing list state : dirty. More options can be introduced into the list to improve performance of the overall shared multiprocessors. Although additional states do increment the complexity of the protocol being utilized and implemented, it would be very much beneficial to the system when the overall performance of the system is increased. The question “Why Additional States are necessary” is answered alongside the introduction of each new state i.e. explanation for the necessity of each and every state has been provided.

Having the following states implemented in the protocol can improve performance.

1. “fresh” and “gone” – Main memory may still be able to respond to a requesting processor if the memory can differentiate between ‘cached and unmodified’ and ‘cached and modified’. Even if the cached value is elsewhere, memory may still be able to respond to the request if the data remains unmodified.

*The “fresh” state shows that memory still has an up-to-date copy of the block, even if the block has been cached and is being shared between processors. As long as the value remains unmodified by any of the processors sharing the data, the memory can keep responding by returning the requested data to the requesting processor.

*The “gone” state indicates that the block has been cached and has been modified. This shows that main memory cant return the data anymore, but has to return the pointer that points to the head of the list.

2. “clean lists” – Although the introduction of these states increase the complexity of handling each access, it does help with the overall performance. In this case, it may be helpful to differentiate the types of read accesses to the cache. One type of read access would be a ‘normal read-only’. Another type of read would be a ‘read with intent to modify later’. The memory in the state is ‘fresh’ when a read-fresh takes place. A read-clean leaves the memory in the ‘gone’ state. Just as with the dirty state, the read-clean allows a write to the block to proceed immediately. When a block is replaced in the cache and the cache is in the ‘clean’ state, no data needs to be written to memory. But with the dirty state, a write-back would be required even if the data had not been modified.

== Examples ==
=== 1 DASH Cache Coherence Protocol ===

DASH (Directory Architecture for SHared memory) is a scalable shared-memory multiprocessor currently being developed at Stanford’s Computer Systems Laboratory. DASH protocol uses point-to-point messages sent between the processors and memories to keep caches consistent.

The DASH coherence protocol is an invalidation-based ownership protocol. A memory block can be in one of three states as indicated by the associated directory entry: (i) uncached-remote, that is not cached by any remote cluster; (ii) shared-remote, that is cached in an unmodified state by one or more remote clusters; or (iii) dirty-remote, that is cached in a modified state by a single remote cluster.

Please see in the figures below: Left - Flow of Read Request to remote memory with directory in dirty-remote state. Right - Flow of Read-Exclusive Request to remote memory with directory in shared-remote state.

[[Image:DASH.jpg]]

Write back request: A dirty cache line that is replaced must be written back to memory. If the home of the memory block is the local cluster, then the data is simply written back to main memory. If the home cluster is remote, then a message is sent to the remote
home which updates the main memory and marks the block uncached-remote.

=== SCI-based CC-NUMA machines ===

SCI is attached to the memory bus of a node. Here, SCI can take part in and "export" cache coherence and memory traffic on the bus and make the memory of the node visible to other nodes. Each node's memory address range can be scaled to span a global address space, giving coherent and transparent access to anywhere in the system to requesting processors. This method is used to form a large-scale cache-coherent shared-memory system, known as a CC-NUMA machine.

= References =

1 The DASH Cache Coherence Protocol
Daniel Lenoski, James Laudon, Kourosh Gharachorloo, Anoop Gupta, John Hennessy
May 1990
ACM SIGARCH Computer Architecture News , Proceedings of the 17th annual international symposium on Computer Architecture ISCA '90,
Volume 18 Issue 3a
Publisher: ACM Press

2. 1596-1992 IEEE standard for scalable coherent interface (SCI).
E-ISBN: 0-7381-1204-6
Year: 1993
Sponsored by: IEEE Computer Society

3. The SGI Origin: A ccnuma Highly Scalable Server
Laudon, J.; Lenoski, D.;
Computer Architecture, 1997. Conference Proceedings. The 24th Annual International Symposium on
June 2-4, 1997 Page(s):241 - 251

4. The Scalable Coherence Interface
David.B.Gustavson
September 1991
"The Scalable Coherence Interface and related standard projects"

5. An analysis of the Scalable Coherent Interface
Eric Rotenberg
June 1995
An Analytical model of the SCI Coherence protocol

6. Interfaces and Standards : SCI
Behrooz. Parhami
1999
Introduction to Parallel Processing: Algorithms and Architectures

7. SCI: Scalable Coherent Interface
Hermann Hellwagner
1999
SCI: Scalable Coherent Interface - Architecture and software for high-performance Compute Clusters

= Further reading =

1. The Stanford Dash multiprocessor
Lenoski, D.; Laudon, J.; Gharachorloo, K.; Weber, W.-D.; Gupta, A.; Hennessy, J.; Horowitz, M.; Lam, M.S.;
Computer
IEEE Volume 25, Issue 3, March 1992 Page(s):63 - 79
Digital Object Identifier 10.1109/2.121510

2. SCI: Scalable Coherent Interface
Hermann Hellwagner
1999
SCI: Scalable Coherent Interface - Architecture and software for high-performance Compute Clusters

= External links =

CSC/ECE 506 Fall 2007/wiki3 2 1r

2007-10-20T01:15:51Z

Arangan: /* Further reading */

''Wiki: SCI. The IEEE Scalable Coherent Interface is a superset of the SSCI protocol we have been considering in class. A lot has been written about it, but it is still difficult to comprehend. Using SSCI as a starting point, explain why additional states are necessary, and give (or cite) examples that demonstrate how they work. Ideally, this would still be an overview of the working of the protocol, referencing more detailed documentation on the Web.''

__TOC__

= Introduction =

In parallel systems, data can be obtained from a variety of sources, which includes local or remote memory modules, secondary storage devices and network interfaces. It would be beneficial if certain aspects of mechanisms that are required to gain access to data are standardized such that it allows easily expandable and flexible designs. Various portions of a parallel system may be modified or even upgraded without any effect on the rest of the design if the system adheres to such standards. Also, utilization of standard interfaces forwards the integration of components and subsystems from several different vendors into efficient and usable parallel systems. The Scalable Coherent Interface is one such standard which facilitates the implementation of large-scale cache-coherent parallel systems.

= Scalable Coherent Interface (SCI) =

The scalable coherent interface has now become a chief hardware based approach to the cache coherence problem in shared memory multiprocessors. SCI is a directory based invalidate coherence protocol. The state of a cache block is distributed to the sharers of that block. Limits that are inherent in bus technology are easily avoided by SCI. The SCI protocol is to provide scalability, coherence and an interface. Scalability is to guarantee that the same mechanisms can be used in single processor systems and large highly parallel multiprocessors. Coherence is to guarantee efficient and integral use of cache memories in distributed shared memory. An interface provides a communication architecture that has multiple values to be brought into a single system and provide smooth inter-operation.

===Goals of SCI===

*High Performance – For distributed or parallel applications, high communication performance must be obtained when using SCI. Usual aspects of such performance would be a low latency, low CPU overhead for operations dealing with communication and high sustained throughput.

*Scalability – This is addressed in many ways. Some are
**Performance scalability – as the number of nodes being added to the system increases.
**Interconnect Distance scalability
**Memory System scalability – chiefly of cache coherence protocols, which should not have limitations to the number of processors or modules in memory that it can handle.
**Technological scalability – utilization of similar mechanisms in large, small-scale, loosely-coupled or tightly-coupled systems. Also, when new advances are made in technology, it must have the ability to readily make use of such advances.
**Economic scalability – use of similar components and mechanisms in high-end, low-volume systems as well as low-end, high-volume systems.
**Addressing capability has no short term practical limits.

*Coherent Memory System – For the purpose of reduction of Average Access Time to data, caches are becoming more important each day for microprocessors.

*Interface Characteristics – This specifies a standard interface to an interconnect that makes several devices to be connected or attached together and to interoperate.

===Operation===

In SCI, every interface does not wait for the signal to propagate before it begins to send the next signal. Also, SCI utilizes multiple links so that, concurrently, several transfers can take place.

Usually, a directory entry is in either of the two states : ‘home’ or ‘gone’. If the state is in ‘home’, then memory can immediately satisfy requests to a block as the block has not been cached by any processor. If the state is ‘gone’, then the block has been cached by a processor and might even be modified. Now, the directory contains a pointer to the first processor in the sharing list for this particular block. Hence, on requesting the data, memory returns the pointer to the first processor on the shared list rather than the data itself. The processor asking for the block now forwards its request to the processor on the top of the shared list. Now, the requesting processor adds itself into the shared list as the new head of the list.

In the SCI protocol, any coherent transaction has three phases.

Memory read – When a processor misses in its cache, It asks for the block in the home directory in memory. If the state of the memory is ‘home’, then the main memory replies with the block to the requesting processor. If the state is ‘gone’, then the main memory returns the head of the shared list of processors for that particular block. Then, memory updates its pointer and puts the requesting processor as the new head.

Cache read – When memory returns a pointer toe the requesting processor instead of data, the processor forwards its request for the block to the head of the doubly linked list. When the cache receives, the head of the list returns the data which might have been modified. The head of the list changes its backward pointer to the requesting processor’s node. Now, the requesting processor becomes the head of the list and it has the cache block.

Cleanup – If the cache miss from the requesting processor is a store, the processor has to first invalidate all other cached copies and then only proceed with the store. The new head gives an invalidate request to the next address on the list i.e. to its next processor. This processor invalidates and gives back a pointer to the next processor on its list. The head of the list uses this new pointer and sends it an invalidate request. This goes on until a NULL pointer is returned. This is the cleanup process for invalidation of other cache copies of the block.

= Simple Scalable Coherent Interface (SSCI) =

In directory-based approach, Every memory block has associated directory information; it keeps track of copies of cached blocks and their states. On a miss, it finds the directory entry, looks it up, and communicates only with the nodes that have copies (if necessary).

There are mainly two approaches: Full-bit vector: For k processors, it maintains k presence bit and 1 dirty bit at the home node. Cache state is represented the same way as in bus-based designs (MSI, MESI, etc.). It has three cache states: EM (exclusive or modified), S (shared), U (unowned). Limitation is - Number of presence bits needed grows as the number of processors.

Memory-based schemes store the information about all cached copies at the home node of the block. Cache-based schemes distribute information about copies among the copies themselves. The home contains a pointer to one cached copy of the block. Each copy contains the identity of the next node that has a copy of the block. The location of the copies is therefore determined through network transactions.

Simple SCI (SSCI) retains similarity with full-bit vector protocol: MESI states in the cache; U, S, EM states in the memory directory; It replaces the presence bits with a pointer.

== Why additional states are necessary? ==

Performance & Correctness (Coherence & Consistency) requirement for the protocol necessitates the additional states in the protocol.

=== Need for Busy state ===

On a scalable multiprocessors without coherent caches, the main-memory module determined the ordering of writes. The order that writes become visible to all processors is the order in which they reached memory.

e.g. If two processors issue read-exclusive requests for a particular word, the home will provide the requestors with the location of
the dirty node. But which request will reach the dirty node first cannot be guaranteed. This creates the need for additional busy state in the directory.

This can be solved by
# holding requests at home or requestor node and serve them in the order of their arrival
# If the block is busy, reject any further request to it & that request will be retried later
# If directory is busy forward request to dirty node & dirty node will serialize the request execution

In non-coherent scalable multiprocessors,
# For write atomicity in invalidation based protocol, current owner of block has to wait until it recieve all invalidation acks and then only it can read/write new value.
# For write completion, current owner of the block need to wait for ack from memory

In Origin, there are 3 busy states indicates that the directory has received the request but is waiting for completion hence another request for the same block won't be accepted.
# Busy read
# Busy read exclusive or upgrade
# Busy uncached read: In case of DMA, if the memory block is being read, no other processor should be allowed to get write access to it.

=== Poison state ===

To improve locality, origin relies on page migration. Hardware keeps reference counts on each page. On every access of memory count is incremented and compared with home node. If count is more than programmable threshold, hardware interrupts one of the local processors to copy the page. Block transfer engine “poison” the source page. Subsequent accesses by other processors receive a bus error and that removes the TLB entry for that location, read the location from the page table & update the TLB with new location. This approach is called - “Lazy TLB shootdown” which reduces the overall cost of migrating memory and changing the virtual-to-physical address mappings.

=== Additional states ===

Additional states can be introduced into the sharing list. The SSCI protocol has only one kind of sharing list state : dirty. More options can be introduced into the list to improve performance of the overall shared multiprocessors. Although additional states do increment the complexity of the protocol being utilized and implemented, it would be very much beneficial to the system when the overall performance of the system is increased. The question “Why Additional States are necessary” is answered alongside the introduction of each new state i.e. explanation for the necessity of each and every state has been provided.

Having the following states implemented in the protocol can improve performance.

1. “fresh” and “gone” – Main memory may still be able to respond to a requesting processor if the memory can differentiate between ‘cached and unmodified’ and ‘cached and modified’. Even if the cached value is elsewhere, memory may still be able to respond to the request if the data remains unmodified.

*The “fresh” state shows that memory still has an up-to-date copy of the block, even if the block has been cached and is being shared between processors. As long as the value remains unmodified by any of the processors sharing the data, the memory can keep responding by returning the requested data to the requesting processor.

*The “gone” state indicates that the block has been cached and has been modified. This shows that main memory cant return the data anymore, but has to return the pointer that points to the head of the list.

2. “clean lists” – Although the introduction of these states increase the complexity of handling each access, it does help with the overall performance. In this case, it may be helpful to differentiate the types of read accesses to the cache. One type of read access would be a ‘normal read-only’. Another type of read would be a ‘read with intent to modify later’. The memory in the state is ‘fresh’ when a read-fresh takes place. A read-clean leaves the memory in the ‘gone’ state. Just as with the dirty state, the read-clean allows a write to the block to proceed immediately. When a block is replaced in the cache and the cache is in the ‘clean’ state, no data needs to be written to memory. But with the dirty state, a write-back would be required even if the data had not been modified.

== Examples ==
=== 1 DASH Cache Coherence Protocol ===

DASH (Directory Architecture for SHared memory) is a scalable shared-memory multiprocessor currently being developed at Stanford’s Computer Systems Laboratory. DASH protocol uses point-to-point messages sent between the processors and memories to keep caches consistent.

The DASH coherence protocol is an invalidation-based ownership protocol. A memory block can be in one of three states as indicated by the associated directory entry: (i) uncached-remote, that is not cached by any remote cluster; (ii) shared-remote, that is cached in an unmodified state by one or more remote clusters; or (iii) dirty-remote, that is cached in a modified state by a single remote cluster.

Please see in the figures below: Left - Flow of Read Request to remote memory with directory in dirty-remote state. Right - Flow of Read-Exclusive Request to remote memory with directory in shared-remote state.

[[Image:DASH.jpg]]

Write back request: A dirty cache line that is replaced must be written back to memory. If the home of the memory block is the local cluster, then the data is simply written back to main memory. If the home cluster is remote, then a message is sent to the remote
home which updates the main memory and marks the block uncached-remote.

= References =

1 The DASH Cache Coherence Protocol
Daniel Lenoski, James Laudon, Kourosh Gharachorloo, Anoop Gupta, John Hennessy
May 1990
ACM SIGARCH Computer Architecture News , Proceedings of the 17th annual international symposium on Computer Architecture ISCA '90,
Volume 18 Issue 3a
Publisher: ACM Press

2. 1596-1992 IEEE standard for scalable coherent interface (SCI).
E-ISBN: 0-7381-1204-6
Year: 1993
Sponsored by: IEEE Computer Society

3. The SGI Origin: A ccnuma Highly Scalable Server
Laudon, J.; Lenoski, D.;
Computer Architecture, 1997. Conference Proceedings. The 24th Annual International Symposium on
June 2-4, 1997 Page(s):241 - 251

4. The Scalable Coherence Interface
David.B.Gustavson
September 1991
"The Scalable Coherence Interface and related standard projects"

5. An analysis of the Scalable Coherent Interface
Eric Rotenberg
June 1995
An Analytical model of the SCI Coherence protocol

6. Interfaces and Standards : SCI
Behrooz. Parhami
1999
Introduction to Parallel Processing: Algorithms and Architectures

7. SCI: Scalable Coherent Interface
Hermann Hellwagner
1999
SCI: Scalable Coherent Interface - Architecture and software for high-performance Compute Clusters

= Further reading =

1. The Stanford Dash multiprocessor
Lenoski, D.; Laudon, J.; Gharachorloo, K.; Weber, W.-D.; Gupta, A.; Hennessy, J.; Horowitz, M.; Lam, M.S.;
Computer
IEEE Volume 25, Issue 3, March 1992 Page(s):63 - 79
Digital Object Identifier 10.1109/2.121510

2. SCI: Scalable Coherent Interface
Hermann Hellwagner
1999
SCI: Scalable Coherent Interface - Architecture and software for high-performance Compute Clusters

= External links =

CSC/ECE 506 Fall 2007/wiki3 2 1r

2007-10-19T22:34:44Z

Arangan:

''Wiki: SCI. The IEEE Scalable Coherent Interface is a superset of the SSCI protocol we have been considering in class. A lot has been written about it, but it is still difficult to comprehend. Using SSCI as a starting point, explain why additional states are necessary, and give (or cite) examples that demonstrate how they work. Ideally, this would still be an overview of the working of the protocol, referencing more detailed documentation on the Web.''

__TOC__

= Introduction =

In parallel systems, data can be obtained from a variety of sources, which includes local or remote memory modules, secondary storage devices and network interfaces. It would be beneficial if certain aspects of mechanisms that are required to gain access to data are standardized such that it allows easily expandable and flexible designs. Various portions of a parallel system may be modified or even upgraded without any effect on the rest of the design if the system adheres to such standards. Also, utilization of standard interfaces forwards the integration of components and subsystems from several different vendors into efficient and usable parallel systems. The Scalable Coherent Interface is one such standard which facilitates the implementation of large-scale cache-coherent parallel systems.

= Scalable Coherent Interface (SCI) =

The scalable coherent interface has now become a chief hardware based approach to the cache coherence problem in shared memory multiprocessors. SCI is a directory based invalidate coherence protocol. The state of a cache block is distributed to the sharers of that block. Limits that are inherent in bus technology are easily avoided by SCI. The SCI protocol is to provide scalability, coherence and an interface. Scalability is to guarantee that the same mechanisms can be used in single processor systems and large highly parallel multiprocessors. Coherence is to guarantee efficient and integral use of cache memories in distributed shared memory. An interface provides a communication architecture that has multiple values to be brought into a single system and provide smooth inter-operation.

===Goals of SCI===

*High Performance – For distributed or parallel applications, high communication performance must be obtained when using SCI. Usual aspects of such performance would be a low latency, low CPU overhead for operations dealing with communication and high sustained throughput.

*Scalability – This is addressed in many ways. Some are
**Performance scalability – as the number of nodes being added to the system increases.
**Interconnect Distance scalability
**Memory System scalability – chiefly of cache coherence protocols, which should not have limitations to the number of processors or modules in memory that it can handle.
**Technological scalability – utilization of similar mechanisms in large, small-scale, loosely-coupled or tightly-coupled systems. Also, when new advances are made in technology, it must have the ability to readily make use of such advances.
**Economic scalability – use of similar components and mechanisms in high-end, low-volume systems as well as low-end, high-volume systems.
**Addressing capability has no short term practical limits.

*Coherent Memory System – For the purpose of reduction of Average Access Time to data, caches are becoming more important each day for microprocessors.

*Interface Characteristics – This specifies a standard interface to an interconnect that makes several devices to be connected or attached together and to interoperate.

===Operation===

In SCI, every interface does not wait for the signal to propagate before it begins to send the next signal. Also, SCI utilizes multiple links so that, concurrently, several transfers can take place.

Usually, a directory entry is in either of the two states : ‘home’ or ‘gone’. If the state is in ‘home’, then memory can immediately satisfy requests to a block as the block has not been cached by any processor. If the state is ‘gone’, then the block has been cached by a processor and might even be modified. Now, the directory contains a pointer to the first processor in the sharing list for this particular block. Hence, on requesting the data, memory returns the pointer to the first processor on the shared list rather than the data itself. The processor asking for the block now forwards its request to the processor on the top of the shared list. Now, the requesting processor adds itself into the shared list as the new head of the list.

In the SCI protocol, any coherent transaction has three phases.

Memory read – When a processor misses in its cache, It asks for the block in the home directory in memory. If the state of the memory is ‘home’, then the main memory replies with the block to the requesting processor. If the state is ‘gone’, then the main memory returns the head of the shared list of processors for that particular block. Then, memory updates its pointer and puts the requesting processor as the new head.

Cache read – When memory returns a pointer toe the requesting processor instead of data, the processor forwards its request for the block to the head of the doubly linked list. When the cache receives, the head of the list returns the data which might have been modified. The head of the list changes its backward pointer to the requesting processor’s node. Now, the requesting processor becomes the head of the list and it has the cache block.

Cleanup – If the cache miss from the requesting processor is a store, the processor has to first invalidate all other cached copies and then only proceed with the store. The new head gives an invalidate request to the next address on the list i.e. to its next processor. This processor invalidates and gives back a pointer to the next processor on its list. The head of the list uses this new pointer and sends it an invalidate request. This goes on until a NULL pointer is returned. This is the cleanup process for invalidation of other cache copies of the block.

= Simple Scalable Coherent Interface (SSCI) =

In directory-based approach, Every memory block has associated directory information; it keeps track of copies of cached blocks and their states. On a miss, it finds the directory entry, looks it up, and communicates only with the nodes that have copies (if necessary).

There are mainly two approaches: Full-bit vector: For k processors, it maintains k presence bit and 1 dirty bit at the home node. Cache state is represented the same way as in bus-based designs (MSI, MESI, etc.). It has three cache states: EM (exclusive or modified), S (shared), U (unowned). Limitation is - Number of presence bits needed grows as the number of processors.

Memory-based schemes store the information about all cached copies at the home node of the block. Cache-based schemes distribute information about copies among the copies themselves. The home contains a pointer to one cached copy of the block. Each copy contains the identity of the next node that has a copy of the block. The location of the copies is therefore determined through network transactions.

Simple SCI (SSCI) retains similarity with full-bit vector protocol: MESI states in the cache; U, S, EM states in the memory directory; It replaces the presence bits with a pointer.

== Why additional states are necessary? ==

Performance & Correctness (Coherence & Consistency) requirement for the protocol necessitates the additional states in the protocol.

=== Need for Busy state ===

On a scalable multiprocessors without coherent caches, the main-memory module determined the ordering of writes. The order that writes become visible to all processors is the order in which they reached memory.

e.g. If two processors issue read-exclusive requests for a particular word, the home will provide the requestors with the location of
the dirty node. But which request will reach the dirty node first cannot be guaranteed. This creates the need for additional busy state in the directory.

This can be solved by
# holding requests at home or requestor node and serve them in the order of their arrival
# If the block is busy, reject any further request to it & that request will be retried later
# If directory is busy forward request to dirty node & dirty node will serialize the request execution

In non-coherent scalable multiprocessors,
# For write atomicity in invalidation based protocol, current owner of block has to wait until it recieve all invalidation acks and then only it can read/write new value.
# For write completion, current owner of the block need to wait for ack from memory

In Origin, there are 3 busy states indicates that the directory has received the request but is waiting for completion hence another request for the same block won't be accepted.
# Busy read
# Busy read exclusive or upgrade
# Busy uncached read: In case of DMA, if the memory block is being read, no other processor should be allowed to get write access to it.

=== Poison state ===

To improve locality, origin relies on page migration. Hardware keeps reference counts on each page. On every access of memory count is incremented and compared with home node. If count is more than programmable threshold, hardware interrupts one of the local processors to copy the page. Block transfer engine “poison” the source page. Subsequent accesses by other processors receive a bus error and that removes the TLB entry for that location, read the location from the page table & update the TLB with new location. This approach is called - “Lazy TLB shootdown” which reduces the overall cost of migrating memory and changing the virtual-to-physical address mappings.

=== Additional states ===

Additional states can be introduced into the sharing list. The SSCI protocol has only one kind of sharing list state : dirty. More options can be introduced into the list to improve performance of the overall shared multiprocessors. Although additional states do increment the complexity of the protocol being utilized and implemented, it would be very much beneficial to the system when the overall performance of the system is increased. The question “Why Additional States are necessary” is answered alongside the introduction of each new state i.e. explanation for the necessity of each and every state has been provided.

Having the following states implemented in the protocol can improve performance.

1. “fresh” and “gone” – Main memory may still be able to respond to a requesting processor if the memory can differentiate between ‘cached and unmodified’ and ‘cached and modified’. Even if the cached value is elsewhere, memory may still be able to respond to the request if the data remains unmodified.

*The “fresh” state shows that memory still has an up-to-date copy of the block, even if the block has been cached and is being shared between processors. As long as the value remains unmodified by any of the processors sharing the data, the memory can keep responding by returning the requested data to the requesting processor.

*The “gone” state indicates that the block has been cached and has been modified. This shows that main memory cant return the data anymore, but has to return the pointer that points to the head of the list.

2. “clean lists” – Although the introduction of these states increase the complexity of handling each access, it does help with the overall performance. In this case, it may be helpful to differentiate the types of read accesses to the cache. One type of read access would be a ‘normal read-only’. Another type of read would be a ‘read with intent to modify later’. The memory in the state is ‘fresh’ when a read-fresh takes place. A read-clean leaves the memory in the ‘gone’ state. Just as with the dirty state, the read-clean allows a write to the block to proceed immediately. When a block is replaced in the cache and the cache is in the ‘clean’ state, no data needs to be written to memory. But with the dirty state, a write-back would be required even if the data had not been modified.

== Examples ==
=== 1 DASH Cache Coherence Protocol ===

DASH (Directory Architecture for SHared memory) is a scalable shared-memory multiprocessor currently being developed at Stanford’s Computer Systems Laboratory. DASH protocol uses point-to-point messages sent between the processors and memories to keep caches consistent.

The DASH coherence protocol is an invalidation-based ownership protocol. A memory block can be in one of three states as indicated by the associated directory entry: (i) uncached-remote, that is not cached by any remote cluster; (ii) shared-remote, that is cached in an unmodified state by one or more remote clusters; or (iii) dirty-remote, that is cached in a modified state by a single remote cluster.

Please see in the figures below: Left - Flow of Read Request to remote memory with directory in dirty-remote state. Right - Flow of Read-Exclusive Request to remote memory with directory in shared-remote state.

[[Image:DASH.jpg]]

Write back request: A dirty cache line that is replaced must be written back to memory. If the home of the memory block is the local cluster, then the data is simply written back to main memory. If the home cluster is remote, then a message is sent to the remote
home which updates the main memory and marks the block uncached-remote.

= References =

1 The DASH Cache Coherence Protocol
Daniel Lenoski, James Laudon, Kourosh Gharachorloo, Anoop Gupta, John Hennessy
May 1990
ACM SIGARCH Computer Architecture News , Proceedings of the 17th annual international symposium on Computer Architecture ISCA '90,
Volume 18 Issue 3a
Publisher: ACM Press

2. 1596-1992 IEEE standard for scalable coherent interface (SCI).
E-ISBN: 0-7381-1204-6
Year: 1993
Sponsored by: IEEE Computer Society

3. The SGI Origin: A ccnuma Highly Scalable Server
Laudon, J.; Lenoski, D.;
Computer Architecture, 1997. Conference Proceedings. The 24th Annual International Symposium on
June 2-4, 1997 Page(s):241 - 251

4. The Scalable Coherence Interface
David.B.Gustavson
September 1991
"The Scalable Coherence Interface and related standard projects"

5. An analysis of the Scalable Coherent Interface
Eric Rotenberg
June 1995
An Analytical model of the SCI Coherence protocol

6. Interfaces and Standards : SCI
Behrooz. Parhami
1999
Introduction to Parallel Processing: Algorithms and Architectures

7. SCI: Scalable Coherent Interface
Hermann Hellwagner
1999
SCI: Scalable Coherent Interface - Architecture and software for high-performance Compute Clusters

= Further reading =

1. The Stanford Dash multiprocessor
Lenoski, D.; Laudon, J.; Gharachorloo, K.; Weber, W.-D.; Gupta, A.; Hennessy, J.; Horowitz, M.; Lam, M.S.;
Computer
IEEE Volume 25, Issue 3, March 1992 Page(s):63 - 79
Digital Object Identifier 10.1109/2.121510

= External links =

CSC/ECE 506 Fall 2007/wiki3 8 38

2007-10-18T23:15:09Z

Arangan: /* References */

''Wiki: SCI. The IEEE Scalable Coherent Interface is a superset of the SSCI protocol we have been considering in class. A lot has been written about it, but it is still difficult to comprehend. Using SSCI as a starting point, explain why additional states are necessary, and give (or cite) examples that demonstrate how they work. Ideally, this would still be an overview of the working of the protocol, referencing more detailed documentation on the Web.''

__TOC__

= Introduction =

In parallel systems, data can be obtained from a variety of sources, which includes local or remote memory modules, secondary storage devices and network interfaces. It would be beneficial if certain aspects of mechanisms that are required to gain access to data are standardized such that it allows easily expandable and flexible designs. Various portions of a parallel system may be modified or even upgraded without any effect on the rest of the design if the system adheres to such standards. Also, utilization of standard interfaces forwards the integration of components and subsystems from several different vendors into efficient and usable parallel systems. The Scalable Coherent Interface is one such standard which facilitates the implementation of large-scale cache-coherent parallel systems.

= Scalable Coherent Interface (SCI) =

The scalable coherent interface has now become a chief hardware based approach to the cache coherence problem in shared memory multiprocessors. SCI is a directory based invalidate coherence protocol. The state of a cache block is distributed to the sharers of that block. Limits that are inherent in bus technology are easily avoided by SCI. The SCI protocol is to provide scalability, coherence and an interface. Scalability is to guarantee that the same mechanisms can be used in single processor systems and large highly parallel multiprocessors. Coherence is to guarantee efficient and integral use of cache memories in distributed shared memory. An interface provides a communication architecture that has multiple values to be brought into a single system and provide smooth inter-operation.

===Goals of SCI===

*High Performance – For distributed or parallel applications, high communication performance must be obtained when using SCI. Usual aspects of such performance would be a low latency, low CPU overhead for operations dealing with communication and high sustained throughput.

*Scalability – This is addressed in many ways. Some are
**Performance scalability – as the number of nodes being added to the system increases.
**Interconnect Distance scalability
**Memory System scalability – chiefly of cache coherence protocols, which should not have limitations to the number of processors or modules in memory that it can handle.
**Technological scalability – utilization of similar mechanisms in large, small-scale, loosely-coupled or tightly-coupled systems. Also, when new advances are made in technology, it must have the ability to readily make use of such advances.
**Economic scalability – use of similar components and mechanisms in high-end, low-volume systems as well as low-end, high-volume systems.
**Addressing capability has no short term practical limits.

*Coherent Memory System – For the purpose of reduction of Average Access Time to data, caches are becoming more important each day for microprocessors.

*Interface Characteristics – This specifies a standard interface to an interconnect that makes several devices to be connected or attached together and to interoperate.

===Operation===

In SCI, every interface does not wait for the signal to propagate before it begins to send the next signal. Also, SCI utilizes multiple links so that, concurrently, several transfers can take place.

Usually, a directory entry is in either of the two states : ‘home’ or ‘gone’. If the state is in ‘home’, then memory can immediately satisfy requests to a block as the block has not been cached by any processor. If the state is ‘gone’, then the block has been cached by a processor and might even be modified. Now, the directory contains a pointer to the first processor in the sharing list for this particular block. Hence, on requesting the data, memory returns the pointer to the first processor on the shared list rather than the data itself. The processor asking for the block now forwards its request to the processor on the top of the shared list. Now, the requesting processor adds itself into the shared list as the new head of the list.

In the SCI protocol, any coherent transaction has three phases.

Memory read – When a processor misses in its cache, It asks for the block in the home directory in memory. If the state of the memory is ‘home’, then the main memory replies with the block to the requesting processor. If the state is ‘gone’, then the main memory returns the head of the shared list of processors for that particular block. Then, memory updates its pointer and puts the requesting processor as the new head.

Cache read – When memory returns a pointer toe the requesting processor instead of data, the processor forwards its request for the block to the head of the doubly linked list. When the cache receives, the head of the list returns the data which might have been modified. The head of the list changes its backward pointer to the requesting processor’s node. Now, the requesting processor becomes the head of the list and it has the cache block.

Cleanup – If the cache miss from the requesting processor is a store, the processor has to first invalidate all other cached copies and then only proceed with the store. The new head gives an invalidate request to the next address on the list i.e. to its next processor. This processor invalidates and gives back a pointer to the next processor on its list. The head of the list uses this new pointer and sends it an invalidate request. This goes on until a NULL pointer is returned. This is the cleanup process for invalidation of other cache copies of the block.

= Simple Scalable Coherent Interface (SSCI) =

In directory-based approach, Every memory block has associated directory information; it keeps track of copies of cached blocks and their states. On a miss, it finds the directory entry, looks it up, and communicates only with the nodes that have copies (if necessary).

There are mainly two approaches: Full-bit vector: For k processors, it maintains k presence bit and 1 dirty bit at the home node. Cache state is represented the same way as in bus-based designs (MSI, MESI, etc.). It has three cache states: EM (exclusive or modified), S (shared), U (unowned). Limitation is - Number of presence bits needed grows as the number of processors.

Memory-based schemes store the information about all cached copies at the home node of the block. Cache-based schemes distribute information about copies among the copies themselves. The home contains a pointer to one cached copy of the block. Each copy contains the identity of the next node that has a copy of the block. The location of the copies is therefore determined through network transactions.

Simple SCI (SSCI) retains similarity with full-bit vector protocol: MESI states in the cache; U, S, EM states in the memory directory; It replaces the presence bits with a pointer.

== Why additional states are necessary? ==

Performance & Correctness (Coherence & Consistency) requirement for the protocol necessitates the additional states in the protocol.

=== Need for Busy state ===

On a scalable multiprocessors without coherent caches, the main-memory module determined the ordering of writes. The order that writes become visible to all processors is the order in which they reached memory.

e.g. If two processors issue read-exclusive requests for a particular word, the home will provide the requestors with the location of
the dirty node. But which request will reach the dirty node first cannot be guaranteed. This creates the need for additional busy state in the directory.

This can be solved by
# holding requests at home or requestor node and serve them in the order of their arrival
# If the block is busy, reject any further request to it & that request will be retried later
# If directory is busy forward request to dirty node & dirty node will serialize the request execution

In non-coherent scalable multiprocessors,
# For write atomicity in invalidation based protocol, current owner of block has to wait until it recieve all invalidation acks and then only it can read/write new value.
# For write completion, current owner of the block need to wait for ack from memory

In Origin, there are 3 busy states indicates that the directory has received the request but is waiting for completion hence another request for the same block won't be accepted.
# Busy read
# Busy read exclusive or upgrade
# Busy uncached read: In case of DMA, if the memory block is being read, no other processor should be allowed to get write access to it.

=== Poison state ===

To improve locality, origin relies on page migration. Hardware keeps reference counts on each page. On every access of memory count is incremented and compared with home node. If count is more than programmable threshold, hardware interrupts one of the local processors to copy the page. Block transfer engine “poison” the source page. Subsequent accesses by other processors receive a bus error and that removes the TLB entry for that location, read the location from the page table & update the TLB with new location. This approach is called - “Lazy TLB shootdown” which reduces the overall cost of migrating memory and changing the virtual-to-physical address mappings.

=== Additional states ===

Additional states can be introduced into the sharing list. The SSCI protocol has only one kind of sharing list state : dirty. More options can be introduced into the list to improve performance of the overall shared multiprocessors. Although additional states do increment the complexity of the protocol being utilized and implemented, it would be very much beneficial to the system when the overall performance of the system is increased. The question “Why Additional States are necessary” is answered alongside the introduction of each new state i.e. explanation for the necessity of each and every state has been provided.

Having the following states implemented in the protocol can improve performance.

1. “fresh” and “gone” – Main memory may still be able to respond to a requesting processor if the memory can differentiate between ‘cached and unmodified’ and ‘cached and modified’. Even if the cached value is elsewhere, memory may still be able to respond to the request if the data remains unmodified.

*The “fresh” state shows that memory still has an up-to-date copy of the block, even if the block has been cached and is being shared between processors. As long as the value remains unmodified by any of the processors sharing the data, the memory can keep responding by returning the requested data to the requesting processor.

*The “gone” state indicates that the block has been cached and has been modified. This shows that main memory cant return the data anymore, but has to return the pointer that points to the head of the list.

2. “clean lists” – Although the introduction of these states increase the complexity of handling each access, it does help with the overall performance. In this case, it may be helpful to differentiate the types of read accesses to the cache. One type of read access would be a ‘normal read-only’. Another type of read would be a ‘read with intent to modify later’. The memory in the state is ‘fresh’ when a read-fresh takes place. A read-clean leaves the memory in the ‘gone’ state. Just as with the dirty state, the read-clean allows a write to the block to proceed immediately. When a block is replaced in the cache and the cache is in the ‘clean’ state, no data needs to be written to memory. But with the dirty state, a write-back would be required even if the data had not been modified.

== Examples ==
=== 1 DASH Cache Coherence Protocol ===

DASH (Directory Architecture for SHared memory) is a scalable shared-memory multiprocessor currently being developed at Stanford’s Computer Systems Laboratory. DASH protocol uses point-to-point messages sent between the processors and memories to keep caches consistent.

The DASH coherence protocol is an invalidation-based ownership protocol. A memory block can be in one of three states as indicated by the associated directory entry: (i) uncached-remote, that is not cached by any remote cluster; (ii) shared-remote, that is cached in an unmodified state by one or more remote clusters; or (iii) dirty-remote, that is cached in a modified state by a single remote cluster.

Please see in the figures below: Left - Flow of Read Request to remote memory with directory in dirty-remote state. Right - Flow of Read-Exclusive Request to remote memory with directory in shared-remote state.

[[Image:DASH.jpg]]

Write back request: A dirty cache line that is replaced must be written back to memory. If the home of the memory block is the local cluster, then the data is simply written back to main memory. If the home cluster is remote, then a message is sent to the remote
home which updates the main memory and marks the block uncached-remote.

= References =

1 The DASH Cache Coherence Protocol
Daniel Lenoski, James Laudon, Kourosh Gharachorloo, Anoop Gupta, John Hennessy
May 1990
ACM SIGARCH Computer Architecture News , Proceedings of the 17th annual international symposium on Computer Architecture ISCA '90,
Volume 18 Issue 3a
Publisher: ACM Press

2. 1596-1992 IEEE standard for scalable coherent interface (SCI).
E-ISBN: 0-7381-1204-6
Year: 1993
Sponsored by: IEEE Computer Society

3. The SGI Origin: A ccnuma Highly Scalable Server
Laudon, J.; Lenoski, D.;
Computer Architecture, 1997. Conference Proceedings. The 24th Annual International Symposium on
June 2-4, 1997 Page(s):241 - 251

4. The Scalable Coherence Interface
David.B.Gustavson
September 1991
"The Scalable Coherence Interface and related standard projects"

5. An analysis of the Scalable Coherent Interface
Eric Rotenberg
June 1995
An Analytical model of the SCI Coherence protocol

6. Interfaces and Standards : SCI
Behrooz. Parhami
1999
Introduction to Parallel Processing: Algorithms and Architectures

7. SCI: Scalable Coherent Interface
Hermann Hellwagner
1999
SCI: Scalable Coherent Interface - Architecture and software for high-performance Compute Clusters

= Further reading =

1. The Stanford Dash multiprocessor
Lenoski, D.; Laudon, J.; Gharachorloo, K.; Weber, W.-D.; Gupta, A.; Hennessy, J.; Horowitz, M.; Lam, M.S.;
Computer
IEEE Volume 25, Issue 3, March 1992 Page(s):63 - 79
Digital Object Identifier 10.1109/2.121510

= External links =

CSC/ECE 506 Fall 2007/wiki3 8 38

2007-10-18T23:14:43Z

Arangan: /* References */

''Wiki: SCI. The IEEE Scalable Coherent Interface is a superset of the SSCI protocol we have been considering in class. A lot has been written about it, but it is still difficult to comprehend. Using SSCI as a starting point, explain why additional states are necessary, and give (or cite) examples that demonstrate how they work. Ideally, this would still be an overview of the working of the protocol, referencing more detailed documentation on the Web.''

__TOC__

= Introduction =

In parallel systems, data can be obtained from a variety of sources, which includes local or remote memory modules, secondary storage devices and network interfaces. It would be beneficial if certain aspects of mechanisms that are required to gain access to data are standardized such that it allows easily expandable and flexible designs. Various portions of a parallel system may be modified or even upgraded without any effect on the rest of the design if the system adheres to such standards. Also, utilization of standard interfaces forwards the integration of components and subsystems from several different vendors into efficient and usable parallel systems. The Scalable Coherent Interface is one such standard which facilitates the implementation of large-scale cache-coherent parallel systems.

= Scalable Coherent Interface (SCI) =

The scalable coherent interface has now become a chief hardware based approach to the cache coherence problem in shared memory multiprocessors. SCI is a directory based invalidate coherence protocol. The state of a cache block is distributed to the sharers of that block. Limits that are inherent in bus technology are easily avoided by SCI. The SCI protocol is to provide scalability, coherence and an interface. Scalability is to guarantee that the same mechanisms can be used in single processor systems and large highly parallel multiprocessors. Coherence is to guarantee efficient and integral use of cache memories in distributed shared memory. An interface provides a communication architecture that has multiple values to be brought into a single system and provide smooth inter-operation.

===Goals of SCI===

*High Performance – For distributed or parallel applications, high communication performance must be obtained when using SCI. Usual aspects of such performance would be a low latency, low CPU overhead for operations dealing with communication and high sustained throughput.

*Scalability – This is addressed in many ways. Some are
**Performance scalability – as the number of nodes being added to the system increases.
**Interconnect Distance scalability
**Memory System scalability – chiefly of cache coherence protocols, which should not have limitations to the number of processors or modules in memory that it can handle.
**Technological scalability – utilization of similar mechanisms in large, small-scale, loosely-coupled or tightly-coupled systems. Also, when new advances are made in technology, it must have the ability to readily make use of such advances.
**Economic scalability – use of similar components and mechanisms in high-end, low-volume systems as well as low-end, high-volume systems.
**Addressing capability has no short term practical limits.

*Coherent Memory System – For the purpose of reduction of Average Access Time to data, caches are becoming more important each day for microprocessors.

*Interface Characteristics – This specifies a standard interface to an interconnect that makes several devices to be connected or attached together and to interoperate.

===Operation===

In SCI, every interface does not wait for the signal to propagate before it begins to send the next signal. Also, SCI utilizes multiple links so that, concurrently, several transfers can take place.

Usually, a directory entry is in either of the two states : ‘home’ or ‘gone’. If the state is in ‘home’, then memory can immediately satisfy requests to a block as the block has not been cached by any processor. If the state is ‘gone’, then the block has been cached by a processor and might even be modified. Now, the directory contains a pointer to the first processor in the sharing list for this particular block. Hence, on requesting the data, memory returns the pointer to the first processor on the shared list rather than the data itself. The processor asking for the block now forwards its request to the processor on the top of the shared list. Now, the requesting processor adds itself into the shared list as the new head of the list.

In the SCI protocol, any coherent transaction has three phases.

Memory read – When a processor misses in its cache, It asks for the block in the home directory in memory. If the state of the memory is ‘home’, then the main memory replies with the block to the requesting processor. If the state is ‘gone’, then the main memory returns the head of the shared list of processors for that particular block. Then, memory updates its pointer and puts the requesting processor as the new head.

Cache read – When memory returns a pointer toe the requesting processor instead of data, the processor forwards its request for the block to the head of the doubly linked list. When the cache receives, the head of the list returns the data which might have been modified. The head of the list changes its backward pointer to the requesting processor’s node. Now, the requesting processor becomes the head of the list and it has the cache block.

Cleanup – If the cache miss from the requesting processor is a store, the processor has to first invalidate all other cached copies and then only proceed with the store. The new head gives an invalidate request to the next address on the list i.e. to its next processor. This processor invalidates and gives back a pointer to the next processor on its list. The head of the list uses this new pointer and sends it an invalidate request. This goes on until a NULL pointer is returned. This is the cleanup process for invalidation of other cache copies of the block.

= Simple Scalable Coherent Interface (SSCI) =

In directory-based approach, Every memory block has associated directory information; it keeps track of copies of cached blocks and their states. On a miss, it finds the directory entry, looks it up, and communicates only with the nodes that have copies (if necessary).

There are mainly two approaches: Full-bit vector: For k processors, it maintains k presence bit and 1 dirty bit at the home node. Cache state is represented the same way as in bus-based designs (MSI, MESI, etc.). It has three cache states: EM (exclusive or modified), S (shared), U (unowned). Limitation is - Number of presence bits needed grows as the number of processors.

Memory-based schemes store the information about all cached copies at the home node of the block. Cache-based schemes distribute information about copies among the copies themselves. The home contains a pointer to one cached copy of the block. Each copy contains the identity of the next node that has a copy of the block. The location of the copies is therefore determined through network transactions.

Simple SCI (SSCI) retains similarity with full-bit vector protocol: MESI states in the cache; U, S, EM states in the memory directory; It replaces the presence bits with a pointer.

== Why additional states are necessary? ==

Performance & Correctness (Coherence & Consistency) requirement for the protocol necessitates the additional states in the protocol.

=== Need for Busy state ===

On a scalable multiprocessors without coherent caches, the main-memory module determined the ordering of writes. The order that writes become visible to all processors is the order in which they reached memory.

e.g. If two processors issue read-exclusive requests for a particular word, the home will provide the requestors with the location of
the dirty node. But which request will reach the dirty node first cannot be guaranteed. This creates the need for additional busy state in the directory.

This can be solved by
# holding requests at home or requestor node and serve them in the order of their arrival
# If the block is busy, reject any further request to it & that request will be retried later
# If directory is busy forward request to dirty node & dirty node will serialize the request execution

In non-coherent scalable multiprocessors,
# For write atomicity in invalidation based protocol, current owner of block has to wait until it recieve all invalidation acks and then only it can read/write new value.
# For write completion, current owner of the block need to wait for ack from memory

In Origin, there are 3 busy states indicates that the directory has received the request but is waiting for completion hence another request for the same block won't be accepted.
# Busy read
# Busy read exclusive or upgrade
# Busy uncached read: In case of DMA, if the memory block is being read, no other processor should be allowed to get write access to it.

=== Poison state ===

To improve locality, origin relies on page migration. Hardware keeps reference counts on each page. On every access of memory count is incremented and compared with home node. If count is more than programmable threshold, hardware interrupts one of the local processors to copy the page. Block transfer engine “poison” the source page. Subsequent accesses by other processors receive a bus error and that removes the TLB entry for that location, read the location from the page table & update the TLB with new location. This approach is called - “Lazy TLB shootdown” which reduces the overall cost of migrating memory and changing the virtual-to-physical address mappings.

=== Additional states ===

Additional states can be introduced into the sharing list. The SSCI protocol has only one kind of sharing list state : dirty. More options can be introduced into the list to improve performance of the overall shared multiprocessors. Although additional states do increment the complexity of the protocol being utilized and implemented, it would be very much beneficial to the system when the overall performance of the system is increased. The question “Why Additional States are necessary” is answered alongside the introduction of each new state i.e. explanation for the necessity of each and every state has been provided.

Having the following states implemented in the protocol can improve performance.

1. “fresh” and “gone” – Main memory may still be able to respond to a requesting processor if the memory can differentiate between ‘cached and unmodified’ and ‘cached and modified’. Even if the cached value is elsewhere, memory may still be able to respond to the request if the data remains unmodified.

*The “fresh” state shows that memory still has an up-to-date copy of the block, even if the block has been cached and is being shared between processors. As long as the value remains unmodified by any of the processors sharing the data, the memory can keep responding by returning the requested data to the requesting processor.

*The “gone” state indicates that the block has been cached and has been modified. This shows that main memory cant return the data anymore, but has to return the pointer that points to the head of the list.

2. “clean lists” – Although the introduction of these states increase the complexity of handling each access, it does help with the overall performance. In this case, it may be helpful to differentiate the types of read accesses to the cache. One type of read access would be a ‘normal read-only’. Another type of read would be a ‘read with intent to modify later’. The memory in the state is ‘fresh’ when a read-fresh takes place. A read-clean leaves the memory in the ‘gone’ state. Just as with the dirty state, the read-clean allows a write to the block to proceed immediately. When a block is replaced in the cache and the cache is in the ‘clean’ state, no data needs to be written to memory. But with the dirty state, a write-back would be required even if the data had not been modified.

== Examples ==
=== 1 DASH Cache Coherence Protocol ===

DASH (Directory Architecture for SHared memory) is a scalable shared-memory multiprocessor currently being developed at Stanford’s Computer Systems Laboratory. DASH protocol uses point-to-point messages sent between the processors and memories to keep caches consistent.

The DASH coherence protocol is an invalidation-based ownership protocol. A memory block can be in one of three states as indicated by the associated directory entry: (i) uncached-remote, that is not cached by any remote cluster; (ii) shared-remote, that is cached in an unmodified state by one or more remote clusters; or (iii) dirty-remote, that is cached in a modified state by a single remote cluster.

Please see in the figures below: Left - Flow of Read Request to remote memory with directory in dirty-remote state. Right - Flow of Read-Exclusive Request to remote memory with directory in shared-remote state.

[[Image:DASH.jpg]]

Write back request: A dirty cache line that is replaced must be written back to memory. If the home of the memory block is the local cluster, then the data is simply written back to main memory. If the home cluster is remote, then a message is sent to the remote
home which updates the main memory and marks the block uncached-remote.

= References =

1 The DASH Cache Coherence Protocol
Daniel Lenoski, James Laudon, Kourosh Gharachorloo, Anoop Gupta, John Hennessy
May 1990
ACM SIGARCH Computer Architecture News , Proceedings of the 17th annual international symposium on Computer Architecture ISCA '90,
Volume 18 Issue 3a
Publisher: ACM Press

2. 1596-1992 IEEE standard for scalable coherent interface (SCI).
E-ISBN: 0-7381-1204-6
Year: 1993
Sponsored by: IEEE Computer Society

3. The SGI Origin: A ccnuma Highly Scalable Server
Laudon, J.; Lenoski, D.;
Computer Architecture, 1997. Conference Proceedings. The 24th Annual International Symposium on
June 2-4, 1997 Page(s):241 - 251

4. The Scalable Coherence Interface
David.B.Gustavson
September 1991
"The Scalable Coherence Interface and related standard projects"

5. An analysis of the Scalable Coherent Interface
Eric Rotenberg
June 1995
An Analytical model of the SCI Coherence protocol

6. Interfaces and Standards : SCI
Behrooz. Parhami
1999
Introduction to Parallel Processing: Algorithms and Architectures

7. SCI: SCalable Coherent Interface
Hermann Hellwagner
1999
SCI: Scalable Coherent Interface - Architecture and software for high-performance Compute Clusters

= Further reading =

1. The Stanford Dash multiprocessor
Lenoski, D.; Laudon, J.; Gharachorloo, K.; Weber, W.-D.; Gupta, A.; Hennessy, J.; Horowitz, M.; Lam, M.S.;
Computer
IEEE Volume 25, Issue 3, March 1992 Page(s):63 - 79
Digital Object Identifier 10.1109/2.121510

= External links =

CSC/ECE 506 Fall 2007/wiki3 8 38

2007-10-18T23:09:01Z

Arangan: /* Introduction */

''Wiki: SCI. The IEEE Scalable Coherent Interface is a superset of the SSCI protocol we have been considering in class. A lot has been written about it, but it is still difficult to comprehend. Using SSCI as a starting point, explain why additional states are necessary, and give (or cite) examples that demonstrate how they work. Ideally, this would still be an overview of the working of the protocol, referencing more detailed documentation on the Web.''

__TOC__

= Introduction =

In parallel systems, data can be obtained from a variety of sources, which includes local or remote memory modules, secondary storage devices and network interfaces. It would be beneficial if certain aspects of mechanisms that are required to gain access to data are standardized such that it allows easily expandable and flexible designs. Various portions of a parallel system may be modified or even upgraded without any effect on the rest of the design if the system adheres to such standards. Also, utilization of standard interfaces forwards the integration of components and subsystems from several different vendors into efficient and usable parallel systems. The Scalable Coherent Interface is one such standard which facilitates the implementation of large-scale cache-coherent parallel systems.

= Scalable Coherent Interface (SCI) =

The scalable coherent interface has now become a chief hardware based approach to the cache coherence problem in shared memory multiprocessors. SCI is a directory based invalidate coherence protocol. The state of a cache block is distributed to the sharers of that block. Limits that are inherent in bus technology are easily avoided by SCI. The SCI protocol is to provide scalability, coherence and an interface. Scalability is to guarantee that the same mechanisms can be used in single processor systems and large highly parallel multiprocessors. Coherence is to guarantee efficient and integral use of cache memories in distributed shared memory. An interface provides a communication architecture that has multiple values to be brought into a single system and provide smooth inter-operation.

===Goals of SCI===

*High Performance – For distributed or parallel applications, high communication performance must be obtained when using SCI. Usual aspects of such performance would be a low latency, low CPU overhead for operations dealing with communication and high sustained throughput.

*Scalability – This is addressed in many ways. Some are
**Performance scalability – as the number of nodes being added to the system increases.
**Interconnect Distance scalability
**Memory System scalability – chiefly of cache coherence protocols, which should not have limitations to the number of processors or modules in memory that it can handle.
**Technological scalability – utilization of similar mechanisms in large, small-scale, loosely-coupled or tightly-coupled systems. Also, when new advances are made in technology, it must have the ability to readily make use of such advances.
**Economic scalability – use of similar components and mechanisms in high-end, low-volume systems as well as low-end, high-volume systems.
**Addressing capability has no short term practical limits.

*Coherent Memory System – For the purpose of reduction of Average Access Time to data, caches are becoming more important each day for microprocessors.

*Interface Characteristics – This specifies a standard interface to an interconnect that makes several devices to be connected or attached together and to interoperate.

===Operation===

In SCI, every interface does not wait for the signal to propagate before it begins to send the next signal. Also, SCI utilizes multiple links so that, concurrently, several transfers can take place.

Usually, a directory entry is in either of the two states : ‘home’ or ‘gone’. If the state is in ‘home’, then memory can immediately satisfy requests to a block as the block has not been cached by any processor. If the state is ‘gone’, then the block has been cached by a processor and might even be modified. Now, the directory contains a pointer to the first processor in the sharing list for this particular block. Hence, on requesting the data, memory returns the pointer to the first processor on the shared list rather than the data itself. The processor asking for the block now forwards its request to the processor on the top of the shared list. Now, the requesting processor adds itself into the shared list as the new head of the list.

In the SCI protocol, any coherent transaction has three phases.

Memory read – When a processor misses in its cache, It asks for the block in the home directory in memory. If the state of the memory is ‘home’, then the main memory replies with the block to the requesting processor. If the state is ‘gone’, then the main memory returns the head of the shared list of processors for that particular block. Then, memory updates its pointer and puts the requesting processor as the new head.

Cache read – When memory returns a pointer toe the requesting processor instead of data, the processor forwards its request for the block to the head of the doubly linked list. When the cache receives, the head of the list returns the data which might have been modified. The head of the list changes its backward pointer to the requesting processor’s node. Now, the requesting processor becomes the head of the list and it has the cache block.

Cleanup – If the cache miss from the requesting processor is a store, the processor has to first invalidate all other cached copies and then only proceed with the store. The new head gives an invalidate request to the next address on the list i.e. to its next processor. This processor invalidates and gives back a pointer to the next processor on its list. The head of the list uses this new pointer and sends it an invalidate request. This goes on until a NULL pointer is returned. This is the cleanup process for invalidation of other cache copies of the block.

= Simple Scalable Coherent Interface (SSCI) =

In directory-based approach, Every memory block has associated directory information; it keeps track of copies of cached blocks and their states. On a miss, it finds the directory entry, looks it up, and communicates only with the nodes that have copies (if necessary).

There are mainly two approaches: Full-bit vector: For k processors, it maintains k presence bit and 1 dirty bit at the home node. Cache state is represented the same way as in bus-based designs (MSI, MESI, etc.). It has three cache states: EM (exclusive or modified), S (shared), U (unowned). Limitation is - Number of presence bits needed grows as the number of processors.

Memory-based schemes store the information about all cached copies at the home node of the block. Cache-based schemes distribute information about copies among the copies themselves. The home contains a pointer to one cached copy of the block. Each copy contains the identity of the next node that has a copy of the block. The location of the copies is therefore determined through network transactions.

Simple SCI (SSCI) retains similarity with full-bit vector protocol: MESI states in the cache; U, S, EM states in the memory directory; It replaces the presence bits with a pointer.

== Why additional states are necessary? ==

Performance & Correctness (Coherence & Consistency) requirement for the protocol necessitates the additional states in the protocol.

=== Need for Busy state ===

On a scalable multiprocessors without coherent caches, the main-memory module determined the ordering of writes. The order that writes become visible to all processors is the order in which they reached memory.

e.g. If two processors issue read-exclusive requests for a particular word, the home will provide the requestors with the location of
the dirty node. But which request will reach the dirty node first cannot be guaranteed. This creates the need for additional busy state in the directory.

This can be solved by
# holding requests at home or requestor node and serve them in the order of their arrival
# If the block is busy, reject any further request to it & that request will be retried later
# If directory is busy forward request to dirty node & dirty node will serialize the request execution

In non-coherent scalable multiprocessors,
# For write atomicity in invalidation based protocol, current owner of block has to wait until it recieve all invalidation acks and then only it can read/write new value.
# For write completion, current owner of the block need to wait for ack from memory

In Origin, there are 3 busy states indicates that the directory has received the request but is waiting for completion hence another request for the same block won't be accepted.
# Busy read
# Busy read exclusive or upgrade
# Busy uncached read: In case of DMA, if the memory block is being read, no other processor should be allowed to get write access to it.

=== Poison state ===

To improve locality, origin relies on page migration. Hardware keeps reference counts on each page. On every access of memory count is incremented and compared with home node. If count is more than programmable threshold, hardware interrupts one of the local processors to copy the page. Block transfer engine “poison” the source page. Subsequent accesses by other processors receive a bus error and that removes the TLB entry for that location, read the location from the page table & update the TLB with new location. This approach is called - “Lazy TLB shootdown” which reduces the overall cost of migrating memory and changing the virtual-to-physical address mappings.

=== Additional states ===

Additional states can be introduced into the sharing list. The SSCI protocol has only one kind of sharing list state : dirty. More options can be introduced into the list to improve performance of the overall shared multiprocessors. Although additional states do increment the complexity of the protocol being utilized and implemented, it would be very much beneficial to the system when the overall performance of the system is increased. The question “Why Additional States are necessary” is answered alongside the introduction of each new state i.e. explanation for the necessity of each and every state has been provided.

Having the following states implemented in the protocol can improve performance.

1. “fresh” and “gone” – Main memory may still be able to respond to a requesting processor if the memory can differentiate between ‘cached and unmodified’ and ‘cached and modified’. Even if the cached value is elsewhere, memory may still be able to respond to the request if the data remains unmodified.

*The “fresh” state shows that memory still has an up-to-date copy of the block, even if the block has been cached and is being shared between processors. As long as the value remains unmodified by any of the processors sharing the data, the memory can keep responding by returning the requested data to the requesting processor.

*The “gone” state indicates that the block has been cached and has been modified. This shows that main memory cant return the data anymore, but has to return the pointer that points to the head of the list.

2. “clean lists” – Although the introduction of these states increase the complexity of handling each access, it does help with the overall performance. In this case, it may be helpful to differentiate the types of read accesses to the cache. One type of read access would be a ‘normal read-only’. Another type of read would be a ‘read with intent to modify later’. The memory in the state is ‘fresh’ when a read-fresh takes place. A read-clean leaves the memory in the ‘gone’ state. Just as with the dirty state, the read-clean allows a write to the block to proceed immediately. When a block is replaced in the cache and the cache is in the ‘clean’ state, no data needs to be written to memory. But with the dirty state, a write-back would be required even if the data had not been modified.

== Examples ==
=== 1 DASH Cache Coherence Protocol ===

DASH (Directory Architecture for SHared memory) is a scalable shared-memory multiprocessor currently being developed at Stanford’s Computer Systems Laboratory. DASH protocol uses point-to-point messages sent between the processors and memories to keep caches consistent.

The DASH coherence protocol is an invalidation-based ownership protocol. A memory block can be in one of three states as indicated by the associated directory entry: (i) uncached-remote, that is not cached by any remote cluster; (ii) shared-remote, that is cached in an unmodified state by one or more remote clusters; or (iii) dirty-remote, that is cached in a modified state by a single remote cluster.

Please see in the figures below: Left - Flow of Read Request to remote memory with directory in dirty-remote state. Right - Flow of Read-Exclusive Request to remote memory with directory in shared-remote state.

[[Image:DASH.jpg]]

Write back request: A dirty cache line that is replaced must be written back to memory. If the home of the memory block is the local cluster, then the data is simply written back to main memory. If the home cluster is remote, then a message is sent to the remote
home which updates the main memory and marks the block uncached-remote.

= References =

1 The DASH Cache Coherence Protocol
Daniel Lenoski, James Laudon, Kourosh Gharachorloo, Anoop Gupta, John Hennessy
May 1990
ACM SIGARCH Computer Architecture News , Proceedings of the 17th annual international symposium on Computer Architecture ISCA '90,
Volume 18 Issue 3a
Publisher: ACM Press

2. 1596-1992 IEEE standard for scalable coherent interface (SCI).
E-ISBN: 0-7381-1204-6
Year: 1993
Sponsored by: IEEE Computer Society

3. The SGI Origin: A ccnuma Highly Scalable Server
Laudon, J.; Lenoski, D.;
Computer Architecture, 1997. Conference Proceedings. The 24th Annual International Symposium on
June 2-4, 1997 Page(s):241 - 251

4. The Scalable Coherence Interface
David.B.Gustavson
September 1991
"The Scalable Coherence Interface and related standard projects"

5. An analysis of the Scalable Coherent Interface
Eric Rotenberg
June 1995
An Analytical model of the SCI Coherence protocol

= Further reading =

1. The Stanford Dash multiprocessor
Lenoski, D.; Laudon, J.; Gharachorloo, K.; Weber, W.-D.; Gupta, A.; Hennessy, J.; Horowitz, M.; Lam, M.S.;
Computer
IEEE Volume 25, Issue 3, March 1992 Page(s):63 - 79
Digital Object Identifier 10.1109/2.121510

= External links =

CSC/ECE 506 Fall 2007/wiki3 8 38

2007-10-18T22:34:38Z

Arangan: /* Need for Busy state */

''Wiki: SCI. The IEEE Scalable Coherent Interface is a superset of the SSCI protocol we have been considering in class. A lot has been written about it, but it is still difficult to comprehend. Using SSCI as a starting point, explain why additional states are necessary, and give (or cite) examples that demonstrate how they work. Ideally, this would still be an overview of the working of the protocol, referencing more detailed documentation on the Web.''

__TOC__

= Introduction =

= Scalable Coherent Interface (SCI) =

The scalable coherent interface has now become a chief hardware based approach to the cache coherence problem in shared memory multiprocessors. SCI is a directory based invalidate coherence protocol. The state of a cache block is distributed to the sharers of that block. Limits that are inherent in bus technology are easily avoided by SCI. The SCI protocol is to provide scalability, coherence and an interface. Scalability is to guarantee that the same mechanisms can be used in single processor systems and large highly parallel multiprocessors. Coherence is to guarantee efficient and integral use of cache memories in distributed shared memory. An interface provides a communication architecture that has multiple values to be brought into a single system and provide smooth inter-operation.

===Goals of SCI===

*High Performance – For distributed or parallel applications, high communication performance must be obtained when using SCI. Usual aspects of such performance would be a low latency, low CPU overhead for operations dealing with communication and high sustained throughput.

*Scalability – This is addressed in many ways. Some are
**Performance scalability – as the number of nodes being added to the system increases.
**Interconnect Distance scalability
**Memory System scalability – chiefly of cache coherence protocols, which should not have limitations to the number of processors or modules in memory that it can handle.
**Technological scalability – utilization of similar mechanisms in large, small-scale, loosely-coupled or tightly-coupled systems. Also, when new advances are made in technology, it must have the ability to readily make use of such advances.
**Economic scalability – use of similar components and mechanisms in high-end, low-volume systems as well as low-end, high-volume systems.
**Addressing capability has no short term practical limits.

*Coherent Memory System – For the purpose of reduction of Average Access Time to data, caches are becoming more important each day for microprocessors.

*Interface Characteristics – This specifies a standard interface to an interconnect that makes several devices to be connected or attached together and to interoperate.

===Operation===

In SCI, every interface does not wait for the signal to propagate before it begins to send the next signal. Also, SCI utilizes multiple links so that, concurrently, several transfers can take place.

Usually, a directory entry is in either of the two states : ‘home’ or ‘gone’. If the state is in ‘home’, then memory can immediately satisfy requests to a block as the block has not been cached by any processor. If the state is ‘gone’, then the block has been cached by a processor and might even be modified. Now, the directory contains a pointer to the first processor in the sharing list for this particular block. Hence, on requesting the data, memory returns the pointer to the first processor on the shared list rather than the data itself. The processor asking for the block now forwards its request to the processor on the top of the shared list. Now, the requesting processor adds itself into the shared list as the new head of the list.

In the SCI protocol, any coherent transaction has three phases.

Memory read – When a processor misses in its cache, It asks for the block in the home directory in memory. If the state of the memory is ‘home’, then the main memory replies with the block to the requesting processor. If the state is ‘gone’, then the main memory returns the head of the shared list of processors for that particular block. Then, memory updates its pointer and puts the requesting processor as the new head.

Cache read – When memory returns a pointer toe the requesting processor instead of data, the processor forwards its request for the block to the head of the doubly linked list. When the cache receives, the head of the list returns the data which might have been modified. The head of the list changes its backward pointer to the requesting processor’s node. Now, the requesting processor becomes the head of the list and it has the cache block.

Cleanup – If the cache miss from the requesting processor is a store, the processor has to first invalidate all other cached copies and then only proceed with the store. The new head gives an invalidate request to the next address on the list i.e. to its next processor. This processor invalidates and gives back a pointer to the next processor on its list. The head of the list uses this new pointer and sends it an invalidate request. This goes on until a NULL pointer is returned. This is the cleanup process for invalidation of other cache copies of the block.

= Simple Scalable Coherent Interface (SSCI) =

In directory-based approach, Every memory block has associated directory information; it keeps track of copies of cached blocks and their states. On a miss, it finds the directory entry, looks it up, and communicates only with the nodes that have copies (if necessary).

There are mainly two approaches: Full-bit vector: For k processors, it maintains k presence bit and 1 dirty bit at the home node. Cache state is represented the same way as in bus-based designs (MSI, MESI, etc.). It has three cache states: EM (exclusive or modified), S (shared), U (unowned). Limitation is - Number of presence bits needed grows as the number of processors.

Memory-based schemes store the information about all cached copies at the home node of the block. Cache-based schemes distribute information about copies among the copies themselves. The home contains a pointer to one cached copy of the block. Each copy contains the identity of the next node that has a copy of the block. The location of the copies is therefore determined through network transactions.

Simple SCI (SSCI) retains similarity with full-bit vector protocol: MESI states in the cache; U, S, EM states in the memory directory; It replaces the presence bits with a pointer.

== Why additional states are necessary? ==

Performance & Correctness (Coherence & Consistency) requirement for the protocol necessitates the additional states in the protocol.

=== Need for Busy state ===

On a scalable multiprocessors without coherent caches, the main-memory module determined the ordering of writes. The order that writes become visible to all processors is the order in which they reached memory.

e.g. If two processors issue read-exclusive requests for a particular word, the home will provide the requestors with the location of
the dirty node. But which request will reach the dirty node first cannot be guaranteed. This creates the need for additional busy state in the directory.

This can be solved by
# holding requests at home or requestor node and serve them in the order of their arrival
# If the block is busy, reject any further request to it & that request will be retried later
# If directory is busy forward request to dirty node & dirty node will serialize the request execution

In non-coherent scalable multiprocessors,
# For write atomicity in invalidation based protocol, current owner of block has to wait until it recieve all invalidation acks and then only it can read/write new value.
# For write completion, current owner of the block need to wait for ack from memory

In Origin, there are 3 busy states indicates that the directory has received the request but is waiting for completion hence another request for the same block won't be accepted.
# Busy read
# Busy read exclusive or upgrade
# Busy uncached read: In case of DMA, if the memory block is being read, no other processor should be allowed to get write access to it.

=== Poison state ===

To improve locality, origin relies on page migration. Hardware keeps reference counts on each page. On every access of memory count is incremented and compared with home node. If count is more than programmable threshold, hardware interrupts one of the local processors to copy the page. Block transfer engine “poison” the source page. Subsequent accesses by other processors receive a bus error and that removes the TLB entry for that location, read the location from the page table & update the TLB with new location. This approach is called - “Lazy TLB shootdown” which reduces the overall cost of migrating memory and changing the virtual-to-physical address mappings.

=== Additional states ===

Additional states can be introduced into the sharing list. The SSCI protocol has only one kind of sharing list state : dirty. More options can be introduced into the list to improve performance of the overall shared multiprocessors. Although additional states do increment the complexity of the protocol being utilized and implemented, it would be very much beneficial to the system when the overall performance of the system is increased. The question “Why Additional States are necessary” is answered alongside the introduction of each new state i.e. explanation for the necessity of each and every state has been provided.

Having the following states implemented in the protocol can improve performance.

1. “fresh” and “gone” – Main memory may still be able to respond to a requesting processor if the memory can differentiate between ‘cached and unmodified’ and ‘cached and modified’. Even if the cached value is elsewhere, memory may still be able to respond to the request if the data remains unmodified.

*The “fresh” state shows that memory still has an up-to-date copy of the block, even if the block has been cached and is being shared between processors. As long as the value remains unmodified by any of the processors sharing the data, the memory can keep responding by returning the requested data to the requesting processor.

*The “gone” state indicates that the block has been cached and has been modified. This shows that main memory cant return the data anymore, but has to return the pointer that points to the head of the list.

2. “clean lists” – Although the introduction of these states increase the complexity of handling each access, it does help with the overall performance. In this case, it may be helpful to differentiate the types of read accesses to the cache. One type of read access would be a ‘normal read-only’. Another type of read would be a ‘read with intent to modify later’. The memory in the state is ‘fresh’ when a read-fresh takes place. A read-clean leaves the memory in the ‘gone’ state. Just as with the dirty state, the read-clean allows a write to the block to proceed immediately. When a block is replaced in the cache and the cache is in the ‘clean’ state, no data needs to be written to memory. But with the dirty state, a write-back would be required even if the data had not been modified.

== Examples ==
=== 1 DASH Cache Coherence Protocol ===

DASH (Directory Architecture for SHared memory) is a scalable shared-memory multiprocessor currently being developed at Stanford’s Computer Systems Laboratory. DASH protocol uses point-to-point messages sent between the processors and memories to keep caches consistent.

The DASH coherence protocol is an invalidation-based ownership protocol. A memory block can be in one of three states as indicated by the associated directory entry: (i) uncached-remote, that is not cached by any remote cluster; (ii) shared-remote, that is cached in an unmodified state by one or more remote clusters; or (iii) dirty-remote, that is cached in a modified state by a single remote cluster.

Please see in the figures below: Left - Flow of Read Request to remote memory with directory in dirty-remote state. Right - Flow of Read-Exclusive Request to remote memory with directory in shared-remote state.

[[Image:DASH.jpg]]

Write back request: A dirty cache line that is replaced must be written back to memory. If the home of the memory block is the local cluster, then the data is simply written back to main memory. If the home cluster is remote, then a message is sent to the remote
home which updates the main memory and marks the block uncached-remote.

= References =

1 The DASH Cache Coherence Protocol
Daniel Lenoski, James Laudon, Kourosh Gharachorloo, Anoop Gupta, John Hennessy
May 1990
ACM SIGARCH Computer Architecture News , Proceedings of the 17th annual international symposium on Computer Architecture ISCA '90,
Volume 18 Issue 3a
Publisher: ACM Press

2. 1596-1992 IEEE standard for scalable coherent interface (SCI).
E-ISBN: 0-7381-1204-6
Year: 1993
Sponsored by: IEEE Computer Society

3. The SGI Origin: A ccnuma Highly Scalable Server
Laudon, J.; Lenoski, D.;
Computer Architecture, 1997. Conference Proceedings. The 24th Annual International Symposium on
June 2-4, 1997 Page(s):241 - 251

4. The Scalable Coherence Interface
David.B.Gustavson
September 1991
"The Scalable Coherence Interface and related standard projects"

5. An analysis of the Scalable Coherent Interface
Eric Rotenberg
June 1995
An Analytical model of the SCI Coherence protocol

= Further reading =

1. The Stanford Dash multiprocessor
Lenoski, D.; Laudon, J.; Gharachorloo, K.; Weber, W.-D.; Gupta, A.; Hennessy, J.; Horowitz, M.; Lam, M.S.;
Computer
IEEE Volume 25, Issue 3, March 1992 Page(s):63 - 79
Digital Object Identifier 10.1109/2.121510

= External links =

CSC/ECE 506 Fall 2007/wiki3 8 38

2007-10-18T22:34:24Z

Arangan: /* Scalable Coherent Interface (SCI) */

''Wiki: SCI. The IEEE Scalable Coherent Interface is a superset of the SSCI protocol we have been considering in class. A lot has been written about it, but it is still difficult to comprehend. Using SSCI as a starting point, explain why additional states are necessary, and give (or cite) examples that demonstrate how they work. Ideally, this would still be an overview of the working of the protocol, referencing more detailed documentation on the Web.''

__TOC__

= Introduction =

= Scalable Coherent Interface (SCI) =

The scalable coherent interface has now become a chief hardware based approach to the cache coherence problem in shared memory multiprocessors. SCI is a directory based invalidate coherence protocol. The state of a cache block is distributed to the sharers of that block. Limits that are inherent in bus technology are easily avoided by SCI. The SCI protocol is to provide scalability, coherence and an interface. Scalability is to guarantee that the same mechanisms can be used in single processor systems and large highly parallel multiprocessors. Coherence is to guarantee efficient and integral use of cache memories in distributed shared memory. An interface provides a communication architecture that has multiple values to be brought into a single system and provide smooth inter-operation.

===Goals of SCI===

*High Performance – For distributed or parallel applications, high communication performance must be obtained when using SCI. Usual aspects of such performance would be a low latency, low CPU overhead for operations dealing with communication and high sustained throughput.

*Scalability – This is addressed in many ways. Some are
**Performance scalability – as the number of nodes being added to the system increases.
**Interconnect Distance scalability
**Memory System scalability – chiefly of cache coherence protocols, which should not have limitations to the number of processors or modules in memory that it can handle.
**Technological scalability – utilization of similar mechanisms in large, small-scale, loosely-coupled or tightly-coupled systems. Also, when new advances are made in technology, it must have the ability to readily make use of such advances.
**Economic scalability – use of similar components and mechanisms in high-end, low-volume systems as well as low-end, high-volume systems.
**Addressing capability has no short term practical limits.

*Coherent Memory System – For the purpose of reduction of Average Access Time to data, caches are becoming more important each day for microprocessors.

*Interface Characteristics – This specifies a standard interface to an interconnect that makes several devices to be connected or attached together and to interoperate.

===Operation===

In SCI, every interface does not wait for the signal to propagate before it begins to send the next signal. Also, SCI utilizes multiple links so that, concurrently, several transfers can take place.

Usually, a directory entry is in either of the two states : ‘home’ or ‘gone’. If the state is in ‘home’, then memory can immediately satisfy requests to a block as the block has not been cached by any processor. If the state is ‘gone’, then the block has been cached by a processor and might even be modified. Now, the directory contains a pointer to the first processor in the sharing list for this particular block. Hence, on requesting the data, memory returns the pointer to the first processor on the shared list rather than the data itself. The processor asking for the block now forwards its request to the processor on the top of the shared list. Now, the requesting processor adds itself into the shared list as the new head of the list.

In the SCI protocol, any coherent transaction has three phases.

Memory read – When a processor misses in its cache, It asks for the block in the home directory in memory. If the state of the memory is ‘home’, then the main memory replies with the block to the requesting processor. If the state is ‘gone’, then the main memory returns the head of the shared list of processors for that particular block. Then, memory updates its pointer and puts the requesting processor as the new head.

Cache read – When memory returns a pointer toe the requesting processor instead of data, the processor forwards its request for the block to the head of the doubly linked list. When the cache receives, the head of the list returns the data which might have been modified. The head of the list changes its backward pointer to the requesting processor’s node. Now, the requesting processor becomes the head of the list and it has the cache block.

Cleanup – If the cache miss from the requesting processor is a store, the processor has to first invalidate all other cached copies and then only proceed with the store. The new head gives an invalidate request to the next address on the list i.e. to its next processor. This processor invalidates and gives back a pointer to the next processor on its list. The head of the list uses this new pointer and sends it an invalidate request. This goes on until a NULL pointer is returned. This is the cleanup process for invalidation of other cache copies of the block.

= Simple Scalable Coherent Interface (SSCI) =

In directory-based approach, Every memory block has associated directory information; it keeps track of copies of cached blocks and their states. On a miss, it finds the directory entry, looks it up, and communicates only with the nodes that have copies (if necessary).

There are mainly two approaches: Full-bit vector: For k processors, it maintains k presence bit and 1 dirty bit at the home node. Cache state is represented the same way as in bus-based designs (MSI, MESI, etc.). It has three cache states: EM (exclusive or modified), S (shared), U (unowned). Limitation is - Number of presence bits needed grows as the number of processors.

Memory-based schemes store the information about all cached copies at the home node of the block. Cache-based schemes distribute information about copies among the copies themselves. The home contains a pointer to one cached copy of the block. Each copy contains the identity of the next node that has a copy of the block. The location of the copies is therefore determined through network transactions.

Simple SCI (SSCI) retains similarity with full-bit vector protocol: MESI states in the cache; U, S, EM states in the memory directory; It replaces the presence bits with a pointer.

== Why additional states are necessary? ==

Performance & Correctness (Coherence & Consistency) requirement for the protocol necessitates the additional states in the protocol.

=== Need for Busy state ===

On a scalable multiprocessors without coherent caches, the main-memory module determined the ordering of writes. The order that writes become visible to all processors is the order in which they reached memory.

e.g. If two processors issue read-exclusive requests for a particular word, the home will provide the requestors with the location of
the dirty node. But which request will reach the dirty node first cannot be guaranteed. This creates the need for additional busy state in the directory.

This can be solved by
# holding requests at home or requestor node and serve them in the order of their arrival
# If the block is busy, reject any further request to it & that request will be retried later
# If directory is busy forward request to dirty node & dirty node will serialize the request execution

In non-coherent scalable multiprocessors,
# For write atomicity in invalidation based protocol, current owner of block has to wait until it recieve all invalidation acks and then only it can read/write new value.
# For write completion, current owner of the block need to wait for ack from memory

In Origin, there are 3 busy states indicates that the directory has received the request but is waiting for completion hence another request for the same block won't be accepted.
# Busy read
# Busy read exclusive or upgrade
# Busy uncached read: In case of DMA, if the memory block is being read, no other processor should be allowed to get write access to it.

=== Poison state ===

To improve locality, origin relies on page migration. Hardware keeps reference counts on each page. On every access of memory count is incremented and compared with home node. If count is more than programmable threshold, hardware interrupts one of the local processors to copy the page. Block transfer engine “poison” the source page. Subsequent accesses by other processors receive a bus error and that removes the TLB entry for that location, read the location from the page table & update the TLB with new location. This approach is called - “Lazy TLB shootdown” which reduces the overall cost of migrating memory and changing the virtual-to-physical address mappings.

=== Additional states ===

Additional states can be introduced into the sharing list. The SSCI protocol has only one kind of sharing list state : dirty. More options can be introduced into the list to improve performance of the overall shared multiprocessors. Although additional states do increment the complexity of the protocol being utilized and implemented, it would be very much beneficial to the system when the overall performance of the system is increased. The question “Why Additional States are necessary” is answered alongside the introduction of each new state i.e. explanation for the necessity of each and every state has been provided.

Having the following states implemented in the protocol can improve performance.

1. “fresh” and “gone” – Main memory may still be able to respond to a requesting processor if the memory can differentiate between ‘cached and unmodified’ and ‘cached and modified’. Even if the cached value is elsewhere, memory may still be able to respond to the request if the data remains unmodified.

*The “fresh” state shows that memory still has an up-to-date copy of the block, even if the block has been cached and is being shared between processors. As long as the value remains unmodified by any of the processors sharing the data, the memory can keep responding by returning the requested data to the requesting processor.

*The “gone” state indicates that the block has been cached and has been modified. This shows that main memory cant return the data anymore, but has to return the pointer that points to the head of the list.

2. “clean lists” – Although the introduction of these states increase the complexity of handling each access, it does help with the overall performance. In this case, it may be helpful to differentiate the types of read accesses to the cache. One type of read access would be a ‘normal read-only’. Another type of read would be a ‘read with intent to modify later’. The memory in the state is ‘fresh’ when a read-fresh takes place. A read-clean leaves the memory in the ‘gone’ state. Just as with the dirty state, the read-clean allows a write to the block to proceed immediately. When a block is replaced in the cache and the cache is in the ‘clean’ state, no data needs to be written to memory. But with the dirty state, a write-back would be required even if the data had not been modified.

== Examples ==
=== 1 DASH Cache Coherence Protocol ===

DASH (Directory Architecture for SHared memory) is a scalable shared-memory multiprocessor currently being developed at Stanford’s Computer Systems Laboratory. DASH protocol uses point-to-point messages sent between the processors and memories to keep caches consistent.

The DASH coherence protocol is an invalidation-based ownership protocol. A memory block can be in one of three states as indicated by the associated directory entry: (i) uncached-remote, that is not cached by any remote cluster; (ii) shared-remote, that is cached in an unmodified state by one or more remote clusters; or (iii) dirty-remote, that is cached in a modified state by a single remote cluster.

Please see in the figures below: Left - Flow of Read Request to remote memory with directory in dirty-remote state. Right - Flow of Read-Exclusive Request to remote memory with directory in shared-remote state.

[[Image:DASH.jpg]]

Write back request: A dirty cache line that is replaced must be written back to memory. If the home of the memory block is the local cluster, then the data is simply written back to main memory. If the home cluster is remote, then a message is sent to the remote
home which updates the main memory and marks the block uncached-remote.

= References =

1 The DASH Cache Coherence Protocol
Daniel Lenoski, James Laudon, Kourosh Gharachorloo, Anoop Gupta, John Hennessy
May 1990
ACM SIGARCH Computer Architecture News , Proceedings of the 17th annual international symposium on Computer Architecture ISCA '90,
Volume 18 Issue 3a
Publisher: ACM Press

2. 1596-1992 IEEE standard for scalable coherent interface (SCI).
E-ISBN: 0-7381-1204-6
Year: 1993
Sponsored by: IEEE Computer Society

3. The SGI Origin: A ccnuma Highly Scalable Server
Laudon, J.; Lenoski, D.;
Computer Architecture, 1997. Conference Proceedings. The 24th Annual International Symposium on
June 2-4, 1997 Page(s):241 - 251

4. The Scalable Coherence Interface
David.B.Gustavson
September 1991
"The Scalable Coherence Interface and related standard projects"

5. An analysis of the Scalable Coherent Interface
Eric Rotenberg
June 1995
An Analytical model of the SCI Coherence protocol

= Further reading =

1. The Stanford Dash multiprocessor
Lenoski, D.; Laudon, J.; Gharachorloo, K.; Weber, W.-D.; Gupta, A.; Hennessy, J.; Horowitz, M.; Lam, M.S.;
Computer
IEEE Volume 25, Issue 3, March 1992 Page(s):63 - 79
Digital Object Identifier 10.1109/2.121510

= External links =

CSC/ECE 506 Fall 2007/wiki3 8 38

2007-10-18T22:33:42Z

Arangan: /* Scalable Coherent Interface (SCI) */

''Wiki: SCI. The IEEE Scalable Coherent Interface is a superset of the SSCI protocol we have been considering in class. A lot has been written about it, but it is still difficult to comprehend. Using SSCI as a starting point, explain why additional states are necessary, and give (or cite) examples that demonstrate how they work. Ideally, this would still be an overview of the working of the protocol, referencing more detailed documentation on the Web.''

__TOC__

= Introduction =

= Scalable Coherent Interface (SCI) =

The scalable coherent interface has now become a chief hardware based approach to the cache coherence problem in shared memory multiprocessors. SCI is a directory based invalidate coherence protocol. The state of a cache block is distributed to the sharers of that block. Limits that are inherent in bus technology are easily avoided by SCI. The SCI protocol is to provide scalability, coherence and an interface. Scalability is to guarantee that the same mechanisms can be used in single processor systems and large highly parallel multiprocessors. Coherence is to guarantee efficient and integral use of cache memories in distributed shared memory. An interface provides a communication architecture that has multiple values to be brought into a single system and provide smooth inter-operation.

===Goals of SCI===

*High Performance – For distributed or parallel applications, high communication performance must be obtained when using SCI. Usual aspects of such performance would be a low latency, low CPU overhead for operations dealing with communication and high sustained throughput.

*Scalability – This is addressed in many ways. Some are
**Performance scalability – as the number of nodes being added to the system increases.
**Interconnect Distance scalability
**Memory System scalability – chiefly of cache coherence protocols, which should not have limitations to the number of processors or modules in memory that it can handle.
**Technological scalability – utilization of similar mechanisms in large, small-scale, loosely-coupled or tightly-coupled systems. Also, when new advances are made in technology, it must have the ability to readily make use of such advances.
**Economic scalability – use of similar components and mechanisms in high-end, low-volume systems as well as low-end, high-volume systems.
**Addressing capability has no short term practical limits.

*Coherent Memory System – For the purpose of reduction of Average Access Time to data, caches are becoming more important each day for microprocessors.

*Interface Characteristics – This specifies a standard interface to an interconnect that makes several devices to be connected or attached together and to interoperate.

In SCI, every interface does not wait for the signal to propagate before it begins to send the next signal. Also, SCI utilizes multiple links so that, concurrently, several transfers can take place.

Usually, a directory entry is in either of the two states : ‘home’ or ‘gone’. If the state is in ‘home’, then memory can immediately satisfy requests to a block as the block has not been cached by any processor. If the state is ‘gone’, then the block has been cached by a processor and might even be modified. Now, the directory contains a pointer to the first processor in the sharing list for this particular block. Hence, on requesting the data, memory returns the pointer to the first processor on the shared list rather than the data itself. The processor asking for the block now forwards its request to the processor on the top of the shared list. Now, the requesting processor adds itself into the shared list as the new head of the list.

In the SCI protocol, any coherent transaction has three phases.

Memory read – When a processor misses in its cache, It asks for the block in the home directory in memory. If the state of the memory is ‘home’, then the main memory replies with the block to the requesting processor. If the state is ‘gone’, then the main memory returns the head of the shared list of processors for that particular block. Then, memory updates its pointer and puts the requesting processor as the new head.

Cache read – When memory returns a pointer toe the requesting processor instead of data, the processor forwards its request for the block to the head of the doubly linked list. When the cache receives, the head of the list returns the data which might have been modified. The head of the list changes its backward pointer to the requesting processor’s node. Now, the requesting processor becomes the head of the list and it has the cache block.

Cleanup – If the cache miss from the requesting processor is a store, the processor has to first invalidate all other cached copies and then only proceed with the store. The new head gives an invalidate request to the next address on the list i.e. to its next processor. This processor invalidates and gives back a pointer to the next processor on its list. The head of the list uses this new pointer and sends it an invalidate request. This goes on until a NULL pointer is returned. This is the cleanup process for invalidation of other cache copies of the block.

= Simple Scalable Coherent Interface (SSCI) =

In directory-based approach, Every memory block has associated directory information; it keeps track of copies of cached blocks and their states. On a miss, it finds the directory entry, looks it up, and communicates only with the nodes that have copies (if necessary).

There are mainly two approaches: Full-bit vector: For k processors, it maintains k presence bit and 1 dirty bit at the home node. Cache state is represented the same way as in bus-based designs (MSI, MESI, etc.). It has three cache states: EM (exclusive or modified), S (shared), U (unowned). Limitation is - Number of presence bits needed grows as the number of processors.

Memory-based schemes store the information about all cached copies at the home node of the block. Cache-based schemes distribute information about copies among the copies themselves. The home contains a pointer to one cached copy of the block. Each copy contains the identity of the next node that has a copy of the block. The location of the copies is therefore determined through network transactions.

Simple SCI (SSCI) retains similarity with full-bit vector protocol: MESI states in the cache; U, S, EM states in the memory directory; It replaces the presence bits with a pointer.

== Why additional states are necessary? ==

Performance & Correctness (Coherence & Consistency) requirement for the protocol necessitates the additional states in the protocol.

=== Need for Busy state ===

On a scalable multiprocessors without coherent caches, the main-memory module determined the ordering of writes. The order that writes become visible to all processors is the order in which they reached memory.

e.g. If two processors issue read-exclusive requests for a particular word, the home will provide the requestors with the location of
the dirty node. But which request will reach the dirty node first cannot be guaranteed. This creates the need for additional busy state in the directory.

This can be solved by
# holding requests at home or requestor node and serve them in the order of their arrival
# If the block is busy, reject any further request to it & that request will be retried later
# If directory is busy forward request to dirty node & dirty node will serialize the request execution

In non-coherent scalable multiprocessors,
# For write atomicity in invalidation based protocol, current owner of block has to wait until it recieve all invalidation acks and then only it can read/write new value.
# For write completion, current owner of the block need to wait for ack from memory

In Origin, there are 3 busy states indicates that the directory has received the request but is waiting for completion hence another request for the same block won't be accepted.
# Busy read
# Busy read exclusive or upgrade
# Busy uncached read: In case of DMA, if the memory block is being read, no other processor should be allowed to get write access to it.

=== Poison state ===

To improve locality, origin relies on page migration. Hardware keeps reference counts on each page. On every access of memory count is incremented and compared with home node. If count is more than programmable threshold, hardware interrupts one of the local processors to copy the page. Block transfer engine “poison” the source page. Subsequent accesses by other processors receive a bus error and that removes the TLB entry for that location, read the location from the page table & update the TLB with new location. This approach is called - “Lazy TLB shootdown” which reduces the overall cost of migrating memory and changing the virtual-to-physical address mappings.

=== Additional states ===

Additional states can be introduced into the sharing list. The SSCI protocol has only one kind of sharing list state : dirty. More options can be introduced into the list to improve performance of the overall shared multiprocessors. Although additional states do increment the complexity of the protocol being utilized and implemented, it would be very much beneficial to the system when the overall performance of the system is increased. The question “Why Additional States are necessary” is answered alongside the introduction of each new state i.e. explanation for the necessity of each and every state has been provided.

Having the following states implemented in the protocol can improve performance.

1. “fresh” and “gone” – Main memory may still be able to respond to a requesting processor if the memory can differentiate between ‘cached and unmodified’ and ‘cached and modified’. Even if the cached value is elsewhere, memory may still be able to respond to the request if the data remains unmodified.

*The “fresh” state shows that memory still has an up-to-date copy of the block, even if the block has been cached and is being shared between processors. As long as the value remains unmodified by any of the processors sharing the data, the memory can keep responding by returning the requested data to the requesting processor.

*The “gone” state indicates that the block has been cached and has been modified. This shows that main memory cant return the data anymore, but has to return the pointer that points to the head of the list.

2. “clean lists” – Although the introduction of these states increase the complexity of handling each access, it does help with the overall performance. In this case, it may be helpful to differentiate the types of read accesses to the cache. One type of read access would be a ‘normal read-only’. Another type of read would be a ‘read with intent to modify later’. The memory in the state is ‘fresh’ when a read-fresh takes place. A read-clean leaves the memory in the ‘gone’ state. Just as with the dirty state, the read-clean allows a write to the block to proceed immediately. When a block is replaced in the cache and the cache is in the ‘clean’ state, no data needs to be written to memory. But with the dirty state, a write-back would be required even if the data had not been modified.

== Examples ==
=== 1 DASH Cache Coherence Protocol ===

DASH (Directory Architecture for SHared memory) is a scalable shared-memory multiprocessor currently being developed at Stanford’s Computer Systems Laboratory. DASH protocol uses point-to-point messages sent between the processors and memories to keep caches consistent.

The DASH coherence protocol is an invalidation-based ownership protocol. A memory block can be in one of three states as indicated by the associated directory entry: (i) uncached-remote, that is not cached by any remote cluster; (ii) shared-remote, that is cached in an unmodified state by one or more remote clusters; or (iii) dirty-remote, that is cached in a modified state by a single remote cluster.

Please see in the figures below: Left - Flow of Read Request to remote memory with directory in dirty-remote state. Right - Flow of Read-Exclusive Request to remote memory with directory in shared-remote state.

[[Image:DASH.jpg]]

Write back request: A dirty cache line that is replaced must be written back to memory. If the home of the memory block is the local cluster, then the data is simply written back to main memory. If the home cluster is remote, then a message is sent to the remote
home which updates the main memory and marks the block uncached-remote.

= References =

1 The DASH Cache Coherence Protocol
Daniel Lenoski, James Laudon, Kourosh Gharachorloo, Anoop Gupta, John Hennessy
May 1990
ACM SIGARCH Computer Architecture News , Proceedings of the 17th annual international symposium on Computer Architecture ISCA '90,
Volume 18 Issue 3a
Publisher: ACM Press

2. 1596-1992 IEEE standard for scalable coherent interface (SCI).
E-ISBN: 0-7381-1204-6
Year: 1993
Sponsored by: IEEE Computer Society

3. The SGI Origin: A ccnuma Highly Scalable Server
Laudon, J.; Lenoski, D.;
Computer Architecture, 1997. Conference Proceedings. The 24th Annual International Symposium on
June 2-4, 1997 Page(s):241 - 251

4. The Scalable Coherence Interface
David.B.Gustavson
September 1991
"The Scalable Coherence Interface and related standard projects"

5. An analysis of the Scalable Coherent Interface
Eric Rotenberg
June 1995
An Analytical model of the SCI Coherence protocol

= Further reading =

1. The Stanford Dash multiprocessor
Lenoski, D.; Laudon, J.; Gharachorloo, K.; Weber, W.-D.; Gupta, A.; Hennessy, J.; Horowitz, M.; Lam, M.S.;
Computer
IEEE Volume 25, Issue 3, March 1992 Page(s):63 - 79
Digital Object Identifier 10.1109/2.121510

= External links =

CSC/ECE 506 Fall 2007/wiki3 8 38

2007-10-18T22:32:59Z

Arangan: /* Additional states */

''Wiki: SCI. The IEEE Scalable Coherent Interface is a superset of the SSCI protocol we have been considering in class. A lot has been written about it, but it is still difficult to comprehend. Using SSCI as a starting point, explain why additional states are necessary, and give (or cite) examples that demonstrate how they work. Ideally, this would still be an overview of the working of the protocol, referencing more detailed documentation on the Web.''

__TOC__

= Introduction =

= Scalable Coherent Interface (SCI) =

The scalable coherent interface has now become a chief hardware based approach to the cache coherence problem in shared memory multiprocessors. SCI is a directory based invalidate coherence protocol. The state of a cache block is distributed to the sharers of that block. Limits that are inherent in bus technology are easily avoided by SCI. The SCI protocol is to provide scalability, coherence and an interface. Scalability is to guarantee that the same mechanisms can be used in single processor systems and large highly parallel multiprocessors. Coherence is to guarantee efficient and integral use of cache memories in distributed shared memory. An interface provides a communication architecture that has multiple values to be brought into a single system and provide smooth inter-operation.

Goals of SCI

*High Performance – For distributed or parallel applications, high communication performance must be obtained when using SCI. Usual aspects of such performance would be a low latency, low CPU overhead for operations dealing with communication and high sustained throughput.

*Scalability – This is addressed in many ways. Some are
**Performance scalability – as the number of nodes being added to the system increases.
**Interconnect Distance scalability
**Memory System scalability – chiefly of cache coherence protocols, which should not have limitations to the number of processors or modules in memory that it can handle.
**Technological scalability – utilization of similar mechanisms in large, small-scale, loosely-coupled or tightly-coupled systems. Also, when new advances are made in technology, it must have the ability to readily make use of such advances.
**Economic scalability – use of similar components and mechanisms in high-end, low-volume systems as well as low-end, high-volume systems.
**Addressing capability has no short term practical limits.

*Coherent Memory System – For the purpose of reduction of Average Access Time to data, caches are becoming more important each day for microprocessors.

*Interface Characteristics – This specifies a standard interface to an interconnect that makes several devices to be connected or attached together and to interoperate.

In SCI, every interface does not wait for the signal to propagate before it begins to send the next signal. Also, SCI utilizes multiple links so that, concurrently, several transfers can take place.

Usually, a directory entry is in either of the two states : ‘home’ or ‘gone’. If the state is in ‘home’, then memory can immediately satisfy requests to a block as the block has not been cached by any processor. If the state is ‘gone’, then the block has been cached by a processor and might even be modified. Now, the directory contains a pointer to the first processor in the sharing list for this particular block. Hence, on requesting the data, memory returns the pointer to the first processor on the shared list rather than the data itself. The processor asking for the block now forwards its request to the processor on the top of the shared list. Now, the requesting processor adds itself into the shared list as the new head of the list.

In the SCI protocol, any coherent transaction has three phases.

Memory read – When a processor misses in its cache, It asks for the block in the home directory in memory. If the state of the memory is ‘home’, then the main memory replies with the block to the requesting processor. If the state is ‘gone’, then the main memory returns the head of the shared list of processors for that particular block. Then, memory updates its pointer and puts the requesting processor as the new head.

Cache read – When memory returns a pointer toe the requesting processor instead of data, the processor forwards its request for the block to the head of the doubly linked list. When the cache receives, the head of the list returns the data which might have been modified. The head of the list changes its backward pointer to the requesting processor’s node. Now, the requesting processor becomes the head of the list and it has the cache block.

Cleanup – If the cache miss from the requesting processor is a store, the processor has to first invalidate all other cached copies and then only proceed with the store. The new head gives an invalidate request to the next address on the list i.e. to its next processor. This processor invalidates and gives back a pointer to the next processor on its list. The head of the list uses this new pointer and sends it an invalidate request. This goes on until a NULL pointer is returned. This is the cleanup process for invalidation of other cache copies of the block.

= Simple Scalable Coherent Interface (SSCI) =

In directory-based approach, Every memory block has associated directory information; it keeps track of copies of cached blocks and their states. On a miss, it finds the directory entry, looks it up, and communicates only with the nodes that have copies (if necessary).

There are mainly two approaches: Full-bit vector: For k processors, it maintains k presence bit and 1 dirty bit at the home node. Cache state is represented the same way as in bus-based designs (MSI, MESI, etc.). It has three cache states: EM (exclusive or modified), S (shared), U (unowned). Limitation is - Number of presence bits needed grows as the number of processors.

Memory-based schemes store the information about all cached copies at the home node of the block. Cache-based schemes distribute information about copies among the copies themselves. The home contains a pointer to one cached copy of the block. Each copy contains the identity of the next node that has a copy of the block. The location of the copies is therefore determined through network transactions.

Simple SCI (SSCI) retains similarity with full-bit vector protocol: MESI states in the cache; U, S, EM states in the memory directory; It replaces the presence bits with a pointer.

== Why additional states are necessary? ==

Performance & Correctness (Coherence & Consistency) requirement for the protocol necessitates the additional states in the protocol.

=== Need for Busy state ===

On a scalable multiprocessors without coherent caches, the main-memory module determined the ordering of writes. The order that writes become visible to all processors is the order in which they reached memory.

e.g. If two processors issue read-exclusive requests for a particular word, the home will provide the requestors with the location of
the dirty node. But which request will reach the dirty node first cannot be guaranteed. This creates the need for additional busy state in the directory.

This can be solved by
# holding requests at home or requestor node and serve them in the order of their arrival
# If the block is busy, reject any further request to it & that request will be retried later
# If directory is busy forward request to dirty node & dirty node will serialize the request execution

In non-coherent scalable multiprocessors,
# For write atomicity in invalidation based protocol, current owner of block has to wait until it recieve all invalidation acks and then only it can read/write new value.
# For write completion, current owner of the block need to wait for ack from memory

In Origin, there are 3 busy states indicates that the directory has received the request but is waiting for completion hence another request for the same block won't be accepted.
# Busy read
# Busy read exclusive or upgrade
# Busy uncached read: In case of DMA, if the memory block is being read, no other processor should be allowed to get write access to it.

=== Poison state ===

To improve locality, origin relies on page migration. Hardware keeps reference counts on each page. On every access of memory count is incremented and compared with home node. If count is more than programmable threshold, hardware interrupts one of the local processors to copy the page. Block transfer engine “poison” the source page. Subsequent accesses by other processors receive a bus error and that removes the TLB entry for that location, read the location from the page table & update the TLB with new location. This approach is called - “Lazy TLB shootdown” which reduces the overall cost of migrating memory and changing the virtual-to-physical address mappings.

=== Additional states ===

Additional states can be introduced into the sharing list. The SSCI protocol has only one kind of sharing list state : dirty. More options can be introduced into the list to improve performance of the overall shared multiprocessors. Although additional states do increment the complexity of the protocol being utilized and implemented, it would be very much beneficial to the system when the overall performance of the system is increased. The question “Why Additional States are necessary” is answered alongside the introduction of each new state i.e. explanation for the necessity of each and every state has been provided.

Having the following states implemented in the protocol can improve performance.

1. “fresh” and “gone” – Main memory may still be able to respond to a requesting processor if the memory can differentiate between ‘cached and unmodified’ and ‘cached and modified’. Even if the cached value is elsewhere, memory may still be able to respond to the request if the data remains unmodified.

*The “fresh” state shows that memory still has an up-to-date copy of the block, even if the block has been cached and is being shared between processors. As long as the value remains unmodified by any of the processors sharing the data, the memory can keep responding by returning the requested data to the requesting processor.

*The “gone” state indicates that the block has been cached and has been modified. This shows that main memory cant return the data anymore, but has to return the pointer that points to the head of the list.

2. “clean lists” – Although the introduction of these states increase the complexity of handling each access, it does help with the overall performance. In this case, it may be helpful to differentiate the types of read accesses to the cache. One type of read access would be a ‘normal read-only’. Another type of read would be a ‘read with intent to modify later’. The memory in the state is ‘fresh’ when a read-fresh takes place. A read-clean leaves the memory in the ‘gone’ state. Just as with the dirty state, the read-clean allows a write to the block to proceed immediately. When a block is replaced in the cache and the cache is in the ‘clean’ state, no data needs to be written to memory. But with the dirty state, a write-back would be required even if the data had not been modified.

== Examples ==
=== 1 DASH Cache Coherence Protocol ===

DASH (Directory Architecture for SHared memory) is a scalable shared-memory multiprocessor currently being developed at Stanford’s Computer Systems Laboratory. DASH protocol uses point-to-point messages sent between the processors and memories to keep caches consistent.

The DASH coherence protocol is an invalidation-based ownership protocol. A memory block can be in one of three states as indicated by the associated directory entry: (i) uncached-remote, that is not cached by any remote cluster; (ii) shared-remote, that is cached in an unmodified state by one or more remote clusters; or (iii) dirty-remote, that is cached in a modified state by a single remote cluster.

Please see in the figures below: Left - Flow of Read Request to remote memory with directory in dirty-remote state. Right - Flow of Read-Exclusive Request to remote memory with directory in shared-remote state.

[[Image:DASH.jpg]]

Write back request: A dirty cache line that is replaced must be written back to memory. If the home of the memory block is the local cluster, then the data is simply written back to main memory. If the home cluster is remote, then a message is sent to the remote
home which updates the main memory and marks the block uncached-remote.

= References =

1 The DASH Cache Coherence Protocol
Daniel Lenoski, James Laudon, Kourosh Gharachorloo, Anoop Gupta, John Hennessy
May 1990
ACM SIGARCH Computer Architecture News , Proceedings of the 17th annual international symposium on Computer Architecture ISCA '90,
Volume 18 Issue 3a
Publisher: ACM Press

2. 1596-1992 IEEE standard for scalable coherent interface (SCI).
E-ISBN: 0-7381-1204-6
Year: 1993
Sponsored by: IEEE Computer Society

3. The SGI Origin: A ccnuma Highly Scalable Server
Laudon, J.; Lenoski, D.;
Computer Architecture, 1997. Conference Proceedings. The 24th Annual International Symposium on
June 2-4, 1997 Page(s):241 - 251

4. The Scalable Coherence Interface
David.B.Gustavson
September 1991
"The Scalable Coherence Interface and related standard projects"

5. An analysis of the Scalable Coherent Interface
Eric Rotenberg
June 1995
An Analytical model of the SCI Coherence protocol

= Further reading =

1. The Stanford Dash multiprocessor
Lenoski, D.; Laudon, J.; Gharachorloo, K.; Weber, W.-D.; Gupta, A.; Hennessy, J.; Horowitz, M.; Lam, M.S.;
Computer
IEEE Volume 25, Issue 3, March 1992 Page(s):63 - 79
Digital Object Identifier 10.1109/2.121510

= External links =

CSC/ECE 506 Fall 2007/wiki3 8 38

2007-10-18T22:32:44Z

Arangan: /* Additional states */

''Wiki: SCI. The IEEE Scalable Coherent Interface is a superset of the SSCI protocol we have been considering in class. A lot has been written about it, but it is still difficult to comprehend. Using SSCI as a starting point, explain why additional states are necessary, and give (or cite) examples that demonstrate how they work. Ideally, this would still be an overview of the working of the protocol, referencing more detailed documentation on the Web.''

__TOC__

= Introduction =

= Scalable Coherent Interface (SCI) =

The scalable coherent interface has now become a chief hardware based approach to the cache coherence problem in shared memory multiprocessors. SCI is a directory based invalidate coherence protocol. The state of a cache block is distributed to the sharers of that block. Limits that are inherent in bus technology are easily avoided by SCI. The SCI protocol is to provide scalability, coherence and an interface. Scalability is to guarantee that the same mechanisms can be used in single processor systems and large highly parallel multiprocessors. Coherence is to guarantee efficient and integral use of cache memories in distributed shared memory. An interface provides a communication architecture that has multiple values to be brought into a single system and provide smooth inter-operation.

Goals of SCI

*High Performance – For distributed or parallel applications, high communication performance must be obtained when using SCI. Usual aspects of such performance would be a low latency, low CPU overhead for operations dealing with communication and high sustained throughput.

*Scalability – This is addressed in many ways. Some are
**Performance scalability – as the number of nodes being added to the system increases.
**Interconnect Distance scalability
**Memory System scalability – chiefly of cache coherence protocols, which should not have limitations to the number of processors or modules in memory that it can handle.
**Technological scalability – utilization of similar mechanisms in large, small-scale, loosely-coupled or tightly-coupled systems. Also, when new advances are made in technology, it must have the ability to readily make use of such advances.
**Economic scalability – use of similar components and mechanisms in high-end, low-volume systems as well as low-end, high-volume systems.
**Addressing capability has no short term practical limits.

*Coherent Memory System – For the purpose of reduction of Average Access Time to data, caches are becoming more important each day for microprocessors.

*Interface Characteristics – This specifies a standard interface to an interconnect that makes several devices to be connected or attached together and to interoperate.

In SCI, every interface does not wait for the signal to propagate before it begins to send the next signal. Also, SCI utilizes multiple links so that, concurrently, several transfers can take place.

Usually, a directory entry is in either of the two states : ‘home’ or ‘gone’. If the state is in ‘home’, then memory can immediately satisfy requests to a block as the block has not been cached by any processor. If the state is ‘gone’, then the block has been cached by a processor and might even be modified. Now, the directory contains a pointer to the first processor in the sharing list for this particular block. Hence, on requesting the data, memory returns the pointer to the first processor on the shared list rather than the data itself. The processor asking for the block now forwards its request to the processor on the top of the shared list. Now, the requesting processor adds itself into the shared list as the new head of the list.

In the SCI protocol, any coherent transaction has three phases.

Memory read – When a processor misses in its cache, It asks for the block in the home directory in memory. If the state of the memory is ‘home’, then the main memory replies with the block to the requesting processor. If the state is ‘gone’, then the main memory returns the head of the shared list of processors for that particular block. Then, memory updates its pointer and puts the requesting processor as the new head.

Cache read – When memory returns a pointer toe the requesting processor instead of data, the processor forwards its request for the block to the head of the doubly linked list. When the cache receives, the head of the list returns the data which might have been modified. The head of the list changes its backward pointer to the requesting processor’s node. Now, the requesting processor becomes the head of the list and it has the cache block.

Cleanup – If the cache miss from the requesting processor is a store, the processor has to first invalidate all other cached copies and then only proceed with the store. The new head gives an invalidate request to the next address on the list i.e. to its next processor. This processor invalidates and gives back a pointer to the next processor on its list. The head of the list uses this new pointer and sends it an invalidate request. This goes on until a NULL pointer is returned. This is the cleanup process for invalidation of other cache copies of the block.

= Simple Scalable Coherent Interface (SSCI) =

In directory-based approach, Every memory block has associated directory information; it keeps track of copies of cached blocks and their states. On a miss, it finds the directory entry, looks it up, and communicates only with the nodes that have copies (if necessary).

There are mainly two approaches: Full-bit vector: For k processors, it maintains k presence bit and 1 dirty bit at the home node. Cache state is represented the same way as in bus-based designs (MSI, MESI, etc.). It has three cache states: EM (exclusive or modified), S (shared), U (unowned). Limitation is - Number of presence bits needed grows as the number of processors.

Memory-based schemes store the information about all cached copies at the home node of the block. Cache-based schemes distribute information about copies among the copies themselves. The home contains a pointer to one cached copy of the block. Each copy contains the identity of the next node that has a copy of the block. The location of the copies is therefore determined through network transactions.

Simple SCI (SSCI) retains similarity with full-bit vector protocol: MESI states in the cache; U, S, EM states in the memory directory; It replaces the presence bits with a pointer.

== Why additional states are necessary? ==

Performance & Correctness (Coherence & Consistency) requirement for the protocol necessitates the additional states in the protocol.

=== Need for Busy state ===

On a scalable multiprocessors without coherent caches, the main-memory module determined the ordering of writes. The order that writes become visible to all processors is the order in which they reached memory.

e.g. If two processors issue read-exclusive requests for a particular word, the home will provide the requestors with the location of
the dirty node. But which request will reach the dirty node first cannot be guaranteed. This creates the need for additional busy state in the directory.

This can be solved by
# holding requests at home or requestor node and serve them in the order of their arrival
# If the block is busy, reject any further request to it & that request will be retried later
# If directory is busy forward request to dirty node & dirty node will serialize the request execution

In non-coherent scalable multiprocessors,
# For write atomicity in invalidation based protocol, current owner of block has to wait until it recieve all invalidation acks and then only it can read/write new value.
# For write completion, current owner of the block need to wait for ack from memory

In Origin, there are 3 busy states indicates that the directory has received the request but is waiting for completion hence another request for the same block won't be accepted.
# Busy read
# Busy read exclusive or upgrade
# Busy uncached read: In case of DMA, if the memory block is being read, no other processor should be allowed to get write access to it.

=== Poison state ===

To improve locality, origin relies on page migration. Hardware keeps reference counts on each page. On every access of memory count is incremented and compared with home node. If count is more than programmable threshold, hardware interrupts one of the local processors to copy the page. Block transfer engine “poison” the source page. Subsequent accesses by other processors receive a bus error and that removes the TLB entry for that location, read the location from the page table & update the TLB with new location. This approach is called - “Lazy TLB shootdown” which reduces the overall cost of migrating memory and changing the virtual-to-physical address mappings.

=== Additional states ===

Additional states can be introduced into the sharing list. The SSCI protocol has only one kind of sharing list state : dirty. More options can be introduced into the list to improve performance of the overall shared multiprocessors. Although additional states do increment the complexity of the protocol being utilized and implemented, it would be very much beneficial to the system when the overall performance of the system is increased. The question “Why Additional States are necessary” is answered alongside the introduction of each new state i.e. explanation for the necessity of each and every state has been provided.

Having the following states implemented in the protocol can improve performance.

1. “fresh” and “gone” – Main memory may still be able to respond to a requesting processor if the memory can differentiate between ‘cached and unmodified’ and ‘cached and modified’. Even if the cached value is elsewhere, memory may still be able to respond to the request if the data remains unmodified.

**The “fresh” state shows that memory still has an up-to-date copy of the block, even if the block has been cached and is being shared between processors. As long as the value remains unmodified by any of the processors sharing the data, the memory can keep responding by returning the requested data to the requesting processor.

**The “gone” state indicates that the block has been cached and has been modified. This shows that main memory cant return the data anymore, but has to return the pointer that points to the head of the list.

2. “clean lists” – Although the introduction of these states increase the complexity of handling each access, it does help with the overall performance. In this case, it may be helpful to differentiate the types of read accesses to the cache. One type of read access would be a ‘normal read-only’. Another type of read would be a ‘read with intent to modify later’. The memory in the state is ‘fresh’ when a read-fresh takes place. A read-clean leaves the memory in the ‘gone’ state. Just as with the dirty state, the read-clean allows a write to the block to proceed immediately. When a block is replaced in the cache and the cache is in the ‘clean’ state, no data needs to be written to memory. But with the dirty state, a write-back would be required even if the data had not been modified.

== Examples ==
=== 1 DASH Cache Coherence Protocol ===

DASH (Directory Architecture for SHared memory) is a scalable shared-memory multiprocessor currently being developed at Stanford’s Computer Systems Laboratory. DASH protocol uses point-to-point messages sent between the processors and memories to keep caches consistent.

The DASH coherence protocol is an invalidation-based ownership protocol. A memory block can be in one of three states as indicated by the associated directory entry: (i) uncached-remote, that is not cached by any remote cluster; (ii) shared-remote, that is cached in an unmodified state by one or more remote clusters; or (iii) dirty-remote, that is cached in a modified state by a single remote cluster.

Please see in the figures below: Left - Flow of Read Request to remote memory with directory in dirty-remote state. Right - Flow of Read-Exclusive Request to remote memory with directory in shared-remote state.

[[Image:DASH.jpg]]

Write back request: A dirty cache line that is replaced must be written back to memory. If the home of the memory block is the local cluster, then the data is simply written back to main memory. If the home cluster is remote, then a message is sent to the remote
home which updates the main memory and marks the block uncached-remote.

= References =

1 The DASH Cache Coherence Protocol
Daniel Lenoski, James Laudon, Kourosh Gharachorloo, Anoop Gupta, John Hennessy
May 1990
ACM SIGARCH Computer Architecture News , Proceedings of the 17th annual international symposium on Computer Architecture ISCA '90,
Volume 18 Issue 3a
Publisher: ACM Press

2. 1596-1992 IEEE standard for scalable coherent interface (SCI).
E-ISBN: 0-7381-1204-6
Year: 1993
Sponsored by: IEEE Computer Society

3. The SGI Origin: A ccnuma Highly Scalable Server
Laudon, J.; Lenoski, D.;
Computer Architecture, 1997. Conference Proceedings. The 24th Annual International Symposium on
June 2-4, 1997 Page(s):241 - 251

4. The Scalable Coherence Interface
David.B.Gustavson
September 1991
"The Scalable Coherence Interface and related standard projects"

5. An analysis of the Scalable Coherent Interface
Eric Rotenberg
June 1995
An Analytical model of the SCI Coherence protocol

= Further reading =

1. The Stanford Dash multiprocessor
Lenoski, D.; Laudon, J.; Gharachorloo, K.; Weber, W.-D.; Gupta, A.; Hennessy, J.; Horowitz, M.; Lam, M.S.;
Computer
IEEE Volume 25, Issue 3, March 1992 Page(s):63 - 79
Digital Object Identifier 10.1109/2.121510

= External links =

CSC/ECE 506 Fall 2007/wiki3 8 38

2007-10-18T22:32:12Z

Arangan: /* Scalable Coherent Interface (SCI) */

''Wiki: SCI. The IEEE Scalable Coherent Interface is a superset of the SSCI protocol we have been considering in class. A lot has been written about it, but it is still difficult to comprehend. Using SSCI as a starting point, explain why additional states are necessary, and give (or cite) examples that demonstrate how they work. Ideally, this would still be an overview of the working of the protocol, referencing more detailed documentation on the Web.''

__TOC__

= Introduction =

= Scalable Coherent Interface (SCI) =

The scalable coherent interface has now become a chief hardware based approach to the cache coherence problem in shared memory multiprocessors. SCI is a directory based invalidate coherence protocol. The state of a cache block is distributed to the sharers of that block. Limits that are inherent in bus technology are easily avoided by SCI. The SCI protocol is to provide scalability, coherence and an interface. Scalability is to guarantee that the same mechanisms can be used in single processor systems and large highly parallel multiprocessors. Coherence is to guarantee efficient and integral use of cache memories in distributed shared memory. An interface provides a communication architecture that has multiple values to be brought into a single system and provide smooth inter-operation.

Goals of SCI

*High Performance – For distributed or parallel applications, high communication performance must be obtained when using SCI. Usual aspects of such performance would be a low latency, low CPU overhead for operations dealing with communication and high sustained throughput.

*Scalability – This is addressed in many ways. Some are
**Performance scalability – as the number of nodes being added to the system increases.
**Interconnect Distance scalability
**Memory System scalability – chiefly of cache coherence protocols, which should not have limitations to the number of processors or modules in memory that it can handle.
**Technological scalability – utilization of similar mechanisms in large, small-scale, loosely-coupled or tightly-coupled systems. Also, when new advances are made in technology, it must have the ability to readily make use of such advances.
**Economic scalability – use of similar components and mechanisms in high-end, low-volume systems as well as low-end, high-volume systems.
**Addressing capability has no short term practical limits.

*Coherent Memory System – For the purpose of reduction of Average Access Time to data, caches are becoming more important each day for microprocessors.

*Interface Characteristics – This specifies a standard interface to an interconnect that makes several devices to be connected or attached together and to interoperate.

In SCI, every interface does not wait for the signal to propagate before it begins to send the next signal. Also, SCI utilizes multiple links so that, concurrently, several transfers can take place.

Usually, a directory entry is in either of the two states : ‘home’ or ‘gone’. If the state is in ‘home’, then memory can immediately satisfy requests to a block as the block has not been cached by any processor. If the state is ‘gone’, then the block has been cached by a processor and might even be modified. Now, the directory contains a pointer to the first processor in the sharing list for this particular block. Hence, on requesting the data, memory returns the pointer to the first processor on the shared list rather than the data itself. The processor asking for the block now forwards its request to the processor on the top of the shared list. Now, the requesting processor adds itself into the shared list as the new head of the list.

In the SCI protocol, any coherent transaction has three phases.

Memory read – When a processor misses in its cache, It asks for the block in the home directory in memory. If the state of the memory is ‘home’, then the main memory replies with the block to the requesting processor. If the state is ‘gone’, then the main memory returns the head of the shared list of processors for that particular block. Then, memory updates its pointer and puts the requesting processor as the new head.

Cache read – When memory returns a pointer toe the requesting processor instead of data, the processor forwards its request for the block to the head of the doubly linked list. When the cache receives, the head of the list returns the data which might have been modified. The head of the list changes its backward pointer to the requesting processor’s node. Now, the requesting processor becomes the head of the list and it has the cache block.

Cleanup – If the cache miss from the requesting processor is a store, the processor has to first invalidate all other cached copies and then only proceed with the store. The new head gives an invalidate request to the next address on the list i.e. to its next processor. This processor invalidates and gives back a pointer to the next processor on its list. The head of the list uses this new pointer and sends it an invalidate request. This goes on until a NULL pointer is returned. This is the cleanup process for invalidation of other cache copies of the block.

= Simple Scalable Coherent Interface (SSCI) =

In directory-based approach, Every memory block has associated directory information; it keeps track of copies of cached blocks and their states. On a miss, it finds the directory entry, looks it up, and communicates only with the nodes that have copies (if necessary).

There are mainly two approaches: Full-bit vector: For k processors, it maintains k presence bit and 1 dirty bit at the home node. Cache state is represented the same way as in bus-based designs (MSI, MESI, etc.). It has three cache states: EM (exclusive or modified), S (shared), U (unowned). Limitation is - Number of presence bits needed grows as the number of processors.

Memory-based schemes store the information about all cached copies at the home node of the block. Cache-based schemes distribute information about copies among the copies themselves. The home contains a pointer to one cached copy of the block. Each copy contains the identity of the next node that has a copy of the block. The location of the copies is therefore determined through network transactions.

Simple SCI (SSCI) retains similarity with full-bit vector protocol: MESI states in the cache; U, S, EM states in the memory directory; It replaces the presence bits with a pointer.

== Why additional states are necessary? ==

Performance & Correctness (Coherence & Consistency) requirement for the protocol necessitates the additional states in the protocol.

=== Need for Busy state ===

On a scalable multiprocessors without coherent caches, the main-memory module determined the ordering of writes. The order that writes become visible to all processors is the order in which they reached memory.

e.g. If two processors issue read-exclusive requests for a particular word, the home will provide the requestors with the location of
the dirty node. But which request will reach the dirty node first cannot be guaranteed. This creates the need for additional busy state in the directory.

This can be solved by
# holding requests at home or requestor node and serve them in the order of their arrival
# If the block is busy, reject any further request to it & that request will be retried later
# If directory is busy forward request to dirty node & dirty node will serialize the request execution

In non-coherent scalable multiprocessors,
# For write atomicity in invalidation based protocol, current owner of block has to wait until it recieve all invalidation acks and then only it can read/write new value.
# For write completion, current owner of the block need to wait for ack from memory

In Origin, there are 3 busy states indicates that the directory has received the request but is waiting for completion hence another request for the same block won't be accepted.
# Busy read
# Busy read exclusive or upgrade
# Busy uncached read: In case of DMA, if the memory block is being read, no other processor should be allowed to get write access to it.

=== Poison state ===

To improve locality, origin relies on page migration. Hardware keeps reference counts on each page. On every access of memory count is incremented and compared with home node. If count is more than programmable threshold, hardware interrupts one of the local processors to copy the page. Block transfer engine “poison” the source page. Subsequent accesses by other processors receive a bus error and that removes the TLB entry for that location, read the location from the page table & update the TLB with new location. This approach is called - “Lazy TLB shootdown” which reduces the overall cost of migrating memory and changing the virtual-to-physical address mappings.

=== Additional states ===

Additional states can be introduced into the sharing list. The SSCI protocol has only one kind of sharing list state : dirty. More options can be introduced into the list to improve performance of the overall shared multiprocessors. Although additional states do increment the complexity of the protocol being utilized and implemented, it would be very much beneficial to the system when the overall performance of the system is increased. The question “Why Additional States are necessary” is answered alongside the introduction of each new state i.e. explanation for the necessity of each and every state has been provided.

Having the following states implemented in the protocol can improve performance.

1. “fresh” and “gone” – Main memory may still be able to respond to a requesting processor if the memory can differentiate between ‘cached and unmodified’ and ‘cached and modified’. Even if the cached value is elsewhere, memory may still be able to respond to the request if the data remains unmodified.

The “fresh” state shows that memory still has an up-to-date copy of the block, even if the block has been cached and is being shared between processors. As long as the value remains unmodified by any of the processors sharing the data, the memory can keep responding by returning the requested data to the requesting processor.

The “gone” state indicates that the block has been cached and has been modified. This shows that main memory cant return the data anymore, but has to return the pointer that points to the head of the list.

2. “clean lists” – Although the introduction of these states increase the complexity of handling each access, it does help with the overall performance. In this case, it may be helpful to differentiate the types of read accesses to the cache. One type of read access would be a ‘normal read-only’. Another type of read would be a ‘read with intent to modify later’. The memory in the state is ‘fresh’ when a read-fresh takes place. A read-clean leaves the memory in the ‘gone’ state. Just as with the dirty state, the read-clean allows a write to the block to proceed immediately. When a block is replaced in the cache and the cache is in the ‘clean’ state, no data needs to be written to memory. But with the dirty state, a write-back would be required even if the data had not been modified.

== Examples ==
=== 1 DASH Cache Coherence Protocol ===

DASH (Directory Architecture for SHared memory) is a scalable shared-memory multiprocessor currently being developed at Stanford’s Computer Systems Laboratory. DASH protocol uses point-to-point messages sent between the processors and memories to keep caches consistent.

The DASH coherence protocol is an invalidation-based ownership protocol. A memory block can be in one of three states as indicated by the associated directory entry: (i) uncached-remote, that is not cached by any remote cluster; (ii) shared-remote, that is cached in an unmodified state by one or more remote clusters; or (iii) dirty-remote, that is cached in a modified state by a single remote cluster.

Please see in the figures below: Left - Flow of Read Request to remote memory with directory in dirty-remote state. Right - Flow of Read-Exclusive Request to remote memory with directory in shared-remote state.

[[Image:DASH.jpg]]

Write back request: A dirty cache line that is replaced must be written back to memory. If the home of the memory block is the local cluster, then the data is simply written back to main memory. If the home cluster is remote, then a message is sent to the remote
home which updates the main memory and marks the block uncached-remote.

= References =

1 The DASH Cache Coherence Protocol
Daniel Lenoski, James Laudon, Kourosh Gharachorloo, Anoop Gupta, John Hennessy
May 1990
ACM SIGARCH Computer Architecture News , Proceedings of the 17th annual international symposium on Computer Architecture ISCA '90,
Volume 18 Issue 3a
Publisher: ACM Press

2. 1596-1992 IEEE standard for scalable coherent interface (SCI).
E-ISBN: 0-7381-1204-6
Year: 1993
Sponsored by: IEEE Computer Society

3. The SGI Origin: A ccnuma Highly Scalable Server
Laudon, J.; Lenoski, D.;
Computer Architecture, 1997. Conference Proceedings. The 24th Annual International Symposium on
June 2-4, 1997 Page(s):241 - 251

4. The Scalable Coherence Interface
David.B.Gustavson
September 1991
"The Scalable Coherence Interface and related standard projects"

5. An analysis of the Scalable Coherent Interface
Eric Rotenberg
June 1995
An Analytical model of the SCI Coherence protocol

= Further reading =

1. The Stanford Dash multiprocessor
Lenoski, D.; Laudon, J.; Gharachorloo, K.; Weber, W.-D.; Gupta, A.; Hennessy, J.; Horowitz, M.; Lam, M.S.;
Computer
IEEE Volume 25, Issue 3, March 1992 Page(s):63 - 79
Digital Object Identifier 10.1109/2.121510

= External links =

CSC/ECE 506 Fall 2007/wiki3 8 38

2007-10-18T22:30:30Z

Arangan: /* Scalable Coherent Interface (SCI) */

''Wiki: SCI. The IEEE Scalable Coherent Interface is a superset of the SSCI protocol we have been considering in class. A lot has been written about it, but it is still difficult to comprehend. Using SSCI as a starting point, explain why additional states are necessary, and give (or cite) examples that demonstrate how they work. Ideally, this would still be an overview of the working of the protocol, referencing more detailed documentation on the Web.''

__TOC__

= Introduction =

= Scalable Coherent Interface (SCI) =

The scalable coherent interface has now become a chief hardware based approach to the cache coherence problem in shared memory multiprocessors. SCI is a directory based invalidate coherence protocol. The state of a cache block is distributed to the sharers of that block. Limits that are inherent in bus technology are easily avoided by SCI. The SCI protocol is to provide scalability, coherence and an interface. Scalability is to guarantee that the same mechanisms can be used in single processor systems and large highly parallel multiprocessors. Coherence is to guarantee efficient and integral use of cache memories in distributed shared memory. An interface provides a communication architecture that has multiple values to be brought into a single system and provide smooth inter-operation.

Goals of SCI

#High Performance – For distributed or parallel applications, high communication performance must be obtained when using SCI. Usual aspects of such performance would be a low latency, low CPU overhead for operations dealing with communication and high sustained throughput.

#Scalability – This is addressed in many ways. Some are
*Performance scalability – as the number of nodes being added to the system increases.
*Interconnect Distance scalability
*Memory System scalability – chiefly of cache coherence protocols, which should not have limitations to the number of processors or modules in memory that it can handle.
*Technological scalability – utilization of similar mechanisms in large, small-scale, loosely-coupled or tightly-coupled systems. Also, when new advances are made in technology, it must have the ability to readily make use of such advances.
*Economic scalability – use of similar components and mechanisms in high-end, low-volume systems as well as low-end, high-volume systems.
*Addressing capability has no short term practical limits.

#Coherent Memory System – For the purpose of reduction of Average Access Time to data, caches are becoming more important each day for microprocessors.

#Interface Characteristics – This specifies a standard interface to an interconnect that makes several devices to be connected or attached together and to interoperate.

In SCI, every interface does not wait for the signal to propagate before it begins to send the next signal. Also, SCI utilizes multiple links so that, concurrently, several transfers can take place.

Usually, a directory entry is in either of the two states : ‘home’ or ‘gone’. If the state is in ‘home’, then memory can immediately satisfy requests to a block as the block has not been cached by any processor. If the state is ‘gone’, then the block has been cached by a processor and might even be modified. Now, the directory contains a pointer to the first processor in the sharing list for this particular block. Hence, on requesting the data, memory returns the pointer to the first processor on the shared list rather than the data itself. The processor asking for the block now forwards its request to the processor on the top of the shared list. Now, the requesting processor adds itself into the shared list as the new head of the list.

In the SCI protocol, any coherent transaction has three phases.

Memory read – When a processor misses in its cache, It asks for the block in the home directory in memory. If the state of the memory is ‘home’, then the main memory replies with the block to the requesting processor. If the state is ‘gone’, then the main memory returns the head of the shared list of processors for that particular block. Then, memory updates its pointer and puts the requesting processor as the new head.

Cache read – When memory returns a pointer toe the requesting processor instead of data, the processor forwards its request for the block to the head of the doubly linked list. When the cache receives, the head of the list returns the data which might have been modified. The head of the list changes its backward pointer to the requesting processor’s node. Now, the requesting processor becomes the head of the list and it has the cache block.

Cleanup – If the cache miss from the requesting processor is a store, the processor has to first invalidate all other cached copies and then only proceed with the store. The new head gives an invalidate request to the next address on the list i.e. to its next processor. This processor invalidates and gives back a pointer to the next processor on its list. The head of the list uses this new pointer and sends it an invalidate request. This goes on until a NULL pointer is returned. This is the cleanup process for invalidation of other cache copies of the block.

= Simple Scalable Coherent Interface (SSCI) =

In directory-based approach, Every memory block has associated directory information; it keeps track of copies of cached blocks and their states. On a miss, it finds the directory entry, looks it up, and communicates only with the nodes that have copies (if necessary).

There are mainly two approaches: Full-bit vector: For k processors, it maintains k presence bit and 1 dirty bit at the home node. Cache state is represented the same way as in bus-based designs (MSI, MESI, etc.). It has three cache states: EM (exclusive or modified), S (shared), U (unowned). Limitation is - Number of presence bits needed grows as the number of processors.

Memory-based schemes store the information about all cached copies at the home node of the block. Cache-based schemes distribute information about copies among the copies themselves. The home contains a pointer to one cached copy of the block. Each copy contains the identity of the next node that has a copy of the block. The location of the copies is therefore determined through network transactions.

Simple SCI (SSCI) retains similarity with full-bit vector protocol: MESI states in the cache; U, S, EM states in the memory directory; It replaces the presence bits with a pointer.

== Why additional states are necessary? ==

Performance & Correctness (Coherence & Consistency) requirement for the protocol necessitates the additional states in the protocol.

=== Need for Busy state ===

On a scalable multiprocessors without coherent caches, the main-memory module determined the ordering of writes. The order that writes become visible to all processors is the order in which they reached memory.

e.g. If two processors issue read-exclusive requests for a particular word, the home will provide the requestors with the location of
the dirty node. But which request will reach the dirty node first cannot be guaranteed. This creates the need for additional busy state in the directory.

This can be solved by
# holding requests at home or requestor node and serve them in the order of their arrival
# If the block is busy, reject any further request to it & that request will be retried later
# If directory is busy forward request to dirty node & dirty node will serialize the request execution

In non-coherent scalable multiprocessors,
# For write atomicity in invalidation based protocol, current owner of block has to wait until it recieve all invalidation acks and then only it can read/write new value.
# For write completion, current owner of the block need to wait for ack from memory

In Origin, there are 3 busy states indicates that the directory has received the request but is waiting for completion hence another request for the same block won't be accepted.
# Busy read
# Busy read exclusive or upgrade
# Busy uncached read: In case of DMA, if the memory block is being read, no other processor should be allowed to get write access to it.

=== Poison state ===

To improve locality, origin relies on page migration. Hardware keeps reference counts on each page. On every access of memory count is incremented and compared with home node. If count is more than programmable threshold, hardware interrupts one of the local processors to copy the page. Block transfer engine “poison” the source page. Subsequent accesses by other processors receive a bus error and that removes the TLB entry for that location, read the location from the page table & update the TLB with new location. This approach is called - “Lazy TLB shootdown” which reduces the overall cost of migrating memory and changing the virtual-to-physical address mappings.

=== Additional states ===

Additional states can be introduced into the sharing list. The SSCI protocol has only one kind of sharing list state : dirty. More options can be introduced into the list to improve performance of the overall shared multiprocessors. Although additional states do increment the complexity of the protocol being utilized and implemented, it would be very much beneficial to the system when the overall performance of the system is increased. The question “Why Additional States are necessary” is answered alongside the introduction of each new state i.e. explanation for the necessity of each and every state has been provided.

Having the following states implemented in the protocol can improve performance.

1. “fresh” and “gone” – Main memory may still be able to respond to a requesting processor if the memory can differentiate between ‘cached and unmodified’ and ‘cached and modified’. Even if the cached value is elsewhere, memory may still be able to respond to the request if the data remains unmodified.

The “fresh” state shows that memory still has an up-to-date copy of the block, even if the block has been cached and is being shared between processors. As long as the value remains unmodified by any of the processors sharing the data, the memory can keep responding by returning the requested data to the requesting processor.

The “gone” state indicates that the block has been cached and has been modified. This shows that main memory cant return the data anymore, but has to return the pointer that points to the head of the list.

2. “clean lists” – Although the introduction of these states increase the complexity of handling each access, it does help with the overall performance. In this case, it may be helpful to differentiate the types of read accesses to the cache. One type of read access would be a ‘normal read-only’. Another type of read would be a ‘read with intent to modify later’. The memory in the state is ‘fresh’ when a read-fresh takes place. A read-clean leaves the memory in the ‘gone’ state. Just as with the dirty state, the read-clean allows a write to the block to proceed immediately. When a block is replaced in the cache and the cache is in the ‘clean’ state, no data needs to be written to memory. But with the dirty state, a write-back would be required even if the data had not been modified.

== Examples ==
=== 1 DASH Cache Coherence Protocol ===

DASH (Directory Architecture for SHared memory) is a scalable shared-memory multiprocessor currently being developed at Stanford’s Computer Systems Laboratory. DASH protocol uses point-to-point messages sent between the processors and memories to keep caches consistent.

The DASH coherence protocol is an invalidation-based ownership protocol. A memory block can be in one of three states as indicated by the associated directory entry: (i) uncached-remote, that is not cached by any remote cluster; (ii) shared-remote, that is cached in an unmodified state by one or more remote clusters; or (iii) dirty-remote, that is cached in a modified state by a single remote cluster.

Please see in the figures below: Left - Flow of Read Request to remote memory with directory in dirty-remote state. Right - Flow of Read-Exclusive Request to remote memory with directory in shared-remote state.

[[Image:DASH.jpg]]

Write back request: A dirty cache line that is replaced must be written back to memory. If the home of the memory block is the local cluster, then the data is simply written back to main memory. If the home cluster is remote, then a message is sent to the remote
home which updates the main memory and marks the block uncached-remote.

= References =

1 The DASH Cache Coherence Protocol
Daniel Lenoski, James Laudon, Kourosh Gharachorloo, Anoop Gupta, John Hennessy
May 1990
ACM SIGARCH Computer Architecture News , Proceedings of the 17th annual international symposium on Computer Architecture ISCA '90,
Volume 18 Issue 3a
Publisher: ACM Press

2. 1596-1992 IEEE standard for scalable coherent interface (SCI).
E-ISBN: 0-7381-1204-6
Year: 1993
Sponsored by: IEEE Computer Society

3. The SGI Origin: A ccnuma Highly Scalable Server
Laudon, J.; Lenoski, D.;
Computer Architecture, 1997. Conference Proceedings. The 24th Annual International Symposium on
June 2-4, 1997 Page(s):241 - 251

4. The Scalable Coherence Interface
David.B.Gustavson
September 1991
"The Scalable Coherence Interface and related standard projects"

5. An analysis of the Scalable Coherent Interface
Eric Rotenberg
June 1995
An Analytical model of the SCI Coherence protocol

= Further reading =

1. The Stanford Dash multiprocessor
Lenoski, D.; Laudon, J.; Gharachorloo, K.; Weber, W.-D.; Gupta, A.; Hennessy, J.; Horowitz, M.; Lam, M.S.;
Computer
IEEE Volume 25, Issue 3, March 1992 Page(s):63 - 79
Digital Object Identifier 10.1109/2.121510

= External links =

CSC/ECE 506 Fall 2007/wiki3 8 38

2007-10-18T22:28:56Z

Arangan: /* Scalable Coherent Interface (SCI) */

''Wiki: SCI. The IEEE Scalable Coherent Interface is a superset of the SSCI protocol we have been considering in class. A lot has been written about it, but it is still difficult to comprehend. Using SSCI as a starting point, explain why additional states are necessary, and give (or cite) examples that demonstrate how they work. Ideally, this would still be an overview of the working of the protocol, referencing more detailed documentation on the Web.''

__TOC__

= Introduction =

= Scalable Coherent Interface (SCI) =

The scalable coherent interface has now become a chief hardware based approach to the cache coherence problem in shared memory multiprocessors. SCI is a directory based invalidate coherence protocol. The state of a cache block is distributed to the sharers of that block. Limits that are inherent in bus technology are easily avoided by SCI. The SCI protocol is to provide scalability, coherence and an interface. Scalability is to guarantee that the same mechanisms can be used in single processor systems and large highly parallel multiprocessors. Coherence is to guarantee efficient and integral use of cache memories in distributed shared memory. An interface provides a communication architecture that has multiple values to be brought into a single system and provide smooth inter-operation.

Goals of SCI

High Performance – For distributed or parallel applications, high communication performance must be obtained when using SCI. Usual aspects of such performance would be a low latency, low CPU overhead for operations dealing with communication and high sustained throughput.

Scalability – This is addressed in many ways. Some are
Performance scalability – as the number of nodes being added to the system increases.
*Interconnect Distance scalability
*Memory System scalability – chiefly of cache coherence protocols, which should not have limitations to the number of processors or modules in memory that it can handle.
*Technological scalability – utilization of similar mechanisms in large, small-scale, loosely-coupled or tightly-coupled systems. Also, when new advances are made in technology, it must have the ability to readily make use of such advances.
*Economic scalability – use of similar components and mechanisms in high-end, low-volume systems as well as low-end, high-volume systems.
*Addressing capability has no short term practical limits.

Coherent Memory System – For the purpose of reduction of Average Access Time to data, caches are becoming more important each day for microprocessors.

Interface Characteristics – This specifies a standard interface to an interconnect that makes several devices to be connected or attached together and to interoperate.

In SCI, every interface does not wait for the signal to propagate before it begins to send the next signal. Also, SCI utilizes multiple links so that, concurrently, several transfers can take place.

Usually, a directory entry is in either of the two states : ‘home’ or ‘gone’. If the state is in ‘home’, then memory can immediately satisfy requests to a block as the block has not been cached by any processor. If the state is ‘gone’, then the block has been cached by a processor and might even be modified. Now, the directory contains a pointer to the first processor in the sharing list for this particular block. Hence, on requesting the data, memory returns the pointer to the first processor on the shared list rather than the data itself. The processor asking for the block now forwards its request to the processor on the top of the shared list. Now, the requesting processor adds itself into the shared list as the new head of the list.

In the SCI protocol, any coherent transaction has three phases.

Memory read – When a processor misses in its cache, It asks for the block in the home directory in memory. If the state of the memory is ‘home’, then the main memory replies with the block to the requesting processor. If the state is ‘gone’, then the main memory returns the head of the shared list of processors for that particular block. Then, memory updates its pointer and puts the requesting processor as the new head.

Cache read – When memory returns a pointer toe the requesting processor instead of data, the processor forwards its request for the block to the head of the doubly linked list. When the cache receives, the head of the list returns the data which might have been modified. The head of the list changes its backward pointer to the requesting processor’s node. Now, the requesting processor becomes the head of the list and it has the cache block.

Cleanup – If the cache miss from the requesting processor is a store, the processor has to first invalidate all other cached copies and then only proceed with the store. The new head gives an invalidate request to the next address on the list i.e. to its next processor. This processor invalidates and gives back a pointer to the next processor on its list. The head of the list uses this new pointer and sends it an invalidate request. This goes on until a NULL pointer is returned. This is the cleanup process for invalidation of other cache copies of the block.

= Simple Scalable Coherent Interface (SSCI) =

In directory-based approach, Every memory block has associated directory information; it keeps track of copies of cached blocks and their states. On a miss, it finds the directory entry, looks it up, and communicates only with the nodes that have copies (if necessary).

There are mainly two approaches: Full-bit vector: For k processors, it maintains k presence bit and 1 dirty bit at the home node. Cache state is represented the same way as in bus-based designs (MSI, MESI, etc.). It has three cache states: EM (exclusive or modified), S (shared), U (unowned). Limitation is - Number of presence bits needed grows as the number of processors.

Memory-based schemes store the information about all cached copies at the home node of the block. Cache-based schemes distribute information about copies among the copies themselves. The home contains a pointer to one cached copy of the block. Each copy contains the identity of the next node that has a copy of the block. The location of the copies is therefore determined through network transactions.

Simple SCI (SSCI) retains similarity with full-bit vector protocol: MESI states in the cache; U, S, EM states in the memory directory; It replaces the presence bits with a pointer.

== Why additional states are necessary? ==

Performance & Correctness (Coherence & Consistency) requirement for the protocol necessitates the additional states in the protocol.

=== Need for Busy state ===

On a scalable multiprocessors without coherent caches, the main-memory module determined the ordering of writes. The order that writes become visible to all processors is the order in which they reached memory.

e.g. If two processors issue read-exclusive requests for a particular word, the home will provide the requestors with the location of
the dirty node. But which request will reach the dirty node first cannot be guaranteed. This creates the need for additional busy state in the directory.

This can be solved by
# holding requests at home or requestor node and serve them in the order of their arrival
# If the block is busy, reject any further request to it & that request will be retried later
# If directory is busy forward request to dirty node & dirty node will serialize the request execution

In non-coherent scalable multiprocessors,
# For write atomicity in invalidation based protocol, current owner of block has to wait until it recieve all invalidation acks and then only it can read/write new value.
# For write completion, current owner of the block need to wait for ack from memory

In Origin, there are 3 busy states indicates that the directory has received the request but is waiting for completion hence another request for the same block won't be accepted.
# Busy read
# Busy read exclusive or upgrade
# Busy uncached read: In case of DMA, if the memory block is being read, no other processor should be allowed to get write access to it.

=== Poison state ===

To improve locality, origin relies on page migration. Hardware keeps reference counts on each page. On every access of memory count is incremented and compared with home node. If count is more than programmable threshold, hardware interrupts one of the local processors to copy the page. Block transfer engine “poison” the source page. Subsequent accesses by other processors receive a bus error and that removes the TLB entry for that location, read the location from the page table & update the TLB with new location. This approach is called - “Lazy TLB shootdown” which reduces the overall cost of migrating memory and changing the virtual-to-physical address mappings.

=== Additional states ===

Additional states can be introduced into the sharing list. The SSCI protocol has only one kind of sharing list state : dirty. More options can be introduced into the list to improve performance of the overall shared multiprocessors. Although additional states do increment the complexity of the protocol being utilized and implemented, it would be very much beneficial to the system when the overall performance of the system is increased. The question “Why Additional States are necessary” is answered alongside the introduction of each new state i.e. explanation for the necessity of each and every state has been provided.

Having the following states implemented in the protocol can improve performance.

1. “fresh” and “gone” – Main memory may still be able to respond to a requesting processor if the memory can differentiate between ‘cached and unmodified’ and ‘cached and modified’. Even if the cached value is elsewhere, memory may still be able to respond to the request if the data remains unmodified.

The “fresh” state shows that memory still has an up-to-date copy of the block, even if the block has been cached and is being shared between processors. As long as the value remains unmodified by any of the processors sharing the data, the memory can keep responding by returning the requested data to the requesting processor.

The “gone” state indicates that the block has been cached and has been modified. This shows that main memory cant return the data anymore, but has to return the pointer that points to the head of the list.

2. “clean lists” – Although the introduction of these states increase the complexity of handling each access, it does help with the overall performance. In this case, it may be helpful to differentiate the types of read accesses to the cache. One type of read access would be a ‘normal read-only’. Another type of read would be a ‘read with intent to modify later’. The memory in the state is ‘fresh’ when a read-fresh takes place. A read-clean leaves the memory in the ‘gone’ state. Just as with the dirty state, the read-clean allows a write to the block to proceed immediately. When a block is replaced in the cache and the cache is in the ‘clean’ state, no data needs to be written to memory. But with the dirty state, a write-back would be required even if the data had not been modified.

== Examples ==
=== 1 DASH Cache Coherence Protocol ===

DASH (Directory Architecture for SHared memory) is a scalable shared-memory multiprocessor currently being developed at Stanford’s Computer Systems Laboratory. DASH protocol uses point-to-point messages sent between the processors and memories to keep caches consistent.

The DASH coherence protocol is an invalidation-based ownership protocol. A memory block can be in one of three states as indicated by the associated directory entry: (i) uncached-remote, that is not cached by any remote cluster; (ii) shared-remote, that is cached in an unmodified state by one or more remote clusters; or (iii) dirty-remote, that is cached in a modified state by a single remote cluster.

Please see in the figures below: Left - Flow of Read Request to remote memory with directory in dirty-remote state. Right - Flow of Read-Exclusive Request to remote memory with directory in shared-remote state.

[[Image:DASH.jpg]]

Write back request: A dirty cache line that is replaced must be written back to memory. If the home of the memory block is the local cluster, then the data is simply written back to main memory. If the home cluster is remote, then a message is sent to the remote
home which updates the main memory and marks the block uncached-remote.

= References =

1 The DASH Cache Coherence Protocol
Daniel Lenoski, James Laudon, Kourosh Gharachorloo, Anoop Gupta, John Hennessy
May 1990
ACM SIGARCH Computer Architecture News , Proceedings of the 17th annual international symposium on Computer Architecture ISCA '90,
Volume 18 Issue 3a
Publisher: ACM Press

2. 1596-1992 IEEE standard for scalable coherent interface (SCI).
E-ISBN: 0-7381-1204-6
Year: 1993
Sponsored by: IEEE Computer Society

3. The SGI Origin: A ccnuma Highly Scalable Server
Laudon, J.; Lenoski, D.;
Computer Architecture, 1997. Conference Proceedings. The 24th Annual International Symposium on
June 2-4, 1997 Page(s):241 - 251

4. The Scalable Coherence Interface
David.B.Gustavson
September 1991
"The Scalable Coherence Interface and related standard projects"

5. An analysis of the Scalable Coherent Interface
Eric Rotenberg
June 1995
An Analytical model of the SCI Coherence protocol

= Further reading =

1. The Stanford Dash multiprocessor
Lenoski, D.; Laudon, J.; Gharachorloo, K.; Weber, W.-D.; Gupta, A.; Hennessy, J.; Horowitz, M.; Lam, M.S.;
Computer
IEEE Volume 25, Issue 3, March 1992 Page(s):63 - 79
Digital Object Identifier 10.1109/2.121510

= External links =

CSC/ECE 506 Fall 2007/wiki3 8 38

2007-10-18T22:24:33Z

Arangan: /* Scalable Coherent Interface (SCI) */

''Wiki: SCI. The IEEE Scalable Coherent Interface is a superset of the SSCI protocol we have been considering in class. A lot has been written about it, but it is still difficult to comprehend. Using SSCI as a starting point, explain why additional states are necessary, and give (or cite) examples that demonstrate how they work. Ideally, this would still be an overview of the working of the protocol, referencing more detailed documentation on the Web.''

__TOC__

= Introduction =

= Scalable Coherent Interface (SCI) =

The scalable coherent interface has now become a chief hardware based approach to the cache coherence problem in shared memory multiprocessors. SCI is a directory based invalidate coherence protocol. The state of a cache block is distributed to the sharers of that block. Limits that are inherent in bus technology are easily avoided by SCI. The SCI protocol is to provide scalability, coherence and an interface. Scalability is to guarantee that the same mechanisms can be used in single processor systems and large highly parallel multiprocessors. Coherence is to guarantee efficient and integral use of cache memories in distributed shared memory. An interface provides a communication architecture that has multiple values to be brought into a single system and provide smooth inter-operation.

Goals of SCI

High Performance – For distributed or parallel applications, high communication performance must be obtained when using SCI. Usual aspects of such performance would be a low latency, low CPU overhead for operations dealing with communication and high sustained throughput.

Scalability – This is addressed in many ways. Some are
•Performance scalability – as the number of nodes being added to the system increases.
•Interconnect Distance scalability
•Memory System scalability – chiefly of cache coherence protocols, which should not have limitations to the number of processors or modules in memory that it can handle.
•Technological scalability – utilization of similar mechanisms in large, small-scale, loosely-coupled or tightly-coupled systems. Also, when new advances are made in technology, it must have the ability to readily make use of such advances.
•Economic scalability – use of similar components and mechanisms in high-end, low-volume systems as well as low-end, high-volume systems.
•Addressing capability has no short term practical limits.

Coherent Memory System – For the purpose of reduction of Average Access Time to data, caches are becoming more important each day for microprocessors.

Interface Characteristics – This specifies a standard interface to an interconnect that makes several devices to be connected or attached together and to interoperate.

In SCI, every interface does not wait for the signal to propagate before it begins to send the next signal. Also, SCI utilizes multiple links so that, concurrently, several transfers can take place.

Usually, a directory entry is in either of the two states : ‘home’ or ‘gone’. If the state is in ‘home’, then memory can immediately satisfy requests to a block as the block has not been cached by any processor. If the state is ‘gone’, then the block has been cached by a processor and might even be modified. Now, the directory contains a pointer to the first processor in the sharing list for this particular block. Hence, on requesting the data, memory returns the pointer to the first processor on the shared list rather than the data itself. The processor asking for the block now forwards its request to the processor on the top of the shared list. Now, the requesting processor adds itself into the shared list as the new head of the list.

In the SCI protocol, any coherent transaction has three phases.

Memory read – When a processor misses in its cache, It asks for the block in the home directory in memory. If the state of the memory is ‘home’, then the main memory replies with the block to the requesting processor. If the state is ‘gone’, then the main memory returns the head of the shared list of processors for that particular block. Then, memory updates its pointer and puts the requesting processor as the new head.

Cache read – When memory returns a pointer toe the requesting processor instead of data, the processor forwards its request for the block to the head of the doubly linked list. When the cache receives, the head of the list returns the data which might have been modified. The head of the list changes its backward pointer to the requesting processor’s node. Now, the requesting processor becomes the head of the list and it has the cache block.

Cleanup – If the cache miss from the requesting processor is a store, the processor has to first invalidate all other cached copies and then only proceed with the store. The new head gives an invalidate request to the next address on the list i.e. to its next processor. This processor invalidates and gives back a pointer to the next processor on its list. The head of the list uses this new pointer and sends it an invalidate request. This goes on until a NULL pointer is returned. This is the cleanup process for invalidation of other cache copies of the block.

= Simple Scalable Coherent Interface (SSCI) =

In directory-based approach, Every memory block has associated directory information; it keeps track of copies of cached blocks and their states. On a miss, it finds the directory entry, looks it up, and communicates only with the nodes that have copies (if necessary).

There are mainly two approaches: Full-bit vector: For k processors, it maintains k presence bit and 1 dirty bit at the home node. Cache state is represented the same way as in bus-based designs (MSI, MESI, etc.). It has three cache states: EM (exclusive or modified), S (shared), U (unowned). Limitation is - Number of presence bits needed grows as the number of processors.

Memory-based schemes store the information about all cached copies at the home node of the block. Cache-based schemes distribute information about copies among the copies themselves. The home contains a pointer to one cached copy of the block. Each copy contains the identity of the next node that has a copy of the block. The location of the copies is therefore determined through network transactions.

Simple SCI (SSCI) retains similarity with full-bit vector protocol: MESI states in the cache; U, S, EM states in the memory directory; It replaces the presence bits with a pointer.

== Why additional states are necessary? ==

Performance & Correctness (Coherence & Consistency) requirement for the protocol necessitates the additional states in the protocol.

=== Need for Busy state ===

On a scalable multiprocessors without coherent caches, the main-memory module determined the ordering of writes. The order that writes become visible to all processors is the order in which they reached memory.

e.g. If two processors issue read-exclusive requests for a particular word, the home will provide the requestors with the location of
the dirty node. But which request will reach the dirty node first cannot be guaranteed. This creates the need for additional busy state in the directory.

This can be solved by
# holding requests at home or requestor node and serve them in the order of their arrival
# If the block is busy, reject any further request to it & that request will be retried later
# If directory is busy forward request to dirty node & dirty node will serialize the request execution

In non-coherent scalable multiprocessors,
# For write atomicity in invalidation based protocol, current owner of block has to wait until it recieve all invalidation acks and then only it can read/write new value.
# For write completion, current owner of the block need to wait for ack from memory

In Origin, there are 3 busy states indicates that the directory has received the request but is waiting for completion hence another request for the same block won't be accepted.
# Busy read
# Busy read exclusive or upgrade
# Busy uncached read: In case of DMA, if the memory block is being read, no other processor should be allowed to get write access to it.

=== Poison state ===

To improve locality, origin relies on page migration. Hardware keeps reference counts on each page. On every access of memory count is incremented and compared with home node. If count is more than programmable threshold, hardware interrupts one of the local processors to copy the page. Block transfer engine “poison” the source page. Subsequent accesses by other processors receive a bus error and that removes the TLB entry for that location, read the location from the page table & update the TLB with new location. This approach is called - “Lazy TLB shootdown” which reduces the overall cost of migrating memory and changing the virtual-to-physical address mappings.

=== Additional states ===

Additional states can be introduced into the sharing list. The SSCI protocol has only one kind of sharing list state : dirty. More options can be introduced into the list to improve performance of the overall shared multiprocessors. Although additional states do increment the complexity of the protocol being utilized and implemented, it would be very much beneficial to the system when the overall performance of the system is increased. The question “Why Additional States are necessary” is answered alongside the introduction of each new state i.e. explanation for the necessity of each and every state has been provided.

Having the following states implemented in the protocol can improve performance.

1. “fresh” and “gone” – Main memory may still be able to respond to a requesting processor if the memory can differentiate between ‘cached and unmodified’ and ‘cached and modified’. Even if the cached value is elsewhere, memory may still be able to respond to the request if the data remains unmodified.

The “fresh” state shows that memory still has an up-to-date copy of the block, even if the block has been cached and is being shared between processors. As long as the value remains unmodified by any of the processors sharing the data, the memory can keep responding by returning the requested data to the requesting processor.

The “gone” state indicates that the block has been cached and has been modified. This shows that main memory cant return the data anymore, but has to return the pointer that points to the head of the list.

2. “clean lists” – Although the introduction of these states increase the complexity of handling each access, it does help with the overall performance. In this case, it may be helpful to differentiate the types of read accesses to the cache. One type of read access would be a ‘normal read-only’. Another type of read would be a ‘read with intent to modify later’. The memory in the state is ‘fresh’ when a read-fresh takes place. A read-clean leaves the memory in the ‘gone’ state. Just as with the dirty state, the read-clean allows a write to the block to proceed immediately. When a block is replaced in the cache and the cache is in the ‘clean’ state, no data needs to be written to memory. But with the dirty state, a write-back would be required even if the data had not been modified.

== Examples ==
=== 1 DASH Cache Coherence Protocol ===

DASH (Directory Architecture for SHared memory) is a scalable shared-memory multiprocessor currently being developed at Stanford’s Computer Systems Laboratory. DASH protocol uses point-to-point messages sent between the processors and memories to keep caches consistent.

The DASH coherence protocol is an invalidation-based ownership protocol. A memory block can be in one of three states as indicated by the associated directory entry: (i) uncached-remote, that is not cached by any remote cluster; (ii) shared-remote, that is cached in an unmodified state by one or more remote clusters; or (iii) dirty-remote, that is cached in a modified state by a single remote cluster.

Please see in the figures below: Left - Flow of Read Request to remote memory with directory in dirty-remote state. Right - Flow of Read-Exclusive Request to remote memory with directory in shared-remote state.

[[Image:DASH.jpg]]

Write back request: A dirty cache line that is replaced must be written back to memory. If the home of the memory block is the local cluster, then the data is simply written back to main memory. If the home cluster is remote, then a message is sent to the remote
home which updates the main memory and marks the block uncached-remote.

= References =

1 The DASH Cache Coherence Protocol
Daniel Lenoski, James Laudon, Kourosh Gharachorloo, Anoop Gupta, John Hennessy
May 1990
ACM SIGARCH Computer Architecture News , Proceedings of the 17th annual international symposium on Computer Architecture ISCA '90,
Volume 18 Issue 3a
Publisher: ACM Press

2. 1596-1992 IEEE standard for scalable coherent interface (SCI).
E-ISBN: 0-7381-1204-6
Year: 1993
Sponsored by: IEEE Computer Society

3. The SGI Origin: A ccnuma Highly Scalable Server
Laudon, J.; Lenoski, D.;
Computer Architecture, 1997. Conference Proceedings. The 24th Annual International Symposium on
June 2-4, 1997 Page(s):241 - 251

4. The Scalable Coherence Interface
David.B.Gustavson
September 1991
"The Scalable Coherence Interface and related standard projects"

5. An analysis of the Scalable Coherent Interface
Eric Rotenberg
June 1995
An Analytical model of the SCI Coherence protocol

= Further reading =

1. The Stanford Dash multiprocessor
Lenoski, D.; Laudon, J.; Gharachorloo, K.; Weber, W.-D.; Gupta, A.; Hennessy, J.; Horowitz, M.; Lam, M.S.;
Computer
IEEE Volume 25, Issue 3, March 1992 Page(s):63 - 79
Digital Object Identifier 10.1109/2.121510

= External links =

CSC/ECE 506 Fall 2007/wiki3 8 38

2007-10-18T22:23:46Z

Arangan: /* Scalable Coherent Interface (SCI) */

''Wiki: SCI. The IEEE Scalable Coherent Interface is a superset of the SSCI protocol we have been considering in class. A lot has been written about it, but it is still difficult to comprehend. Using SSCI as a starting point, explain why additional states are necessary, and give (or cite) examples that demonstrate how they work. Ideally, this would still be an overview of the working of the protocol, referencing more detailed documentation on the Web.''

__TOC__

= Introduction =

= Scalable Coherent Interface (SCI) =

The scalable coherent interface has now become a chief hardware based approach to the cache coherence problem in shared memory multiprocessors. SCI is a directory based invalidate coherence protocol. The state of a cache block is distributed to the sharers of that block. Limits that are inherent in bus technology are easily avoided by SCI. The SCI protocol is to provide scalability, coherence and an interface. Scalability is to guarantee that the same mechanisms can be used in single processor systems and large highly parallel multiprocessors. Coherence is to guarantee efficient and integral use of cache memories in distributed shared memory. An interface provides a communication architecture that has multiple values to be brought into a single system and provide smooth inter-operation.

Goals of SCI

High Performance – For distributed or parallel applications, high communication performance must be obtained when using SCI. Usual aspects of such performance would be a low latency, low CPU overhead for operations dealing with communication and high sustained throughput.

Scalability – This is addressed in many ways. Some are
• Performance scalability – as the number of nodes being added to the system increases.
• Interconnect Distance scalability
• Memory System scalability – chiefly of cache coherence protocols, which should not have limitations to the number of processors or modules in memory that it can handle.
• Technological scalability – utilization of similar mechanisms in large, small-scale, loosely-coupled or tightly-coupled systems. Also, when new advances are made in technology, it must have the ability to readily make use of such advances.
• Economic scalability – use of similar components and mechanisms in high-end, low-volume systems as well as low-end, high-volume systems.
• Addressing capability has no short term practical limits.

Coherent Memory System – For the purpose of reduction of Average Access Time to data, caches are becoming more important each day for microprocessors.

Interface Characteristics – This specifies a standard interface to an interconnect that makes several devices to be connected or attached together and to interoperate.

In SCI, every interface does not wait for the signal to propagate before it begins to send the next signal. Also, SCI utilizes multiple links so that, concurrently, several transfers can take place.

Usually, a directory entry is in either of the two states : ‘home’ or ‘gone’. If the state is in ‘home’, then memory can immediately satisfy requests to a block as the block has not been cached by any processor. If the state is ‘gone’, then the block has been cached by a processor and might even be modified. Now, the directory contains a pointer to the first processor in the sharing list for this particular block. Hence, on requesting the data, memory returns the pointer to the first processor on the shared list rather than the data itself. The processor asking for the block now forwards its request to the processor on the top of the shared list. Now, the requesting processor adds itself into the shared list as the new head of the list.

In the SCI protocol, any coherent transaction has three phases.

Memory read – When a processor misses in its cache, It asks for the block in the home directory in memory. If the state of the memory is ‘home’, then the main memory replies with the block to the requesting processor. If the state is ‘gone’, then the main memory returns the head of the shared list of processors for that particular block. Then, memory updates its pointer and puts the requesting processor as the new head.

Cache read – When memory returns a pointer toe the requesting processor instead of data, the processor forwards its request for the block to the head of the doubly linked list. When the cache receives, the head of the list returns the data which might have been modified. The head of the list changes its backward pointer to the requesting processor’s node. Now, the requesting processor becomes the head of the list and it has the cache block.

Cleanup – If the cache miss from the requesting processor is a store, the processor has to first invalidate all other cached copies and then only proceed with the store. The new head gives an invalidate request to the next address on the list i.e. to its next processor. This processor invalidates and gives back a pointer to the next processor on its list. The head of the list uses this new pointer and sends it an invalidate request. This goes on until a NULL pointer is returned. This is the cleanup process for invalidation of other cache copies of the block.

= Simple Scalable Coherent Interface (SSCI) =

In directory-based approach, Every memory block has associated directory information; it keeps track of copies of cached blocks and their states. On a miss, it finds the directory entry, looks it up, and communicates only with the nodes that have copies (if necessary).

There are mainly two approaches: Full-bit vector: For k processors, it maintains k presence bit and 1 dirty bit at the home node. Cache state is represented the same way as in bus-based designs (MSI, MESI, etc.). It has three cache states: EM (exclusive or modified), S (shared), U (unowned). Limitation is - Number of presence bits needed grows as the number of processors.

Memory-based schemes store the information about all cached copies at the home node of the block. Cache-based schemes distribute information about copies among the copies themselves. The home contains a pointer to one cached copy of the block. Each copy contains the identity of the next node that has a copy of the block. The location of the copies is therefore determined through network transactions.

Simple SCI (SSCI) retains similarity with full-bit vector protocol: MESI states in the cache; U, S, EM states in the memory directory; It replaces the presence bits with a pointer.

== Why additional states are necessary? ==

Performance & Correctness (Coherence & Consistency) requirement for the protocol necessitates the additional states in the protocol.

=== Need for Busy state ===

On a scalable multiprocessors without coherent caches, the main-memory module determined the ordering of writes. The order that writes become visible to all processors is the order in which they reached memory.

e.g. If two processors issue read-exclusive requests for a particular word, the home will provide the requestors with the location of
the dirty node. But which request will reach the dirty node first cannot be guaranteed. This creates the need for additional busy state in the directory.

This can be solved by
# holding requests at home or requestor node and serve them in the order of their arrival
# If the block is busy, reject any further request to it & that request will be retried later
# If directory is busy forward request to dirty node & dirty node will serialize the request execution

In non-coherent scalable multiprocessors,
# For write atomicity in invalidation based protocol, current owner of block has to wait until it recieve all invalidation acks and then only it can read/write new value.
# For write completion, current owner of the block need to wait for ack from memory

In Origin, there are 3 busy states indicates that the directory has received the request but is waiting for completion hence another request for the same block won't be accepted.
# Busy read
# Busy read exclusive or upgrade
# Busy uncached read: In case of DMA, if the memory block is being read, no other processor should be allowed to get write access to it.

=== Poison state ===

To improve locality, origin relies on page migration. Hardware keeps reference counts on each page. On every access of memory count is incremented and compared with home node. If count is more than programmable threshold, hardware interrupts one of the local processors to copy the page. Block transfer engine “poison” the source page. Subsequent accesses by other processors receive a bus error and that removes the TLB entry for that location, read the location from the page table & update the TLB with new location. This approach is called - “Lazy TLB shootdown” which reduces the overall cost of migrating memory and changing the virtual-to-physical address mappings.

=== Additional states ===

Additional states can be introduced into the sharing list. The SSCI protocol has only one kind of sharing list state : dirty. More options can be introduced into the list to improve performance of the overall shared multiprocessors. Although additional states do increment the complexity of the protocol being utilized and implemented, it would be very much beneficial to the system when the overall performance of the system is increased. The question “Why Additional States are necessary” is answered alongside the introduction of each new state i.e. explanation for the necessity of each and every state has been provided.

Having the following states implemented in the protocol can improve performance.

1. “fresh” and “gone” – Main memory may still be able to respond to a requesting processor if the memory can differentiate between ‘cached and unmodified’ and ‘cached and modified’. Even if the cached value is elsewhere, memory may still be able to respond to the request if the data remains unmodified.

The “fresh” state shows that memory still has an up-to-date copy of the block, even if the block has been cached and is being shared between processors. As long as the value remains unmodified by any of the processors sharing the data, the memory can keep responding by returning the requested data to the requesting processor.

The “gone” state indicates that the block has been cached and has been modified. This shows that main memory cant return the data anymore, but has to return the pointer that points to the head of the list.

2. “clean lists” – Although the introduction of these states increase the complexity of handling each access, it does help with the overall performance. In this case, it may be helpful to differentiate the types of read accesses to the cache. One type of read access would be a ‘normal read-only’. Another type of read would be a ‘read with intent to modify later’. The memory in the state is ‘fresh’ when a read-fresh takes place. A read-clean leaves the memory in the ‘gone’ state. Just as with the dirty state, the read-clean allows a write to the block to proceed immediately. When a block is replaced in the cache and the cache is in the ‘clean’ state, no data needs to be written to memory. But with the dirty state, a write-back would be required even if the data had not been modified.

== Examples ==
=== 1 DASH Cache Coherence Protocol ===

DASH (Directory Architecture for SHared memory) is a scalable shared-memory multiprocessor currently being developed at Stanford’s Computer Systems Laboratory. DASH protocol uses point-to-point messages sent between the processors and memories to keep caches consistent.

The DASH coherence protocol is an invalidation-based ownership protocol. A memory block can be in one of three states as indicated by the associated directory entry: (i) uncached-remote, that is not cached by any remote cluster; (ii) shared-remote, that is cached in an unmodified state by one or more remote clusters; or (iii) dirty-remote, that is cached in a modified state by a single remote cluster.

Please see in the figures below: Left - Flow of Read Request to remote memory with directory in dirty-remote state. Right - Flow of Read-Exclusive Request to remote memory with directory in shared-remote state.

[[Image:DASH.jpg]]

Write back request: A dirty cache line that is replaced must be written back to memory. If the home of the memory block is the local cluster, then the data is simply written back to main memory. If the home cluster is remote, then a message is sent to the remote
home which updates the main memory and marks the block uncached-remote.

= References =

1 The DASH Cache Coherence Protocol
Daniel Lenoski, James Laudon, Kourosh Gharachorloo, Anoop Gupta, John Hennessy
May 1990
ACM SIGARCH Computer Architecture News , Proceedings of the 17th annual international symposium on Computer Architecture ISCA '90,
Volume 18 Issue 3a
Publisher: ACM Press

2. 1596-1992 IEEE standard for scalable coherent interface (SCI).
E-ISBN: 0-7381-1204-6
Year: 1993
Sponsored by: IEEE Computer Society

3. The SGI Origin: A ccnuma Highly Scalable Server
Laudon, J.; Lenoski, D.;
Computer Architecture, 1997. Conference Proceedings. The 24th Annual International Symposium on
June 2-4, 1997 Page(s):241 - 251

4. The Scalable Coherence Interface
David.B.Gustavson
September 1991
"The Scalable Coherence Interface and related standard projects"

5. An analysis of the Scalable Coherent Interface
Eric Rotenberg
June 1995
An Analytical model of the SCI Coherence protocol

= Further reading =

1. The Stanford Dash multiprocessor
Lenoski, D.; Laudon, J.; Gharachorloo, K.; Weber, W.-D.; Gupta, A.; Hennessy, J.; Horowitz, M.; Lam, M.S.;
Computer
IEEE Volume 25, Issue 3, March 1992 Page(s):63 - 79
Digital Object Identifier 10.1109/2.121510

= External links =

CSC/ECE 506 Fall 2007/wiki3 8 38

2007-10-18T01:24:22Z

Arangan: /* References */

CSC/ECE 506 Fall 2007/wiki3 8 38

2007-10-17T22:41:10Z

Arangan: /* Why additional states are necessary? */

CSC/ECE 506 Fall 2007/wiki3 8 38

2007-10-17T22:34:15Z

Arangan: /* Why additional states are necessary? */

CSC/ECE 506 Fall 2007/wiki3 8 38

2007-10-17T22:33:21Z

Arangan: /* Why additional states are necessary? */

CSC/ECE 506 Fall 2007/wiki3 8 38

2007-10-17T22:32:06Z

Arangan: /* Why additional states are necessary? */

CSC/ECE 506 Fall 2007/wiki3 8 38

2007-10-17T18:21:02Z

Arangan: /* SCI */

CSC/ECE 506 Fall 2007/wiki1 10 aj

2007-09-10T21:55:39Z

Arangan: /* Systolic Architecture */

'''( Section 1.2.6)'''

== New developments in dataflow and systolic architectures ==

In the 1980s, a steady progress of parallel computing gave way for various unique parallel architectures that underwent extensive research but were not truly implemented to its full extent as when compared to architectures such as Shared Address Programming, Message Passing and Data Parallel Processing. Such architectures are still in its infancy and though its concepts have been utilized in various approaches for increasing processing efficiency; the actual architectural implementation is not widely employed for the sake of parallel computing.

Two such architectures are Dataflow Architecture and Systolic Architecture.

== Dataflow Architecture ==

'''Developments in Dataflow Architecture'''

(Reference: Based on ideas presented in the Wikipedia article at: http://en.wikipedia.org/wiki/Dataflow_architecture )

Dataflow architectures (from a hardware standpoint) was an important research topic in the 1970s and early 1980s. Interest in the field has subsided in recent years due to the inability to resolve certain inherent problems in the Dataflow architectural model. To understand, this, the Dataflow concept is summarized below.

'''Summary of the Dataflow Idea'''

In its essence, a Dataflow architecture executes instructions based on whether the input arguments to the instructions are available. There is no program counter as present in a Von Neumann computer. To indicate the dependency of instructions, tags were used. When the tags contained simple memory addresses, the design was a static Dataflow machine. However, static designs couldn’t allow for multiple instances of a routine.

Dynamic Dataflow machines used Content-Addressable memory (CAM)(i.e. the tags were stored in memory) to solve this problem.
The programs were loaded in the CAM. When the tagged operands of an instruction became available, the CAM would send it to an '''execution unit'''. After execution, the output data and it’s tags were sent back to the CAM (as a '''data token'''). The CAM would then execute the next instruction whose dependencies had been satisfied.

Since, the CAM could identify instructions whose tags were not dependant on any unexecuted instruction, parallelization was possible.
There were however, major problems related to -

1) broadcasting the data tokens in a massively parallel system efficiently.

2) dispatching instruction tokens in a massively parallel system efficiently.

3) A real program had a huge number of dependencies. Building a CAM large enough for this proved difficult.

Due to these issues, new developments in this area were largely stagnant.

'''New Developments in the Dataflow Architectural Model'''

(Referenced from: http://en.wikipedia.org/wiki/Out-of-order_execution)
1) A subset of the Dataflow model - '''Out of order execution''' is widely used in present day architecture. It uses the conventional Von Neumann Architecture to run '''Execution Windows''' in the usual order. Within the Window however, instructions are run according to the Dataflow Paradigm thus enabling parallelization and efficient utilization of CPU cycles.

(Referenced form: http://wavescalar.cs.washington.edu/)
2)The WaveScalar Instruction Set Architecture and Execution Model, currently being developed at the University of Washington attempts at building an architectural model that can work with current Imperative languages.

(Referenced from: http://portal.acm.org/citation.cfm?id=1032450 & http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?tp=&arnumber=1332593&isnumber=29428)

3)Calstrom and Boden propose a '''packet instruction set computer (PISC) architecture''' which would employ dataflow architecture for network processors. They detailed a 40 gb/s network processor using Dataflow architecture.

(Referenced from: http://ieeexplore.ieee.org/Xplore/login.jsp?url=/iel2/3036/8629/00379757.pdf?isnumber=&arnumber=379757)
4) Data flow architecture is also being evaluated for DSP processors as outlined in the referenced paper by Lee et al. Their proposed design was for a static Dataflow processor with 9 parallel processors for the multi-standard video codec processor.

== Systolic Architecture ==

'''Emergence of Systolic Arrays'''

During the 1980s, tremendous technological advances were being made in the field of VLSI circuits, which led to the birth of several different concepts of highly parallel computing systems. One particular approach to such parallelism was using Systolic arrays, which constituted of a high number of processing elements known as cells, which were interconnected for local communication. This model utilizes a pipe-network arrangement of data processing units. Whenever a data object arrives, an operation is triggered in a data processing unit, unlike a central processing unit which has program counters. Several data streams are sent and received by a systolic array and several data counters are required to propagate data streams. This simple methodology facilitates parallelism.

'''Definition, Function and Applications'''

They are called systolic arrays; because data, similar to human blood circulation, circulates within the array and also may interact with other data. Some results that have been computed in certain cells are pumped into new cells as input, similar to the heart (Hence the analogy to a systolic mechanism of parallelism). H. T. Kung and Charles E. Leiserson invented the concept of systolic arrays being utilized as agents for parallel processing.

The Systolic Architecture is a mesh-like, interconnected set of processing elements, or cells, which interact with each other and pass on results that become inputs for the next computational time-step. Such proximity of high performance processors provide a highly required space efficiency, which is usually the main factor in parallel architecture. The structure of such systolic arrays may be non linear and multi-directional pathways may exist between processing elements. Even though this architecture is quite similar to Single-Instruction-Multiple-Data, unlike SIMD, each processing element is capable of processing a unique, different operation. The main feature of systolic architecture is that its highly specialized computations can be performed on simple parallel processors and localized communication patterns.

The systolic array is an architectural concept that is highly suited for very high integration technology. Communication between these mesh-connected processors occurs only amongst neighbors. Both control and data flow are local. Each processor has a communication register that can be read by any of its neighbors.

A Systolic array is a network of processing elements that has features like

• Modularity : each systolic array element is a module

• Synchrony : a global clock times the data through the network

• Locality (both spatial and temporal) : cells have local communication and a unit time delay

• Regularity : processing elements are homogeneously interconnected

• Parallel Computing and Pipelinability : architecture achieves high speed

The two main Systolic Architectural Features are

• Balancing Computation with I/O.

• Concurrency and Communication.

But other notable features of the architecture are its cost effectiveness, its regular & simple design, the modularity in arrays is quite high, the number of processors working simultaneously is high and also the local communication is fast – making overall execution time much faster.

''Types of Systolic Arrays''

1. Single Dimensional Input/Output Linear Array
- This is usually utilized for single input/output
[[Image:Systolic01.jpg]]

2. Two Dimensional Input/Output Linear Array
- This allows for more input/output and control over a linear array
[[Image:Systolic02.jpg]]

3. Perimeter Input/Output Planar Array
- This topology allows input/output through its boundary or perimeter cells only
[[Image:Systolic03.jpg]]

4. Three Dimensional Input/Output Focal Plane Array
- This facilitates input/output to each and every systolic cell in the array
[[Image:Systolic04.jpg]]

A few applications of Systolic Arrays

• Matrix Multiplication

• Solution of linear equations to problems in image processing

• Computer graphics

• Cryptography

• Computer tomography

• Matrix Inversion and Decomposition

• Convolution

• Polynomial Evaluation

• Lattice filters for seismic and speech signal processing

• Artificial Neural Networks

For a complete explanation on the invention of systolic arrays, utilize the following link : http://www.patentstorm.us/patents/4727503-fulltext.html

'''New Developments'''

''iWarp''

In 1989, the Intel Corporation launched the iWarp as a product and produced more than 1500 iWarp systems. The underlying architecture was based on systolic arrays and data streams being utilized for sending and receiving data between elements on the system. Even though the system was well suited for several applications, Intel stopped actively marketing the product in the mid-90s. It is still a useful tool for research in the parallel computing direction and it was one of the first successful steps towards true parallelism in the early ages of parallel computing.

''Image Processing and Morphology''

A two-dimensional systolic array consisting of cells can be implemented, based on parallel modules which provide internal pipeline operation where every cell can be configured according to a control word. Also, the array can be provided with a group of image buffers, that can reduce the number of accesses to data memory may also extend the array capabilities, allowing the possibility of chaining interconnection of multiple processing blocks.

''Digital Signal Processors''

Low cost, systolic architectural microprocessors utilizes a DSP systolic array, which makes it possible to build low cost ICs capable of providing high speed digital video services. This requires high processing capabilities, and parallelism is a requirement in such areas. Systolic architectures are better suited for such purposes as they are of much lower cost due to their space-efficiency and interconnectivity.

''Tilera Tile64''

Arranged in a two dimensional grid, the Tilera 64 is a tile that has 64 processing elements or cells which are interconnected through a five way switch and it provides a two level hierarchical cache. The architecture is completely based on systolic arrays and this sixty-four element multi-core processor is being released in September 2007. It is already predicted to be utilized extensively in the field of embedded systems for advanced networking, unified thread management, digital video, web services, etc.

'''Why Systolic Architecture has not truly evolved with time'''
(to the extent of other architectures, is because)

1. Global Synchronization takes more time due to signal delays within the network

2. Bandwidth requirements are tremendous for both processor and memory

3. Fault tolerance is very poor as interconnection protocols are lacking

Also, a certain ‘parallel overhead’ point occurs in Systolic Arrays where beyond that point, adding more processors does not increase the efficiency of processing as it would normally be expected. There may even be a point beyond which adding more processors may lead to slower execution time for the architecture.

'''References'''

1. (Reference: Based on ideas presented in the Wikipedia article at:http://en.wikipedia.org/wiki/Systolic_array)

2. Systolic Algorithms: Concepts, Synthesis and Evolution - Siang W.Song, University of Sau Paulo, Department Of Computer Science
http://citeseer.ist.psu.edu/cache/papers/cs/30734/http:zSzzSzwww.ime.usp.brzSz~songzSzpaperszSzcimpa.pdf/systolic-algorithms-concepts-synthesis.pdf

3. Instruction Systolic Array (ISA) - H.W. Lang, FH Flensburg
http://www.iti.fh-flensburg.de/lang/papers/isa/isa1.htm

4. Systolic Arrays : Patent Description
http://www.patentstorm.us/patents/4727503-fulltext.html

5. iWarp
http://www.cs.cmu.edu/~iwarp/

6. Tilera Tile64
http://www.tilera.com/products/processors.php

7. FPGA-Based Customizable Systolic Architecture for Image Processing Applications : Proceedings of the 2005 International Conference on Reconfigurable Computing and FPGAs (ReConFig'05) on Reconfigurable Computing and FPGAs - Griselda Saldana, Miguel Arias-Estrada (IEEE Computer Society)

8. Summary of Systolic Architectures (Slides) - Syeda Mohsina Afroze
http://web.cecs.pdx.edu/~mperkows/temp/May13/070-Systolic-Processors.pdf

CSC/ECE 506 Fall 2007/wiki1 10 aj

2007-09-06T02:58:02Z

Arangan: /* Systolic Architecture */

'''( Section 1.2.6)'''

== New developments in dataflow and systolic architectures ==

In the 1980s, a steady progress of parallel computing gave way for various unique parallel architectures that underwent extensive research but were not truly implemented to its full extent as when compared to architectures such as Shared Address Programming, Message Passing and Data Parallel Processing. Such architectures are still in its infancy and though its concepts have been utilized in various approaches for increasing processing efficiency; the actual architectural implementation is not widely employed for the sake of parallel computing.

Two such architectures are Dataflow Architecture and Systolic Architecture.

== Dataflow Architecture ==

'''Developments in Dataflow Architecture'''

(Reference: Based on ideas presented in the Wikipedia article at: http://en.wikipedia.org/wiki/Dataflow_architecture )

Dataflow architectures (from a hardware standpoint) was an important research topic in the 1970s and early 1980s. Interest in the field has subsided in recent years due to the inability to resolve certain inherent problems in the Dataflow architectural model. To understand, this, the Dataflow concept is summarized below.

'''Summary of the Dataflow Idea'''

In its essence, a Dataflow architecture executes instructions based on whether the input arguments to the instructions are available. There is no program counter as present in a Von Neumann computer. To indicate the dependency of instructions, tags were used. When the tags contained simple memory addresses, the design was a static Dataflow machine. However, static designs couldn’t allow for multiple instances of a routine.

Dynamic Dataflow machines used Content-Addressable memory (CAM)(i.e. the tags were stored in memory) to solve this problem.
The programs were loaded in the CAM. When the tagged operands of an instruction became available, the CAM would send it to an '''execution unit'''. After execution, the output data and it’s tags were sent back to the CAM (as a '''data token'''). The CAM would then execute the next instruction whose dependencies had been satisfied.

Since, the CAM could identify instructions whose tags were not dependant on any unexecuted instruction, parallelization was possible.
There were however, major problems related to -

1) broadcasting the data tokens in a massively parallel system efficiently.

2) dispatching instruction tokens in a massively parallel system efficiently.

3) A real program had a huge number of dependencies. Building a CAM large enough for this proved difficult.

Due to these issues, new developments in this area were largely stagnant.

'''New Developments in the Dataflow Architectural Model'''

(Referenced from: http://en.wikipedia.org/wiki/Out-of-order_execution)
1) A subset of the Dataflow model - '''Out of order execution''' is widely used in present day architecture. It uses the conventional Von Neumann Architecture to run '''Execution Windows''' in the usual order. Within the Window however, instructions are run according to the Dataflow Paradigm thus enabling parallelization and efficient utilization of CPU cycles.

(Referenced form: http://wavescalar.cs.washington.edu/)
2)The WaveScalar Instruction Set Architecture and Execution Model, currently being developed at the University of Washington attempts at building an architectural model that can work with current Imperative languages.

(Referenced from: http://portal.acm.org/citation.cfm?id=1032450 & http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?tp=&arnumber=1332593&isnumber=29428)

3)Calstrom and Boden propose a '''packet instruction set computer (PISC) architecture''' which would employ dataflow architecture for network processors. They detailed a 40 gb/s network processor using Dataflow architecture.

(Referenced from: http://ieeexplore.ieee.org/Xplore/login.jsp?url=/iel2/3036/8629/00379757.pdf?isnumber=&arnumber=379757)
4) Data flow architecture is also being evaluated for DSP processors as outlined in the referenced paper by Lee et al. Their proposed design was for a static Dataflow processor with 9 parallel processors for the multi-standard video codec processor.

== Systolic Architecture ==

'''Emergence of Systolic Arrays'''

During the 1980s, tremendous technological advances were being made in the field of VLSI circuits, which led to the birth of a several different concepts of highly parallel computing systems. One particular approach to such parallelism was using Systolic arrays, which constituted of a high number of processing elements known as cells, which were interconnected for local communication. This model utilizes a pipe-network arrangement of data processing units. Whenever a data object arrives, an operation is triggered in a data processing unit, unlike a central processing unit which has program counters. Several data streams are sent and received by a systolic array and several data counters are required to propagate data streams. This simple methodology facilitates parallelism.

'''Definition, Function and Applications'''

They are called systolic arrays; because data, similar to human blood circulation, circulates within the array and also may interact with other data. Some results that have been computed in certain cells are pumped into new cells as input, similar to the heart (Hence the analogy to a systolic mechanism of parallelism). H. T. Kung and Charles E. Leiserson invented the concept of systolic arrays being utilized as agents for parallel processing.

The Systolic Architecture is a mesh-like, interconnected set of processing elements, or cells, which interact with each other and pass on results that become inputs for the next computational time-step. Such proximity of high performance processors provide a highly required space efficiency, which is usually the main factor in parallel architecture. The structure of such systolic arrays may be non linear and multi-directional pathways may exist between processing elements. Even though this architecture is quite similar to Single-Instruction-Multiple-Data, unlike SIMD, each processing element is capable of processing a unique, different operation. The main feature of systolic architecture is that its highly specialized computations can be performed on simple parallel processors and localized communication patterns.

The systolic array is an architectural concept that is highly suited for very high integration technology. Communication between these mesh-connected processors occurs only amongst neighbors. Both control and data flow are local. Each processor has a communication register that can be read by any of its neighbors.

A Systolic array is a network of processing elements that has features like

• Modularity : each systolic array element is a module

• Synchrony : a global clock times the data through the network

• Locality (both spatial and temporal) : cells have local communication and a unit time delay

• Regularity : processing elements are homogeneously interconnected

• Parallel Computing and Pipelinability : architecture achieves high speed

The two main Systolic Architectural Features are

• Balancing Computation with I/O.

• Concurrency and Communication.

But other notable features of the architecture are its cost effectiveness, its regular & simple design, the modularity in arrays is quite high, the number of processors working simultaneously is high and also the local communication is fast – making overall execution time much faster.

''Types of Systolic Arrays''

1. Single Dimensional Input/Output Linear Array
- This is usually utilized for single input/output
[[Image:Systolic01.jpg]]

2. Two Dimensional Input/Output Linear Array
- This allows for more input/output and control over a linear array
[[Image:Systolic02.jpg]]

3. Perimeter Input/Output Planar Array
- This topology allows input/output through its boundary or perimeter cells only
[[Image:Systolic03.jpg]]

4. Three Dimensional Input/Output Focal Plane Array
- This facilitates input/output to each and every systolic cell in the array
[[Image:Systolic04.jpg]]

A few applications of Systolic Arrays

• Matrix Multiplication

• Solution of linear equations to problems in image processing

• Computer graphics

• Cryptography

• Computer tomography

• Matrix Inversion and Decomposition

• Convolution

• Polynomial Evaluation

• Lattice filters for seismic and speech signal processing

• Artificial Neural Networks

For a complete explanation on the invention of systolic arrays, utilize the following link : http://www.patentstorm.us/patents/4727503-fulltext.html

'''New Developments'''

''iWarp''

In 1989, the Intel Corporation launched the iWarp as a product and produced more than 1500 iWarp systems. The underlying architecture was based on systolic arrays and data streams being utilized for sending and receiving data between elements on the system. Even though the system was well suited for several applications, Intel stopped actively marketing the product in the mid-90s. It is still a useful tool for research in the parallel computing direction and it was one of the first successful steps towards true parallelism in the early ages of parallel computing.

''Image Processing and Morphology''

A two-dimensional systolic array consisting of cells can be implemented, based on parallel modules which provide internal pipeline operation where every cell can be configured according to a control word. Also, the array can be provided with a group of image buffers, that can reduce the number of accesses to data memory may also extend the array capabilities, allowing the possibility of chaining interconnection of multiple processing blocks.

''Digital Signal Processors''

Low cost, systolic architectural microprocessors utilizes a DSP systolic array, which makes it possible to build low cost ICs capable of providing high speed digital video services. This requires high processing capabilities, and parallelism is a requirement in such areas. Systolic architectures are better suited for such purposes as they are of much lower cost due to their space-efficiency and interconnectivity.

''Tilera Tile64''

Arranged in a two dimensional grid, the Tilera 64 is a tile that has 64 processing elements or cells which are interconnected through a five way switch and it provides a two level hierarchical cache. The architecture is completely based on systolic arrays and this sixty-four element multi-core processor is being released in September 2007. It is already predicted to be utilized extensively in the field of embedded systems for advanced networking, unified thread management, digital video, web services, etc.

'''Why Systolic Architecture has not truly evolved with time'''
(to the extent of other architectures, is because)

1. Global Synchronization takes more time due to signal delays within the network

2. Bandwidth requirements are tremendous for both processor and memory

3. Fault tolerance is very poor as interconnection protocols are lacking

Also, a certain ‘parallel overhead’ point occurs in Systolic Arrays where beyond that point, adding more processors does not increase the efficiency of processing as it would normally be expected. There may even be a point beyond which adding more processors may lead to slower execution time for the architecture.

'''References'''

1. (Reference: Based on ideas presented in the Wikipedia article at:http://en.wikipedia.org/wiki/Systolic_array)

2. Systolic Algorithms: Concepts, Synthesis and Evolution - Siang W.Song, University of Sau Paulo, Department Of Computer Science
http://citeseer.ist.psu.edu/cache/papers/cs/30734/http:zSzzSzwww.ime.usp.brzSz~songzSzpaperszSzcimpa.pdf/systolic-algorithms-concepts-synthesis.pdf

3. Instruction Systolic Array (ISA) - H.W. Lang, FH Flensburg
http://www.iti.fh-flensburg.de/lang/papers/isa/isa1.htm

4. Systolic Arrays : Patent Description
http://www.patentstorm.us/patents/4727503-fulltext.html

5. iWarp
http://www.cs.cmu.edu/~iwarp/

6. Tilera Tile64
http://www.tilera.com/products/processors.php

7. FPGA-Based Customizable Systolic Architecture for Image Processing Applications : Proceedings of the 2005 International Conference on Reconfigurable Computing and FPGAs (ReConFig'05) on Reconfigurable Computing and FPGAs - Griselda Saldana, Miguel Arias-Estrada (IEEE Computer Society)

8. Summary of Systolic Architectures (Slides) - Syeda Mohsina Afroze
http://web.cecs.pdx.edu/~mperkows/temp/May13/070-Systolic-Processors.pdf

CSC/ECE 506 Fall 2007/wiki1 10 aj

2007-09-06T02:52:05Z

Arangan: /* Systolic Architecture */

'''( Section 1.2.6)'''

== New developments in dataflow and systolic architectures ==

In the 1980s, a steady progress of parallel computing gave way for various unique parallel architectures that underwent extensive research but were not truly implemented to its full extent as when compared to architectures such as Shared Address Programming, Message Passing and Data Parallel Processing. Such architectures are still in its infancy and though its concepts have been utilized in various approaches for increasing processing efficiency; the actual architectural implementation is not widely employed for the sake of parallel computing.

Two such architectures are Dataflow Architecture and Systolic Architecture.

== Dataflow Architecture ==

'''Developments in Dataflow Architecture'''

(Reference: Based on ideas presented in the Wikipedia article at: http://en.wikipedia.org/wiki/Dataflow_architecture )

Dataflow architectures (from a hardware standpoint) was an important research topic in the 1970s and early 1980s. Interest in the field has subsided in recent years due to the inability to resolve certain inherent problems in the Dataflow architectural model. To understand, this, the Dataflow concept is summarized below.

'''Summary of the Dataflow Idea'''

In its essence, a Dataflow architecture executes instructions based on whether the input arguments to the instructions are available. There is no program counter as present in a Von Neumann computer. To indicate the dependency of instructions, tags were used. When the tags contained simple memory addresses, the design was a static Dataflow machine. However, static designs couldn’t allow for multiple instances of a routine.

Dynamic Dataflow machines used Content-Addressable memory (CAM)(i.e. the tags were stored in memory) to solve this problem.
The programs were loaded in the CAM. When the tagged operands of an instruction became available, the CAM would send it to an '''execution unit'''. After execution, the output data and it’s tags were sent back to the CAM (as a '''data token'''). The CAM would then execute the next instruction whose dependencies had been satisfied.

Since, the CAM could identify instructions whose tags were not dependant on any unexecuted instruction, parallelization was possible.
There were however, major problems related to -

1) broadcasting the data tokens in a massively parallel system efficiently.

2) dispatching instruction tokens in a massively parallel system efficiently.

3) A real program had a huge number of dependencies. Building a CAM large enough for this proved difficult.

Due to these issues, new developments in this area were largely stagnant.

'''New Developments in the Dataflow Architectural Model'''

(Referenced from: http://en.wikipedia.org/wiki/Out-of-order_execution)
1) A subset of the Dataflow model - '''Out of order execution''' is widely used in present day architecture. It uses the conventional Von Neumann Architecture to run '''Execution Windows''' in the usual order. Within the Window however, instructions are run according to the Dataflow Paradigm thus enabling parallelization and efficient utilization of CPU cycles.

(Referenced form: http://wavescalar.cs.washington.edu/)
2)The WaveScalar Instruction Set Architecture and Execution Model, currently being developed at the University of Washington attempts at building an architectural model that can work with current Imperative languages.

(Referenced from: http://portal.acm.org/citation.cfm?id=1032450 & http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?tp=&arnumber=1332593&isnumber=29428)

3)Calstrom and Boden propose a '''packet instruction set computer (PISC) architecture''' which would employ dataflow architecture for network processors. They detailed a 40 gb/s network processor using Dataflow architecture.

(Referenced from: http://ieeexplore.ieee.org/Xplore/login.jsp?url=/iel2/3036/8629/00379757.pdf?isnumber=&arnumber=379757)
4) Data flow architecture is also being evaluated for DSP processors as outlined in the referenced paper by Lee et al. Their proposed design was for a static Dataflow processor with 9 parallel processors for the multi-standard video codec processor.

== Systolic Architecture ==

'''Emergence of Systolic Arrays'''

During the 1980s, tremendous technological advances were being made in the field of VLSI circuits, which led to the birth of a several different concepts of highly parallel computing systems. One particular approach to such parallelism was using Systolic arrays, which constituted of a high number of processing elements known as cells, which were interconnected for local communication. This model utilizes a pipe-network arrangement of data processing units. Whenever a data object arrives, an operation is triggered in a data processing unit, unlike a central processing unit which has program counters. Several data streams are sent and received by a systolic array and several data counters are required to propagate data streams. This simple methodology facilitates parallelism.

'''Definition, Function and Applications'''

They are called systolic arrays; because data, similar to human blood circulation, circulates within the array and also may interact with other data. Some results that have been computed in certain cells are pumped into new cells as input, similar to the heart (Hence the analogy to a systolic mechanism of parallelism). H. T. Kung and Charles E. Leiserson invented the concept of systolic arrays being utilized as agents for parallel processing.

The Systolic Architecture is a mesh-like, interconnected set of processing elements, or cells, which interact with each other and pass on results that become inputs for the next computational time-step. Such proximity of high performance processors provide a highly required space efficiency, which is usually the main factor in parallel architecture. The structure of such systolic arrays may be non linear and multi-directional pathways may exist between processing elements. Even though this architecture is quite similar to Single-Instruction-Multiple-Data, unlike SIMD, each processing element is capable of processing a unique, different operation. The main feature of systolic architecture is that its highly specialized computations can be performed on simple parallel processors and localized communication patterns.

The systolic array is an architectural concept that is highly suited for very high integration technology. Communication between these mesh-connected processors occurs only amongst neighbors. Both control and data flow are local. Each processor has a communication register that can be read by any of its neighbors.

A Systolic array is a network of processing elements that has features like

• Modularity : each systolic array element is a module

• Synchrony : a global clock times the data through the network

• Locality (both spatial and temporal) : cells have local communication and a unit time delay

• Regularity : processing elements are homogeneously interconnected

• Parallel Computing and Pipelinability : architecture achieves high speed

The two main Systolic Architectural Features are

• Balancing Computation with I/O.

• Concurrency and Communication.

But other notable features of the architecture are its cost effectiveness, its regular & simple design, the modularity in arrays is quite high, the number of processors working simultaneously is high and also the local communication is fast – making overall execution time much faster.

''Types of Systolic Arrays''

1. Single Dimensional Input/Output Linear Array
- This is usually utilized for single input/output
[[Image:Systolic01.jpg]]

2. Two Dimensional Input/Output Linear Array
- This allows for more input/output and control over a linear array
[[Image:Systolic02.jpg]]

3. Perimeter Input/Output Planar Array
- This topology allows input/output through its boundary or perimeter cells only
[[Image:Systolic03.jpg]]

4. Three Dimensional Input/Output Focal Plane Array
- This facilitates input/output to each and every systolic cell in the array
[[Image:Systolic04.jpg]]

A few applications of Systolic Arrays

• Matrix Multiplication

• Solution of linear equations to problems in image processing

• Computer graphics

• Cryptography

• Computer tomography

• Matrix Inversion and Decomposition

• Convolution

• Polynomial Evaluation

• Lattice filters for seismic and speech signal processing

• Artificial Neural Networks

For a complete explanation on the invention of systolic arrays, utilize the following link : http://www.patentstorm.us/patents/4727503-fulltext.html

'''New Developments'''

''iWarp''

In 1989, the Intel Corporation launched the iWarp as a product and produced more than 1500 iWarp systems. The underlying architecture was based on systolic arrays and data streams being utilized for sending and receiving data between elements on the system. Even though the system was well suited for several applications, Intel stopped actively marketing the product in the mid-90s. It is still a useful tool for research in the parallel computing direction and it was one of the first successful steps towards true parallelism in the early ages of parallel computing.

''Image Processing and Morphology''

A two-dimensional systolic array consisting of cells can be implemented, based on parallel modules which provide internal pipeline operation where every cell can be configured according to a control word. Also, the array can be provided with a group of image buffers, that can reduce the number of accesses to data memory may also extend the array capabilities, allowing the possibility of chaining interconnection of multiple processing blocks.

''Digital Signal Processors''

Low cost, systolic architectural microprocessors utilizes a DSP systolic array, which makes it possible to build low cost ICs capable of providing high speed digital video services. This requires high processing capabilities, and parallelism is a requirement in such areas. Systolic architectures are better suited for such purposes as they are of much lower cost due to their space-efficiency and interconnectivity.

''Tilera Tile64''

Arranged in a two dimensional grid, the Tilera 64 is a tile that has 64 processing elements or cells which are interconnected through a five way switch and it provides a two level hierarchical cache. The architecture is completely based on systolic arrays and this sixty-four element multi-core processor is being released in September 2007. It is already predicted to be utilized extensively in the field of embedded systems for advanced networking, unified thread management, digital video, web services, etc.

'''Why Systolic Architecture has not truly evolved with time'''
(to the extent of other architectures, is because)

1. Global Synchronization takes more time due to signal delays within the network

2. Bandwidth requirements are tremendous for both processor and memory

3. Fault tolerance is very poor due as interconnection protocols are lacking

Also, a certain ‘parallel overhead’ point occurs in Systolic Arrays where beyond that point, adding more processors does not increase the efficiency of processing as it would normally be expected. There may even be a point beyond which adding more processors may even lead to slower execution time for the architecture.

'''References'''

1. (Reference: Based on ideas presented in the Wikipedia article at:http://en.wikipedia.org/wiki/Systolic_array)

2. Systolic Algorithms: Concepts, Synthesis and Evolution - Siang W.Song, University of Sau Paulo, Department Of Computer Science
http://citeseer.ist.psu.edu/cache/papers/cs/30734/http:zSzzSzwww.ime.usp.brzSz~songzSzpaperszSzcimpa.pdf/systolic-algorithms-concepts-synthesis.pdf

3. Instruction Systolic Array (ISA) - H.W. Lang, FH Flensburg
http://www.iti.fh-flensburg.de/lang/papers/isa/isa1.htm

4. Systolic Arrays : Patent Description
http://www.patentstorm.us/patents/4727503-fulltext.html

5. iWarp
http://www.cs.cmu.edu/~iwarp/

6. Tilera Tile64
http://www.tilera.com/products/processors.php

7. FPGA-Based Customizable Systolic Architecture for Image Processing Applications : Proceedings of the 2005 International Conference on Reconfigurable Computing and FPGAs (ReConFig'05) on Reconfigurable Computing and FPGAs - Griselda Saldana, Miguel Arias-Estrada (IEEE Computer Society)

8. Summary of Systolic Architectures (Slides) - Syeda Mohsina Afroze
http://web.cecs.pdx.edu/~mperkows/temp/May13/070-Systolic-Processors.pdf

CSC/ECE 506 Fall 2007/wiki1 10 aj

2007-09-06T02:45:56Z

Arangan: /* Systolic Architecture */

'''( Section 1.2.6)'''

== New developments in dataflow and systolic architectures ==

In the 1980s, a steady progress of parallel computing gave way for various unique parallel architectures that underwent extensive research but were not truly implemented to its full extent as when compared to architectures such as Shared Address Programming, Message Passing and Data Parallel Processing. Such architectures are still in its infancy and though its concepts have been utilized in various approaches for increasing processing efficiency; the actual architectural implementation is not widely employed for the sake of parallel computing.

Two such architectures are Dataflow Architecture and Systolic Architecture.

== Dataflow Architecture ==

'''Developments in Dataflow Architecture'''

(Reference: Based on ideas presented in the Wikipedia article at: http://en.wikipedia.org/wiki/Dataflow_architecture )

Dataflow architectures (from a hardware standpoint) was an important research topic in the 1970s and early 1980s. Interest in the field has subsided in recent years due to the inability to resolve certain inherent problems in the Dataflow architectural model. To understand, this, the Dataflow concept is summarized below.

'''Summary of the Dataflow Idea'''

In its essence, a Dataflow architecture executes instructions based on whether the input arguments to the instructions are available. There is no program counter as present in a Von Neumann computer. To indicate the dependency of instructions, tags were used. When the tags contained simple memory addresses, the design was a static Dataflow machine. However, static designs couldn’t allow for multiple instances of a routine.

Dynamic Dataflow machines used Content-Addressable memory (CAM)(i.e. the tags were stored in memory) to solve this problem.
The programs were loaded in the CAM. When the tagged operands of an instruction became available, the CAM would send it to an '''execution unit'''. After execution, the output data and it’s tags were sent back to the CAM (as a '''data token'''). The CAM would then execute the next instruction whose dependencies had been satisfied.

Since, the CAM could identify instructions whose tags were not dependant on any unexecuted instruction, parallelization was possible.
There were however, major problems related to -

1) broadcasting the data tokens in a massively parallel system efficiently.

2) dispatching instruction tokens in a massively parallel system efficiently.

3) A real program had a huge number of dependencies. Building a CAM large enough for this proved difficult.

Due to these issues, new developments in this area were largely stagnant.

'''New Developments in the Dataflow Architectural Model'''

(Referenced from: http://en.wikipedia.org/wiki/Out-of-order_execution)
1) A subset of the Dataflow model - '''Out of order execution''' is widely used in present day architecture. It uses the conventional Von Neumann Architecture to run '''Execution Windows''' in the usual order. Within the Window however, instructions are run according to the Dataflow Paradigm thus enabling parallelization and efficient utilization of CPU cycles.

(Referenced form: http://wavescalar.cs.washington.edu/)
2)The WaveScalar Instruction Set Architecture and Execution Model, currently being developed at the University of Washington attempts at building an architectural model that can work with current Imperative languages.

(Referenced from: http://portal.acm.org/citation.cfm?id=1032450 & http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?tp=&arnumber=1332593&isnumber=29428)

3)Calstrom and Boden propose a '''packet instruction set computer (PISC) architecture''' which would employ dataflow architecture for network processors. They detailed a 40 gb/s network processor using Dataflow architecture.

(Referenced from: http://ieeexplore.ieee.org/Xplore/login.jsp?url=/iel2/3036/8629/00379757.pdf?isnumber=&arnumber=379757)
4) Data flow architecture is also being evaluated for DSP processors as outlined in the referenced paper by Lee et al. Their proposed design was for a static Dataflow processor with 9 parallel processors for the multi-standard video codec processor.

== Systolic Architecture ==

'''Emergence of Systolic Arrays'''

During the 1980s, tremendous technological advances were being made in the field of VLSI circuits, which led to the birth of a several different concepts of highly parallel computing systems. One particular approach to such parallelism was using Systolic arrays, which constituted of a high number of processing elements known as cells, which were interconnected for local communication. This model utilizes a pipe-network arrangement of data processing units. Whenever a data object arrives, an operation is triggered in a data processing unit, unlike a central processing unit which has program counters. Several data streams are sent and received by a systolic array and several data counters are required to propagate data streams. This simple methodology facilitates parallelism.

'''Definition, Function and Applications'''

They are called systolic arrays; because data, similar to human blood circulation, circulates within the array and also may interact with other data. Some results that have been computed in certain cells are pumped into new cells as input, similar to the heart (Hence the analogy to a systolic mechanism of parallelism). H. T. Kung and Charles E. Leiserson invented the concept of systolic arrays being utilized as agents for parallel processing.

The Systolic Architecture is a mesh-like, interconnected set of processing elements, or cells, which interact with each other and pass on results that become inputs for the next computational time-step. Such proximity of high performance processors provide a highly required space efficiency, which is usually the main factor in parallel architecture. The structure of such systolic arrays may be non linear and multi-directional pathways may exist between processing elements. Even though this architecture is quite similar to Single-Instruction-Multiple-Data, unlike SIMD, each processing element is capable of processing a unique, different operation. The main feature of systolic architecture is that its highly specialized computations can be performed on simple parallel processors and localized communication patterns.

The systolic array is an architectural concept that is highly suited for very high integration technology. Communication between these mesh-connected processors occurs only amongst neighbors. Both control and data flow are local. Each processor has a communication register that can be read by any of its neighbors.

A Systolic array is a network of processing elements that has features like

• Modularity : each systolic array element is a module

• Synchrony : a global clock times the data through the network

• Locality (both spatial and temporal) : cells have local communication and a unit time delay

• Regularity : processing elements are homogeneously interconnected

• Parallel Computing and Pipelinability : architecture achieves high speed

The two main Systolic Architectural Features are

• Balancing Computation with I/O.

• Concurrency and Communication.

But other notable features of the architecture are its cost effectiveness, its regular & simple design, the modularity in arrays is quite high, the number of processors working simultaneously is high and also the local communication is fast – making overall execution time much faster.

''Types of Systolic Arrays''

1. Single Dimensional Input/Output Linear Array
- This is usually utilized for single input/output
[[Image:Systolic01.jpg]]

2. Two Dimensional Input/Output Linear Array
- This allows for more input/output and control over a linear array
[[Image:Systolic02.jpg]]

3. Perimeter Input/Output Planar Array
- This topology allows input/output through its boundary or perimeter cells only
[[Image:Systolic03.jpg]]

4. Three Dimensional Input/Output Focal Plane Array
- This facilitates input/output to each and every systolic cell in the array
[[Image:Systolic04.jpg]]

A few applications of Systolic Arrays

• Matrix Multiplication

• Solution of linear equations to problems in image processing

• Computer graphics

• Cryptography

• Computer tomography

• Matrix Inversion and Decomposition

• Convolution

• Polynomial Evaluation

• Lattice filters for seismic and speech signal processing

• Artificial Neural Networks

For a complete explanation on the invention of systolic arrays, utilize the following link : http://www.patentstorm.us/patents/4727503-fulltext.html

'''New Developments'''

''iWarp''

In 1989, the Intel Corporation launched the iWarp as a product and produced more than 1500 iWarp systems. The underlying architecture was based on systolic arrays and data streams being utilized for sending and receiving data between elements on the system. Even though the system was well suited for several applications, Intel stopped actively marketing the product in the mid-90s. It is still a useful tool for research in the parallel computing direction and it was one of the first successful steps towards true parallelism in the early ages of parallel computing.

''Image Processing and Morphology''

A two-dimensional systolic array consisting of cells can be implemented, based on parallel modules which provide internal pipeline operation where every cell can be configured according to a control word. Also, the array can be provided with a group of image buffers, that can reduce the number of accesses to data memory may also extend the array capabilities, allowing the possibility of chaining interconnection of multiple processing blocks.

''Digital Signal Processors''

Low cost, systolic architectural microprocessors utilizes a DSP systolic array, which makes it possible to build low cost ICs capable of providing high speed digital video services. This requires high processing capabilities, and parallelism is a requirement in such areas. Systolic architectures are better suited for such purposes as they are of much lower cost due to their space-efficiency and interconnectivity.

''Tilera Tile64''

Arranged in a two dimensional grid, the Tilera 64 is a tile that has 64 processing elements or cells which are interconnected through a five way switch and it provides a two level hierarchical cache. The architecture is completely based on systolic arrays and this sixty-four element multi-core processor is being released in September 2007. It is already predicted to be utilized extensively in the field of embedded systems for advanced networking, unified thread management, digital video, web services, etc.

'''Why Systolic Architecture has not truly evolved with time'''
(to the extent of other architectures, is because)

1. Global Synchronization takes more time due to signal delays within the network

2. Bandwidth requirements are tremendous for both processor and memory

3. Fault tolerance is very poor due as interconnection protocols are lacking

Also, a certain ‘parallel overhead’ point occurs in Systolic Arrays where beyond that point, adding more processors does not increase the efficiency of processing as it would normally be expected. There may even be a point beyond which adding more processors may even lead to slower execution time for the architecture.

'''References'''

1. (Reference: Based on ideas presented in the Wikipedia article at:http://en.wikipedia.org/wiki/Systolic_array)

2. Systolic Algorithms: Concepts, Synthesis and Evolution - Siang W.Song, University of Sau Paulo, Department Of Computer Science
http://citeseer.ist.psu.edu/cache/papers/cs/30734/http:zSzzSzwww.ime.usp.brzSz~songzSzpaperszSzcimpa.pdf/systolic-algorithms-concepts-synthesis.pdf

3. Instruction Systolic Array (ISA) - H.W. Lang, FH Flensburg
http://www.iti.fh-flensburg.de/lang/papers/isa/isa1.htm

4. Systolic Arrays : Patent Description
http://www.patentstorm.us/patents/4727503-fulltext.html

5. iWarp
http://www.cs.cmu.edu/~iwarp/

6. Tilera Tile64
http://www.tilera.com/products/processors.php

7. FPGA-Based Customizable Systolic Architecture for Image Processing Applications : Proceedings of the 2005 International Conference on Reconfigurable Computing and FPGAs (ReConFig'05) on Reconfigurable Computing and FPGAs - Griselda Saldana, Miguel Arias-Estrada (IEEE Computer Society)

8. Summary of Systolic Architectures (Slides)
http://web.cecs.pdx.edu/~mperkows/temp/May13/070-Systolic-Processors.pdf

CSC/ECE 506 Fall 2007/wiki1 10 aj

2007-09-06T02:45:14Z

Arangan: /* Systolic Architecture */

'''( Section 1.2.6)'''

== New developments in dataflow and systolic architectures ==

In the 1980s, a steady progress of parallel computing gave way for various unique parallel architectures that underwent extensive research but were not truly implemented to its full extent as when compared to architectures such as Shared Address Programming, Message Passing and Data Parallel Processing. Such architectures are still in its infancy and though its concepts have been utilized in various approaches for increasing processing efficiency; the actual architectural implementation is not widely employed for the sake of parallel computing.

Two such architectures are Dataflow Architecture and Systolic Architecture.

== Dataflow Architecture ==

'''Developments in Dataflow Architecture'''

(Reference: Based on ideas presented in the Wikipedia article at: http://en.wikipedia.org/wiki/Dataflow_architecture )

Dataflow architectures (from a hardware standpoint) was an important research topic in the 1970s and early 1980s. Interest in the field has subsided in recent years due to the inability to resolve certain inherent problems in the Dataflow architectural model. To understand, this, the Dataflow concept is summarized below.

'''Summary of the Dataflow Idea'''

In its essence, a Dataflow architecture executes instructions based on whether the input arguments to the instructions are available. There is no program counter as present in a Von Neumann computer. To indicate the dependency of instructions, tags were used. When the tags contained simple memory addresses, the design was a static Dataflow machine. However, static designs couldn’t allow for multiple instances of a routine.

Dynamic Dataflow machines used Content-Addressable memory (CAM)(i.e. the tags were stored in memory) to solve this problem.
The programs were loaded in the CAM. When the tagged operands of an instruction became available, the CAM would send it to an '''execution unit'''. After execution, the output data and it’s tags were sent back to the CAM (as a '''data token'''). The CAM would then execute the next instruction whose dependencies had been satisfied.

Since, the CAM could identify instructions whose tags were not dependant on any unexecuted instruction, parallelization was possible.
There were however, major problems related to -

1) broadcasting the data tokens in a massively parallel system efficiently.

2) dispatching instruction tokens in a massively parallel system efficiently.

3) A real program had a huge number of dependencies. Building a CAM large enough for this proved difficult.

Due to these issues, new developments in this area were largely stagnant.

'''New Developments in the Dataflow Architectural Model'''

(Referenced from: http://en.wikipedia.org/wiki/Out-of-order_execution)
1) A subset of the Dataflow model - '''Out of order execution''' is widely used in present day architecture. It uses the conventional Von Neumann Architecture to run '''Execution Windows''' in the usual order. Within the Window however, instructions are run according to the Dataflow Paradigm thus enabling parallelization and efficient utilization of CPU cycles.

(Referenced form: http://wavescalar.cs.washington.edu/)
2)The WaveScalar Instruction Set Architecture and Execution Model, currently being developed at the University of Washington attempts at building an architectural model that can work with current Imperative languages.

(Referenced from: http://portal.acm.org/citation.cfm?id=1032450 & http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?tp=&arnumber=1332593&isnumber=29428)

3)Calstrom and Boden propose a '''packet instruction set computer (PISC) architecture''' which would employ dataflow architecture for network processors. They detailed a 40 gb/s network processor using Dataflow architecture.

(Referenced from: http://ieeexplore.ieee.org/Xplore/login.jsp?url=/iel2/3036/8629/00379757.pdf?isnumber=&arnumber=379757)
4) Data flow architecture is also being evaluated for DSP processors as outlined in the referenced paper by Lee et al. Their proposed design was for a static Dataflow processor with 9 parallel processors for the multi-standard video codec processor.

== Systolic Architecture ==

'''Emergence of Systolic Arrays'''

During the 1980s, tremendous technological advances were being made in the field of VLSI circuits, which led to the birth of a several different concepts of highly parallel computing systems. One particular approach to such parallelism was using Systolic arrays, which constituted of a high number of processing elements known as cells, which were interconnected for local communication. This model utilizes a pipe-network arrangement of data processing units. Whenever a data object arrives, an operation is triggered in a data processing unit, unlike a central processing unit which has program counters. Several data streams are sent and received by a systolic array and several data counters are required to propagate data streams. This simple methodology facilitates parallelism.

'''Definition, Function and Applications'''

They are called systolic arrays; because data, similar to human blood circulation, circulates within the array and also may interact with other data. Some results that have been computed in certain cells are pumped into new cells as input, similar to the heart (Hence the analogy to a systolic mechanism of parallelism). H. T. Kung and Charles E. Leiserson invented the concept of systolic arrays being utilized as agents for parallel processing.

The Systolic Architecture is a mesh-like, interconnected set of processing elements, or cells, which interact with each other and pass on results that become inputs for the next computational time-step. Such proximity of high performance processors provide a highly required space efficiency, which is usually the main factor in parallel architecture. The structure of such systolic arrays may be non linear and multi-directional pathways may exist between processing elements. Even though this architecture is quite similar to Single-Instruction-Multiple-Data, unlike SIMD, each processing element is capable of processing a unique, different operation. The main feature of systolic architecture is that its highly specialized computations can be performed on simple parallel processors and localized communication patterns.

The systolic array is an architectural concept that is highly suited for very high integration technology. Communication between these mesh-connected processors occurs only amongst neighbors. Both control and data flow are local. Each processor has a communication register that can be read by any of its neighbors.

A Systolic array is a network of processing elements that has features like

• Modularity : each systolic array element is a module

• Synchrony : a global clock times the data through the network

• Locality (both spatial and temporal) : cells have local communication and a unit time delay

• Regularity : processing elements are homogeneously interconnected

• Parallel Computing and Pipelinability : architecture achieves high speed

The two main Systolic Architectural Features are

• Balancing Computation with I/O.

• Concurrency and Communication.

But other notable features of the architecture are its cost effectiveness, its regular & simple design, the modularity in arrays is quite high, the number of processors working simultaneously is high and also the local communication is fast – making overall execution time much faster.

''Types of Systolic Arrays''

1. Single Dimensional Input/Output Linear Array
- This is usually utilized for single input/output
[[Image:Systolic01.jpg]]

2. Two Dimensional Input/Output Linear Array
- This allows for more input/output and control over a linear array
[[Image:Systolic02.jpg]]

3. Perimeter Input/Output Planar Array
- This topology allows input/output through its boundary or perimeter cells only
[[Image:Systolic03.jpg]]

4. Three Dimensional Input/Output Focal Plane Array
- This facilitates input/output to each and every systolic cell in the array
[[Image:Systolic04.jpg]]

A few applications of Systolic Arrays

• Matrix Multiplication

• Solution of linear equations to problems in image processing

• Computer graphics

• Cryptography

• Computer tomography

• Matrix Inversion and Decomposition

• Convolution

• Polynomial Evaluation

• Lattice filters for seismic and speech signal processing

• Artificial Neural Networks

For a complete explanation on the invention of systolic arrays, utilize the following link : http://www.patentstorm.us/patents/4727503-fulltext.html

'''New Developments'''

''iWarp''

In 1989, the Intel Corporation launched the iWarp as a product and produced more than 1500 iWarp systems. The underlying architecture was based on systolic arrays and data streams being utilized for sending and receiving data between elements on the system. Even though the system was well suited for several applications, Intel stopped actively marketing the product in the mid-90s. It is still a useful tool for research in the parallel computing direction and it was one of the first successful steps towards true parallelism in the early ages of parallel computing.

''Image Processing and Morphology''

A two-dimensional systolic array consisting of cells can be implemented, based on parallel modules which provide internal pipeline operation where every cell can be configured according to a control word. Also, the array can be provided with a group of image buffers, that can reduce the number of accesses to data memory may also extend the array capabilities, allowing the possibility of chaining interconnection of multiple processing blocks.

''Digital Signal Processors''

Low cost, systolic architectural microprocessors utilizes a DSP systolic array, which makes it possible to build low cost ICs capable of providing high speed digital video services. This requires high processing capabilities, and parallelism is a requirement in such areas. Systolic architectures are better suited for such purposes as they are of much lower cost due to their space-efficiency and interconnectivity.

''Tilera Tile64''

Arranged in a two dimensional grid, the Tilera 64 is a tile that has 64 processing elements or cells which are interconnected through a five way switch and it provides a two level hierarchical cache. The architecture is completely based on systolic arrays and this sixty-four element multi-core processor is being released in September 2007. It is already predicted to be utilized extensively in the field of embedded systems for advanced networking, unified thread management, digital video, web services, etc.

'''Why Systolic Architecture has not truly evolved with time'''
(to the extent of other architectures is because)

1. Global Synchronization takes more time due to signal delays within the network

2. Bandwidth requirements are tremendous for both processor and memory

3. Fault tolerance is very poor due as interconnection protocols are lacking

Also, a certain ‘parallel overhead’ point occurs in Systolic Arrays where beyond that point, adding more processors does not increase the efficiency of processing as it would normally be expected. There may even be a point beyond which adding more processors may even lead to slower execution time for the architecture.

'''References'''

1. (Reference: Based on ideas presented in the Wikipedia article at:http://en.wikipedia.org/wiki/Systolic_array)

2. Systolic Algorithms: Concepts, Synthesis and Evolution - Siang W.Song, University of Sau Paulo, Department Of Computer Science
http://citeseer.ist.psu.edu/cache/papers/cs/30734/http:zSzzSzwww.ime.usp.brzSz~songzSzpaperszSzcimpa.pdf/systolic-algorithms-concepts-synthesis.pdf

3. Instruction Systolic Array (ISA) - H.W. Lang, FH Flensburg
http://www.iti.fh-flensburg.de/lang/papers/isa/isa1.htm

4. Systolic Arrays : Patent Description
http://www.patentstorm.us/patents/4727503-fulltext.html

5. iWarp
http://www.cs.cmu.edu/~iwarp/

6. Tilera Tile64
http://www.tilera.com/products/processors.php

7. FPGA-Based Customizable Systolic Architecture for Image Processing Applications : Proceedings of the 2005 International Conference on Reconfigurable Computing and FPGAs (ReConFig'05) on Reconfigurable Computing and FPGAs - Griselda Saldana, Miguel Arias-Estrada (IEEE Computer Society)

8. Summary of Systolic Architectures (Slides)
http://web.cecs.pdx.edu/~mperkows/temp/May13/070-Systolic-Processors.pdf

CSC/ECE 506 Fall 2007/wiki1 10 aj

2007-09-06T02:44:27Z

Arangan: /* Systolic Architecture */

'''( Section 1.2.6)'''

== New developments in dataflow and systolic architectures ==

In the 1980s, a steady progress of parallel computing gave way for various unique parallel architectures that underwent extensive research but were not truly implemented to its full extent as when compared to architectures such as Shared Address Programming, Message Passing and Data Parallel Processing. Such architectures are still in its infancy and though its concepts have been utilized in various approaches for increasing processing efficiency; the actual architectural implementation is not widely employed for the sake of parallel computing.

Two such architectures are Dataflow Architecture and Systolic Architecture.

== Dataflow Architecture ==

'''Developments in Dataflow Architecture'''

(Reference: Based on ideas presented in the Wikipedia article at: http://en.wikipedia.org/wiki/Dataflow_architecture )

Dataflow architectures (from a hardware standpoint) was an important research topic in the 1970s and early 1980s. Interest in the field has subsided in recent years due to the inability to resolve certain inherent problems in the Dataflow architectural model. To understand, this, the Dataflow concept is summarized below.

'''Summary of the Dataflow Idea'''

In its essence, a Dataflow architecture executes instructions based on whether the input arguments to the instructions are available. There is no program counter as present in a Von Neumann computer. To indicate the dependency of instructions, tags were used. When the tags contained simple memory addresses, the design was a static Dataflow machine. However, static designs couldn’t allow for multiple instances of a routine.

Dynamic Dataflow machines used Content-Addressable memory (CAM)(i.e. the tags were stored in memory) to solve this problem.
The programs were loaded in the CAM. When the tagged operands of an instruction became available, the CAM would send it to an '''execution unit'''. After execution, the output data and it’s tags were sent back to the CAM (as a '''data token'''). The CAM would then execute the next instruction whose dependencies had been satisfied.

Since, the CAM could identify instructions whose tags were not dependant on any unexecuted instruction, parallelization was possible.
There were however, major problems related to -

1) broadcasting the data tokens in a massively parallel system efficiently.

2) dispatching instruction tokens in a massively parallel system efficiently.

3) A real program had a huge number of dependencies. Building a CAM large enough for this proved difficult.

Due to these issues, new developments in this area were largely stagnant.

'''New Developments in the Dataflow Architectural Model'''

(Referenced from: http://en.wikipedia.org/wiki/Out-of-order_execution)
1) A subset of the Dataflow model - '''Out of order execution''' is widely used in present day architecture. It uses the conventional Von Neumann Architecture to run '''Execution Windows''' in the usual order. Within the Window however, instructions are run according to the Dataflow Paradigm thus enabling parallelization and efficient utilization of CPU cycles.

(Referenced form: http://wavescalar.cs.washington.edu/)
2)The WaveScalar Instruction Set Architecture and Execution Model, currently being developed at the University of Washington attempts at building an architectural model that can work with current Imperative languages.

(Referenced from: http://portal.acm.org/citation.cfm?id=1032450 & http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?tp=&arnumber=1332593&isnumber=29428)

3)Calstrom and Boden propose a '''packet instruction set computer (PISC) architecture''' which would employ dataflow architecture for network processors. They detailed a 40 gb/s network processor using Dataflow architecture.

(Referenced from: http://ieeexplore.ieee.org/Xplore/login.jsp?url=/iel2/3036/8629/00379757.pdf?isnumber=&arnumber=379757)
4) Data flow architecture is also being evaluated for DSP processors as outlined in the referenced paper by Lee et al. Their proposed design was for a static Dataflow processor with 9 parallel processors for the multi-standard video codec processor.

== Systolic Architecture ==

'''Emergence of Systolic Arrays'''

During the 1980s, tremendous technological advances were being made in the field of VLSI circuits, which led to the birth of a several different concepts of highly parallel computing systems. One particular approach to such parallelism was using Systolic arrays, which constituted of a high number of processing elements known as cells, which were interconnected for local communication. This model utilizes a pipe-network arrangement of data processing units. Whenever a data object arrives, an operation is triggered in a data processing unit, unlike a central processing unit which has program counters. Several data streams are sent and received by a systolic array and several data counters are required to propagate data streams. This simple methodology facilitates parallelism.

'''Definition, Function and Applications'''

They are called systolic arrays; because data, similar to human blood circulation, circulates within the array and also may interact with other data. Some results that have been computed in certain cells are pumped into new cells as input, similar to the heart (Hence the analogy to a systolic mechanism of parallelism). H. T. Kung and Charles E. Leiserson invented the concept of systolic arrays being utilized as agents for parallel processing.

The Systolic Architecture is a mesh-like, interconnected set of processing elements, or cells, which interact with each other and pass on results that become inputs for the next computational time-step. Such proximity of high performance processors provide a highly required space efficiency, which is usually the main factor in parallel architecture. The structure of such systolic arrays may be non linear and multi-directional pathways may exist between processing elements. Even though this architecture is quite similar to Single-Instruction-Multiple-Data, unlike SIMD, each processing element is capable of processing a unique, different operation. The main feature of systolic architecture is that its highly specialized computations can be performed on simple parallel processors and localized communication patterns.

The systolic array is an architectural concept that is highly suited for very high integration technology. Communication between these mesh-connected processors occurs only amongst neighbors. Both control and data flow are local. Each processor has a communication register that can be read by any of its neighbors.

A Systolic array is a network of processing elements that has features like

• Modularity : each systolic array element is a module

• Synchrony : a global clock times the data through the network

• Locality (both spatial and temporal) : cells have local communication and a unit time delay

• Regularity : processing elements are homogeneously interconnected

• Parallel Computing and Pipelinability : architecture achieves high speed

The two main Systolic Architectural Features are

• Balancing Computation with I/O.

• Concurrency and Communication.

But other notable features of the architecture are its cost effectiveness, its regular & simple design, the modularity in arrays is quite high, the number of processors working simultaneously is high and also the local communication is fast – making overall execution time much faster.

''Types of Systolic Arrays''

1. Single Dimensional Input/Output Linear Array
- This is usually utilized for single input/output
[[Image:Systolic01.jpg]]

2. Two Dimensional Input/Output Linear Array
- This allows for more input/output and control over a linear array
[[Image:Systolic02.jpg]]

3. Perimeter Input/Output Planar Array
- This topology allows input/output through its boundary or perimeter cells only
[[Image:Systolic03.jpg]]

4. Three Dimensional Input/Output Focal Plane Array
- This facilitates input/output to each and every systolic cell in the array
[[Image:Systolic04.jpg]]

A few applications of Systolic Arrays

• Matrix Multiplication

• Solution of linear equations to problems in image processing

• Computer graphics

• Cryptography

• Computer tomography

• Matrix Inversion and Decomposition

• Convolution

• Polynomial Evaluation

• Lattice filters for seismic and speech signal processing

• Artificial Neural Networks

For a complete explanation on the invention of systolic arrays, utilize the following link : http://www.patentstorm.us/patents/4727503-fulltext.html

'''New Developments'''

''iWarp''

In 1989, the Intel Corporation launched the iWarp as a product and produced more than 1500 iWarp systems. The underlying architecture was based on systolic arrays and data streams being utilized for sending and receiving data between elements on the system. Even though the system was well suited for several applications, Intel stopped actively marketing the product in the mid-90s. It is still a useful tool for research in the parallel computing direction and it was one of the first successful steps towards true parallelism in the early ages of parallel computing.

''Image Processing and Morphology''

A two-dimensional systolic array consisting of cells can be implemented, based on parallel modules which provide internal pipeline operation where every cell can be configured according to a control word. Also, the array can be provided with a group of image buffers, that can reduce the number of accesses to data memory may also extend the array capabilities, allowing the possibility of chaining interconnection of multiple processing blocks.

''Digital Signal Processors''

Low cost, systolic architectural microprocessors utilizes a DSP systolic array, which makes it possible to build low cost ICs capable of providing high speed digital video services. This requires high processing capabilities, and parallelism is a requirement in such areas. Systolic architectures are better suited for such purposes as they are of much lower cost due to their space-efficiency and interconnectivity.

''Tilera Tile64''

Arranged in a two dimensional grid, the Tilera 64 is a tile that has 64 processing elements or cells which are interconnected through a five way switch and it provides a two level hierarchical cache. The architecture is completely based on systolic arrays and this sixty-four element multi-core processor is being released in September 2007. It is already predicted to be utilized extensively in the field of embedded systems for advanced networking, unified thread management, digital video, web services, etc.

''Why Systolic Architecture has not truly evolved with time''
(to the extent of other architectures is because)
1. Global Synchronization takes more time due to signal delays within the network
2. Bandwidth requirements are tremendous for both processor and memory
3. Fault tolerance is very poor due as interconnection protocols are lacking

Also, a certain ‘parallel overhead’ point occurs in Systolic Arrays where beyond that point, adding more processors does not increase the efficiency of processing as it would normally be expected. There may even be a point beyond which adding more processors may even lead to slower execution time for the architecture.

'''References'''

1. (Reference: Based on ideas presented in the Wikipedia article at:http://en.wikipedia.org/wiki/Systolic_array)

2. Systolic Algorithms: Concepts, Synthesis and Evolution - Siang W.Song, University of Sau Paulo, Department Of Computer Science
http://citeseer.ist.psu.edu/cache/papers/cs/30734/http:zSzzSzwww.ime.usp.brzSz~songzSzpaperszSzcimpa.pdf/systolic-algorithms-concepts-synthesis.pdf

3. Instruction Systolic Array (ISA) - H.W. Lang, FH Flensburg
http://www.iti.fh-flensburg.de/lang/papers/isa/isa1.htm

4. Systolic Arrays : Patent Description
http://www.patentstorm.us/patents/4727503-fulltext.html

5. iWarp
http://www.cs.cmu.edu/~iwarp/

6. Tilera Tile64
http://www.tilera.com/products/processors.php

7. FPGA-Based Customizable Systolic Architecture for Image Processing Applications : Proceedings of the 2005 International Conference on Reconfigurable Computing and FPGAs (ReConFig'05) on Reconfigurable Computing and FPGAs - Griselda Saldana, Miguel Arias-Estrada (IEEE Computer Society)

8. Summary of Systolic Architectures (Slides)
http://web.cecs.pdx.edu/~mperkows/temp/May13/070-Systolic-Processors.pdf

CSC/ECE 506 Fall 2007/wiki1 10 aj

2007-09-06T02:42:38Z

Arangan: /* Systolic Architecture */

'''( Section 1.2.6)'''

== New developments in dataflow and systolic architectures ==

In the 1980s, a steady progress of parallel computing gave way for various unique parallel architectures that underwent extensive research but were not truly implemented to its full extent as when compared to architectures such as Shared Address Programming, Message Passing and Data Parallel Processing. Such architectures are still in its infancy and though its concepts have been utilized in various approaches for increasing processing efficiency; the actual architectural implementation is not widely employed for the sake of parallel computing.

Two such architectures are Dataflow Architecture and Systolic Architecture.

== Dataflow Architecture ==

'''Developments in Dataflow Architecture'''

(Reference: Based on ideas presented in the Wikipedia article at: http://en.wikipedia.org/wiki/Dataflow_architecture )

Dataflow architectures (from a hardware standpoint) was an important research topic in the 1970s and early 1980s. Interest in the field has subsided in recent years due to the inability to resolve certain inherent problems in the Dataflow architectural model. To understand, this, the Dataflow concept is summarized below.

'''Summary of the Dataflow Idea'''

In its essence, a Dataflow architecture executes instructions based on whether the input arguments to the instructions are available. There is no program counter as present in a Von Neumann computer. To indicate the dependency of instructions, tags were used. When the tags contained simple memory addresses, the design was a static Dataflow machine. However, static designs couldn’t allow for multiple instances of a routine.

Dynamic Dataflow machines used Content-Addressable memory (CAM)(i.e. the tags were stored in memory) to solve this problem.
The programs were loaded in the CAM. When the tagged operands of an instruction became available, the CAM would send it to an '''execution unit'''. After execution, the output data and it’s tags were sent back to the CAM (as a '''data token'''). The CAM would then execute the next instruction whose dependencies had been satisfied.

Since, the CAM could identify instructions whose tags were not dependant on any unexecuted instruction, parallelization was possible.
There were however, major problems related to -

1) broadcasting the data tokens in a massively parallel system efficiently.

2) dispatching instruction tokens in a massively parallel system efficiently.

3) A real program had a huge number of dependencies. Building a CAM large enough for this proved difficult.

Due to these issues, new developments in this area were largely stagnant.

'''New Developments in the Dataflow Architectural Model'''

(Referenced from: http://en.wikipedia.org/wiki/Out-of-order_execution)
1) A subset of the Dataflow model - '''Out of order execution''' is widely used in present day architecture. It uses the conventional Von Neumann Architecture to run '''Execution Windows''' in the usual order. Within the Window however, instructions are run according to the Dataflow Paradigm thus enabling parallelization and efficient utilization of CPU cycles.

(Referenced form: http://wavescalar.cs.washington.edu/)
2)The WaveScalar Instruction Set Architecture and Execution Model, currently being developed at the University of Washington attempts at building an architectural model that can work with current Imperative languages.

(Referenced from: http://portal.acm.org/citation.cfm?id=1032450 & http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?tp=&arnumber=1332593&isnumber=29428)

3)Calstrom and Boden propose a '''packet instruction set computer (PISC) architecture''' which would employ dataflow architecture for network processors. They detailed a 40 gb/s network processor using Dataflow architecture.

(Referenced from: http://ieeexplore.ieee.org/Xplore/login.jsp?url=/iel2/3036/8629/00379757.pdf?isnumber=&arnumber=379757)
4) Data flow architecture is also being evaluated for DSP processors as outlined in the referenced paper by Lee et al. Their proposed design was for a static Dataflow processor with 9 parallel processors for the multi-standard video codec processor.

== Systolic Architecture ==

'''Emergence of Systolic Arrays'''

During the 1980s, tremendous technological advances were being made in the field of VLSI circuits, which led to the birth of a several different concepts of highly parallel computing systems. One particular approach to such parallelism was using Systolic arrays, which constituted of a high number of processing elements known as cells, which were interconnected for local communication. This model utilizes a pipe-network arrangement of data processing units. Whenever a data object arrives, an operation is triggered in a data processing unit, unlike a central processing unit which has program counters. Several data streams are sent and received by a systolic array and several data counters are required to propagate data streams. This simple methodology facilitates parallelism.

'''Definition, Function and Applications'''

They are called systolic arrays; because data, similar to human blood circulation, circulates within the array and also may interact with other data. Some results that have been computed in certain cells are pumped into new cells as input, similar to the heart (Hence the analogy to a systolic mechanism of parallelism). H. T. Kung and Charles E. Leiserson invented the concept of systolic arrays being utilized as agents for parallel processing.

The Systolic Architecture is a mesh-like, interconnected set of processing elements, or cells, which interact with each other and pass on results that become inputs for the next computational time-step. Such proximity of high performance processors provide a highly required space efficiency, which is usually the main factor in parallel architecture. The structure of such systolic arrays may be non linear and multi-directional pathways may exist between processing elements. Even though this architecture is quite similar to Single-Instruction-Multiple-Data, unlike SIMD, each processing element is capable of processing a unique, different operation. The main feature of systolic architecture is that its highly specialized computations can be performed on simple parallel processors and localized communication patterns.

The systolic array is an architectural concept that is highly suited for very high integration technology. Communication between these mesh-connected processors occurs only amongst neighbors. Both control and data flow are local. Each processor has a communication register that can be read by any of its neighbors.

A Systolic array is a network of processing elements that has features like

• Modularity : each systolic array element is a module

• Synchrony : a global clock times the data through the network

• Locality (both spatial and temporal) : cells have local communication and a unit time delay

• Regularity : processing elements are homogeneously interconnected

• Parallel Computing and Pipelinability : architecture achieves high speed

The two main Systolic Architectural Features are

• Balancing Computation with I/O.

• Concurrency and Communication.

But other notable features of the architecture are its cost effectiveness, its regular & simple design, the modularity in arrays is quite high, the number of processors working simultaneously is high and also the local communication is fast – making overall execution time much faster.

''Types of Systolic Arrays''

1. Single Dimensional Input/Output Linear Array
- This is usually utilized for single input/output
[[Image:Systolic01.jpg]]

2. Two Dimensional Input/Output Linear Array
- This allows for more input/output and control over a linear array
[[Image:Systolic02.jpg]]

3. Perimeter Input/Output Planar Array
- This topology allows input/output through its boundary or perimeter cells only
[[Image:Systolic03.jpg]]

4. Three Dimensional Input/Output Focal Plane Array
- This facilitates input/output to each and every systolic cell in the array
[[Image:Systolic04.jpg]]

A few applications of Systolic Arrays

• Matrix Multiplication

• Solution of linear equations to problems in image processing

• Computer graphics

• Cryptography

• Computer tomography

• Matrix Inversion and Decomposition

• Convolution

• Polynomial Evaluation

• Lattice filters for seismic and speech signal processing

• Artificial Neural Networks

For a complete explanation on the invention of systolic arrays, utilize the following link : http://www.patentstorm.us/patents/4727503-fulltext.html

'''New Developments'''

''iWarp''

In 1989, the Intel Corporation launched the iWarp as a product and produced more than 1500 iWarp systems. The underlying architecture was based on systolic arrays and data streams being utilized for sending and receiving data between elements on the system. Even though the system was well suited for several applications, Intel stopped actively marketing the product in the mid-90s. It is still a useful tool for research in the parallel computing direction and it was one of the first successful steps towards true parallelism in the early ages of parallel computing.

''Image Processing and Morphology''

A two-dimensional systolic array consisting of cells can be implemented, based on parallel modules which provide internal pipeline operation where every cell can be configured according to a control word. Also, the array can be provided with a group of image buffers, that can reduce the number of accesses to data memory may also extend the array capabilities, allowing the possibility of chaining interconnection of multiple processing blocks.

''Digital Signal Processors''

Low cost, systolic architectural microprocessors utilizes a DSP systolic array, which makes it possible to build low cost ICs capable of providing high speed digital video services. This requires high processing capabilities, and parallelism is a requirement in such areas. Systolic architectures are better suited for such purposes as they are of much lower cost due to their space-efficiency and interconnectivity.

''Tilera Tile64''

Arranged in a two dimensional grid, the Tilera 64 is a tile that has 64 processing elements or cells which are interconnected through a five way switch and it provides a two level hierarchical cache. The architecture is completely based on systolic arrays and this sixty-four element multi-core processor is being released in September 2007. It is already predicted to be utilized extensively in the field of embedded systems for advanced networking, unified thread management, digital video, web services, etc.

'''References'''

1. (Reference: Based on ideas presented in the Wikipedia article at:http://en.wikipedia.org/wiki/Systolic_array)

2. Systolic Algorithms: Concepts, Synthesis and Evolution - Siang W.Song, University of Sau Paulo, Department Of Computer Science
http://citeseer.ist.psu.edu/cache/papers/cs/30734/http:zSzzSzwww.ime.usp.brzSz~songzSzpaperszSzcimpa.pdf/systolic-algorithms-concepts-synthesis.pdf

3. Instruction Systolic Array (ISA) - H.W. Lang, FH Flensburg
http://www.iti.fh-flensburg.de/lang/papers/isa/isa1.htm

4. Systolic Arrays : Patent Description
http://www.patentstorm.us/patents/4727503-fulltext.html

5. iWarp
http://www.cs.cmu.edu/~iwarp/

6. Tilera Tile64
http://www.tilera.com/products/processors.php

7. FPGA-Based Customizable Systolic Architecture for Image Processing Applications : Proceedings of the 2005 International Conference on Reconfigurable Computing and FPGAs (ReConFig'05) on Reconfigurable Computing and FPGAs - Griselda Saldana, Miguel Arias-Estrada (IEEE Computer Society)

8. Summary of Systolic Architectures (Slides)
http://web.cecs.pdx.edu/~mperkows/temp/May13/070-Systolic-Processors.pdf

CSC/ECE 506 Fall 2007/wiki1 10 aj

2007-09-06T02:42:14Z

Arangan: /* Systolic Architecture */

'''( Section 1.2.6)'''

== New developments in dataflow and systolic architectures ==

In the 1980s, a steady progress of parallel computing gave way for various unique parallel architectures that underwent extensive research but were not truly implemented to its full extent as when compared to architectures such as Shared Address Programming, Message Passing and Data Parallel Processing. Such architectures are still in its infancy and though its concepts have been utilized in various approaches for increasing processing efficiency; the actual architectural implementation is not widely employed for the sake of parallel computing.

Two such architectures are Dataflow Architecture and Systolic Architecture.

== Dataflow Architecture ==

'''Developments in Dataflow Architecture'''

(Reference: Based on ideas presented in the Wikipedia article at: http://en.wikipedia.org/wiki/Dataflow_architecture )

Dataflow architectures (from a hardware standpoint) was an important research topic in the 1970s and early 1980s. Interest in the field has subsided in recent years due to the inability to resolve certain inherent problems in the Dataflow architectural model. To understand, this, the Dataflow concept is summarized below.

'''Summary of the Dataflow Idea'''

In its essence, a Dataflow architecture executes instructions based on whether the input arguments to the instructions are available. There is no program counter as present in a Von Neumann computer. To indicate the dependency of instructions, tags were used. When the tags contained simple memory addresses, the design was a static Dataflow machine. However, static designs couldn’t allow for multiple instances of a routine.

Dynamic Dataflow machines used Content-Addressable memory (CAM)(i.e. the tags were stored in memory) to solve this problem.
The programs were loaded in the CAM. When the tagged operands of an instruction became available, the CAM would send it to an '''execution unit'''. After execution, the output data and it’s tags were sent back to the CAM (as a '''data token'''). The CAM would then execute the next instruction whose dependencies had been satisfied.

Since, the CAM could identify instructions whose tags were not dependant on any unexecuted instruction, parallelization was possible.
There were however, major problems related to -

1) broadcasting the data tokens in a massively parallel system efficiently.

2) dispatching instruction tokens in a massively parallel system efficiently.

3) A real program had a huge number of dependencies. Building a CAM large enough for this proved difficult.

Due to these issues, new developments in this area were largely stagnant.

'''New Developments in the Dataflow Architectural Model'''

(Referenced from: http://en.wikipedia.org/wiki/Out-of-order_execution)
1) A subset of the Dataflow model - '''Out of order execution''' is widely used in present day architecture. It uses the conventional Von Neumann Architecture to run '''Execution Windows''' in the usual order. Within the Window however, instructions are run according to the Dataflow Paradigm thus enabling parallelization and efficient utilization of CPU cycles.

(Referenced form: http://wavescalar.cs.washington.edu/)
2)The WaveScalar Instruction Set Architecture and Execution Model, currently being developed at the University of Washington attempts at building an architectural model that can work with current Imperative languages.

(Referenced from: http://portal.acm.org/citation.cfm?id=1032450 & http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?tp=&arnumber=1332593&isnumber=29428)

3)Calstrom and Boden propose a '''packet instruction set computer (PISC) architecture''' which would employ dataflow architecture for network processors. They detailed a 40 gb/s network processor using Dataflow architecture.

(Referenced from: http://ieeexplore.ieee.org/Xplore/login.jsp?url=/iel2/3036/8629/00379757.pdf?isnumber=&arnumber=379757)
4) Data flow architecture is also being evaluated for DSP processors as outlined in the referenced paper by Lee et al. Their proposed design was for a static Dataflow processor with 9 parallel processors for the multi-standard video codec processor.

== Systolic Architecture ==

'''Emergence of Systolic Arrays'''

During the 1980s, tremendous technological advances were being made in the field of VLSI circuits, which led to the birth of a several different concepts of highly parallel computing systems. One particular approach to such parallelism was using Systolic arrays, which constituted of a high number of processing elements known as cells, which were interconnected for local communication. This model utilizes a pipe-network arrangement of data processing units. Whenever a data object arrives, an operation is triggered in a data processing unit, unlike a central processing unit which has program counters. Several data streams are sent and received by a systolic array and several data counters are required to propagate data streams. This simple methodology facilitates parallelism.

'''Definition, Function and Applications'''

They are called systolic arrays; because data, similar to human blood circulation, circulates within the array and also may interact with other data. Some results that have been computed in certain cells are pumped into new cells as input, similar to the heart (Hence the analogy to a systolic mechanism of parallelism). H. T. Kung and Charles E. Leiserson invented the concept of systolic arrays being utilized as agents for parallel processing.

The Systolic Architecture is a mesh-like, interconnected set of processing elements, or cells, which interact with each other and pass on results that become inputs for the next computational time-step. Such proximity of high performance processors provide a highly required space efficiency, which is usually the main factor in parallel architecture. The structure of such systolic arrays may be non linear and multi-directional pathways may exist between processing elements. Even though this architecture is quite similar to Single-Instruction-Multiple-Data, unlike SIMD, each processing element is capable of processing a unique, different operation. The main feature of systolic architecture is that its highly specialized computations can be performed on simple parallel processors and localized communication patterns.

The systolic array is an architectural concept that is highly suited for very high integration technology. Communication between these mesh-connected processors occurs only amongst neighbors. Both control and data flow are local. Each processor has a communication register that can be read by any of its neighbors.

A Systolic array is a network of processing elements that has features like

• Modularity : each systolic array element is a module

• Synchrony : a global clock times the data through the network

• Locality (both spatial and temporal) : cells have local communication and a unit time delay

• Regularity : processing elements are homogeneously interconnected

• Parallel Computing and Pipelinability : architecture achieves high speed

The two main Systolic Architectural Features are

• Balancing Computation with I/O.

• Concurrency and Communication.

But other notable features of the architecture are its cost effectiveness, its regular & simple design, the modularity in arrays is quite high, the number of processors working simultaneously is high and also the local communication is fast – making overall execution time much faster.

''Types of Systolic Arrays''

1. Single Dimensional Input/Output Linear Array
- This is usually utilized for single input/output
[[Image:Systolic01.jpg]]

2. Two Dimensional Input/Output Linear Array
- This allows for more input/output and control over a linear array
[[Image:Systolic02.jpg]]

3. Perimeter Input/Output Planar Array
- This topology allows input/output through its boundary or perimeter cells only
[[Image:Systolic03.jpg]]

4. Three Dimensional Input/Output Focal Plane Array
- This facilitates input/output to each and every systolic cell in the array
[[Image:Systolic04.jpg]]

A few applications of Systolic Arrays

• Matrix Multiplication

• Solution of linear equations to problems in image processing

• Computer graphics

• Cryptography

• Computer tomography

• Matrix Inversion and Decomposition

• Convolution

• Polynomial Evaluation

• Lattice filters for seismic and speech signal processing

• Artificial Neural Networks

For a complete explanation on the invention of systolic arrays, utilize the following link : http://www.patentstorm.us/patents/4727503-fulltext.html

'''New Developments'''

''iWarp''

In 1989, the Intel Corporation launched the iWarp as a product and produced more than 1500 iWarp systems. The underlying architecture was based on systolic arrays and data streams being utilized for sending and receiving data between elements on the system. Even though the system was well suited for several applications, Intel stopped actively marketing the product in the mid-90s. It is still a useful tool for research in the parallel computing direction and it was one of the first successful steps towards true parallelism in the early ages of parallel computing.

''Image Processing and Morphology''

A two-dimensional systolic array consisting of cells can be implemented, based on parallel modules which provide internal pipeline operation where every cell can be configured according to a control word. Also, the array can be provided with a group of image buffers, that can reduce the number of accesses to data memory may also extend the array capabilities, allowing the possibility of chaining interconnection of multiple processing blocks.

''Digital Signal Processors''

Low cost, systolic architectural microprocessors utilizes a DSP systolic array, which makes it possible to build low cost ICs capable of providing high speed digital video services. This requires high processing capabilities, and parallelism is a requirement in such areas. Systolic architectures are better suited for such purposes as they are of much lower cost due to their space-efficiency and interconnectivity.

''Tilera Tile64''

Arranged in a two dimensional grid, the Tilera 64 is a tile that has 64 processing elements or cells which are interconnected through a five way switch and it provides a two level hierarchical cache. The architecture is completely based on systolic arrays and this sixty-four element multi-core processor is being released in September 2007. It is already predicted to be utilized extensively in the field of embedded systems for advanced networking, unified thread management, digital video, web services, etc.

'''References'''

1. (Reference: Based on ideas presented in the Wikipedia article at:http://en.wikipedia.org/wiki/Systolic_array)

2. Systolic Algorithms: Concepts, Synthesis and Evolution - Siang W.Song, University of Sau Paulo, Department Of Computer Science
http://citeseer.ist.psu.edu/cache/papers/cs/30734/http:zSzzSzwww.ime.usp.brzSz~songzSzpaperszSzcimpa.pdf/systolic-algorithms-concepts-synthesis.pdf

3. Instruction Systolic Array (ISA) - H.W. Lang, FH Flensburg
http://www.iti.fh-flensburg.de/lang/papers/isa/isa1.htm

4. Systolic Arrays : Patent Description
http://www.patentstorm.us/patents/4727503-fulltext.html

5. iWarp
http://www.cs.cmu.edu/~iwarp/

6. Tilera Tile64
http://www.tilera.com/products/processors.php

7. FPGA-Based Customizable Systolic Architecture for Image Processing Applications : Proceedings of the 2005 International Conference on Reconfigurable Computing and FPGAs (ReConFig'05) on Reconfigurable Computing and FPGAs - Griselda Saldana, Miguel Arias-Estrada (IEEE Computer Society)

8. Summary of Systolic Architectures (Slides)
http://web.cecs.pdx.edu/~mperkows/temp/May13/070-Systolic-Processors.pdf

File:Systolic04.jpg

2007-09-06T02:41:15Z

Arangan:

File:Systolic03.jpg

2007-09-06T02:40:53Z

Arangan:

File:Systolic02.jpg

2007-09-06T02:40:37Z

Arangan:

File:Systolic01.jpg

2007-09-06T02:40:04Z

Arangan:

CSC/ECE 506 Fall 2007/wiki1 10 aj

2007-09-06T02:39:10Z

Arangan: /* Systolic Architecture */

'''( Section 1.2.6)'''

== New developments in dataflow and systolic architectures ==

In the 1980s, a steady progress of parallel computing gave way for various unique parallel architectures that underwent extensive research but were not truly implemented to its full extent as when compared to architectures such as Shared Address Programming, Message Passing and Data Parallel Processing. Such architectures are still in its infancy and though its concepts have been utilized in various approaches for increasing processing efficiency; the actual architectural implementation is not widely employed for the sake of parallel computing.

Two such architectures are Dataflow Architecture and Systolic Architecture.

== Dataflow Architecture ==

'''Developments in Dataflow Architecture'''

(Reference: Based on ideas presented in the Wikipedia article at: http://en.wikipedia.org/wiki/Dataflow_architecture )

Dataflow architectures (from a hardware standpoint) was an important research topic in the 1970s and early 1980s. Interest in the field has subsided in recent years due to the inability to resolve certain inherent problems in the Dataflow architectural model. To understand, this, the Dataflow concept is summarized below.

'''Summary of the Dataflow Idea'''

In its essence, a Dataflow architecture executes instructions based on whether the input arguments to the instructions are available. There is no program counter as present in a Von Neumann computer. To indicate the dependency of instructions, tags were used. When the tags contained simple memory addresses, the design was a static Dataflow machine. However, static designs couldn’t allow for multiple instances of a routine.

Dynamic Dataflow machines used Content-Addressable memory (CAM)(i.e. the tags were stored in memory) to solve this problem.
The programs were loaded in the CAM. When the tagged operands of an instruction became available, the CAM would send it to an '''execution unit'''. After execution, the output data and it’s tags were sent back to the CAM (as a '''data token'''). The CAM would then execute the next instruction whose dependencies had been satisfied.

Since, the CAM could identify instructions whose tags were not dependant on any unexecuted instruction, parallelization was possible.
There were however, major problems related to -

1) broadcasting the data tokens in a massively parallel system efficiently.

2) dispatching instruction tokens in a massively parallel system efficiently.

3) A real program had a huge number of dependencies. Building a CAM large enough for this proved difficult.

Due to these issues, new developments in this area were largely stagnant.

'''New Developments in the Dataflow Architectural Model'''

(Referenced from: http://en.wikipedia.org/wiki/Out-of-order_execution)
1) A subset of the Dataflow model - '''Out of order execution''' is widely used in present day architecture. It uses the conventional Von Neumann Architecture to run '''Execution Windows''' in the usual order. Within the Window however, instructions are run according to the Dataflow Paradigm thus enabling parallelization and efficient utilization of CPU cycles.

(Referenced form: http://wavescalar.cs.washington.edu/)
2)The WaveScalar Instruction Set Architecture and Execution Model, currently being developed at the University of Washington attempts at building an architectural model that can work with current Imperative languages.

(Referenced from: http://portal.acm.org/citation.cfm?id=1032450 & http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?tp=&arnumber=1332593&isnumber=29428)

3)Calstrom and Boden propose a '''packet instruction set computer (PISC) architecture''' which would employ dataflow architecture for network processors. They detailed a 40 gb/s network processor using Dataflow architecture.

(Referenced from: http://ieeexplore.ieee.org/Xplore/login.jsp?url=/iel2/3036/8629/00379757.pdf?isnumber=&arnumber=379757)
4) Data flow architecture is also being evaluated for DSP processors as outlined in the referenced paper by Lee et al. Their proposed design was for a static Dataflow processor with 9 parallel processors for the multi-standard video codec processor.

== Systolic Architecture ==

'''Emergence of Systolic Arrays'''

During the 1980s, tremendous technological advances were being made in the field of VLSI circuits, which led to the birth of a several different concepts of highly parallel computing systems. One particular approach to such parallelism was using Systolic arrays, which constituted of a high number of processing elements known as cells, which were interconnected for local communication. This model utilizes a pipe-network arrangement of data processing units. Whenever a data object arrives, an operation is triggered in a data processing unit, unlike a central processing unit which has program counters. Several data streams are sent and received by a systolic array and several data counters are required to propagate data streams. This simple methodology facilitates parallelism.

'''Definition, Function and Applications'''

They are called systolic arrays; because data, similar to human blood circulation, circulates within the array and also may interact with other data. Some results that have been computed in certain cells are pumped into new cells as input, similar to the heart (Hence the analogy to a systolic mechanism of parallelism). H. T. Kung and Charles E. Leiserson invented the concept of systolic arrays being utilized as agents for parallel processing.

The Systolic Architecture is a mesh-like, interconnected set of processing elements, or cells, which interact with each other and pass on results that become inputs for the next computational time-step. Such proximity of high performance processors provide a highly required space efficiency, which is usually the main factor in parallel architecture. The structure of such systolic arrays may be non linear and multi-directional pathways may exist between processing elements. Even though this architecture is quite similar to Single-Instruction-Multiple-Data, unlike SIMD, each processing element is capable of processing a unique, different operation. The main feature of systolic architecture is that its highly specialized computations can be performed on simple parallel processors and localized communication patterns.

The systolic array is an architectural concept that is highly suited for very high integration technology. Communication between these mesh-connected processors occurs only amongst neighbors. Both control and data flow are local. Each processor has a communication register that can be read by any of its neighbors.

A Systolic array is a network of processing elements that has features like

• Modularity : each systolic array element is a module

• Synchrony : a global clock times the data through the network

• Locality (both spatial and temporal) : cells have local communication and a unit time delay

• Regularity : processing elements are homogeneously interconnected

• Parallel Computing and Pipelinability : architecture achieves high speed

The two main Systolic Architectural Features are

• Balancing Computation with I/O.

• Concurrency and Communication.

But other notable features of the architecture are its cost effectiveness, its regular & simple design, the modularity in arrays is quite high, the number of processors working simultaneously is high and also the local communication is fast – making overall execution time much faster.

''Types of Systolic Arrays''

1. Single Dimensional Input/Output Linear Array
- This is usually utilized for single input/output
[[Image:Systolic01.jpg]]

2. Two Dimensional Input/Output Linear Array
- This allows for more input/output and control over a linear array
[[Image:Systolic02.jpg]]

3. Perimeter Input/Output Planar Array
- This topology allows input/output through its boundary or perimeter cells only
[[Image:Systolic03.jpg]]

4. Three Dimensional Input/Output Focal Plane Array
- This facilitates input/output to each and every systolic cell in the array
[[Image:Systolic04.jpg]]

A few applications of Systolic Arrays
• Matrix Multiplication
• Solution of linear equations to problems in image processing
• Computer graphics
• Cryptography
• Computer tomography
• Matrix Inversion and Decomposition.
• Convolution
• Polynomial Evaluation
• Lattice filters for seismic and speech signal processing
• Artificial Neural Networks

For a complete explanation on the invention of systolic arrays, utilize the following link : http://www.patentstorm.us/patents/4727503-fulltext.html

'''New Developments'''

''iWarp''

In 1989, the Intel Corporation launched the iWarp as a product and produced more than 1500 iWarp systems. The underlying architecture was based on systolic arrays and data streams being utilized for sending and receiving data between elements on the system. Even though the system was well suited for several applications, Intel stopped actively marketing the product in the mid-90s. It is still a useful tool for research in the parallel computing direction and it was one of the first successful steps towards true parallelism in the early ages of parallel computing.

''Image Processing and Morphology''

A two-dimensional systolic array consisting of cells can be implemented, based on parallel modules which provide internal pipeline operation where every cell can be configured according to a control word. Also, the array can be provided with a group of image buffers, that can reduce the number of accesses to data memory may also extend the array capabilities, allowing the possibility of chaining interconnection of multiple processing blocks.

''Digital Signal Processors''

Low cost, systolic architectural microprocessors utilizes a DSP systolic array, which makes it possible to build low cost ICs capable of providing high speed digital video services. This requires high processing capabilities, and parallelism is a requirement in such areas. Systolic architectures are better suited for such purposes as they are of much lower cost due to their space-efficiency and interconnectivity.

''Tilera Tile64''

Arranged in a two dimensional grid, the Tilera 64 is a tile that has 64 processing elements or cells which are interconnected through a five way switch and it provides a two level hierarchical cache. The architecture is completely based on systolic arrays and this sixty-four element multi-core processor is being released in September 2007. It is already predicted to be utilized extensively in the field of embedded systems for advanced networking, unified thread management, digital video, web services, etc.

'''References'''

1. (Reference: Based on ideas presented in the Wikipedia article at:http://en.wikipedia.org/wiki/Systolic_array)

2. Systolic Algorithms: Concepts, Synthesis and Evolution - Siang W.Song, University of Sau Paulo, Department Of Computer Science
http://citeseer.ist.psu.edu/cache/papers/cs/30734/http:zSzzSzwww.ime.usp.brzSz~songzSzpaperszSzcimpa.pdf/systolic-algorithms-concepts-synthesis.pdf

3. Instruction Systolic Array (ISA) - H.W. Lang, FH Flensburg
http://www.iti.fh-flensburg.de/lang/papers/isa/isa1.htm

4. Systolic Arrays : Patent Description
http://www.patentstorm.us/patents/4727503-fulltext.html

5. iWarp
http://www.cs.cmu.edu/~iwarp/

6. Tilera Tile64
http://www.tilera.com/products/processors.php

7. FPGA-Based Customizable Systolic Architecture for Image Processing Applications : Proceedings of the 2005 International Conference on Reconfigurable Computing and FPGAs (ReConFig'05) on Reconfigurable Computing and FPGAs - Griselda Saldana, Miguel Arias-Estrada (IEEE Computer Society)

8. Summary of Systolic Architectures (Slides)
http://web.cecs.pdx.edu/~mperkows/temp/May13/070-Systolic-Processors.pdf

File:Systolic04.JPG

2007-09-06T02:37:20Z

Arangan:

File:Systolic03.JPG

2007-09-06T02:37:04Z

Arangan:

File:Systolic02.JPG

2007-09-06T02:36:55Z

Arangan: