https://wiki.expertiza.ncsu.edu/api.php?action=feedcontributions&feedformat=atom&user=Schakra8Expertiza_Wiki - User contributions [en]2026-07-01T22:51:38ZUser contributionsMediaWiki 1.41.0https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/9b_sc&diff=74692CSC/ECE 506 Spring 2013/9b sc2013-04-04T00:14:22Z<p>Schakra8: /* Combining Tree Barrier */</p>
<hr />
<div>= Instruction and Previous Work =<br />
Changes are made as per the instructions given in https://docs.google.com/a/ncsu.edu/document/d/18ap34rWJImZjMAtGbANK9lZMon4NgwqDb5UY9Z8ksLM/edit<br />
<br />
Previous Version can be found here: http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_506_Spring_2011/ch9_ms<br />
<br />
= Synchronization =<br />
In addition to a proper cache coherency model, it is important for a multiprocessor system to provide support for synchronization. This must be provided at the hardware level, and can also be implemented to some degree in software. The most common types of hardware synchronization are locks and barriers, which are discussed in this chapter.<br />
<br />
== Hardware vs. Operating System Synchronization ==<br />
Synchronizations are characterized by three parts: acquire, wait, and release. Both hardware and operating system synchronization rely on atomic instructions for acquire and release, but the implementations for the wait portion of a synchronization can vary. Hardware implementations usually use busy-waiting, where a process repeatedly tests a variable waiting for it to change. The operating system can wait using blocking, where the process suspends itself until woken up by another process.<br />
<br />
Blocking has the advantage of freeing up the processor for use by other processes, but requires operating system functions to work and thus has a lot of overhead. Busy-waiting has much less overhead, but uses processor and cache resources while waiting. Because of these tradeoffs, it is usually better to use busy-waiting for short wait periods, and blocking for long wait periods.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
= Lock Implementations =<br />
<br />
Locks are an important concept when programming for multiprocessor systems. The purpose of a lock is to protect the code inside the lock, known as a critical section. It must be certain that while some thread X has entered the critical section, another thread Y is not also inside of the critical section and possibly modifying critical values. While thread X has entered the critical section, thread Y must wait until X has exited before entering. This can be accomplished in a variety of ways of varying complexity and performance.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these four criteria:<br />
<br />
*Acquisition Latency - How much time does it take to acquire the lock?<br />
*Traffic - How much bus traffic is generated by threads attempting to acquire the lock?<br />
*Fairness - FIFO vs. Luck<br />
*Storage - How much storage is needed compared to the number of threads?<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Acquisition Latency ===<br />
We want a low acquisition latency, especially for applications that repeatedly acquire and release locks. Over the run time of a program, the acquisition latency compounded many times could have a large performance affect. However, low acquisition latency has to be balanced against other factors when implementing a lock.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
Traffic is an important consideration when evaluating lock performance. If acquiring lock causes a lot of bus traffic, it will not scale well as the number of threads increases. Eventually the bus will become choked.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Fairness ===<br />
In general, threads should acquire a lock in them same order that the locks were requested. If this does not happen, as the number of threads increases so does the chance that a thread will become starved.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Storage ===<br />
The amount of storage needed per lock should be small enough such that it scales to a large number of threads without any problems. If a large amount of storage is required, then multiple threads could cause locks to consume too much memory.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
<br />
Two common lock implementations are test&set and test and test&set. In terms of performance, both have a low storage cost (scalable) and are not fair. Test&set has a low acquisition latency but is not scalable due do high bus traffic when there is a lot of contention for the lock. Test-and-test&set has a higher acquisition latency but scales better because it generates less bus traffic. The next figure shows performance comparisons for these two lock implementations.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
[[Image:lockperf.png|frame|none|Performance comparison of test&set and test-and-test&set (spin on read)<sup><span id="3body">[[#3foot|[3]]]</span></sup>]]<br />
<br />
=== Improving performance ===<br />
Optimizing software can improve the performance of locks. For instance, inserting a delay between a processor noticing a release and then trying to acquire a lock can reduce contention and bus traffic, and increase the performance of the locks. TCP-like back off algorithms can be used to adjust this delay depending on the number of threads contending for the lock.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
Another way to improve performance is to insert a delay between memory references so as to limit the bandwidth (traffic) each processor can use for spinning.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
One way to guarantee fairness is to queue lock attempts in shared memory so that they happen in order.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
== Atomic Instructions ==<br />
<br />
Since a multiprocessor system cannot disable interrupts as an effective method to execute code atomically, there must be hardware support for atomic operations.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
The ability to execute an atomic instruction is a requirement for most lock implementations. It is important that when a processor attempts to set a lock, either the lock is fully set and the thread is able to enter into the critical section, or the lock is not set and it appears that none of the instructions required to set the lock have executed. <br />
<br />
In the x86 instruction set the opcode CMPXCHG (compare and exchange) can be used in a lock implementation in order to guarantee atomicity. This function works by sending a destination and a source. The accumulator is compared to the destination and, if they are equal, loaded with the source. If they are not equal the accumulator is loaded with the destination value. In order to assure that this is executed atomically the opcode must be issued with the LOCK prefix. This is useful in implementing some locks, such as ticket locks.<sup><span id="5body">[[#5foot|[5]]]</span></sup> The syntax for the instruction is LOCK CMPXCHG SOURCE DESTINATION. Here, the source has to be a register while the destination can be either Register or Memory. An example of LOCK CMPXCHG is implementation in dis-assembly of Linux kernel's rtc_cmos_read(). The CPU locks needs to be copied and and then updated to a non zero value if the lock is to be applied. But there is a possibility when the value got copied earlier and changed later. This causes a race condition. Hence,it helps in briefing the busy wait loop by preventing race condition. The pseudo code for same is given as below<sup><span id="10body">[[#10foot|[10]]]</span></sup>:<br />
<br />
spin: # see if the lock-variable is clear<br />
mov cmos_lock, %eax<br />
test %eax, %eax<br />
jnz spin<br />
# ok, now we try to grab the lock<br />
lock cmpxchg %edx, cmos_lock<br />
# did another CPU grab it first? <br />
test %eax, %eax<br />
jnz spin<br />
<br />
== Hand-off Lock ==<br />
<br />
Another type of lock that is not discussed in the text is known as the "hand-off" lock. In this lock the first thread acquires the lock if no other thread is currently locked (since it is the first thread). When another thread attempts to gain the lock it will see that the lock is in use and adds itself to the queue. Once done this thread can sleep until called by the thread with the lock. Once the thread in the lock is finished, it will pass the lock to the next thread in the queue<sup><span id="6body">[[#6foot|[6]]]</span></sup>. Waiting thread gives up processor so that other threads (e.g. the thread with the lock) can run more quickly. Someone wakes up thread when the lock is free. The separation of ready queue from that of waiting queue fastens the process of execution. In the below example, the lock is given to a waiting thread by the unlock method. The interrupts are enabled only when the thread is placed in waiting queue and disabled once its given the lock. <br />
<br />
lock () {<br />
disable interrupts<br />
if (value == FREE) {<br />
value = BUSY<br />
} else {<br />
add thread to queue of threads waiting for<br />
this lock<br />
switch to next runnable thread<br />
}<br />
enable interrupts<br />
}<br />
unlock () {<br />
disable interrupts<br />
value = FREE<br />
if (any thread is waiting for this lock) {<br />
move waiting thread from waiting queue to<br />
ready queue<br />
value = BUSY<br />
}<br />
enable interrupts<br />
} <br />
<br />
But this fails if interrupt happens after thread enable interrupts Lock() adds thread to wait queue Lock() enables interrupts Interrupts causes pre-emption, i.e. switch to another thread. Pre-emption moves thread to ready queue. Now thread is on two queues (wait and ready)! Also, switch is likely to be a critical section Adding thread to wait queue and switching to next thread must be atomic. Solution to this problem is waiting thread leaves interrupts disabled when it calls switch. Next thread to run has the responsibility of re-enabling interrupts before returning to user code. When waiting thread wakes up, it returns from switch with interrupts disabled (from the last thread)<sup><span id="11body">[[#11foot|[11]]]</span></sup><br />
<br />
== Avoiding Locks ==<br />
<br />
There are many reasons why a programmer should attempt to write programs in such a way as to avoid locks if possible. There are many problems that can arise with the use of locks.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
One of the must well known issues is deadlock. This can occur when the threads are waiting to acquire the lock, but the lock will never be unlocked. For example, there are two processes A and B and they need two methods (1 and 2) to be grabbed for completion of a task which are shared by both A and B. This can create a deadlock when A grabs 1 and B grabs 2. This reaches to a situation where both A and B cannot change their states. The solution to this problem is to set a condition where in A process can lock method 2 only if t has locked method 1.<br />
<br />
Another problem with using locks is that the performance is not optimal, as often a lock is used when there is only a chance of conflict. This approach to programming yields slower performance than what might be possible with other methods. This also leads to questions of granularity, that is how much of the code should be protected under the critical section. The programmer must decide between many small (fine grain) locks or fewer, more encompassing locks. This decision can greatly effect the performance of the program.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
To avoid locks, lock free programming techniques like compare and swap and sequential consistency can be followed. This pattern typically involves copying a shared variable to a local variable, performing some speculative work, and attempting to publish the changes using CAS<sup><span id="12body">[[#12foot|[12]]]</span></sup>:<br />
<br />
void LockFreeQueue::push(Node* newHead)<br />
{<br />
for (;;)<br />
{<br />
// Copy a shared variable (m_Head) to a local.<br />
Node* oldHead = m_Head;<br />
<br />
// Do some speculative work, not yet visible to other threads.<br />
newHead->next = oldHead;<br />
<br />
// Next, attempt to publish our changes to the shared variable.<br />
// If the shared variable hasn't changed, the CAS succeeds and we return.<br />
// Otherwise, repeat.<br />
if (_InterlockedCompareExchange(&m_Head, newHead, oldHead) == oldHead)<br />
return;<br />
}<br />
}<br />
The above code basically follows the Read-Write-Update order and makes sure that each thread has its local copy to work on and gets it updated sequentially so that the next thread has the correct value. If the value is not correct, write fails<br />
<br />
= Barrier Implementations =<br />
<br />
In many ways, barriers are simpler than locks. A barrier is simply a point in a program where one or more threads must reach before the parallel program is allowed to continue. When using barriers, a programmer does not have to be concerned with advanced topics such as fairness, that are required when programming for locks.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these two criteria:<br />
<br />
*Latency - The time required to enter and exit a barrier.<br />
*Traffic - Communications overhead required by the barrier.<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Latency ===<br />
<br />
Ideally the latency of a barrier should be small, but for barriers that don't scale well latency can increase as the number of threads increases.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
<br />
Low traffic is also good, as we do not want excess buss traffic to prevent scalability.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
{| class="wikitable" border="1"<br />
|+Performance Comparison of Barrier Implementations<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
|-<br />
! Criteria<br />
! Sense Reversal<br />
Centralized Barrier<br />
! Combining Tree<br />
Barrier<br />
! Barrier Network<br />
(Hardware)<br />
|-<br />
| Latency<br />
| O(1)<br />
| O(log p)<br />
| O(log p)<br />
|-<br />
| Traffic<br />
| O(p<sup>2</sup>)<br />
| O(p)<br />
| moved to a separate network<br />
|}<br />
<br />
== Sense-Reversal Barrier ==<br />
<br />
This barrier is a centralized barrier where a single count variable protected by a lock is shared among all the threads. Each thread on reaching the barrier increments its value and waits till the value of variable has reached the number of threads for which barrier was implemented.Since all the threads are spinning around a single variable the miss rate is scales quadratically with number of processors.<br />
<br />
[[Image:Ch9wiki01.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
A sense-reversing barrier is a more elegant and practical solution to the problem of reusing barriers. A phase’s sense is a Boolean value: true for even-numbered phases and false otherwise. Each SenseBarrier object has a Boolean sense field indicating the sense of the currently executing phase. Each thread keeps its current sense as a thread-local object. Initially the barrier’s sense is the complement of the local sense of all the threads. When a thread calls await(), it checks whether it is the last thread to decrement the counter. If so, it reverses the barrier’s sense and continues. Otherwise, it spins waiting for the balancer’s sense field to change to match its own local sense.<sup><span id="13body">[[#13foot|[13]]]</span></sup><br />
<br />
struct barrier {<br />
shared int count; <br />
// a Fetch&Inc/Dec() object with initial value n <br />
// this object supports also read and write<br />
boolean sense;<br />
// initially FALSE;<br />
boolean mysense[n];<br />
// initially, psense[i] = TRUE,<br />
// for each 1 ≤ i ≤ n.<br />
};<br />
void await(struct barrier *B) { <br />
// code for process pi<br />
int position = Get&Dec(B->count); <br />
if (position == 1) {<br />
B->count = n;<br />
B->sense = B->mysense[i]; <br />
}<br />
else {<br />
while (B->sense!= B->mysense[i]) <br />
noop;<br />
}<br />
B->mysense[i] = 1-B->mysense[i];<br />
}<br />
<br />
== Combining Tree Barrier ==<br />
<br />
This barrier is a distributed barrier where group of processors form clusters and updates value of a local variable. The local variable on reaching a value equal to number of threads updating it, proceeds to increment another variable higher in hierarchy in the combining tree. When the variable in the highest level of hierarchy in the combining tree reaches its max value it is considered that all the threads have reached the barrier and synchronization is complete.<br />
<br />
Since in this case all the threads are updated local variables in form of smaller groups the miss rate is not as high as sense-reversal barrier.<br />
The diagram below shows the operation of combining tree barrier with threads grouped in two groups. The variables C0 and C1 are local to each group and C2 is the variable that is at higher level of hierarchy in the tree.<br />
<br />
[[Image:Ch9wiki02.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
Another example in form of pseudo code is given below:<br />
<br />
procedure combining_barrier <br />
combining_barrier_aux(mynode) // join the barrier <br />
sense := not sense // for next barrier <br />
procedure combining_barrier_aux(nodepointer : ^node) <br />
with nodepointer^ do <br />
if fetch_and_decrement(&count) = 1 // last to reach this node <br />
if parent != nil <br />
combining_barrier_aux(parent) <br />
count := k // prepare for next barrier<br />
nodesense := not nodesense // release waiting processors<br />
repeat until nodesense = sense<br />
<br />
Here, each processor starts at a leaf node of the tree and the leaf count value is decreased. The last descendant to reach each node in tree gets to continue further. The processor reaching the root wakes up and retraces its path through tree and unblocks siblings at each node along path.<br />
<br />
== Tournament Barrier ==<br />
<br />
In tournament barrier the threads are considered to be leaves at the end of a binary tree and each node represents a flag. Two threads compete with each other and the loser thread is allowed to set the flag and move to higher level and compete to lose with the loser thread from other section of binary tree. Thus the thread which completes last is able to set the highest flag in the binary tree. On setting the flag it indicates to all the threads that barrier has been completed and thus synchronization is achieved.<br />
<br />
[[Image:Ch9wiki03.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
An example of tournament barrier is given as follows:<br />
<br />
void await(boolean mySense) {<br />
if (top) {<br />
return;<br />
} else if (parent != null) {<br />
while (flag != mySense) {};<br />
parent.await(mySense);<br />
partner.flag = mySense;<br />
} else {<br />
partner.flag = mySense;<br />
while (flag != mySense) {};<br />
}}}<br />
Here, the parent variable stores the value i.e. null if not a winner else a winner, and then waits for a partner (represented by flag) and then performs synchronization. Otherwise a natural loser is chosen and partnered with current thread.<br />
<br />
== Disseminating Barrier ==<br />
<br />
In this barrier each thread maintains a record of the activity of other threads. For every round i with n threads, thread A notifies thread (A + 2i) mod n. Thus after logn rounds all the threads are aware of the status of every other thread running and whether it has reached the barrier. Overall synchronization is achieved by implication from a carefully chosen sequence of pairwise synchronizations. <br />
<br />
[[Image:Ch9wiki04.png]]<sup><span id="9body">[[#9foot|[9]]]</span></sup><br />
<br />
An example of this type of barrier can be found below:<br />
<br />
for [s = 0 to stages-1] { ## there will be ceiling(log_2 p) stages<br />
work out who my out-going partner is at stage s;<br />
<await (arrive[partner][s] == 0);><br />
arrive[partner][s] = 1;<br />
<await (arrive[myid][s] == 1);><br />
arrive[myid][s] = 0;<br />
}<br />
Here, each thread xecutes the same code, choosing partners for the pairwise synchs as a function of its own identifier and the internal<br />
iteration.Our own arrive flag is now set by an “in-coming” partner, who is distinct from our “out-going” partner (except when p=2).<br />
<br />
= References = <br />
<span id="1foot">[[#1body|1.]]</span> David Culler, J.P. Singh and Anoop Gupta. Parallel Computer Architecture: A Hardware/Software Approach. Morgan Kaufmann. August 1998.<br><br />
<span id="2foot">[[#2body|2.]]</span> Yan Solihin. Fundamentals of Parallel Computer Architecture: Multichip and Multicore Systems. Solihin Books. August 2009.<br><br />
<span id="3foot">[[#3body|3.]]</span> Thomas E. Anderson. The Performance of Spin Lock Alternatives for Shared-memory Multiprocessors. http://www.cc.gatech.edu/classes/AY2010/cs4210_fall/papers/anderson-spinlock.pdf<br><br />
<span id="4foot">[[#4body|4.]]</span> http://www.statemaster.com/encyclopedia/Lock-(computer-science)<br><br />
<span id="5foot">[[#5body|5.]]</span> http://faydoc.tripod.com/cpu/cmpxchg.htm<br><br />
<span id="6foot">[[#6body|6.]]</span> http://www.cs.duke.edu/courses/fall09/cps110/slides/threads3.ppt<br><br />
<span id="7foot">[[#7body|7.]]</span> Scalability evaluation of barrier algorithms for OpenMP. http://www2.cs.uh.edu/~hpctools/pub/iwomp-barrier.pdf<br><br />
<span id="8foot">[[#8body|8.]]</span> Carwyn Ball and Mark Bull. Barrier Synchronization in Java. http://www.ukhec.ac.uk/publications/reports/synch_java.pdf<br><br />
<span id="9foot">[[#9body|9.]]</span> http://www.cs.brown.edu/courses/cs176/barrier.ppt<br><br />
<span id="10foot">[[#10body|10.]]</span>http://heather.cs.ucdavis.edu/~matloff/50/PLN/lock.pdf<br><br />
<span id="11foot">[[#11body|11.]]</span>http://www.cs.duke.edu/courses/fall09/cps110/handouts/threads3.pdf<br><br />
<span id="12foot">[[#12body|12.]]</span>http://preshing.com/20120612/an-introduction-to-lock-free-programming<br><br />
<span id="13foot">[[#13body|13.]]</span>http://my.safaribooksonline.com/book/software-engineering-and-development/9780123705914/barriers/ch17lev1sec3#X2ludGVybmFsX0h0bWxWaWV3P3htbGlkPTk3ODAxMjM3MDU5MTQlMkZjaDE3bGV2MXNlYzMmcXVlcnk9<br></div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/9b_sc&diff=74687CSC/ECE 506 Spring 2013/9b sc2013-04-04T00:08:36Z<p>Schakra8: /* Tournament Barrier */</p>
<hr />
<div>= Instruction and Previous Work =<br />
Changes are made as per the instructions given in https://docs.google.com/a/ncsu.edu/document/d/18ap34rWJImZjMAtGbANK9lZMon4NgwqDb5UY9Z8ksLM/edit<br />
<br />
Previous Version can be found here: http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_506_Spring_2011/ch9_ms<br />
<br />
= Synchronization =<br />
In addition to a proper cache coherency model, it is important for a multiprocessor system to provide support for synchronization. This must be provided at the hardware level, and can also be implemented to some degree in software. The most common types of hardware synchronization are locks and barriers, which are discussed in this chapter.<br />
<br />
== Hardware vs. Operating System Synchronization ==<br />
Synchronizations are characterized by three parts: acquire, wait, and release. Both hardware and operating system synchronization rely on atomic instructions for acquire and release, but the implementations for the wait portion of a synchronization can vary. Hardware implementations usually use busy-waiting, where a process repeatedly tests a variable waiting for it to change. The operating system can wait using blocking, where the process suspends itself until woken up by another process.<br />
<br />
Blocking has the advantage of freeing up the processor for use by other processes, but requires operating system functions to work and thus has a lot of overhead. Busy-waiting has much less overhead, but uses processor and cache resources while waiting. Because of these tradeoffs, it is usually better to use busy-waiting for short wait periods, and blocking for long wait periods.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
= Lock Implementations =<br />
<br />
Locks are an important concept when programming for multiprocessor systems. The purpose of a lock is to protect the code inside the lock, known as a critical section. It must be certain that while some thread X has entered the critical section, another thread Y is not also inside of the critical section and possibly modifying critical values. While thread X has entered the critical section, thread Y must wait until X has exited before entering. This can be accomplished in a variety of ways of varying complexity and performance.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these four criteria:<br />
<br />
*Acquisition Latency - How much time does it take to acquire the lock?<br />
*Traffic - How much bus traffic is generated by threads attempting to acquire the lock?<br />
*Fairness - FIFO vs. Luck<br />
*Storage - How much storage is needed compared to the number of threads?<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Acquisition Latency ===<br />
We want a low acquisition latency, especially for applications that repeatedly acquire and release locks. Over the run time of a program, the acquisition latency compounded many times could have a large performance affect. However, low acquisition latency has to be balanced against other factors when implementing a lock.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
Traffic is an important consideration when evaluating lock performance. If acquiring lock causes a lot of bus traffic, it will not scale well as the number of threads increases. Eventually the bus will become choked.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Fairness ===<br />
In general, threads should acquire a lock in them same order that the locks were requested. If this does not happen, as the number of threads increases so does the chance that a thread will become starved.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Storage ===<br />
The amount of storage needed per lock should be small enough such that it scales to a large number of threads without any problems. If a large amount of storage is required, then multiple threads could cause locks to consume too much memory.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
<br />
Two common lock implementations are test&set and test and test&set. In terms of performance, both have a low storage cost (scalable) and are not fair. Test&set has a low acquisition latency but is not scalable due do high bus traffic when there is a lot of contention for the lock. Test-and-test&set has a higher acquisition latency but scales better because it generates less bus traffic. The next figure shows performance comparisons for these two lock implementations.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
[[Image:lockperf.png|frame|none|Performance comparison of test&set and test-and-test&set (spin on read)<sup><span id="3body">[[#3foot|[3]]]</span></sup>]]<br />
<br />
=== Improving performance ===<br />
Optimizing software can improve the performance of locks. For instance, inserting a delay between a processor noticing a release and then trying to acquire a lock can reduce contention and bus traffic, and increase the performance of the locks. TCP-like back off algorithms can be used to adjust this delay depending on the number of threads contending for the lock.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
Another way to improve performance is to insert a delay between memory references so as to limit the bandwidth (traffic) each processor can use for spinning.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
One way to guarantee fairness is to queue lock attempts in shared memory so that they happen in order.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
== Atomic Instructions ==<br />
<br />
Since a multiprocessor system cannot disable interrupts as an effective method to execute code atomically, there must be hardware support for atomic operations.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
The ability to execute an atomic instruction is a requirement for most lock implementations. It is important that when a processor attempts to set a lock, either the lock is fully set and the thread is able to enter into the critical section, or the lock is not set and it appears that none of the instructions required to set the lock have executed. <br />
<br />
In the x86 instruction set the opcode CMPXCHG (compare and exchange) can be used in a lock implementation in order to guarantee atomicity. This function works by sending a destination and a source. The accumulator is compared to the destination and, if they are equal, loaded with the source. If they are not equal the accumulator is loaded with the destination value. In order to assure that this is executed atomically the opcode must be issued with the LOCK prefix. This is useful in implementing some locks, such as ticket locks.<sup><span id="5body">[[#5foot|[5]]]</span></sup> The syntax for the instruction is LOCK CMPXCHG SOURCE DESTINATION. Here, the source has to be a register while the destination can be either Register or Memory. An example of LOCK CMPXCHG is implementation in dis-assembly of Linux kernel's rtc_cmos_read(). The CPU locks needs to be copied and and then updated to a non zero value if the lock is to be applied. But there is a possibility when the value got copied earlier and changed later. This causes a race condition. Hence,it helps in briefing the busy wait loop by preventing race condition. The pseudo code for same is given as below<sup><span id="10body">[[#10foot|[10]]]</span></sup>:<br />
<br />
spin: # see if the lock-variable is clear<br />
mov cmos_lock, %eax<br />
test %eax, %eax<br />
jnz spin<br />
# ok, now we try to grab the lock<br />
lock cmpxchg %edx, cmos_lock<br />
# did another CPU grab it first? <br />
test %eax, %eax<br />
jnz spin<br />
<br />
== Hand-off Lock ==<br />
<br />
Another type of lock that is not discussed in the text is known as the "hand-off" lock. In this lock the first thread acquires the lock if no other thread is currently locked (since it is the first thread). When another thread attempts to gain the lock it will see that the lock is in use and adds itself to the queue. Once done this thread can sleep until called by the thread with the lock. Once the thread in the lock is finished, it will pass the lock to the next thread in the queue<sup><span id="6body">[[#6foot|[6]]]</span></sup>. Waiting thread gives up processor so that other threads (e.g. the thread with the lock) can run more quickly. Someone wakes up thread when the lock is free. The separation of ready queue from that of waiting queue fastens the process of execution. In the below example, the lock is given to a waiting thread by the unlock method. The interrupts are enabled only when the thread is placed in waiting queue and disabled once its given the lock. <br />
<br />
lock () {<br />
disable interrupts<br />
if (value == FREE) {<br />
value = BUSY<br />
} else {<br />
add thread to queue of threads waiting for<br />
this lock<br />
switch to next runnable thread<br />
}<br />
enable interrupts<br />
}<br />
unlock () {<br />
disable interrupts<br />
value = FREE<br />
if (any thread is waiting for this lock) {<br />
move waiting thread from waiting queue to<br />
ready queue<br />
value = BUSY<br />
}<br />
enable interrupts<br />
} <br />
<br />
But this fails if interrupt happens after thread enable interrupts Lock() adds thread to wait queue Lock() enables interrupts Interrupts causes pre-emption, i.e. switch to another thread. Pre-emption moves thread to ready queue. Now thread is on two queues (wait and ready)! Also, switch is likely to be a critical section Adding thread to wait queue and switching to next thread must be atomic. Solution to this problem is waiting thread leaves interrupts disabled when it calls switch. Next thread to run has the responsibility of re-enabling interrupts before returning to user code. When waiting thread wakes up, it returns from switch with interrupts disabled (from the last thread)<sup><span id="11body">[[#11foot|[11]]]</span></sup><br />
<br />
== Avoiding Locks ==<br />
<br />
There are many reasons why a programmer should attempt to write programs in such a way as to avoid locks if possible. There are many problems that can arise with the use of locks.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
One of the must well known issues is deadlock. This can occur when the threads are waiting to acquire the lock, but the lock will never be unlocked. For example, there are two processes A and B and they need two methods (1 and 2) to be grabbed for completion of a task which are shared by both A and B. This can create a deadlock when A grabs 1 and B grabs 2. This reaches to a situation where both A and B cannot change their states. The solution to this problem is to set a condition where in A process can lock method 2 only if t has locked method 1.<br />
<br />
Another problem with using locks is that the performance is not optimal, as often a lock is used when there is only a chance of conflict. This approach to programming yields slower performance than what might be possible with other methods. This also leads to questions of granularity, that is how much of the code should be protected under the critical section. The programmer must decide between many small (fine grain) locks or fewer, more encompassing locks. This decision can greatly effect the performance of the program.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
To avoid locks, lock free programming techniques like compare and swap and sequential consistency can be followed. This pattern typically involves copying a shared variable to a local variable, performing some speculative work, and attempting to publish the changes using CAS<sup><span id="12body">[[#12foot|[12]]]</span></sup>:<br />
<br />
void LockFreeQueue::push(Node* newHead)<br />
{<br />
for (;;)<br />
{<br />
// Copy a shared variable (m_Head) to a local.<br />
Node* oldHead = m_Head;<br />
<br />
// Do some speculative work, not yet visible to other threads.<br />
newHead->next = oldHead;<br />
<br />
// Next, attempt to publish our changes to the shared variable.<br />
// If the shared variable hasn't changed, the CAS succeeds and we return.<br />
// Otherwise, repeat.<br />
if (_InterlockedCompareExchange(&m_Head, newHead, oldHead) == oldHead)<br />
return;<br />
}<br />
}<br />
The above code basically follows the Read-Write-Update order and makes sure that each thread has its local copy to work on and gets it updated sequentially so that the next thread has the correct value. If the value is not correct, write fails<br />
<br />
= Barrier Implementations =<br />
<br />
In many ways, barriers are simpler than locks. A barrier is simply a point in a program where one or more threads must reach before the parallel program is allowed to continue. When using barriers, a programmer does not have to be concerned with advanced topics such as fairness, that are required when programming for locks.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these two criteria:<br />
<br />
*Latency - The time required to enter and exit a barrier.<br />
*Traffic - Communications overhead required by the barrier.<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Latency ===<br />
<br />
Ideally the latency of a barrier should be small, but for barriers that don't scale well latency can increase as the number of threads increases.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
<br />
Low traffic is also good, as we do not want excess buss traffic to prevent scalability.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
{| class="wikitable" border="1"<br />
|+Performance Comparison of Barrier Implementations<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
|-<br />
! Criteria<br />
! Sense Reversal<br />
Centralized Barrier<br />
! Combining Tree<br />
Barrier<br />
! Barrier Network<br />
(Hardware)<br />
|-<br />
| Latency<br />
| O(1)<br />
| O(log p)<br />
| O(log p)<br />
|-<br />
| Traffic<br />
| O(p<sup>2</sup>)<br />
| O(p)<br />
| moved to a separate network<br />
|}<br />
<br />
== Sense-Reversal Barrier ==<br />
<br />
This barrier is a centralized barrier where a single count variable protected by a lock is shared among all the threads. Each thread on reaching the barrier increments its value and waits till the value of variable has reached the number of threads for which barrier was implemented.Since all the threads are spinning around a single variable the miss rate is scales quadratically with number of processors.<br />
<br />
[[Image:Ch9wiki01.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
A sense-reversing barrier is a more elegant and practical solution to the problem of reusing barriers. A phase’s sense is a Boolean value: true for even-numbered phases and false otherwise. Each SenseBarrier object has a Boolean sense field indicating the sense of the currently executing phase. Each thread keeps its current sense as a thread-local object. Initially the barrier’s sense is the complement of the local sense of all the threads. When a thread calls await(), it checks whether it is the last thread to decrement the counter. If so, it reverses the barrier’s sense and continues. Otherwise, it spins waiting for the balancer’s sense field to change to match its own local sense.<sup><span id="13body">[[#13foot|[13]]]</span></sup><br />
<br />
struct barrier {<br />
shared int count; <br />
// a Fetch&Inc/Dec() object with initial value n <br />
// this object supports also read and write<br />
boolean sense;<br />
// initially FALSE;<br />
boolean mysense[n];<br />
// initially, psense[i] = TRUE,<br />
// for each 1 ≤ i ≤ n.<br />
};<br />
void await(struct barrier *B) { <br />
// code for process pi<br />
int position = Get&Dec(B->count); <br />
if (position == 1) {<br />
B->count = n;<br />
B->sense = B->mysense[i]; <br />
}<br />
else {<br />
while (B->sense!= B->mysense[i]) <br />
noop;<br />
}<br />
B->mysense[i] = 1-B->mysense[i];<br />
}<br />
<br />
== Combining Tree Barrier ==<br />
<br />
This barrier is a distributed barrier where group of processors form clusters and updates value of a local variable. The local variable on reaching a value equal to number of threads updating it, proceeds to increment another variable higher in hierarchy in the combining tree. When the variable in the highest level of hierarchy in the combining tree reaches its max value it is considered that all the threads have reached the barrier and synchronization is complete.<br />
<br />
Since in this case all the threads are updated local variables in form of smaller groups the miss rate is not as high as sense-reversal barrier.<br />
<br />
The diagram below shows the operation of combining tree barrier with threads grouped in two groups. The variables C0 and C1 are local to each group and C2 is the variable that is at higher level of hierarchy in the tree.<br />
<br />
[[Image:Ch9wiki02.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Tournament Barrier ==<br />
<br />
In tournament barrier the threads are considered to be leaves at the end of a binary tree and each node represents a flag. Two threads compete with each other and the loser thread is allowed to set the flag and move to higher level and compete to lose with the loser thread from other section of binary tree. Thus the thread which completes last is able to set the highest flag in the binary tree. On setting the flag it indicates to all the threads that barrier has been completed and thus synchronization is achieved.<br />
<br />
[[Image:Ch9wiki03.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
An example of tournament barrier is given as follows:<br />
<br />
void await(boolean mySense) {<br />
if (top) {<br />
return;<br />
} else if (parent != null) {<br />
while (flag != mySense) {};<br />
parent.await(mySense);<br />
partner.flag = mySense;<br />
} else {<br />
partner.flag = mySense;<br />
while (flag != mySense) {};<br />
}}}<br />
Here, the parent variable stores the value i.e. null if not a winner else a winner, and then waits for a partner (represented by flag) and then performs synchronization. Otherwise a natural loser is chosen and partnered with current thread.<br />
<br />
== Disseminating Barrier ==<br />
<br />
In this barrier each thread maintains a record of the activity of other threads. For every round i with n threads, thread A notifies thread (A + 2i) mod n. Thus after logn rounds all the threads are aware of the status of every other thread running and whether it has reached the barrier. Overall synchronization is achieved by implication from a carefully chosen sequence of pairwise synchronizations. <br />
<br />
[[Image:Ch9wiki04.png]]<sup><span id="9body">[[#9foot|[9]]]</span></sup><br />
<br />
An example of this type of barrier can be found below:<br />
<br />
for [s = 0 to stages-1] { ## there will be ceiling(log_2 p) stages<br />
work out who my out-going partner is at stage s;<br />
<await (arrive[partner][s] == 0);><br />
arrive[partner][s] = 1;<br />
<await (arrive[myid][s] == 1);><br />
arrive[myid][s] = 0;<br />
}<br />
Here, each thread xecutes the same code, choosing partners for the pairwise synchs as a function of its own identifier and the internal<br />
iteration.Our own arrive flag is now set by an “in-coming” partner, who is distinct from our “out-going” partner (except when p=2).<br />
<br />
= References = <br />
<span id="1foot">[[#1body|1.]]</span> David Culler, J.P. Singh and Anoop Gupta. Parallel Computer Architecture: A Hardware/Software Approach. Morgan Kaufmann. August 1998.<br><br />
<span id="2foot">[[#2body|2.]]</span> Yan Solihin. Fundamentals of Parallel Computer Architecture: Multichip and Multicore Systems. Solihin Books. August 2009.<br><br />
<span id="3foot">[[#3body|3.]]</span> Thomas E. Anderson. The Performance of Spin Lock Alternatives for Shared-memory Multiprocessors. http://www.cc.gatech.edu/classes/AY2010/cs4210_fall/papers/anderson-spinlock.pdf<br><br />
<span id="4foot">[[#4body|4.]]</span> http://www.statemaster.com/encyclopedia/Lock-(computer-science)<br><br />
<span id="5foot">[[#5body|5.]]</span> http://faydoc.tripod.com/cpu/cmpxchg.htm<br><br />
<span id="6foot">[[#6body|6.]]</span> http://www.cs.duke.edu/courses/fall09/cps110/slides/threads3.ppt<br><br />
<span id="7foot">[[#7body|7.]]</span> Scalability evaluation of barrier algorithms for OpenMP. http://www2.cs.uh.edu/~hpctools/pub/iwomp-barrier.pdf<br><br />
<span id="8foot">[[#8body|8.]]</span> Carwyn Ball and Mark Bull. Barrier Synchronization in Java. http://www.ukhec.ac.uk/publications/reports/synch_java.pdf<br><br />
<span id="9foot">[[#9body|9.]]</span> http://www.cs.brown.edu/courses/cs176/barrier.ppt<br><br />
<span id="10foot">[[#10body|10.]]</span>http://heather.cs.ucdavis.edu/~matloff/50/PLN/lock.pdf<br><br />
<span id="11foot">[[#11body|11.]]</span>http://www.cs.duke.edu/courses/fall09/cps110/handouts/threads3.pdf<br><br />
<span id="12foot">[[#12body|12.]]</span>http://preshing.com/20120612/an-introduction-to-lock-free-programming<br><br />
<span id="13foot">[[#13body|13.]]</span>http://my.safaribooksonline.com/book/software-engineering-and-development/9780123705914/barriers/ch17lev1sec3#X2ludGVybmFsX0h0bWxWaWV3P3htbGlkPTk3ODAxMjM3MDU5MTQlMkZjaDE3bGV2MXNlYzMmcXVlcnk9<br></div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/9b_sc&diff=74682CSC/ECE 506 Spring 2013/9b sc2013-04-03T23:59:01Z<p>Schakra8: /* Disseminating Barrier */</p>
<hr />
<div>= Instruction and Previous Work =<br />
Changes are made as per the instructions given in https://docs.google.com/a/ncsu.edu/document/d/18ap34rWJImZjMAtGbANK9lZMon4NgwqDb5UY9Z8ksLM/edit<br />
<br />
Previous Version can be found here: http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_506_Spring_2011/ch9_ms<br />
<br />
= Synchronization =<br />
In addition to a proper cache coherency model, it is important for a multiprocessor system to provide support for synchronization. This must be provided at the hardware level, and can also be implemented to some degree in software. The most common types of hardware synchronization are locks and barriers, which are discussed in this chapter.<br />
<br />
== Hardware vs. Operating System Synchronization ==<br />
Synchronizations are characterized by three parts: acquire, wait, and release. Both hardware and operating system synchronization rely on atomic instructions for acquire and release, but the implementations for the wait portion of a synchronization can vary. Hardware implementations usually use busy-waiting, where a process repeatedly tests a variable waiting for it to change. The operating system can wait using blocking, where the process suspends itself until woken up by another process.<br />
<br />
Blocking has the advantage of freeing up the processor for use by other processes, but requires operating system functions to work and thus has a lot of overhead. Busy-waiting has much less overhead, but uses processor and cache resources while waiting. Because of these tradeoffs, it is usually better to use busy-waiting for short wait periods, and blocking for long wait periods.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
= Lock Implementations =<br />
<br />
Locks are an important concept when programming for multiprocessor systems. The purpose of a lock is to protect the code inside the lock, known as a critical section. It must be certain that while some thread X has entered the critical section, another thread Y is not also inside of the critical section and possibly modifying critical values. While thread X has entered the critical section, thread Y must wait until X has exited before entering. This can be accomplished in a variety of ways of varying complexity and performance.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these four criteria:<br />
<br />
*Acquisition Latency - How much time does it take to acquire the lock?<br />
*Traffic - How much bus traffic is generated by threads attempting to acquire the lock?<br />
*Fairness - FIFO vs. Luck<br />
*Storage - How much storage is needed compared to the number of threads?<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Acquisition Latency ===<br />
We want a low acquisition latency, especially for applications that repeatedly acquire and release locks. Over the run time of a program, the acquisition latency compounded many times could have a large performance affect. However, low acquisition latency has to be balanced against other factors when implementing a lock.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
Traffic is an important consideration when evaluating lock performance. If acquiring lock causes a lot of bus traffic, it will not scale well as the number of threads increases. Eventually the bus will become choked.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Fairness ===<br />
In general, threads should acquire a lock in them same order that the locks were requested. If this does not happen, as the number of threads increases so does the chance that a thread will become starved.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Storage ===<br />
The amount of storage needed per lock should be small enough such that it scales to a large number of threads without any problems. If a large amount of storage is required, then multiple threads could cause locks to consume too much memory.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
<br />
Two common lock implementations are test&set and test and test&set. In terms of performance, both have a low storage cost (scalable) and are not fair. Test&set has a low acquisition latency but is not scalable due do high bus traffic when there is a lot of contention for the lock. Test-and-test&set has a higher acquisition latency but scales better because it generates less bus traffic. The next figure shows performance comparisons for these two lock implementations.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
[[Image:lockperf.png|frame|none|Performance comparison of test&set and test-and-test&set (spin on read)<sup><span id="3body">[[#3foot|[3]]]</span></sup>]]<br />
<br />
=== Improving performance ===<br />
Optimizing software can improve the performance of locks. For instance, inserting a delay between a processor noticing a release and then trying to acquire a lock can reduce contention and bus traffic, and increase the performance of the locks. TCP-like back off algorithms can be used to adjust this delay depending on the number of threads contending for the lock.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
Another way to improve performance is to insert a delay between memory references so as to limit the bandwidth (traffic) each processor can use for spinning.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
One way to guarantee fairness is to queue lock attempts in shared memory so that they happen in order.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
== Atomic Instructions ==<br />
<br />
Since a multiprocessor system cannot disable interrupts as an effective method to execute code atomically, there must be hardware support for atomic operations.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
The ability to execute an atomic instruction is a requirement for most lock implementations. It is important that when a processor attempts to set a lock, either the lock is fully set and the thread is able to enter into the critical section, or the lock is not set and it appears that none of the instructions required to set the lock have executed. <br />
<br />
In the x86 instruction set the opcode CMPXCHG (compare and exchange) can be used in a lock implementation in order to guarantee atomicity. This function works by sending a destination and a source. The accumulator is compared to the destination and, if they are equal, loaded with the source. If they are not equal the accumulator is loaded with the destination value. In order to assure that this is executed atomically the opcode must be issued with the LOCK prefix. This is useful in implementing some locks, such as ticket locks.<sup><span id="5body">[[#5foot|[5]]]</span></sup> The syntax for the instruction is LOCK CMPXCHG SOURCE DESTINATION. Here, the source has to be a register while the destination can be either Register or Memory. An example of LOCK CMPXCHG is implementation in dis-assembly of Linux kernel's rtc_cmos_read(). The CPU locks needs to be copied and and then updated to a non zero value if the lock is to be applied. But there is a possibility when the value got copied earlier and changed later. This causes a race condition. Hence,it helps in briefing the busy wait loop by preventing race condition. The pseudo code for same is given as below<sup><span id="10body">[[#10foot|[10]]]</span></sup>:<br />
<br />
spin: # see if the lock-variable is clear<br />
mov cmos_lock, %eax<br />
test %eax, %eax<br />
jnz spin<br />
# ok, now we try to grab the lock<br />
lock cmpxchg %edx, cmos_lock<br />
# did another CPU grab it first? <br />
test %eax, %eax<br />
jnz spin<br />
<br />
== Hand-off Lock ==<br />
<br />
Another type of lock that is not discussed in the text is known as the "hand-off" lock. In this lock the first thread acquires the lock if no other thread is currently locked (since it is the first thread). When another thread attempts to gain the lock it will see that the lock is in use and adds itself to the queue. Once done this thread can sleep until called by the thread with the lock. Once the thread in the lock is finished, it will pass the lock to the next thread in the queue<sup><span id="6body">[[#6foot|[6]]]</span></sup>. Waiting thread gives up processor so that other threads (e.g. the thread with the lock) can run more quickly. Someone wakes up thread when the lock is free. The separation of ready queue from that of waiting queue fastens the process of execution. In the below example, the lock is given to a waiting thread by the unlock method. The interrupts are enabled only when the thread is placed in waiting queue and disabled once its given the lock. <br />
<br />
lock () {<br />
disable interrupts<br />
if (value == FREE) {<br />
value = BUSY<br />
} else {<br />
add thread to queue of threads waiting for<br />
this lock<br />
switch to next runnable thread<br />
}<br />
enable interrupts<br />
}<br />
unlock () {<br />
disable interrupts<br />
value = FREE<br />
if (any thread is waiting for this lock) {<br />
move waiting thread from waiting queue to<br />
ready queue<br />
value = BUSY<br />
}<br />
enable interrupts<br />
} <br />
<br />
But this fails if interrupt happens after thread enable interrupts Lock() adds thread to wait queue Lock() enables interrupts Interrupts causes pre-emption, i.e. switch to another thread. Pre-emption moves thread to ready queue. Now thread is on two queues (wait and ready)! Also, switch is likely to be a critical section Adding thread to wait queue and switching to next thread must be atomic. Solution to this problem is waiting thread leaves interrupts disabled when it calls switch. Next thread to run has the responsibility of re-enabling interrupts before returning to user code. When waiting thread wakes up, it returns from switch with interrupts disabled (from the last thread)<sup><span id="11body">[[#11foot|[11]]]</span></sup><br />
<br />
== Avoiding Locks ==<br />
<br />
There are many reasons why a programmer should attempt to write programs in such a way as to avoid locks if possible. There are many problems that can arise with the use of locks.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
One of the must well known issues is deadlock. This can occur when the threads are waiting to acquire the lock, but the lock will never be unlocked. For example, there are two processes A and B and they need two methods (1 and 2) to be grabbed for completion of a task which are shared by both A and B. This can create a deadlock when A grabs 1 and B grabs 2. This reaches to a situation where both A and B cannot change their states. The solution to this problem is to set a condition where in A process can lock method 2 only if t has locked method 1.<br />
<br />
Another problem with using locks is that the performance is not optimal, as often a lock is used when there is only a chance of conflict. This approach to programming yields slower performance than what might be possible with other methods. This also leads to questions of granularity, that is how much of the code should be protected under the critical section. The programmer must decide between many small (fine grain) locks or fewer, more encompassing locks. This decision can greatly effect the performance of the program.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
To avoid locks, lock free programming techniques like compare and swap and sequential consistency can be followed. This pattern typically involves copying a shared variable to a local variable, performing some speculative work, and attempting to publish the changes using CAS<sup><span id="12body">[[#12foot|[12]]]</span></sup>:<br />
<br />
void LockFreeQueue::push(Node* newHead)<br />
{<br />
for (;;)<br />
{<br />
// Copy a shared variable (m_Head) to a local.<br />
Node* oldHead = m_Head;<br />
<br />
// Do some speculative work, not yet visible to other threads.<br />
newHead->next = oldHead;<br />
<br />
// Next, attempt to publish our changes to the shared variable.<br />
// If the shared variable hasn't changed, the CAS succeeds and we return.<br />
// Otherwise, repeat.<br />
if (_InterlockedCompareExchange(&m_Head, newHead, oldHead) == oldHead)<br />
return;<br />
}<br />
}<br />
The above code basically follows the Read-Write-Update order and makes sure that each thread has its local copy to work on and gets it updated sequentially so that the next thread has the correct value. If the value is not correct, write fails<br />
<br />
= Barrier Implementations =<br />
<br />
In many ways, barriers are simpler than locks. A barrier is simply a point in a program where one or more threads must reach before the parallel program is allowed to continue. When using barriers, a programmer does not have to be concerned with advanced topics such as fairness, that are required when programming for locks.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these two criteria:<br />
<br />
*Latency - The time required to enter and exit a barrier.<br />
*Traffic - Communications overhead required by the barrier.<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Latency ===<br />
<br />
Ideally the latency of a barrier should be small, but for barriers that don't scale well latency can increase as the number of threads increases.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
<br />
Low traffic is also good, as we do not want excess buss traffic to prevent scalability.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
{| class="wikitable" border="1"<br />
|+Performance Comparison of Barrier Implementations<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
|-<br />
! Criteria<br />
! Sense Reversal<br />
Centralized Barrier<br />
! Combining Tree<br />
Barrier<br />
! Barrier Network<br />
(Hardware)<br />
|-<br />
| Latency<br />
| O(1)<br />
| O(log p)<br />
| O(log p)<br />
|-<br />
| Traffic<br />
| O(p<sup>2</sup>)<br />
| O(p)<br />
| moved to a separate network<br />
|}<br />
<br />
== Sense-Reversal Barrier ==<br />
<br />
This barrier is a centralized barrier where a single count variable protected by a lock is shared among all the threads. Each thread on reaching the barrier increments its value and waits till the value of variable has reached the number of threads for which barrier was implemented.Since all the threads are spinning around a single variable the miss rate is scales quadratically with number of processors.<br />
<br />
[[Image:Ch9wiki01.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
A sense-reversing barrier is a more elegant and practical solution to the problem of reusing barriers. A phase’s sense is a Boolean value: true for even-numbered phases and false otherwise. Each SenseBarrier object has a Boolean sense field indicating the sense of the currently executing phase. Each thread keeps its current sense as a thread-local object. Initially the barrier’s sense is the complement of the local sense of all the threads. When a thread calls await(), it checks whether it is the last thread to decrement the counter. If so, it reverses the barrier’s sense and continues. Otherwise, it spins waiting for the balancer’s sense field to change to match its own local sense.<sup><span id="13body">[[#13foot|[13]]]</span></sup><br />
<br />
struct barrier {<br />
shared int count; <br />
// a Fetch&Inc/Dec() object with initial value n <br />
// this object supports also read and write<br />
boolean sense;<br />
// initially FALSE;<br />
boolean mysense[n];<br />
// initially, psense[i] = TRUE,<br />
// for each 1 ≤ i ≤ n.<br />
};<br />
void await(struct barrier *B) { <br />
// code for process pi<br />
int position = Get&Dec(B->count); <br />
if (position == 1) {<br />
B->count = n;<br />
B->sense = B->mysense[i]; <br />
}<br />
else {<br />
while (B->sense!= B->mysense[i]) <br />
noop;<br />
}<br />
B->mysense[i] = 1-B->mysense[i];<br />
}<br />
<br />
== Combining Tree Barrier ==<br />
<br />
This barrier is a distributed barrier where group of processors form clusters and updates value of a local variable. The local variable on reaching a value equal to number of threads updating it, proceeds to increment another variable higher in hierarchy in the combining tree. When the variable in the highest level of hierarchy in the combining tree reaches its max value it is considered that all the threads have reached the barrier and synchronization is complete.<br />
<br />
Since in this case all the threads are updated local variables in form of smaller groups the miss rate is not as high as sense-reversal barrier.<br />
<br />
The diagram below shows the operation of combining tree barrier with threads grouped in two groups. The variables C0 and C1 are local to each group and C2 is the variable that is at higher level of hierarchy in the tree.<br />
<br />
[[Image:Ch9wiki02.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Tournament Barrier ==<br />
<br />
In tournament barrier the threads are considered to be leaves at the end of a binary tree and each node represents a flag. Two threads compete with each other and the loser thread is allowed to set the flag and move to higher level and compete to lose with the loser thread from other section of binary tree. Thus the thread which completes last is able to set the highest flag in the binary tree. On setting the flag it indicates to all the threads that barrier has been completed and thus synchronization is achieved.<br />
<br />
[[Image:Ch9wiki03.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Disseminating Barrier ==<br />
<br />
In this barrier each thread maintains a record of the activity of other threads. For every round i with n threads, thread A notifies thread (A + 2i) mod n. Thus after logn rounds all the threads are aware of the status of every other thread running and whether it has reached the barrier. Overall synchronization is achieved by implication from a carefully chosen sequence of pairwise synchronizations. <br />
<br />
[[Image:Ch9wiki04.png]]<sup><span id="9body">[[#9foot|[9]]]</span></sup><br />
<br />
An example of this type of barrier can be found below:<br />
<br />
for [s = 0 to stages-1] { ## there will be ceiling(log_2 p) stages<br />
work out who my out-going partner is at stage s;<br />
<await (arrive[partner][s] == 0);><br />
arrive[partner][s] = 1;<br />
<await (arrive[myid][s] == 1);><br />
arrive[myid][s] = 0;<br />
}<br />
Here, each thread xecutes the same code, choosing partners for the pairwise synchs as a function of its own identifier and the internal<br />
iteration.Our own arrive flag is now set by an “in-coming” partner, who is distinct from our “out-going” partner (except when p=2).<br />
<br />
= References = <br />
<span id="1foot">[[#1body|1.]]</span> David Culler, J.P. Singh and Anoop Gupta. Parallel Computer Architecture: A Hardware/Software Approach. Morgan Kaufmann. August 1998.<br><br />
<span id="2foot">[[#2body|2.]]</span> Yan Solihin. Fundamentals of Parallel Computer Architecture: Multichip and Multicore Systems. Solihin Books. August 2009.<br><br />
<span id="3foot">[[#3body|3.]]</span> Thomas E. Anderson. The Performance of Spin Lock Alternatives for Shared-memory Multiprocessors. http://www.cc.gatech.edu/classes/AY2010/cs4210_fall/papers/anderson-spinlock.pdf<br><br />
<span id="4foot">[[#4body|4.]]</span> http://www.statemaster.com/encyclopedia/Lock-(computer-science)<br><br />
<span id="5foot">[[#5body|5.]]</span> http://faydoc.tripod.com/cpu/cmpxchg.htm<br><br />
<span id="6foot">[[#6body|6.]]</span> http://www.cs.duke.edu/courses/fall09/cps110/slides/threads3.ppt<br><br />
<span id="7foot">[[#7body|7.]]</span> Scalability evaluation of barrier algorithms for OpenMP. http://www2.cs.uh.edu/~hpctools/pub/iwomp-barrier.pdf<br><br />
<span id="8foot">[[#8body|8.]]</span> Carwyn Ball and Mark Bull. Barrier Synchronization in Java. http://www.ukhec.ac.uk/publications/reports/synch_java.pdf<br><br />
<span id="9foot">[[#9body|9.]]</span> http://www.cs.brown.edu/courses/cs176/barrier.ppt<br><br />
<span id="10foot">[[#10body|10.]]</span>http://heather.cs.ucdavis.edu/~matloff/50/PLN/lock.pdf<br><br />
<span id="11foot">[[#11body|11.]]</span>http://www.cs.duke.edu/courses/fall09/cps110/handouts/threads3.pdf<br><br />
<span id="12foot">[[#12body|12.]]</span>http://preshing.com/20120612/an-introduction-to-lock-free-programming<br><br />
<span id="13foot">[[#13body|13.]]</span>http://my.safaribooksonline.com/book/software-engineering-and-development/9780123705914/barriers/ch17lev1sec3#X2ludGVybmFsX0h0bWxWaWV3P3htbGlkPTk3ODAxMjM3MDU5MTQlMkZjaDE3bGV2MXNlYzMmcXVlcnk9<br></div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/9b_sc&diff=74680CSC/ECE 506 Spring 2013/9b sc2013-04-03T23:56:07Z<p>Schakra8: /* References */</p>
<hr />
<div>= Instruction and Previous Work =<br />
Changes are made as per the instructions given in https://docs.google.com/a/ncsu.edu/document/d/18ap34rWJImZjMAtGbANK9lZMon4NgwqDb5UY9Z8ksLM/edit<br />
<br />
Previous Version can be found here: http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_506_Spring_2011/ch9_ms<br />
<br />
= Synchronization =<br />
In addition to a proper cache coherency model, it is important for a multiprocessor system to provide support for synchronization. This must be provided at the hardware level, and can also be implemented to some degree in software. The most common types of hardware synchronization are locks and barriers, which are discussed in this chapter.<br />
<br />
== Hardware vs. Operating System Synchronization ==<br />
Synchronizations are characterized by three parts: acquire, wait, and release. Both hardware and operating system synchronization rely on atomic instructions for acquire and release, but the implementations for the wait portion of a synchronization can vary. Hardware implementations usually use busy-waiting, where a process repeatedly tests a variable waiting for it to change. The operating system can wait using blocking, where the process suspends itself until woken up by another process.<br />
<br />
Blocking has the advantage of freeing up the processor for use by other processes, but requires operating system functions to work and thus has a lot of overhead. Busy-waiting has much less overhead, but uses processor and cache resources while waiting. Because of these tradeoffs, it is usually better to use busy-waiting for short wait periods, and blocking for long wait periods.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
= Lock Implementations =<br />
<br />
Locks are an important concept when programming for multiprocessor systems. The purpose of a lock is to protect the code inside the lock, known as a critical section. It must be certain that while some thread X has entered the critical section, another thread Y is not also inside of the critical section and possibly modifying critical values. While thread X has entered the critical section, thread Y must wait until X has exited before entering. This can be accomplished in a variety of ways of varying complexity and performance.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these four criteria:<br />
<br />
*Acquisition Latency - How much time does it take to acquire the lock?<br />
*Traffic - How much bus traffic is generated by threads attempting to acquire the lock?<br />
*Fairness - FIFO vs. Luck<br />
*Storage - How much storage is needed compared to the number of threads?<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Acquisition Latency ===<br />
We want a low acquisition latency, especially for applications that repeatedly acquire and release locks. Over the run time of a program, the acquisition latency compounded many times could have a large performance affect. However, low acquisition latency has to be balanced against other factors when implementing a lock.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
Traffic is an important consideration when evaluating lock performance. If acquiring lock causes a lot of bus traffic, it will not scale well as the number of threads increases. Eventually the bus will become choked.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Fairness ===<br />
In general, threads should acquire a lock in them same order that the locks were requested. If this does not happen, as the number of threads increases so does the chance that a thread will become starved.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Storage ===<br />
The amount of storage needed per lock should be small enough such that it scales to a large number of threads without any problems. If a large amount of storage is required, then multiple threads could cause locks to consume too much memory.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
<br />
Two common lock implementations are test&set and test and test&set. In terms of performance, both have a low storage cost (scalable) and are not fair. Test&set has a low acquisition latency but is not scalable due do high bus traffic when there is a lot of contention for the lock. Test-and-test&set has a higher acquisition latency but scales better because it generates less bus traffic. The next figure shows performance comparisons for these two lock implementations.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
[[Image:lockperf.png|frame|none|Performance comparison of test&set and test-and-test&set (spin on read)<sup><span id="3body">[[#3foot|[3]]]</span></sup>]]<br />
<br />
=== Improving performance ===<br />
Optimizing software can improve the performance of locks. For instance, inserting a delay between a processor noticing a release and then trying to acquire a lock can reduce contention and bus traffic, and increase the performance of the locks. TCP-like back off algorithms can be used to adjust this delay depending on the number of threads contending for the lock.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
Another way to improve performance is to insert a delay between memory references so as to limit the bandwidth (traffic) each processor can use for spinning.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
One way to guarantee fairness is to queue lock attempts in shared memory so that they happen in order.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
== Atomic Instructions ==<br />
<br />
Since a multiprocessor system cannot disable interrupts as an effective method to execute code atomically, there must be hardware support for atomic operations.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
The ability to execute an atomic instruction is a requirement for most lock implementations. It is important that when a processor attempts to set a lock, either the lock is fully set and the thread is able to enter into the critical section, or the lock is not set and it appears that none of the instructions required to set the lock have executed. <br />
<br />
In the x86 instruction set the opcode CMPXCHG (compare and exchange) can be used in a lock implementation in order to guarantee atomicity. This function works by sending a destination and a source. The accumulator is compared to the destination and, if they are equal, loaded with the source. If they are not equal the accumulator is loaded with the destination value. In order to assure that this is executed atomically the opcode must be issued with the LOCK prefix. This is useful in implementing some locks, such as ticket locks.<sup><span id="5body">[[#5foot|[5]]]</span></sup> The syntax for the instruction is LOCK CMPXCHG SOURCE DESTINATION. Here, the source has to be a register while the destination can be either Register or Memory. An example of LOCK CMPXCHG is implementation in dis-assembly of Linux kernel's rtc_cmos_read(). The CPU locks needs to be copied and and then updated to a non zero value if the lock is to be applied. But there is a possibility when the value got copied earlier and changed later. This causes a race condition. Hence,it helps in briefing the busy wait loop by preventing race condition. The pseudo code for same is given as below<sup><span id="10body">[[#10foot|[10]]]</span></sup>:<br />
<br />
spin: # see if the lock-variable is clear<br />
mov cmos_lock, %eax<br />
test %eax, %eax<br />
jnz spin<br />
# ok, now we try to grab the lock<br />
lock cmpxchg %edx, cmos_lock<br />
# did another CPU grab it first? <br />
test %eax, %eax<br />
jnz spin<br />
<br />
== Hand-off Lock ==<br />
<br />
Another type of lock that is not discussed in the text is known as the "hand-off" lock. In this lock the first thread acquires the lock if no other thread is currently locked (since it is the first thread). When another thread attempts to gain the lock it will see that the lock is in use and adds itself to the queue. Once done this thread can sleep until called by the thread with the lock. Once the thread in the lock is finished, it will pass the lock to the next thread in the queue<sup><span id="6body">[[#6foot|[6]]]</span></sup>. Waiting thread gives up processor so that other threads (e.g. the thread with the lock) can run more quickly. Someone wakes up thread when the lock is free. The separation of ready queue from that of waiting queue fastens the process of execution. In the below example, the lock is given to a waiting thread by the unlock method. The interrupts are enabled only when the thread is placed in waiting queue and disabled once its given the lock. <br />
<br />
lock () {<br />
disable interrupts<br />
if (value == FREE) {<br />
value = BUSY<br />
} else {<br />
add thread to queue of threads waiting for<br />
this lock<br />
switch to next runnable thread<br />
}<br />
enable interrupts<br />
}<br />
unlock () {<br />
disable interrupts<br />
value = FREE<br />
if (any thread is waiting for this lock) {<br />
move waiting thread from waiting queue to<br />
ready queue<br />
value = BUSY<br />
}<br />
enable interrupts<br />
} <br />
<br />
But this fails if interrupt happens after thread enable interrupts Lock() adds thread to wait queue Lock() enables interrupts Interrupts causes pre-emption, i.e. switch to another thread. Pre-emption moves thread to ready queue. Now thread is on two queues (wait and ready)! Also, switch is likely to be a critical section Adding thread to wait queue and switching to next thread must be atomic. Solution to this problem is waiting thread leaves interrupts disabled when it calls switch. Next thread to run has the responsibility of re-enabling interrupts before returning to user code. When waiting thread wakes up, it returns from switch with interrupts disabled (from the last thread)<sup><span id="11body">[[#11foot|[11]]]</span></sup><br />
<br />
== Avoiding Locks ==<br />
<br />
There are many reasons why a programmer should attempt to write programs in such a way as to avoid locks if possible. There are many problems that can arise with the use of locks.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
One of the must well known issues is deadlock. This can occur when the threads are waiting to acquire the lock, but the lock will never be unlocked. For example, there are two processes A and B and they need two methods (1 and 2) to be grabbed for completion of a task which are shared by both A and B. This can create a deadlock when A grabs 1 and B grabs 2. This reaches to a situation where both A and B cannot change their states. The solution to this problem is to set a condition where in A process can lock method 2 only if t has locked method 1.<br />
<br />
Another problem with using locks is that the performance is not optimal, as often a lock is used when there is only a chance of conflict. This approach to programming yields slower performance than what might be possible with other methods. This also leads to questions of granularity, that is how much of the code should be protected under the critical section. The programmer must decide between many small (fine grain) locks or fewer, more encompassing locks. This decision can greatly effect the performance of the program.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
To avoid locks, lock free programming techniques like compare and swap and sequential consistency can be followed. This pattern typically involves copying a shared variable to a local variable, performing some speculative work, and attempting to publish the changes using CAS<sup><span id="12body">[[#12foot|[12]]]</span></sup>:<br />
<br />
void LockFreeQueue::push(Node* newHead)<br />
{<br />
for (;;)<br />
{<br />
// Copy a shared variable (m_Head) to a local.<br />
Node* oldHead = m_Head;<br />
<br />
// Do some speculative work, not yet visible to other threads.<br />
newHead->next = oldHead;<br />
<br />
// Next, attempt to publish our changes to the shared variable.<br />
// If the shared variable hasn't changed, the CAS succeeds and we return.<br />
// Otherwise, repeat.<br />
if (_InterlockedCompareExchange(&m_Head, newHead, oldHead) == oldHead)<br />
return;<br />
}<br />
}<br />
The above code basically follows the Read-Write-Update order and makes sure that each thread has its local copy to work on and gets it updated sequentially so that the next thread has the correct value. If the value is not correct, write fails<br />
<br />
= Barrier Implementations =<br />
<br />
In many ways, barriers are simpler than locks. A barrier is simply a point in a program where one or more threads must reach before the parallel program is allowed to continue. When using barriers, a programmer does not have to be concerned with advanced topics such as fairness, that are required when programming for locks.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these two criteria:<br />
<br />
*Latency - The time required to enter and exit a barrier.<br />
*Traffic - Communications overhead required by the barrier.<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Latency ===<br />
<br />
Ideally the latency of a barrier should be small, but for barriers that don't scale well latency can increase as the number of threads increases.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
<br />
Low traffic is also good, as we do not want excess buss traffic to prevent scalability.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
{| class="wikitable" border="1"<br />
|+Performance Comparison of Barrier Implementations<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
|-<br />
! Criteria<br />
! Sense Reversal<br />
Centralized Barrier<br />
! Combining Tree<br />
Barrier<br />
! Barrier Network<br />
(Hardware)<br />
|-<br />
| Latency<br />
| O(1)<br />
| O(log p)<br />
| O(log p)<br />
|-<br />
| Traffic<br />
| O(p<sup>2</sup>)<br />
| O(p)<br />
| moved to a separate network<br />
|}<br />
<br />
== Sense-Reversal Barrier ==<br />
<br />
This barrier is a centralized barrier where a single count variable protected by a lock is shared among all the threads. Each thread on reaching the barrier increments its value and waits till the value of variable has reached the number of threads for which barrier was implemented.Since all the threads are spinning around a single variable the miss rate is scales quadratically with number of processors.<br />
<br />
[[Image:Ch9wiki01.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
A sense-reversing barrier is a more elegant and practical solution to the problem of reusing barriers. A phase’s sense is a Boolean value: true for even-numbered phases and false otherwise. Each SenseBarrier object has a Boolean sense field indicating the sense of the currently executing phase. Each thread keeps its current sense as a thread-local object. Initially the barrier’s sense is the complement of the local sense of all the threads. When a thread calls await(), it checks whether it is the last thread to decrement the counter. If so, it reverses the barrier’s sense and continues. Otherwise, it spins waiting for the balancer’s sense field to change to match its own local sense.<sup><span id="13body">[[#13foot|[13]]]</span></sup><br />
<br />
struct barrier {<br />
shared int count; <br />
// a Fetch&Inc/Dec() object with initial value n <br />
// this object supports also read and write<br />
boolean sense;<br />
// initially FALSE;<br />
boolean mysense[n];<br />
// initially, psense[i] = TRUE,<br />
// for each 1 ≤ i ≤ n.<br />
};<br />
void await(struct barrier *B) { <br />
// code for process pi<br />
int position = Get&Dec(B->count); <br />
if (position == 1) {<br />
B->count = n;<br />
B->sense = B->mysense[i]; <br />
}<br />
else {<br />
while (B->sense!= B->mysense[i]) <br />
noop;<br />
}<br />
B->mysense[i] = 1-B->mysense[i];<br />
}<br />
<br />
== Combining Tree Barrier ==<br />
<br />
This barrier is a distributed barrier where group of processors form clusters and updates value of a local variable. The local variable on reaching a value equal to number of threads updating it, proceeds to increment another variable higher in hierarchy in the combining tree. When the variable in the highest level of hierarchy in the combining tree reaches its max value it is considered that all the threads have reached the barrier and synchronization is complete.<br />
<br />
Since in this case all the threads are updated local variables in form of smaller groups the miss rate is not as high as sense-reversal barrier.<br />
<br />
The diagram below shows the operation of combining tree barrier with threads grouped in two groups. The variables C0 and C1 are local to each group and C2 is the variable that is at higher level of hierarchy in the tree.<br />
<br />
[[Image:Ch9wiki02.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Tournament Barrier ==<br />
<br />
In tournament barrier the threads are considered to be leaves at the end of a binary tree and each node represents a flag. Two threads compete with each other and the loser thread is allowed to set the flag and move to higher level and compete to lose with the loser thread from other section of binary tree. Thus the thread which completes last is able to set the highest flag in the binary tree. On setting the flag it indicates to all the threads that barrier has been completed and thus synchronization is achieved.<br />
<br />
[[Image:Ch9wiki03.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Disseminating Barrier ==<br />
<br />
In this barrier each thread maintains a record of the activity of other threads. For every round i with n threads, thread A notifies thread (A + 2i) mod n. Thus after logn rounds all the threads are aware of the status of every other thread running and whether it has reached the barrier.<br />
<br />
[[Image:Ch9wiki04.png]]<sup><span id="9body">[[#9foot|[9]]]</span></sup><br />
<br />
= References = <br />
<span id="1foot">[[#1body|1.]]</span> David Culler, J.P. Singh and Anoop Gupta. Parallel Computer Architecture: A Hardware/Software Approach. Morgan Kaufmann. August 1998.<br><br />
<span id="2foot">[[#2body|2.]]</span> Yan Solihin. Fundamentals of Parallel Computer Architecture: Multichip and Multicore Systems. Solihin Books. August 2009.<br><br />
<span id="3foot">[[#3body|3.]]</span> Thomas E. Anderson. The Performance of Spin Lock Alternatives for Shared-memory Multiprocessors. http://www.cc.gatech.edu/classes/AY2010/cs4210_fall/papers/anderson-spinlock.pdf<br><br />
<span id="4foot">[[#4body|4.]]</span> http://www.statemaster.com/encyclopedia/Lock-(computer-science)<br><br />
<span id="5foot">[[#5body|5.]]</span> http://faydoc.tripod.com/cpu/cmpxchg.htm<br><br />
<span id="6foot">[[#6body|6.]]</span> http://www.cs.duke.edu/courses/fall09/cps110/slides/threads3.ppt<br><br />
<span id="7foot">[[#7body|7.]]</span> Scalability evaluation of barrier algorithms for OpenMP. http://www2.cs.uh.edu/~hpctools/pub/iwomp-barrier.pdf<br><br />
<span id="8foot">[[#8body|8.]]</span> Carwyn Ball and Mark Bull. Barrier Synchronization in Java. http://www.ukhec.ac.uk/publications/reports/synch_java.pdf<br><br />
<span id="9foot">[[#9body|9.]]</span> http://www.cs.brown.edu/courses/cs176/barrier.ppt<br><br />
<span id="10foot">[[#10body|10.]]</span>http://heather.cs.ucdavis.edu/~matloff/50/PLN/lock.pdf<br><br />
<span id="11foot">[[#11body|11.]]</span>http://www.cs.duke.edu/courses/fall09/cps110/handouts/threads3.pdf<br><br />
<span id="12foot">[[#12body|12.]]</span>http://preshing.com/20120612/an-introduction-to-lock-free-programming<br><br />
<span id="13foot">[[#13body|13.]]</span>http://my.safaribooksonline.com/book/software-engineering-and-development/9780123705914/barriers/ch17lev1sec3#X2ludGVybmFsX0h0bWxWaWV3P3htbGlkPTk3ODAxMjM3MDU5MTQlMkZjaDE3bGV2MXNlYzMmcXVlcnk9<br></div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/9b_sc&diff=74679CSC/ECE 506 Spring 2013/9b sc2013-04-03T23:55:26Z<p>Schakra8: /* Sense-Reversal Barrier */</p>
<hr />
<div>= Instruction and Previous Work =<br />
Changes are made as per the instructions given in https://docs.google.com/a/ncsu.edu/document/d/18ap34rWJImZjMAtGbANK9lZMon4NgwqDb5UY9Z8ksLM/edit<br />
<br />
Previous Version can be found here: http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_506_Spring_2011/ch9_ms<br />
<br />
= Synchronization =<br />
In addition to a proper cache coherency model, it is important for a multiprocessor system to provide support for synchronization. This must be provided at the hardware level, and can also be implemented to some degree in software. The most common types of hardware synchronization are locks and barriers, which are discussed in this chapter.<br />
<br />
== Hardware vs. Operating System Synchronization ==<br />
Synchronizations are characterized by three parts: acquire, wait, and release. Both hardware and operating system synchronization rely on atomic instructions for acquire and release, but the implementations for the wait portion of a synchronization can vary. Hardware implementations usually use busy-waiting, where a process repeatedly tests a variable waiting for it to change. The operating system can wait using blocking, where the process suspends itself until woken up by another process.<br />
<br />
Blocking has the advantage of freeing up the processor for use by other processes, but requires operating system functions to work and thus has a lot of overhead. Busy-waiting has much less overhead, but uses processor and cache resources while waiting. Because of these tradeoffs, it is usually better to use busy-waiting for short wait periods, and blocking for long wait periods.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
= Lock Implementations =<br />
<br />
Locks are an important concept when programming for multiprocessor systems. The purpose of a lock is to protect the code inside the lock, known as a critical section. It must be certain that while some thread X has entered the critical section, another thread Y is not also inside of the critical section and possibly modifying critical values. While thread X has entered the critical section, thread Y must wait until X has exited before entering. This can be accomplished in a variety of ways of varying complexity and performance.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these four criteria:<br />
<br />
*Acquisition Latency - How much time does it take to acquire the lock?<br />
*Traffic - How much bus traffic is generated by threads attempting to acquire the lock?<br />
*Fairness - FIFO vs. Luck<br />
*Storage - How much storage is needed compared to the number of threads?<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Acquisition Latency ===<br />
We want a low acquisition latency, especially for applications that repeatedly acquire and release locks. Over the run time of a program, the acquisition latency compounded many times could have a large performance affect. However, low acquisition latency has to be balanced against other factors when implementing a lock.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
Traffic is an important consideration when evaluating lock performance. If acquiring lock causes a lot of bus traffic, it will not scale well as the number of threads increases. Eventually the bus will become choked.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Fairness ===<br />
In general, threads should acquire a lock in them same order that the locks were requested. If this does not happen, as the number of threads increases so does the chance that a thread will become starved.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Storage ===<br />
The amount of storage needed per lock should be small enough such that it scales to a large number of threads without any problems. If a large amount of storage is required, then multiple threads could cause locks to consume too much memory.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
<br />
Two common lock implementations are test&set and test and test&set. In terms of performance, both have a low storage cost (scalable) and are not fair. Test&set has a low acquisition latency but is not scalable due do high bus traffic when there is a lot of contention for the lock. Test-and-test&set has a higher acquisition latency but scales better because it generates less bus traffic. The next figure shows performance comparisons for these two lock implementations.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
[[Image:lockperf.png|frame|none|Performance comparison of test&set and test-and-test&set (spin on read)<sup><span id="3body">[[#3foot|[3]]]</span></sup>]]<br />
<br />
=== Improving performance ===<br />
Optimizing software can improve the performance of locks. For instance, inserting a delay between a processor noticing a release and then trying to acquire a lock can reduce contention and bus traffic, and increase the performance of the locks. TCP-like back off algorithms can be used to adjust this delay depending on the number of threads contending for the lock.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
Another way to improve performance is to insert a delay between memory references so as to limit the bandwidth (traffic) each processor can use for spinning.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
One way to guarantee fairness is to queue lock attempts in shared memory so that they happen in order.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
== Atomic Instructions ==<br />
<br />
Since a multiprocessor system cannot disable interrupts as an effective method to execute code atomically, there must be hardware support for atomic operations.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
The ability to execute an atomic instruction is a requirement for most lock implementations. It is important that when a processor attempts to set a lock, either the lock is fully set and the thread is able to enter into the critical section, or the lock is not set and it appears that none of the instructions required to set the lock have executed. <br />
<br />
In the x86 instruction set the opcode CMPXCHG (compare and exchange) can be used in a lock implementation in order to guarantee atomicity. This function works by sending a destination and a source. The accumulator is compared to the destination and, if they are equal, loaded with the source. If they are not equal the accumulator is loaded with the destination value. In order to assure that this is executed atomically the opcode must be issued with the LOCK prefix. This is useful in implementing some locks, such as ticket locks.<sup><span id="5body">[[#5foot|[5]]]</span></sup> The syntax for the instruction is LOCK CMPXCHG SOURCE DESTINATION. Here, the source has to be a register while the destination can be either Register or Memory. An example of LOCK CMPXCHG is implementation in dis-assembly of Linux kernel's rtc_cmos_read(). The CPU locks needs to be copied and and then updated to a non zero value if the lock is to be applied. But there is a possibility when the value got copied earlier and changed later. This causes a race condition. Hence,it helps in briefing the busy wait loop by preventing race condition. The pseudo code for same is given as below<sup><span id="10body">[[#10foot|[10]]]</span></sup>:<br />
<br />
spin: # see if the lock-variable is clear<br />
mov cmos_lock, %eax<br />
test %eax, %eax<br />
jnz spin<br />
# ok, now we try to grab the lock<br />
lock cmpxchg %edx, cmos_lock<br />
# did another CPU grab it first? <br />
test %eax, %eax<br />
jnz spin<br />
<br />
== Hand-off Lock ==<br />
<br />
Another type of lock that is not discussed in the text is known as the "hand-off" lock. In this lock the first thread acquires the lock if no other thread is currently locked (since it is the first thread). When another thread attempts to gain the lock it will see that the lock is in use and adds itself to the queue. Once done this thread can sleep until called by the thread with the lock. Once the thread in the lock is finished, it will pass the lock to the next thread in the queue<sup><span id="6body">[[#6foot|[6]]]</span></sup>. Waiting thread gives up processor so that other threads (e.g. the thread with the lock) can run more quickly. Someone wakes up thread when the lock is free. The separation of ready queue from that of waiting queue fastens the process of execution. In the below example, the lock is given to a waiting thread by the unlock method. The interrupts are enabled only when the thread is placed in waiting queue and disabled once its given the lock. <br />
<br />
lock () {<br />
disable interrupts<br />
if (value == FREE) {<br />
value = BUSY<br />
} else {<br />
add thread to queue of threads waiting for<br />
this lock<br />
switch to next runnable thread<br />
}<br />
enable interrupts<br />
}<br />
unlock () {<br />
disable interrupts<br />
value = FREE<br />
if (any thread is waiting for this lock) {<br />
move waiting thread from waiting queue to<br />
ready queue<br />
value = BUSY<br />
}<br />
enable interrupts<br />
} <br />
<br />
But this fails if interrupt happens after thread enable interrupts Lock() adds thread to wait queue Lock() enables interrupts Interrupts causes pre-emption, i.e. switch to another thread. Pre-emption moves thread to ready queue. Now thread is on two queues (wait and ready)! Also, switch is likely to be a critical section Adding thread to wait queue and switching to next thread must be atomic. Solution to this problem is waiting thread leaves interrupts disabled when it calls switch. Next thread to run has the responsibility of re-enabling interrupts before returning to user code. When waiting thread wakes up, it returns from switch with interrupts disabled (from the last thread)<sup><span id="11body">[[#11foot|[11]]]</span></sup><br />
<br />
== Avoiding Locks ==<br />
<br />
There are many reasons why a programmer should attempt to write programs in such a way as to avoid locks if possible. There are many problems that can arise with the use of locks.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
One of the must well known issues is deadlock. This can occur when the threads are waiting to acquire the lock, but the lock will never be unlocked. For example, there are two processes A and B and they need two methods (1 and 2) to be grabbed for completion of a task which are shared by both A and B. This can create a deadlock when A grabs 1 and B grabs 2. This reaches to a situation where both A and B cannot change their states. The solution to this problem is to set a condition where in A process can lock method 2 only if t has locked method 1.<br />
<br />
Another problem with using locks is that the performance is not optimal, as often a lock is used when there is only a chance of conflict. This approach to programming yields slower performance than what might be possible with other methods. This also leads to questions of granularity, that is how much of the code should be protected under the critical section. The programmer must decide between many small (fine grain) locks or fewer, more encompassing locks. This decision can greatly effect the performance of the program.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
To avoid locks, lock free programming techniques like compare and swap and sequential consistency can be followed. This pattern typically involves copying a shared variable to a local variable, performing some speculative work, and attempting to publish the changes using CAS<sup><span id="12body">[[#12foot|[12]]]</span></sup>:<br />
<br />
void LockFreeQueue::push(Node* newHead)<br />
{<br />
for (;;)<br />
{<br />
// Copy a shared variable (m_Head) to a local.<br />
Node* oldHead = m_Head;<br />
<br />
// Do some speculative work, not yet visible to other threads.<br />
newHead->next = oldHead;<br />
<br />
// Next, attempt to publish our changes to the shared variable.<br />
// If the shared variable hasn't changed, the CAS succeeds and we return.<br />
// Otherwise, repeat.<br />
if (_InterlockedCompareExchange(&m_Head, newHead, oldHead) == oldHead)<br />
return;<br />
}<br />
}<br />
The above code basically follows the Read-Write-Update order and makes sure that each thread has its local copy to work on and gets it updated sequentially so that the next thread has the correct value. If the value is not correct, write fails<br />
<br />
= Barrier Implementations =<br />
<br />
In many ways, barriers are simpler than locks. A barrier is simply a point in a program where one or more threads must reach before the parallel program is allowed to continue. When using barriers, a programmer does not have to be concerned with advanced topics such as fairness, that are required when programming for locks.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these two criteria:<br />
<br />
*Latency - The time required to enter and exit a barrier.<br />
*Traffic - Communications overhead required by the barrier.<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Latency ===<br />
<br />
Ideally the latency of a barrier should be small, but for barriers that don't scale well latency can increase as the number of threads increases.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
<br />
Low traffic is also good, as we do not want excess buss traffic to prevent scalability.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
{| class="wikitable" border="1"<br />
|+Performance Comparison of Barrier Implementations<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
|-<br />
! Criteria<br />
! Sense Reversal<br />
Centralized Barrier<br />
! Combining Tree<br />
Barrier<br />
! Barrier Network<br />
(Hardware)<br />
|-<br />
| Latency<br />
| O(1)<br />
| O(log p)<br />
| O(log p)<br />
|-<br />
| Traffic<br />
| O(p<sup>2</sup>)<br />
| O(p)<br />
| moved to a separate network<br />
|}<br />
<br />
== Sense-Reversal Barrier ==<br />
<br />
This barrier is a centralized barrier where a single count variable protected by a lock is shared among all the threads. Each thread on reaching the barrier increments its value and waits till the value of variable has reached the number of threads for which barrier was implemented.Since all the threads are spinning around a single variable the miss rate is scales quadratically with number of processors.<br />
<br />
[[Image:Ch9wiki01.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
A sense-reversing barrier is a more elegant and practical solution to the problem of reusing barriers. A phase’s sense is a Boolean value: true for even-numbered phases and false otherwise. Each SenseBarrier object has a Boolean sense field indicating the sense of the currently executing phase. Each thread keeps its current sense as a thread-local object. Initially the barrier’s sense is the complement of the local sense of all the threads. When a thread calls await(), it checks whether it is the last thread to decrement the counter. If so, it reverses the barrier’s sense and continues. Otherwise, it spins waiting for the balancer’s sense field to change to match its own local sense.<sup><span id="13body">[[#13foot|[13]]]</span></sup><br />
<br />
struct barrier {<br />
shared int count; <br />
// a Fetch&Inc/Dec() object with initial value n <br />
// this object supports also read and write<br />
boolean sense;<br />
// initially FALSE;<br />
boolean mysense[n];<br />
// initially, psense[i] = TRUE,<br />
// for each 1 ≤ i ≤ n.<br />
};<br />
void await(struct barrier *B) { <br />
// code for process pi<br />
int position = Get&Dec(B->count); <br />
if (position == 1) {<br />
B->count = n;<br />
B->sense = B->mysense[i]; <br />
}<br />
else {<br />
while (B->sense!= B->mysense[i]) <br />
noop;<br />
}<br />
B->mysense[i] = 1-B->mysense[i];<br />
}<br />
<br />
== Combining Tree Barrier ==<br />
<br />
This barrier is a distributed barrier where group of processors form clusters and updates value of a local variable. The local variable on reaching a value equal to number of threads updating it, proceeds to increment another variable higher in hierarchy in the combining tree. When the variable in the highest level of hierarchy in the combining tree reaches its max value it is considered that all the threads have reached the barrier and synchronization is complete.<br />
<br />
Since in this case all the threads are updated local variables in form of smaller groups the miss rate is not as high as sense-reversal barrier.<br />
<br />
The diagram below shows the operation of combining tree barrier with threads grouped in two groups. The variables C0 and C1 are local to each group and C2 is the variable that is at higher level of hierarchy in the tree.<br />
<br />
[[Image:Ch9wiki02.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Tournament Barrier ==<br />
<br />
In tournament barrier the threads are considered to be leaves at the end of a binary tree and each node represents a flag. Two threads compete with each other and the loser thread is allowed to set the flag and move to higher level and compete to lose with the loser thread from other section of binary tree. Thus the thread which completes last is able to set the highest flag in the binary tree. On setting the flag it indicates to all the threads that barrier has been completed and thus synchronization is achieved.<br />
<br />
[[Image:Ch9wiki03.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Disseminating Barrier ==<br />
<br />
In this barrier each thread maintains a record of the activity of other threads. For every round i with n threads, thread A notifies thread (A + 2i) mod n. Thus after logn rounds all the threads are aware of the status of every other thread running and whether it has reached the barrier.<br />
<br />
[[Image:Ch9wiki04.png]]<sup><span id="9body">[[#9foot|[9]]]</span></sup><br />
<br />
= References = <br />
<span id="1foot">[[#1body|1.]]</span> David Culler, J.P. Singh and Anoop Gupta. Parallel Computer Architecture: A Hardware/Software Approach. Morgan Kaufmann. August 1998.<br><br />
<span id="2foot">[[#2body|2.]]</span> Yan Solihin. Fundamentals of Parallel Computer Architecture: Multichip and Multicore Systems. Solihin Books. August 2009.<br><br />
<span id="3foot">[[#3body|3.]]</span> Thomas E. Anderson. The Performance of Spin Lock Alternatives for Shared-memory Multiprocessors. http://www.cc.gatech.edu/classes/AY2010/cs4210_fall/papers/anderson-spinlock.pdf<br><br />
<span id="4foot">[[#4body|4.]]</span> http://www.statemaster.com/encyclopedia/Lock-(computer-science)<br><br />
<span id="5foot">[[#5body|5.]]</span> http://faydoc.tripod.com/cpu/cmpxchg.htm<br><br />
<span id="6foot">[[#6body|6.]]</span> http://www.cs.duke.edu/courses/fall09/cps110/slides/threads3.ppt<br><br />
<span id="7foot">[[#7body|7.]]</span> Scalability evaluation of barrier algorithms for OpenMP. http://www2.cs.uh.edu/~hpctools/pub/iwomp-barrier.pdf<br><br />
<span id="8foot">[[#8body|8.]]</span> Carwyn Ball and Mark Bull. Barrier Synchronization in Java. http://www.ukhec.ac.uk/publications/reports/synch_java.pdf<br><br />
<span id="9foot">[[#9body|9.]]</span> http://www.cs.brown.edu/courses/cs176/barrier.ppt<br><br />
<span id="10foot">[[#10body|10.]]</span>http://heather.cs.ucdavis.edu/~matloff/50/PLN/lock.pdf<br><br />
<span id="11foot">[[#11body|11.]]</span>http://www.cs.duke.edu/courses/fall09/cps110/handouts/threads3.pdf<br><br />
<span id="12foot">[[#12body|12.]]</span>http://preshing.com/20120612/an-introduction-to-lock-free-programming<br></div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/9b_sc&diff=74674CSC/ECE 506 Spring 2013/9b sc2013-04-03T23:35:16Z<p>Schakra8: /* Performance Comparison */</p>
<hr />
<div>= Instruction and Previous Work =<br />
Changes are made as per the instructions given in https://docs.google.com/a/ncsu.edu/document/d/18ap34rWJImZjMAtGbANK9lZMon4NgwqDb5UY9Z8ksLM/edit<br />
<br />
Previous Version can be found here: http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_506_Spring_2011/ch9_ms<br />
<br />
= Synchronization =<br />
In addition to a proper cache coherency model, it is important for a multiprocessor system to provide support for synchronization. This must be provided at the hardware level, and can also be implemented to some degree in software. The most common types of hardware synchronization are locks and barriers, which are discussed in this chapter.<br />
<br />
== Hardware vs. Operating System Synchronization ==<br />
Synchronizations are characterized by three parts: acquire, wait, and release. Both hardware and operating system synchronization rely on atomic instructions for acquire and release, but the implementations for the wait portion of a synchronization can vary. Hardware implementations usually use busy-waiting, where a process repeatedly tests a variable waiting for it to change. The operating system can wait using blocking, where the process suspends itself until woken up by another process.<br />
<br />
Blocking has the advantage of freeing up the processor for use by other processes, but requires operating system functions to work and thus has a lot of overhead. Busy-waiting has much less overhead, but uses processor and cache resources while waiting. Because of these tradeoffs, it is usually better to use busy-waiting for short wait periods, and blocking for long wait periods.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
= Lock Implementations =<br />
<br />
Locks are an important concept when programming for multiprocessor systems. The purpose of a lock is to protect the code inside the lock, known as a critical section. It must be certain that while some thread X has entered the critical section, another thread Y is not also inside of the critical section and possibly modifying critical values. While thread X has entered the critical section, thread Y must wait until X has exited before entering. This can be accomplished in a variety of ways of varying complexity and performance.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these four criteria:<br />
<br />
*Acquisition Latency - How much time does it take to acquire the lock?<br />
*Traffic - How much bus traffic is generated by threads attempting to acquire the lock?<br />
*Fairness - FIFO vs. Luck<br />
*Storage - How much storage is needed compared to the number of threads?<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Acquisition Latency ===<br />
We want a low acquisition latency, especially for applications that repeatedly acquire and release locks. Over the run time of a program, the acquisition latency compounded many times could have a large performance affect. However, low acquisition latency has to be balanced against other factors when implementing a lock.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
Traffic is an important consideration when evaluating lock performance. If acquiring lock causes a lot of bus traffic, it will not scale well as the number of threads increases. Eventually the bus will become choked.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Fairness ===<br />
In general, threads should acquire a lock in them same order that the locks were requested. If this does not happen, as the number of threads increases so does the chance that a thread will become starved.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Storage ===<br />
The amount of storage needed per lock should be small enough such that it scales to a large number of threads without any problems. If a large amount of storage is required, then multiple threads could cause locks to consume too much memory.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
<br />
Two common lock implementations are test&set and test and test&set. In terms of performance, both have a low storage cost (scalable) and are not fair. Test&set has a low acquisition latency but is not scalable due do high bus traffic when there is a lot of contention for the lock. Test-and-test&set has a higher acquisition latency but scales better because it generates less bus traffic. The next figure shows performance comparisons for these two lock implementations.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
[[Image:lockperf.png|frame|none|Performance comparison of test&set and test-and-test&set (spin on read)<sup><span id="3body">[[#3foot|[3]]]</span></sup>]]<br />
<br />
=== Improving performance ===<br />
Optimizing software can improve the performance of locks. For instance, inserting a delay between a processor noticing a release and then trying to acquire a lock can reduce contention and bus traffic, and increase the performance of the locks. TCP-like back off algorithms can be used to adjust this delay depending on the number of threads contending for the lock.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
Another way to improve performance is to insert a delay between memory references so as to limit the bandwidth (traffic) each processor can use for spinning.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
One way to guarantee fairness is to queue lock attempts in shared memory so that they happen in order.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
== Atomic Instructions ==<br />
<br />
Since a multiprocessor system cannot disable interrupts as an effective method to execute code atomically, there must be hardware support for atomic operations.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
The ability to execute an atomic instruction is a requirement for most lock implementations. It is important that when a processor attempts to set a lock, either the lock is fully set and the thread is able to enter into the critical section, or the lock is not set and it appears that none of the instructions required to set the lock have executed. <br />
<br />
In the x86 instruction set the opcode CMPXCHG (compare and exchange) can be used in a lock implementation in order to guarantee atomicity. This function works by sending a destination and a source. The accumulator is compared to the destination and, if they are equal, loaded with the source. If they are not equal the accumulator is loaded with the destination value. In order to assure that this is executed atomically the opcode must be issued with the LOCK prefix. This is useful in implementing some locks, such as ticket locks.<sup><span id="5body">[[#5foot|[5]]]</span></sup> The syntax for the instruction is LOCK CMPXCHG SOURCE DESTINATION. Here, the source has to be a register while the destination can be either Register or Memory. An example of LOCK CMPXCHG is implementation in dis-assembly of Linux kernel's rtc_cmos_read(). The CPU locks needs to be copied and and then updated to a non zero value if the lock is to be applied. But there is a possibility when the value got copied earlier and changed later. This causes a race condition. Hence,it helps in briefing the busy wait loop by preventing race condition. The pseudo code for same is given as below<sup><span id="10body">[[#10foot|[10]]]</span></sup>:<br />
<br />
spin: # see if the lock-variable is clear<br />
mov cmos_lock, %eax<br />
test %eax, %eax<br />
jnz spin<br />
# ok, now we try to grab the lock<br />
lock cmpxchg %edx, cmos_lock<br />
# did another CPU grab it first? <br />
test %eax, %eax<br />
jnz spin<br />
<br />
== Hand-off Lock ==<br />
<br />
Another type of lock that is not discussed in the text is known as the "hand-off" lock. In this lock the first thread acquires the lock if no other thread is currently locked (since it is the first thread). When another thread attempts to gain the lock it will see that the lock is in use and adds itself to the queue. Once done this thread can sleep until called by the thread with the lock. Once the thread in the lock is finished, it will pass the lock to the next thread in the queue<sup><span id="6body">[[#6foot|[6]]]</span></sup>. Waiting thread gives up processor so that other threads (e.g. the thread with the lock) can run more quickly. Someone wakes up thread when the lock is free. The separation of ready queue from that of waiting queue fastens the process of execution. In the below example, the lock is given to a waiting thread by the unlock method. The interrupts are enabled only when the thread is placed in waiting queue and disabled once its given the lock. <br />
<br />
lock () {<br />
disable interrupts<br />
if (value == FREE) {<br />
value = BUSY<br />
} else {<br />
add thread to queue of threads waiting for<br />
this lock<br />
switch to next runnable thread<br />
}<br />
enable interrupts<br />
}<br />
unlock () {<br />
disable interrupts<br />
value = FREE<br />
if (any thread is waiting for this lock) {<br />
move waiting thread from waiting queue to<br />
ready queue<br />
value = BUSY<br />
}<br />
enable interrupts<br />
} <br />
<br />
But this fails if interrupt happens after thread enable interrupts Lock() adds thread to wait queue Lock() enables interrupts Interrupts causes pre-emption, i.e. switch to another thread. Pre-emption moves thread to ready queue. Now thread is on two queues (wait and ready)! Also, switch is likely to be a critical section Adding thread to wait queue and switching to next thread must be atomic. Solution to this problem is waiting thread leaves interrupts disabled when it calls switch. Next thread to run has the responsibility of re-enabling interrupts before returning to user code. When waiting thread wakes up, it returns from switch with interrupts disabled (from the last thread)<sup><span id="11body">[[#11foot|[11]]]</span></sup><br />
<br />
== Avoiding Locks ==<br />
<br />
There are many reasons why a programmer should attempt to write programs in such a way as to avoid locks if possible. There are many problems that can arise with the use of locks.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
One of the must well known issues is deadlock. This can occur when the threads are waiting to acquire the lock, but the lock will never be unlocked. For example, there are two processes A and B and they need two methods (1 and 2) to be grabbed for completion of a task which are shared by both A and B. This can create a deadlock when A grabs 1 and B grabs 2. This reaches to a situation where both A and B cannot change their states. The solution to this problem is to set a condition where in A process can lock method 2 only if t has locked method 1.<br />
<br />
Another problem with using locks is that the performance is not optimal, as often a lock is used when there is only a chance of conflict. This approach to programming yields slower performance than what might be possible with other methods. This also leads to questions of granularity, that is how much of the code should be protected under the critical section. The programmer must decide between many small (fine grain) locks or fewer, more encompassing locks. This decision can greatly effect the performance of the program.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
To avoid locks, lock free programming techniques like compare and swap and sequential consistency can be followed. This pattern typically involves copying a shared variable to a local variable, performing some speculative work, and attempting to publish the changes using CAS<sup><span id="12body">[[#12foot|[12]]]</span></sup>:<br />
<br />
void LockFreeQueue::push(Node* newHead)<br />
{<br />
for (;;)<br />
{<br />
// Copy a shared variable (m_Head) to a local.<br />
Node* oldHead = m_Head;<br />
<br />
// Do some speculative work, not yet visible to other threads.<br />
newHead->next = oldHead;<br />
<br />
// Next, attempt to publish our changes to the shared variable.<br />
// If the shared variable hasn't changed, the CAS succeeds and we return.<br />
// Otherwise, repeat.<br />
if (_InterlockedCompareExchange(&m_Head, newHead, oldHead) == oldHead)<br />
return;<br />
}<br />
}<br />
The above code basically follows the Read-Write-Update order and makes sure that each thread has its local copy to work on and gets it updated sequentially so that the next thread has the correct value. If the value is not correct, write fails<br />
<br />
= Barrier Implementations =<br />
<br />
In many ways, barriers are simpler than locks. A barrier is simply a point in a program where one or more threads must reach before the parallel program is allowed to continue. When using barriers, a programmer does not have to be concerned with advanced topics such as fairness, that are required when programming for locks.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these two criteria:<br />
<br />
*Latency - The time required to enter and exit a barrier.<br />
*Traffic - Communications overhead required by the barrier.<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Latency ===<br />
<br />
Ideally the latency of a barrier should be small, but for barriers that don't scale well latency can increase as the number of threads increases.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
<br />
Low traffic is also good, as we do not want excess buss traffic to prevent scalability.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
{| class="wikitable" border="1"<br />
|+Performance Comparison of Barrier Implementations<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
|-<br />
! Criteria<br />
! Sense Reversal<br />
Centralized Barrier<br />
! Combining Tree<br />
Barrier<br />
! Barrier Network<br />
(Hardware)<br />
|-<br />
| Latency<br />
| O(1)<br />
| O(log p)<br />
| O(log p)<br />
|-<br />
| Traffic<br />
| O(p<sup>2</sup>)<br />
| O(p)<br />
| moved to a separate network<br />
|}<br />
<br />
== Sense-Reversal Barrier ==<br />
<br />
This barrier is a centralized barrier where a single count variable protected by a lock is shared among all the threads. Each thread on reaching the barrier increments its value and waits till the value of variable has reached the number of threads for which barrier was implemented.<br />
<br />
Since all the threads are spinning around a single variable the miss rate is scales quadratically with number of processors.<br />
<br />
[[Image:Ch9wiki01.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Combining Tree Barrier ==<br />
<br />
This barrier is a distributed barrier where group of processors form clusters and updates value of a local variable. The local variable on reaching a value equal to number of threads updating it, proceeds to increment another variable higher in hierarchy in the combining tree. When the variable in the highest level of hierarchy in the combining tree reaches its max value it is considered that all the threads have reached the barrier and synchronization is complete.<br />
<br />
Since in this case all the threads are updated local variables in form of smaller groups the miss rate is not as high as sense-reversal barrier.<br />
<br />
The diagram below shows the operation of combining tree barrier with threads grouped in two groups. The variables C0 and C1 are local to each group and C2 is the variable that is at higher level of hierarchy in the tree.<br />
<br />
[[Image:Ch9wiki02.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Tournament Barrier ==<br />
<br />
In tournament barrier the threads are considered to be leaves at the end of a binary tree and each node represents a flag. Two threads compete with each other and the loser thread is allowed to set the flag and move to higher level and compete to lose with the loser thread from other section of binary tree. Thus the thread which completes last is able to set the highest flag in the binary tree. On setting the flag it indicates to all the threads that barrier has been completed and thus synchronization is achieved.<br />
<br />
[[Image:Ch9wiki03.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Disseminating Barrier ==<br />
<br />
In this barrier each thread maintains a record of the activity of other threads. For every round i with n threads, thread A notifies thread (A + 2i) mod n. Thus after logn rounds all the threads are aware of the status of every other thread running and whether it has reached the barrier.<br />
<br />
[[Image:Ch9wiki04.png]]<sup><span id="9body">[[#9foot|[9]]]</span></sup><br />
<br />
= References = <br />
<span id="1foot">[[#1body|1.]]</span> David Culler, J.P. Singh and Anoop Gupta. Parallel Computer Architecture: A Hardware/Software Approach. Morgan Kaufmann. August 1998.<br><br />
<span id="2foot">[[#2body|2.]]</span> Yan Solihin. Fundamentals of Parallel Computer Architecture: Multichip and Multicore Systems. Solihin Books. August 2009.<br><br />
<span id="3foot">[[#3body|3.]]</span> Thomas E. Anderson. The Performance of Spin Lock Alternatives for Shared-memory Multiprocessors. http://www.cc.gatech.edu/classes/AY2010/cs4210_fall/papers/anderson-spinlock.pdf<br><br />
<span id="4foot">[[#4body|4.]]</span> http://www.statemaster.com/encyclopedia/Lock-(computer-science)<br><br />
<span id="5foot">[[#5body|5.]]</span> http://faydoc.tripod.com/cpu/cmpxchg.htm<br><br />
<span id="6foot">[[#6body|6.]]</span> http://www.cs.duke.edu/courses/fall09/cps110/slides/threads3.ppt<br><br />
<span id="7foot">[[#7body|7.]]</span> Scalability evaluation of barrier algorithms for OpenMP. http://www2.cs.uh.edu/~hpctools/pub/iwomp-barrier.pdf<br><br />
<span id="8foot">[[#8body|8.]]</span> Carwyn Ball and Mark Bull. Barrier Synchronization in Java. http://www.ukhec.ac.uk/publications/reports/synch_java.pdf<br><br />
<span id="9foot">[[#9body|9.]]</span> http://www.cs.brown.edu/courses/cs176/barrier.ppt<br><br />
<span id="10foot">[[#10body|10.]]</span>http://heather.cs.ucdavis.edu/~matloff/50/PLN/lock.pdf<br><br />
<span id="11foot">[[#11body|11.]]</span>http://www.cs.duke.edu/courses/fall09/cps110/handouts/threads3.pdf<br><br />
<span id="12foot">[[#12body|12.]]</span>http://preshing.com/20120612/an-introduction-to-lock-free-programming<br></div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/9b_sc&diff=74673CSC/ECE 506 Spring 2013/9b sc2013-04-03T23:32:09Z<p>Schakra8: </p>
<hr />
<div>= Instruction and Previous Work =<br />
Changes are made as per the instructions given in https://docs.google.com/a/ncsu.edu/document/d/18ap34rWJImZjMAtGbANK9lZMon4NgwqDb5UY9Z8ksLM/edit<br />
<br />
Previous Version can be found here: http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_506_Spring_2011/ch9_ms<br />
<br />
= Synchronization =<br />
In addition to a proper cache coherency model, it is important for a multiprocessor system to provide support for synchronization. This must be provided at the hardware level, and can also be implemented to some degree in software. The most common types of hardware synchronization are locks and barriers, which are discussed in this chapter.<br />
<br />
== Hardware vs. Operating System Synchronization ==<br />
Synchronizations are characterized by three parts: acquire, wait, and release. Both hardware and operating system synchronization rely on atomic instructions for acquire and release, but the implementations for the wait portion of a synchronization can vary. Hardware implementations usually use busy-waiting, where a process repeatedly tests a variable waiting for it to change. The operating system can wait using blocking, where the process suspends itself until woken up by another process.<br />
<br />
Blocking has the advantage of freeing up the processor for use by other processes, but requires operating system functions to work and thus has a lot of overhead. Busy-waiting has much less overhead, but uses processor and cache resources while waiting. Because of these tradeoffs, it is usually better to use busy-waiting for short wait periods, and blocking for long wait periods.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
= Lock Implementations =<br />
<br />
Locks are an important concept when programming for multiprocessor systems. The purpose of a lock is to protect the code inside the lock, known as a critical section. It must be certain that while some thread X has entered the critical section, another thread Y is not also inside of the critical section and possibly modifying critical values. While thread X has entered the critical section, thread Y must wait until X has exited before entering. This can be accomplished in a variety of ways of varying complexity and performance.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these four criteria:<br />
<br />
*Acquisition Latency - How much time does it take to acquire the lock?<br />
*Traffic - How much bus traffic is generated by threads attempting to acquire the lock?<br />
*Fairness - FIFO vs. Luck<br />
*Storage - How much storage is needed compared to the number of threads?<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Acquisition Latency ===<br />
We want a low acquisition latency, especially for applications that repeatedly acquire and release locks. Over the run time of a program, the acquisition latency compounded many times could have a large performance affect. However, low acquisition latency has to be balanced against other factors when implementing a lock.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
Traffic is an important consideration when evaluating lock performance. If acquiring lock causes a lot of bus traffic, it will not scale well as the number of threads increases. Eventually the bus will become choked.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Fairness ===<br />
In general, threads should acquire a lock in them same order that the locks were requested. If this does not happen, as the number of threads increases so does the chance that a thread will become starved.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Storage ===<br />
The amount of storage needed per lock should be small enough such that it scales to a large number of threads without any problems. If a large amount of storage is required, then multiple threads could cause locks to consume too much memory.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
<br />
Two common lock implementations are test&set and test and test&set. In terms of performance, both have a low storage cost (scalable) and are not fair. Test&set has a low acquisition latency but is not scalable due do high bus traffic when there is a lot of contention for the lock. Test-and-test&set has a higher acquisition latency but scales better because it generates less bus traffic. The next figure shows performance comparisons for these two lock implementations.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
[[Image:lockperf.png|frame|none|Performance comparison of test&set and test-and-test&set (spin on read)<sup><span id="3body">[[#3foot|[3]]]</span></sup>]]<br />
<br />
=== Improving performance ===<br />
Optimizing software can improve the performance of locks. For instance, inserting a delay between a processor noticing a release and then trying to acquire a lock can reduce contention and bus traffic, and increase the performance of the locks. TCP-like back off algorithms can be used to adjust this delay depending on the number of threads contending for the lock.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
Another way to improve performance is to insert a delay between memory references so as to limit the bandwidth (traffic) each processor can use for spinning.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
One way to guarantee fairness is to queue lock attempts in shared memory so that they happen in order.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
== Atomic Instructions ==<br />
<br />
Since a multiprocessor system cannot disable interrupts as an effective method to execute code atomically, there must be hardware support for atomic operations.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
The ability to execute an atomic instruction is a requirement for most lock implementations. It is important that when a processor attempts to set a lock, either the lock is fully set and the thread is able to enter into the critical section, or the lock is not set and it appears that none of the instructions required to set the lock have executed. <br />
<br />
In the x86 instruction set the opcode CMPXCHG (compare and exchange) can be used in a lock implementation in order to guarantee atomicity. This function works by sending a destination and a source. The accumulator is compared to the destination and, if they are equal, loaded with the source. If they are not equal the accumulator is loaded with the destination value. In order to assure that this is executed atomically the opcode must be issued with the LOCK prefix. This is useful in implementing some locks, such as ticket locks.<sup><span id="5body">[[#5foot|[5]]]</span></sup> The syntax for the instruction is LOCK CMPXCHG SOURCE DESTINATION. Here, the source has to be a register while the destination can be either Register or Memory. An example of LOCK CMPXCHG is implementation in dis-assembly of Linux kernel's rtc_cmos_read(). The CPU locks needs to be copied and and then updated to a non zero value if the lock is to be applied. But there is a possibility when the value got copied earlier and changed later. This causes a race condition. Hence,it helps in briefing the busy wait loop by preventing race condition. The pseudo code for same is given as below<sup><span id="10body">[[#10foot|[10]]]</span></sup>:<br />
<br />
spin: # see if the lock-variable is clear<br />
mov cmos_lock, %eax<br />
test %eax, %eax<br />
jnz spin<br />
# ok, now we try to grab the lock<br />
lock cmpxchg %edx, cmos_lock<br />
# did another CPU grab it first? <br />
test %eax, %eax<br />
jnz spin<br />
<br />
== Hand-off Lock ==<br />
<br />
Another type of lock that is not discussed in the text is known as the "hand-off" lock. In this lock the first thread acquires the lock if no other thread is currently locked (since it is the first thread). When another thread attempts to gain the lock it will see that the lock is in use and adds itself to the queue. Once done this thread can sleep until called by the thread with the lock. Once the thread in the lock is finished, it will pass the lock to the next thread in the queue<sup><span id="6body">[[#6foot|[6]]]</span></sup>. Waiting thread gives up processor so that other threads (e.g. the thread with the lock) can run more quickly. Someone wakes up thread when the lock is free. The separation of ready queue from that of waiting queue fastens the process of execution. In the below example, the lock is given to a waiting thread by the unlock method. The interrupts are enabled only when the thread is placed in waiting queue and disabled once its given the lock. <br />
<br />
lock () {<br />
disable interrupts<br />
if (value == FREE) {<br />
value = BUSY<br />
} else {<br />
add thread to queue of threads waiting for<br />
this lock<br />
switch to next runnable thread<br />
}<br />
enable interrupts<br />
}<br />
unlock () {<br />
disable interrupts<br />
value = FREE<br />
if (any thread is waiting for this lock) {<br />
move waiting thread from waiting queue to<br />
ready queue<br />
value = BUSY<br />
}<br />
enable interrupts<br />
} <br />
<br />
But this fails if interrupt happens after thread enable interrupts Lock() adds thread to wait queue Lock() enables interrupts Interrupts causes pre-emption, i.e. switch to another thread. Pre-emption moves thread to ready queue. Now thread is on two queues (wait and ready)! Also, switch is likely to be a critical section Adding thread to wait queue and switching to next thread must be atomic. Solution to this problem is waiting thread leaves interrupts disabled when it calls switch. Next thread to run has the responsibility of re-enabling interrupts before returning to user code. When waiting thread wakes up, it returns from switch with interrupts disabled (from the last thread)<sup><span id="11body">[[#11foot|[11]]]</span></sup><br />
<br />
== Avoiding Locks ==<br />
<br />
There are many reasons why a programmer should attempt to write programs in such a way as to avoid locks if possible. There are many problems that can arise with the use of locks.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
One of the must well known issues is deadlock. This can occur when the threads are waiting to acquire the lock, but the lock will never be unlocked. For example, there are two processes A and B and they need two methods (1 and 2) to be grabbed for completion of a task which are shared by both A and B. This can create a deadlock when A grabs 1 and B grabs 2. This reaches to a situation where both A and B cannot change their states. The solution to this problem is to set a condition where in A process can lock method 2 only if t has locked method 1.<br />
<br />
Another problem with using locks is that the performance is not optimal, as often a lock is used when there is only a chance of conflict. This approach to programming yields slower performance than what might be possible with other methods. This also leads to questions of granularity, that is how much of the code should be protected under the critical section. The programmer must decide between many small (fine grain) locks or fewer, more encompassing locks. This decision can greatly effect the performance of the program.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
To avoid locks, lock free programming techniques like compare and swap and sequential consistency can be followed. This pattern typically involves copying a shared variable to a local variable, performing some speculative work, and attempting to publish the changes using CAS<sup><span id="12body">[[#12foot|[12]]]</span></sup>:<br />
<br />
void LockFreeQueue::push(Node* newHead)<br />
{<br />
for (;;)<br />
{<br />
// Copy a shared variable (m_Head) to a local.<br />
Node* oldHead = m_Head;<br />
<br />
// Do some speculative work, not yet visible to other threads.<br />
newHead->next = oldHead;<br />
<br />
// Next, attempt to publish our changes to the shared variable.<br />
// If the shared variable hasn't changed, the CAS succeeds and we return.<br />
// Otherwise, repeat.<br />
if (_InterlockedCompareExchange(&m_Head, newHead, oldHead) == oldHead)<br />
return;<br />
}<br />
}<br />
The above code basically follows the Read-Write-Update order and makes sure that each thread has its local copy to work on and gets it updated sequentially so that the next thread has the correct value. If the value is not correct, write fails<br />
<br />
= Barrier Implementations =<br />
<br />
In many ways, barriers are simpler than locks. A barrier is simply a point in a program where one or more threads must reach before the parallel program is allowed to continue. When using barriers, a programmer does not have to be concerned with advanced topics such as fairness, that are required when programming for locks.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these two criteria:<br />
<br />
*Latency - The time required to enter and exit a barrier.<br />
*Traffic - Communications overhead required by the barrier.<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Latency ===<br />
<br />
Ideally the latency of a barrier should be small, but for barriers that don't scale well latency can increase as the number of threads increases.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
<br />
Low traffic is also good, as we do not want excess buss traffic to prevent scalability.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
{| class="wikitable" border="1"<br />
|+Performance Comparison of Barrier Implementations<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
|-<br />
! Criteria<br />
! Sense Reversal<br />
Centralized Barrier<br />
! Combining Tree<br />
Barrier<br />
! Barrier Network<br />
(Hardware)<br />
|-<br />
| Latency<br />
| O(1)<br />
| O(log p)<br />
| O(log p)<br />
|-<br />
| Traffic<br />
| O(p<sup>2</sup>)<br />
| O(p)<br />
| moved to a separate network<br />
|}<br />
<br />
<br />
<br />
[[Image:Ch9wiki05.png|frame|none|Additive Schwarz Preconditioned Conjugate Gradient (ASPCG) kernel on Altix System - This figure shows the timings of the ASPCG kernel using the different barrier implementations. It can be seen that the blocking barrier does not scale with number of threads as with increase in number of threads the contention increases.<sup><span id="7body">[[#7foot|[7]]]</span></sup>]]<br />
<br />
<br />
<br />
[[Image:Ch9wiki06.png|frame|none|EPCC Microbenchmark - This figure shows the timings to implement a barrier of the EPCC Microbenchmark using the different barrier implementations. It can be seen that the blocking barrier/centralized blocking barrier does not scale with number of threads as with increase in number of threads the contention increases.<sup><span id="7body">[[#7foot|[7]]]</span></sup>]]<br />
<br />
== Sense-Reversal Barrier ==<br />
<br />
This barrier is a centralized barrier where a single count variable protected by a lock is shared among all the threads. Each thread on reaching the barrier increments its value and waits till the value of variable has reached the number of threads for which barrier was implemented.<br />
<br />
Since all the threads are spinning around a single variable the miss rate is scales quadratically with number of processors.<br />
<br />
[[Image:Ch9wiki01.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Combining Tree Barrier ==<br />
<br />
This barrier is a distributed barrier where group of processors form clusters and updates value of a local variable. The local variable on reaching a value equal to number of threads updating it, proceeds to increment another variable higher in hierarchy in the combining tree. When the variable in the highest level of hierarchy in the combining tree reaches its max value it is considered that all the threads have reached the barrier and synchronization is complete.<br />
<br />
Since in this case all the threads are updated local variables in form of smaller groups the miss rate is not as high as sense-reversal barrier.<br />
<br />
The diagram below shows the operation of combining tree barrier with threads grouped in two groups. The variables C0 and C1 are local to each group and C2 is the variable that is at higher level of hierarchy in the tree.<br />
<br />
[[Image:Ch9wiki02.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Tournament Barrier ==<br />
<br />
In tournament barrier the threads are considered to be leaves at the end of a binary tree and each node represents a flag. Two threads compete with each other and the loser thread is allowed to set the flag and move to higher level and compete to lose with the loser thread from other section of binary tree. Thus the thread which completes last is able to set the highest flag in the binary tree. On setting the flag it indicates to all the threads that barrier has been completed and thus synchronization is achieved.<br />
<br />
[[Image:Ch9wiki03.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Disseminating Barrier ==<br />
<br />
In this barrier each thread maintains a record of the activity of other threads. For every round i with n threads, thread A notifies thread (A + 2i) mod n. Thus after logn rounds all the threads are aware of the status of every other thread running and whether it has reached the barrier.<br />
<br />
[[Image:Ch9wiki04.png]]<sup><span id="9body">[[#9foot|[9]]]</span></sup><br />
<br />
= References = <br />
<span id="1foot">[[#1body|1.]]</span> David Culler, J.P. Singh and Anoop Gupta. Parallel Computer Architecture: A Hardware/Software Approach. Morgan Kaufmann. August 1998.<br><br />
<span id="2foot">[[#2body|2.]]</span> Yan Solihin. Fundamentals of Parallel Computer Architecture: Multichip and Multicore Systems. Solihin Books. August 2009.<br><br />
<span id="3foot">[[#3body|3.]]</span> Thomas E. Anderson. The Performance of Spin Lock Alternatives for Shared-memory Multiprocessors. http://www.cc.gatech.edu/classes/AY2010/cs4210_fall/papers/anderson-spinlock.pdf<br><br />
<span id="4foot">[[#4body|4.]]</span> http://www.statemaster.com/encyclopedia/Lock-(computer-science)<br><br />
<span id="5foot">[[#5body|5.]]</span> http://faydoc.tripod.com/cpu/cmpxchg.htm<br><br />
<span id="6foot">[[#6body|6.]]</span> http://www.cs.duke.edu/courses/fall09/cps110/slides/threads3.ppt<br><br />
<span id="7foot">[[#7body|7.]]</span> Scalability evaluation of barrier algorithms for OpenMP. http://www2.cs.uh.edu/~hpctools/pub/iwomp-barrier.pdf<br><br />
<span id="8foot">[[#8body|8.]]</span> Carwyn Ball and Mark Bull. Barrier Synchronization in Java. http://www.ukhec.ac.uk/publications/reports/synch_java.pdf<br><br />
<span id="9foot">[[#9body|9.]]</span> http://www.cs.brown.edu/courses/cs176/barrier.ppt<br><br />
<span id="10foot">[[#10body|10.]]</span>http://heather.cs.ucdavis.edu/~matloff/50/PLN/lock.pdf<br><br />
<span id="11foot">[[#11body|11.]]</span>http://www.cs.duke.edu/courses/fall09/cps110/handouts/threads3.pdf<br><br />
<span id="12foot">[[#12body|12.]]</span>http://preshing.com/20120612/an-introduction-to-lock-free-programming<br></div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/9b_sc&diff=74672CSC/ECE 506 Spring 2013/9b sc2013-04-03T23:26:31Z<p>Schakra8: /* Avoiding Locks */</p>
<hr />
<div>= Synchronization =<br />
In addition to a proper cache coherency model, it is important for a multiprocessor system to provide support for synchronization. This must be provided at the hardware level, and can also be implemented to some degree in software. The most common types of hardware synchronization are locks and barriers, which are discussed in this chapter.<br />
<br />
== Hardware vs. Operating System Synchronization ==<br />
Synchronizations are characterized by three parts: acquire, wait, and release. Both hardware and operating system synchronization rely on atomic instructions for acquire and release, but the implementations for the wait portion of a synchronization can vary. Hardware implementations usually use busy-waiting, where a process repeatedly tests a variable waiting for it to change. The operating system can wait using blocking, where the process suspends itself until woken up by another process.<br />
<br />
Blocking has the advantage of freeing up the processor for use by other processes, but requires operating system functions to work and thus has a lot of overhead. Busy-waiting has much less overhead, but uses processor and cache resources while waiting. Because of these tradeoffs, it is usually better to use busy-waiting for short wait periods, and blocking for long wait periods.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
= Lock Implementations =<br />
<br />
Locks are an important concept when programming for multiprocessor systems. The purpose of a lock is to protect the code inside the lock, known as a critical section. It must be certain that while some thread X has entered the critical section, another thread Y is not also inside of the critical section and possibly modifying critical values. While thread X has entered the critical section, thread Y must wait until X has exited before entering. This can be accomplished in a variety of ways of varying complexity and performance.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these four criteria:<br />
<br />
*Acquisition Latency - How much time does it take to acquire the lock?<br />
*Traffic - How much bus traffic is generated by threads attempting to acquire the lock?<br />
*Fairness - FIFO vs. Luck<br />
*Storage - How much storage is needed compared to the number of threads?<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Acquisition Latency ===<br />
We want a low acquisition latency, especially for applications that repeatedly acquire and release locks. Over the run time of a program, the acquisition latency compounded many times could have a large performance affect. However, low acquisition latency has to be balanced against other factors when implementing a lock.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
Traffic is an important consideration when evaluating lock performance. If acquiring lock causes a lot of bus traffic, it will not scale well as the number of threads increases. Eventually the bus will become choked.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Fairness ===<br />
In general, threads should acquire a lock in them same order that the locks were requested. If this does not happen, as the number of threads increases so does the chance that a thread will become starved.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Storage ===<br />
The amount of storage needed per lock should be small enough such that it scales to a large number of threads without any problems. If a large amount of storage is required, then multiple threads could cause locks to consume too much memory.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
<br />
Two common lock implementations are test&set and test and test&set. In terms of performance, both have a low storage cost (scalable) and are not fair. Test&set has a low acquisition latency but is not scalable due do high bus traffic when there is a lot of contention for the lock. Test-and-test&set has a higher acquisition latency but scales better because it generates less bus traffic. The next figure shows performance comparisons for these two lock implementations.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
[[Image:lockperf.png|frame|none|Performance comparison of test&set and test-and-test&set (spin on read)<sup><span id="3body">[[#3foot|[3]]]</span></sup>]]<br />
<br />
=== Improving performance ===<br />
Optimizing software can improve the performance of locks. For instance, inserting a delay between a processor noticing a release and then trying to acquire a lock can reduce contention and bus traffic, and increase the performance of the locks. TCP-like back off algorithms can be used to adjust this delay depending on the number of threads contending for the lock.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
Another way to improve performance is to insert a delay between memory references so as to limit the bandwidth (traffic) each processor can use for spinning.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
One way to guarantee fairness is to queue lock attempts in shared memory so that they happen in order.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
== Atomic Instructions ==<br />
<br />
Since a multiprocessor system cannot disable interrupts as an effective method to execute code atomically, there must be hardware support for atomic operations.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
The ability to execute an atomic instruction is a requirement for most lock implementations. It is important that when a processor attempts to set a lock, either the lock is fully set and the thread is able to enter into the critical section, or the lock is not set and it appears that none of the instructions required to set the lock have executed. <br />
<br />
In the x86 instruction set the opcode CMPXCHG (compare and exchange) can be used in a lock implementation in order to guarantee atomicity. This function works by sending a destination and a source. The accumulator is compared to the destination and, if they are equal, loaded with the source. If they are not equal the accumulator is loaded with the destination value. In order to assure that this is executed atomically the opcode must be issued with the LOCK prefix. This is useful in implementing some locks, such as ticket locks.<sup><span id="5body">[[#5foot|[5]]]</span></sup> The syntax for the instruction is LOCK CMPXCHG SOURCE DESTINATION. Here, the source has to be a register while the destination can be either Register or Memory. An example of LOCK CMPXCHG is implementation in dis-assembly of Linux kernel's rtc_cmos_read(). The CPU locks needs to be copied and and then updated to a non zero value if the lock is to be applied. But there is a possibility when the value got copied earlier and changed later. This causes a race condition. Hence,it helps in briefing the busy wait loop by preventing race condition. The pseudo code for same is given as below<sup><span id="10body">[[#10foot|[10]]]</span></sup>:<br />
<br />
spin: # see if the lock-variable is clear<br />
mov cmos_lock, %eax<br />
test %eax, %eax<br />
jnz spin<br />
# ok, now we try to grab the lock<br />
lock cmpxchg %edx, cmos_lock<br />
# did another CPU grab it first? <br />
test %eax, %eax<br />
jnz spin<br />
<br />
== Hand-off Lock ==<br />
<br />
Another type of lock that is not discussed in the text is known as the "hand-off" lock. In this lock the first thread acquires the lock if no other thread is currently locked (since it is the first thread). When another thread attempts to gain the lock it will see that the lock is in use and adds itself to the queue. Once done this thread can sleep until called by the thread with the lock. Once the thread in the lock is finished, it will pass the lock to the next thread in the queue<sup><span id="6body">[[#6foot|[6]]]</span></sup>. Waiting thread gives up processor so that other threads (e.g. the thread with the lock) can run more quickly. Someone wakes up thread when the lock is free. The separation of ready queue from that of waiting queue fastens the process of execution. In the below example, the lock is given to a waiting thread by the unlock method. The interrupts are enabled only when the thread is placed in waiting queue and disabled once its given the lock. <br />
<br />
lock () {<br />
disable interrupts<br />
if (value == FREE) {<br />
value = BUSY<br />
} else {<br />
add thread to queue of threads waiting for<br />
this lock<br />
switch to next runnable thread<br />
}<br />
enable interrupts<br />
}<br />
unlock () {<br />
disable interrupts<br />
value = FREE<br />
if (any thread is waiting for this lock) {<br />
move waiting thread from waiting queue to<br />
ready queue<br />
value = BUSY<br />
}<br />
enable interrupts<br />
} <br />
<br />
But this fails if interrupt happens after thread enable interrupts Lock() adds thread to wait queue Lock() enables interrupts Interrupts causes pre-emption, i.e. switch to another thread. Pre-emption moves thread to ready queue. Now thread is on two queues (wait and ready)! Also, switch is likely to be a critical section Adding thread to wait queue and switching to next thread must be atomic. Solution to this problem is waiting thread leaves interrupts disabled when it calls switch. Next thread to run has the responsibility of re-enabling interrupts before returning to user code. When waiting thread wakes up, it returns from switch with interrupts disabled (from the last thread)<sup><span id="11body">[[#11foot|[11]]]</span></sup><br />
<br />
== Avoiding Locks ==<br />
<br />
There are many reasons why a programmer should attempt to write programs in such a way as to avoid locks if possible. There are many problems that can arise with the use of locks.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
One of the must well known issues is deadlock. This can occur when the threads are waiting to acquire the lock, but the lock will never be unlocked. For example, there are two processes A and B and they need two methods (1 and 2) to be grabbed for completion of a task which are shared by both A and B. This can create a deadlock when A grabs 1 and B grabs 2. This reaches to a situation where both A and B cannot change their states. The solution to this problem is to set a condition where in A process can lock method 2 only if t has locked method 1.<br />
<br />
Another problem with using locks is that the performance is not optimal, as often a lock is used when there is only a chance of conflict. This approach to programming yields slower performance than what might be possible with other methods. This also leads to questions of granularity, that is how much of the code should be protected under the critical section. The programmer must decide between many small (fine grain) locks or fewer, more encompassing locks. This decision can greatly effect the performance of the program.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
To avoid locks, lock free programming techniques like compare and swap and sequential consistency can be followed. This pattern typically involves copying a shared variable to a local variable, performing some speculative work, and attempting to publish the changes using CAS<sup><span id="12body">[[#12foot|[12]]]</span></sup>:<br />
<br />
void LockFreeQueue::push(Node* newHead)<br />
{<br />
for (;;)<br />
{<br />
// Copy a shared variable (m_Head) to a local.<br />
Node* oldHead = m_Head;<br />
<br />
// Do some speculative work, not yet visible to other threads.<br />
newHead->next = oldHead;<br />
<br />
// Next, attempt to publish our changes to the shared variable.<br />
// If the shared variable hasn't changed, the CAS succeeds and we return.<br />
// Otherwise, repeat.<br />
if (_InterlockedCompareExchange(&m_Head, newHead, oldHead) == oldHead)<br />
return;<br />
}<br />
}<br />
The above code basically follows the Read-Write-Update order and makes sure that each thread has its local copy to work on and gets it updated sequentially so that the next thread has the correct value. If the value is not correct, write fails<br />
<br />
= Barrier Implementations =<br />
<br />
In many ways, barriers are simpler than locks. A barrier is simply a point in a program where one or more threads must reach before the parallel program is allowed to continue. When using barriers, a programmer does not have to be concerned with advanced topics such as fairness, that are required when programming for locks.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these two criteria:<br />
<br />
*Latency - The time required to enter and exit a barrier.<br />
*Traffic - Communications overhead required by the barrier.<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Latency ===<br />
<br />
Ideally the latency of a barrier should be small, but for barriers that don't scale well latency can increase as the number of threads increases.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
<br />
Low traffic is also good, as we do not want excess buss traffic to prevent scalability.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
{| class="wikitable" border="1"<br />
|+Performance Comparison of Barrier Implementations<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
|-<br />
! Criteria<br />
! Sense Reversal<br />
Centralized Barrier<br />
! Combining Tree<br />
Barrier<br />
! Barrier Network<br />
(Hardware)<br />
|-<br />
| Latency<br />
| O(1)<br />
| O(log p)<br />
| O(log p)<br />
|-<br />
| Traffic<br />
| O(p<sup>2</sup>)<br />
| O(p)<br />
| moved to a separate network<br />
|}<br />
<br />
<br />
<br />
[[Image:Ch9wiki05.png|frame|none|Additive Schwarz Preconditioned Conjugate Gradient (ASPCG) kernel on Altix System - This figure shows the timings of the ASPCG kernel using the different barrier implementations. It can be seen that the blocking barrier does not scale with number of threads as with increase in number of threads the contention increases.<sup><span id="7body">[[#7foot|[7]]]</span></sup>]]<br />
<br />
<br />
<br />
[[Image:Ch9wiki06.png|frame|none|EPCC Microbenchmark - This figure shows the timings to implement a barrier of the EPCC Microbenchmark using the different barrier implementations. It can be seen that the blocking barrier/centralized blocking barrier does not scale with number of threads as with increase in number of threads the contention increases.<sup><span id="7body">[[#7foot|[7]]]</span></sup>]]<br />
<br />
== Sense-Reversal Barrier ==<br />
<br />
This barrier is a centralized barrier where a single count variable protected by a lock is shared among all the threads. Each thread on reaching the barrier increments its value and waits till the value of variable has reached the number of threads for which barrier was implemented.<br />
<br />
Since all the threads are spinning around a single variable the miss rate is scales quadratically with number of processors.<br />
<br />
[[Image:Ch9wiki01.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Combining Tree Barrier ==<br />
<br />
This barrier is a distributed barrier where group of processors form clusters and updates value of a local variable. The local variable on reaching a value equal to number of threads updating it, proceeds to increment another variable higher in hierarchy in the combining tree. When the variable in the highest level of hierarchy in the combining tree reaches its max value it is considered that all the threads have reached the barrier and synchronization is complete.<br />
<br />
Since in this case all the threads are updated local variables in form of smaller groups the miss rate is not as high as sense-reversal barrier.<br />
<br />
The diagram below shows the operation of combining tree barrier with threads grouped in two groups. The variables C0 and C1 are local to each group and C2 is the variable that is at higher level of hierarchy in the tree.<br />
<br />
[[Image:Ch9wiki02.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Tournament Barrier ==<br />
<br />
In tournament barrier the threads are considered to be leaves at the end of a binary tree and each node represents a flag. Two threads compete with each other and the loser thread is allowed to set the flag and move to higher level and compete to lose with the loser thread from other section of binary tree. Thus the thread which completes last is able to set the highest flag in the binary tree. On setting the flag it indicates to all the threads that barrier has been completed and thus synchronization is achieved.<br />
<br />
[[Image:Ch9wiki03.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Disseminating Barrier ==<br />
<br />
In this barrier each thread maintains a record of the activity of other threads. For every round i with n threads, thread A notifies thread (A + 2i) mod n. Thus after logn rounds all the threads are aware of the status of every other thread running and whether it has reached the barrier.<br />
<br />
[[Image:Ch9wiki04.png]]<sup><span id="9body">[[#9foot|[9]]]</span></sup><br />
<br />
= References = <br />
<span id="1foot">[[#1body|1.]]</span> David Culler, J.P. Singh and Anoop Gupta. Parallel Computer Architecture: A Hardware/Software Approach. Morgan Kaufmann. August 1998.<br><br />
<span id="2foot">[[#2body|2.]]</span> Yan Solihin. Fundamentals of Parallel Computer Architecture: Multichip and Multicore Systems. Solihin Books. August 2009.<br><br />
<span id="3foot">[[#3body|3.]]</span> Thomas E. Anderson. The Performance of Spin Lock Alternatives for Shared-memory Multiprocessors. http://www.cc.gatech.edu/classes/AY2010/cs4210_fall/papers/anderson-spinlock.pdf<br><br />
<span id="4foot">[[#4body|4.]]</span> http://www.statemaster.com/encyclopedia/Lock-(computer-science)<br><br />
<span id="5foot">[[#5body|5.]]</span> http://faydoc.tripod.com/cpu/cmpxchg.htm<br><br />
<span id="6foot">[[#6body|6.]]</span> http://www.cs.duke.edu/courses/fall09/cps110/slides/threads3.ppt<br><br />
<span id="7foot">[[#7body|7.]]</span> Scalability evaluation of barrier algorithms for OpenMP. http://www2.cs.uh.edu/~hpctools/pub/iwomp-barrier.pdf<br><br />
<span id="8foot">[[#8body|8.]]</span> Carwyn Ball and Mark Bull. Barrier Synchronization in Java. http://www.ukhec.ac.uk/publications/reports/synch_java.pdf<br><br />
<span id="9foot">[[#9body|9.]]</span> http://www.cs.brown.edu/courses/cs176/barrier.ppt<br><br />
<span id="10foot">[[#10body|10.]]</span>http://heather.cs.ucdavis.edu/~matloff/50/PLN/lock.pdf<br><br />
<span id="11foot">[[#11body|11.]]</span>http://www.cs.duke.edu/courses/fall09/cps110/handouts/threads3.pdf<br><br />
<span id="12foot">[[#12body|12.]]</span>http://preshing.com/20120612/an-introduction-to-lock-free-programming<br></div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/9b_sc&diff=74671CSC/ECE 506 Spring 2013/9b sc2013-04-03T23:23:51Z<p>Schakra8: /* References */</p>
<hr />
<div>= Synchronization =<br />
In addition to a proper cache coherency model, it is important for a multiprocessor system to provide support for synchronization. This must be provided at the hardware level, and can also be implemented to some degree in software. The most common types of hardware synchronization are locks and barriers, which are discussed in this chapter.<br />
<br />
== Hardware vs. Operating System Synchronization ==<br />
Synchronizations are characterized by three parts: acquire, wait, and release. Both hardware and operating system synchronization rely on atomic instructions for acquire and release, but the implementations for the wait portion of a synchronization can vary. Hardware implementations usually use busy-waiting, where a process repeatedly tests a variable waiting for it to change. The operating system can wait using blocking, where the process suspends itself until woken up by another process.<br />
<br />
Blocking has the advantage of freeing up the processor for use by other processes, but requires operating system functions to work and thus has a lot of overhead. Busy-waiting has much less overhead, but uses processor and cache resources while waiting. Because of these tradeoffs, it is usually better to use busy-waiting for short wait periods, and blocking for long wait periods.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
= Lock Implementations =<br />
<br />
Locks are an important concept when programming for multiprocessor systems. The purpose of a lock is to protect the code inside the lock, known as a critical section. It must be certain that while some thread X has entered the critical section, another thread Y is not also inside of the critical section and possibly modifying critical values. While thread X has entered the critical section, thread Y must wait until X has exited before entering. This can be accomplished in a variety of ways of varying complexity and performance.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these four criteria:<br />
<br />
*Acquisition Latency - How much time does it take to acquire the lock?<br />
*Traffic - How much bus traffic is generated by threads attempting to acquire the lock?<br />
*Fairness - FIFO vs. Luck<br />
*Storage - How much storage is needed compared to the number of threads?<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Acquisition Latency ===<br />
We want a low acquisition latency, especially for applications that repeatedly acquire and release locks. Over the run time of a program, the acquisition latency compounded many times could have a large performance affect. However, low acquisition latency has to be balanced against other factors when implementing a lock.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
Traffic is an important consideration when evaluating lock performance. If acquiring lock causes a lot of bus traffic, it will not scale well as the number of threads increases. Eventually the bus will become choked.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Fairness ===<br />
In general, threads should acquire a lock in them same order that the locks were requested. If this does not happen, as the number of threads increases so does the chance that a thread will become starved.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Storage ===<br />
The amount of storage needed per lock should be small enough such that it scales to a large number of threads without any problems. If a large amount of storage is required, then multiple threads could cause locks to consume too much memory.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
<br />
Two common lock implementations are test&set and test and test&set. In terms of performance, both have a low storage cost (scalable) and are not fair. Test&set has a low acquisition latency but is not scalable due do high bus traffic when there is a lot of contention for the lock. Test-and-test&set has a higher acquisition latency but scales better because it generates less bus traffic. The next figure shows performance comparisons for these two lock implementations.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
[[Image:lockperf.png|frame|none|Performance comparison of test&set and test-and-test&set (spin on read)<sup><span id="3body">[[#3foot|[3]]]</span></sup>]]<br />
<br />
=== Improving performance ===<br />
Optimizing software can improve the performance of locks. For instance, inserting a delay between a processor noticing a release and then trying to acquire a lock can reduce contention and bus traffic, and increase the performance of the locks. TCP-like back off algorithms can be used to adjust this delay depending on the number of threads contending for the lock.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
Another way to improve performance is to insert a delay between memory references so as to limit the bandwidth (traffic) each processor can use for spinning.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
One way to guarantee fairness is to queue lock attempts in shared memory so that they happen in order.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
== Atomic Instructions ==<br />
<br />
Since a multiprocessor system cannot disable interrupts as an effective method to execute code atomically, there must be hardware support for atomic operations.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
The ability to execute an atomic instruction is a requirement for most lock implementations. It is important that when a processor attempts to set a lock, either the lock is fully set and the thread is able to enter into the critical section, or the lock is not set and it appears that none of the instructions required to set the lock have executed. <br />
<br />
In the x86 instruction set the opcode CMPXCHG (compare and exchange) can be used in a lock implementation in order to guarantee atomicity. This function works by sending a destination and a source. The accumulator is compared to the destination and, if they are equal, loaded with the source. If they are not equal the accumulator is loaded with the destination value. In order to assure that this is executed atomically the opcode must be issued with the LOCK prefix. This is useful in implementing some locks, such as ticket locks.<sup><span id="5body">[[#5foot|[5]]]</span></sup> The syntax for the instruction is LOCK CMPXCHG SOURCE DESTINATION. Here, the source has to be a register while the destination can be either Register or Memory. An example of LOCK CMPXCHG is implementation in dis-assembly of Linux kernel's rtc_cmos_read(). The CPU locks needs to be copied and and then updated to a non zero value if the lock is to be applied. But there is a possibility when the value got copied earlier and changed later. This causes a race condition. Hence,it helps in briefing the busy wait loop by preventing race condition. The pseudo code for same is given as below<sup><span id="10body">[[#10foot|[10]]]</span></sup>:<br />
<br />
spin: # see if the lock-variable is clear<br />
mov cmos_lock, %eax<br />
test %eax, %eax<br />
jnz spin<br />
# ok, now we try to grab the lock<br />
lock cmpxchg %edx, cmos_lock<br />
# did another CPU grab it first? <br />
test %eax, %eax<br />
jnz spin<br />
<br />
== Hand-off Lock ==<br />
<br />
Another type of lock that is not discussed in the text is known as the "hand-off" lock. In this lock the first thread acquires the lock if no other thread is currently locked (since it is the first thread). When another thread attempts to gain the lock it will see that the lock is in use and adds itself to the queue. Once done this thread can sleep until called by the thread with the lock. Once the thread in the lock is finished, it will pass the lock to the next thread in the queue<sup><span id="6body">[[#6foot|[6]]]</span></sup>. Waiting thread gives up processor so that other threads (e.g. the thread with the lock) can run more quickly. Someone wakes up thread when the lock is free. The separation of ready queue from that of waiting queue fastens the process of execution. In the below example, the lock is given to a waiting thread by the unlock method. The interrupts are enabled only when the thread is placed in waiting queue and disabled once its given the lock. <br />
<br />
lock () {<br />
disable interrupts<br />
if (value == FREE) {<br />
value = BUSY<br />
} else {<br />
add thread to queue of threads waiting for<br />
this lock<br />
switch to next runnable thread<br />
}<br />
enable interrupts<br />
}<br />
unlock () {<br />
disable interrupts<br />
value = FREE<br />
if (any thread is waiting for this lock) {<br />
move waiting thread from waiting queue to<br />
ready queue<br />
value = BUSY<br />
}<br />
enable interrupts<br />
} <br />
<br />
But this fails if interrupt happens after thread enable interrupts Lock() adds thread to wait queue Lock() enables interrupts Interrupts causes pre-emption, i.e. switch to another thread. Pre-emption moves thread to ready queue. Now thread is on two queues (wait and ready)! Also, switch is likely to be a critical section Adding thread to wait queue and switching to next thread must be atomic. Solution to this problem is waiting thread leaves interrupts disabled when it calls switch. Next thread to run has the responsibility of re-enabling interrupts before returning to user code. When waiting thread wakes up, it returns from switch with interrupts disabled (from the last thread)<sup><span id="11body">[[#11foot|[11]]]</span></sup><br />
<br />
== Avoiding Locks ==<br />
<br />
There are many reasons why a programmer should attempt to write programs in such a way as to avoid locks if possible. There are many problems that can arise with the use of locks.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
One of the must well known issues is deadlock. This can occur when the threads are waiting to acquire the lock, but the lock will never be unlocked. For example, there are two processes A and B and they need two methods (1 and 2) to be grabbed for completion of a task which are shared by both A and B. This can create a deadlock when A grabs 1 and B grabs 2. This reaches to a situation where both A and B cannot change their states. The solution to this problem is to set a condition where in A process can lock method 2 only if t has locked method 1.<br />
<br />
Another problem with using locks is that the performance is not optimal, as often a lock is used when there is only a chance of conflict. This approach to programming yields slower performance than what might be possible with other methods. This also leads to questions of granularity, that is how much of the code should be protected under the critical section. The programmer must decide between many small (fine grain) locks or fewer, more encompassing locks. This decision can greatly effect the performance of the program.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
To avoid locks, lock free programming techniques like compare and swap and sequential consistency can be followed. This pattern typically involves copying a shared variable to a local variable, performing some speculative work, and attempting to publish the changes using CAS<sup><span id="12body">[[#12foot|[12]]]</span></sup>:<br />
<br />
void LockFreeQueue::push(Node* newHead)<br />
{<br />
for (;;)<br />
{<br />
// Copy a shared variable (m_Head) to a local.<br />
Node* oldHead = m_Head;<br />
<br />
// Do some speculative work, not yet visible to other threads.<br />
newHead->next = oldHead;<br />
<br />
// Next, attempt to publish our changes to the shared variable.<br />
// If the shared variable hasn't changed, the CAS succeeds and we return.<br />
// Otherwise, repeat.<br />
if (_InterlockedCompareExchange(&m_Head, newHead, oldHead) == oldHead)<br />
return;<br />
}<br />
}<br />
<br />
= Barrier Implementations =<br />
<br />
In many ways, barriers are simpler than locks. A barrier is simply a point in a program where one or more threads must reach before the parallel program is allowed to continue. When using barriers, a programmer does not have to be concerned with advanced topics such as fairness, that are required when programming for locks.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these two criteria:<br />
<br />
*Latency - The time required to enter and exit a barrier.<br />
*Traffic - Communications overhead required by the barrier.<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Latency ===<br />
<br />
Ideally the latency of a barrier should be small, but for barriers that don't scale well latency can increase as the number of threads increases.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
<br />
Low traffic is also good, as we do not want excess buss traffic to prevent scalability.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
{| class="wikitable" border="1"<br />
|+Performance Comparison of Barrier Implementations<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
|-<br />
! Criteria<br />
! Sense Reversal<br />
Centralized Barrier<br />
! Combining Tree<br />
Barrier<br />
! Barrier Network<br />
(Hardware)<br />
|-<br />
| Latency<br />
| O(1)<br />
| O(log p)<br />
| O(log p)<br />
|-<br />
| Traffic<br />
| O(p<sup>2</sup>)<br />
| O(p)<br />
| moved to a separate network<br />
|}<br />
<br />
<br />
<br />
[[Image:Ch9wiki05.png|frame|none|Additive Schwarz Preconditioned Conjugate Gradient (ASPCG) kernel on Altix System - This figure shows the timings of the ASPCG kernel using the different barrier implementations. It can be seen that the blocking barrier does not scale with number of threads as with increase in number of threads the contention increases.<sup><span id="7body">[[#7foot|[7]]]</span></sup>]]<br />
<br />
<br />
<br />
[[Image:Ch9wiki06.png|frame|none|EPCC Microbenchmark - This figure shows the timings to implement a barrier of the EPCC Microbenchmark using the different barrier implementations. It can be seen that the blocking barrier/centralized blocking barrier does not scale with number of threads as with increase in number of threads the contention increases.<sup><span id="7body">[[#7foot|[7]]]</span></sup>]]<br />
<br />
== Sense-Reversal Barrier ==<br />
<br />
This barrier is a centralized barrier where a single count variable protected by a lock is shared among all the threads. Each thread on reaching the barrier increments its value and waits till the value of variable has reached the number of threads for which barrier was implemented.<br />
<br />
Since all the threads are spinning around a single variable the miss rate is scales quadratically with number of processors.<br />
<br />
[[Image:Ch9wiki01.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Combining Tree Barrier ==<br />
<br />
This barrier is a distributed barrier where group of processors form clusters and updates value of a local variable. The local variable on reaching a value equal to number of threads updating it, proceeds to increment another variable higher in hierarchy in the combining tree. When the variable in the highest level of hierarchy in the combining tree reaches its max value it is considered that all the threads have reached the barrier and synchronization is complete.<br />
<br />
Since in this case all the threads are updated local variables in form of smaller groups the miss rate is not as high as sense-reversal barrier.<br />
<br />
The diagram below shows the operation of combining tree barrier with threads grouped in two groups. The variables C0 and C1 are local to each group and C2 is the variable that is at higher level of hierarchy in the tree.<br />
<br />
[[Image:Ch9wiki02.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Tournament Barrier ==<br />
<br />
In tournament barrier the threads are considered to be leaves at the end of a binary tree and each node represents a flag. Two threads compete with each other and the loser thread is allowed to set the flag and move to higher level and compete to lose with the loser thread from other section of binary tree. Thus the thread which completes last is able to set the highest flag in the binary tree. On setting the flag it indicates to all the threads that barrier has been completed and thus synchronization is achieved.<br />
<br />
[[Image:Ch9wiki03.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Disseminating Barrier ==<br />
<br />
In this barrier each thread maintains a record of the activity of other threads. For every round i with n threads, thread A notifies thread (A + 2i) mod n. Thus after logn rounds all the threads are aware of the status of every other thread running and whether it has reached the barrier.<br />
<br />
[[Image:Ch9wiki04.png]]<sup><span id="9body">[[#9foot|[9]]]</span></sup><br />
<br />
= References = <br />
<span id="1foot">[[#1body|1.]]</span> David Culler, J.P. Singh and Anoop Gupta. Parallel Computer Architecture: A Hardware/Software Approach. Morgan Kaufmann. August 1998.<br><br />
<span id="2foot">[[#2body|2.]]</span> Yan Solihin. Fundamentals of Parallel Computer Architecture: Multichip and Multicore Systems. Solihin Books. August 2009.<br><br />
<span id="3foot">[[#3body|3.]]</span> Thomas E. Anderson. The Performance of Spin Lock Alternatives for Shared-memory Multiprocessors. http://www.cc.gatech.edu/classes/AY2010/cs4210_fall/papers/anderson-spinlock.pdf<br><br />
<span id="4foot">[[#4body|4.]]</span> http://www.statemaster.com/encyclopedia/Lock-(computer-science)<br><br />
<span id="5foot">[[#5body|5.]]</span> http://faydoc.tripod.com/cpu/cmpxchg.htm<br><br />
<span id="6foot">[[#6body|6.]]</span> http://www.cs.duke.edu/courses/fall09/cps110/slides/threads3.ppt<br><br />
<span id="7foot">[[#7body|7.]]</span> Scalability evaluation of barrier algorithms for OpenMP. http://www2.cs.uh.edu/~hpctools/pub/iwomp-barrier.pdf<br><br />
<span id="8foot">[[#8body|8.]]</span> Carwyn Ball and Mark Bull. Barrier Synchronization in Java. http://www.ukhec.ac.uk/publications/reports/synch_java.pdf<br><br />
<span id="9foot">[[#9body|9.]]</span> http://www.cs.brown.edu/courses/cs176/barrier.ppt<br><br />
<span id="10foot">[[#10body|10.]]</span>http://heather.cs.ucdavis.edu/~matloff/50/PLN/lock.pdf<br><br />
<span id="11foot">[[#11body|11.]]</span>http://www.cs.duke.edu/courses/fall09/cps110/handouts/threads3.pdf<br><br />
<span id="12foot">[[#12body|12.]]</span>http://preshing.com/20120612/an-introduction-to-lock-free-programming<br></div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/9b_sc&diff=74670CSC/ECE 506 Spring 2013/9b sc2013-04-03T23:23:01Z<p>Schakra8: /* Avoiding Locks */</p>
<hr />
<div>= Synchronization =<br />
In addition to a proper cache coherency model, it is important for a multiprocessor system to provide support for synchronization. This must be provided at the hardware level, and can also be implemented to some degree in software. The most common types of hardware synchronization are locks and barriers, which are discussed in this chapter.<br />
<br />
== Hardware vs. Operating System Synchronization ==<br />
Synchronizations are characterized by three parts: acquire, wait, and release. Both hardware and operating system synchronization rely on atomic instructions for acquire and release, but the implementations for the wait portion of a synchronization can vary. Hardware implementations usually use busy-waiting, where a process repeatedly tests a variable waiting for it to change. The operating system can wait using blocking, where the process suspends itself until woken up by another process.<br />
<br />
Blocking has the advantage of freeing up the processor for use by other processes, but requires operating system functions to work and thus has a lot of overhead. Busy-waiting has much less overhead, but uses processor and cache resources while waiting. Because of these tradeoffs, it is usually better to use busy-waiting for short wait periods, and blocking for long wait periods.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
= Lock Implementations =<br />
<br />
Locks are an important concept when programming for multiprocessor systems. The purpose of a lock is to protect the code inside the lock, known as a critical section. It must be certain that while some thread X has entered the critical section, another thread Y is not also inside of the critical section and possibly modifying critical values. While thread X has entered the critical section, thread Y must wait until X has exited before entering. This can be accomplished in a variety of ways of varying complexity and performance.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these four criteria:<br />
<br />
*Acquisition Latency - How much time does it take to acquire the lock?<br />
*Traffic - How much bus traffic is generated by threads attempting to acquire the lock?<br />
*Fairness - FIFO vs. Luck<br />
*Storage - How much storage is needed compared to the number of threads?<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Acquisition Latency ===<br />
We want a low acquisition latency, especially for applications that repeatedly acquire and release locks. Over the run time of a program, the acquisition latency compounded many times could have a large performance affect. However, low acquisition latency has to be balanced against other factors when implementing a lock.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
Traffic is an important consideration when evaluating lock performance. If acquiring lock causes a lot of bus traffic, it will not scale well as the number of threads increases. Eventually the bus will become choked.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Fairness ===<br />
In general, threads should acquire a lock in them same order that the locks were requested. If this does not happen, as the number of threads increases so does the chance that a thread will become starved.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Storage ===<br />
The amount of storage needed per lock should be small enough such that it scales to a large number of threads without any problems. If a large amount of storage is required, then multiple threads could cause locks to consume too much memory.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
<br />
Two common lock implementations are test&set and test and test&set. In terms of performance, both have a low storage cost (scalable) and are not fair. Test&set has a low acquisition latency but is not scalable due do high bus traffic when there is a lot of contention for the lock. Test-and-test&set has a higher acquisition latency but scales better because it generates less bus traffic. The next figure shows performance comparisons for these two lock implementations.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
[[Image:lockperf.png|frame|none|Performance comparison of test&set and test-and-test&set (spin on read)<sup><span id="3body">[[#3foot|[3]]]</span></sup>]]<br />
<br />
=== Improving performance ===<br />
Optimizing software can improve the performance of locks. For instance, inserting a delay between a processor noticing a release and then trying to acquire a lock can reduce contention and bus traffic, and increase the performance of the locks. TCP-like back off algorithms can be used to adjust this delay depending on the number of threads contending for the lock.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
Another way to improve performance is to insert a delay between memory references so as to limit the bandwidth (traffic) each processor can use for spinning.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
One way to guarantee fairness is to queue lock attempts in shared memory so that they happen in order.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
== Atomic Instructions ==<br />
<br />
Since a multiprocessor system cannot disable interrupts as an effective method to execute code atomically, there must be hardware support for atomic operations.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
The ability to execute an atomic instruction is a requirement for most lock implementations. It is important that when a processor attempts to set a lock, either the lock is fully set and the thread is able to enter into the critical section, or the lock is not set and it appears that none of the instructions required to set the lock have executed. <br />
<br />
In the x86 instruction set the opcode CMPXCHG (compare and exchange) can be used in a lock implementation in order to guarantee atomicity. This function works by sending a destination and a source. The accumulator is compared to the destination and, if they are equal, loaded with the source. If they are not equal the accumulator is loaded with the destination value. In order to assure that this is executed atomically the opcode must be issued with the LOCK prefix. This is useful in implementing some locks, such as ticket locks.<sup><span id="5body">[[#5foot|[5]]]</span></sup> The syntax for the instruction is LOCK CMPXCHG SOURCE DESTINATION. Here, the source has to be a register while the destination can be either Register or Memory. An example of LOCK CMPXCHG is implementation in dis-assembly of Linux kernel's rtc_cmos_read(). The CPU locks needs to be copied and and then updated to a non zero value if the lock is to be applied. But there is a possibility when the value got copied earlier and changed later. This causes a race condition. Hence,it helps in briefing the busy wait loop by preventing race condition. The pseudo code for same is given as below<sup><span id="10body">[[#10foot|[10]]]</span></sup>:<br />
<br />
spin: # see if the lock-variable is clear<br />
mov cmos_lock, %eax<br />
test %eax, %eax<br />
jnz spin<br />
# ok, now we try to grab the lock<br />
lock cmpxchg %edx, cmos_lock<br />
# did another CPU grab it first? <br />
test %eax, %eax<br />
jnz spin<br />
<br />
== Hand-off Lock ==<br />
<br />
Another type of lock that is not discussed in the text is known as the "hand-off" lock. In this lock the first thread acquires the lock if no other thread is currently locked (since it is the first thread). When another thread attempts to gain the lock it will see that the lock is in use and adds itself to the queue. Once done this thread can sleep until called by the thread with the lock. Once the thread in the lock is finished, it will pass the lock to the next thread in the queue<sup><span id="6body">[[#6foot|[6]]]</span></sup>. Waiting thread gives up processor so that other threads (e.g. the thread with the lock) can run more quickly. Someone wakes up thread when the lock is free. The separation of ready queue from that of waiting queue fastens the process of execution. In the below example, the lock is given to a waiting thread by the unlock method. The interrupts are enabled only when the thread is placed in waiting queue and disabled once its given the lock. <br />
<br />
lock () {<br />
disable interrupts<br />
if (value == FREE) {<br />
value = BUSY<br />
} else {<br />
add thread to queue of threads waiting for<br />
this lock<br />
switch to next runnable thread<br />
}<br />
enable interrupts<br />
}<br />
unlock () {<br />
disable interrupts<br />
value = FREE<br />
if (any thread is waiting for this lock) {<br />
move waiting thread from waiting queue to<br />
ready queue<br />
value = BUSY<br />
}<br />
enable interrupts<br />
} <br />
<br />
But this fails if interrupt happens after thread enable interrupts Lock() adds thread to wait queue Lock() enables interrupts Interrupts causes pre-emption, i.e. switch to another thread. Pre-emption moves thread to ready queue. Now thread is on two queues (wait and ready)! Also, switch is likely to be a critical section Adding thread to wait queue and switching to next thread must be atomic. Solution to this problem is waiting thread leaves interrupts disabled when it calls switch. Next thread to run has the responsibility of re-enabling interrupts before returning to user code. When waiting thread wakes up, it returns from switch with interrupts disabled (from the last thread)<sup><span id="11body">[[#11foot|[11]]]</span></sup><br />
<br />
== Avoiding Locks ==<br />
<br />
There are many reasons why a programmer should attempt to write programs in such a way as to avoid locks if possible. There are many problems that can arise with the use of locks.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
One of the must well known issues is deadlock. This can occur when the threads are waiting to acquire the lock, but the lock will never be unlocked. For example, there are two processes A and B and they need two methods (1 and 2) to be grabbed for completion of a task which are shared by both A and B. This can create a deadlock when A grabs 1 and B grabs 2. This reaches to a situation where both A and B cannot change their states. The solution to this problem is to set a condition where in A process can lock method 2 only if t has locked method 1.<br />
<br />
Another problem with using locks is that the performance is not optimal, as often a lock is used when there is only a chance of conflict. This approach to programming yields slower performance than what might be possible with other methods. This also leads to questions of granularity, that is how much of the code should be protected under the critical section. The programmer must decide between many small (fine grain) locks or fewer, more encompassing locks. This decision can greatly effect the performance of the program.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
To avoid locks, lock free programming techniques like compare and swap and sequential consistency can be followed. This pattern typically involves copying a shared variable to a local variable, performing some speculative work, and attempting to publish the changes using CAS<sup><span id="12body">[[#12foot|[12]]]</span></sup>:<br />
<br />
void LockFreeQueue::push(Node* newHead)<br />
{<br />
for (;;)<br />
{<br />
// Copy a shared variable (m_Head) to a local.<br />
Node* oldHead = m_Head;<br />
<br />
// Do some speculative work, not yet visible to other threads.<br />
newHead->next = oldHead;<br />
<br />
// Next, attempt to publish our changes to the shared variable.<br />
// If the shared variable hasn't changed, the CAS succeeds and we return.<br />
// Otherwise, repeat.<br />
if (_InterlockedCompareExchange(&m_Head, newHead, oldHead) == oldHead)<br />
return;<br />
}<br />
}<br />
<br />
= Barrier Implementations =<br />
<br />
In many ways, barriers are simpler than locks. A barrier is simply a point in a program where one or more threads must reach before the parallel program is allowed to continue. When using barriers, a programmer does not have to be concerned with advanced topics such as fairness, that are required when programming for locks.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these two criteria:<br />
<br />
*Latency - The time required to enter and exit a barrier.<br />
*Traffic - Communications overhead required by the barrier.<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Latency ===<br />
<br />
Ideally the latency of a barrier should be small, but for barriers that don't scale well latency can increase as the number of threads increases.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
<br />
Low traffic is also good, as we do not want excess buss traffic to prevent scalability.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
{| class="wikitable" border="1"<br />
|+Performance Comparison of Barrier Implementations<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
|-<br />
! Criteria<br />
! Sense Reversal<br />
Centralized Barrier<br />
! Combining Tree<br />
Barrier<br />
! Barrier Network<br />
(Hardware)<br />
|-<br />
| Latency<br />
| O(1)<br />
| O(log p)<br />
| O(log p)<br />
|-<br />
| Traffic<br />
| O(p<sup>2</sup>)<br />
| O(p)<br />
| moved to a separate network<br />
|}<br />
<br />
<br />
<br />
[[Image:Ch9wiki05.png|frame|none|Additive Schwarz Preconditioned Conjugate Gradient (ASPCG) kernel on Altix System - This figure shows the timings of the ASPCG kernel using the different barrier implementations. It can be seen that the blocking barrier does not scale with number of threads as with increase in number of threads the contention increases.<sup><span id="7body">[[#7foot|[7]]]</span></sup>]]<br />
<br />
<br />
<br />
[[Image:Ch9wiki06.png|frame|none|EPCC Microbenchmark - This figure shows the timings to implement a barrier of the EPCC Microbenchmark using the different barrier implementations. It can be seen that the blocking barrier/centralized blocking barrier does not scale with number of threads as with increase in number of threads the contention increases.<sup><span id="7body">[[#7foot|[7]]]</span></sup>]]<br />
<br />
== Sense-Reversal Barrier ==<br />
<br />
This barrier is a centralized barrier where a single count variable protected by a lock is shared among all the threads. Each thread on reaching the barrier increments its value and waits till the value of variable has reached the number of threads for which barrier was implemented.<br />
<br />
Since all the threads are spinning around a single variable the miss rate is scales quadratically with number of processors.<br />
<br />
[[Image:Ch9wiki01.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Combining Tree Barrier ==<br />
<br />
This barrier is a distributed barrier where group of processors form clusters and updates value of a local variable. The local variable on reaching a value equal to number of threads updating it, proceeds to increment another variable higher in hierarchy in the combining tree. When the variable in the highest level of hierarchy in the combining tree reaches its max value it is considered that all the threads have reached the barrier and synchronization is complete.<br />
<br />
Since in this case all the threads are updated local variables in form of smaller groups the miss rate is not as high as sense-reversal barrier.<br />
<br />
The diagram below shows the operation of combining tree barrier with threads grouped in two groups. The variables C0 and C1 are local to each group and C2 is the variable that is at higher level of hierarchy in the tree.<br />
<br />
[[Image:Ch9wiki02.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Tournament Barrier ==<br />
<br />
In tournament barrier the threads are considered to be leaves at the end of a binary tree and each node represents a flag. Two threads compete with each other and the loser thread is allowed to set the flag and move to higher level and compete to lose with the loser thread from other section of binary tree. Thus the thread which completes last is able to set the highest flag in the binary tree. On setting the flag it indicates to all the threads that barrier has been completed and thus synchronization is achieved.<br />
<br />
[[Image:Ch9wiki03.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Disseminating Barrier ==<br />
<br />
In this barrier each thread maintains a record of the activity of other threads. For every round i with n threads, thread A notifies thread (A + 2i) mod n. Thus after logn rounds all the threads are aware of the status of every other thread running and whether it has reached the barrier.<br />
<br />
[[Image:Ch9wiki04.png]]<sup><span id="9body">[[#9foot|[9]]]</span></sup><br />
<br />
= References = <br />
<span id="1foot">[[#1body|1.]]</span> David Culler, J.P. Singh and Anoop Gupta. Parallel Computer Architecture: A Hardware/Software Approach. Morgan Kaufmann. August 1998.<br><br />
<span id="2foot">[[#2body|2.]]</span> Yan Solihin. Fundamentals of Parallel Computer Architecture: Multichip and Multicore Systems. Solihin Books. August 2009.<br><br />
<span id="3foot">[[#3body|3.]]</span> Thomas E. Anderson. The Performance of Spin Lock Alternatives for Shared-memory Multiprocessors. http://www.cc.gatech.edu/classes/AY2010/cs4210_fall/papers/anderson-spinlock.pdf<br><br />
<span id="4foot">[[#4body|4.]]</span> http://www.statemaster.com/encyclopedia/Lock-(computer-science)<br><br />
<span id="5foot">[[#5body|5.]]</span> http://faydoc.tripod.com/cpu/cmpxchg.htm<br><br />
<span id="6foot">[[#6body|6.]]</span> http://www.cs.duke.edu/courses/fall09/cps110/slides/threads3.ppt<br><br />
<span id="7foot">[[#7body|7.]]</span> Scalability evaluation of barrier algorithms for OpenMP. http://www2.cs.uh.edu/~hpctools/pub/iwomp-barrier.pdf<br><br />
<span id="8foot">[[#8body|8.]]</span> Carwyn Ball and Mark Bull. Barrier Synchronization in Java. http://www.ukhec.ac.uk/publications/reports/synch_java.pdf<br><br />
<span id="9foot">[[#9body|9.]]</span> http://www.cs.brown.edu/courses/cs176/barrier.ppt<br><br />
<span id="10foot">[[#10body|10.]]</span>http://heather.cs.ucdavis.edu/~matloff/50/PLN/lock.pdf<br><br />
<span id="11foot">[[#11body|11.]]</span>http://www.cs.duke.edu/courses/fall09/cps110/handouts/threads3.pdf<br></div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/9b_sc&diff=74658CSC/ECE 506 Spring 2013/9b sc2013-04-03T20:46:46Z<p>Schakra8: /* Avoiding Locks */</p>
<hr />
<div>= Synchronization =<br />
In addition to a proper cache coherency model, it is important for a multiprocessor system to provide support for synchronization. This must be provided at the hardware level, and can also be implemented to some degree in software. The most common types of hardware synchronization are locks and barriers, which are discussed in this chapter.<br />
<br />
== Hardware vs. Operating System Synchronization ==<br />
Synchronizations are characterized by three parts: acquire, wait, and release. Both hardware and operating system synchronization rely on atomic instructions for acquire and release, but the implementations for the wait portion of a synchronization can vary. Hardware implementations usually use busy-waiting, where a process repeatedly tests a variable waiting for it to change. The operating system can wait using blocking, where the process suspends itself until woken up by another process.<br />
<br />
Blocking has the advantage of freeing up the processor for use by other processes, but requires operating system functions to work and thus has a lot of overhead. Busy-waiting has much less overhead, but uses processor and cache resources while waiting. Because of these tradeoffs, it is usually better to use busy-waiting for short wait periods, and blocking for long wait periods.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
= Lock Implementations =<br />
<br />
Locks are an important concept when programming for multiprocessor systems. The purpose of a lock is to protect the code inside the lock, known as a critical section. It must be certain that while some thread X has entered the critical section, another thread Y is not also inside of the critical section and possibly modifying critical values. While thread X has entered the critical section, thread Y must wait until X has exited before entering. This can be accomplished in a variety of ways of varying complexity and performance.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these four criteria:<br />
<br />
*Acquisition Latency - How much time does it take to acquire the lock?<br />
*Traffic - How much bus traffic is generated by threads attempting to acquire the lock?<br />
*Fairness - FIFO vs. Luck<br />
*Storage - How much storage is needed compared to the number of threads?<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Acquisition Latency ===<br />
We want a low acquisition latency, especially for applications that repeatedly acquire and release locks. Over the run time of a program, the acquisition latency compounded many times could have a large performance affect. However, low acquisition latency has to be balanced against other factors when implementing a lock.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
Traffic is an important consideration when evaluating lock performance. If acquiring lock causes a lot of bus traffic, it will not scale well as the number of threads increases. Eventually the bus will become choked.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Fairness ===<br />
In general, threads should acquire a lock in them same order that the locks were requested. If this does not happen, as the number of threads increases so does the chance that a thread will become starved.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Storage ===<br />
The amount of storage needed per lock should be small enough such that it scales to a large number of threads without any problems. If a large amount of storage is required, then multiple threads could cause locks to consume too much memory.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
<br />
Two common lock implementations are test&set and test and test&set. In terms of performance, both have a low storage cost (scalable) and are not fair. Test&set has a low acquisition latency but is not scalable due do high bus traffic when there is a lot of contention for the lock. Test-and-test&set has a higher acquisition latency but scales better because it generates less bus traffic. The next figure shows performance comparisons for these two lock implementations.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
[[Image:lockperf.png|frame|none|Performance comparison of test&set and test-and-test&set (spin on read)<sup><span id="3body">[[#3foot|[3]]]</span></sup>]]<br />
<br />
=== Improving performance ===<br />
Optimizing software can improve the performance of locks. For instance, inserting a delay between a processor noticing a release and then trying to acquire a lock can reduce contention and bus traffic, and increase the performance of the locks. TCP-like back off algorithms can be used to adjust this delay depending on the number of threads contending for the lock.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
Another way to improve performance is to insert a delay between memory references so as to limit the bandwidth (traffic) each processor can use for spinning.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
One way to guarantee fairness is to queue lock attempts in shared memory so that they happen in order.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
== Atomic Instructions ==<br />
<br />
Since a multiprocessor system cannot disable interrupts as an effective method to execute code atomically, there must be hardware support for atomic operations.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
The ability to execute an atomic instruction is a requirement for most lock implementations. It is important that when a processor attempts to set a lock, either the lock is fully set and the thread is able to enter into the critical section, or the lock is not set and it appears that none of the instructions required to set the lock have executed. <br />
<br />
In the x86 instruction set the opcode CMPXCHG (compare and exchange) can be used in a lock implementation in order to guarantee atomicity. This function works by sending a destination and a source. The accumulator is compared to the destination and, if they are equal, loaded with the source. If they are not equal the accumulator is loaded with the destination value. In order to assure that this is executed atomically the opcode must be issued with the LOCK prefix. This is useful in implementing some locks, such as ticket locks.<sup><span id="5body">[[#5foot|[5]]]</span></sup> The syntax for the instruction is LOCK CMPXCHG SOURCE DESTINATION. Here, the source has to be a register while the destination can be either Register or Memory. An example of LOCK CMPXCHG is implementation in dis-assembly of Linux kernel's rtc_cmos_read(). The CPU locks needs to be copied and and then updated to a non zero value if the lock is to be applied. But there is a possibility when the value got copied earlier and changed later. This causes a race condition. Hence,it helps in briefing the busy wait loop by preventing race condition. The pseudo code for same is given as below<sup><span id="10body">[[#10foot|[10]]]</span></sup>:<br />
<br />
spin: # see if the lock-variable is clear<br />
mov cmos_lock, %eax<br />
test %eax, %eax<br />
jnz spin<br />
# ok, now we try to grab the lock<br />
lock cmpxchg %edx, cmos_lock<br />
# did another CPU grab it first? <br />
test %eax, %eax<br />
jnz spin<br />
<br />
== Hand-off Lock ==<br />
<br />
Another type of lock that is not discussed in the text is known as the "hand-off" lock. In this lock the first thread acquires the lock if no other thread is currently locked (since it is the first thread). When another thread attempts to gain the lock it will see that the lock is in use and adds itself to the queue. Once done this thread can sleep until called by the thread with the lock. Once the thread in the lock is finished, it will pass the lock to the next thread in the queue<sup><span id="6body">[[#6foot|[6]]]</span></sup>. Waiting thread gives up processor so that other threads (e.g. the thread with the lock) can run more quickly. Someone wakes up thread when the lock is free. The separation of ready queue from that of waiting queue fastens the process of execution. In the below example, the lock is given to a waiting thread by the unlock method. The interrupts are enabled only when the thread is placed in waiting queue and disabled once its given the lock. <br />
<br />
lock () {<br />
disable interrupts<br />
if (value == FREE) {<br />
value = BUSY<br />
} else {<br />
add thread to queue of threads waiting for<br />
this lock<br />
switch to next runnable thread<br />
}<br />
enable interrupts<br />
}<br />
unlock () {<br />
disable interrupts<br />
value = FREE<br />
if (any thread is waiting for this lock) {<br />
move waiting thread from waiting queue to<br />
ready queue<br />
value = BUSY<br />
}<br />
enable interrupts<br />
} <br />
<br />
But this fails if interrupt happens after thread enable interrupts Lock() adds thread to wait queue Lock() enables interrupts Interrupts causes pre-emption, i.e. switch to another thread. Pre-emption moves thread to ready queue. Now thread is on two queues (wait and ready)! Also, switch is likely to be a critical section Adding thread to wait queue and switching to next thread must be atomic. Solution to this problem is waiting thread leaves interrupts disabled when it calls switch. Next thread to run has the responsibility of re-enabling interrupts before returning to user code. When waiting thread wakes up, it returns from switch with interrupts disabled (from the last thread)<sup><span id="11body">[[#11foot|[11]]]</span></sup><br />
<br />
== Avoiding Locks ==<br />
<br />
There are many reasons why a programmer should attempt to write programs in such a way as to avoid locks if possible. There are many problems that can arise with the use of locks.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
One of the must well known issues is deadlock. This can occur when the threads are waiting to acquire the lock, but the lock will never be unlocked. For example, there are two processes A and B and they need two methods (1 and 2) to be grabbed for completion of a task which are shared by both A and B. This can create a deadlock when A grabs 1 and B grabs 2. This reaches to a situation where both A and B cannot change their states. The solution to this problem is to set a condition where in A process can lock method 2 only if t has locked method 1.<br />
<br />
Another problem with using locks is that the performance is not optimal, as often a lock is used when there is only a chance of conflict. This approach to programming yields slower performance than what might be possible with other methods. This also leads to questions of granularity, that is how much of the code should be protected under the critical section. The programmer must decide between many small (fine grain) locks or fewer, more encompassing locks. This decision can greatly effect the performance of the program.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
= Barrier Implementations =<br />
<br />
In many ways, barriers are simpler than locks. A barrier is simply a point in a program where one or more threads must reach before the parallel program is allowed to continue. When using barriers, a programmer does not have to be concerned with advanced topics such as fairness, that are required when programming for locks.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these two criteria:<br />
<br />
*Latency - The time required to enter and exit a barrier.<br />
*Traffic - Communications overhead required by the barrier.<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Latency ===<br />
<br />
Ideally the latency of a barrier should be small, but for barriers that don't scale well latency can increase as the number of threads increases.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
<br />
Low traffic is also good, as we do not want excess buss traffic to prevent scalability.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
{| class="wikitable" border="1"<br />
|+Performance Comparison of Barrier Implementations<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
|-<br />
! Criteria<br />
! Sense Reversal<br />
Centralized Barrier<br />
! Combining Tree<br />
Barrier<br />
! Barrier Network<br />
(Hardware)<br />
|-<br />
| Latency<br />
| O(1)<br />
| O(log p)<br />
| O(log p)<br />
|-<br />
| Traffic<br />
| O(p<sup>2</sup>)<br />
| O(p)<br />
| moved to a separate network<br />
|}<br />
<br />
<br />
<br />
[[Image:Ch9wiki05.png|frame|none|Additive Schwarz Preconditioned Conjugate Gradient (ASPCG) kernel on Altix System - This figure shows the timings of the ASPCG kernel using the different barrier implementations. It can be seen that the blocking barrier does not scale with number of threads as with increase in number of threads the contention increases.<sup><span id="7body">[[#7foot|[7]]]</span></sup>]]<br />
<br />
<br />
<br />
[[Image:Ch9wiki06.png|frame|none|EPCC Microbenchmark - This figure shows the timings to implement a barrier of the EPCC Microbenchmark using the different barrier implementations. It can be seen that the blocking barrier/centralized blocking barrier does not scale with number of threads as with increase in number of threads the contention increases.<sup><span id="7body">[[#7foot|[7]]]</span></sup>]]<br />
<br />
== Sense-Reversal Barrier ==<br />
<br />
This barrier is a centralized barrier where a single count variable protected by a lock is shared among all the threads. Each thread on reaching the barrier increments its value and waits till the value of variable has reached the number of threads for which barrier was implemented.<br />
<br />
Since all the threads are spinning around a single variable the miss rate is scales quadratically with number of processors.<br />
<br />
[[Image:Ch9wiki01.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Combining Tree Barrier ==<br />
<br />
This barrier is a distributed barrier where group of processors form clusters and updates value of a local variable. The local variable on reaching a value equal to number of threads updating it, proceeds to increment another variable higher in hierarchy in the combining tree. When the variable in the highest level of hierarchy in the combining tree reaches its max value it is considered that all the threads have reached the barrier and synchronization is complete.<br />
<br />
Since in this case all the threads are updated local variables in form of smaller groups the miss rate is not as high as sense-reversal barrier.<br />
<br />
The diagram below shows the operation of combining tree barrier with threads grouped in two groups. The variables C0 and C1 are local to each group and C2 is the variable that is at higher level of hierarchy in the tree.<br />
<br />
[[Image:Ch9wiki02.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Tournament Barrier ==<br />
<br />
In tournament barrier the threads are considered to be leaves at the end of a binary tree and each node represents a flag. Two threads compete with each other and the loser thread is allowed to set the flag and move to higher level and compete to lose with the loser thread from other section of binary tree. Thus the thread which completes last is able to set the highest flag in the binary tree. On setting the flag it indicates to all the threads that barrier has been completed and thus synchronization is achieved.<br />
<br />
[[Image:Ch9wiki03.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Disseminating Barrier ==<br />
<br />
In this barrier each thread maintains a record of the activity of other threads. For every round i with n threads, thread A notifies thread (A + 2i) mod n. Thus after logn rounds all the threads are aware of the status of every other thread running and whether it has reached the barrier.<br />
<br />
[[Image:Ch9wiki04.png]]<sup><span id="9body">[[#9foot|[9]]]</span></sup><br />
<br />
= References = <br />
<span id="1foot">[[#1body|1.]]</span> David Culler, J.P. Singh and Anoop Gupta. Parallel Computer Architecture: A Hardware/Software Approach. Morgan Kaufmann. August 1998.<br><br />
<span id="2foot">[[#2body|2.]]</span> Yan Solihin. Fundamentals of Parallel Computer Architecture: Multichip and Multicore Systems. Solihin Books. August 2009.<br><br />
<span id="3foot">[[#3body|3.]]</span> Thomas E. Anderson. The Performance of Spin Lock Alternatives for Shared-memory Multiprocessors. http://www.cc.gatech.edu/classes/AY2010/cs4210_fall/papers/anderson-spinlock.pdf<br><br />
<span id="4foot">[[#4body|4.]]</span> http://www.statemaster.com/encyclopedia/Lock-(computer-science)<br><br />
<span id="5foot">[[#5body|5.]]</span> http://faydoc.tripod.com/cpu/cmpxchg.htm<br><br />
<span id="6foot">[[#6body|6.]]</span> http://www.cs.duke.edu/courses/fall09/cps110/slides/threads3.ppt<br><br />
<span id="7foot">[[#7body|7.]]</span> Scalability evaluation of barrier algorithms for OpenMP. http://www2.cs.uh.edu/~hpctools/pub/iwomp-barrier.pdf<br><br />
<span id="8foot">[[#8body|8.]]</span> Carwyn Ball and Mark Bull. Barrier Synchronization in Java. http://www.ukhec.ac.uk/publications/reports/synch_java.pdf<br><br />
<span id="9foot">[[#9body|9.]]</span> http://www.cs.brown.edu/courses/cs176/barrier.ppt<br><br />
<span id="10foot">[[#10body|10.]]</span>http://heather.cs.ucdavis.edu/~matloff/50/PLN/lock.pdf<br><br />
<span id="11foot">[[#11body|11.]]</span>http://www.cs.duke.edu/courses/fall09/cps110/handouts/threads3.pdf<br></div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/9b_sc&diff=74657CSC/ECE 506 Spring 2013/9b sc2013-04-03T20:35:20Z<p>Schakra8: /* References */</p>
<hr />
<div>= Synchronization =<br />
In addition to a proper cache coherency model, it is important for a multiprocessor system to provide support for synchronization. This must be provided at the hardware level, and can also be implemented to some degree in software. The most common types of hardware synchronization are locks and barriers, which are discussed in this chapter.<br />
<br />
== Hardware vs. Operating System Synchronization ==<br />
Synchronizations are characterized by three parts: acquire, wait, and release. Both hardware and operating system synchronization rely on atomic instructions for acquire and release, but the implementations for the wait portion of a synchronization can vary. Hardware implementations usually use busy-waiting, where a process repeatedly tests a variable waiting for it to change. The operating system can wait using blocking, where the process suspends itself until woken up by another process.<br />
<br />
Blocking has the advantage of freeing up the processor for use by other processes, but requires operating system functions to work and thus has a lot of overhead. Busy-waiting has much less overhead, but uses processor and cache resources while waiting. Because of these tradeoffs, it is usually better to use busy-waiting for short wait periods, and blocking for long wait periods.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
= Lock Implementations =<br />
<br />
Locks are an important concept when programming for multiprocessor systems. The purpose of a lock is to protect the code inside the lock, known as a critical section. It must be certain that while some thread X has entered the critical section, another thread Y is not also inside of the critical section and possibly modifying critical values. While thread X has entered the critical section, thread Y must wait until X has exited before entering. This can be accomplished in a variety of ways of varying complexity and performance.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these four criteria:<br />
<br />
*Acquisition Latency - How much time does it take to acquire the lock?<br />
*Traffic - How much bus traffic is generated by threads attempting to acquire the lock?<br />
*Fairness - FIFO vs. Luck<br />
*Storage - How much storage is needed compared to the number of threads?<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Acquisition Latency ===<br />
We want a low acquisition latency, especially for applications that repeatedly acquire and release locks. Over the run time of a program, the acquisition latency compounded many times could have a large performance affect. However, low acquisition latency has to be balanced against other factors when implementing a lock.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
Traffic is an important consideration when evaluating lock performance. If acquiring lock causes a lot of bus traffic, it will not scale well as the number of threads increases. Eventually the bus will become choked.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Fairness ===<br />
In general, threads should acquire a lock in them same order that the locks were requested. If this does not happen, as the number of threads increases so does the chance that a thread will become starved.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Storage ===<br />
The amount of storage needed per lock should be small enough such that it scales to a large number of threads without any problems. If a large amount of storage is required, then multiple threads could cause locks to consume too much memory.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
<br />
Two common lock implementations are test&set and test and test&set. In terms of performance, both have a low storage cost (scalable) and are not fair. Test&set has a low acquisition latency but is not scalable due do high bus traffic when there is a lot of contention for the lock. Test-and-test&set has a higher acquisition latency but scales better because it generates less bus traffic. The next figure shows performance comparisons for these two lock implementations.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
[[Image:lockperf.png|frame|none|Performance comparison of test&set and test-and-test&set (spin on read)<sup><span id="3body">[[#3foot|[3]]]</span></sup>]]<br />
<br />
=== Improving performance ===<br />
Optimizing software can improve the performance of locks. For instance, inserting a delay between a processor noticing a release and then trying to acquire a lock can reduce contention and bus traffic, and increase the performance of the locks. TCP-like back off algorithms can be used to adjust this delay depending on the number of threads contending for the lock.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
Another way to improve performance is to insert a delay between memory references so as to limit the bandwidth (traffic) each processor can use for spinning.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
One way to guarantee fairness is to queue lock attempts in shared memory so that they happen in order.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
== Atomic Instructions ==<br />
<br />
Since a multiprocessor system cannot disable interrupts as an effective method to execute code atomically, there must be hardware support for atomic operations.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
The ability to execute an atomic instruction is a requirement for most lock implementations. It is important that when a processor attempts to set a lock, either the lock is fully set and the thread is able to enter into the critical section, or the lock is not set and it appears that none of the instructions required to set the lock have executed. <br />
<br />
In the x86 instruction set the opcode CMPXCHG (compare and exchange) can be used in a lock implementation in order to guarantee atomicity. This function works by sending a destination and a source. The accumulator is compared to the destination and, if they are equal, loaded with the source. If they are not equal the accumulator is loaded with the destination value. In order to assure that this is executed atomically the opcode must be issued with the LOCK prefix. This is useful in implementing some locks, such as ticket locks.<sup><span id="5body">[[#5foot|[5]]]</span></sup> The syntax for the instruction is LOCK CMPXCHG SOURCE DESTINATION. Here, the source has to be a register while the destination can be either Register or Memory. An example of LOCK CMPXCHG is implementation in dis-assembly of Linux kernel's rtc_cmos_read(). The CPU locks needs to be copied and and then updated to a non zero value if the lock is to be applied. But there is a possibility when the value got copied earlier and changed later. This causes a race condition. Hence,it helps in briefing the busy wait loop by preventing race condition. The pseudo code for same is given as below<sup><span id="10body">[[#10foot|[10]]]</span></sup>:<br />
<br />
spin: # see if the lock-variable is clear<br />
mov cmos_lock, %eax<br />
test %eax, %eax<br />
jnz spin<br />
# ok, now we try to grab the lock<br />
lock cmpxchg %edx, cmos_lock<br />
# did another CPU grab it first? <br />
test %eax, %eax<br />
jnz spin<br />
<br />
== Hand-off Lock ==<br />
<br />
Another type of lock that is not discussed in the text is known as the "hand-off" lock. In this lock the first thread acquires the lock if no other thread is currently locked (since it is the first thread). When another thread attempts to gain the lock it will see that the lock is in use and adds itself to the queue. Once done this thread can sleep until called by the thread with the lock. Once the thread in the lock is finished, it will pass the lock to the next thread in the queue<sup><span id="6body">[[#6foot|[6]]]</span></sup>. Waiting thread gives up processor so that other threads (e.g. the thread with the lock) can run more quickly. Someone wakes up thread when the lock is free. The separation of ready queue from that of waiting queue fastens the process of execution. In the below example, the lock is given to a waiting thread by the unlock method. The interrupts are enabled only when the thread is placed in waiting queue and disabled once its given the lock. <br />
<br />
lock () {<br />
disable interrupts<br />
if (value == FREE) {<br />
value = BUSY<br />
} else {<br />
add thread to queue of threads waiting for<br />
this lock<br />
switch to next runnable thread<br />
}<br />
enable interrupts<br />
}<br />
unlock () {<br />
disable interrupts<br />
value = FREE<br />
if (any thread is waiting for this lock) {<br />
move waiting thread from waiting queue to<br />
ready queue<br />
value = BUSY<br />
}<br />
enable interrupts<br />
} <br />
<br />
But this fails if interrupt happens after thread enable interrupts Lock() adds thread to wait queue Lock() enables interrupts Interrupts causes pre-emption, i.e. switch to another thread. Pre-emption moves thread to ready queue. Now thread is on two queues (wait and ready)! Also, switch is likely to be a critical section Adding thread to wait queue and switching to next thread must be atomic. Solution to this problem is waiting thread leaves interrupts disabled when it calls switch. Next thread to run has the responsibility of re-enabling interrupts before returning to user code. When waiting thread wakes up, it returns from switch with interrupts disabled (from the last thread)<sup><span id="11body">[[#11foot|[11]]]</span></sup><br />
<br />
== Avoiding Locks ==<br />
<br />
There are many reasons why a programmer should attempt to write programs in such a way as to avoid locks if possible. There are many problems that can arise with the use of locks.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
One of the must well known issues is deadlock. This can occur when the threads are waiting to acquire the lock, but the lock will never be unlocked. For example if 2 threads are both spinning on a lock that is locked, they will continue to spin forever, as each thread 'thinks' that the other is inside the critical section.<br />
<br />
Another problem with using locks is that the performance is not optimal, as often a lock is used when there is only a chance of conflict. This approach to programming yields slower performance than what might be possible with other methods. This also leads to questions of granularity, that is how much of the code should be protected under the critical section. The programmer must decide between many small (fine grain) locks or fewer, more encompassing locks. This decision can greatly effect the performance of the program.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
= Barrier Implementations =<br />
<br />
In many ways, barriers are simpler than locks. A barrier is simply a point in a program where one or more threads must reach before the parallel program is allowed to continue. When using barriers, a programmer does not have to be concerned with advanced topics such as fairness, that are required when programming for locks.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these two criteria:<br />
<br />
*Latency - The time required to enter and exit a barrier.<br />
*Traffic - Communications overhead required by the barrier.<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Latency ===<br />
<br />
Ideally the latency of a barrier should be small, but for barriers that don't scale well latency can increase as the number of threads increases.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
<br />
Low traffic is also good, as we do not want excess buss traffic to prevent scalability.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
{| class="wikitable" border="1"<br />
|+Performance Comparison of Barrier Implementations<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
|-<br />
! Criteria<br />
! Sense Reversal<br />
Centralized Barrier<br />
! Combining Tree<br />
Barrier<br />
! Barrier Network<br />
(Hardware)<br />
|-<br />
| Latency<br />
| O(1)<br />
| O(log p)<br />
| O(log p)<br />
|-<br />
| Traffic<br />
| O(p<sup>2</sup>)<br />
| O(p)<br />
| moved to a separate network<br />
|}<br />
<br />
<br />
<br />
[[Image:Ch9wiki05.png|frame|none|Additive Schwarz Preconditioned Conjugate Gradient (ASPCG) kernel on Altix System - This figure shows the timings of the ASPCG kernel using the different barrier implementations. It can be seen that the blocking barrier does not scale with number of threads as with increase in number of threads the contention increases.<sup><span id="7body">[[#7foot|[7]]]</span></sup>]]<br />
<br />
<br />
<br />
[[Image:Ch9wiki06.png|frame|none|EPCC Microbenchmark - This figure shows the timings to implement a barrier of the EPCC Microbenchmark using the different barrier implementations. It can be seen that the blocking barrier/centralized blocking barrier does not scale with number of threads as with increase in number of threads the contention increases.<sup><span id="7body">[[#7foot|[7]]]</span></sup>]]<br />
<br />
== Sense-Reversal Barrier ==<br />
<br />
This barrier is a centralized barrier where a single count variable protected by a lock is shared among all the threads. Each thread on reaching the barrier increments its value and waits till the value of variable has reached the number of threads for which barrier was implemented.<br />
<br />
Since all the threads are spinning around a single variable the miss rate is scales quadratically with number of processors.<br />
<br />
[[Image:Ch9wiki01.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Combining Tree Barrier ==<br />
<br />
This barrier is a distributed barrier where group of processors form clusters and updates value of a local variable. The local variable on reaching a value equal to number of threads updating it, proceeds to increment another variable higher in hierarchy in the combining tree. When the variable in the highest level of hierarchy in the combining tree reaches its max value it is considered that all the threads have reached the barrier and synchronization is complete.<br />
<br />
Since in this case all the threads are updated local variables in form of smaller groups the miss rate is not as high as sense-reversal barrier.<br />
<br />
The diagram below shows the operation of combining tree barrier with threads grouped in two groups. The variables C0 and C1 are local to each group and C2 is the variable that is at higher level of hierarchy in the tree.<br />
<br />
[[Image:Ch9wiki02.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Tournament Barrier ==<br />
<br />
In tournament barrier the threads are considered to be leaves at the end of a binary tree and each node represents a flag. Two threads compete with each other and the loser thread is allowed to set the flag and move to higher level and compete to lose with the loser thread from other section of binary tree. Thus the thread which completes last is able to set the highest flag in the binary tree. On setting the flag it indicates to all the threads that barrier has been completed and thus synchronization is achieved.<br />
<br />
[[Image:Ch9wiki03.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Disseminating Barrier ==<br />
<br />
In this barrier each thread maintains a record of the activity of other threads. For every round i with n threads, thread A notifies thread (A + 2i) mod n. Thus after logn rounds all the threads are aware of the status of every other thread running and whether it has reached the barrier.<br />
<br />
[[Image:Ch9wiki04.png]]<sup><span id="9body">[[#9foot|[9]]]</span></sup><br />
<br />
= References = <br />
<span id="1foot">[[#1body|1.]]</span> David Culler, J.P. Singh and Anoop Gupta. Parallel Computer Architecture: A Hardware/Software Approach. Morgan Kaufmann. August 1998.<br><br />
<span id="2foot">[[#2body|2.]]</span> Yan Solihin. Fundamentals of Parallel Computer Architecture: Multichip and Multicore Systems. Solihin Books. August 2009.<br><br />
<span id="3foot">[[#3body|3.]]</span> Thomas E. Anderson. The Performance of Spin Lock Alternatives for Shared-memory Multiprocessors. http://www.cc.gatech.edu/classes/AY2010/cs4210_fall/papers/anderson-spinlock.pdf<br><br />
<span id="4foot">[[#4body|4.]]</span> http://www.statemaster.com/encyclopedia/Lock-(computer-science)<br><br />
<span id="5foot">[[#5body|5.]]</span> http://faydoc.tripod.com/cpu/cmpxchg.htm<br><br />
<span id="6foot">[[#6body|6.]]</span> http://www.cs.duke.edu/courses/fall09/cps110/slides/threads3.ppt<br><br />
<span id="7foot">[[#7body|7.]]</span> Scalability evaluation of barrier algorithms for OpenMP. http://www2.cs.uh.edu/~hpctools/pub/iwomp-barrier.pdf<br><br />
<span id="8foot">[[#8body|8.]]</span> Carwyn Ball and Mark Bull. Barrier Synchronization in Java. http://www.ukhec.ac.uk/publications/reports/synch_java.pdf<br><br />
<span id="9foot">[[#9body|9.]]</span> http://www.cs.brown.edu/courses/cs176/barrier.ppt<br><br />
<span id="10foot">[[#10body|10.]]</span>http://heather.cs.ucdavis.edu/~matloff/50/PLN/lock.pdf<br><br />
<span id="11foot">[[#11body|11.]]</span>http://www.cs.duke.edu/courses/fall09/cps110/handouts/threads3.pdf<br></div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/9b_sc&diff=74656CSC/ECE 506 Spring 2013/9b sc2013-04-03T20:34:39Z<p>Schakra8: /* Hand-off Lock */</p>
<hr />
<div>= Synchronization =<br />
In addition to a proper cache coherency model, it is important for a multiprocessor system to provide support for synchronization. This must be provided at the hardware level, and can also be implemented to some degree in software. The most common types of hardware synchronization are locks and barriers, which are discussed in this chapter.<br />
<br />
== Hardware vs. Operating System Synchronization ==<br />
Synchronizations are characterized by three parts: acquire, wait, and release. Both hardware and operating system synchronization rely on atomic instructions for acquire and release, but the implementations for the wait portion of a synchronization can vary. Hardware implementations usually use busy-waiting, where a process repeatedly tests a variable waiting for it to change. The operating system can wait using blocking, where the process suspends itself until woken up by another process.<br />
<br />
Blocking has the advantage of freeing up the processor for use by other processes, but requires operating system functions to work and thus has a lot of overhead. Busy-waiting has much less overhead, but uses processor and cache resources while waiting. Because of these tradeoffs, it is usually better to use busy-waiting for short wait periods, and blocking for long wait periods.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
= Lock Implementations =<br />
<br />
Locks are an important concept when programming for multiprocessor systems. The purpose of a lock is to protect the code inside the lock, known as a critical section. It must be certain that while some thread X has entered the critical section, another thread Y is not also inside of the critical section and possibly modifying critical values. While thread X has entered the critical section, thread Y must wait until X has exited before entering. This can be accomplished in a variety of ways of varying complexity and performance.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these four criteria:<br />
<br />
*Acquisition Latency - How much time does it take to acquire the lock?<br />
*Traffic - How much bus traffic is generated by threads attempting to acquire the lock?<br />
*Fairness - FIFO vs. Luck<br />
*Storage - How much storage is needed compared to the number of threads?<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Acquisition Latency ===<br />
We want a low acquisition latency, especially for applications that repeatedly acquire and release locks. Over the run time of a program, the acquisition latency compounded many times could have a large performance affect. However, low acquisition latency has to be balanced against other factors when implementing a lock.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
Traffic is an important consideration when evaluating lock performance. If acquiring lock causes a lot of bus traffic, it will not scale well as the number of threads increases. Eventually the bus will become choked.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Fairness ===<br />
In general, threads should acquire a lock in them same order that the locks were requested. If this does not happen, as the number of threads increases so does the chance that a thread will become starved.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Storage ===<br />
The amount of storage needed per lock should be small enough such that it scales to a large number of threads without any problems. If a large amount of storage is required, then multiple threads could cause locks to consume too much memory.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
<br />
Two common lock implementations are test&set and test and test&set. In terms of performance, both have a low storage cost (scalable) and are not fair. Test&set has a low acquisition latency but is not scalable due do high bus traffic when there is a lot of contention for the lock. Test-and-test&set has a higher acquisition latency but scales better because it generates less bus traffic. The next figure shows performance comparisons for these two lock implementations.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
[[Image:lockperf.png|frame|none|Performance comparison of test&set and test-and-test&set (spin on read)<sup><span id="3body">[[#3foot|[3]]]</span></sup>]]<br />
<br />
=== Improving performance ===<br />
Optimizing software can improve the performance of locks. For instance, inserting a delay between a processor noticing a release and then trying to acquire a lock can reduce contention and bus traffic, and increase the performance of the locks. TCP-like back off algorithms can be used to adjust this delay depending on the number of threads contending for the lock.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
Another way to improve performance is to insert a delay between memory references so as to limit the bandwidth (traffic) each processor can use for spinning.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
One way to guarantee fairness is to queue lock attempts in shared memory so that they happen in order.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
== Atomic Instructions ==<br />
<br />
Since a multiprocessor system cannot disable interrupts as an effective method to execute code atomically, there must be hardware support for atomic operations.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
The ability to execute an atomic instruction is a requirement for most lock implementations. It is important that when a processor attempts to set a lock, either the lock is fully set and the thread is able to enter into the critical section, or the lock is not set and it appears that none of the instructions required to set the lock have executed. <br />
<br />
In the x86 instruction set the opcode CMPXCHG (compare and exchange) can be used in a lock implementation in order to guarantee atomicity. This function works by sending a destination and a source. The accumulator is compared to the destination and, if they are equal, loaded with the source. If they are not equal the accumulator is loaded with the destination value. In order to assure that this is executed atomically the opcode must be issued with the LOCK prefix. This is useful in implementing some locks, such as ticket locks.<sup><span id="5body">[[#5foot|[5]]]</span></sup> The syntax for the instruction is LOCK CMPXCHG SOURCE DESTINATION. Here, the source has to be a register while the destination can be either Register or Memory. An example of LOCK CMPXCHG is implementation in dis-assembly of Linux kernel's rtc_cmos_read(). The CPU locks needs to be copied and and then updated to a non zero value if the lock is to be applied. But there is a possibility when the value got copied earlier and changed later. This causes a race condition. Hence,it helps in briefing the busy wait loop by preventing race condition. The pseudo code for same is given as below<sup><span id="10body">[[#10foot|[10]]]</span></sup>:<br />
<br />
spin: # see if the lock-variable is clear<br />
mov cmos_lock, %eax<br />
test %eax, %eax<br />
jnz spin<br />
# ok, now we try to grab the lock<br />
lock cmpxchg %edx, cmos_lock<br />
# did another CPU grab it first? <br />
test %eax, %eax<br />
jnz spin<br />
<br />
== Hand-off Lock ==<br />
<br />
Another type of lock that is not discussed in the text is known as the "hand-off" lock. In this lock the first thread acquires the lock if no other thread is currently locked (since it is the first thread). When another thread attempts to gain the lock it will see that the lock is in use and adds itself to the queue. Once done this thread can sleep until called by the thread with the lock. Once the thread in the lock is finished, it will pass the lock to the next thread in the queue<sup><span id="6body">[[#6foot|[6]]]</span></sup>. Waiting thread gives up processor so that other threads (e.g. the thread with the lock) can run more quickly. Someone wakes up thread when the lock is free. The separation of ready queue from that of waiting queue fastens the process of execution. In the below example, the lock is given to a waiting thread by the unlock method. The interrupts are enabled only when the thread is placed in waiting queue and disabled once its given the lock. <br />
<br />
lock () {<br />
disable interrupts<br />
if (value == FREE) {<br />
value = BUSY<br />
} else {<br />
add thread to queue of threads waiting for<br />
this lock<br />
switch to next runnable thread<br />
}<br />
enable interrupts<br />
}<br />
unlock () {<br />
disable interrupts<br />
value = FREE<br />
if (any thread is waiting for this lock) {<br />
move waiting thread from waiting queue to<br />
ready queue<br />
value = BUSY<br />
}<br />
enable interrupts<br />
} <br />
<br />
But this fails if interrupt happens after thread enable interrupts Lock() adds thread to wait queue Lock() enables interrupts Interrupts causes pre-emption, i.e. switch to another thread. Pre-emption moves thread to ready queue. Now thread is on two queues (wait and ready)! Also, switch is likely to be a critical section Adding thread to wait queue and switching to next thread must be atomic. Solution to this problem is waiting thread leaves interrupts disabled when it calls switch. Next thread to run has the responsibility of re-enabling interrupts before returning to user code. When waiting thread wakes up, it returns from switch with interrupts disabled (from the last thread)<sup><span id="11body">[[#11foot|[11]]]</span></sup><br />
<br />
== Avoiding Locks ==<br />
<br />
There are many reasons why a programmer should attempt to write programs in such a way as to avoid locks if possible. There are many problems that can arise with the use of locks.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
One of the must well known issues is deadlock. This can occur when the threads are waiting to acquire the lock, but the lock will never be unlocked. For example if 2 threads are both spinning on a lock that is locked, they will continue to spin forever, as each thread 'thinks' that the other is inside the critical section.<br />
<br />
Another problem with using locks is that the performance is not optimal, as often a lock is used when there is only a chance of conflict. This approach to programming yields slower performance than what might be possible with other methods. This also leads to questions of granularity, that is how much of the code should be protected under the critical section. The programmer must decide between many small (fine grain) locks or fewer, more encompassing locks. This decision can greatly effect the performance of the program.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
= Barrier Implementations =<br />
<br />
In many ways, barriers are simpler than locks. A barrier is simply a point in a program where one or more threads must reach before the parallel program is allowed to continue. When using barriers, a programmer does not have to be concerned with advanced topics such as fairness, that are required when programming for locks.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these two criteria:<br />
<br />
*Latency - The time required to enter and exit a barrier.<br />
*Traffic - Communications overhead required by the barrier.<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Latency ===<br />
<br />
Ideally the latency of a barrier should be small, but for barriers that don't scale well latency can increase as the number of threads increases.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
<br />
Low traffic is also good, as we do not want excess buss traffic to prevent scalability.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
{| class="wikitable" border="1"<br />
|+Performance Comparison of Barrier Implementations<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
|-<br />
! Criteria<br />
! Sense Reversal<br />
Centralized Barrier<br />
! Combining Tree<br />
Barrier<br />
! Barrier Network<br />
(Hardware)<br />
|-<br />
| Latency<br />
| O(1)<br />
| O(log p)<br />
| O(log p)<br />
|-<br />
| Traffic<br />
| O(p<sup>2</sup>)<br />
| O(p)<br />
| moved to a separate network<br />
|}<br />
<br />
<br />
<br />
[[Image:Ch9wiki05.png|frame|none|Additive Schwarz Preconditioned Conjugate Gradient (ASPCG) kernel on Altix System - This figure shows the timings of the ASPCG kernel using the different barrier implementations. It can be seen that the blocking barrier does not scale with number of threads as with increase in number of threads the contention increases.<sup><span id="7body">[[#7foot|[7]]]</span></sup>]]<br />
<br />
<br />
<br />
[[Image:Ch9wiki06.png|frame|none|EPCC Microbenchmark - This figure shows the timings to implement a barrier of the EPCC Microbenchmark using the different barrier implementations. It can be seen that the blocking barrier/centralized blocking barrier does not scale with number of threads as with increase in number of threads the contention increases.<sup><span id="7body">[[#7foot|[7]]]</span></sup>]]<br />
<br />
== Sense-Reversal Barrier ==<br />
<br />
This barrier is a centralized barrier where a single count variable protected by a lock is shared among all the threads. Each thread on reaching the barrier increments its value and waits till the value of variable has reached the number of threads for which barrier was implemented.<br />
<br />
Since all the threads are spinning around a single variable the miss rate is scales quadratically with number of processors.<br />
<br />
[[Image:Ch9wiki01.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Combining Tree Barrier ==<br />
<br />
This barrier is a distributed barrier where group of processors form clusters and updates value of a local variable. The local variable on reaching a value equal to number of threads updating it, proceeds to increment another variable higher in hierarchy in the combining tree. When the variable in the highest level of hierarchy in the combining tree reaches its max value it is considered that all the threads have reached the barrier and synchronization is complete.<br />
<br />
Since in this case all the threads are updated local variables in form of smaller groups the miss rate is not as high as sense-reversal barrier.<br />
<br />
The diagram below shows the operation of combining tree barrier with threads grouped in two groups. The variables C0 and C1 are local to each group and C2 is the variable that is at higher level of hierarchy in the tree.<br />
<br />
[[Image:Ch9wiki02.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Tournament Barrier ==<br />
<br />
In tournament barrier the threads are considered to be leaves at the end of a binary tree and each node represents a flag. Two threads compete with each other and the loser thread is allowed to set the flag and move to higher level and compete to lose with the loser thread from other section of binary tree. Thus the thread which completes last is able to set the highest flag in the binary tree. On setting the flag it indicates to all the threads that barrier has been completed and thus synchronization is achieved.<br />
<br />
[[Image:Ch9wiki03.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Disseminating Barrier ==<br />
<br />
In this barrier each thread maintains a record of the activity of other threads. For every round i with n threads, thread A notifies thread (A + 2i) mod n. Thus after logn rounds all the threads are aware of the status of every other thread running and whether it has reached the barrier.<br />
<br />
[[Image:Ch9wiki04.png]]<sup><span id="9body">[[#9foot|[9]]]</span></sup><br />
<br />
= References = <br />
<span id="1foot">[[#1body|1.]]</span> David Culler, J.P. Singh and Anoop Gupta. Parallel Computer Architecture: A Hardware/Software Approach. Morgan Kaufmann. August 1998.<br><br />
<span id="2foot">[[#2body|2.]]</span> Yan Solihin. Fundamentals of Parallel Computer Architecture: Multichip and Multicore Systems. Solihin Books. August 2009.<br><br />
<span id="3foot">[[#3body|3.]]</span> Thomas E. Anderson. The Performance of Spin Lock Alternatives for Shared-memory Multiprocessors. http://www.cc.gatech.edu/classes/AY2010/cs4210_fall/papers/anderson-spinlock.pdf<br><br />
<span id="4foot">[[#4body|4.]]</span> http://www.statemaster.com/encyclopedia/Lock-(computer-science)<br><br />
<span id="5foot">[[#5body|5.]]</span> http://faydoc.tripod.com/cpu/cmpxchg.htm<br><br />
<span id="6foot">[[#6body|6.]]</span> http://www.cs.duke.edu/courses/fall09/cps110/slides/threads3.ppt<br><br />
<span id="7foot">[[#7body|7.]]</span> Scalability evaluation of barrier algorithms for OpenMP. http://www2.cs.uh.edu/~hpctools/pub/iwomp-barrier.pdf<br><br />
<span id="8foot">[[#8body|8.]]</span> Carwyn Ball and Mark Bull. Barrier Synchronization in Java. http://www.ukhec.ac.uk/publications/reports/synch_java.pdf<br><br />
<span id="9foot">[[#9body|9.]]</span> http://www.cs.brown.edu/courses/cs176/barrier.ppt<br><br />
<span id="10foot">[[#10body|10.]]</span>http://heather.cs.ucdavis.edu/~matloff/50/PLN/lock.pdf<br></div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/9b_sc&diff=74655CSC/ECE 506 Spring 2013/9b sc2013-04-03T19:30:01Z<p>Schakra8: /* Atomic Instructions */</p>
<hr />
<div>= Synchronization =<br />
In addition to a proper cache coherency model, it is important for a multiprocessor system to provide support for synchronization. This must be provided at the hardware level, and can also be implemented to some degree in software. The most common types of hardware synchronization are locks and barriers, which are discussed in this chapter.<br />
<br />
== Hardware vs. Operating System Synchronization ==<br />
Synchronizations are characterized by three parts: acquire, wait, and release. Both hardware and operating system synchronization rely on atomic instructions for acquire and release, but the implementations for the wait portion of a synchronization can vary. Hardware implementations usually use busy-waiting, where a process repeatedly tests a variable waiting for it to change. The operating system can wait using blocking, where the process suspends itself until woken up by another process.<br />
<br />
Blocking has the advantage of freeing up the processor for use by other processes, but requires operating system functions to work and thus has a lot of overhead. Busy-waiting has much less overhead, but uses processor and cache resources while waiting. Because of these tradeoffs, it is usually better to use busy-waiting for short wait periods, and blocking for long wait periods.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
= Lock Implementations =<br />
<br />
Locks are an important concept when programming for multiprocessor systems. The purpose of a lock is to protect the code inside the lock, known as a critical section. It must be certain that while some thread X has entered the critical section, another thread Y is not also inside of the critical section and possibly modifying critical values. While thread X has entered the critical section, thread Y must wait until X has exited before entering. This can be accomplished in a variety of ways of varying complexity and performance.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these four criteria:<br />
<br />
*Acquisition Latency - How much time does it take to acquire the lock?<br />
*Traffic - How much bus traffic is generated by threads attempting to acquire the lock?<br />
*Fairness - FIFO vs. Luck<br />
*Storage - How much storage is needed compared to the number of threads?<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Acquisition Latency ===<br />
We want a low acquisition latency, especially for applications that repeatedly acquire and release locks. Over the run time of a program, the acquisition latency compounded many times could have a large performance affect. However, low acquisition latency has to be balanced against other factors when implementing a lock.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
Traffic is an important consideration when evaluating lock performance. If acquiring lock causes a lot of bus traffic, it will not scale well as the number of threads increases. Eventually the bus will become choked.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Fairness ===<br />
In general, threads should acquire a lock in them same order that the locks were requested. If this does not happen, as the number of threads increases so does the chance that a thread will become starved.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Storage ===<br />
The amount of storage needed per lock should be small enough such that it scales to a large number of threads without any problems. If a large amount of storage is required, then multiple threads could cause locks to consume too much memory.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
<br />
Two common lock implementations are test&set and test and test&set. In terms of performance, both have a low storage cost (scalable) and are not fair. Test&set has a low acquisition latency but is not scalable due do high bus traffic when there is a lot of contention for the lock. Test-and-test&set has a higher acquisition latency but scales better because it generates less bus traffic. The next figure shows performance comparisons for these two lock implementations.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
[[Image:lockperf.png|frame|none|Performance comparison of test&set and test-and-test&set (spin on read)<sup><span id="3body">[[#3foot|[3]]]</span></sup>]]<br />
<br />
=== Improving performance ===<br />
Optimizing software can improve the performance of locks. For instance, inserting a delay between a processor noticing a release and then trying to acquire a lock can reduce contention and bus traffic, and increase the performance of the locks. TCP-like back off algorithms can be used to adjust this delay depending on the number of threads contending for the lock.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
Another way to improve performance is to insert a delay between memory references so as to limit the bandwidth (traffic) each processor can use for spinning.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
One way to guarantee fairness is to queue lock attempts in shared memory so that they happen in order.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
== Atomic Instructions ==<br />
<br />
Since a multiprocessor system cannot disable interrupts as an effective method to execute code atomically, there must be hardware support for atomic operations.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
The ability to execute an atomic instruction is a requirement for most lock implementations. It is important that when a processor attempts to set a lock, either the lock is fully set and the thread is able to enter into the critical section, or the lock is not set and it appears that none of the instructions required to set the lock have executed. <br />
<br />
In the x86 instruction set the opcode CMPXCHG (compare and exchange) can be used in a lock implementation in order to guarantee atomicity. This function works by sending a destination and a source. The accumulator is compared to the destination and, if they are equal, loaded with the source. If they are not equal the accumulator is loaded with the destination value. In order to assure that this is executed atomically the opcode must be issued with the LOCK prefix. This is useful in implementing some locks, such as ticket locks.<sup><span id="5body">[[#5foot|[5]]]</span></sup> The syntax for the instruction is LOCK CMPXCHG SOURCE DESTINATION. Here, the source has to be a register while the destination can be either Register or Memory. An example of LOCK CMPXCHG is implementation in dis-assembly of Linux kernel's rtc_cmos_read(). The CPU locks needs to be copied and and then updated to a non zero value if the lock is to be applied. But there is a possibility when the value got copied earlier and changed later. This causes a race condition. Hence,it helps in briefing the busy wait loop by preventing race condition. The pseudo code for same is given as below<sup><span id="10body">[[#10foot|[10]]]</span></sup>:<br />
<br />
spin: # see if the lock-variable is clear<br />
mov cmos_lock, %eax<br />
test %eax, %eax<br />
jnz spin<br />
# ok, now we try to grab the lock<br />
lock cmpxchg %edx, cmos_lock<br />
# did another CPU grab it first? <br />
test %eax, %eax<br />
jnz spin<br />
<br />
== Hand-off Lock ==<br />
<br />
Another type of lock that is not discussed in the text is known as the "hand-off" lock. In this lock the first thread acquires the lock if no other thread is currently locked (since it is the first thread). When another thread attempts to gain the lock it will see that the lock is in use and adds itself to the queue. Once done this thread can sleep until called by the thread with the lock. Once the thread in the lock is finished, it will pass the lock to the next thread in the queue.<sup><span id="6body">[[#6foot|[6]]]</span></sup><br />
<br />
== Avoiding Locks ==<br />
<br />
There are many reasons why a programmer should attempt to write programs in such a way as to avoid locks if possible. There are many problems that can arise with the use of locks.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
One of the must well known issues is deadlock. This can occur when the threads are waiting to acquire the lock, but the lock will never be unlocked. For example if 2 threads are both spinning on a lock that is locked, they will continue to spin forever, as each thread 'thinks' that the other is inside the critical section.<br />
<br />
Another problem with using locks is that the performance is not optimal, as often a lock is used when there is only a chance of conflict. This approach to programming yields slower performance than what might be possible with other methods. This also leads to questions of granularity, that is how much of the code should be protected under the critical section. The programmer must decide between many small (fine grain) locks or fewer, more encompassing locks. This decision can greatly effect the performance of the program.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
= Barrier Implementations =<br />
<br />
In many ways, barriers are simpler than locks. A barrier is simply a point in a program where one or more threads must reach before the parallel program is allowed to continue. When using barriers, a programmer does not have to be concerned with advanced topics such as fairness, that are required when programming for locks.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these two criteria:<br />
<br />
*Latency - The time required to enter and exit a barrier.<br />
*Traffic - Communications overhead required by the barrier.<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Latency ===<br />
<br />
Ideally the latency of a barrier should be small, but for barriers that don't scale well latency can increase as the number of threads increases.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
<br />
Low traffic is also good, as we do not want excess buss traffic to prevent scalability.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
{| class="wikitable" border="1"<br />
|+Performance Comparison of Barrier Implementations<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
|-<br />
! Criteria<br />
! Sense Reversal<br />
Centralized Barrier<br />
! Combining Tree<br />
Barrier<br />
! Barrier Network<br />
(Hardware)<br />
|-<br />
| Latency<br />
| O(1)<br />
| O(log p)<br />
| O(log p)<br />
|-<br />
| Traffic<br />
| O(p<sup>2</sup>)<br />
| O(p)<br />
| moved to a separate network<br />
|}<br />
<br />
<br />
<br />
[[Image:Ch9wiki05.png|frame|none|Additive Schwarz Preconditioned Conjugate Gradient (ASPCG) kernel on Altix System - This figure shows the timings of the ASPCG kernel using the different barrier implementations. It can be seen that the blocking barrier does not scale with number of threads as with increase in number of threads the contention increases.<sup><span id="7body">[[#7foot|[7]]]</span></sup>]]<br />
<br />
<br />
<br />
[[Image:Ch9wiki06.png|frame|none|EPCC Microbenchmark - This figure shows the timings to implement a barrier of the EPCC Microbenchmark using the different barrier implementations. It can be seen that the blocking barrier/centralized blocking barrier does not scale with number of threads as with increase in number of threads the contention increases.<sup><span id="7body">[[#7foot|[7]]]</span></sup>]]<br />
<br />
== Sense-Reversal Barrier ==<br />
<br />
This barrier is a centralized barrier where a single count variable protected by a lock is shared among all the threads. Each thread on reaching the barrier increments its value and waits till the value of variable has reached the number of threads for which barrier was implemented.<br />
<br />
Since all the threads are spinning around a single variable the miss rate is scales quadratically with number of processors.<br />
<br />
[[Image:Ch9wiki01.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Combining Tree Barrier ==<br />
<br />
This barrier is a distributed barrier where group of processors form clusters and updates value of a local variable. The local variable on reaching a value equal to number of threads updating it, proceeds to increment another variable higher in hierarchy in the combining tree. When the variable in the highest level of hierarchy in the combining tree reaches its max value it is considered that all the threads have reached the barrier and synchronization is complete.<br />
<br />
Since in this case all the threads are updated local variables in form of smaller groups the miss rate is not as high as sense-reversal barrier.<br />
<br />
The diagram below shows the operation of combining tree barrier with threads grouped in two groups. The variables C0 and C1 are local to each group and C2 is the variable that is at higher level of hierarchy in the tree.<br />
<br />
[[Image:Ch9wiki02.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Tournament Barrier ==<br />
<br />
In tournament barrier the threads are considered to be leaves at the end of a binary tree and each node represents a flag. Two threads compete with each other and the loser thread is allowed to set the flag and move to higher level and compete to lose with the loser thread from other section of binary tree. Thus the thread which completes last is able to set the highest flag in the binary tree. On setting the flag it indicates to all the threads that barrier has been completed and thus synchronization is achieved.<br />
<br />
[[Image:Ch9wiki03.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Disseminating Barrier ==<br />
<br />
In this barrier each thread maintains a record of the activity of other threads. For every round i with n threads, thread A notifies thread (A + 2i) mod n. Thus after logn rounds all the threads are aware of the status of every other thread running and whether it has reached the barrier.<br />
<br />
[[Image:Ch9wiki04.png]]<sup><span id="9body">[[#9foot|[9]]]</span></sup><br />
<br />
= References = <br />
<span id="1foot">[[#1body|1.]]</span> David Culler, J.P. Singh and Anoop Gupta. Parallel Computer Architecture: A Hardware/Software Approach. Morgan Kaufmann. August 1998.<br><br />
<span id="2foot">[[#2body|2.]]</span> Yan Solihin. Fundamentals of Parallel Computer Architecture: Multichip and Multicore Systems. Solihin Books. August 2009.<br><br />
<span id="3foot">[[#3body|3.]]</span> Thomas E. Anderson. The Performance of Spin Lock Alternatives for Shared-memory Multiprocessors. http://www.cc.gatech.edu/classes/AY2010/cs4210_fall/papers/anderson-spinlock.pdf<br><br />
<span id="4foot">[[#4body|4.]]</span> http://www.statemaster.com/encyclopedia/Lock-(computer-science)<br><br />
<span id="5foot">[[#5body|5.]]</span> http://faydoc.tripod.com/cpu/cmpxchg.htm<br><br />
<span id="6foot">[[#6body|6.]]</span> http://www.cs.duke.edu/courses/fall09/cps110/slides/threads3.ppt<br><br />
<span id="7foot">[[#7body|7.]]</span> Scalability evaluation of barrier algorithms for OpenMP. http://www2.cs.uh.edu/~hpctools/pub/iwomp-barrier.pdf<br><br />
<span id="8foot">[[#8body|8.]]</span> Carwyn Ball and Mark Bull. Barrier Synchronization in Java. http://www.ukhec.ac.uk/publications/reports/synch_java.pdf<br><br />
<span id="9foot">[[#9body|9.]]</span> http://www.cs.brown.edu/courses/cs176/barrier.ppt<br><br />
<span id="10foot">[[#10body|10.]]</span>http://heather.cs.ucdavis.edu/~matloff/50/PLN/lock.pdf<br></div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/9b_sc&diff=74654CSC/ECE 506 Spring 2013/9b sc2013-04-03T19:28:19Z<p>Schakra8: </p>
<hr />
<div>= Synchronization =<br />
In addition to a proper cache coherency model, it is important for a multiprocessor system to provide support for synchronization. This must be provided at the hardware level, and can also be implemented to some degree in software. The most common types of hardware synchronization are locks and barriers, which are discussed in this chapter.<br />
<br />
== Hardware vs. Operating System Synchronization ==<br />
Synchronizations are characterized by three parts: acquire, wait, and release. Both hardware and operating system synchronization rely on atomic instructions for acquire and release, but the implementations for the wait portion of a synchronization can vary. Hardware implementations usually use busy-waiting, where a process repeatedly tests a variable waiting for it to change. The operating system can wait using blocking, where the process suspends itself until woken up by another process.<br />
<br />
Blocking has the advantage of freeing up the processor for use by other processes, but requires operating system functions to work and thus has a lot of overhead. Busy-waiting has much less overhead, but uses processor and cache resources while waiting. Because of these tradeoffs, it is usually better to use busy-waiting for short wait periods, and blocking for long wait periods.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
= Lock Implementations =<br />
<br />
Locks are an important concept when programming for multiprocessor systems. The purpose of a lock is to protect the code inside the lock, known as a critical section. It must be certain that while some thread X has entered the critical section, another thread Y is not also inside of the critical section and possibly modifying critical values. While thread X has entered the critical section, thread Y must wait until X has exited before entering. This can be accomplished in a variety of ways of varying complexity and performance.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these four criteria:<br />
<br />
*Acquisition Latency - How much time does it take to acquire the lock?<br />
*Traffic - How much bus traffic is generated by threads attempting to acquire the lock?<br />
*Fairness - FIFO vs. Luck<br />
*Storage - How much storage is needed compared to the number of threads?<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Acquisition Latency ===<br />
We want a low acquisition latency, especially for applications that repeatedly acquire and release locks. Over the run time of a program, the acquisition latency compounded many times could have a large performance affect. However, low acquisition latency has to be balanced against other factors when implementing a lock.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
Traffic is an important consideration when evaluating lock performance. If acquiring lock causes a lot of bus traffic, it will not scale well as the number of threads increases. Eventually the bus will become choked.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Fairness ===<br />
In general, threads should acquire a lock in them same order that the locks were requested. If this does not happen, as the number of threads increases so does the chance that a thread will become starved.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Storage ===<br />
The amount of storage needed per lock should be small enough such that it scales to a large number of threads without any problems. If a large amount of storage is required, then multiple threads could cause locks to consume too much memory.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
<br />
Two common lock implementations are test&set and test and test&set. In terms of performance, both have a low storage cost (scalable) and are not fair. Test&set has a low acquisition latency but is not scalable due do high bus traffic when there is a lot of contention for the lock. Test-and-test&set has a higher acquisition latency but scales better because it generates less bus traffic. The next figure shows performance comparisons for these two lock implementations.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
[[Image:lockperf.png|frame|none|Performance comparison of test&set and test-and-test&set (spin on read)<sup><span id="3body">[[#3foot|[3]]]</span></sup>]]<br />
<br />
=== Improving performance ===<br />
Optimizing software can improve the performance of locks. For instance, inserting a delay between a processor noticing a release and then trying to acquire a lock can reduce contention and bus traffic, and increase the performance of the locks. TCP-like back off algorithms can be used to adjust this delay depending on the number of threads contending for the lock.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
Another way to improve performance is to insert a delay between memory references so as to limit the bandwidth (traffic) each processor can use for spinning.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
One way to guarantee fairness is to queue lock attempts in shared memory so that they happen in order.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
== Atomic Instructions ==<br />
<br />
Since a multiprocessor system cannot disable interrupts as an effective method to execute code atomically, there must be hardware support for atomic operations.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
The ability to execute an atomic instruction is a requirement for most lock implementations. It is important that when a processor attempts to set a lock, either the lock is fully set and the thread is able to enter into the critical section, or the lock is not set and it appears that none of the instructions required to set the lock have executed. <br />
<br />
In the x86 instruction set the opcode CMPXCHG (compare and exchange) can be used in a lock implementation in order to guarantee atomicity. This function works by sending a destination and a source. The accumulator is compared to the destination and, if they are equal, loaded with the source. If they are not equal the accumulator is loaded with the destination value. In order to assure that this is executed atomically the opcode must be issued with the LOCK prefix. This is useful in implementing some locks, such as ticket locks.<sup><span id="5body">[[#5foot|[5]]]</span></sup> The syntax for the instruction is LOCK CMPXCHG SOURCE DESTINATION. Here, the source has to be a register while the destination can be either Register or Memory. An example of LOCK CMPXCHG is implementation in dis-assembly of Linux kernel's rtc_cmos_read(). The CPU locks needs to be copied and and then updated to a non zero value if the lock is to be applied. But there is a possibility when the value got copied earlier and changed later. This causes a race condition. Hence,it helps in briefing the busy wait loop by preventing race condition. The pseudo code for same is given as below<sup><span id="10body">[[#10foot|[10]]]</span></sup>:<br />
spin: # see if the lock-variable is clear<br />
mov cmos_lock, %eax<br />
test %eax, %eax<br />
jnz spin<br />
# ok, now we try to grab the lock<br />
lock cmpxchg %edx, cmos_lock<br />
# did another CPU grab it first? <br />
test %eax, %eax<br />
jnz spin<br />
<br />
<br />
== Hand-off Lock ==<br />
<br />
Another type of lock that is not discussed in the text is known as the "hand-off" lock. In this lock the first thread acquires the lock if no other thread is currently locked (since it is the first thread). When another thread attempts to gain the lock it will see that the lock is in use and adds itself to the queue. Once done this thread can sleep until called by the thread with the lock. Once the thread in the lock is finished, it will pass the lock to the next thread in the queue.<sup><span id="6body">[[#6foot|[6]]]</span></sup><br />
<br />
== Avoiding Locks ==<br />
<br />
There are many reasons why a programmer should attempt to write programs in such a way as to avoid locks if possible. There are many problems that can arise with the use of locks.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
One of the must well known issues is deadlock. This can occur when the threads are waiting to acquire the lock, but the lock will never be unlocked. For example if 2 threads are both spinning on a lock that is locked, they will continue to spin forever, as each thread 'thinks' that the other is inside the critical section.<br />
<br />
Another problem with using locks is that the performance is not optimal, as often a lock is used when there is only a chance of conflict. This approach to programming yields slower performance than what might be possible with other methods. This also leads to questions of granularity, that is how much of the code should be protected under the critical section. The programmer must decide between many small (fine grain) locks or fewer, more encompassing locks. This decision can greatly effect the performance of the program.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
= Barrier Implementations =<br />
<br />
In many ways, barriers are simpler than locks. A barrier is simply a point in a program where one or more threads must reach before the parallel program is allowed to continue. When using barriers, a programmer does not have to be concerned with advanced topics such as fairness, that are required when programming for locks.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these two criteria:<br />
<br />
*Latency - The time required to enter and exit a barrier.<br />
*Traffic - Communications overhead required by the barrier.<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Latency ===<br />
<br />
Ideally the latency of a barrier should be small, but for barriers that don't scale well latency can increase as the number of threads increases.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
<br />
Low traffic is also good, as we do not want excess buss traffic to prevent scalability.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
{| class="wikitable" border="1"<br />
|+Performance Comparison of Barrier Implementations<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
|-<br />
! Criteria<br />
! Sense Reversal<br />
Centralized Barrier<br />
! Combining Tree<br />
Barrier<br />
! Barrier Network<br />
(Hardware)<br />
|-<br />
| Latency<br />
| O(1)<br />
| O(log p)<br />
| O(log p)<br />
|-<br />
| Traffic<br />
| O(p<sup>2</sup>)<br />
| O(p)<br />
| moved to a separate network<br />
|}<br />
<br />
<br />
<br />
[[Image:Ch9wiki05.png|frame|none|Additive Schwarz Preconditioned Conjugate Gradient (ASPCG) kernel on Altix System - This figure shows the timings of the ASPCG kernel using the different barrier implementations. It can be seen that the blocking barrier does not scale with number of threads as with increase in number of threads the contention increases.<sup><span id="7body">[[#7foot|[7]]]</span></sup>]]<br />
<br />
<br />
<br />
[[Image:Ch9wiki06.png|frame|none|EPCC Microbenchmark - This figure shows the timings to implement a barrier of the EPCC Microbenchmark using the different barrier implementations. It can be seen that the blocking barrier/centralized blocking barrier does not scale with number of threads as with increase in number of threads the contention increases.<sup><span id="7body">[[#7foot|[7]]]</span></sup>]]<br />
<br />
== Sense-Reversal Barrier ==<br />
<br />
This barrier is a centralized barrier where a single count variable protected by a lock is shared among all the threads. Each thread on reaching the barrier increments its value and waits till the value of variable has reached the number of threads for which barrier was implemented.<br />
<br />
Since all the threads are spinning around a single variable the miss rate is scales quadratically with number of processors.<br />
<br />
[[Image:Ch9wiki01.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Combining Tree Barrier ==<br />
<br />
This barrier is a distributed barrier where group of processors form clusters and updates value of a local variable. The local variable on reaching a value equal to number of threads updating it, proceeds to increment another variable higher in hierarchy in the combining tree. When the variable in the highest level of hierarchy in the combining tree reaches its max value it is considered that all the threads have reached the barrier and synchronization is complete.<br />
<br />
Since in this case all the threads are updated local variables in form of smaller groups the miss rate is not as high as sense-reversal barrier.<br />
<br />
The diagram below shows the operation of combining tree barrier with threads grouped in two groups. The variables C0 and C1 are local to each group and C2 is the variable that is at higher level of hierarchy in the tree.<br />
<br />
[[Image:Ch9wiki02.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Tournament Barrier ==<br />
<br />
In tournament barrier the threads are considered to be leaves at the end of a binary tree and each node represents a flag. Two threads compete with each other and the loser thread is allowed to set the flag and move to higher level and compete to lose with the loser thread from other section of binary tree. Thus the thread which completes last is able to set the highest flag in the binary tree. On setting the flag it indicates to all the threads that barrier has been completed and thus synchronization is achieved.<br />
<br />
[[Image:Ch9wiki03.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Disseminating Barrier ==<br />
<br />
In this barrier each thread maintains a record of the activity of other threads. For every round i with n threads, thread A notifies thread (A + 2i) mod n. Thus after logn rounds all the threads are aware of the status of every other thread running and whether it has reached the barrier.<br />
<br />
[[Image:Ch9wiki04.png]]<sup><span id="9body">[[#9foot|[9]]]</span></sup><br />
<br />
= References = <br />
<span id="1foot">[[#1body|1.]]</span> David Culler, J.P. Singh and Anoop Gupta. Parallel Computer Architecture: A Hardware/Software Approach. Morgan Kaufmann. August 1998.<br><br />
<span id="2foot">[[#2body|2.]]</span> Yan Solihin. Fundamentals of Parallel Computer Architecture: Multichip and Multicore Systems. Solihin Books. August 2009.<br><br />
<span id="3foot">[[#3body|3.]]</span> Thomas E. Anderson. The Performance of Spin Lock Alternatives for Shared-memory Multiprocessors. http://www.cc.gatech.edu/classes/AY2010/cs4210_fall/papers/anderson-spinlock.pdf<br><br />
<span id="4foot">[[#4body|4.]]</span> http://www.statemaster.com/encyclopedia/Lock-(computer-science)<br><br />
<span id="5foot">[[#5body|5.]]</span> http://faydoc.tripod.com/cpu/cmpxchg.htm<br><br />
<span id="6foot">[[#6body|6.]]</span> http://www.cs.duke.edu/courses/fall09/cps110/slides/threads3.ppt<br><br />
<span id="7foot">[[#7body|7.]]</span> Scalability evaluation of barrier algorithms for OpenMP. http://www2.cs.uh.edu/~hpctools/pub/iwomp-barrier.pdf<br><br />
<span id="8foot">[[#8body|8.]]</span> Carwyn Ball and Mark Bull. Barrier Synchronization in Java. http://www.ukhec.ac.uk/publications/reports/synch_java.pdf<br><br />
<span id="9foot">[[#9body|9.]]</span> http://www.cs.brown.edu/courses/cs176/barrier.ppt<br><br />
<span id="10foot">[[#10body|10.]]</span>http://heather.cs.ucdavis.edu/~matloff/50/PLN/lock.pdf<br></div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/9b_sc&diff=74652CSC/ECE 506 Spring 2013/9b sc2013-04-03T18:49:56Z<p>Schakra8: Created page with "= Synchronization = In addition to a proper cache coherency model, it is important for a multiprocessor system to provide support for synchronization. This must be provided at th..."</p>
<hr />
<div>= Synchronization =<br />
In addition to a proper cache coherency model, it is important for a multiprocessor system to provide support for synchronization. This must be provided at the hardware level, and can also be implemented to some degree in software. The most common types of hardware synchronization are locks and barriers, which are discussed in this chapter.<br />
<br />
== Hardware vs. Operating System Synchronization ==<br />
Synchronizations are characterized by three parts: acquire, wait, and release. Both hardware and operating system synchronization rely on atomic instructions for acquire and release, but the implementations for the wait portion of a synchronization can vary. Hardware implementations usually use busy-waiting, where a process repeatedly tests a variable waiting for it to change. The operating system can wait using blocking, where the process suspends itself until woken up by another process.<br />
<br />
Blocking has the advantage of freeing up the processor for use by other processes, but requires operating system functions to work and thus has a lot of overhead. Busy-waiting has much less overhead, but uses processor and cache resources while waiting. Because of these tradeoffs, it is usually better to use busy-waiting for short wait periods, and blocking for long wait periods.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
= Lock Implementations =<br />
<br />
Locks are an important concept when programming for multiprocessor systems. The purpose of a lock is to protect the code inside the lock, known as a critical section. It must be certain that while some thread X has entered the critical section, another thread Y is not also inside of the critical section and possibly modifying critical values. While thread X has entered the critical section, thread Y must wait until X has exited before entering. This can be accomplished in a variety of ways of varying complexity and performance.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these four criteria:<br />
<br />
*Acquisition Latency - How much time does it take to acquire the lock?<br />
*Traffic - How much bus traffic is generated by threads attempting to acquire the lock?<br />
*Fairness - FIFO vs. Luck<br />
*Storage - How much storage is needed compared to the number of threads?<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Acquisition Latency ===<br />
We want a low acquisition latency, especially for applications that repeatedly acquire and release locks. Over the run time of a program, the acquisition latency compounded many times could have a large performance affect. However, low acquisition latency has to be balanced against other factors when implementing a lock.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
Traffic is an important consideration when evaluating lock performance. If acquiring lock causes a lot of bus traffic, it will not scale well as the number of threads increases. Eventually the bus will become choked.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Fairness ===<br />
In general, threads should acquire a lock in them same order that the locks were requested. If this does not happen, as the number of threads increases so does the chance that a thread will become starved.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Storage ===<br />
The amount of storage needed per lock should be small enough such that it scales to a large number of threads without any problems. If a large amount of storage is required, then multiple threads could cause locks to consume too much memory.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
<br />
Two common lock implementations are test&set and test and test&set. In terms of performance, both have a low storage cost (scalable) and are not fair. Test&set has a low acquisition latency but is not scalable due do high bus traffic when there is a lot of contention for the lock. Test-and-test&set has a higher acquisition latency but scales better because it generates less bus traffic. The next figure shows performance comparisons for these two lock implementations.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
[[Image:lockperf.png|frame|none|Performance comparison of test&set and test-and-test&set (spin on read)<sup><span id="3body">[[#3foot|[3]]]</span></sup>]]<br />
<br />
=== Improving performance ===<br />
Optimizing software can improve the performance of locks. For instance, inserting a delay between a processor noticing a release and then trying to acquire a lock can reduce contention and bus traffic, and increase the performance of the locks. TCP-like back off algorithms can be used to adjust this delay depending on the number of threads contending for the lock.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
Another way to improve performance is to insert a delay between memory references so as to limit the bandwidth (traffic) each processor can use for spinning.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
One way to guarantee fairness is to queue lock attempts in shared memory so that they happen in order.<sup><span>[[#3foot|[3]]]</span></sup><br />
<br />
== Atomic Instructions ==<br />
<br />
Since a multiprocessor system cannot disable interrupts as an effective method to execute code atomically, there must be hardware support for atomic operations.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
The ability to execute an atomic instruction is a requirement for most lock implementations. It is important that when a processor attempts to set a lock, either the lock is fully set and the thread is able to enter into the critical section, or the lock is not set and it appears that none of the instructions required to set the lock have executed. <br />
<br />
In the x86 instruction set the opcode CMPXCHG (compare and exchange) can be used in a lock implementation in order to guarantee atomicity. This function works by sending a destination and a source. The accumulator is compared to the destination and, if they are equal, loaded with the source. If they are not equal the accumulator is loaded with the destination value. In order to assure that this is executed atomically the opcode must be issued with the LOCK prefix. This is useful in implementing some locks, such as ticket locks.<sup><span id="5body">[[#5foot|[5]]]</span></sup><br />
<br />
== Hand-off Lock ==<br />
<br />
Another type of lock that is not discussed in the text is known as the "hand-off" lock. In this lock the first thread acquires the lock if no other thread is currently locked (since it is the first thread). When another thread attempts to gain the lock it will see that the lock is in use and adds itself to the queue. Once done this thread can sleep until called by the thread with the lock. Once the thread in the lock is finished, it will pass the lock to the next thread in the queue.<sup><span id="6body">[[#6foot|[6]]]</span></sup><br />
<br />
== Avoiding Locks ==<br />
<br />
There are many reasons why a programmer should attempt to write programs in such a way as to avoid locks if possible. There are many problems that can arise with the use of locks.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
One of the must well known issues is deadlock. This can occur when the threads are waiting to acquire the lock, but the lock will never be unlocked. For example if 2 threads are both spinning on a lock that is locked, they will continue to spin forever, as each thread 'thinks' that the other is inside the critical section.<br />
<br />
Another problem with using locks is that the performance is not optimal, as often a lock is used when there is only a chance of conflict. This approach to programming yields slower performance than what might be possible with other methods. This also leads to questions of granularity, that is how much of the code should be protected under the critical section. The programmer must decide between many small (fine grain) locks or fewer, more encompassing locks. This decision can greatly effect the performance of the program.<sup><span id="4body">[[#4foot|[4]]]</span></sup><br />
<br />
= Barrier Implementations =<br />
<br />
In many ways, barriers are simpler than locks. A barrier is simply a point in a program where one or more threads must reach before the parallel program is allowed to continue. When using barriers, a programmer does not have to be concerned with advanced topics such as fairness, that are required when programming for locks.<br />
<br />
== Performance Evaluation ==<br />
<br />
To evaluate lock performance, we can use these two criteria:<br />
<br />
*Latency - The time required to enter and exit a barrier.<br />
*Traffic - Communications overhead required by the barrier.<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
<br />
=== Latency ===<br />
<br />
Ideally the latency of a barrier should be small, but for barriers that don't scale well latency can increase as the number of threads increases.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Traffic ===<br />
<br />
Low traffic is also good, as we do not want excess buss traffic to prevent scalability.<sup><span id="1body">[[#1foot|[1]]]</span></sup><br />
<br />
=== Performance Comparison ===<br />
{| class="wikitable" border="1"<br />
|+Performance Comparison of Barrier Implementations<sup><span id="2body">[[#2foot|[2]]]</span></sup><br />
|-<br />
! Criteria<br />
! Sense Reversal<br />
Centralized Barrier<br />
! Combining Tree<br />
Barrier<br />
! Barrier Network<br />
(Hardware)<br />
|-<br />
| Latency<br />
| O(1)<br />
| O(log p)<br />
| O(log p)<br />
|-<br />
| Traffic<br />
| O(p<sup>2</sup>)<br />
| O(p)<br />
| moved to a separate network<br />
|}<br />
<br />
<br />
<br />
[[Image:Ch9wiki05.png|frame|none|Additive Schwarz Preconditioned Conjugate Gradient (ASPCG) kernel on Altix System - This figure shows the timings of the ASPCG kernel using the different barrier implementations. It can be seen that the blocking barrier does not scale with number of threads as with increase in number of threads the contention increases.<sup><span id="7body">[[#7foot|[7]]]</span></sup>]]<br />
<br />
<br />
<br />
[[Image:Ch9wiki06.png|frame|none|EPCC Microbenchmark - This figure shows the timings to implement a barrier of the EPCC Microbenchmark using the different barrier implementations. It can be seen that the blocking barrier/centralized blocking barrier does not scale with number of threads as with increase in number of threads the contention increases.<sup><span id="7body">[[#7foot|[7]]]</span></sup>]]<br />
<br />
== Sense-Reversal Barrier ==<br />
<br />
This barrier is a centralized barrier where a single count variable protected by a lock is shared among all the threads. Each thread on reaching the barrier increments its value and waits till the value of variable has reached the number of threads for which barrier was implemented.<br />
<br />
Since all the threads are spinning around a single variable the miss rate is scales quadratically with number of processors.<br />
<br />
[[Image:Ch9wiki01.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Combining Tree Barrier ==<br />
<br />
This barrier is a distributed barrier where group of processors form clusters and updates value of a local variable. The local variable on reaching a value equal to number of threads updating it, proceeds to increment another variable higher in hierarchy in the combining tree. When the variable in the highest level of hierarchy in the combining tree reaches its max value it is considered that all the threads have reached the barrier and synchronization is complete.<br />
<br />
Since in this case all the threads are updated local variables in form of smaller groups the miss rate is not as high as sense-reversal barrier.<br />
<br />
The diagram below shows the operation of combining tree barrier with threads grouped in two groups. The variables C0 and C1 are local to each group and C2 is the variable that is at higher level of hierarchy in the tree.<br />
<br />
[[Image:Ch9wiki02.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Tournament Barrier ==<br />
<br />
In tournament barrier the threads are considered to be leaves at the end of a binary tree and each node represents a flag. Two threads compete with each other and the loser thread is allowed to set the flag and move to higher level and compete to lose with the loser thread from other section of binary tree. Thus the thread which completes last is able to set the highest flag in the binary tree. On setting the flag it indicates to all the threads that barrier has been completed and thus synchronization is achieved.<br />
<br />
[[Image:Ch9wiki03.png]]<sup><span id="8body">[[#8foot|[8]]]</span></sup><br />
<br />
== Disseminating Barrier ==<br />
<br />
In this barrier each thread maintains a record of the activity of other threads. For every round i with n threads, thread A notifies thread (A + 2i) mod n. Thus after logn rounds all the threads are aware of the status of every other thread running and whether it has reached the barrier.<br />
<br />
[[Image:Ch9wiki04.png]]<sup><span id="9body">[[#9foot|[9]]]</span></sup><br />
<br />
= References = <br />
<span id="1foot">[[#1body|1.]]</span> David Culler, J.P. Singh and Anoop Gupta. Parallel Computer Architecture: A Hardware/Software Approach. Morgan Kaufmann. August 1998.<br><br />
<span id="2foot">[[#2body|2.]]</span> Yan Solihin. Fundamentals of Parallel Computer Architecture: Multichip and Multicore Systems. Solihin Books. August 2009.<br><br />
<span id="3foot">[[#3body|3.]]</span> Thomas E. Anderson. The Performance of Spin Lock Alternatives for Shared-memory Multiprocessors. http://www.cc.gatech.edu/classes/AY2010/cs4210_fall/papers/anderson-spinlock.pdf<br><br />
<span id="4foot">[[#4body|4.]]</span> http://www.statemaster.com/encyclopedia/Lock-(computer-science)<br><br />
<span id="5foot">[[#5body|5.]]</span> http://faydoc.tripod.com/cpu/cmpxchg.htm<br><br />
<span id="6foot">[[#6body|6.]]</span> http://www.cs.duke.edu/courses/fall09/cps110/slides/threads3.ppt<br><br />
<span id="7foot">[[#7body|7.]]</span> Scalability evaluation of barrier algorithms for OpenMP. http://www2.cs.uh.edu/~hpctools/pub/iwomp-barrier.pdf<br><br />
<span id="8foot">[[#8body|8.]]</span> Carwyn Ball and Mark Bull. Barrier Synchronization in Java. http://www.ukhec.ac.uk/publications/reports/synch_java.pdf<br><br />
<span id="9foot">[[#9body|9.]]</span> http://www.cs.brown.edu/courses/cs176/barrier.ppt<br></div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=ECE506_Main_Page&diff=74651ECE506 Main Page2013-04-03T18:49:23Z<p>Schakra8: /* Supplements to Solihin Text */</p>
<hr />
<div>This page serves as a portal for all wiki material related to CSC506 and ECE506. Link to any new wiki pages from this page, and add links to any current pages.<br />
<br />
=Supplements to Solihin Text=<br />
<br />
Post links to the textbook supplements in this section.<br />
*Chapter 2 [[CSC/ECE 506 Spring 2011/ch2 dm | CSC/ECE 506 Spring 2011/ch2 dm]]<br />
*Chapter 2 [[Parallel_Programming_Models | Parallel Programming Models]]<br />
*Chapter 2 (Still being revised) [[CSC/ECE 506 Spring 2011/ch2 cl | CSC/ECE 506 Spring 2011/ch2 cl]]<br />
*Chapter 2a [[ CSC/ECE 506 Spring 2011/ch2a mc | Current Data-Parallel Architectures ]]<br />
*Chapter 2a [[ CSC/ECE 506 Spring_2012/2a va ]]<br />
*Chapter 2b [[CSC/ECE 506 Spring 2012/ch2b cm | CSC/ECE 506 Spring 2012/ch2b cm]]<br />
*Chapter 2b [[ECE506_CSC/ECE_506_Spring_2012/2b_az | CSC/ECE 506 Spring 2012/2b az - Data-Parallel Processing with the AMD HD 6900 Series Graphics Processing Unit]]<br />
*Chapter 3 (Final Revision) [[ CSC/ECE 506 Spring 2011/ch3 ab | Parallel Architecture Mechanisms and Programming Models ]]<br />
*Chapter 4a[[ CSC/ECE 506 Spring 2011/ch4a ob | Parallelization of Nelder Mead Algorithm ]]<br />
*Chapter 4a (Under Construction) [[ CSC/ECE_506_Spring_2011/ch4a_bm | Parallelization of Algorithms ]]<br />
*Chapter 4a [[ CSC/ECE 506 Spring 2011/ch4a zz | CSC/ECE 506 Spring 2011/ch4a zz ]]<br />
*Chapter 4b [[Chapter 4b CSC/ECE 506 Spring 2011 / ch4b]]<br />
*Chapter 5a [[ CSC/ECE 506 Spring 2012/ch5a ja | CSC/ECE 506 Spring 2012/ch5a ja ]]<br />
*Chapter 9a [[CSC/ECE 506 Spring 2012/ch9a cm | CSC/ECE 506 Spring 2012/ch9a cm]]<br />
*Chapter 6a (Under Construction) [[ CSC/ECE 506 Spring 2011/ch6a jp | CSC/ECE 506 Spring 2011/ch6a jp ]]<br />
*Chapter 6a (Under Construction) [[ CSC/ECE 506 Spring 2011/ch6a ep | CSC/ECE 506 Spring 2011/ch6a ep ]]<br />
*Chapter 6b (Ready for First Review) [[CSC/ECE 506 Spring 2011/ch6b ab | CSC/ECE 506 Spring 2011/ch6b ab]]<br />
*Chapter 7 (Under Construction) [[CSC/ECE 506 Spring 2011/ch7 jp | CSC/ECE 506 Spring 2011/ch7 jp]]<br />
*Chapter 8 [[CSC/ECE 506 Spring 2011/ch8 mc | CSC/ECE 506 Spring 2011/ch8 mc]]<br />
*Chapter 10 (Under Construction) [[CSC/ECE 506 Spring 2011/ch10 sb | CSC/ECE 506 Spring 2011/ch10 sb]]<br />
*Chapter 10 [[CSC/ECE 506 Spring 2012/ch10 sj | CSC/ECE 506 Spring 2012/ch10 sj]]<br />
*Chapter 10a [[CSC/ECE_506_Spring_2011/ch10a_dc | CSC/ECE_506_Spring_2011/ch10a_dc]]<br />
*Chapter 11 [[CSC/ECE_506_Spring_2011/ch11_BB_EP | Chapter 11 Supplement]]<br />
*Chapter 11 [[Scalable_Coherent_Interface | SCI (Scalable Coherent Interface) ]]<br />
*Chapter 12 [[ CSC/ECE 506 Spring 2011/ch12 ob | Interconnection Network Topologies and Routing Algorithms]]<br />
*Chapter 12 (Ready for Final Review) [[ CSC/ECE 506 Spring 2011/ch12 aj | Interconnection Network Topologies and Routing Algorithms]]<br />
*Chapter 12 [[ CSC/ECE 506 Spring 2011/ch12 | Interconnection Network Topologies]]<br />
*[[CSC/ECE 506 Spring 2012/1a ry]]<br />
*[[CSC/ECE 506 Spring 2012/1c dm]]<br />
*[[CSC/ECE 506 Spring 2012/1c cl]]<br />
*[[CSC/ECE 506 Spring 2012/1a mw]]<br />
*[[CSC/ECE 506 Spring 2012/3a yw]]<br />
*[[CSC/ECE 506 Spring 2012/7b yw]]<br />
*[[CSC/ECE 506 Spring 2012/3b sk]]<br />
*[[CSC/ECE 506 Spring 2012/4b rs]]<br />
*[[CSC/ECE 506 Spring 2012/6b am]]<br />
*[[CSC/ECE 506 Spring 2012/8a cj]]<br />
*[[CSC/ECE 506 Spring 2012/10a dr]]<br />
*[[CSC/ECE 506 Spring 2012/10a jp]]<br />
*[[CSC/ECE 506 Spring 2012/9a ms]]<br />
*[[CSC/ECE 506 Spring 2012/10b sr]]<br />
*Chapter 11a [[ECE506_CSC/ECE_506_Spring_2012/11a_az | CSC/ECE 506 Spring 2012/11a az - Performance of DSM system]]<br />
*[[CSC/ECE 506 Spring 2012/12b jh]]<br />
*[[CSC/ECE 506 Spring 2010/8a fu]]<br />
*[[CSC/ECE 506 Spring 2010/8a sk]]<br />
*[[CSC/ECE 506 Spring 2012/11a ht]]<br />
*[[CSC/ECE 506 Spring 2013/1b dj]]<br />
*[[CSC/ECE 506 Spring 2013/1a sp]]<br />
*[[CSC/ECE 506 Spring 2013/1d ks]]<br />
*[[CSC/ECE 506 Spring 2013/2b so]]<br />
*[[CSC/ECE 506 Spring 2013/1c ad]]<br />
*[[CSC/ECE 506 Spring 2013/3b xz]]<br />
*[[CSC/ECE_506_Spring_2013/4a_aj]]<br />
*[[CSC/ECE_506_Spring_2013/4a_ss]]<br />
*[[CSC/ECE_506_Spring_2013/1a_ag]]<br />
* Chapter 3a [[CSC/ECE_506_Spring_2013/3a_bs]]<br />
* Chapter 6a [[CSC/ECE_506_Spring_2013/6a_cs]]<br />
* Chapter 5a [[CSC/ECE_506_Spring_2013/5a_ks]]<br />
* Chapter 8a [[CSC/ECE_506_Spring_2013/8a_an]]<br />
* Chapter 7a [[CSC/ECE_506_Spring_2013/7a_bs]]<br />
* Chapter 8b [[CSC/ECE_506_Spring_2013/8b_ap]]<br />
* Chpater 8c [[CSC/ECE_506_Spring_2013/8c_da]]<br />
* Chpater 10a [[CSC/ECE_506_Spring_2013/10a_os]]<br />
* Chapter 10c [[CSC/ECE_506_Spring_2013/10c_ks]]<br />
*[[CSC/ECE_506_Spring_2013/9b_sc]]</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=ECE506_Main_Page&diff=72632ECE506 Main Page2013-02-12T20:59:29Z<p>Schakra8: /* Supplements to Solihin Text */</p>
<hr />
<div>This page serves as a portal for all wiki material related to CSC506 and ECE506. Link to any new wiki pages from this page, and add links to any current pages.<br />
<br />
=Supplements to Solihin Text=<br />
<br />
Post links to the textbook supplements in this section.<br />
*Chapter 2 [[CSC/ECE 506 Spring 2011/ch2 dm | CSC/ECE 506 Spring 2011/ch2 dm]]<br />
*Chapter 2 [[Parallel_Programming_Models | Parallel Programming Models]]<br />
*Chapter 2 (Still being revised) [[CSC/ECE 506 Spring 2011/ch2 cl | CSC/ECE 506 Spring 2011/ch2 cl]]<br />
*Chapter 2a [[ CSC/ECE 506 Spring 2011/ch2a mc | Current Data-Parallel Architectures ]]<br />
*Chapter 2a [[ CSC/ECE 506 Spring_2012/2a va ]]<br />
*Chapter 2b [[CSC/ECE 506 Spring 2012/ch2b cm | CSC/ECE 506 Spring 2012/ch2b cm]]<br />
*Chapter 2b [[ECE506_CSC/ECE_506_Spring_2012/2b_az | CSC/ECE 506 Spring 2012/2b az - Data-Parallel Processing with the AMD HD 6900 Series Graphics Processing Unit]]<br />
*Chapter 3 (Final Revision) [[ CSC/ECE 506 Spring 2011/ch3 ab | Parallel Architecture Mechanisms and Programming Models ]]<br />
*Chapter 4a[[ CSC/ECE 506 Spring 2011/ch4a ob | Parallelization of Nelder Mead Algorithm ]]<br />
*Chapter 4a (Under Construction) [[ CSC/ECE_506_Spring_2011/ch4a_bm | Parallelization of Algorithms ]]<br />
*Chapter 4a [[ CSC/ECE 506 Spring 2011/ch4a zz | CSC/ECE 506 Spring 2011/ch4a zz ]]<br />
*Chapter 4b [[Chapter 4b CSC/ECE 506 Spring 2011 / ch4b]]<br />
*Chapter 5a [[ CSC/ECE 506 Spring 2012/ch5a ja | CSC/ECE 506 Spring 2012/ch5a ja ]]<br />
*Chapter 9a [[CSC/ECE 506 Spring 2012/ch9a cm | CSC/ECE 506 Spring 2012/ch9a cm]]<br />
*Chapter 6a (Under Construction) [[ CSC/ECE 506 Spring 2011/ch6a jp | CSC/ECE 506 Spring 2011/ch6a jp ]]<br />
*Chapter 6a (Under Construction) [[ CSC/ECE 506 Spring 2011/ch6a ep | CSC/ECE 506 Spring 2011/ch6a ep ]]<br />
*Chapter 6b (Ready for First Review) [[CSC/ECE 506 Spring 2011/ch6b ab | CSC/ECE 506 Spring 2011/ch6b ab]]<br />
*Chapter 7 (Under Construction) [[CSC/ECE 506 Spring 2011/ch7 jp | CSC/ECE 506 Spring 2011/ch7 jp]]<br />
*Chapter 8 [[CSC/ECE 506 Spring 2011/ch8 mc | CSC/ECE 506 Spring 2011/ch8 mc]]<br />
*Chapter 10 (Under Construction) [[CSC/ECE 506 Spring 2011/ch10 sb | CSC/ECE 506 Spring 2011/ch10 sb]]<br />
*Chapter 10 [[CSC/ECE 506 Spring 2012/ch10 sj | CSC/ECE 506 Spring 2012/ch10 sj]]<br />
*Chapter 10a [[CSC/ECE_506_Spring_2011/ch10a_dc | CSC/ECE_506_Spring_2011/ch10a_dc]]<br />
*Chapter 11 [[CSC/ECE_506_Spring_2011/ch11_BB_EP | Chapter 11 Supplement]]<br />
*Chapter 11 [[Scalable_Coherent_Interface | SCI (Scalable Coherent Interface) ]]<br />
*Chapter 12 [[ CSC/ECE 506 Spring 2011/ch12 ob | Interconnection Network Topologies and Routing Algorithms]]<br />
*Chapter 12 (Ready for Final Review) [[ CSC/ECE 506 Spring 2011/ch12 aj | Interconnection Network Topologies and Routing Algorithms]]<br />
*Chapter 12 [[ CSC/ECE 506 Spring 2011/ch12 | Interconnection Network Topologies]]<br />
*[[CSC/ECE 506 Spring 2012/1a ry]]<br />
*[[CSC/ECE 506 Spring 2012/1c dm]]<br />
*[[CSC/ECE 506 Spring 2012/1c cl]]<br />
*[[CSC/ECE 506 Spring 2012/1a mw]]<br />
*[[CSC/ECE 506 Spring 2012/3a yw]]<br />
*[[CSC/ECE 506 Spring 2012/7b yw]]<br />
*[[CSC/ECE 506 Spring 2012/3b sk]]<br />
*[[CSC/ECE 506 Spring 2012/4b rs]]<br />
*[[CSC/ECE 506 Spring 2012/6b am]]<br />
*[[CSC/ECE 506 Spring 2012/8a cj]]<br />
*[[CSC/ECE 506 Spring 2012/10a dr]]<br />
*[[CSC/ECE 506 Spring 2012/10a jp]]<br />
*[[CSC/ECE 506 Spring 2012/9a ms]]<br />
*[[CSC/ECE 506 Spring 2012/10b sr]]<br />
*Chapter 11a [[ECE506_CSC/ECE_506_Spring_2012/11a_az | CSC/ECE 506 Spring 2012/11a az - Performance of DSM system]]<br />
*[[CSC/ECE 506 Spring 2012/12b jh]]<br />
*[[CSC/ECE 506 Spring 2010/8a fu]]<br />
*[[CSC/ECE 506 Spring 2010/8a sk]]<br />
*[[CSC/ECE 506 Spring 2012/11a ht]]<br />
*[[CSC/ECE 506 Spring 2013/1b dj]]<br />
*[[CSC/ECE 506 Spring 2013/1a sp]]<br />
*[[CSC/ECE 506 Spring 2013/1d ks]]<br />
*[[CSC/ECE 506 Spring 2013/2b so]]<br />
*[[CSC/ECE 506 Spring 2013/1c ad]]<br />
*[[CSC/ECE 506 Spring 2013/3b xz]]<br />
*[[CSC/ECE_506_Spring_2013/4a_aj]]<br />
*[[CSC/ECE_506_Spring_2013/4a_ss]]</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=ECE506_Main_Page&diff=72631ECE506 Main Page2013-02-12T20:59:01Z<p>Schakra8: /* Supplements to Solihin Text */</p>
<hr />
<div>This page serves as a portal for all wiki material related to CSC506 and ECE506. Link to any new wiki pages from this page, and add links to any current pages.<br />
<br />
=Supplements to Solihin Text=<br />
<br />
Post links to the textbook supplements in this section.<br />
*Chapter 2 [[CSC/ECE 506 Spring 2011/ch2 dm | CSC/ECE 506 Spring 2011/ch2 dm]]<br />
*Chapter 2 [[Parallel_Programming_Models | Parallel Programming Models]]<br />
*Chapter 2 (Still being revised) [[CSC/ECE 506 Spring 2011/ch2 cl | CSC/ECE 506 Spring 2011/ch2 cl]]<br />
*Chapter 2a [[ CSC/ECE 506 Spring 2011/ch2a mc | Current Data-Parallel Architectures ]]<br />
*Chapter 2a [[ CSC/ECE 506 Spring_2012/2a va ]]<br />
*Chapter 2b [[CSC/ECE 506 Spring 2012/ch2b cm | CSC/ECE 506 Spring 2012/ch2b cm]]<br />
*Chapter 2b [[ECE506_CSC/ECE_506_Spring_2012/2b_az | CSC/ECE 506 Spring 2012/2b az - Data-Parallel Processing with the AMD HD 6900 Series Graphics Processing Unit]]<br />
*Chapter 3 (Final Revision) [[ CSC/ECE 506 Spring 2011/ch3 ab | Parallel Architecture Mechanisms and Programming Models ]]<br />
*Chapter 4a[[ CSC/ECE 506 Spring 2011/ch4a ob | Parallelization of Nelder Mead Algorithm ]]<br />
*Chapter 4a (Under Construction) [[ CSC/ECE_506_Spring_2011/ch4a_bm | Parallelization of Algorithms ]]<br />
*Chapter 4a [[ CSC/ECE 506 Spring 2011/ch4a zz | CSC/ECE 506 Spring 2011/ch4a zz ]]<br />
*Chapter 4b [[Chapter 4b CSC/ECE 506 Spring 2011 / ch4b]]<br />
*Chapter 5a [[ CSC/ECE 506 Spring 2012/ch5a ja | CSC/ECE 506 Spring 2012/ch5a ja ]]<br />
*Chapter 9a [[CSC/ECE 506 Spring 2012/ch9a cm | CSC/ECE 506 Spring 2012/ch9a cm]]<br />
*Chapter 6a (Under Construction) [[ CSC/ECE 506 Spring 2011/ch6a jp | CSC/ECE 506 Spring 2011/ch6a jp ]]<br />
*Chapter 6a (Under Construction) [[ CSC/ECE 506 Spring 2011/ch6a ep | CSC/ECE 506 Spring 2011/ch6a ep ]]<br />
*Chapter 6b (Ready for First Review) [[CSC/ECE 506 Spring 2011/ch6b ab | CSC/ECE 506 Spring 2011/ch6b ab]]<br />
*Chapter 7 (Under Construction) [[CSC/ECE 506 Spring 2011/ch7 jp | CSC/ECE 506 Spring 2011/ch7 jp]]<br />
*Chapter 8 [[CSC/ECE 506 Spring 2011/ch8 mc | CSC/ECE 506 Spring 2011/ch8 mc]]<br />
*Chapter 10 (Under Construction) [[CSC/ECE 506 Spring 2011/ch10 sb | CSC/ECE 506 Spring 2011/ch10 sb]]<br />
*Chapter 10 [[CSC/ECE 506 Spring 2012/ch10 sj | CSC/ECE 506 Spring 2012/ch10 sj]]<br />
*Chapter 10a [[CSC/ECE_506_Spring_2011/ch10a_dc | CSC/ECE_506_Spring_2011/ch10a_dc]]<br />
*Chapter 11 [[CSC/ECE_506_Spring_2011/ch11_BB_EP | Chapter 11 Supplement]]<br />
*Chapter 11 [[Scalable_Coherent_Interface | SCI (Scalable Coherent Interface) ]]<br />
*Chapter 12 [[ CSC/ECE 506 Spring 2011/ch12 ob | Interconnection Network Topologies and Routing Algorithms]]<br />
*Chapter 12 (Ready for Final Review) [[ CSC/ECE 506 Spring 2011/ch12 aj | Interconnection Network Topologies and Routing Algorithms]]<br />
*Chapter 12 [[ CSC/ECE 506 Spring 2011/ch12 | Interconnection Network Topologies]]<br />
*[[CSC/ECE 506 Spring 2012/1a ry]]<br />
*[[CSC/ECE 506 Spring 2012/1c dm]]<br />
*[[CSC/ECE 506 Spring 2012/1c cl]]<br />
*[[CSC/ECE 506 Spring 2012/1a mw]]<br />
*[[CSC/ECE 506 Spring 2012/3a yw]]<br />
*[[CSC/ECE 506 Spring 2012/7b yw]]<br />
*[[CSC/ECE 506 Spring 2012/3b sk]]<br />
*[[CSC/ECE 506 Spring 2012/4b rs]]<br />
*[[CSC/ECE 506 Spring 2012/6b am]]<br />
*[[CSC/ECE 506 Spring 2012/8a cj]]<br />
*[[CSC/ECE 506 Spring 2012/10a dr]]<br />
*[[CSC/ECE 506 Spring 2012/10a jp]]<br />
*[[CSC/ECE 506 Spring 2012/9a ms]]<br />
*[[CSC/ECE 506 Spring 2012/10b sr]]<br />
*Chapter 11a [[ECE506_CSC/ECE_506_Spring_2012/11a_az | CSC/ECE 506 Spring 2012/11a az - Performance of DSM system]]<br />
*[[CSC/ECE 506 Spring 2012/12b jh]]<br />
*[[CSC/ECE 506 Spring 2010/8a fu]]<br />
*[[CSC/ECE 506 Spring 2010/8a sk]]<br />
*[[CSC/ECE 506 Spring 2012/11a ht]]<br />
*[[CSC/ECE 506 Spring 2013/1b dj]]<br />
*[[MISD Architecture(CSC/ECE 506 Spring 2013/1a sp)]]<br />
*[[CSC/ECE 506 Spring 2013/1d ks]]<br />
*[[CSC/ECE 506 Spring 2013/2b so]]<br />
*[[CSC/ECE 506 Spring 2013/1c ad]]<br />
*[[CSC/ECE 506 Spring 2013/3b xz]]<br />
*[[CSC/ECE_506_Spring_2013/4a_aj]]<br />
*[[CSC/ECE_506_Spring_2013/4a_ss]]</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=ECE506_Main_Page&diff=72630ECE506 Main Page2013-02-12T20:58:36Z<p>Schakra8: /* Supplements to Solihin Text */</p>
<hr />
<div>This page serves as a portal for all wiki material related to CSC506 and ECE506. Link to any new wiki pages from this page, and add links to any current pages.<br />
<br />
=Supplements to Solihin Text=<br />
<br />
Post links to the textbook supplements in this section.<br />
*Chapter 2 [[CSC/ECE 506 Spring 2011/ch2 dm | CSC/ECE 506 Spring 2011/ch2 dm]]<br />
*Chapter 2 [[Parallel_Programming_Models | Parallel Programming Models]]<br />
*Chapter 2 (Still being revised) [[CSC/ECE 506 Spring 2011/ch2 cl | CSC/ECE 506 Spring 2011/ch2 cl]]<br />
*Chapter 2a [[ CSC/ECE 506 Spring 2011/ch2a mc | Current Data-Parallel Architectures ]]<br />
*Chapter 2a [[ CSC/ECE 506 Spring_2012/2a va ]]<br />
*Chapter 2b [[CSC/ECE 506 Spring 2012/ch2b cm | CSC/ECE 506 Spring 2012/ch2b cm]]<br />
*Chapter 2b [[ECE506_CSC/ECE_506_Spring_2012/2b_az | CSC/ECE 506 Spring 2012/2b az - Data-Parallel Processing with the AMD HD 6900 Series Graphics Processing Unit]]<br />
*Chapter 3 (Final Revision) [[ CSC/ECE 506 Spring 2011/ch3 ab | Parallel Architecture Mechanisms and Programming Models ]]<br />
*Chapter 4a[[ CSC/ECE 506 Spring 2011/ch4a ob | Parallelization of Nelder Mead Algorithm ]]<br />
*Chapter 4a (Under Construction) [[ CSC/ECE_506_Spring_2011/ch4a_bm | Parallelization of Algorithms ]]<br />
*Chapter 4a [[ CSC/ECE 506 Spring 2011/ch4a zz | CSC/ECE 506 Spring 2011/ch4a zz ]]<br />
*Chapter 4b [[Chapter 4b CSC/ECE 506 Spring 2011 / ch4b]]<br />
*Chapter 5a [[ CSC/ECE 506 Spring 2012/ch5a ja | CSC/ECE 506 Spring 2012/ch5a ja ]]<br />
*Chapter 9a [[CSC/ECE 506 Spring 2012/ch9a cm | CSC/ECE 506 Spring 2012/ch9a cm]]<br />
*Chapter 6a (Under Construction) [[ CSC/ECE 506 Spring 2011/ch6a jp | CSC/ECE 506 Spring 2011/ch6a jp ]]<br />
*Chapter 6a (Under Construction) [[ CSC/ECE 506 Spring 2011/ch6a ep | CSC/ECE 506 Spring 2011/ch6a ep ]]<br />
*Chapter 6b (Ready for First Review) [[CSC/ECE 506 Spring 2011/ch6b ab | CSC/ECE 506 Spring 2011/ch6b ab]]<br />
*Chapter 7 (Under Construction) [[CSC/ECE 506 Spring 2011/ch7 jp | CSC/ECE 506 Spring 2011/ch7 jp]]<br />
*Chapter 8 [[CSC/ECE 506 Spring 2011/ch8 mc | CSC/ECE 506 Spring 2011/ch8 mc]]<br />
*Chapter 10 (Under Construction) [[CSC/ECE 506 Spring 2011/ch10 sb | CSC/ECE 506 Spring 2011/ch10 sb]]<br />
*Chapter 10 [[CSC/ECE 506 Spring 2012/ch10 sj | CSC/ECE 506 Spring 2012/ch10 sj]]<br />
*Chapter 10a [[CSC/ECE_506_Spring_2011/ch10a_dc | CSC/ECE_506_Spring_2011/ch10a_dc]]<br />
*Chapter 11 [[CSC/ECE_506_Spring_2011/ch11_BB_EP | Chapter 11 Supplement]]<br />
*Chapter 11 [[Scalable_Coherent_Interface | SCI (Scalable Coherent Interface) ]]<br />
*Chapter 12 [[ CSC/ECE 506 Spring 2011/ch12 ob | Interconnection Network Topologies and Routing Algorithms]]<br />
*Chapter 12 (Ready for Final Review) [[ CSC/ECE 506 Spring 2011/ch12 aj | Interconnection Network Topologies and Routing Algorithms]]<br />
*Chapter 12 [[ CSC/ECE 506 Spring 2011/ch12 | Interconnection Network Topologies]]<br />
*[[CSC/ECE 506 Spring 2012/1a ry]]<br />
*[[CSC/ECE 506 Spring 2012/1c dm]]<br />
*[[CSC/ECE 506 Spring 2012/1c cl]]<br />
*[[CSC/ECE 506 Spring 2012/1a mw]]<br />
*[[CSC/ECE 506 Spring 2012/3a yw]]<br />
*[[CSC/ECE 506 Spring 2012/7b yw]]<br />
*[[CSC/ECE 506 Spring 2012/3b sk]]<br />
*[[CSC/ECE 506 Spring 2012/4b rs]]<br />
*[[CSC/ECE 506 Spring 2012/6b am]]<br />
*[[CSC/ECE 506 Spring 2012/8a cj]]<br />
*[[CSC/ECE 506 Spring 2012/10a dr]]<br />
*[[CSC/ECE 506 Spring 2012/10a jp]]<br />
*[[CSC/ECE 506 Spring 2012/9a ms]]<br />
*[[CSC/ECE 506 Spring 2012/10b sr]]<br />
*Chapter 11a [[ECE506_CSC/ECE_506_Spring_2012/11a_az | CSC/ECE 506 Spring 2012/11a az - Performance of DSM system]]<br />
*[[CSC/ECE 506 Spring 2012/12b jh]]<br />
*[[CSC/ECE 506 Spring 2010/8a fu]]<br />
*[[CSC/ECE 506 Spring 2010/8a sk]]<br />
*[[CSC/ECE 506 Spring 2012/11a ht]]<br />
*[[CSC/ECE 506 Spring 2013/1b dj]]<br />
*[[<b>MISD Architecture</b>(CSC/ECE 506 Spring 2013/1a sp)]]<br />
*[[CSC/ECE 506 Spring 2013/1d ks]]<br />
*[[CSC/ECE 506 Spring 2013/2b so]]<br />
*[[CSC/ECE 506 Spring 2013/1c ad]]<br />
*[[CSC/ECE 506 Spring 2013/3b xz]]<br />
*[[CSC/ECE_506_Spring_2013/4a_aj]]<br />
*[[CSC/ECE_506_Spring_2013/4a_ss]]</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/1a_sp&diff=72487CSC/ECE 506 Spring 2013/1a sp2013-02-11T04:45:07Z<p>Schakra8: /* = */</p>
<hr />
<div>==Multiple Instruction Single Data (MISD) Architecture==<br />
<br />
Multiple Instruction Single Data, or MISD, architecture is one of the four general categories of multi-processor or multi-core architectures described by Flynn's Taxonomy.<br />
<br />
===MISD Architectures within the Context of Flynn’s Taxonomy<ref>http://en.wikipedia.org/wiki/Flynn's_taxonomy</ref><ref>http://www.phy.ornl.gov/csep/ca/node11.html</ref>===<br />
<br />
Flynn's Taxonomy categorizes parallel computing architectures based on the number of concurrent information streams that flowed into the processor. According to Flynn's Taxonomy, there are two kinds of logical streams of information that flows into a parallel computer, the Instruction (or Control) Stream and the Data Stream. Note that for the purposes of classification using Flynn’s Taxonomy, it does not matter whether or not the streams are physically independent of each other on the hardware, only that the streams can be logically treated as independent. Based on how many of each type of stream flows into the parallel computer (single vs. multiple), Flynn's Taxonomy divides parallel architectures into four categories. They are Single Instruction, Single Data (SISD), Single Instruction, Multiple Data (SIMD), Multiple Instruction, Single Data (MISD), and Multiple Instruction, Multiple Data (MIMD). <br />
<br />
The following table (Figure 1) shows the categories of different parallel architectures according to Flynn:<ref>https://computing.llnl.gov/tutorials/parallel_comp/#Flynn</ref><br />
<br />
[[Image:Flynn's Taxonomy.PNG|thumb|center|400px|Figure 1. [http://en.wikipedia.org/wiki/Michael_J._Flynn Flynn]'s Taxonomy [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
===Overview of the MISD Architecture===<br />
<br />
In order to be categorized as MISD architecture, there must be at least two Instruction streams and there can only be one Data stream as shown on Figure 2.[[Image:MISD.PNG|thumb|right|100px|Figure 2. MISD Architecture [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]] As per the description according to Flynn’s Taxonomy, each processing elements should receive the same data and execute different instructions on the same data simultaneously. <br />
<br />
There are very few implementations of the classical MISD architecture as described by Flynn’s Taxonomy, none of which are commercial, general purpose parallel computers since the MIMD and SIMD are often better suited to common data parallel techniques. Under most circumstances MIMD and SIMD architectures tend to provide better scaling and more efficient use of computational resources. Instead of commercial use, MISD architectures are mainly used to create custom hardware in order to solve specific problems. The majority of the implementations have been created are in academia.<br />
<br />
Over the years, several architectures have been described as MISD even though the architectures do not completely follow the description in Flynn’s Taxonomy. One commonly cited example is the Pipeline architecture where the data is passed from processing element to processing element and each processing element executes a different instruction. It is often argued that this is not a true MISD architecture as the data changes as it moves through the pipeline. Similarly, other architectures or applications exist which may be called MISD by some while others disagree. The following section looks at some of those implementations.<br />
<br />
==Implementations of MISD architecture==<br />
<br />
Some major implementations of MISD are in the fields of systolic arrays, pattern matching, flight control systems and fault tolerance systems. These MISD implementations are discussed below.<br />
<br />
===Systolic Array===<br />
<br />
A systolic array is an arrangement of processors in an array where data flows synchronously across the array between neighbors, usually with different data flowing in different directions. Each Processor at each step takes in data from one or more neighbors, processes it and, in the next step, outputs results in the opposite direction. <br />
The systolic array paradigm, data-stream-driven by data counters, is the counterpart of the [http://en.wikipedia.org/wiki/Von_Neumann_model von Neumann paradigm], instruction-stream-driven by a program counter. Because a systolic array usually sends and receives multiple data streams, and multiple data counters are needed to generate these data streams, it supports data parallelism.<ref>http://en.wikipedia.org/wiki/Systolic_array</ref><br />
<br />
====Type of Systolic Arrays<ref>Johnson, K.T.; Hurson, A.R.; Shirazi, B.; , "General-purpose systolic arrays," Computer , vol.26, no.11, pp.20-31, Nov. 1993 doi: 10.1109/2.241423</ref>====<br />
<br />
=====Special-purpose systolic array=====<br />
<br />
An array of hardwired systolic processing elements tailored for a specific application. Typically, many tens or hundreds of cells fit on a single chip. One of the major applications of special-purpose systolic array is in matrix operations. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Systolic_1.png Figure 3] illustrates the algorithm for the sum of a scalar product, computed in a single systolic element. Here, a’s and b’s are synchronously shifted through the processing element to be available for next element. These data synchronously exits the processing element unmodified for the next element. The sum of the products is then shifted out of the accumulator.<br />
[[Image:systolic_1.png|thumb|centre|250px|Figure 3: The algorithm for the sum of a scalar product, computed in systolic element [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
=====General-purpose systolic array=====<br />
<br />
It is an array of systolic processing elements, which gets adapted to a variety of applications via programming or reconfiguration. Array topologies can be either programmable or reconfigurable. Likewise, array cells are either programmable or reconfigurable. This is referred to as Systolic topologies.<br />
<br />
======<font size="2">Programmable Systolic Array</font>======<br />
<br />
It is an array of programmable systolic elements that operates either in SIMD or MIMD fashion. Either the arrays interconnect or each processing unit is programmable and a program controls dataflow through the elements. Programmable systolic arrays are programmable either at a high level or a low level. At either level, programmable arrays can be categorized as either SIMD or MIMD machines.<br />
<br />
<br />
<br />
For the latter approach, the workstation downloads a program to each MIMD (Figure 4) systolic cell. Each cell may be loaded with a different program, or all the cells in the array may be loaded with the same program. Each cell's architecture is somewhat similar to the conventional von Neumann architecture: It contains a control unit, an ALU, and local memory. MIMD systolic cells have more local memory than their SIMD counterparts to support the von Neumann-style organization.<br />
[[Image:systolic_2.png|thumb|centre|250px|Figure 4: The systolic product of two 3x3 matrices [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
<br />
This architecture is defined as Multiple Instruction Multiple Data (MIMD) architecture in [http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf 5]. The architecture has multiple instruction streams for the PEs and a single data stream passing through all the PEs. Thus, it can also be defined as Multiple Instruction Single Data (MISD) architecture. The architecture of Systolic array configuration are controversial as explained in the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Architecture_of_systolic_arrays_as_against_MISD_architecture section 4.1.2.]<br />
<br />
[[Image:systolic_4.png|thumb|centre|250px|Figure 5: General organization of MIMD programmable linear systolic arrays [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
<br />
======<font size="2">Reconfigurable Systolic Array</font>======<br />
[[Image:reconfig.jpg|thumb|right|250px|Figure 6: Block Diagram of the RSA Architecture [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
It is an array of systolic elements that can be programmed at the lowest level. Recent gate density advances in FPGA technology have produced a low-level, reconfigurable systolic array architecture that bridges the gap between special-purpose arrays and the more versatile, programmable general-purpose arrays. The FPGA architecture is unusual because a single hardware platform can be logically reconfigured as an exact duplicate of a special-purpose systolic array. <br />
<br />
The RSA circuit design is based on systolic array architecture consisting of PEs interconnected via SWs as depicted in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Reconfig.jpg Figure 6]. The homogeneous characteristic of the Reconfigurable Systolic Array (RSA) architecture, where each reconfigurable processing element (PE) cell is connected to its nearest neighbors via configurable switch (SW) elements, enables array expansion for parallel processing and facilitates time sharing computation of high-throughput data by individual PEs. Both the PEs and SWs can be reconfigured dynamically with the former as an arithmetic processor and the latter as a flexible router linking the neighboring PE cells. The RSA shifts reconfiguration and input signals into the PEs and SWs on separate data bus which enables the circuit to continue its operation while the reconfiguration is in process.<br />
<br />
====Architecture of systolic arrays as against MISD architecture====<br />
[[Image:comp.png|thumb|right|250px|Figure 7.Comparison between Architecture of systolic arrays and MISD]]<br />
<br />
As from the above mentioned configurations of the Systolic Arrays, it is seen that generally the configurations have multiple processing elements executing different instructions from dedicated instruction streams for each processing element. There is a single data stream that connects the adjacent PEs. Thus, systolic array can be defined as an MISD architecture.<br />
<br />
Many authors say that as the data read as input by one processing element is processed data output of the adjacent PE. The data stream cannot be considered as single because all the data paths do not carry the same data to all the PEs. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Comp.png Figure 7] shows the difference between the Data Stream for Systolic Arrays and the MISD architecture. Thus the systolic array should be considered as “Multiple Data” architecture and not Single Data architecture.<br />
<br />
===Fault Tolerant Systems<ref>http://en.wikipedia.org/wiki/Fault-tolerant_computer_system#Types_of_fault_tolerance</ref>===<br />
<br />
The fault tolerant systems are designed to handle the possible failures in software, hardware or interfaces. The hardware faults include hard disk failures, input or output device failures, etc. and the software and interface faults include driver failures; operator errors, installing unexpected software etc. The hardware faults can be detected and identified by implementing redundant hardware and multiple backups. The software faults can be tolerable by removing the program errors by executing the software redundantly or by implementing small programs that take over the tasks that crash or generate errors.<br />
<br />
[[Image:fault.png|thumb|centre|250px|Figure 8 MISD as fault tolerant architecture]]<br />
<br />
In computer systems, the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Single_Instruction.2C_Multiple_Data_streams_.28SIMD.29 SIMD], [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD] and [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instruction.2C_Multiple_Data_streams_.28MIMD.29 MIMD] architectures facilitate the implementation of the fault tolerance systems by multiple instruction streams or multiple data streams or both. Fault tolerance on computations can be implemented by multiple processors (likely with different architectures) executing the algorithms on the same set of data. The output of each processor is compared with that of the others and M out of N majority voting method is used to determine the faulty processor. Thus MISD architecture is utilized to get the fault tolerance on critical computations.<br />
<br />
There are various examples of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture] being used as fault tolerant architecture. The major examples being flight control systems, nuclear power plants, satellite systems, super collider experiment systems, etc. Here, the flight control system is explained as an example of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture].<br />
<br />
====The Flight Control System – MISD Example for fault tolerance====<br />
<br />
A [http://en.wikipedia.org/wiki/Fly-by-wire Fly-By-Wire] system is used to replace the manual flight control by an electronic control interface. The movements of the flight control in the cockpit are converted to electronic signals and are transmitted to the actuators by wires. The control computers use the feedback from the sensors to compute and control the movement of the actuators to provide the expected response. These computers also perform the task to stabilize the aircraft and perform other tasks without the knowledge of the pilot. Flight control systems must meet extremely high levels of accuracy and functional integrity.<br />
<br />
There are redundant flight control computers present in the flight control system. If one of the flight-control computers crashes, gets damaged or is affected by electromagnetic pulses, the other computer can overrule the faulty one and hence the flight of the aircraft is unharmed.The number of redundant flight control computers is generally more than two, so that any computer whose results disagree with the others is ruled out to be faulty and is either ignored or rebooted.<br />
<br />
=====Multiple Processors Implementation in Boeing 777<ref>Yeh, Y.C.;, "Triple-triple redundant 777 primary flight computer," Aerospace Applications Conference, 1996. Proceedings., 1996 IEEE , vol.1, no., pp.293-307 vol.1, 3-10 Feb 1996<br />
doi: 10.1109/AERO.1996.495891</ref>=====<br />
<br />
In modern computers, the redundant flight control computations are carried out by multiprocessor systems. The triple redundant 777 primary flight computer, has the architecture as shown in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Fig13.png Figure 9].<br />
[[Image:fig13.png|thumb|centre|250px|Figure 9: Architecture of triple redundant 777 primary flight computer [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-7 6]]]<br />
<br />
The system has three primary flight control computers, each of them having three lanes with different processors. The flight control program is compiled for each of the processors which get the input data from the same data bus but drive the output on their individual control bus. Thus each processor executes different instructions but they process the same data. Thus, it is the best suited example of Multiple Instruction Single Data (MISD) architecture.<br />
<br />
The three processors selected for the flight control system of [http://en.wikipedia.org/wiki/Boeing_777 Boeing 777] were [http://en.wikipedia.org/wiki/Intel_80486 Intel 80486], [http://en.wikipedia.org/wiki/Motorola_68040 Motorola 68040] and [http://en.wikipedia.org/wiki/AMD_Am29000 AMD 29050]. The dissimilar processors lead to dissimilar interface hardware circuits and compilers. Each lane of the flight control computer is data synchronized with the other lanes so that all of the lanes read the same frame of data from the flight sensors. As the outputs of each lane can be different, the median value of the outputs is used to select the output of the lane to be considered. The lane which has the median value select hardware selected is said to be in “command mode” whereas the other lanes are said to be in “monitoring mode”. It receives the data from the other Primary Flight Computer (PFC) lanes and performs a median select of the outputs. This provides a fault blocking mechanism before the fault detection and identification by the cross-lane monitoring system. Thus, the MISD based multi computer architecture is capable of detecting generic errors in compilers or in complex hardware devices providing assurance beyond reasonable doubt of the dependability of the Fly-By-Wire system.<br />
<br />
The above mentioned system clearly has individual Instruction Streams as the architecture of each processor is different, thus different instruction sets and different instruction streams. These processors have frame synchronized input data which means they have same set of data to work upon which is fed from a single data stream. Thus the flight control system can be classified under MISD architecture.<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System<ref>M.Gajer;,“ Parallel Image Processing on The TMS320C80 Multiprocessor System”, http://focus.ti.com/pdfs/univ/11-CommunicationAudioSpeech.pdf</ref>===<br />
<br />
The image pre-processing system of this Multiprocessor, the pre-image processing time needed to be made shorter. To achieve the aim of reducing the pre-processing time, different types of image processing architectures for ‘C80 were presented. For example MIMD architecture is used for image histogram equalization; SIMD architecture for median filtering; MISD architecture for directional edge detection.<br />
<br />
The MISD architecture can be explained as follows. Here, each processor performs different operations on the same data set. It is mentioned at many places that MISD architecture was defined only for the sake of completeness of classification due to difficulty in practice to find a computer based on MISD architecture<ref>TMS320C80 System Level Synopsis, Texas Instruments, 1995</ref>. But in case of Image Processing, this assumption need not be true as here making many different operations on same image is rare and hence MISD architecture is a best fit for such cases. As an example of implementation of image processing operations for the MISD architecture the edge detection operation will be considered. The edges should be detected in four different directions (north, south, west and east) by the medium of four different filter masks. Each of the parallel processors of the TMS320C80 calculates one of the directional edges. Finally one obtains four images with detected edges, each in different direction.<br />
<br />
The image processing in computer is very often a multistage process composed of many stages. For example, an algorithm detecting contours of the objects in an image. The algorithm for achieving the before mentioned is composed of four basic image processing operation. The image acquired from the camera in the first step undergoes low-pass filtering, during which it is smoothed and noise is also eliminated. In the next step image is thresholding is performed and each pixel gets black or white depending on that if its grey level is below or above threshold value. The resulting image is a binary image from which edges are detected by the usage of the Laplace filter. In the last step the image with detected contours undergoes logic filtering in the purpose of elimination of single black pixels placed on the white background. For implementing the above processes parallel, a five-stage pipeline was used. The first stage of this pipeline constitutes the master processor which accounts for control of the image acquisition process. Further pipe line stages were implemented for 4 parallel processors as mentioned in table 1.<br />
<br />
<table border=1 align="center"><br />
<tr><td>Stage</td><td>Processor</td><td>Operation</td></tr><br />
<tr><td>1</td><td>MP</td><td>Image acquisition</td></tr><br />
<tr><td>2</td><td>PP0/td><td>Low-pass Filtering</td></tr><br />
<tr><td>3</td><td>PP1</td><td>Thresholding</td></tr><br />
<tr><td>4</td><td>PP2</td><td>Laplace Filtering</td></tr><br />
<tr><td>5</td><td>PP3</td><td>Logic Filtering</td></tr> <br />
</table><br />
<p align="center">Table 1: Pipeline structure for image edge detection[8]</p><br />
<br />
===Recursive Pattern Matching using MISD Architecture <ref>Arne Halaas, Børge Svingen, Magnar Nedland, Pål Sætrom, Ola Snøve, Jr., and Olaf René Birkeland. 2004. A recursive MISD architecture for pattern matching. IEEE Trans. Very Large Scale Integr. Syst. 12, 7 (July 2004), 727-734. DOI=10.1109/TVLSI.2004.830918 http://dx.doi.org/10.1109/TVLSI.2004.830918</ref>===<br />
<br />
The situations where several patterns are required to be matched concurrently in data and no persistent index can be created for look-up requires the use of Online multipattern approximate searching. This algorithm requires evaluation of the hit function H(Si,P) which is the function working on strings and pattern,and requires partitioning P into smaller components and combining the individual results. A recursive approach is required and there exists a need to evaluate the Si value in parallel. Thus it helps in using an architecture with two complete binary trees which take care of the data distribution and result processing. The architecture is implemented with as many PE's that are practically possible within the implementation constraints. The data from the stream is fed to the root node of the data distribution tree and simultaneously distributed to several collection of PE's belonging to separate queries thus implementing the MISD architecture with a single data stream and multiple instruction streams.<br />
<br />
==Conclusion==<br />
The MISD architecture is one of the four types of parallel architecture described in Flynn’s Taxonomy. By definition, the MISD architecture has multiple instruction streams and a single data stream. The definition also describes this architecture as requiring that all processing elements receive the same data, but most commonly cited implementations of this class of architecture do not adhere to this detail. The result is there is debate among some experts on whether or not certain implementations are truly MISD.<br />
<br />
MISD architectures are not commonly found in industry as they tend to lack the scalability and efficiency of resource use of the MIMD and SIMD class of architectures. However, MISD architectures are efficient for solving specific classes of problems often found in research. Some examples of its use in research include image processing and systolic arrays. While rare, some implementations do exist outside of academic settings. One such example is fault tolerant systems such as the flight control system on the Boeing 777.<br />
<br />
=='''References'''==<br />
<references/><br />
<br />
=='''Glossary'''==<br />
*'''ALU''': Arithmetic Logic Unit, the control unit in a computer<br />
<br />
*'''CMU''': Carnegie Mellon University<br />
<br />
*'''Fly-By-Wire''': System that replaces the conventional manual flight controls of an aircraft with an electronic interface<br />
<br />
*'''FPGA''': Field Programmable Gate Array<br />
<br />
*'''MIMD''': Multiple Instruction Multiple Data<br />
<br />
*'''MISD''': Multiple Instruction Single Data<br />
<br />
*'''PE''': Processing Element<br />
<br />
*'''PFC''': Primary Flight Computer<br />
<br />
*'''SIMD''': Single Instruction Multiple Data<br />
<br />
*'''SISD''': Single Instruction Single Data<br />
<br />
==Quiz==<br />
<br />
'''Q1: Using the classical definition of the MISD architecture from Flynn’s Taxonomy, what is the minimum number of input streams that must be present to be classified as MISD architecture?'''<br />
<br />
A. One information stream, the data stream, the number of instruction streams do not matter.<br />
<br />
B. Two information streams, one instruction stream, one data stream.<br />
<br />
C. Three information streams, two instruction streams, one data stream.<br />
<br />
D. The number of information streams is irrelevant as per Flynn’s Taxonomy.<br />
<br />
''A1: C, Three information streams, two instruction streams, one data stream.''<br />
<br />
<br />
'''Q2: Why is the Pipeline architecture often cited as an example of MISD?'''<br />
<br />
A. The Pipeline architecture is exactly like the definition of MISD architecture from Flynn’s Taxonomy.<br />
<br />
B. The data to be processed travels linearly across all processing elements with each processing element executing a different instruction and passing it on to the next processing element. This line of data is cited to be a single “stream.”<br />
<br />
C. The data stream sends the same data to all processing elements for every cycle.<br />
<br />
D. Flynn based his description of MISD architecture on the Pipeline architecture in his paper that introduced Flynn’s Taxonomy.<br />
<br />
''A2: B, The data to be processed travels linearly across all processing elements with each processing element executing a different instruction and passing it on to the next processing element. This line of data is cited to be a single “stream.”''<br />
<br />
<br />
'''Q3: Why is the Pipeline architecture NOT considered a true MISD architecture by many experts?'''<br />
<br />
A. This is incorrect; there is consensus that the Pipeline architecture is synonymous with the MISD architecture described by Flynn’s Taxonomy.<br />
<br />
B. There is only a single, global information/control stream in the Pipeline architecture.<br />
<br />
C. The data passed to each processing element is different as it reflects the result of the processing of all the previous processing elements. Flynn’s Taxonomy describes the same data going to every processing element.<br />
<br />
D. It has been proven that it is impossible to build a MISD architecture parallel computer as described by Flynn’s Taxonomy.<br />
<br />
''A3: C, The data passed to each processing element is different as it reflects the result of the processing of all the previous processing elements. Flynn’s Taxonomy describes the same data going to every processing element.''<br />
<br />
<br />
'''Q4: True or False? MISD architecture is the most common parallel computing architecture used for general computing tasks.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A4: B, False, it is often not the best suited architecture for general computing tasks and as such not common outside of academic research.''<br />
<br />
<br />
'''Q5: When classifying a particular parallel architecture, it was noted that there was only one physical connection to provide all information streams, is it possible for this architecture to be classified as MISD architecture? (Note that there are three choices, you need to select the correct reason for the Yes/No/Maybe answer.)'''<br />
<br />
A. Yes, Flynn’s taxonomy does not require the number of physical connections in the hardware correspond to the number of logical streams.<br />
<br />
B. Maybe, we do not have enough information about the architecture to rule out the possibility of this parallel computer having a MISD architecture. We will need to see how the connections are physically laid out on the chip.<br />
<br />
C. No, there must be multiple physical information streams.<br />
<br />
''A5. A, Yes, Flynn’s taxonomy does not require the number of physical connections in the hardware correspond to the number of logical streams.''<br />
<br />
<br />
'''Q6. True or False? To be a MISD architecture, it requires that every processing element is identical.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A6. B, False, it is not required by definition of Flynn’s Taxonomy that all processing elements are identical. See example in the wiki reading about the fault tolerant flight control system on the Boeing 777.''<br />
<br />
<br />
'''Q7: Which of these is an application of the MISD architecture?'''<br />
<br />
A. Keyword search<br />
<br />
B. Controlling an assembly line.<br />
<br />
C. Home computing<br />
<br />
D. Image edge detection.<br />
<br />
''A7: D, Image edge detection.''<br />
<br />
<br />
'''Q8. True or False? The systolic array is another name for the MISD architecture.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A8. B, False, systolic arrays may be implemented using a MISD architecture but they also may be other architectures such as SIMD and MIMD.''<br />
<br />
<br />
'''Q9. True or False? MISD is the considered the easiest parallel architecture to scale.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A9. False, MIMD and SIMD are often used because of advantages in scaling.''<br />
<br />
<br />
'''Q10. Flynn’s Taxonomy relies on the number of logical information streams to classify the types of parallel architectures. What is the difference between a logical information stream and a physical information stream?'''<br />
<br />
A. A logical information stream is a connection between the ALU and the location of the data that is responsible for passing information about boolean (logical) computations, where the physical information stream it the connection between computer and other parts of the network.<br />
<br />
B. Physical information streams are determined by how the information is physically transported around chip while logical information streams are one layer of abstraction above the physical information streams. Logical streams describe conceptually how the information on the stream can be treated.<br />
<br />
C. Logical and physical information streams transport different kinds of information but are no different in implementation.<br />
<br />
D. There is no difference; the two terms can be used interchangeably.<br />
<br />
''A10. B, Physical information streams are determined by how the information is physically transported around chip while logical information streams are one layer of abstraction above the physical information streams. Logical streams describe conceptually how the information on the stream can be treated.''</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/1a_sp&diff=72486CSC/ECE 506 Spring 2013/1a sp2013-02-11T04:44:15Z<p>Schakra8: /* Edge Detection of Images on TMS320C80 Multiprocessor SystemM.Gajer;,“ Parallel Image Processing on The TMS320C80 Multiprocessor System”, http://focus.ti.com/pdfs/univ/11-CommunicationAudioSpeech.pdf */</p>
<hr />
<div>==Multiple Instruction Single Data (MISD) Architecture==<br />
<br />
Multiple Instruction Single Data, or MISD, architecture is one of the four general categories of multi-processor or multi-core architectures described by Flynn's Taxonomy.<br />
<br />
===MISD Architectures within the Context of Flynn’s Taxonomy<ref>http://en.wikipedia.org/wiki/Flynn's_taxonomy</ref><ref>http://www.phy.ornl.gov/csep/ca/node11.html</ref>===<br />
<br />
Flynn's Taxonomy categorizes parallel computing architectures based on the number of concurrent information streams that flowed into the processor. According to Flynn's Taxonomy, there are two kinds of logical streams of information that flows into a parallel computer, the Instruction (or Control) Stream and the Data Stream. Note that for the purposes of classification using Flynn’s Taxonomy, it does not matter whether or not the streams are physically independent of each other on the hardware, only that the streams can be logically treated as independent. Based on how many of each type of stream flows into the parallel computer (single vs. multiple), Flynn's Taxonomy divides parallel architectures into four categories. They are Single Instruction, Single Data (SISD), Single Instruction, Multiple Data (SIMD), Multiple Instruction, Single Data (MISD), and Multiple Instruction, Multiple Data (MIMD). <br />
<br />
The following table (Figure 1) shows the categories of different parallel architectures according to Flynn:<ref>https://computing.llnl.gov/tutorials/parallel_comp/#Flynn</ref><br />
<br />
[[Image:Flynn's Taxonomy.PNG|thumb|center|400px|Figure 1. [http://en.wikipedia.org/wiki/Michael_J._Flynn Flynn]'s Taxonomy [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
===Overview of the MISD Architecture===<br />
<br />
In order to be categorized as MISD architecture, there must be at least two Instruction streams and there can only be one Data stream as shown on Figure 2.[[Image:MISD.PNG|thumb|right|100px|Figure 2. MISD Architecture [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]] As per the description according to Flynn’s Taxonomy, each processing elements should receive the same data and execute different instructions on the same data simultaneously. <br />
<br />
There are very few implementations of the classical MISD architecture as described by Flynn’s Taxonomy, none of which are commercial, general purpose parallel computers since the MIMD and SIMD are often better suited to common data parallel techniques. Under most circumstances MIMD and SIMD architectures tend to provide better scaling and more efficient use of computational resources. Instead of commercial use, MISD architectures are mainly used to create custom hardware in order to solve specific problems. The majority of the implementations have been created are in academia.<br />
<br />
Over the years, several architectures have been described as MISD even though the architectures do not completely follow the description in Flynn’s Taxonomy. One commonly cited example is the Pipeline architecture where the data is passed from processing element to processing element and each processing element executes a different instruction. It is often argued that this is not a true MISD architecture as the data changes as it moves through the pipeline. Similarly, other architectures or applications exist which may be called MISD by some while others disagree. The following section looks at some of those implementations.<br />
<br />
==Implementations of MISD architecture==<br />
<br />
Some major implementations of MISD are in the fields of systolic arrays, pattern matching, flight control systems and fault tolerance systems. These MISD implementations are discussed below.<br />
<br />
===Systolic Array===<br />
<br />
A systolic array is an arrangement of processors in an array where data flows synchronously across the array between neighbors, usually with different data flowing in different directions. Each Processor at each step takes in data from one or more neighbors, processes it and, in the next step, outputs results in the opposite direction. <br />
The systolic array paradigm, data-stream-driven by data counters, is the counterpart of the [http://en.wikipedia.org/wiki/Von_Neumann_model von Neumann paradigm], instruction-stream-driven by a program counter. Because a systolic array usually sends and receives multiple data streams, and multiple data counters are needed to generate these data streams, it supports data parallelism.<ref>http://en.wikipedia.org/wiki/Systolic_array</ref><br />
<br />
====Type of Systolic Arrays<ref>Johnson, K.T.; Hurson, A.R.; Shirazi, B.; , "General-purpose systolic arrays," Computer , vol.26, no.11, pp.20-31, Nov. 1993 doi: 10.1109/2.241423</ref>====<br />
<br />
=====Special-purpose systolic array=====<br />
<br />
An array of hardwired systolic processing elements tailored for a specific application. Typically, many tens or hundreds of cells fit on a single chip. One of the major applications of special-purpose systolic array is in matrix operations. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Systolic_1.png Figure 3] illustrates the algorithm for the sum of a scalar product, computed in a single systolic element. Here, a’s and b’s are synchronously shifted through the processing element to be available for next element. These data synchronously exits the processing element unmodified for the next element. The sum of the products is then shifted out of the accumulator.<br />
[[Image:systolic_1.png|thumb|centre|250px|Figure 3: The algorithm for the sum of a scalar product, computed in systolic element [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
=====General-purpose systolic array=====<br />
<br />
It is an array of systolic processing elements, which gets adapted to a variety of applications via programming or reconfiguration. Array topologies can be either programmable or reconfigurable. Likewise, array cells are either programmable or reconfigurable. This is referred to as Systolic topologies.<br />
<br />
======<font size="2">Programmable Systolic Array</font>======<br />
<br />
It is an array of programmable systolic elements that operates either in SIMD or MIMD fashion. Either the arrays interconnect or each processing unit is programmable and a program controls dataflow through the elements. Programmable systolic arrays are programmable either at a high level or a low level. At either level, programmable arrays can be categorized as either SIMD or MIMD machines.<br />
<br />
<br />
<br />
For the latter approach, the workstation downloads a program to each MIMD (Figure 4) systolic cell. Each cell may be loaded with a different program, or all the cells in the array may be loaded with the same program. Each cell's architecture is somewhat similar to the conventional von Neumann architecture: It contains a control unit, an ALU, and local memory. MIMD systolic cells have more local memory than their SIMD counterparts to support the von Neumann-style organization.<br />
[[Image:systolic_2.png|thumb|centre|250px|Figure 4: The systolic product of two 3x3 matrices [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
<br />
This architecture is defined as Multiple Instruction Multiple Data (MIMD) architecture in [http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf 5]. The architecture has multiple instruction streams for the PEs and a single data stream passing through all the PEs. Thus, it can also be defined as Multiple Instruction Single Data (MISD) architecture. The architecture of Systolic array configuration are controversial as explained in the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Architecture_of_systolic_arrays_as_against_MISD_architecture section 4.1.2.]<br />
<br />
[[Image:systolic_4.png|thumb|centre|250px|Figure 5: General organization of MIMD programmable linear systolic arrays [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
<br />
======<font size="2">Reconfigurable Systolic Array</font>======<br />
[[Image:reconfig.jpg|thumb|right|250px|Figure 6: Block Diagram of the RSA Architecture [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
It is an array of systolic elements that can be programmed at the lowest level. Recent gate density advances in FPGA technology have produced a low-level, reconfigurable systolic array architecture that bridges the gap between special-purpose arrays and the more versatile, programmable general-purpose arrays. The FPGA architecture is unusual because a single hardware platform can be logically reconfigured as an exact duplicate of a special-purpose systolic array. <br />
<br />
The RSA circuit design is based on systolic array architecture consisting of PEs interconnected via SWs as depicted in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Reconfig.jpg Figure 6]. The homogeneous characteristic of the Reconfigurable Systolic Array (RSA) architecture, where each reconfigurable processing element (PE) cell is connected to its nearest neighbors via configurable switch (SW) elements, enables array expansion for parallel processing and facilitates time sharing computation of high-throughput data by individual PEs. Both the PEs and SWs can be reconfigured dynamically with the former as an arithmetic processor and the latter as a flexible router linking the neighboring PE cells. The RSA shifts reconfiguration and input signals into the PEs and SWs on separate data bus which enables the circuit to continue its operation while the reconfiguration is in process.<br />
<br />
====Architecture of systolic arrays as against MISD architecture====<br />
[[Image:comp.png|thumb|right|250px|Figure 7.Comparison between Architecture of systolic arrays and MISD]]<br />
<br />
As from the above mentioned configurations of the Systolic Arrays, it is seen that generally the configurations have multiple processing elements executing different instructions from dedicated instruction streams for each processing element. There is a single data stream that connects the adjacent PEs. Thus, systolic array can be defined as an MISD architecture.<br />
<br />
Many authors say that as the data read as input by one processing element is processed data output of the adjacent PE. The data stream cannot be considered as single because all the data paths do not carry the same data to all the PEs. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Comp.png Figure 7] shows the difference between the Data Stream for Systolic Arrays and the MISD architecture. Thus the systolic array should be considered as “Multiple Data” architecture and not Single Data architecture.<br />
<br />
===Fault Tolerant Systems<ref>http://en.wikipedia.org/wiki/Fault-tolerant_computer_system#Types_of_fault_tolerance</ref>===<br />
<br />
The fault tolerant systems are designed to handle the possible failures in software, hardware or interfaces. The hardware faults include hard disk failures, input or output device failures, etc. and the software and interface faults include driver failures; operator errors, installing unexpected software etc. The hardware faults can be detected and identified by implementing redundant hardware and multiple backups. The software faults can be tolerable by removing the program errors by executing the software redundantly or by implementing small programs that take over the tasks that crash or generate errors.<br />
<br />
[[Image:fault.png|thumb|centre|250px|Figure 8 MISD as fault tolerant architecture]]<br />
<br />
In computer systems, the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Single_Instruction.2C_Multiple_Data_streams_.28SIMD.29 SIMD], [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD] and [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instruction.2C_Multiple_Data_streams_.28MIMD.29 MIMD] architectures facilitate the implementation of the fault tolerance systems by multiple instruction streams or multiple data streams or both. Fault tolerance on computations can be implemented by multiple processors (likely with different architectures) executing the algorithms on the same set of data. The output of each processor is compared with that of the others and M out of N majority voting method is used to determine the faulty processor. Thus MISD architecture is utilized to get the fault tolerance on critical computations.<br />
<br />
There are various examples of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture] being used as fault tolerant architecture. The major examples being flight control systems, nuclear power plants, satellite systems, super collider experiment systems, etc. Here, the flight control system is explained as an example of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture].<br />
<br />
====The Flight Control System – MISD Example for fault tolerance====<br />
<br />
A [http://en.wikipedia.org/wiki/Fly-by-wire Fly-By-Wire] system is used to replace the manual flight control by an electronic control interface. The movements of the flight control in the cockpit are converted to electronic signals and are transmitted to the actuators by wires. The control computers use the feedback from the sensors to compute and control the movement of the actuators to provide the expected response. These computers also perform the task to stabilize the aircraft and perform other tasks without the knowledge of the pilot. Flight control systems must meet extremely high levels of accuracy and functional integrity.<br />
<br />
There are redundant flight control computers present in the flight control system. If one of the flight-control computers crashes, gets damaged or is affected by electromagnetic pulses, the other computer can overrule the faulty one and hence the flight of the aircraft is unharmed.The number of redundant flight control computers is generally more than two, so that any computer whose results disagree with the others is ruled out to be faulty and is either ignored or rebooted.<br />
<br />
=====Multiple Processors Implementation in Boeing 777<ref>Yeh, Y.C.;, "Triple-triple redundant 777 primary flight computer," Aerospace Applications Conference, 1996. Proceedings., 1996 IEEE , vol.1, no., pp.293-307 vol.1, 3-10 Feb 1996<br />
doi: 10.1109/AERO.1996.495891</ref>=====<br />
<br />
In modern computers, the redundant flight control computations are carried out by multiprocessor systems. The triple redundant 777 primary flight computer, has the architecture as shown in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Fig13.png Figure 9].<br />
[[Image:fig13.png|thumb|centre|250px|Figure 9: Architecture of triple redundant 777 primary flight computer [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-7 6]]]<br />
<br />
The system has three primary flight control computers, each of them having three lanes with different processors. The flight control program is compiled for each of the processors which get the input data from the same data bus but drive the output on their individual control bus. Thus each processor executes different instructions but they process the same data. Thus, it is the best suited example of Multiple Instruction Single Data (MISD) architecture.<br />
<br />
The three processors selected for the flight control system of [http://en.wikipedia.org/wiki/Boeing_777 Boeing 777] were [http://en.wikipedia.org/wiki/Intel_80486 Intel 80486], [http://en.wikipedia.org/wiki/Motorola_68040 Motorola 68040] and [http://en.wikipedia.org/wiki/AMD_Am29000 AMD 29050]. The dissimilar processors lead to dissimilar interface hardware circuits and compilers. Each lane of the flight control computer is data synchronized with the other lanes so that all of the lanes read the same frame of data from the flight sensors. As the outputs of each lane can be different, the median value of the outputs is used to select the output of the lane to be considered. The lane which has the median value select hardware selected is said to be in “command mode” whereas the other lanes are said to be in “monitoring mode”. It receives the data from the other Primary Flight Computer (PFC) lanes and performs a median select of the outputs. This provides a fault blocking mechanism before the fault detection and identification by the cross-lane monitoring system. Thus, the MISD based multi computer architecture is capable of detecting generic errors in compilers or in complex hardware devices providing assurance beyond reasonable doubt of the dependability of the Fly-By-Wire system.<br />
<br />
The above mentioned system clearly has individual Instruction Streams as the architecture of each processor is different, thus different instruction sets and different instruction streams. These processors have frame synchronized input data which means they have same set of data to work upon which is fed from a single data stream. Thus the flight control system can be classified under MISD architecture.<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System===<ref>M.Gajer;,“ Parallel Image Processing on The TMS320C80 Multiprocessor System”, http://focus.ti.com/pdfs/univ/11-CommunicationAudioSpeech.pdf</ref><br />
<br />
The image pre-processing system of this Multiprocessor, the pre-image processing time needed to be made shorter. To achieve the aim of reducing the pre-processing time, different types of image processing architectures for ‘C80 were presented. For example MIMD architecture is used for image histogram equalization; SIMD architecture for median filtering; MISD architecture for directional edge detection.<br />
<br />
The MISD architecture can be explained as follows. Here, each processor performs different operations on the same data set. It is mentioned at many places that MISD architecture was defined only for the sake of completeness of classification due to difficulty in practice to find a computer based on MISD architecture<ref>TMS320C80 System Level Synopsis, Texas Instruments, 1995</ref>. But in case of Image Processing, this assumption need not be true as here making many different operations on same image is rare and hence MISD architecture is a best fit for such cases. As an example of implementation of image processing operations for the MISD architecture the edge detection operation will be considered. The edges should be detected in four different directions (north, south, west and east) by the medium of four different filter masks. Each of the parallel processors of the TMS320C80 calculates one of the directional edges. Finally one obtains four images with detected edges, each in different direction.<br />
<br />
The image processing in computer is very often a multistage process composed of many stages. For example, an algorithm detecting contours of the objects in an image. The algorithm for achieving the before mentioned is composed of four basic image processing operation. The image acquired from the camera in the first step undergoes low-pass filtering, during which it is smoothed and noise is also eliminated. In the next step image is thresholding is performed and each pixel gets black or white depending on that if its grey level is below or above threshold value. The resulting image is a binary image from which edges are detected by the usage of the Laplace filter. In the last step the image with detected contours undergoes logic filtering in the purpose of elimination of single black pixels placed on the white background. For implementing the above processes parallel, a five-stage pipeline was used. The first stage of this pipeline constitutes the master processor which accounts for control of the image acquisition process. Further pipe line stages were implemented for 4 parallel processors as mentioned in table 1.<br />
<br />
<table border=1 align="center"><br />
<tr><td>Stage</td><td>Processor</td><td>Operation</td></tr><br />
<tr><td>1</td><td>MP</td><td>Image acquisition</td></tr><br />
<tr><td>2</td><td>PP0/td><td>Low-pass Filtering</td></tr><br />
<tr><td>3</td><td>PP1</td><td>Thresholding</td></tr><br />
<tr><td>4</td><td>PP2</td><td>Laplace Filtering</td></tr><br />
<tr><td>5</td><td>PP3</td><td>Logic Filtering</td></tr> <br />
</table><br />
<p align="center">Table 1: Pipeline structure for image edge detection[8]</p><br />
<br />
===Recursive Pattern Matching using MISD Architecture <ref>Arne Halaas, Børge Svingen, Magnar Nedland, Pål Sætrom, Ola Snøve, Jr., and Olaf René Birkeland. 2004. A recursive MISD architecture for pattern matching. IEEE Trans. Very Large Scale Integr. Syst. 12, 7 (July 2004), 727-734. DOI=10.1109/TVLSI.2004.830918 http://dx.doi.org/10.1109/TVLSI.2004.830918</ref>===<br />
<br />
The situations where several patterns are required to be matched concurrently in data and no persistent index can be created for look-up requires the use of Online multipattern approximate searching. This algorithm requires evaluation of the hit function H(Si,P) which is the function working on strings and pattern,and requires partitioning P into smaller components and combining the individual results. A recursive approach is required and there exists a need to evaluate the Si value in parallel. Thus it helps in using an architecture with two complete binary trees which take care of the data distribution and result processing. The architecture is implemented with as many PE's that are practically possible within the implementation constraints. The data from the stream is fed to the root node of the data distribution tree and simultaneously distributed to several collection of PE's belonging to separate queries thus implementing the MISD architecture with a single data stream and multiple instruction streams.<br />
<br />
==Conclusion==<br />
The MISD architecture is one of the four types of parallel architecture described in Flynn’s Taxonomy. By definition, the MISD architecture has multiple instruction streams and a single data stream. The definition also describes this architecture as requiring that all processing elements receive the same data, but most commonly cited implementations of this class of architecture do not adhere to this detail. The result is there is debate among some experts on whether or not certain implementations are truly MISD.<br />
<br />
MISD architectures are not commonly found in industry as they tend to lack the scalability and efficiency of resource use of the MIMD and SIMD class of architectures. However, MISD architectures are efficient for solving specific classes of problems often found in research. Some examples of its use in research include image processing and systolic arrays. While rare, some implementations do exist outside of academic settings. One such example is fault tolerant systems such as the flight control system on the Boeing 777.<br />
<br />
=='''References'''==<br />
<references/><br />
<br />
=='''Glossary'''==<br />
*'''ALU''': Arithmetic Logic Unit, the control unit in a computer<br />
<br />
*'''CMU''': Carnegie Mellon University<br />
<br />
*'''Fly-By-Wire''': System that replaces the conventional manual flight controls of an aircraft with an electronic interface<br />
<br />
*'''FPGA''': Field Programmable Gate Array<br />
<br />
*'''MIMD''': Multiple Instruction Multiple Data<br />
<br />
*'''MISD''': Multiple Instruction Single Data<br />
<br />
*'''PE''': Processing Element<br />
<br />
*'''PFC''': Primary Flight Computer<br />
<br />
*'''SIMD''': Single Instruction Multiple Data<br />
<br />
*'''SISD''': Single Instruction Single Data<br />
<br />
==Quiz==<br />
<br />
'''Q1: Using the classical definition of the MISD architecture from Flynn’s Taxonomy, what is the minimum number of input streams that must be present to be classified as MISD architecture?'''<br />
<br />
A. One information stream, the data stream, the number of instruction streams do not matter.<br />
<br />
B. Two information streams, one instruction stream, one data stream.<br />
<br />
C. Three information streams, two instruction streams, one data stream.<br />
<br />
D. The number of information streams is irrelevant as per Flynn’s Taxonomy.<br />
<br />
''A1: C, Three information streams, two instruction streams, one data stream.''<br />
<br />
<br />
'''Q2: Why is the Pipeline architecture often cited as an example of MISD?'''<br />
<br />
A. The Pipeline architecture is exactly like the definition of MISD architecture from Flynn’s Taxonomy.<br />
<br />
B. The data to be processed travels linearly across all processing elements with each processing element executing a different instruction and passing it on to the next processing element. This line of data is cited to be a single “stream.”<br />
<br />
C. The data stream sends the same data to all processing elements for every cycle.<br />
<br />
D. Flynn based his description of MISD architecture on the Pipeline architecture in his paper that introduced Flynn’s Taxonomy.<br />
<br />
''A2: B, The data to be processed travels linearly across all processing elements with each processing element executing a different instruction and passing it on to the next processing element. This line of data is cited to be a single “stream.”''<br />
<br />
<br />
'''Q3: Why is the Pipeline architecture NOT considered a true MISD architecture by many experts?'''<br />
<br />
A. This is incorrect; there is consensus that the Pipeline architecture is synonymous with the MISD architecture described by Flynn’s Taxonomy.<br />
<br />
B. There is only a single, global information/control stream in the Pipeline architecture.<br />
<br />
C. The data passed to each processing element is different as it reflects the result of the processing of all the previous processing elements. Flynn’s Taxonomy describes the same data going to every processing element.<br />
<br />
D. It has been proven that it is impossible to build a MISD architecture parallel computer as described by Flynn’s Taxonomy.<br />
<br />
''A3: C, The data passed to each processing element is different as it reflects the result of the processing of all the previous processing elements. Flynn’s Taxonomy describes the same data going to every processing element.''<br />
<br />
<br />
'''Q4: True or False? MISD architecture is the most common parallel computing architecture used for general computing tasks.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A4: B, False, it is often not the best suited architecture for general computing tasks and as such not common outside of academic research.''<br />
<br />
<br />
'''Q5: When classifying a particular parallel architecture, it was noted that there was only one physical connection to provide all information streams, is it possible for this architecture to be classified as MISD architecture? (Note that there are three choices, you need to select the correct reason for the Yes/No/Maybe answer.)'''<br />
<br />
A. Yes, Flynn’s taxonomy does not require the number of physical connections in the hardware correspond to the number of logical streams.<br />
<br />
B. Maybe, we do not have enough information about the architecture to rule out the possibility of this parallel computer having a MISD architecture. We will need to see how the connections are physically laid out on the chip.<br />
<br />
C. No, there must be multiple physical information streams.<br />
<br />
''A5. A, Yes, Flynn’s taxonomy does not require the number of physical connections in the hardware correspond to the number of logical streams.''<br />
<br />
<br />
'''Q6. True or False? To be a MISD architecture, it requires that every processing element is identical.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A6. B, False, it is not required by definition of Flynn’s Taxonomy that all processing elements are identical. See example in the wiki reading about the fault tolerant flight control system on the Boeing 777.''<br />
<br />
<br />
'''Q7: Which of these is an application of the MISD architecture?'''<br />
<br />
A. Keyword search<br />
<br />
B. Controlling an assembly line.<br />
<br />
C. Home computing<br />
<br />
D. Image edge detection.<br />
<br />
''A7: D, Image edge detection.''<br />
<br />
<br />
'''Q8. True or False? The systolic array is another name for the MISD architecture.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A8. B, False, systolic arrays may be implemented using a MISD architecture but they also may be other architectures such as SIMD and MIMD.''<br />
<br />
<br />
'''Q9. True or False? MISD is the considered the easiest parallel architecture to scale.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A9. False, MIMD and SIMD are often used because of advantages in scaling.''<br />
<br />
<br />
'''Q10. Flynn’s Taxonomy relies on the number of logical information streams to classify the types of parallel architectures. What is the difference between a logical information stream and a physical information stream?'''<br />
<br />
A. A logical information stream is a connection between the ALU and the location of the data that is responsible for passing information about boolean (logical) computations, where the physical information stream it the connection between computer and other parts of the network.<br />
<br />
B. Physical information streams are determined by how the information is physically transported around chip while logical information streams are one layer of abstraction above the physical information streams. Logical streams describe conceptually how the information on the stream can be treated.<br />
<br />
C. Logical and physical information streams transport different kinds of information but are no different in implementation.<br />
<br />
D. There is no difference; the two terms can be used interchangeably.<br />
<br />
''A10. B, Physical information streams are determined by how the information is physically transported around chip while logical information streams are one layer of abstraction above the physical information streams. Logical streams describe conceptually how the information on the stream can be treated.''</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/1a_sp&diff=72485CSC/ECE 506 Spring 2013/1a sp2013-02-11T04:43:44Z<p>Schakra8: /* Edge Detection of Images on TMS320C80 Multiprocessor SystemArne Halaas, Børge Svingen, Magnar Nedland, Pål Sætrom, Ola Snøve, Jr., and Olaf René Birkeland. 2004. A recursive MISD architecture for pattern matching. IEEE Trans. Very Large Scale</p>
<hr />
<div>==Multiple Instruction Single Data (MISD) Architecture==<br />
<br />
Multiple Instruction Single Data, or MISD, architecture is one of the four general categories of multi-processor or multi-core architectures described by Flynn's Taxonomy.<br />
<br />
===MISD Architectures within the Context of Flynn’s Taxonomy<ref>http://en.wikipedia.org/wiki/Flynn's_taxonomy</ref><ref>http://www.phy.ornl.gov/csep/ca/node11.html</ref>===<br />
<br />
Flynn's Taxonomy categorizes parallel computing architectures based on the number of concurrent information streams that flowed into the processor. According to Flynn's Taxonomy, there are two kinds of logical streams of information that flows into a parallel computer, the Instruction (or Control) Stream and the Data Stream. Note that for the purposes of classification using Flynn’s Taxonomy, it does not matter whether or not the streams are physically independent of each other on the hardware, only that the streams can be logically treated as independent. Based on how many of each type of stream flows into the parallel computer (single vs. multiple), Flynn's Taxonomy divides parallel architectures into four categories. They are Single Instruction, Single Data (SISD), Single Instruction, Multiple Data (SIMD), Multiple Instruction, Single Data (MISD), and Multiple Instruction, Multiple Data (MIMD). <br />
<br />
The following table (Figure 1) shows the categories of different parallel architectures according to Flynn:<ref>https://computing.llnl.gov/tutorials/parallel_comp/#Flynn</ref><br />
<br />
[[Image:Flynn's Taxonomy.PNG|thumb|center|400px|Figure 1. [http://en.wikipedia.org/wiki/Michael_J._Flynn Flynn]'s Taxonomy [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
===Overview of the MISD Architecture===<br />
<br />
In order to be categorized as MISD architecture, there must be at least two Instruction streams and there can only be one Data stream as shown on Figure 2.[[Image:MISD.PNG|thumb|right|100px|Figure 2. MISD Architecture [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]] As per the description according to Flynn’s Taxonomy, each processing elements should receive the same data and execute different instructions on the same data simultaneously. <br />
<br />
There are very few implementations of the classical MISD architecture as described by Flynn’s Taxonomy, none of which are commercial, general purpose parallel computers since the MIMD and SIMD are often better suited to common data parallel techniques. Under most circumstances MIMD and SIMD architectures tend to provide better scaling and more efficient use of computational resources. Instead of commercial use, MISD architectures are mainly used to create custom hardware in order to solve specific problems. The majority of the implementations have been created are in academia.<br />
<br />
Over the years, several architectures have been described as MISD even though the architectures do not completely follow the description in Flynn’s Taxonomy. One commonly cited example is the Pipeline architecture where the data is passed from processing element to processing element and each processing element executes a different instruction. It is often argued that this is not a true MISD architecture as the data changes as it moves through the pipeline. Similarly, other architectures or applications exist which may be called MISD by some while others disagree. The following section looks at some of those implementations.<br />
<br />
==Implementations of MISD architecture==<br />
<br />
Some major implementations of MISD are in the fields of systolic arrays, pattern matching, flight control systems and fault tolerance systems. These MISD implementations are discussed below.<br />
<br />
===Systolic Array===<br />
<br />
A systolic array is an arrangement of processors in an array where data flows synchronously across the array between neighbors, usually with different data flowing in different directions. Each Processor at each step takes in data from one or more neighbors, processes it and, in the next step, outputs results in the opposite direction. <br />
The systolic array paradigm, data-stream-driven by data counters, is the counterpart of the [http://en.wikipedia.org/wiki/Von_Neumann_model von Neumann paradigm], instruction-stream-driven by a program counter. Because a systolic array usually sends and receives multiple data streams, and multiple data counters are needed to generate these data streams, it supports data parallelism.<ref>http://en.wikipedia.org/wiki/Systolic_array</ref><br />
<br />
====Type of Systolic Arrays<ref>Johnson, K.T.; Hurson, A.R.; Shirazi, B.; , "General-purpose systolic arrays," Computer , vol.26, no.11, pp.20-31, Nov. 1993 doi: 10.1109/2.241423</ref>====<br />
<br />
=====Special-purpose systolic array=====<br />
<br />
An array of hardwired systolic processing elements tailored for a specific application. Typically, many tens or hundreds of cells fit on a single chip. One of the major applications of special-purpose systolic array is in matrix operations. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Systolic_1.png Figure 3] illustrates the algorithm for the sum of a scalar product, computed in a single systolic element. Here, a’s and b’s are synchronously shifted through the processing element to be available for next element. These data synchronously exits the processing element unmodified for the next element. The sum of the products is then shifted out of the accumulator.<br />
[[Image:systolic_1.png|thumb|centre|250px|Figure 3: The algorithm for the sum of a scalar product, computed in systolic element [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
=====General-purpose systolic array=====<br />
<br />
It is an array of systolic processing elements, which gets adapted to a variety of applications via programming or reconfiguration. Array topologies can be either programmable or reconfigurable. Likewise, array cells are either programmable or reconfigurable. This is referred to as Systolic topologies.<br />
<br />
======<font size="2">Programmable Systolic Array</font>======<br />
<br />
It is an array of programmable systolic elements that operates either in SIMD or MIMD fashion. Either the arrays interconnect or each processing unit is programmable and a program controls dataflow through the elements. Programmable systolic arrays are programmable either at a high level or a low level. At either level, programmable arrays can be categorized as either SIMD or MIMD machines.<br />
<br />
<br />
<br />
For the latter approach, the workstation downloads a program to each MIMD (Figure 4) systolic cell. Each cell may be loaded with a different program, or all the cells in the array may be loaded with the same program. Each cell's architecture is somewhat similar to the conventional von Neumann architecture: It contains a control unit, an ALU, and local memory. MIMD systolic cells have more local memory than their SIMD counterparts to support the von Neumann-style organization.<br />
[[Image:systolic_2.png|thumb|centre|250px|Figure 4: The systolic product of two 3x3 matrices [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
<br />
This architecture is defined as Multiple Instruction Multiple Data (MIMD) architecture in [http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf 5]. The architecture has multiple instruction streams for the PEs and a single data stream passing through all the PEs. Thus, it can also be defined as Multiple Instruction Single Data (MISD) architecture. The architecture of Systolic array configuration are controversial as explained in the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Architecture_of_systolic_arrays_as_against_MISD_architecture section 4.1.2.]<br />
<br />
[[Image:systolic_4.png|thumb|centre|250px|Figure 5: General organization of MIMD programmable linear systolic arrays [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
<br />
======<font size="2">Reconfigurable Systolic Array</font>======<br />
[[Image:reconfig.jpg|thumb|right|250px|Figure 6: Block Diagram of the RSA Architecture [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
It is an array of systolic elements that can be programmed at the lowest level. Recent gate density advances in FPGA technology have produced a low-level, reconfigurable systolic array architecture that bridges the gap between special-purpose arrays and the more versatile, programmable general-purpose arrays. The FPGA architecture is unusual because a single hardware platform can be logically reconfigured as an exact duplicate of a special-purpose systolic array. <br />
<br />
The RSA circuit design is based on systolic array architecture consisting of PEs interconnected via SWs as depicted in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Reconfig.jpg Figure 6]. The homogeneous characteristic of the Reconfigurable Systolic Array (RSA) architecture, where each reconfigurable processing element (PE) cell is connected to its nearest neighbors via configurable switch (SW) elements, enables array expansion for parallel processing and facilitates time sharing computation of high-throughput data by individual PEs. Both the PEs and SWs can be reconfigured dynamically with the former as an arithmetic processor and the latter as a flexible router linking the neighboring PE cells. The RSA shifts reconfiguration and input signals into the PEs and SWs on separate data bus which enables the circuit to continue its operation while the reconfiguration is in process.<br />
<br />
====Architecture of systolic arrays as against MISD architecture====<br />
[[Image:comp.png|thumb|right|250px|Figure 7.Comparison between Architecture of systolic arrays and MISD]]<br />
<br />
As from the above mentioned configurations of the Systolic Arrays, it is seen that generally the configurations have multiple processing elements executing different instructions from dedicated instruction streams for each processing element. There is a single data stream that connects the adjacent PEs. Thus, systolic array can be defined as an MISD architecture.<br />
<br />
Many authors say that as the data read as input by one processing element is processed data output of the adjacent PE. The data stream cannot be considered as single because all the data paths do not carry the same data to all the PEs. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Comp.png Figure 7] shows the difference between the Data Stream for Systolic Arrays and the MISD architecture. Thus the systolic array should be considered as “Multiple Data” architecture and not Single Data architecture.<br />
<br />
===Fault Tolerant Systems<ref>http://en.wikipedia.org/wiki/Fault-tolerant_computer_system#Types_of_fault_tolerance</ref>===<br />
<br />
The fault tolerant systems are designed to handle the possible failures in software, hardware or interfaces. The hardware faults include hard disk failures, input or output device failures, etc. and the software and interface faults include driver failures; operator errors, installing unexpected software etc. The hardware faults can be detected and identified by implementing redundant hardware and multiple backups. The software faults can be tolerable by removing the program errors by executing the software redundantly or by implementing small programs that take over the tasks that crash or generate errors.<br />
<br />
[[Image:fault.png|thumb|centre|250px|Figure 8 MISD as fault tolerant architecture]]<br />
<br />
In computer systems, the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Single_Instruction.2C_Multiple_Data_streams_.28SIMD.29 SIMD], [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD] and [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instruction.2C_Multiple_Data_streams_.28MIMD.29 MIMD] architectures facilitate the implementation of the fault tolerance systems by multiple instruction streams or multiple data streams or both. Fault tolerance on computations can be implemented by multiple processors (likely with different architectures) executing the algorithms on the same set of data. The output of each processor is compared with that of the others and M out of N majority voting method is used to determine the faulty processor. Thus MISD architecture is utilized to get the fault tolerance on critical computations.<br />
<br />
There are various examples of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture] being used as fault tolerant architecture. The major examples being flight control systems, nuclear power plants, satellite systems, super collider experiment systems, etc. Here, the flight control system is explained as an example of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture].<br />
<br />
====The Flight Control System – MISD Example for fault tolerance====<br />
<br />
A [http://en.wikipedia.org/wiki/Fly-by-wire Fly-By-Wire] system is used to replace the manual flight control by an electronic control interface. The movements of the flight control in the cockpit are converted to electronic signals and are transmitted to the actuators by wires. The control computers use the feedback from the sensors to compute and control the movement of the actuators to provide the expected response. These computers also perform the task to stabilize the aircraft and perform other tasks without the knowledge of the pilot. Flight control systems must meet extremely high levels of accuracy and functional integrity.<br />
<br />
There are redundant flight control computers present in the flight control system. If one of the flight-control computers crashes, gets damaged or is affected by electromagnetic pulses, the other computer can overrule the faulty one and hence the flight of the aircraft is unharmed.The number of redundant flight control computers is generally more than two, so that any computer whose results disagree with the others is ruled out to be faulty and is either ignored or rebooted.<br />
<br />
=====Multiple Processors Implementation in Boeing 777<ref>Yeh, Y.C.;, "Triple-triple redundant 777 primary flight computer," Aerospace Applications Conference, 1996. Proceedings., 1996 IEEE , vol.1, no., pp.293-307 vol.1, 3-10 Feb 1996<br />
doi: 10.1109/AERO.1996.495891</ref>=====<br />
<br />
In modern computers, the redundant flight control computations are carried out by multiprocessor systems. The triple redundant 777 primary flight computer, has the architecture as shown in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Fig13.png Figure 9].<br />
[[Image:fig13.png|thumb|centre|250px|Figure 9: Architecture of triple redundant 777 primary flight computer [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-7 6]]]<br />
<br />
The system has three primary flight control computers, each of them having three lanes with different processors. The flight control program is compiled for each of the processors which get the input data from the same data bus but drive the output on their individual control bus. Thus each processor executes different instructions but they process the same data. Thus, it is the best suited example of Multiple Instruction Single Data (MISD) architecture.<br />
<br />
The three processors selected for the flight control system of [http://en.wikipedia.org/wiki/Boeing_777 Boeing 777] were [http://en.wikipedia.org/wiki/Intel_80486 Intel 80486], [http://en.wikipedia.org/wiki/Motorola_68040 Motorola 68040] and [http://en.wikipedia.org/wiki/AMD_Am29000 AMD 29050]. The dissimilar processors lead to dissimilar interface hardware circuits and compilers. Each lane of the flight control computer is data synchronized with the other lanes so that all of the lanes read the same frame of data from the flight sensors. As the outputs of each lane can be different, the median value of the outputs is used to select the output of the lane to be considered. The lane which has the median value select hardware selected is said to be in “command mode” whereas the other lanes are said to be in “monitoring mode”. It receives the data from the other Primary Flight Computer (PFC) lanes and performs a median select of the outputs. This provides a fault blocking mechanism before the fault detection and identification by the cross-lane monitoring system. Thus, the MISD based multi computer architecture is capable of detecting generic errors in compilers or in complex hardware devices providing assurance beyond reasonable doubt of the dependability of the Fly-By-Wire system.<br />
<br />
The above mentioned system clearly has individual Instruction Streams as the architecture of each processor is different, thus different instruction sets and different instruction streams. These processors have frame synchronized input data which means they have same set of data to work upon which is fed from a single data stream. Thus the flight control system can be classified under MISD architecture.<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System<ref>M.Gajer;,“ Parallel Image Processing on The TMS320C80 Multiprocessor System”, http://focus.ti.com/pdfs/univ/11-CommunicationAudioSpeech.pdf</ref>===<br />
<br />
The image pre-processing system of this Multiprocessor, the pre-image processing time needed to be made shorter. To achieve the aim of reducing the pre-processing time, different types of image processing architectures for ‘C80 were presented. For example MIMD architecture is used for image histogram equalization; SIMD architecture for median filtering; MISD architecture for directional edge detection.<br />
<br />
The MISD architecture can be explained as follows. Here, each processor performs different operations on the same data set. It is mentioned at many places that MISD architecture was defined only for the sake of completeness of classification due to difficulty in practice to find a computer based on MISD architecture<ref>TMS320C80 System Level Synopsis, Texas Instruments, 1995</ref>. But in case of Image Processing, this assumption need not be true as here making many different operations on same image is rare and hence MISD architecture is a best fit for such cases. As an example of implementation of image processing operations for the MISD architecture the edge detection operation will be considered. The edges should be detected in four different directions (north, south, west and east) by the medium of four different filter masks. Each of the parallel processors of the TMS320C80 calculates one of the directional edges. Finally one obtains four images with detected edges, each in different direction.<br />
<br />
The image processing in computer is very often a multistage process composed of many stages. For example, an algorithm detecting contours of the objects in an image. The algorithm for achieving the before mentioned is composed of four basic image processing operation. The image acquired from the camera in the first step undergoes low-pass filtering, during which it is smoothed and noise is also eliminated. In the next step image is thresholding is performed and each pixel gets black or white depending on that if its grey level is below or above threshold value. The resulting image is a binary image from which edges are detected by the usage of the Laplace filter. In the last step the image with detected contours undergoes logic filtering in the purpose of elimination of single black pixels placed on the white background. For implementing the above processes parallel, a five-stage pipeline was used. The first stage of this pipeline constitutes the master processor which accounts for control of the image acquisition process. Further pipe line stages were implemented for 4 parallel processors as mentioned in table 1.<br />
<br />
<table border=1 align="center"><br />
<tr><td>Stage</td><td>Processor</td><td>Operation</td></tr><br />
<tr><td>1</td><td>MP</td><td>Image acquisition</td></tr><br />
<tr><td>2</td><td>PP0/td><td>Low-pass Filtering</td></tr><br />
<tr><td>3</td><td>PP1</td><td>Thresholding</td></tr><br />
<tr><td>4</td><td>PP2</td><td>Laplace Filtering</td></tr><br />
<tr><td>5</td><td>PP3</td><td>Logic Filtering</td></tr> <br />
</table><br />
<p align="center">Table 1: Pipeline structure for image edge detection[8]</p><br />
<br />
===Recursive Pattern Matching using MISD Architecture <ref>Arne Halaas, Børge Svingen, Magnar Nedland, Pål Sætrom, Ola Snøve, Jr., and Olaf René Birkeland. 2004. A recursive MISD architecture for pattern matching. IEEE Trans. Very Large Scale Integr. Syst. 12, 7 (July 2004), 727-734. DOI=10.1109/TVLSI.2004.830918 http://dx.doi.org/10.1109/TVLSI.2004.830918</ref>===<br />
<br />
The situations where several patterns are required to be matched concurrently in data and no persistent index can be created for look-up requires the use of Online multipattern approximate searching. This algorithm requires evaluation of the hit function H(Si,P) which is the function working on strings and pattern,and requires partitioning P into smaller components and combining the individual results. A recursive approach is required and there exists a need to evaluate the Si value in parallel. Thus it helps in using an architecture with two complete binary trees which take care of the data distribution and result processing. The architecture is implemented with as many PE's that are practically possible within the implementation constraints. The data from the stream is fed to the root node of the data distribution tree and simultaneously distributed to several collection of PE's belonging to separate queries thus implementing the MISD architecture with a single data stream and multiple instruction streams.<br />
<br />
==Conclusion==<br />
The MISD architecture is one of the four types of parallel architecture described in Flynn’s Taxonomy. By definition, the MISD architecture has multiple instruction streams and a single data stream. The definition also describes this architecture as requiring that all processing elements receive the same data, but most commonly cited implementations of this class of architecture do not adhere to this detail. The result is there is debate among some experts on whether or not certain implementations are truly MISD.<br />
<br />
MISD architectures are not commonly found in industry as they tend to lack the scalability and efficiency of resource use of the MIMD and SIMD class of architectures. However, MISD architectures are efficient for solving specific classes of problems often found in research. Some examples of its use in research include image processing and systolic arrays. While rare, some implementations do exist outside of academic settings. One such example is fault tolerant systems such as the flight control system on the Boeing 777.<br />
<br />
=='''References'''==<br />
<references/><br />
<br />
=='''Glossary'''==<br />
*'''ALU''': Arithmetic Logic Unit, the control unit in a computer<br />
<br />
*'''CMU''': Carnegie Mellon University<br />
<br />
*'''Fly-By-Wire''': System that replaces the conventional manual flight controls of an aircraft with an electronic interface<br />
<br />
*'''FPGA''': Field Programmable Gate Array<br />
<br />
*'''MIMD''': Multiple Instruction Multiple Data<br />
<br />
*'''MISD''': Multiple Instruction Single Data<br />
<br />
*'''PE''': Processing Element<br />
<br />
*'''PFC''': Primary Flight Computer<br />
<br />
*'''SIMD''': Single Instruction Multiple Data<br />
<br />
*'''SISD''': Single Instruction Single Data<br />
<br />
==Quiz==<br />
<br />
'''Q1: Using the classical definition of the MISD architecture from Flynn’s Taxonomy, what is the minimum number of input streams that must be present to be classified as MISD architecture?'''<br />
<br />
A. One information stream, the data stream, the number of instruction streams do not matter.<br />
<br />
B. Two information streams, one instruction stream, one data stream.<br />
<br />
C. Three information streams, two instruction streams, one data stream.<br />
<br />
D. The number of information streams is irrelevant as per Flynn’s Taxonomy.<br />
<br />
''A1: C, Three information streams, two instruction streams, one data stream.''<br />
<br />
<br />
'''Q2: Why is the Pipeline architecture often cited as an example of MISD?'''<br />
<br />
A. The Pipeline architecture is exactly like the definition of MISD architecture from Flynn’s Taxonomy.<br />
<br />
B. The data to be processed travels linearly across all processing elements with each processing element executing a different instruction and passing it on to the next processing element. This line of data is cited to be a single “stream.”<br />
<br />
C. The data stream sends the same data to all processing elements for every cycle.<br />
<br />
D. Flynn based his description of MISD architecture on the Pipeline architecture in his paper that introduced Flynn’s Taxonomy.<br />
<br />
''A2: B, The data to be processed travels linearly across all processing elements with each processing element executing a different instruction and passing it on to the next processing element. This line of data is cited to be a single “stream.”''<br />
<br />
<br />
'''Q3: Why is the Pipeline architecture NOT considered a true MISD architecture by many experts?'''<br />
<br />
A. This is incorrect; there is consensus that the Pipeline architecture is synonymous with the MISD architecture described by Flynn’s Taxonomy.<br />
<br />
B. There is only a single, global information/control stream in the Pipeline architecture.<br />
<br />
C. The data passed to each processing element is different as it reflects the result of the processing of all the previous processing elements. Flynn’s Taxonomy describes the same data going to every processing element.<br />
<br />
D. It has been proven that it is impossible to build a MISD architecture parallel computer as described by Flynn’s Taxonomy.<br />
<br />
''A3: C, The data passed to each processing element is different as it reflects the result of the processing of all the previous processing elements. Flynn’s Taxonomy describes the same data going to every processing element.''<br />
<br />
<br />
'''Q4: True or False? MISD architecture is the most common parallel computing architecture used for general computing tasks.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A4: B, False, it is often not the best suited architecture for general computing tasks and as such not common outside of academic research.''<br />
<br />
<br />
'''Q5: When classifying a particular parallel architecture, it was noted that there was only one physical connection to provide all information streams, is it possible for this architecture to be classified as MISD architecture? (Note that there are three choices, you need to select the correct reason for the Yes/No/Maybe answer.)'''<br />
<br />
A. Yes, Flynn’s taxonomy does not require the number of physical connections in the hardware correspond to the number of logical streams.<br />
<br />
B. Maybe, we do not have enough information about the architecture to rule out the possibility of this parallel computer having a MISD architecture. We will need to see how the connections are physically laid out on the chip.<br />
<br />
C. No, there must be multiple physical information streams.<br />
<br />
''A5. A, Yes, Flynn’s taxonomy does not require the number of physical connections in the hardware correspond to the number of logical streams.''<br />
<br />
<br />
'''Q6. True or False? To be a MISD architecture, it requires that every processing element is identical.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A6. B, False, it is not required by definition of Flynn’s Taxonomy that all processing elements are identical. See example in the wiki reading about the fault tolerant flight control system on the Boeing 777.''<br />
<br />
<br />
'''Q7: Which of these is an application of the MISD architecture?'''<br />
<br />
A. Keyword search<br />
<br />
B. Controlling an assembly line.<br />
<br />
C. Home computing<br />
<br />
D. Image edge detection.<br />
<br />
''A7: D, Image edge detection.''<br />
<br />
<br />
'''Q8. True or False? The systolic array is another name for the MISD architecture.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A8. B, False, systolic arrays may be implemented using a MISD architecture but they also may be other architectures such as SIMD and MIMD.''<br />
<br />
<br />
'''Q9. True or False? MISD is the considered the easiest parallel architecture to scale.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A9. False, MIMD and SIMD are often used because of advantages in scaling.''<br />
<br />
<br />
'''Q10. Flynn’s Taxonomy relies on the number of logical information streams to classify the types of parallel architectures. What is the difference between a logical information stream and a physical information stream?'''<br />
<br />
A. A logical information stream is a connection between the ALU and the location of the data that is responsible for passing information about boolean (logical) computations, where the physical information stream it the connection between computer and other parts of the network.<br />
<br />
B. Physical information streams are determined by how the information is physically transported around chip while logical information streams are one layer of abstraction above the physical information streams. Logical streams describe conceptually how the information on the stream can be treated.<br />
<br />
C. Logical and physical information streams transport different kinds of information but are no different in implementation.<br />
<br />
D. There is no difference; the two terms can be used interchangeably.<br />
<br />
''A10. B, Physical information streams are determined by how the information is physically transported around chip while logical information streams are one layer of abstraction above the physical information streams. Logical streams describe conceptually how the information on the stream can be treated.''</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/1a_sp&diff=72484CSC/ECE 506 Spring 2013/1a sp2013-02-11T04:37:38Z<p>Schakra8: /* Edge Detection of Images on TMS320C80 Multiprocessor SystemM.Gajer;,“ Parallel Image Processing on The TMS320C80 Multiprocessor System”, http://focus.ti.com/pdfs/univ/11-CommunicationAudioSpeech.pdf */</p>
<hr />
<div>==Multiple Instruction Single Data (MISD) Architecture==<br />
<br />
Multiple Instruction Single Data, or MISD, architecture is one of the four general categories of multi-processor or multi-core architectures described by Flynn's Taxonomy.<br />
<br />
===MISD Architectures within the Context of Flynn’s Taxonomy<ref>http://en.wikipedia.org/wiki/Flynn's_taxonomy</ref><ref>http://www.phy.ornl.gov/csep/ca/node11.html</ref>===<br />
<br />
Flynn's Taxonomy categorizes parallel computing architectures based on the number of concurrent information streams that flowed into the processor. According to Flynn's Taxonomy, there are two kinds of logical streams of information that flows into a parallel computer, the Instruction (or Control) Stream and the Data Stream. Note that for the purposes of classification using Flynn’s Taxonomy, it does not matter whether or not the streams are physically independent of each other on the hardware, only that the streams can be logically treated as independent. Based on how many of each type of stream flows into the parallel computer (single vs. multiple), Flynn's Taxonomy divides parallel architectures into four categories. They are Single Instruction, Single Data (SISD), Single Instruction, Multiple Data (SIMD), Multiple Instruction, Single Data (MISD), and Multiple Instruction, Multiple Data (MIMD). <br />
<br />
The following table (Figure 1) shows the categories of different parallel architectures according to Flynn:<ref>https://computing.llnl.gov/tutorials/parallel_comp/#Flynn</ref><br />
<br />
[[Image:Flynn's Taxonomy.PNG|thumb|center|400px|Figure 1. [http://en.wikipedia.org/wiki/Michael_J._Flynn Flynn]'s Taxonomy [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
===Overview of the MISD Architecture===<br />
<br />
In order to be categorized as MISD architecture, there must be at least two Instruction streams and there can only be one Data stream as shown on Figure 2.[[Image:MISD.PNG|thumb|right|100px|Figure 2. MISD Architecture [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]] As per the description according to Flynn’s Taxonomy, each processing elements should receive the same data and execute different instructions on the same data simultaneously. <br />
<br />
There are very few implementations of the classical MISD architecture as described by Flynn’s Taxonomy, none of which are commercial, general purpose parallel computers since the MIMD and SIMD are often better suited to common data parallel techniques. Under most circumstances MIMD and SIMD architectures tend to provide better scaling and more efficient use of computational resources. Instead of commercial use, MISD architectures are mainly used to create custom hardware in order to solve specific problems. The majority of the implementations have been created are in academia.<br />
<br />
Over the years, several architectures have been described as MISD even though the architectures do not completely follow the description in Flynn’s Taxonomy. One commonly cited example is the Pipeline architecture where the data is passed from processing element to processing element and each processing element executes a different instruction. It is often argued that this is not a true MISD architecture as the data changes as it moves through the pipeline. Similarly, other architectures or applications exist which may be called MISD by some while others disagree. The following section looks at some of those implementations.<br />
<br />
==Implementations of MISD architecture==<br />
<br />
Some major implementations of MISD are in the fields of systolic arrays, pattern matching, flight control systems and fault tolerance systems. These MISD implementations are discussed below.<br />
<br />
===Systolic Array===<br />
<br />
A systolic array is an arrangement of processors in an array where data flows synchronously across the array between neighbors, usually with different data flowing in different directions. Each Processor at each step takes in data from one or more neighbors, processes it and, in the next step, outputs results in the opposite direction. <br />
The systolic array paradigm, data-stream-driven by data counters, is the counterpart of the [http://en.wikipedia.org/wiki/Von_Neumann_model von Neumann paradigm], instruction-stream-driven by a program counter. Because a systolic array usually sends and receives multiple data streams, and multiple data counters are needed to generate these data streams, it supports data parallelism.<ref>http://en.wikipedia.org/wiki/Systolic_array</ref><br />
<br />
====Type of Systolic Arrays<ref>Johnson, K.T.; Hurson, A.R.; Shirazi, B.; , "General-purpose systolic arrays," Computer , vol.26, no.11, pp.20-31, Nov. 1993 doi: 10.1109/2.241423</ref>====<br />
<br />
=====Special-purpose systolic array=====<br />
<br />
An array of hardwired systolic processing elements tailored for a specific application. Typically, many tens or hundreds of cells fit on a single chip. One of the major applications of special-purpose systolic array is in matrix operations. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Systolic_1.png Figure 3] illustrates the algorithm for the sum of a scalar product, computed in a single systolic element. Here, a’s and b’s are synchronously shifted through the processing element to be available for next element. These data synchronously exits the processing element unmodified for the next element. The sum of the products is then shifted out of the accumulator.<br />
[[Image:systolic_1.png|thumb|centre|250px|Figure 3: The algorithm for the sum of a scalar product, computed in systolic element [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
=====General-purpose systolic array=====<br />
<br />
It is an array of systolic processing elements, which gets adapted to a variety of applications via programming or reconfiguration. Array topologies can be either programmable or reconfigurable. Likewise, array cells are either programmable or reconfigurable. This is referred to as Systolic topologies.<br />
<br />
======<font size="2">Programmable Systolic Array</font>======<br />
<br />
It is an array of programmable systolic elements that operates either in SIMD or MIMD fashion. Either the arrays interconnect or each processing unit is programmable and a program controls dataflow through the elements. Programmable systolic arrays are programmable either at a high level or a low level. At either level, programmable arrays can be categorized as either SIMD or MIMD machines.<br />
<br />
<br />
<br />
For the latter approach, the workstation downloads a program to each MIMD (Figure 4) systolic cell. Each cell may be loaded with a different program, or all the cells in the array may be loaded with the same program. Each cell's architecture is somewhat similar to the conventional von Neumann architecture: It contains a control unit, an ALU, and local memory. MIMD systolic cells have more local memory than their SIMD counterparts to support the von Neumann-style organization.<br />
[[Image:systolic_2.png|thumb|centre|250px|Figure 4: The systolic product of two 3x3 matrices [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
<br />
This architecture is defined as Multiple Instruction Multiple Data (MIMD) architecture in [http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf 5]. The architecture has multiple instruction streams for the PEs and a single data stream passing through all the PEs. Thus, it can also be defined as Multiple Instruction Single Data (MISD) architecture. The architecture of Systolic array configuration are controversial as explained in the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Architecture_of_systolic_arrays_as_against_MISD_architecture section 4.1.2.]<br />
<br />
[[Image:systolic_4.png|thumb|centre|250px|Figure 5: General organization of MIMD programmable linear systolic arrays [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
<br />
======<font size="2">Reconfigurable Systolic Array</font>======<br />
[[Image:reconfig.jpg|thumb|right|250px|Figure 6: Block Diagram of the RSA Architecture [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
It is an array of systolic elements that can be programmed at the lowest level. Recent gate density advances in FPGA technology have produced a low-level, reconfigurable systolic array architecture that bridges the gap between special-purpose arrays and the more versatile, programmable general-purpose arrays. The FPGA architecture is unusual because a single hardware platform can be logically reconfigured as an exact duplicate of a special-purpose systolic array. <br />
<br />
The RSA circuit design is based on systolic array architecture consisting of PEs interconnected via SWs as depicted in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Reconfig.jpg Figure 6]. The homogeneous characteristic of the Reconfigurable Systolic Array (RSA) architecture, where each reconfigurable processing element (PE) cell is connected to its nearest neighbors via configurable switch (SW) elements, enables array expansion for parallel processing and facilitates time sharing computation of high-throughput data by individual PEs. Both the PEs and SWs can be reconfigured dynamically with the former as an arithmetic processor and the latter as a flexible router linking the neighboring PE cells. The RSA shifts reconfiguration and input signals into the PEs and SWs on separate data bus which enables the circuit to continue its operation while the reconfiguration is in process.<br />
<br />
====Architecture of systolic arrays as against MISD architecture====<br />
[[Image:comp.png|thumb|right|250px|Figure 7.Comparison between Architecture of systolic arrays and MISD]]<br />
<br />
As from the above mentioned configurations of the Systolic Arrays, it is seen that generally the configurations have multiple processing elements executing different instructions from dedicated instruction streams for each processing element. There is a single data stream that connects the adjacent PEs. Thus, systolic array can be defined as an MISD architecture.<br />
<br />
Many authors say that as the data read as input by one processing element is processed data output of the adjacent PE. The data stream cannot be considered as single because all the data paths do not carry the same data to all the PEs. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Comp.png Figure 7] shows the difference between the Data Stream for Systolic Arrays and the MISD architecture. Thus the systolic array should be considered as “Multiple Data” architecture and not Single Data architecture.<br />
<br />
===Fault Tolerant Systems<ref>http://en.wikipedia.org/wiki/Fault-tolerant_computer_system#Types_of_fault_tolerance</ref>===<br />
<br />
The fault tolerant systems are designed to handle the possible failures in software, hardware or interfaces. The hardware faults include hard disk failures, input or output device failures, etc. and the software and interface faults include driver failures; operator errors, installing unexpected software etc. The hardware faults can be detected and identified by implementing redundant hardware and multiple backups. The software faults can be tolerable by removing the program errors by executing the software redundantly or by implementing small programs that take over the tasks that crash or generate errors.<br />
<br />
[[Image:fault.png|thumb|centre|250px|Figure 8 MISD as fault tolerant architecture]]<br />
<br />
In computer systems, the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Single_Instruction.2C_Multiple_Data_streams_.28SIMD.29 SIMD], [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD] and [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instruction.2C_Multiple_Data_streams_.28MIMD.29 MIMD] architectures facilitate the implementation of the fault tolerance systems by multiple instruction streams or multiple data streams or both. Fault tolerance on computations can be implemented by multiple processors (likely with different architectures) executing the algorithms on the same set of data. The output of each processor is compared with that of the others and M out of N majority voting method is used to determine the faulty processor. Thus MISD architecture is utilized to get the fault tolerance on critical computations.<br />
<br />
There are various examples of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture] being used as fault tolerant architecture. The major examples being flight control systems, nuclear power plants, satellite systems, super collider experiment systems, etc. Here, the flight control system is explained as an example of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture].<br />
<br />
====The Flight Control System – MISD Example for fault tolerance====<br />
<br />
A [http://en.wikipedia.org/wiki/Fly-by-wire Fly-By-Wire] system is used to replace the manual flight control by an electronic control interface. The movements of the flight control in the cockpit are converted to electronic signals and are transmitted to the actuators by wires. The control computers use the feedback from the sensors to compute and control the movement of the actuators to provide the expected response. These computers also perform the task to stabilize the aircraft and perform other tasks without the knowledge of the pilot. Flight control systems must meet extremely high levels of accuracy and functional integrity.<br />
<br />
There are redundant flight control computers present in the flight control system. If one of the flight-control computers crashes, gets damaged or is affected by electromagnetic pulses, the other computer can overrule the faulty one and hence the flight of the aircraft is unharmed.The number of redundant flight control computers is generally more than two, so that any computer whose results disagree with the others is ruled out to be faulty and is either ignored or rebooted.<br />
<br />
=====Multiple Processors Implementation in Boeing 777<ref>Yeh, Y.C.;, "Triple-triple redundant 777 primary flight computer," Aerospace Applications Conference, 1996. Proceedings., 1996 IEEE , vol.1, no., pp.293-307 vol.1, 3-10 Feb 1996<br />
doi: 10.1109/AERO.1996.495891</ref>=====<br />
<br />
In modern computers, the redundant flight control computations are carried out by multiprocessor systems. The triple redundant 777 primary flight computer, has the architecture as shown in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Fig13.png Figure 9].<br />
[[Image:fig13.png|thumb|centre|250px|Figure 9: Architecture of triple redundant 777 primary flight computer [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-7 6]]]<br />
<br />
The system has three primary flight control computers, each of them having three lanes with different processors. The flight control program is compiled for each of the processors which get the input data from the same data bus but drive the output on their individual control bus. Thus each processor executes different instructions but they process the same data. Thus, it is the best suited example of Multiple Instruction Single Data (MISD) architecture.<br />
<br />
The three processors selected for the flight control system of [http://en.wikipedia.org/wiki/Boeing_777 Boeing 777] were [http://en.wikipedia.org/wiki/Intel_80486 Intel 80486], [http://en.wikipedia.org/wiki/Motorola_68040 Motorola 68040] and [http://en.wikipedia.org/wiki/AMD_Am29000 AMD 29050]. The dissimilar processors lead to dissimilar interface hardware circuits and compilers. Each lane of the flight control computer is data synchronized with the other lanes so that all of the lanes read the same frame of data from the flight sensors. As the outputs of each lane can be different, the median value of the outputs is used to select the output of the lane to be considered. The lane which has the median value select hardware selected is said to be in “command mode” whereas the other lanes are said to be in “monitoring mode”. It receives the data from the other Primary Flight Computer (PFC) lanes and performs a median select of the outputs. This provides a fault blocking mechanism before the fault detection and identification by the cross-lane monitoring system. Thus, the MISD based multi computer architecture is capable of detecting generic errors in compilers or in complex hardware devices providing assurance beyond reasonable doubt of the dependability of the Fly-By-Wire system.<br />
<br />
The above mentioned system clearly has individual Instruction Streams as the architecture of each processor is different, thus different instruction sets and different instruction streams. These processors have frame synchronized input data which means they have same set of data to work upon which is fed from a single data stream. Thus the flight control system can be classified under MISD architecture.<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System<ref>M.Gajer;,“ Parallel Image Processing on The TMS320C80 Multiprocessor System”, http://focus.ti.com/pdfs/univ/11-CommunicationAudioSpeech.pdf</ref>===<br />
<br />
The image pre-processing system of this Multiprocessor, the pre-image processing time needed to be made shorter. To achieve the aim of reducing the pre-processing time, different types of image processing architectures for ‘C80 were presented. For example MIMD architecture is used for image histogram equalization; SIMD architecture for median filtering; MISD architecture for directional edge detection.<br />
<br />
The MISD architecture can be explained as follows. Here, each processor performs different operations on the same data set. It is mentioned at many places that MISD architecture was defined only for the sake of completeness of classification due to difficulty in practice to find a computer based on MISD architecture<ref>TMS320C80 System Level Synopsis, Texas Instruments, 1995</ref>. But in case of Image Processing, this assumption need not be true as here making many different operations on same image is rare and hence MISD architecture is a best fit for such cases. As an example of implementation of image processing operations for the MISD architecture the edge detection operation will be considered. The edges should be detected in four different directions (north, south, west and east) by the medium of four different filter masks. Each of the parallel processors of the TMS320C80 calculates one of the directional edges. Finally one obtains four images with detected edges, each in different direction.<br />
<br />
The image processing in computer is very often a multistage process composed of many stages. For example, an algorithm detecting contours of the objects in an image. The algorithm for achieving the before mentioned is composed of four basic image processing operation. The image acquired from the camera in the first step undergoes low-pass filtering, during which it is smoothed and noise is also eliminated. In the next step image is thresholding is performed and each pixel gets black or white depending on that if its grey level is below or above threshold value. The resulting image is a binary image from which edges are detected by the usage of the Laplace filter. In the last step the image with detected contours undergoes logic filtering in the purpose of elimination of single black pixels placed on the white background. For implementing the above processes parallel, a five-stage pipeline was used. The first stage of this pipeline constitutes the master processor which accounts for control of the image acquisition process. Further pipe line stages were implemented for 4 parallel processors as mentioned in table 1.<br />
<br />
<table border=1 align="center"><br />
<tr><td>Stage</td><td>Processor</td><td>Operation</td></tr><br />
<tr><td>1</td><td>MP</td><td>Image acquisition</td></tr><br />
<tr><td>2</td><td>PP0/td><td>Low-pass Filtering</td></tr><br />
<tr><td>3</td><td>PP1</td><td>Thresholding</td></tr><br />
<tr><td>4</td><td>PP2</td><td>Laplace Filtering</td></tr><br />
<tr><td>5</td><td>PP3</td><td>Logic Filtering</td></tr> <br />
</table><br />
<p align="center">Table 1: Pipeline structure for image edge detection[8]</p><br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System<ref>Arne Halaas, Børge Svingen, Magnar Nedland, Pål Sætrom, Ola Snøve, Jr., and Olaf René Birkeland. 2004. A recursive MISD architecture for pattern matching. IEEE Trans. Very Large Scale Integr. Syst. 12, 7 (July 2004), 727-734. DOI=10.1109/TVLSI.2004.830918 http://dx.doi.org/10.1109/TVLSI.2004.830918</ref>===<br />
<br />
The situations where several patterns are required to be matched concurrently in data and no persistent index can be created for look-up requires the use of Online multipattern approximate searching. This algorithm requires evaluation of the hit function H(Si,P) which is the function working on strings and pattern,and requires partitioning P into smaller components and combining the individual results. A recursive approach is required and there exists a need to evaluate the Si value in parallel. Thus it helps in using an architecture with two complete binary trees which take care of the data distribution and result processing. The architecture is implemented with as many PE's that are practically possible within the implementation constraints. The data from the stream is fed to the root node of the data distribution tree and simultaneously distributed to several collection of PE's belonging to separate queries thus implementing the MISD architecture with a single data stream and multiple instruction streams.<br />
<br />
==Conclusion==<br />
The MISD architecture is one of the four types of parallel architecture described in Flynn’s Taxonomy. By definition, the MISD architecture has multiple instruction streams and a single data stream. The definition also describes this architecture as requiring that all processing elements receive the same data, but most commonly cited implementations of this class of architecture do not adhere to this detail. The result is there is debate among some experts on whether or not certain implementations are truly MISD.<br />
<br />
MISD architectures are not commonly found in industry as they tend to lack the scalability and efficiency of resource use of the MIMD and SIMD class of architectures. However, MISD architectures are efficient for solving specific classes of problems often found in research. Some examples of its use in research include image processing and systolic arrays. While rare, some implementations do exist outside of academic settings. One such example is fault tolerant systems such as the flight control system on the Boeing 777.<br />
<br />
=='''References'''==<br />
<references/><br />
<br />
=='''Glossary'''==<br />
*'''ALU''': Arithmetic Logic Unit, the control unit in a computer<br />
<br />
*'''CMU''': Carnegie Mellon University<br />
<br />
*'''Fly-By-Wire''': System that replaces the conventional manual flight controls of an aircraft with an electronic interface<br />
<br />
*'''FPGA''': Field Programmable Gate Array<br />
<br />
*'''MIMD''': Multiple Instruction Multiple Data<br />
<br />
*'''MISD''': Multiple Instruction Single Data<br />
<br />
*'''PE''': Processing Element<br />
<br />
*'''PFC''': Primary Flight Computer<br />
<br />
*'''SIMD''': Single Instruction Multiple Data<br />
<br />
*'''SISD''': Single Instruction Single Data<br />
<br />
==Quiz==<br />
<br />
'''Q1: Using the classical definition of the MISD architecture from Flynn’s Taxonomy, what is the minimum number of input streams that must be present to be classified as MISD architecture?'''<br />
<br />
A. One information stream, the data stream, the number of instruction streams do not matter.<br />
<br />
B. Two information streams, one instruction stream, one data stream.<br />
<br />
C. Three information streams, two instruction streams, one data stream.<br />
<br />
D. The number of information streams is irrelevant as per Flynn’s Taxonomy.<br />
<br />
''A1: C, Three information streams, two instruction streams, one data stream.''<br />
<br />
<br />
'''Q2: Why is the Pipeline architecture often cited as an example of MISD?'''<br />
<br />
A. The Pipeline architecture is exactly like the definition of MISD architecture from Flynn’s Taxonomy.<br />
<br />
B. The data to be processed travels linearly across all processing elements with each processing element executing a different instruction and passing it on to the next processing element. This line of data is cited to be a single “stream.”<br />
<br />
C. The data stream sends the same data to all processing elements for every cycle.<br />
<br />
D. Flynn based his description of MISD architecture on the Pipeline architecture in his paper that introduced Flynn’s Taxonomy.<br />
<br />
''A2: B, The data to be processed travels linearly across all processing elements with each processing element executing a different instruction and passing it on to the next processing element. This line of data is cited to be a single “stream.”''<br />
<br />
<br />
'''Q3: Why is the Pipeline architecture NOT considered a true MISD architecture by many experts?'''<br />
<br />
A. This is incorrect; there is consensus that the Pipeline architecture is synonymous with the MISD architecture described by Flynn’s Taxonomy.<br />
<br />
B. There is only a single, global information/control stream in the Pipeline architecture.<br />
<br />
C. The data passed to each processing element is different as it reflects the result of the processing of all the previous processing elements. Flynn’s Taxonomy describes the same data going to every processing element.<br />
<br />
D. It has been proven that it is impossible to build a MISD architecture parallel computer as described by Flynn’s Taxonomy.<br />
<br />
''A3: C, The data passed to each processing element is different as it reflects the result of the processing of all the previous processing elements. Flynn’s Taxonomy describes the same data going to every processing element.''<br />
<br />
<br />
'''Q4: True or False? MISD architecture is the most common parallel computing architecture used for general computing tasks.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A4: B, False, it is often not the best suited architecture for general computing tasks and as such not common outside of academic research.''<br />
<br />
<br />
'''Q5: When classifying a particular parallel architecture, it was noted that there was only one physical connection to provide all information streams, is it possible for this architecture to be classified as MISD architecture? (Note that there are three choices, you need to select the correct reason for the Yes/No/Maybe answer.)'''<br />
<br />
A. Yes, Flynn’s taxonomy does not require the number of physical connections in the hardware correspond to the number of logical streams.<br />
<br />
B. Maybe, we do not have enough information about the architecture to rule out the possibility of this parallel computer having a MISD architecture. We will need to see how the connections are physically laid out on the chip.<br />
<br />
C. No, there must be multiple physical information streams.<br />
<br />
''A5. A, Yes, Flynn’s taxonomy does not require the number of physical connections in the hardware correspond to the number of logical streams.''<br />
<br />
<br />
'''Q6. True or False? To be a MISD architecture, it requires that every processing element is identical.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A6. B, False, it is not required by definition of Flynn’s Taxonomy that all processing elements are identical. See example in the wiki reading about the fault tolerant flight control system on the Boeing 777.''<br />
<br />
<br />
'''Q7: Which of these is an application of the MISD architecture?'''<br />
<br />
A. Keyword search<br />
<br />
B. Controlling an assembly line.<br />
<br />
C. Home computing<br />
<br />
D. Image edge detection.<br />
<br />
''A7: D, Image edge detection.''<br />
<br />
<br />
'''Q8. True or False? The systolic array is another name for the MISD architecture.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A8. B, False, systolic arrays may be implemented using a MISD architecture but they also may be other architectures such as SIMD and MIMD.''<br />
<br />
<br />
'''Q9. True or False? MISD is the considered the easiest parallel architecture to scale.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A9. False, MIMD and SIMD are often used because of advantages in scaling.''<br />
<br />
<br />
'''Q10. Flynn’s Taxonomy relies on the number of logical information streams to classify the types of parallel architectures. What is the difference between a logical information stream and a physical information stream?'''<br />
<br />
A. A logical information stream is a connection between the ALU and the location of the data that is responsible for passing information about boolean (logical) computations, where the physical information stream it the connection between computer and other parts of the network.<br />
<br />
B. Physical information streams are determined by how the information is physically transported around chip while logical information streams are one layer of abstraction above the physical information streams. Logical streams describe conceptually how the information on the stream can be treated.<br />
<br />
C. Logical and physical information streams transport different kinds of information but are no different in implementation.<br />
<br />
D. There is no difference; the two terms can be used interchangeably.<br />
<br />
''A10. B, Physical information streams are determined by how the information is physically transported around chip while logical information streams are one layer of abstraction above the physical information streams. Logical streams describe conceptually how the information on the stream can be treated.''</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/1a_sp&diff=72483CSC/ECE 506 Spring 2013/1a sp2013-02-11T04:36:19Z<p>Schakra8: /* Fault Tolerant Systemshttp://en.wikipedia.org/wiki/Fault-tolerant_computer_system#Types_of_fault_tolerance */</p>
<hr />
<div>==Multiple Instruction Single Data (MISD) Architecture==<br />
<br />
Multiple Instruction Single Data, or MISD, architecture is one of the four general categories of multi-processor or multi-core architectures described by Flynn's Taxonomy.<br />
<br />
===MISD Architectures within the Context of Flynn’s Taxonomy<ref>http://en.wikipedia.org/wiki/Flynn's_taxonomy</ref><ref>http://www.phy.ornl.gov/csep/ca/node11.html</ref>===<br />
<br />
Flynn's Taxonomy categorizes parallel computing architectures based on the number of concurrent information streams that flowed into the processor. According to Flynn's Taxonomy, there are two kinds of logical streams of information that flows into a parallel computer, the Instruction (or Control) Stream and the Data Stream. Note that for the purposes of classification using Flynn’s Taxonomy, it does not matter whether or not the streams are physically independent of each other on the hardware, only that the streams can be logically treated as independent. Based on how many of each type of stream flows into the parallel computer (single vs. multiple), Flynn's Taxonomy divides parallel architectures into four categories. They are Single Instruction, Single Data (SISD), Single Instruction, Multiple Data (SIMD), Multiple Instruction, Single Data (MISD), and Multiple Instruction, Multiple Data (MIMD). <br />
<br />
The following table (Figure 1) shows the categories of different parallel architectures according to Flynn:<ref>https://computing.llnl.gov/tutorials/parallel_comp/#Flynn</ref><br />
<br />
[[Image:Flynn's Taxonomy.PNG|thumb|center|400px|Figure 1. [http://en.wikipedia.org/wiki/Michael_J._Flynn Flynn]'s Taxonomy [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
===Overview of the MISD Architecture===<br />
<br />
In order to be categorized as MISD architecture, there must be at least two Instruction streams and there can only be one Data stream as shown on Figure 2.[[Image:MISD.PNG|thumb|right|100px|Figure 2. MISD Architecture [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]] As per the description according to Flynn’s Taxonomy, each processing elements should receive the same data and execute different instructions on the same data simultaneously. <br />
<br />
There are very few implementations of the classical MISD architecture as described by Flynn’s Taxonomy, none of which are commercial, general purpose parallel computers since the MIMD and SIMD are often better suited to common data parallel techniques. Under most circumstances MIMD and SIMD architectures tend to provide better scaling and more efficient use of computational resources. Instead of commercial use, MISD architectures are mainly used to create custom hardware in order to solve specific problems. The majority of the implementations have been created are in academia.<br />
<br />
Over the years, several architectures have been described as MISD even though the architectures do not completely follow the description in Flynn’s Taxonomy. One commonly cited example is the Pipeline architecture where the data is passed from processing element to processing element and each processing element executes a different instruction. It is often argued that this is not a true MISD architecture as the data changes as it moves through the pipeline. Similarly, other architectures or applications exist which may be called MISD by some while others disagree. The following section looks at some of those implementations.<br />
<br />
==Implementations of MISD architecture==<br />
<br />
Some major implementations of MISD are in the fields of systolic arrays, pattern matching, flight control systems and fault tolerance systems. These MISD implementations are discussed below.<br />
<br />
===Systolic Array===<br />
<br />
A systolic array is an arrangement of processors in an array where data flows synchronously across the array between neighbors, usually with different data flowing in different directions. Each Processor at each step takes in data from one or more neighbors, processes it and, in the next step, outputs results in the opposite direction. <br />
The systolic array paradigm, data-stream-driven by data counters, is the counterpart of the [http://en.wikipedia.org/wiki/Von_Neumann_model von Neumann paradigm], instruction-stream-driven by a program counter. Because a systolic array usually sends and receives multiple data streams, and multiple data counters are needed to generate these data streams, it supports data parallelism.<ref>http://en.wikipedia.org/wiki/Systolic_array</ref><br />
<br />
====Type of Systolic Arrays<ref>Johnson, K.T.; Hurson, A.R.; Shirazi, B.; , "General-purpose systolic arrays," Computer , vol.26, no.11, pp.20-31, Nov. 1993 doi: 10.1109/2.241423</ref>====<br />
<br />
=====Special-purpose systolic array=====<br />
<br />
An array of hardwired systolic processing elements tailored for a specific application. Typically, many tens or hundreds of cells fit on a single chip. One of the major applications of special-purpose systolic array is in matrix operations. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Systolic_1.png Figure 3] illustrates the algorithm for the sum of a scalar product, computed in a single systolic element. Here, a’s and b’s are synchronously shifted through the processing element to be available for next element. These data synchronously exits the processing element unmodified for the next element. The sum of the products is then shifted out of the accumulator.<br />
[[Image:systolic_1.png|thumb|centre|250px|Figure 3: The algorithm for the sum of a scalar product, computed in systolic element [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
=====General-purpose systolic array=====<br />
<br />
It is an array of systolic processing elements, which gets adapted to a variety of applications via programming or reconfiguration. Array topologies can be either programmable or reconfigurable. Likewise, array cells are either programmable or reconfigurable. This is referred to as Systolic topologies.<br />
<br />
======<font size="2">Programmable Systolic Array</font>======<br />
<br />
It is an array of programmable systolic elements that operates either in SIMD or MIMD fashion. Either the arrays interconnect or each processing unit is programmable and a program controls dataflow through the elements. Programmable systolic arrays are programmable either at a high level or a low level. At either level, programmable arrays can be categorized as either SIMD or MIMD machines.<br />
<br />
<br />
<br />
For the latter approach, the workstation downloads a program to each MIMD (Figure 4) systolic cell. Each cell may be loaded with a different program, or all the cells in the array may be loaded with the same program. Each cell's architecture is somewhat similar to the conventional von Neumann architecture: It contains a control unit, an ALU, and local memory. MIMD systolic cells have more local memory than their SIMD counterparts to support the von Neumann-style organization.<br />
[[Image:systolic_2.png|thumb|centre|250px|Figure 4: The systolic product of two 3x3 matrices [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
<br />
This architecture is defined as Multiple Instruction Multiple Data (MIMD) architecture in [http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf 5]. The architecture has multiple instruction streams for the PEs and a single data stream passing through all the PEs. Thus, it can also be defined as Multiple Instruction Single Data (MISD) architecture. The architecture of Systolic array configuration are controversial as explained in the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Architecture_of_systolic_arrays_as_against_MISD_architecture section 4.1.2.]<br />
<br />
[[Image:systolic_4.png|thumb|centre|250px|Figure 5: General organization of MIMD programmable linear systolic arrays [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
<br />
======<font size="2">Reconfigurable Systolic Array</font>======<br />
[[Image:reconfig.jpg|thumb|right|250px|Figure 6: Block Diagram of the RSA Architecture [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
It is an array of systolic elements that can be programmed at the lowest level. Recent gate density advances in FPGA technology have produced a low-level, reconfigurable systolic array architecture that bridges the gap between special-purpose arrays and the more versatile, programmable general-purpose arrays. The FPGA architecture is unusual because a single hardware platform can be logically reconfigured as an exact duplicate of a special-purpose systolic array. <br />
<br />
The RSA circuit design is based on systolic array architecture consisting of PEs interconnected via SWs as depicted in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Reconfig.jpg Figure 6]. The homogeneous characteristic of the Reconfigurable Systolic Array (RSA) architecture, where each reconfigurable processing element (PE) cell is connected to its nearest neighbors via configurable switch (SW) elements, enables array expansion for parallel processing and facilitates time sharing computation of high-throughput data by individual PEs. Both the PEs and SWs can be reconfigured dynamically with the former as an arithmetic processor and the latter as a flexible router linking the neighboring PE cells. The RSA shifts reconfiguration and input signals into the PEs and SWs on separate data bus which enables the circuit to continue its operation while the reconfiguration is in process.<br />
<br />
====Architecture of systolic arrays as against MISD architecture====<br />
[[Image:comp.png|thumb|right|250px|Figure 7.Comparison between Architecture of systolic arrays and MISD]]<br />
<br />
As from the above mentioned configurations of the Systolic Arrays, it is seen that generally the configurations have multiple processing elements executing different instructions from dedicated instruction streams for each processing element. There is a single data stream that connects the adjacent PEs. Thus, systolic array can be defined as an MISD architecture.<br />
<br />
Many authors say that as the data read as input by one processing element is processed data output of the adjacent PE. The data stream cannot be considered as single because all the data paths do not carry the same data to all the PEs. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Comp.png Figure 7] shows the difference between the Data Stream for Systolic Arrays and the MISD architecture. Thus the systolic array should be considered as “Multiple Data” architecture and not Single Data architecture.<br />
<br />
===Fault Tolerant Systems<ref>http://en.wikipedia.org/wiki/Fault-tolerant_computer_system#Types_of_fault_tolerance</ref>===<br />
<br />
The fault tolerant systems are designed to handle the possible failures in software, hardware or interfaces. The hardware faults include hard disk failures, input or output device failures, etc. and the software and interface faults include driver failures; operator errors, installing unexpected software etc. The hardware faults can be detected and identified by implementing redundant hardware and multiple backups. The software faults can be tolerable by removing the program errors by executing the software redundantly or by implementing small programs that take over the tasks that crash or generate errors.<br />
<br />
[[Image:fault.png|thumb|centre|250px|Figure 8 MISD as fault tolerant architecture]]<br />
<br />
In computer systems, the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Single_Instruction.2C_Multiple_Data_streams_.28SIMD.29 SIMD], [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD] and [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instruction.2C_Multiple_Data_streams_.28MIMD.29 MIMD] architectures facilitate the implementation of the fault tolerance systems by multiple instruction streams or multiple data streams or both. Fault tolerance on computations can be implemented by multiple processors (likely with different architectures) executing the algorithms on the same set of data. The output of each processor is compared with that of the others and M out of N majority voting method is used to determine the faulty processor. Thus MISD architecture is utilized to get the fault tolerance on critical computations.<br />
<br />
There are various examples of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture] being used as fault tolerant architecture. The major examples being flight control systems, nuclear power plants, satellite systems, super collider experiment systems, etc. Here, the flight control system is explained as an example of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture].<br />
<br />
====The Flight Control System – MISD Example for fault tolerance====<br />
<br />
A [http://en.wikipedia.org/wiki/Fly-by-wire Fly-By-Wire] system is used to replace the manual flight control by an electronic control interface. The movements of the flight control in the cockpit are converted to electronic signals and are transmitted to the actuators by wires. The control computers use the feedback from the sensors to compute and control the movement of the actuators to provide the expected response. These computers also perform the task to stabilize the aircraft and perform other tasks without the knowledge of the pilot. Flight control systems must meet extremely high levels of accuracy and functional integrity.<br />
<br />
There are redundant flight control computers present in the flight control system. If one of the flight-control computers crashes, gets damaged or is affected by electromagnetic pulses, the other computer can overrule the faulty one and hence the flight of the aircraft is unharmed.The number of redundant flight control computers is generally more than two, so that any computer whose results disagree with the others is ruled out to be faulty and is either ignored or rebooted.<br />
<br />
=====Multiple Processors Implementation in Boeing 777<ref>Yeh, Y.C.;, "Triple-triple redundant 777 primary flight computer," Aerospace Applications Conference, 1996. Proceedings., 1996 IEEE , vol.1, no., pp.293-307 vol.1, 3-10 Feb 1996<br />
doi: 10.1109/AERO.1996.495891</ref>=====<br />
<br />
In modern computers, the redundant flight control computations are carried out by multiprocessor systems. The triple redundant 777 primary flight computer, has the architecture as shown in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Fig13.png Figure 9].<br />
[[Image:fig13.png|thumb|centre|250px|Figure 9: Architecture of triple redundant 777 primary flight computer [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-7 6]]]<br />
<br />
The system has three primary flight control computers, each of them having three lanes with different processors. The flight control program is compiled for each of the processors which get the input data from the same data bus but drive the output on their individual control bus. Thus each processor executes different instructions but they process the same data. Thus, it is the best suited example of Multiple Instruction Single Data (MISD) architecture.<br />
<br />
The three processors selected for the flight control system of [http://en.wikipedia.org/wiki/Boeing_777 Boeing 777] were [http://en.wikipedia.org/wiki/Intel_80486 Intel 80486], [http://en.wikipedia.org/wiki/Motorola_68040 Motorola 68040] and [http://en.wikipedia.org/wiki/AMD_Am29000 AMD 29050]. The dissimilar processors lead to dissimilar interface hardware circuits and compilers. Each lane of the flight control computer is data synchronized with the other lanes so that all of the lanes read the same frame of data from the flight sensors. As the outputs of each lane can be different, the median value of the outputs is used to select the output of the lane to be considered. The lane which has the median value select hardware selected is said to be in “command mode” whereas the other lanes are said to be in “monitoring mode”. It receives the data from the other Primary Flight Computer (PFC) lanes and performs a median select of the outputs. This provides a fault blocking mechanism before the fault detection and identification by the cross-lane monitoring system. Thus, the MISD based multi computer architecture is capable of detecting generic errors in compilers or in complex hardware devices providing assurance beyond reasonable doubt of the dependability of the Fly-By-Wire system.<br />
<br />
The above mentioned system clearly has individual Instruction Streams as the architecture of each processor is different, thus different instruction sets and different instruction streams. These processors have frame synchronized input data which means they have same set of data to work upon which is fed from a single data stream. Thus the flight control system can be classified under MISD architecture.<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System<ref>M.Gajer;,“ Parallel Image Processing on The TMS320C80 Multiprocessor System”, http://focus.ti.com/pdfs/univ/11-CommunicationAudioSpeech.pdf</ref>===<br />
<br />
The image pre-processing system of this Multiprocessor, the pre-image processing time needed to be made shorter. To achieve the aim of reducing the pre-processing time, different types of image processing architectures for ‘C80 were presented. For example MIMD architecture is used for image histogram equalization; SIMD architecture for median filtering; MISD architecture for directional edge detection.<br />
<br />
The MISD architecture can be explained as follows. Here, each processor performs different operations on the same data set. It is mentioned at many places that MISD architecture was defined only for the sake of completeness of classification due to difficulty in practice to find a computer based on MISD architecture<ref>TMS320C80 System Level Synopsis, Texas Instruments, 1995</ref>. But in case of Image Processing, this assumption need not be true as here making many different operations on same image is rare and hence MISD architecture is a best fit for such cases. As an example of implementation of image processing operations for the MISD architecture the edge detection operation will be considered. The edges should be detected in four different directions (north, south, west and east) by the medium of four different filter masks. Each of the parallel processors of the TMS320C80 calculates one of the directional edges. Finally one obtains four images with detected edges, each in different direction.<br />
<br />
The image processing in computer is very often a multistage process composed of many stages. For example, an algorithm detecting contours of the objects in an image. The algorithm for achieving the before mentioned is composed of four basic image processing operation. The image acquired from the camera in the first step undergoes low-pass filtering, during which it is smoothed and noise is also eliminated. In the next step image is thresholding is performed and each pixel gets black or white depending on that if its grey level is below or above threshold value. The resulting image is a binary image from which edges are detected by the usage of the Laplace filter. In the last step the image with detected contours undergoes logic filtering in the purpose of elimination of single black pixels placed on the white background. For implementing the above processes parallel, a five-stage pipeline was used. The first stage of this pipeline constitutes the master processor which accounts for control of the image acquisition process. Further pipe line stages were implemented for 4 parallel processors as mentioned in table 1.<br />
<table border=1><br />
<tr><td>Stage</td><td>Processor</td><td>Operation</td></tr><br />
<tr><td>1</td><td>MP</td><td>Image acquisition</td></tr><br />
<tr><td>2</td><td>PP0/td><td>Low-pass Filtering</td></tr><br />
<tr><td>3</td><td>PP1</td><td>Thresholding</td></tr><br />
<tr><td>4</td><td>PP2</td><td>Laplace Filtering</td></tr><br />
<tr><td>5</td><td>PP3</td><td>Logic Filtering</td></tr> <br />
</table><br />
Table 1: Pipeline structure for image edge detection[8]<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System<ref>Arne Halaas, Børge Svingen, Magnar Nedland, Pål Sætrom, Ola Snøve, Jr., and Olaf René Birkeland. 2004. A recursive MISD architecture for pattern matching. IEEE Trans. Very Large Scale Integr. Syst. 12, 7 (July 2004), 727-734. DOI=10.1109/TVLSI.2004.830918 http://dx.doi.org/10.1109/TVLSI.2004.830918</ref>===<br />
<br />
The situations where several patterns are required to be matched concurrently in data and no persistent index can be created for look-up requires the use of Online multipattern approximate searching. This algorithm requires evaluation of the hit function H(Si,P) which is the function working on strings and pattern,and requires partitioning P into smaller components and combining the individual results. A recursive approach is required and there exists a need to evaluate the Si value in parallel. Thus it helps in using an architecture with two complete binary trees which take care of the data distribution and result processing. The architecture is implemented with as many PE's that are practically possible within the implementation constraints. The data from the stream is fed to the root node of the data distribution tree and simultaneously distributed to several collection of PE's belonging to separate queries thus implementing the MISD architecture with a single data stream and multiple instruction streams.<br />
<br />
==Conclusion==<br />
The MISD architecture is one of the four types of parallel architecture described in Flynn’s Taxonomy. By definition, the MISD architecture has multiple instruction streams and a single data stream. The definition also describes this architecture as requiring that all processing elements receive the same data, but most commonly cited implementations of this class of architecture do not adhere to this detail. The result is there is debate among some experts on whether or not certain implementations are truly MISD.<br />
<br />
MISD architectures are not commonly found in industry as they tend to lack the scalability and efficiency of resource use of the MIMD and SIMD class of architectures. However, MISD architectures are efficient for solving specific classes of problems often found in research. Some examples of its use in research include image processing and systolic arrays. While rare, some implementations do exist outside of academic settings. One such example is fault tolerant systems such as the flight control system on the Boeing 777.<br />
<br />
=='''References'''==<br />
<references/><br />
<br />
=='''Glossary'''==<br />
*'''ALU''': Arithmetic Logic Unit, the control unit in a computer<br />
<br />
*'''CMU''': Carnegie Mellon University<br />
<br />
*'''Fly-By-Wire''': System that replaces the conventional manual flight controls of an aircraft with an electronic interface<br />
<br />
*'''FPGA''': Field Programmable Gate Array<br />
<br />
*'''MIMD''': Multiple Instruction Multiple Data<br />
<br />
*'''MISD''': Multiple Instruction Single Data<br />
<br />
*'''PE''': Processing Element<br />
<br />
*'''PFC''': Primary Flight Computer<br />
<br />
*'''SIMD''': Single Instruction Multiple Data<br />
<br />
*'''SISD''': Single Instruction Single Data<br />
<br />
==Quiz==<br />
<br />
'''Q1: Using the classical definition of the MISD architecture from Flynn’s Taxonomy, what is the minimum number of input streams that must be present to be classified as MISD architecture?'''<br />
<br />
A. One information stream, the data stream, the number of instruction streams do not matter.<br />
<br />
B. Two information streams, one instruction stream, one data stream.<br />
<br />
C. Three information streams, two instruction streams, one data stream.<br />
<br />
D. The number of information streams is irrelevant as per Flynn’s Taxonomy.<br />
<br />
''A1: C, Three information streams, two instruction streams, one data stream.''<br />
<br />
<br />
'''Q2: Why is the Pipeline architecture often cited as an example of MISD?'''<br />
<br />
A. The Pipeline architecture is exactly like the definition of MISD architecture from Flynn’s Taxonomy.<br />
<br />
B. The data to be processed travels linearly across all processing elements with each processing element executing a different instruction and passing it on to the next processing element. This line of data is cited to be a single “stream.”<br />
<br />
C. The data stream sends the same data to all processing elements for every cycle.<br />
<br />
D. Flynn based his description of MISD architecture on the Pipeline architecture in his paper that introduced Flynn’s Taxonomy.<br />
<br />
''A2: B, The data to be processed travels linearly across all processing elements with each processing element executing a different instruction and passing it on to the next processing element. This line of data is cited to be a single “stream.”''<br />
<br />
<br />
'''Q3: Why is the Pipeline architecture NOT considered a true MISD architecture by many experts?'''<br />
<br />
A. This is incorrect; there is consensus that the Pipeline architecture is synonymous with the MISD architecture described by Flynn’s Taxonomy.<br />
<br />
B. There is only a single, global information/control stream in the Pipeline architecture.<br />
<br />
C. The data passed to each processing element is different as it reflects the result of the processing of all the previous processing elements. Flynn’s Taxonomy describes the same data going to every processing element.<br />
<br />
D. It has been proven that it is impossible to build a MISD architecture parallel computer as described by Flynn’s Taxonomy.<br />
<br />
''A3: C, The data passed to each processing element is different as it reflects the result of the processing of all the previous processing elements. Flynn’s Taxonomy describes the same data going to every processing element.''<br />
<br />
<br />
'''Q4: True or False? MISD architecture is the most common parallel computing architecture used for general computing tasks.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A4: B, False, it is often not the best suited architecture for general computing tasks and as such not common outside of academic research.''<br />
<br />
<br />
'''Q5: When classifying a particular parallel architecture, it was noted that there was only one physical connection to provide all information streams, is it possible for this architecture to be classified as MISD architecture? (Note that there are three choices, you need to select the correct reason for the Yes/No/Maybe answer.)'''<br />
<br />
A. Yes, Flynn’s taxonomy does not require the number of physical connections in the hardware correspond to the number of logical streams.<br />
<br />
B. Maybe, we do not have enough information about the architecture to rule out the possibility of this parallel computer having a MISD architecture. We will need to see how the connections are physically laid out on the chip.<br />
<br />
C. No, there must be multiple physical information streams.<br />
<br />
''A5. A, Yes, Flynn’s taxonomy does not require the number of physical connections in the hardware correspond to the number of logical streams.''<br />
<br />
<br />
'''Q6. True or False? To be a MISD architecture, it requires that every processing element is identical.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A6. B, False, it is not required by definition of Flynn’s Taxonomy that all processing elements are identical. See example in the wiki reading about the fault tolerant flight control system on the Boeing 777.''<br />
<br />
<br />
'''Q7: Which of these is an application of the MISD architecture?'''<br />
<br />
A. Keyword search<br />
<br />
B. Controlling an assembly line.<br />
<br />
C. Home computing<br />
<br />
D. Image edge detection.<br />
<br />
''A7: D, Image edge detection.''<br />
<br />
<br />
'''Q8. True or False? The systolic array is another name for the MISD architecture.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A8. B, False, systolic arrays may be implemented using a MISD architecture but they also may be other architectures such as SIMD and MIMD.''<br />
<br />
<br />
'''Q9. True or False? MISD is the considered the easiest parallel architecture to scale.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A9. False, MIMD and SIMD are often used because of advantages in scaling.''<br />
<br />
<br />
'''Q10. Flynn’s Taxonomy relies on the number of logical information streams to classify the types of parallel architectures. What is the difference between a logical information stream and a physical information stream?'''<br />
<br />
A. A logical information stream is a connection between the ALU and the location of the data that is responsible for passing information about boolean (logical) computations, where the physical information stream it the connection between computer and other parts of the network.<br />
<br />
B. Physical information streams are determined by how the information is physically transported around chip while logical information streams are one layer of abstraction above the physical information streams. Logical streams describe conceptually how the information on the stream can be treated.<br />
<br />
C. Logical and physical information streams transport different kinds of information but are no different in implementation.<br />
<br />
D. There is no difference; the two terms can be used interchangeably.<br />
<br />
''A10. B, Physical information streams are determined by how the information is physically transported around chip while logical information streams are one layer of abstraction above the physical information streams. Logical streams describe conceptually how the information on the stream can be treated.''</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/1a_sp&diff=72482CSC/ECE 506 Spring 2013/1a sp2013-02-11T04:33:43Z<p>Schakra8: /* Programmable Systolic Array */</p>
<hr />
<div>==Multiple Instruction Single Data (MISD) Architecture==<br />
<br />
Multiple Instruction Single Data, or MISD, architecture is one of the four general categories of multi-processor or multi-core architectures described by Flynn's Taxonomy.<br />
<br />
===MISD Architectures within the Context of Flynn’s Taxonomy<ref>http://en.wikipedia.org/wiki/Flynn's_taxonomy</ref><ref>http://www.phy.ornl.gov/csep/ca/node11.html</ref>===<br />
<br />
Flynn's Taxonomy categorizes parallel computing architectures based on the number of concurrent information streams that flowed into the processor. According to Flynn's Taxonomy, there are two kinds of logical streams of information that flows into a parallel computer, the Instruction (or Control) Stream and the Data Stream. Note that for the purposes of classification using Flynn’s Taxonomy, it does not matter whether or not the streams are physically independent of each other on the hardware, only that the streams can be logically treated as independent. Based on how many of each type of stream flows into the parallel computer (single vs. multiple), Flynn's Taxonomy divides parallel architectures into four categories. They are Single Instruction, Single Data (SISD), Single Instruction, Multiple Data (SIMD), Multiple Instruction, Single Data (MISD), and Multiple Instruction, Multiple Data (MIMD). <br />
<br />
The following table (Figure 1) shows the categories of different parallel architectures according to Flynn:<ref>https://computing.llnl.gov/tutorials/parallel_comp/#Flynn</ref><br />
<br />
[[Image:Flynn's Taxonomy.PNG|thumb|center|400px|Figure 1. [http://en.wikipedia.org/wiki/Michael_J._Flynn Flynn]'s Taxonomy [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
===Overview of the MISD Architecture===<br />
<br />
In order to be categorized as MISD architecture, there must be at least two Instruction streams and there can only be one Data stream as shown on Figure 2.[[Image:MISD.PNG|thumb|right|100px|Figure 2. MISD Architecture [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]] As per the description according to Flynn’s Taxonomy, each processing elements should receive the same data and execute different instructions on the same data simultaneously. <br />
<br />
There are very few implementations of the classical MISD architecture as described by Flynn’s Taxonomy, none of which are commercial, general purpose parallel computers since the MIMD and SIMD are often better suited to common data parallel techniques. Under most circumstances MIMD and SIMD architectures tend to provide better scaling and more efficient use of computational resources. Instead of commercial use, MISD architectures are mainly used to create custom hardware in order to solve specific problems. The majority of the implementations have been created are in academia.<br />
<br />
Over the years, several architectures have been described as MISD even though the architectures do not completely follow the description in Flynn’s Taxonomy. One commonly cited example is the Pipeline architecture where the data is passed from processing element to processing element and each processing element executes a different instruction. It is often argued that this is not a true MISD architecture as the data changes as it moves through the pipeline. Similarly, other architectures or applications exist which may be called MISD by some while others disagree. The following section looks at some of those implementations.<br />
<br />
==Implementations of MISD architecture==<br />
<br />
Some major implementations of MISD are in the fields of systolic arrays, pattern matching, flight control systems and fault tolerance systems. These MISD implementations are discussed below.<br />
<br />
===Systolic Array===<br />
<br />
A systolic array is an arrangement of processors in an array where data flows synchronously across the array between neighbors, usually with different data flowing in different directions. Each Processor at each step takes in data from one or more neighbors, processes it and, in the next step, outputs results in the opposite direction. <br />
The systolic array paradigm, data-stream-driven by data counters, is the counterpart of the [http://en.wikipedia.org/wiki/Von_Neumann_model von Neumann paradigm], instruction-stream-driven by a program counter. Because a systolic array usually sends and receives multiple data streams, and multiple data counters are needed to generate these data streams, it supports data parallelism.<ref>http://en.wikipedia.org/wiki/Systolic_array</ref><br />
<br />
====Type of Systolic Arrays<ref>Johnson, K.T.; Hurson, A.R.; Shirazi, B.; , "General-purpose systolic arrays," Computer , vol.26, no.11, pp.20-31, Nov. 1993 doi: 10.1109/2.241423</ref>====<br />
<br />
=====Special-purpose systolic array=====<br />
<br />
An array of hardwired systolic processing elements tailored for a specific application. Typically, many tens or hundreds of cells fit on a single chip. One of the major applications of special-purpose systolic array is in matrix operations. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Systolic_1.png Figure 3] illustrates the algorithm for the sum of a scalar product, computed in a single systolic element. Here, a’s and b’s are synchronously shifted through the processing element to be available for next element. These data synchronously exits the processing element unmodified for the next element. The sum of the products is then shifted out of the accumulator.<br />
[[Image:systolic_1.png|thumb|centre|250px|Figure 3: The algorithm for the sum of a scalar product, computed in systolic element [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
=====General-purpose systolic array=====<br />
<br />
It is an array of systolic processing elements, which gets adapted to a variety of applications via programming or reconfiguration. Array topologies can be either programmable or reconfigurable. Likewise, array cells are either programmable or reconfigurable. This is referred to as Systolic topologies.<br />
<br />
======<font size="2">Programmable Systolic Array</font>======<br />
<br />
It is an array of programmable systolic elements that operates either in SIMD or MIMD fashion. Either the arrays interconnect or each processing unit is programmable and a program controls dataflow through the elements. Programmable systolic arrays are programmable either at a high level or a low level. At either level, programmable arrays can be categorized as either SIMD or MIMD machines.<br />
<br />
<br />
<br />
For the latter approach, the workstation downloads a program to each MIMD (Figure 4) systolic cell. Each cell may be loaded with a different program, or all the cells in the array may be loaded with the same program. Each cell's architecture is somewhat similar to the conventional von Neumann architecture: It contains a control unit, an ALU, and local memory. MIMD systolic cells have more local memory than their SIMD counterparts to support the von Neumann-style organization.<br />
[[Image:systolic_2.png|thumb|centre|250px|Figure 4: The systolic product of two 3x3 matrices [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
<br />
This architecture is defined as Multiple Instruction Multiple Data (MIMD) architecture in [http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf 5]. The architecture has multiple instruction streams for the PEs and a single data stream passing through all the PEs. Thus, it can also be defined as Multiple Instruction Single Data (MISD) architecture. The architecture of Systolic array configuration are controversial as explained in the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Architecture_of_systolic_arrays_as_against_MISD_architecture section 4.1.2.]<br />
<br />
[[Image:systolic_4.png|thumb|centre|250px|Figure 5: General organization of MIMD programmable linear systolic arrays [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
<br />
======<font size="2">Reconfigurable Systolic Array</font>======<br />
[[Image:reconfig.jpg|thumb|right|250px|Figure 6: Block Diagram of the RSA Architecture [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
It is an array of systolic elements that can be programmed at the lowest level. Recent gate density advances in FPGA technology have produced a low-level, reconfigurable systolic array architecture that bridges the gap between special-purpose arrays and the more versatile, programmable general-purpose arrays. The FPGA architecture is unusual because a single hardware platform can be logically reconfigured as an exact duplicate of a special-purpose systolic array. <br />
<br />
The RSA circuit design is based on systolic array architecture consisting of PEs interconnected via SWs as depicted in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Reconfig.jpg Figure 6]. The homogeneous characteristic of the Reconfigurable Systolic Array (RSA) architecture, where each reconfigurable processing element (PE) cell is connected to its nearest neighbors via configurable switch (SW) elements, enables array expansion for parallel processing and facilitates time sharing computation of high-throughput data by individual PEs. Both the PEs and SWs can be reconfigured dynamically with the former as an arithmetic processor and the latter as a flexible router linking the neighboring PE cells. The RSA shifts reconfiguration and input signals into the PEs and SWs on separate data bus which enables the circuit to continue its operation while the reconfiguration is in process.<br />
<br />
====Architecture of systolic arrays as against MISD architecture====<br />
[[Image:comp.png|thumb|right|250px|Figure 7.Comparison between Architecture of systolic arrays and MISD]]<br />
<br />
As from the above mentioned configurations of the Systolic Arrays, it is seen that generally the configurations have multiple processing elements executing different instructions from dedicated instruction streams for each processing element. There is a single data stream that connects the adjacent PEs. Thus, systolic array can be defined as an MISD architecture.<br />
<br />
Many authors say that as the data read as input by one processing element is processed data output of the adjacent PE. The data stream cannot be considered as single because all the data paths do not carry the same data to all the PEs. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Comp.png Figure 7] shows the difference between the Data Stream for Systolic Arrays and the MISD architecture. Thus the systolic array should be considered as “Multiple Data” architecture and not Single Data architecture.<br />
<br />
===Fault Tolerant Systems<ref>http://en.wikipedia.org/wiki/Fault-tolerant_computer_system#Types_of_fault_tolerance</ref>===<br />
<br />
The fault tolerant systems are designed to handle the possible failures in software, hardware or interfaces. The hardware faults include hard disk failures, input or output device failures, etc. and the software and interface faults include driver failures; operator errors, installing unexpected software etc. The hardware faults can be detected and identified by implementing redundant hardware and multiple backups. The software faults can be tolerable by removing the program errors by executing the software redundantly or by implementing small programs that take over the tasks that crash or generate errors.<br />
<br />
[[Image:fault.png|thumb|right|250px|Figure 8 MISD as fault tolerant architecture]]<br />
<br />
In computer systems, the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Single_Instruction.2C_Multiple_Data_streams_.28SIMD.29 SIMD], [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD] and [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instruction.2C_Multiple_Data_streams_.28MIMD.29 MIMD] architectures facilitate the implementation of the fault tolerance systems by multiple instruction streams or multiple data streams or both. Fault tolerance on computations can be implemented by multiple processors (likely with different architectures) executing the algorithms on the same set of data. The output of each processor is compared with that of the others and M out of N majority voting method is used to determine the faulty processor. Thus MISD architecture is utilized to get the fault tolerance on critical computations.<br />
<br />
There are various examples of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture] being used as fault tolerant architecture. The major examples being flight control systems, nuclear power plants, satellite systems, super collider experiment systems, etc. Here, the flight control system is explained as an example of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture].<br />
<br />
====The Flight Control System – MISD Example for fault tolerance====<br />
<br />
A [http://en.wikipedia.org/wiki/Fly-by-wire Fly-By-Wire] system is used to replace the manual flight control by an electronic control interface. The movements of the flight control in the cockpit are converted to electronic signals and are transmitted to the actuators by wires. The control computers use the feedback from the sensors to compute and control the movement of the actuators to provide the expected response. These computers also perform the task to stabilize the aircraft and perform other tasks without the knowledge of the pilot. Flight control systems must meet extremely high levels of accuracy and functional integrity.<br />
<br />
There are redundant flight control computers present in the flight control system. If one of the flight-control computers crashes, gets damaged or is affected by electromagnetic pulses, the other computer can overrule the faulty one and hence the flight of the aircraft is unharmed. [[Image:fig13.png|thumb|right|250px|Figure 9: Architecture of triple redundant 777 primary flight computer [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-7 6]]]The number of redundant flight control computers is generally more than two, so that any computer whose results disagree with the others is ruled out to be faulty and is either ignored or rebooted.<br />
<br />
=====Multiple Processors Implementation in Boeing 777<ref>Yeh, Y.C.;, "Triple-triple redundant 777 primary flight computer," Aerospace Applications Conference, 1996. Proceedings., 1996 IEEE , vol.1, no., pp.293-307 vol.1, 3-10 Feb 1996<br />
doi: 10.1109/AERO.1996.495891</ref>=====<br />
<br />
In modern computers, the redundant flight control computations are carried out by multiprocessor systems. The triple redundant 777 primary flight computer, has the architecture as shown in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Fig13.png Figure 9].<br />
<br />
The system has three primary flight control computers, each of them having three lanes with different processors. The flight control program is compiled for each of the processors which get the input data from the same data bus but drive the output on their individual control bus. Thus each processor executes different instructions but they process the same data. Thus, it is the best suited example of Multiple Instruction Single Data (MISD) architecture.<br />
<br />
The three processors selected for the flight control system of [http://en.wikipedia.org/wiki/Boeing_777 Boeing 777] were [http://en.wikipedia.org/wiki/Intel_80486 Intel 80486], [http://en.wikipedia.org/wiki/Motorola_68040 Motorola 68040] and [http://en.wikipedia.org/wiki/AMD_Am29000 AMD 29050]. The dissimilar processors lead to dissimilar interface hardware circuits and compilers. Each lane of the flight control computer is data synchronized with the other lanes so that all of the lanes read the same frame of data from the flight sensors. As the outputs of each lane can be different, the median value of the outputs is used to select the output of the lane to be considered. The lane which has the median value select hardware selected is said to be in “command mode” whereas the other lanes are said to be in “monitoring mode”. It receives the data from the other Primary Flight Computer (PFC) lanes and performs a median select of the outputs. This provides a fault blocking mechanism before the fault detection and identification by the cross-lane monitoring system. Thus, the MISD based multi computer architecture is capable of detecting generic errors in compilers or in complex hardware devices providing assurance beyond reasonable doubt of the dependability of the Fly-By-Wire system.<br />
<br />
The above mentioned system clearly has individual Instruction Streams as the architecture of each processor is different, thus different instruction sets and different instruction streams. These processors have frame synchronized input data which means they have same set of data to work upon which is fed from a single data stream. Thus the flight control system can be classified under MISD architecture.<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System<ref>M.Gajer;,“ Parallel Image Processing on The TMS320C80 Multiprocessor System”, http://focus.ti.com/pdfs/univ/11-CommunicationAudioSpeech.pdf</ref>===<br />
<br />
The image pre-processing system of this Multiprocessor, the pre-image processing time needed to be made shorter. To achieve the aim of reducing the pre-processing time, different types of image processing architectures for ‘C80 were presented. For example MIMD architecture is used for image histogram equalization; SIMD architecture for median filtering; MISD architecture for directional edge detection.<br />
<br />
The MISD architecture can be explained as follows. Here, each processor performs different operations on the same data set. It is mentioned at many places that MISD architecture was defined only for the sake of completeness of classification due to difficulty in practice to find a computer based on MISD architecture<ref>TMS320C80 System Level Synopsis, Texas Instruments, 1995</ref>. But in case of Image Processing, this assumption need not be true as here making many different operations on same image is rare and hence MISD architecture is a best fit for such cases. As an example of implementation of image processing operations for the MISD architecture the edge detection operation will be considered. The edges should be detected in four different directions (north, south, west and east) by the medium of four different filter masks. Each of the parallel processors of the TMS320C80 calculates one of the directional edges. Finally one obtains four images with detected edges, each in different direction.<br />
<br />
The image processing in computer is very often a multistage process composed of many stages. For example, an algorithm detecting contours of the objects in an image. The algorithm for achieving the before mentioned is composed of four basic image processing operation. The image acquired from the camera in the first step undergoes low-pass filtering, during which it is smoothed and noise is also eliminated. In the next step image is thresholding is performed and each pixel gets black or white depending on that if its grey level is below or above threshold value. The resulting image is a binary image from which edges are detected by the usage of the Laplace filter. In the last step the image with detected contours undergoes logic filtering in the purpose of elimination of single black pixels placed on the white background. For implementing the above processes parallel, a five-stage pipeline was used. The first stage of this pipeline constitutes the master processor which accounts for control of the image acquisition process. Further pipe line stages were implemented for 4 parallel processors as mentioned in table 1.<br />
<table border=1><br />
<tr><td>Stage</td><td>Processor</td><td>Operation</td></tr><br />
<tr><td>1</td><td>MP</td><td>Image acquisition</td></tr><br />
<tr><td>2</td><td>PP0/td><td>Low-pass Filtering</td></tr><br />
<tr><td>3</td><td>PP1</td><td>Thresholding</td></tr><br />
<tr><td>4</td><td>PP2</td><td>Laplace Filtering</td></tr><br />
<tr><td>5</td><td>PP3</td><td>Logic Filtering</td></tr> <br />
</table><br />
Table 1: Pipeline structure for image edge detection[8]<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System<ref>Arne Halaas, Børge Svingen, Magnar Nedland, Pål Sætrom, Ola Snøve, Jr., and Olaf René Birkeland. 2004. A recursive MISD architecture for pattern matching. IEEE Trans. Very Large Scale Integr. Syst. 12, 7 (July 2004), 727-734. DOI=10.1109/TVLSI.2004.830918 http://dx.doi.org/10.1109/TVLSI.2004.830918</ref>===<br />
<br />
The situations where several patterns are required to be matched concurrently in data and no persistent index can be created for look-up requires the use of Online multipattern approximate searching. This algorithm requires evaluation of the hit function H(Si,P) which is the function working on strings and pattern,and requires partitioning P into smaller components and combining the individual results. A recursive approach is required and there exists a need to evaluate the Si value in parallel. Thus it helps in using an architecture with two complete binary trees which take care of the data distribution and result processing. The architecture is implemented with as many PE's that are practically possible within the implementation constraints. The data from the stream is fed to the root node of the data distribution tree and simultaneously distributed to several collection of PE's belonging to separate queries thus implementing the MISD architecture with a single data stream and multiple instruction streams.<br />
<br />
==Conclusion==<br />
The MISD architecture is one of the four types of parallel architecture described in Flynn’s Taxonomy. By definition, the MISD architecture has multiple instruction streams and a single data stream. The definition also describes this architecture as requiring that all processing elements receive the same data, but most commonly cited implementations of this class of architecture do not adhere to this detail. The result is there is debate among some experts on whether or not certain implementations are truly MISD.<br />
<br />
MISD architectures are not commonly found in industry as they tend to lack the scalability and efficiency of resource use of the MIMD and SIMD class of architectures. However, MISD architectures are efficient for solving specific classes of problems often found in research. Some examples of its use in research include image processing and systolic arrays. While rare, some implementations do exist outside of academic settings. One such example is fault tolerant systems such as the flight control system on the Boeing 777.<br />
<br />
=='''References'''==<br />
<references/><br />
<br />
=='''Glossary'''==<br />
*'''ALU''': Arithmetic Logic Unit, the control unit in a computer<br />
<br />
*'''CMU''': Carnegie Mellon University<br />
<br />
*'''Fly-By-Wire''': System that replaces the conventional manual flight controls of an aircraft with an electronic interface<br />
<br />
*'''FPGA''': Field Programmable Gate Array<br />
<br />
*'''MIMD''': Multiple Instruction Multiple Data<br />
<br />
*'''MISD''': Multiple Instruction Single Data<br />
<br />
*'''PE''': Processing Element<br />
<br />
*'''PFC''': Primary Flight Computer<br />
<br />
*'''SIMD''': Single Instruction Multiple Data<br />
<br />
*'''SISD''': Single Instruction Single Data<br />
<br />
==Quiz==<br />
<br />
'''Q1: Using the classical definition of the MISD architecture from Flynn’s Taxonomy, what is the minimum number of input streams that must be present to be classified as MISD architecture?'''<br />
<br />
A. One information stream, the data stream, the number of instruction streams do not matter.<br />
<br />
B. Two information streams, one instruction stream, one data stream.<br />
<br />
C. Three information streams, two instruction streams, one data stream.<br />
<br />
D. The number of information streams is irrelevant as per Flynn’s Taxonomy.<br />
<br />
''A1: C, Three information streams, two instruction streams, one data stream.''<br />
<br />
<br />
'''Q2: Why is the Pipeline architecture often cited as an example of MISD?'''<br />
<br />
A. The Pipeline architecture is exactly like the definition of MISD architecture from Flynn’s Taxonomy.<br />
<br />
B. The data to be processed travels linearly across all processing elements with each processing element executing a different instruction and passing it on to the next processing element. This line of data is cited to be a single “stream.”<br />
<br />
C. The data stream sends the same data to all processing elements for every cycle.<br />
<br />
D. Flynn based his description of MISD architecture on the Pipeline architecture in his paper that introduced Flynn’s Taxonomy.<br />
<br />
''A2: B, The data to be processed travels linearly across all processing elements with each processing element executing a different instruction and passing it on to the next processing element. This line of data is cited to be a single “stream.”''<br />
<br />
<br />
'''Q3: Why is the Pipeline architecture NOT considered a true MISD architecture by many experts?'''<br />
<br />
A. This is incorrect; there is consensus that the Pipeline architecture is synonymous with the MISD architecture described by Flynn’s Taxonomy.<br />
<br />
B. There is only a single, global information/control stream in the Pipeline architecture.<br />
<br />
C. The data passed to each processing element is different as it reflects the result of the processing of all the previous processing elements. Flynn’s Taxonomy describes the same data going to every processing element.<br />
<br />
D. It has been proven that it is impossible to build a MISD architecture parallel computer as described by Flynn’s Taxonomy.<br />
<br />
''A3: C, The data passed to each processing element is different as it reflects the result of the processing of all the previous processing elements. Flynn’s Taxonomy describes the same data going to every processing element.''<br />
<br />
<br />
'''Q4: True or False? MISD architecture is the most common parallel computing architecture used for general computing tasks.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A4: B, False, it is often not the best suited architecture for general computing tasks and as such not common outside of academic research.''<br />
<br />
<br />
'''Q5: When classifying a particular parallel architecture, it was noted that there was only one physical connection to provide all information streams, is it possible for this architecture to be classified as MISD architecture? (Note that there are three choices, you need to select the correct reason for the Yes/No/Maybe answer.)'''<br />
<br />
A. Yes, Flynn’s taxonomy does not require the number of physical connections in the hardware correspond to the number of logical streams.<br />
<br />
B. Maybe, we do not have enough information about the architecture to rule out the possibility of this parallel computer having a MISD architecture. We will need to see how the connections are physically laid out on the chip.<br />
<br />
C. No, there must be multiple physical information streams.<br />
<br />
''A5. A, Yes, Flynn’s taxonomy does not require the number of physical connections in the hardware correspond to the number of logical streams.''<br />
<br />
<br />
'''Q6. True or False? To be a MISD architecture, it requires that every processing element is identical.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A6. B, False, it is not required by definition of Flynn’s Taxonomy that all processing elements are identical. See example in the wiki reading about the fault tolerant flight control system on the Boeing 777.''<br />
<br />
<br />
'''Q7: Which of these is an application of the MISD architecture?'''<br />
<br />
A. Keyword search<br />
<br />
B. Controlling an assembly line.<br />
<br />
C. Home computing<br />
<br />
D. Image edge detection.<br />
<br />
''A7: D, Image edge detection.''<br />
<br />
<br />
'''Q8. True or False? The systolic array is another name for the MISD architecture.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A8. B, False, systolic arrays may be implemented using a MISD architecture but they also may be other architectures such as SIMD and MIMD.''<br />
<br />
<br />
'''Q9. True or False? MISD is the considered the easiest parallel architecture to scale.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A9. False, MIMD and SIMD are often used because of advantages in scaling.''<br />
<br />
<br />
'''Q10. Flynn’s Taxonomy relies on the number of logical information streams to classify the types of parallel architectures. What is the difference between a logical information stream and a physical information stream?'''<br />
<br />
A. A logical information stream is a connection between the ALU and the location of the data that is responsible for passing information about boolean (logical) computations, where the physical information stream it the connection between computer and other parts of the network.<br />
<br />
B. Physical information streams are determined by how the information is physically transported around chip while logical information streams are one layer of abstraction above the physical information streams. Logical streams describe conceptually how the information on the stream can be treated.<br />
<br />
C. Logical and physical information streams transport different kinds of information but are no different in implementation.<br />
<br />
D. There is no difference; the two terms can be used interchangeably.<br />
<br />
''A10. B, Physical information streams are determined by how the information is physically transported around chip while logical information streams are one layer of abstraction above the physical information streams. Logical streams describe conceptually how the information on the stream can be treated.''</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/1a_sp&diff=72481CSC/ECE 506 Spring 2013/1a sp2013-02-11T04:32:11Z<p>Schakra8: /* Special-purpose systolic array */</p>
<hr />
<div>==Multiple Instruction Single Data (MISD) Architecture==<br />
<br />
Multiple Instruction Single Data, or MISD, architecture is one of the four general categories of multi-processor or multi-core architectures described by Flynn's Taxonomy.<br />
<br />
===MISD Architectures within the Context of Flynn’s Taxonomy<ref>http://en.wikipedia.org/wiki/Flynn's_taxonomy</ref><ref>http://www.phy.ornl.gov/csep/ca/node11.html</ref>===<br />
<br />
Flynn's Taxonomy categorizes parallel computing architectures based on the number of concurrent information streams that flowed into the processor. According to Flynn's Taxonomy, there are two kinds of logical streams of information that flows into a parallel computer, the Instruction (or Control) Stream and the Data Stream. Note that for the purposes of classification using Flynn’s Taxonomy, it does not matter whether or not the streams are physically independent of each other on the hardware, only that the streams can be logically treated as independent. Based on how many of each type of stream flows into the parallel computer (single vs. multiple), Flynn's Taxonomy divides parallel architectures into four categories. They are Single Instruction, Single Data (SISD), Single Instruction, Multiple Data (SIMD), Multiple Instruction, Single Data (MISD), and Multiple Instruction, Multiple Data (MIMD). <br />
<br />
The following table (Figure 1) shows the categories of different parallel architectures according to Flynn:<ref>https://computing.llnl.gov/tutorials/parallel_comp/#Flynn</ref><br />
<br />
[[Image:Flynn's Taxonomy.PNG|thumb|center|400px|Figure 1. [http://en.wikipedia.org/wiki/Michael_J._Flynn Flynn]'s Taxonomy [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
===Overview of the MISD Architecture===<br />
<br />
In order to be categorized as MISD architecture, there must be at least two Instruction streams and there can only be one Data stream as shown on Figure 2.[[Image:MISD.PNG|thumb|right|100px|Figure 2. MISD Architecture [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]] As per the description according to Flynn’s Taxonomy, each processing elements should receive the same data and execute different instructions on the same data simultaneously. <br />
<br />
There are very few implementations of the classical MISD architecture as described by Flynn’s Taxonomy, none of which are commercial, general purpose parallel computers since the MIMD and SIMD are often better suited to common data parallel techniques. Under most circumstances MIMD and SIMD architectures tend to provide better scaling and more efficient use of computational resources. Instead of commercial use, MISD architectures are mainly used to create custom hardware in order to solve specific problems. The majority of the implementations have been created are in academia.<br />
<br />
Over the years, several architectures have been described as MISD even though the architectures do not completely follow the description in Flynn’s Taxonomy. One commonly cited example is the Pipeline architecture where the data is passed from processing element to processing element and each processing element executes a different instruction. It is often argued that this is not a true MISD architecture as the data changes as it moves through the pipeline. Similarly, other architectures or applications exist which may be called MISD by some while others disagree. The following section looks at some of those implementations.<br />
<br />
==Implementations of MISD architecture==<br />
<br />
Some major implementations of MISD are in the fields of systolic arrays, pattern matching, flight control systems and fault tolerance systems. These MISD implementations are discussed below.<br />
<br />
===Systolic Array===<br />
<br />
A systolic array is an arrangement of processors in an array where data flows synchronously across the array between neighbors, usually with different data flowing in different directions. Each Processor at each step takes in data from one or more neighbors, processes it and, in the next step, outputs results in the opposite direction. <br />
The systolic array paradigm, data-stream-driven by data counters, is the counterpart of the [http://en.wikipedia.org/wiki/Von_Neumann_model von Neumann paradigm], instruction-stream-driven by a program counter. Because a systolic array usually sends and receives multiple data streams, and multiple data counters are needed to generate these data streams, it supports data parallelism.<ref>http://en.wikipedia.org/wiki/Systolic_array</ref><br />
<br />
====Type of Systolic Arrays<ref>Johnson, K.T.; Hurson, A.R.; Shirazi, B.; , "General-purpose systolic arrays," Computer , vol.26, no.11, pp.20-31, Nov. 1993 doi: 10.1109/2.241423</ref>====<br />
<br />
=====Special-purpose systolic array=====<br />
<br />
An array of hardwired systolic processing elements tailored for a specific application. Typically, many tens or hundreds of cells fit on a single chip. One of the major applications of special-purpose systolic array is in matrix operations. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Systolic_1.png Figure 3] illustrates the algorithm for the sum of a scalar product, computed in a single systolic element. Here, a’s and b’s are synchronously shifted through the processing element to be available for next element. These data synchronously exits the processing element unmodified for the next element. The sum of the products is then shifted out of the accumulator.<br />
[[Image:systolic_1.png|thumb|centre|250px|Figure 3: The algorithm for the sum of a scalar product, computed in systolic element [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
=====General-purpose systolic array=====<br />
<br />
It is an array of systolic processing elements, which gets adapted to a variety of applications via programming or reconfiguration. Array topologies can be either programmable or reconfigurable. Likewise, array cells are either programmable or reconfigurable. This is referred to as Systolic topologies.<br />
<br />
======<font size="2">Programmable Systolic Array</font>======<br />
<br />
It is an array of programmable systolic elements that operates either in SIMD or MIMD fashion. Either the arrays interconnect or each processing unit is programmable and a program controls dataflow through the elements. Programmable systolic arrays are programmable either at a high level or a low level. At either level, programmable arrays can be categorized as either SIMD or MIMD machines.<br />
<br />
[[Image:systolic_4.png|thumb|right|250px|Figure 5: General organization of MIMD programmable linear systolic arrays [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
<br />
For the latter approach, the workstation downloads a program to each MIMD (Figure 4) systolic cell. Each cell may be loaded with a different program, or all the cells in the array may be loaded with the same program. Each cell's architecture is somewhat similar to the conventional von Neumann architecture: It contains a control unit, an ALU, and local memory. MIMD systolic cells have more local memory than their SIMD counterparts to support the von Neumann-style organization.<br />
<br />
This architecture is defined as Multiple Instruction Multiple Data (MIMD) architecture in [http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf 5]. The architecture has multiple instruction streams for the PEs and a single data stream passing through all the PEs. Thus, it can also be defined as Multiple Instruction Single Data (MISD) architecture. The architecture of Systolic array configuration are controversial as explained in the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Architecture_of_systolic_arrays_as_against_MISD_architecture section 4.1.2.]<br />
<br />
======<font size="2">Reconfigurable Systolic Array</font>======<br />
[[Image:reconfig.jpg|thumb|right|250px|Figure 6: Block Diagram of the RSA Architecture [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
It is an array of systolic elements that can be programmed at the lowest level. Recent gate density advances in FPGA technology have produced a low-level, reconfigurable systolic array architecture that bridges the gap between special-purpose arrays and the more versatile, programmable general-purpose arrays. The FPGA architecture is unusual because a single hardware platform can be logically reconfigured as an exact duplicate of a special-purpose systolic array. <br />
<br />
The RSA circuit design is based on systolic array architecture consisting of PEs interconnected via SWs as depicted in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Reconfig.jpg Figure 6]. The homogeneous characteristic of the Reconfigurable Systolic Array (RSA) architecture, where each reconfigurable processing element (PE) cell is connected to its nearest neighbors via configurable switch (SW) elements, enables array expansion for parallel processing and facilitates time sharing computation of high-throughput data by individual PEs. Both the PEs and SWs can be reconfigured dynamically with the former as an arithmetic processor and the latter as a flexible router linking the neighboring PE cells. The RSA shifts reconfiguration and input signals into the PEs and SWs on separate data bus which enables the circuit to continue its operation while the reconfiguration is in process.<br />
<br />
====Architecture of systolic arrays as against MISD architecture====<br />
[[Image:comp.png|thumb|right|250px|Figure 7.Comparison between Architecture of systolic arrays and MISD]]<br />
<br />
As from the above mentioned configurations of the Systolic Arrays, it is seen that generally the configurations have multiple processing elements executing different instructions from dedicated instruction streams for each processing element. There is a single data stream that connects the adjacent PEs. Thus, systolic array can be defined as an MISD architecture.<br />
<br />
Many authors say that as the data read as input by one processing element is processed data output of the adjacent PE. The data stream cannot be considered as single because all the data paths do not carry the same data to all the PEs. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Comp.png Figure 7] shows the difference between the Data Stream for Systolic Arrays and the MISD architecture. Thus the systolic array should be considered as “Multiple Data” architecture and not Single Data architecture.<br />
<br />
===Fault Tolerant Systems<ref>http://en.wikipedia.org/wiki/Fault-tolerant_computer_system#Types_of_fault_tolerance</ref>===<br />
<br />
The fault tolerant systems are designed to handle the possible failures in software, hardware or interfaces. The hardware faults include hard disk failures, input or output device failures, etc. and the software and interface faults include driver failures; operator errors, installing unexpected software etc. The hardware faults can be detected and identified by implementing redundant hardware and multiple backups. The software faults can be tolerable by removing the program errors by executing the software redundantly or by implementing small programs that take over the tasks that crash or generate errors.<br />
<br />
[[Image:fault.png|thumb|right|250px|Figure 8 MISD as fault tolerant architecture]]<br />
<br />
In computer systems, the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Single_Instruction.2C_Multiple_Data_streams_.28SIMD.29 SIMD], [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD] and [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instruction.2C_Multiple_Data_streams_.28MIMD.29 MIMD] architectures facilitate the implementation of the fault tolerance systems by multiple instruction streams or multiple data streams or both. Fault tolerance on computations can be implemented by multiple processors (likely with different architectures) executing the algorithms on the same set of data. The output of each processor is compared with that of the others and M out of N majority voting method is used to determine the faulty processor. Thus MISD architecture is utilized to get the fault tolerance on critical computations.<br />
<br />
There are various examples of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture] being used as fault tolerant architecture. The major examples being flight control systems, nuclear power plants, satellite systems, super collider experiment systems, etc. Here, the flight control system is explained as an example of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture].<br />
<br />
====The Flight Control System – MISD Example for fault tolerance====<br />
<br />
A [http://en.wikipedia.org/wiki/Fly-by-wire Fly-By-Wire] system is used to replace the manual flight control by an electronic control interface. The movements of the flight control in the cockpit are converted to electronic signals and are transmitted to the actuators by wires. The control computers use the feedback from the sensors to compute and control the movement of the actuators to provide the expected response. These computers also perform the task to stabilize the aircraft and perform other tasks without the knowledge of the pilot. Flight control systems must meet extremely high levels of accuracy and functional integrity.<br />
<br />
There are redundant flight control computers present in the flight control system. If one of the flight-control computers crashes, gets damaged or is affected by electromagnetic pulses, the other computer can overrule the faulty one and hence the flight of the aircraft is unharmed. [[Image:fig13.png|thumb|right|250px|Figure 9: Architecture of triple redundant 777 primary flight computer [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-7 6]]]The number of redundant flight control computers is generally more than two, so that any computer whose results disagree with the others is ruled out to be faulty and is either ignored or rebooted.<br />
<br />
=====Multiple Processors Implementation in Boeing 777<ref>Yeh, Y.C.;, "Triple-triple redundant 777 primary flight computer," Aerospace Applications Conference, 1996. Proceedings., 1996 IEEE , vol.1, no., pp.293-307 vol.1, 3-10 Feb 1996<br />
doi: 10.1109/AERO.1996.495891</ref>=====<br />
<br />
In modern computers, the redundant flight control computations are carried out by multiprocessor systems. The triple redundant 777 primary flight computer, has the architecture as shown in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Fig13.png Figure 9].<br />
<br />
The system has three primary flight control computers, each of them having three lanes with different processors. The flight control program is compiled for each of the processors which get the input data from the same data bus but drive the output on their individual control bus. Thus each processor executes different instructions but they process the same data. Thus, it is the best suited example of Multiple Instruction Single Data (MISD) architecture.<br />
<br />
The three processors selected for the flight control system of [http://en.wikipedia.org/wiki/Boeing_777 Boeing 777] were [http://en.wikipedia.org/wiki/Intel_80486 Intel 80486], [http://en.wikipedia.org/wiki/Motorola_68040 Motorola 68040] and [http://en.wikipedia.org/wiki/AMD_Am29000 AMD 29050]. The dissimilar processors lead to dissimilar interface hardware circuits and compilers. Each lane of the flight control computer is data synchronized with the other lanes so that all of the lanes read the same frame of data from the flight sensors. As the outputs of each lane can be different, the median value of the outputs is used to select the output of the lane to be considered. The lane which has the median value select hardware selected is said to be in “command mode” whereas the other lanes are said to be in “monitoring mode”. It receives the data from the other Primary Flight Computer (PFC) lanes and performs a median select of the outputs. This provides a fault blocking mechanism before the fault detection and identification by the cross-lane monitoring system. Thus, the MISD based multi computer architecture is capable of detecting generic errors in compilers or in complex hardware devices providing assurance beyond reasonable doubt of the dependability of the Fly-By-Wire system.<br />
<br />
The above mentioned system clearly has individual Instruction Streams as the architecture of each processor is different, thus different instruction sets and different instruction streams. These processors have frame synchronized input data which means they have same set of data to work upon which is fed from a single data stream. Thus the flight control system can be classified under MISD architecture.<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System<ref>M.Gajer;,“ Parallel Image Processing on The TMS320C80 Multiprocessor System”, http://focus.ti.com/pdfs/univ/11-CommunicationAudioSpeech.pdf</ref>===<br />
<br />
The image pre-processing system of this Multiprocessor, the pre-image processing time needed to be made shorter. To achieve the aim of reducing the pre-processing time, different types of image processing architectures for ‘C80 were presented. For example MIMD architecture is used for image histogram equalization; SIMD architecture for median filtering; MISD architecture for directional edge detection.<br />
<br />
The MISD architecture can be explained as follows. Here, each processor performs different operations on the same data set. It is mentioned at many places that MISD architecture was defined only for the sake of completeness of classification due to difficulty in practice to find a computer based on MISD architecture<ref>TMS320C80 System Level Synopsis, Texas Instruments, 1995</ref>. But in case of Image Processing, this assumption need not be true as here making many different operations on same image is rare and hence MISD architecture is a best fit for such cases. As an example of implementation of image processing operations for the MISD architecture the edge detection operation will be considered. The edges should be detected in four different directions (north, south, west and east) by the medium of four different filter masks. Each of the parallel processors of the TMS320C80 calculates one of the directional edges. Finally one obtains four images with detected edges, each in different direction.<br />
<br />
The image processing in computer is very often a multistage process composed of many stages. For example, an algorithm detecting contours of the objects in an image. The algorithm for achieving the before mentioned is composed of four basic image processing operation. The image acquired from the camera in the first step undergoes low-pass filtering, during which it is smoothed and noise is also eliminated. In the next step image is thresholding is performed and each pixel gets black or white depending on that if its grey level is below or above threshold value. The resulting image is a binary image from which edges are detected by the usage of the Laplace filter. In the last step the image with detected contours undergoes logic filtering in the purpose of elimination of single black pixels placed on the white background. For implementing the above processes parallel, a five-stage pipeline was used. The first stage of this pipeline constitutes the master processor which accounts for control of the image acquisition process. Further pipe line stages were implemented for 4 parallel processors as mentioned in table 1.<br />
<table border=1><br />
<tr><td>Stage</td><td>Processor</td><td>Operation</td></tr><br />
<tr><td>1</td><td>MP</td><td>Image acquisition</td></tr><br />
<tr><td>2</td><td>PP0/td><td>Low-pass Filtering</td></tr><br />
<tr><td>3</td><td>PP1</td><td>Thresholding</td></tr><br />
<tr><td>4</td><td>PP2</td><td>Laplace Filtering</td></tr><br />
<tr><td>5</td><td>PP3</td><td>Logic Filtering</td></tr> <br />
</table><br />
Table 1: Pipeline structure for image edge detection[8]<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System<ref>Arne Halaas, Børge Svingen, Magnar Nedland, Pål Sætrom, Ola Snøve, Jr., and Olaf René Birkeland. 2004. A recursive MISD architecture for pattern matching. IEEE Trans. Very Large Scale Integr. Syst. 12, 7 (July 2004), 727-734. DOI=10.1109/TVLSI.2004.830918 http://dx.doi.org/10.1109/TVLSI.2004.830918</ref>===<br />
<br />
The situations where several patterns are required to be matched concurrently in data and no persistent index can be created for look-up requires the use of Online multipattern approximate searching. This algorithm requires evaluation of the hit function H(Si,P) which is the function working on strings and pattern,and requires partitioning P into smaller components and combining the individual results. A recursive approach is required and there exists a need to evaluate the Si value in parallel. Thus it helps in using an architecture with two complete binary trees which take care of the data distribution and result processing. The architecture is implemented with as many PE's that are practically possible within the implementation constraints. The data from the stream is fed to the root node of the data distribution tree and simultaneously distributed to several collection of PE's belonging to separate queries thus implementing the MISD architecture with a single data stream and multiple instruction streams.<br />
<br />
==Conclusion==<br />
The MISD architecture is one of the four types of parallel architecture described in Flynn’s Taxonomy. By definition, the MISD architecture has multiple instruction streams and a single data stream. The definition also describes this architecture as requiring that all processing elements receive the same data, but most commonly cited implementations of this class of architecture do not adhere to this detail. The result is there is debate among some experts on whether or not certain implementations are truly MISD.<br />
<br />
MISD architectures are not commonly found in industry as they tend to lack the scalability and efficiency of resource use of the MIMD and SIMD class of architectures. However, MISD architectures are efficient for solving specific classes of problems often found in research. Some examples of its use in research include image processing and systolic arrays. While rare, some implementations do exist outside of academic settings. One such example is fault tolerant systems such as the flight control system on the Boeing 777.<br />
<br />
=='''References'''==<br />
<references/><br />
<br />
=='''Glossary'''==<br />
*'''ALU''': Arithmetic Logic Unit, the control unit in a computer<br />
<br />
*'''CMU''': Carnegie Mellon University<br />
<br />
*'''Fly-By-Wire''': System that replaces the conventional manual flight controls of an aircraft with an electronic interface<br />
<br />
*'''FPGA''': Field Programmable Gate Array<br />
<br />
*'''MIMD''': Multiple Instruction Multiple Data<br />
<br />
*'''MISD''': Multiple Instruction Single Data<br />
<br />
*'''PE''': Processing Element<br />
<br />
*'''PFC''': Primary Flight Computer<br />
<br />
*'''SIMD''': Single Instruction Multiple Data<br />
<br />
*'''SISD''': Single Instruction Single Data<br />
<br />
==Quiz==<br />
<br />
'''Q1: Using the classical definition of the MISD architecture from Flynn’s Taxonomy, what is the minimum number of input streams that must be present to be classified as MISD architecture?'''<br />
<br />
A. One information stream, the data stream, the number of instruction streams do not matter.<br />
<br />
B. Two information streams, one instruction stream, one data stream.<br />
<br />
C. Three information streams, two instruction streams, one data stream.<br />
<br />
D. The number of information streams is irrelevant as per Flynn’s Taxonomy.<br />
<br />
''A1: C, Three information streams, two instruction streams, one data stream.''<br />
<br />
<br />
'''Q2: Why is the Pipeline architecture often cited as an example of MISD?'''<br />
<br />
A. The Pipeline architecture is exactly like the definition of MISD architecture from Flynn’s Taxonomy.<br />
<br />
B. The data to be processed travels linearly across all processing elements with each processing element executing a different instruction and passing it on to the next processing element. This line of data is cited to be a single “stream.”<br />
<br />
C. The data stream sends the same data to all processing elements for every cycle.<br />
<br />
D. Flynn based his description of MISD architecture on the Pipeline architecture in his paper that introduced Flynn’s Taxonomy.<br />
<br />
''A2: B, The data to be processed travels linearly across all processing elements with each processing element executing a different instruction and passing it on to the next processing element. This line of data is cited to be a single “stream.”''<br />
<br />
<br />
'''Q3: Why is the Pipeline architecture NOT considered a true MISD architecture by many experts?'''<br />
<br />
A. This is incorrect; there is consensus that the Pipeline architecture is synonymous with the MISD architecture described by Flynn’s Taxonomy.<br />
<br />
B. There is only a single, global information/control stream in the Pipeline architecture.<br />
<br />
C. The data passed to each processing element is different as it reflects the result of the processing of all the previous processing elements. Flynn’s Taxonomy describes the same data going to every processing element.<br />
<br />
D. It has been proven that it is impossible to build a MISD architecture parallel computer as described by Flynn’s Taxonomy.<br />
<br />
''A3: C, The data passed to each processing element is different as it reflects the result of the processing of all the previous processing elements. Flynn’s Taxonomy describes the same data going to every processing element.''<br />
<br />
<br />
'''Q4: True or False? MISD architecture is the most common parallel computing architecture used for general computing tasks.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A4: B, False, it is often not the best suited architecture for general computing tasks and as such not common outside of academic research.''<br />
<br />
<br />
'''Q5: When classifying a particular parallel architecture, it was noted that there was only one physical connection to provide all information streams, is it possible for this architecture to be classified as MISD architecture? (Note that there are three choices, you need to select the correct reason for the Yes/No/Maybe answer.)'''<br />
<br />
A. Yes, Flynn’s taxonomy does not require the number of physical connections in the hardware correspond to the number of logical streams.<br />
<br />
B. Maybe, we do not have enough information about the architecture to rule out the possibility of this parallel computer having a MISD architecture. We will need to see how the connections are physically laid out on the chip.<br />
<br />
C. No, there must be multiple physical information streams.<br />
<br />
''A5. A, Yes, Flynn’s taxonomy does not require the number of physical connections in the hardware correspond to the number of logical streams.''<br />
<br />
<br />
'''Q6. True or False? To be a MISD architecture, it requires that every processing element is identical.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A6. B, False, it is not required by definition of Flynn’s Taxonomy that all processing elements are identical. See example in the wiki reading about the fault tolerant flight control system on the Boeing 777.''<br />
<br />
<br />
'''Q7: Which of these is an application of the MISD architecture?'''<br />
<br />
A. Keyword search<br />
<br />
B. Controlling an assembly line.<br />
<br />
C. Home computing<br />
<br />
D. Image edge detection.<br />
<br />
''A7: D, Image edge detection.''<br />
<br />
<br />
'''Q8. True or False? The systolic array is another name for the MISD architecture.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A8. B, False, systolic arrays may be implemented using a MISD architecture but they also may be other architectures such as SIMD and MIMD.''<br />
<br />
<br />
'''Q9. True or False? MISD is the considered the easiest parallel architecture to scale.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A9. False, MIMD and SIMD are often used because of advantages in scaling.''<br />
<br />
<br />
'''Q10. Flynn’s Taxonomy relies on the number of logical information streams to classify the types of parallel architectures. What is the difference between a logical information stream and a physical information stream?'''<br />
<br />
A. A logical information stream is a connection between the ALU and the location of the data that is responsible for passing information about boolean (logical) computations, where the physical information stream it the connection between computer and other parts of the network.<br />
<br />
B. Physical information streams are determined by how the information is physically transported around chip while logical information streams are one layer of abstraction above the physical information streams. Logical streams describe conceptually how the information on the stream can be treated.<br />
<br />
C. Logical and physical information streams transport different kinds of information but are no different in implementation.<br />
<br />
D. There is no difference; the two terms can be used interchangeably.<br />
<br />
''A10. B, Physical information streams are determined by how the information is physically transported around chip while logical information streams are one layer of abstraction above the physical information streams. Logical streams describe conceptually how the information on the stream can be treated.''</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/1a_sp&diff=72480CSC/ECE 506 Spring 2013/1a sp2013-02-11T04:31:33Z<p>Schakra8: /* Special-purpose systolic array */</p>
<hr />
<div>==Multiple Instruction Single Data (MISD) Architecture==<br />
<br />
Multiple Instruction Single Data, or MISD, architecture is one of the four general categories of multi-processor or multi-core architectures described by Flynn's Taxonomy.<br />
<br />
===MISD Architectures within the Context of Flynn’s Taxonomy<ref>http://en.wikipedia.org/wiki/Flynn's_taxonomy</ref><ref>http://www.phy.ornl.gov/csep/ca/node11.html</ref>===<br />
<br />
Flynn's Taxonomy categorizes parallel computing architectures based on the number of concurrent information streams that flowed into the processor. According to Flynn's Taxonomy, there are two kinds of logical streams of information that flows into a parallel computer, the Instruction (or Control) Stream and the Data Stream. Note that for the purposes of classification using Flynn’s Taxonomy, it does not matter whether or not the streams are physically independent of each other on the hardware, only that the streams can be logically treated as independent. Based on how many of each type of stream flows into the parallel computer (single vs. multiple), Flynn's Taxonomy divides parallel architectures into four categories. They are Single Instruction, Single Data (SISD), Single Instruction, Multiple Data (SIMD), Multiple Instruction, Single Data (MISD), and Multiple Instruction, Multiple Data (MIMD). <br />
<br />
The following table (Figure 1) shows the categories of different parallel architectures according to Flynn:<ref>https://computing.llnl.gov/tutorials/parallel_comp/#Flynn</ref><br />
<br />
[[Image:Flynn's Taxonomy.PNG|thumb|center|400px|Figure 1. [http://en.wikipedia.org/wiki/Michael_J._Flynn Flynn]'s Taxonomy [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
===Overview of the MISD Architecture===<br />
<br />
In order to be categorized as MISD architecture, there must be at least two Instruction streams and there can only be one Data stream as shown on Figure 2.[[Image:MISD.PNG|thumb|right|100px|Figure 2. MISD Architecture [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]] As per the description according to Flynn’s Taxonomy, each processing elements should receive the same data and execute different instructions on the same data simultaneously. <br />
<br />
There are very few implementations of the classical MISD architecture as described by Flynn’s Taxonomy, none of which are commercial, general purpose parallel computers since the MIMD and SIMD are often better suited to common data parallel techniques. Under most circumstances MIMD and SIMD architectures tend to provide better scaling and more efficient use of computational resources. Instead of commercial use, MISD architectures are mainly used to create custom hardware in order to solve specific problems. The majority of the implementations have been created are in academia.<br />
<br />
Over the years, several architectures have been described as MISD even though the architectures do not completely follow the description in Flynn’s Taxonomy. One commonly cited example is the Pipeline architecture where the data is passed from processing element to processing element and each processing element executes a different instruction. It is often argued that this is not a true MISD architecture as the data changes as it moves through the pipeline. Similarly, other architectures or applications exist which may be called MISD by some while others disagree. The following section looks at some of those implementations.<br />
<br />
==Implementations of MISD architecture==<br />
<br />
Some major implementations of MISD are in the fields of systolic arrays, pattern matching, flight control systems and fault tolerance systems. These MISD implementations are discussed below.<br />
<br />
===Systolic Array===<br />
<br />
A systolic array is an arrangement of processors in an array where data flows synchronously across the array between neighbors, usually with different data flowing in different directions. Each Processor at each step takes in data from one or more neighbors, processes it and, in the next step, outputs results in the opposite direction. <br />
The systolic array paradigm, data-stream-driven by data counters, is the counterpart of the [http://en.wikipedia.org/wiki/Von_Neumann_model von Neumann paradigm], instruction-stream-driven by a program counter. Because a systolic array usually sends and receives multiple data streams, and multiple data counters are needed to generate these data streams, it supports data parallelism.<ref>http://en.wikipedia.org/wiki/Systolic_array</ref><br />
<br />
====Type of Systolic Arrays<ref>Johnson, K.T.; Hurson, A.R.; Shirazi, B.; , "General-purpose systolic arrays," Computer , vol.26, no.11, pp.20-31, Nov. 1993 doi: 10.1109/2.241423</ref>====<br />
<br />
=====Special-purpose systolic array=====<br />
<br />
An array of hardwired systolic processing elements tailored for a specific application. Typically, many tens or hundreds of cells fit on a single chip. One of the major applications of special-purpose systolic array is in matrix operations. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Systolic_1.png Figure 3] illustrates the algorithm for the sum of a scalar product, computed in a single systolic element. Here, a’s and b’s are synchronously shifted through the processing element to be available for next element. These data synchronously exits the processing element unmodified for the next element. The sum of the products is then shifted out of the accumulator.<br />
[[Image:systolic_1.png|thumb|left|250px|Figure 3: The algorithm for the sum of a scalar product, computed in systolic element [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
[[Image:systolic_2.png|thumb|right|250px|Figure 4: The systolic product of two 3x3 matrices [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
=====General-purpose systolic array=====<br />
<br />
It is an array of systolic processing elements, which gets adapted to a variety of applications via programming or reconfiguration. Array topologies can be either programmable or reconfigurable. Likewise, array cells are either programmable or reconfigurable. This is referred to as Systolic topologies.<br />
<br />
======<font size="2">Programmable Systolic Array</font>======<br />
<br />
It is an array of programmable systolic elements that operates either in SIMD or MIMD fashion. Either the arrays interconnect or each processing unit is programmable and a program controls dataflow through the elements. Programmable systolic arrays are programmable either at a high level or a low level. At either level, programmable arrays can be categorized as either SIMD or MIMD machines.<br />
<br />
[[Image:systolic_4.png|thumb|right|250px|Figure 5: General organization of MIMD programmable linear systolic arrays [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
<br />
For the latter approach, the workstation downloads a program to each MIMD (Figure 4) systolic cell. Each cell may be loaded with a different program, or all the cells in the array may be loaded with the same program. Each cell's architecture is somewhat similar to the conventional von Neumann architecture: It contains a control unit, an ALU, and local memory. MIMD systolic cells have more local memory than their SIMD counterparts to support the von Neumann-style organization.<br />
<br />
This architecture is defined as Multiple Instruction Multiple Data (MIMD) architecture in [http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf 5]. The architecture has multiple instruction streams for the PEs and a single data stream passing through all the PEs. Thus, it can also be defined as Multiple Instruction Single Data (MISD) architecture. The architecture of Systolic array configuration are controversial as explained in the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Architecture_of_systolic_arrays_as_against_MISD_architecture section 4.1.2.]<br />
<br />
======<font size="2">Reconfigurable Systolic Array</font>======<br />
[[Image:reconfig.jpg|thumb|right|250px|Figure 6: Block Diagram of the RSA Architecture [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
It is an array of systolic elements that can be programmed at the lowest level. Recent gate density advances in FPGA technology have produced a low-level, reconfigurable systolic array architecture that bridges the gap between special-purpose arrays and the more versatile, programmable general-purpose arrays. The FPGA architecture is unusual because a single hardware platform can be logically reconfigured as an exact duplicate of a special-purpose systolic array. <br />
<br />
The RSA circuit design is based on systolic array architecture consisting of PEs interconnected via SWs as depicted in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Reconfig.jpg Figure 6]. The homogeneous characteristic of the Reconfigurable Systolic Array (RSA) architecture, where each reconfigurable processing element (PE) cell is connected to its nearest neighbors via configurable switch (SW) elements, enables array expansion for parallel processing and facilitates time sharing computation of high-throughput data by individual PEs. Both the PEs and SWs can be reconfigured dynamically with the former as an arithmetic processor and the latter as a flexible router linking the neighboring PE cells. The RSA shifts reconfiguration and input signals into the PEs and SWs on separate data bus which enables the circuit to continue its operation while the reconfiguration is in process.<br />
<br />
====Architecture of systolic arrays as against MISD architecture====<br />
[[Image:comp.png|thumb|right|250px|Figure 7.Comparison between Architecture of systolic arrays and MISD]]<br />
<br />
As from the above mentioned configurations of the Systolic Arrays, it is seen that generally the configurations have multiple processing elements executing different instructions from dedicated instruction streams for each processing element. There is a single data stream that connects the adjacent PEs. Thus, systolic array can be defined as an MISD architecture.<br />
<br />
Many authors say that as the data read as input by one processing element is processed data output of the adjacent PE. The data stream cannot be considered as single because all the data paths do not carry the same data to all the PEs. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Comp.png Figure 7] shows the difference between the Data Stream for Systolic Arrays and the MISD architecture. Thus the systolic array should be considered as “Multiple Data” architecture and not Single Data architecture.<br />
<br />
===Fault Tolerant Systems<ref>http://en.wikipedia.org/wiki/Fault-tolerant_computer_system#Types_of_fault_tolerance</ref>===<br />
<br />
The fault tolerant systems are designed to handle the possible failures in software, hardware or interfaces. The hardware faults include hard disk failures, input or output device failures, etc. and the software and interface faults include driver failures; operator errors, installing unexpected software etc. The hardware faults can be detected and identified by implementing redundant hardware and multiple backups. The software faults can be tolerable by removing the program errors by executing the software redundantly or by implementing small programs that take over the tasks that crash or generate errors.<br />
<br />
[[Image:fault.png|thumb|right|250px|Figure 8 MISD as fault tolerant architecture]]<br />
<br />
In computer systems, the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Single_Instruction.2C_Multiple_Data_streams_.28SIMD.29 SIMD], [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD] and [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instruction.2C_Multiple_Data_streams_.28MIMD.29 MIMD] architectures facilitate the implementation of the fault tolerance systems by multiple instruction streams or multiple data streams or both. Fault tolerance on computations can be implemented by multiple processors (likely with different architectures) executing the algorithms on the same set of data. The output of each processor is compared with that of the others and M out of N majority voting method is used to determine the faulty processor. Thus MISD architecture is utilized to get the fault tolerance on critical computations.<br />
<br />
There are various examples of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture] being used as fault tolerant architecture. The major examples being flight control systems, nuclear power plants, satellite systems, super collider experiment systems, etc. Here, the flight control system is explained as an example of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture].<br />
<br />
====The Flight Control System – MISD Example for fault tolerance====<br />
<br />
A [http://en.wikipedia.org/wiki/Fly-by-wire Fly-By-Wire] system is used to replace the manual flight control by an electronic control interface. The movements of the flight control in the cockpit are converted to electronic signals and are transmitted to the actuators by wires. The control computers use the feedback from the sensors to compute and control the movement of the actuators to provide the expected response. These computers also perform the task to stabilize the aircraft and perform other tasks without the knowledge of the pilot. Flight control systems must meet extremely high levels of accuracy and functional integrity.<br />
<br />
There are redundant flight control computers present in the flight control system. If one of the flight-control computers crashes, gets damaged or is affected by electromagnetic pulses, the other computer can overrule the faulty one and hence the flight of the aircraft is unharmed. [[Image:fig13.png|thumb|right|250px|Figure 9: Architecture of triple redundant 777 primary flight computer [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-7 6]]]The number of redundant flight control computers is generally more than two, so that any computer whose results disagree with the others is ruled out to be faulty and is either ignored or rebooted.<br />
<br />
=====Multiple Processors Implementation in Boeing 777<ref>Yeh, Y.C.;, "Triple-triple redundant 777 primary flight computer," Aerospace Applications Conference, 1996. Proceedings., 1996 IEEE , vol.1, no., pp.293-307 vol.1, 3-10 Feb 1996<br />
doi: 10.1109/AERO.1996.495891</ref>=====<br />
<br />
In modern computers, the redundant flight control computations are carried out by multiprocessor systems. The triple redundant 777 primary flight computer, has the architecture as shown in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Fig13.png Figure 9].<br />
<br />
The system has three primary flight control computers, each of them having three lanes with different processors. The flight control program is compiled for each of the processors which get the input data from the same data bus but drive the output on their individual control bus. Thus each processor executes different instructions but they process the same data. Thus, it is the best suited example of Multiple Instruction Single Data (MISD) architecture.<br />
<br />
The three processors selected for the flight control system of [http://en.wikipedia.org/wiki/Boeing_777 Boeing 777] were [http://en.wikipedia.org/wiki/Intel_80486 Intel 80486], [http://en.wikipedia.org/wiki/Motorola_68040 Motorola 68040] and [http://en.wikipedia.org/wiki/AMD_Am29000 AMD 29050]. The dissimilar processors lead to dissimilar interface hardware circuits and compilers. Each lane of the flight control computer is data synchronized with the other lanes so that all of the lanes read the same frame of data from the flight sensors. As the outputs of each lane can be different, the median value of the outputs is used to select the output of the lane to be considered. The lane which has the median value select hardware selected is said to be in “command mode” whereas the other lanes are said to be in “monitoring mode”. It receives the data from the other Primary Flight Computer (PFC) lanes and performs a median select of the outputs. This provides a fault blocking mechanism before the fault detection and identification by the cross-lane monitoring system. Thus, the MISD based multi computer architecture is capable of detecting generic errors in compilers or in complex hardware devices providing assurance beyond reasonable doubt of the dependability of the Fly-By-Wire system.<br />
<br />
The above mentioned system clearly has individual Instruction Streams as the architecture of each processor is different, thus different instruction sets and different instruction streams. These processors have frame synchronized input data which means they have same set of data to work upon which is fed from a single data stream. Thus the flight control system can be classified under MISD architecture.<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System<ref>M.Gajer;,“ Parallel Image Processing on The TMS320C80 Multiprocessor System”, http://focus.ti.com/pdfs/univ/11-CommunicationAudioSpeech.pdf</ref>===<br />
<br />
The image pre-processing system of this Multiprocessor, the pre-image processing time needed to be made shorter. To achieve the aim of reducing the pre-processing time, different types of image processing architectures for ‘C80 were presented. For example MIMD architecture is used for image histogram equalization; SIMD architecture for median filtering; MISD architecture for directional edge detection.<br />
<br />
The MISD architecture can be explained as follows. Here, each processor performs different operations on the same data set. It is mentioned at many places that MISD architecture was defined only for the sake of completeness of classification due to difficulty in practice to find a computer based on MISD architecture<ref>TMS320C80 System Level Synopsis, Texas Instruments, 1995</ref>. But in case of Image Processing, this assumption need not be true as here making many different operations on same image is rare and hence MISD architecture is a best fit for such cases. As an example of implementation of image processing operations for the MISD architecture the edge detection operation will be considered. The edges should be detected in four different directions (north, south, west and east) by the medium of four different filter masks. Each of the parallel processors of the TMS320C80 calculates one of the directional edges. Finally one obtains four images with detected edges, each in different direction.<br />
<br />
The image processing in computer is very often a multistage process composed of many stages. For example, an algorithm detecting contours of the objects in an image. The algorithm for achieving the before mentioned is composed of four basic image processing operation. The image acquired from the camera in the first step undergoes low-pass filtering, during which it is smoothed and noise is also eliminated. In the next step image is thresholding is performed and each pixel gets black or white depending on that if its grey level is below or above threshold value. The resulting image is a binary image from which edges are detected by the usage of the Laplace filter. In the last step the image with detected contours undergoes logic filtering in the purpose of elimination of single black pixels placed on the white background. For implementing the above processes parallel, a five-stage pipeline was used. The first stage of this pipeline constitutes the master processor which accounts for control of the image acquisition process. Further pipe line stages were implemented for 4 parallel processors as mentioned in table 1.<br />
<table border=1><br />
<tr><td>Stage</td><td>Processor</td><td>Operation</td></tr><br />
<tr><td>1</td><td>MP</td><td>Image acquisition</td></tr><br />
<tr><td>2</td><td>PP0/td><td>Low-pass Filtering</td></tr><br />
<tr><td>3</td><td>PP1</td><td>Thresholding</td></tr><br />
<tr><td>4</td><td>PP2</td><td>Laplace Filtering</td></tr><br />
<tr><td>5</td><td>PP3</td><td>Logic Filtering</td></tr> <br />
</table><br />
Table 1: Pipeline structure for image edge detection[8]<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System<ref>Arne Halaas, Børge Svingen, Magnar Nedland, Pål Sætrom, Ola Snøve, Jr., and Olaf René Birkeland. 2004. A recursive MISD architecture for pattern matching. IEEE Trans. Very Large Scale Integr. Syst. 12, 7 (July 2004), 727-734. DOI=10.1109/TVLSI.2004.830918 http://dx.doi.org/10.1109/TVLSI.2004.830918</ref>===<br />
<br />
The situations where several patterns are required to be matched concurrently in data and no persistent index can be created for look-up requires the use of Online multipattern approximate searching. This algorithm requires evaluation of the hit function H(Si,P) which is the function working on strings and pattern,and requires partitioning P into smaller components and combining the individual results. A recursive approach is required and there exists a need to evaluate the Si value in parallel. Thus it helps in using an architecture with two complete binary trees which take care of the data distribution and result processing. The architecture is implemented with as many PE's that are practically possible within the implementation constraints. The data from the stream is fed to the root node of the data distribution tree and simultaneously distributed to several collection of PE's belonging to separate queries thus implementing the MISD architecture with a single data stream and multiple instruction streams.<br />
<br />
==Conclusion==<br />
The MISD architecture is one of the four types of parallel architecture described in Flynn’s Taxonomy. By definition, the MISD architecture has multiple instruction streams and a single data stream. The definition also describes this architecture as requiring that all processing elements receive the same data, but most commonly cited implementations of this class of architecture do not adhere to this detail. The result is there is debate among some experts on whether or not certain implementations are truly MISD.<br />
<br />
MISD architectures are not commonly found in industry as they tend to lack the scalability and efficiency of resource use of the MIMD and SIMD class of architectures. However, MISD architectures are efficient for solving specific classes of problems often found in research. Some examples of its use in research include image processing and systolic arrays. While rare, some implementations do exist outside of academic settings. One such example is fault tolerant systems such as the flight control system on the Boeing 777.<br />
<br />
=='''References'''==<br />
<references/><br />
<br />
=='''Glossary'''==<br />
*'''ALU''': Arithmetic Logic Unit, the control unit in a computer<br />
<br />
*'''CMU''': Carnegie Mellon University<br />
<br />
*'''Fly-By-Wire''': System that replaces the conventional manual flight controls of an aircraft with an electronic interface<br />
<br />
*'''FPGA''': Field Programmable Gate Array<br />
<br />
*'''MIMD''': Multiple Instruction Multiple Data<br />
<br />
*'''MISD''': Multiple Instruction Single Data<br />
<br />
*'''PE''': Processing Element<br />
<br />
*'''PFC''': Primary Flight Computer<br />
<br />
*'''SIMD''': Single Instruction Multiple Data<br />
<br />
*'''SISD''': Single Instruction Single Data<br />
<br />
==Quiz==<br />
<br />
'''Q1: Using the classical definition of the MISD architecture from Flynn’s Taxonomy, what is the minimum number of input streams that must be present to be classified as MISD architecture?'''<br />
<br />
A. One information stream, the data stream, the number of instruction streams do not matter.<br />
<br />
B. Two information streams, one instruction stream, one data stream.<br />
<br />
C. Three information streams, two instruction streams, one data stream.<br />
<br />
D. The number of information streams is irrelevant as per Flynn’s Taxonomy.<br />
<br />
''A1: C, Three information streams, two instruction streams, one data stream.''<br />
<br />
<br />
'''Q2: Why is the Pipeline architecture often cited as an example of MISD?'''<br />
<br />
A. The Pipeline architecture is exactly like the definition of MISD architecture from Flynn’s Taxonomy.<br />
<br />
B. The data to be processed travels linearly across all processing elements with each processing element executing a different instruction and passing it on to the next processing element. This line of data is cited to be a single “stream.”<br />
<br />
C. The data stream sends the same data to all processing elements for every cycle.<br />
<br />
D. Flynn based his description of MISD architecture on the Pipeline architecture in his paper that introduced Flynn’s Taxonomy.<br />
<br />
''A2: B, The data to be processed travels linearly across all processing elements with each processing element executing a different instruction and passing it on to the next processing element. This line of data is cited to be a single “stream.”''<br />
<br />
<br />
'''Q3: Why is the Pipeline architecture NOT considered a true MISD architecture by many experts?'''<br />
<br />
A. This is incorrect; there is consensus that the Pipeline architecture is synonymous with the MISD architecture described by Flynn’s Taxonomy.<br />
<br />
B. There is only a single, global information/control stream in the Pipeline architecture.<br />
<br />
C. The data passed to each processing element is different as it reflects the result of the processing of all the previous processing elements. Flynn’s Taxonomy describes the same data going to every processing element.<br />
<br />
D. It has been proven that it is impossible to build a MISD architecture parallel computer as described by Flynn’s Taxonomy.<br />
<br />
''A3: C, The data passed to each processing element is different as it reflects the result of the processing of all the previous processing elements. Flynn’s Taxonomy describes the same data going to every processing element.''<br />
<br />
<br />
'''Q4: True or False? MISD architecture is the most common parallel computing architecture used for general computing tasks.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A4: B, False, it is often not the best suited architecture for general computing tasks and as such not common outside of academic research.''<br />
<br />
<br />
'''Q5: When classifying a particular parallel architecture, it was noted that there was only one physical connection to provide all information streams, is it possible for this architecture to be classified as MISD architecture? (Note that there are three choices, you need to select the correct reason for the Yes/No/Maybe answer.)'''<br />
<br />
A. Yes, Flynn’s taxonomy does not require the number of physical connections in the hardware correspond to the number of logical streams.<br />
<br />
B. Maybe, we do not have enough information about the architecture to rule out the possibility of this parallel computer having a MISD architecture. We will need to see how the connections are physically laid out on the chip.<br />
<br />
C. No, there must be multiple physical information streams.<br />
<br />
''A5. A, Yes, Flynn’s taxonomy does not require the number of physical connections in the hardware correspond to the number of logical streams.''<br />
<br />
<br />
'''Q6. True or False? To be a MISD architecture, it requires that every processing element is identical.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A6. B, False, it is not required by definition of Flynn’s Taxonomy that all processing elements are identical. See example in the wiki reading about the fault tolerant flight control system on the Boeing 777.''<br />
<br />
<br />
'''Q7: Which of these is an application of the MISD architecture?'''<br />
<br />
A. Keyword search<br />
<br />
B. Controlling an assembly line.<br />
<br />
C. Home computing<br />
<br />
D. Image edge detection.<br />
<br />
''A7: D, Image edge detection.''<br />
<br />
<br />
'''Q8. True or False? The systolic array is another name for the MISD architecture.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A8. B, False, systolic arrays may be implemented using a MISD architecture but they also may be other architectures such as SIMD and MIMD.''<br />
<br />
<br />
'''Q9. True or False? MISD is the considered the easiest parallel architecture to scale.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A9. False, MIMD and SIMD are often used because of advantages in scaling.''<br />
<br />
<br />
'''Q10. Flynn’s Taxonomy relies on the number of logical information streams to classify the types of parallel architectures. What is the difference between a logical information stream and a physical information stream?'''<br />
<br />
A. A logical information stream is a connection between the ALU and the location of the data that is responsible for passing information about boolean (logical) computations, where the physical information stream it the connection between computer and other parts of the network.<br />
<br />
B. Physical information streams are determined by how the information is physically transported around chip while logical information streams are one layer of abstraction above the physical information streams. Logical streams describe conceptually how the information on the stream can be treated.<br />
<br />
C. Logical and physical information streams transport different kinds of information but are no different in implementation.<br />
<br />
D. There is no difference; the two terms can be used interchangeably.<br />
<br />
''A10. B, Physical information streams are determined by how the information is physically transported around chip while logical information streams are one layer of abstraction above the physical information streams. Logical streams describe conceptually how the information on the stream can be treated.''</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/1a_sp&diff=72479CSC/ECE 506 Spring 2013/1a sp2013-02-11T04:24:52Z<p>Schakra8: /* Overview of the MISD Architecture */</p>
<hr />
<div>==Multiple Instruction Single Data (MISD) Architecture==<br />
<br />
Multiple Instruction Single Data, or MISD, architecture is one of the four general categories of multi-processor or multi-core architectures described by Flynn's Taxonomy.<br />
<br />
===MISD Architectures within the Context of Flynn’s Taxonomy<ref>http://en.wikipedia.org/wiki/Flynn's_taxonomy</ref><ref>http://www.phy.ornl.gov/csep/ca/node11.html</ref>===<br />
<br />
Flynn's Taxonomy categorizes parallel computing architectures based on the number of concurrent information streams that flowed into the processor. According to Flynn's Taxonomy, there are two kinds of logical streams of information that flows into a parallel computer, the Instruction (or Control) Stream and the Data Stream. Note that for the purposes of classification using Flynn’s Taxonomy, it does not matter whether or not the streams are physically independent of each other on the hardware, only that the streams can be logically treated as independent. Based on how many of each type of stream flows into the parallel computer (single vs. multiple), Flynn's Taxonomy divides parallel architectures into four categories. They are Single Instruction, Single Data (SISD), Single Instruction, Multiple Data (SIMD), Multiple Instruction, Single Data (MISD), and Multiple Instruction, Multiple Data (MIMD). <br />
<br />
The following table (Figure 1) shows the categories of different parallel architectures according to Flynn:<ref>https://computing.llnl.gov/tutorials/parallel_comp/#Flynn</ref><br />
<br />
[[Image:Flynn's Taxonomy.PNG|thumb|center|400px|Figure 1. [http://en.wikipedia.org/wiki/Michael_J._Flynn Flynn]'s Taxonomy [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
===Overview of the MISD Architecture===<br />
<br />
In order to be categorized as MISD architecture, there must be at least two Instruction streams and there can only be one Data stream as shown on Figure 2.[[Image:MISD.PNG|thumb|right|100px|Figure 2. MISD Architecture [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]] As per the description according to Flynn’s Taxonomy, each processing elements should receive the same data and execute different instructions on the same data simultaneously. <br />
<br />
There are very few implementations of the classical MISD architecture as described by Flynn’s Taxonomy, none of which are commercial, general purpose parallel computers since the MIMD and SIMD are often better suited to common data parallel techniques. Under most circumstances MIMD and SIMD architectures tend to provide better scaling and more efficient use of computational resources. Instead of commercial use, MISD architectures are mainly used to create custom hardware in order to solve specific problems. The majority of the implementations have been created are in academia.<br />
<br />
Over the years, several architectures have been described as MISD even though the architectures do not completely follow the description in Flynn’s Taxonomy. One commonly cited example is the Pipeline architecture where the data is passed from processing element to processing element and each processing element executes a different instruction. It is often argued that this is not a true MISD architecture as the data changes as it moves through the pipeline. Similarly, other architectures or applications exist which may be called MISD by some while others disagree. The following section looks at some of those implementations.<br />
<br />
==Implementations of MISD architecture==<br />
<br />
Some major implementations of MISD are in the fields of systolic arrays, pattern matching, flight control systems and fault tolerance systems. These MISD implementations are discussed below.<br />
<br />
===Systolic Array===<br />
<br />
A systolic array is an arrangement of processors in an array where data flows synchronously across the array between neighbors, usually with different data flowing in different directions. Each Processor at each step takes in data from one or more neighbors, processes it and, in the next step, outputs results in the opposite direction. <br />
The systolic array paradigm, data-stream-driven by data counters, is the counterpart of the [http://en.wikipedia.org/wiki/Von_Neumann_model von Neumann paradigm], instruction-stream-driven by a program counter. Because a systolic array usually sends and receives multiple data streams, and multiple data counters are needed to generate these data streams, it supports data parallelism.<ref>http://en.wikipedia.org/wiki/Systolic_array</ref><br />
<br />
====Type of Systolic Arrays<ref>Johnson, K.T.; Hurson, A.R.; Shirazi, B.; , "General-purpose systolic arrays," Computer , vol.26, no.11, pp.20-31, Nov. 1993 doi: 10.1109/2.241423</ref>====<br />
<br />
=====Special-purpose systolic array=====<br />
[[Image:systolic_1.png|thumb|right|250px|Figure 3: The algorithm for the sum of a scalar product, computed in systolic element [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
[[Image:systolic_2.png|thumb|right|250px|Figure 4: The systolic product of two 3x3 matrices [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
An array of hardwired systolic processing elements tailored for a specific application. Typically, many tens or hundreds of cells fit on a single chip. One of the major applications of special-purpose systolic array is in matrix operations. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Systolic_1.png Figure 3] illustrates the algorithm for the sum of a scalar product, computed in a single systolic element. Here, a’s and b’s are synchronously shifted through the processing element to be available for next element. These data synchronously exits the processing element unmodified for the next element. The sum of the products is then shifted out of the accumulator.<br />
<br />
=====General-purpose systolic array=====<br />
<br />
It is an array of systolic processing elements, which gets adapted to a variety of applications via programming or reconfiguration. Array topologies can be either programmable or reconfigurable. Likewise, array cells are either programmable or reconfigurable. This is referred to as Systolic topologies.<br />
<br />
======<font size="2">Programmable Systolic Array</font>======<br />
<br />
It is an array of programmable systolic elements that operates either in SIMD or MIMD fashion. Either the arrays interconnect or each processing unit is programmable and a program controls dataflow through the elements. Programmable systolic arrays are programmable either at a high level or a low level. At either level, programmable arrays can be categorized as either SIMD or MIMD machines.<br />
<br />
[[Image:systolic_4.png|thumb|right|250px|Figure 5: General organization of MIMD programmable linear systolic arrays [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
<br />
For the latter approach, the workstation downloads a program to each MIMD (Figure 4) systolic cell. Each cell may be loaded with a different program, or all the cells in the array may be loaded with the same program. Each cell's architecture is somewhat similar to the conventional von Neumann architecture: It contains a control unit, an ALU, and local memory. MIMD systolic cells have more local memory than their SIMD counterparts to support the von Neumann-style organization.<br />
<br />
This architecture is defined as Multiple Instruction Multiple Data (MIMD) architecture in [http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf 5]. The architecture has multiple instruction streams for the PEs and a single data stream passing through all the PEs. Thus, it can also be defined as Multiple Instruction Single Data (MISD) architecture. The architecture of Systolic array configuration are controversial as explained in the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Architecture_of_systolic_arrays_as_against_MISD_architecture section 4.1.2.]<br />
<br />
======<font size="2">Reconfigurable Systolic Array</font>======<br />
[[Image:reconfig.jpg|thumb|right|250px|Figure 6: Block Diagram of the RSA Architecture [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
It is an array of systolic elements that can be programmed at the lowest level. Recent gate density advances in FPGA technology have produced a low-level, reconfigurable systolic array architecture that bridges the gap between special-purpose arrays and the more versatile, programmable general-purpose arrays. The FPGA architecture is unusual because a single hardware platform can be logically reconfigured as an exact duplicate of a special-purpose systolic array. <br />
<br />
The RSA circuit design is based on systolic array architecture consisting of PEs interconnected via SWs as depicted in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Reconfig.jpg Figure 6]. The homogeneous characteristic of the Reconfigurable Systolic Array (RSA) architecture, where each reconfigurable processing element (PE) cell is connected to its nearest neighbors via configurable switch (SW) elements, enables array expansion for parallel processing and facilitates time sharing computation of high-throughput data by individual PEs. Both the PEs and SWs can be reconfigured dynamically with the former as an arithmetic processor and the latter as a flexible router linking the neighboring PE cells. The RSA shifts reconfiguration and input signals into the PEs and SWs on separate data bus which enables the circuit to continue its operation while the reconfiguration is in process.<br />
<br />
====Architecture of systolic arrays as against MISD architecture====<br />
[[Image:comp.png|thumb|right|250px|Figure 7.Comparison between Architecture of systolic arrays and MISD]]<br />
<br />
As from the above mentioned configurations of the Systolic Arrays, it is seen that generally the configurations have multiple processing elements executing different instructions from dedicated instruction streams for each processing element. There is a single data stream that connects the adjacent PEs. Thus, systolic array can be defined as an MISD architecture.<br />
<br />
Many authors say that as the data read as input by one processing element is processed data output of the adjacent PE. The data stream cannot be considered as single because all the data paths do not carry the same data to all the PEs. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Comp.png Figure 7] shows the difference between the Data Stream for Systolic Arrays and the MISD architecture. Thus the systolic array should be considered as “Multiple Data” architecture and not Single Data architecture.<br />
<br />
===Fault Tolerant Systems<ref>http://en.wikipedia.org/wiki/Fault-tolerant_computer_system#Types_of_fault_tolerance</ref>===<br />
<br />
The fault tolerant systems are designed to handle the possible failures in software, hardware or interfaces. The hardware faults include hard disk failures, input or output device failures, etc. and the software and interface faults include driver failures; operator errors, installing unexpected software etc. The hardware faults can be detected and identified by implementing redundant hardware and multiple backups. The software faults can be tolerable by removing the program errors by executing the software redundantly or by implementing small programs that take over the tasks that crash or generate errors.<br />
<br />
[[Image:fault.png|thumb|right|250px|Figure 8 MISD as fault tolerant architecture]]<br />
<br />
In computer systems, the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Single_Instruction.2C_Multiple_Data_streams_.28SIMD.29 SIMD], [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD] and [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instruction.2C_Multiple_Data_streams_.28MIMD.29 MIMD] architectures facilitate the implementation of the fault tolerance systems by multiple instruction streams or multiple data streams or both. Fault tolerance on computations can be implemented by multiple processors (likely with different architectures) executing the algorithms on the same set of data. The output of each processor is compared with that of the others and M out of N majority voting method is used to determine the faulty processor. Thus MISD architecture is utilized to get the fault tolerance on critical computations.<br />
<br />
There are various examples of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture] being used as fault tolerant architecture. The major examples being flight control systems, nuclear power plants, satellite systems, super collider experiment systems, etc. Here, the flight control system is explained as an example of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture].<br />
<br />
====The Flight Control System – MISD Example for fault tolerance====<br />
<br />
A [http://en.wikipedia.org/wiki/Fly-by-wire Fly-By-Wire] system is used to replace the manual flight control by an electronic control interface. The movements of the flight control in the cockpit are converted to electronic signals and are transmitted to the actuators by wires. The control computers use the feedback from the sensors to compute and control the movement of the actuators to provide the expected response. These computers also perform the task to stabilize the aircraft and perform other tasks without the knowledge of the pilot. Flight control systems must meet extremely high levels of accuracy and functional integrity.<br />
<br />
There are redundant flight control computers present in the flight control system. If one of the flight-control computers crashes, gets damaged or is affected by electromagnetic pulses, the other computer can overrule the faulty one and hence the flight of the aircraft is unharmed. [[Image:fig13.png|thumb|right|250px|Figure 9: Architecture of triple redundant 777 primary flight computer [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-7 6]]]The number of redundant flight control computers is generally more than two, so that any computer whose results disagree with the others is ruled out to be faulty and is either ignored or rebooted.<br />
<br />
=====Multiple Processors Implementation in Boeing 777<ref>Yeh, Y.C.;, "Triple-triple redundant 777 primary flight computer," Aerospace Applications Conference, 1996. Proceedings., 1996 IEEE , vol.1, no., pp.293-307 vol.1, 3-10 Feb 1996<br />
doi: 10.1109/AERO.1996.495891</ref>=====<br />
<br />
In modern computers, the redundant flight control computations are carried out by multiprocessor systems. The triple redundant 777 primary flight computer, has the architecture as shown in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Fig13.png Figure 9].<br />
<br />
The system has three primary flight control computers, each of them having three lanes with different processors. The flight control program is compiled for each of the processors which get the input data from the same data bus but drive the output on their individual control bus. Thus each processor executes different instructions but they process the same data. Thus, it is the best suited example of Multiple Instruction Single Data (MISD) architecture.<br />
<br />
The three processors selected for the flight control system of [http://en.wikipedia.org/wiki/Boeing_777 Boeing 777] were [http://en.wikipedia.org/wiki/Intel_80486 Intel 80486], [http://en.wikipedia.org/wiki/Motorola_68040 Motorola 68040] and [http://en.wikipedia.org/wiki/AMD_Am29000 AMD 29050]. The dissimilar processors lead to dissimilar interface hardware circuits and compilers. Each lane of the flight control computer is data synchronized with the other lanes so that all of the lanes read the same frame of data from the flight sensors. As the outputs of each lane can be different, the median value of the outputs is used to select the output of the lane to be considered. The lane which has the median value select hardware selected is said to be in “command mode” whereas the other lanes are said to be in “monitoring mode”. It receives the data from the other Primary Flight Computer (PFC) lanes and performs a median select of the outputs. This provides a fault blocking mechanism before the fault detection and identification by the cross-lane monitoring system. Thus, the MISD based multi computer architecture is capable of detecting generic errors in compilers or in complex hardware devices providing assurance beyond reasonable doubt of the dependability of the Fly-By-Wire system.<br />
<br />
The above mentioned system clearly has individual Instruction Streams as the architecture of each processor is different, thus different instruction sets and different instruction streams. These processors have frame synchronized input data which means they have same set of data to work upon which is fed from a single data stream. Thus the flight control system can be classified under MISD architecture.<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System<ref>M.Gajer;,“ Parallel Image Processing on The TMS320C80 Multiprocessor System”, http://focus.ti.com/pdfs/univ/11-CommunicationAudioSpeech.pdf</ref>===<br />
<br />
The image pre-processing system of this Multiprocessor, the pre-image processing time needed to be made shorter. To achieve the aim of reducing the pre-processing time, different types of image processing architectures for ‘C80 were presented. For example MIMD architecture is used for image histogram equalization; SIMD architecture for median filtering; MISD architecture for directional edge detection.<br />
<br />
The MISD architecture can be explained as follows. Here, each processor performs different operations on the same data set. It is mentioned at many places that MISD architecture was defined only for the sake of completeness of classification due to difficulty in practice to find a computer based on MISD architecture<ref>TMS320C80 System Level Synopsis, Texas Instruments, 1995</ref>. But in case of Image Processing, this assumption need not be true as here making many different operations on same image is rare and hence MISD architecture is a best fit for such cases. As an example of implementation of image processing operations for the MISD architecture the edge detection operation will be considered. The edges should be detected in four different directions (north, south, west and east) by the medium of four different filter masks. Each of the parallel processors of the TMS320C80 calculates one of the directional edges. Finally one obtains four images with detected edges, each in different direction.<br />
<br />
The image processing in computer is very often a multistage process composed of many stages. For example, an algorithm detecting contours of the objects in an image. The algorithm for achieving the before mentioned is composed of four basic image processing operation. The image acquired from the camera in the first step undergoes low-pass filtering, during which it is smoothed and noise is also eliminated. In the next step image is thresholding is performed and each pixel gets black or white depending on that if its grey level is below or above threshold value. The resulting image is a binary image from which edges are detected by the usage of the Laplace filter. In the last step the image with detected contours undergoes logic filtering in the purpose of elimination of single black pixels placed on the white background. For implementing the above processes parallel, a five-stage pipeline was used. The first stage of this pipeline constitutes the master processor which accounts for control of the image acquisition process. Further pipe line stages were implemented for 4 parallel processors as mentioned in table 1.<br />
<table border=1><br />
<tr><td>Stage</td><td>Processor</td><td>Operation</td></tr><br />
<tr><td>1</td><td>MP</td><td>Image acquisition</td></tr><br />
<tr><td>2</td><td>PP0/td><td>Low-pass Filtering</td></tr><br />
<tr><td>3</td><td>PP1</td><td>Thresholding</td></tr><br />
<tr><td>4</td><td>PP2</td><td>Laplace Filtering</td></tr><br />
<tr><td>5</td><td>PP3</td><td>Logic Filtering</td></tr> <br />
</table><br />
Table 1: Pipeline structure for image edge detection[8]<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System<ref>Arne Halaas, Børge Svingen, Magnar Nedland, Pål Sætrom, Ola Snøve, Jr., and Olaf René Birkeland. 2004. A recursive MISD architecture for pattern matching. IEEE Trans. Very Large Scale Integr. Syst. 12, 7 (July 2004), 727-734. DOI=10.1109/TVLSI.2004.830918 http://dx.doi.org/10.1109/TVLSI.2004.830918</ref>===<br />
<br />
The situations where several patterns are required to be matched concurrently in data and no persistent index can be created for look-up requires the use of Online multipattern approximate searching. This algorithm requires evaluation of the hit function H(Si,P) which is the function working on strings and pattern,and requires partitioning P into smaller components and combining the individual results. A recursive approach is required and there exists a need to evaluate the Si value in parallel. Thus it helps in using an architecture with two complete binary trees which take care of the data distribution and result processing. The architecture is implemented with as many PE's that are practically possible within the implementation constraints. The data from the stream is fed to the root node of the data distribution tree and simultaneously distributed to several collection of PE's belonging to separate queries thus implementing the MISD architecture with a single data stream and multiple instruction streams.<br />
<br />
==Conclusion==<br />
The MISD architecture is one of the four types of parallel architecture described in Flynn’s Taxonomy. By definition, the MISD architecture has multiple instruction streams and a single data stream. The definition also describes this architecture as requiring that all processing elements receive the same data, but most commonly cited implementations of this class of architecture do not adhere to this detail. The result is there is debate among some experts on whether or not certain implementations are truly MISD.<br />
<br />
MISD architectures are not commonly found in industry as they tend to lack the scalability and efficiency of resource use of the MIMD and SIMD class of architectures. However, MISD architectures are efficient for solving specific classes of problems often found in research. Some examples of its use in research include image processing and systolic arrays. While rare, some implementations do exist outside of academic settings. One such example is fault tolerant systems such as the flight control system on the Boeing 777.<br />
<br />
=='''References'''==<br />
<references/><br />
<br />
=='''Glossary'''==<br />
*'''ALU''': Arithmetic Logic Unit, the control unit in a computer<br />
<br />
*'''CMU''': Carnegie Mellon University<br />
<br />
*'''Fly-By-Wire''': System that replaces the conventional manual flight controls of an aircraft with an electronic interface<br />
<br />
*'''FPGA''': Field Programmable Gate Array<br />
<br />
*'''MIMD''': Multiple Instruction Multiple Data<br />
<br />
*'''MISD''': Multiple Instruction Single Data<br />
<br />
*'''PE''': Processing Element<br />
<br />
*'''PFC''': Primary Flight Computer<br />
<br />
*'''SIMD''': Single Instruction Multiple Data<br />
<br />
*'''SISD''': Single Instruction Single Data<br />
<br />
==Quiz==<br />
<br />
'''Q1: Using the classical definition of the MISD architecture from Flynn’s Taxonomy, what is the minimum number of input streams that must be present to be classified as MISD architecture?'''<br />
<br />
A. One information stream, the data stream, the number of instruction streams do not matter.<br />
<br />
B. Two information streams, one instruction stream, one data stream.<br />
<br />
C. Three information streams, two instruction streams, one data stream.<br />
<br />
D. The number of information streams is irrelevant as per Flynn’s Taxonomy.<br />
<br />
''A1: C, Three information streams, two instruction streams, one data stream.''<br />
<br />
<br />
'''Q2: Why is the Pipeline architecture often cited as an example of MISD?'''<br />
<br />
A. The Pipeline architecture is exactly like the definition of MISD architecture from Flynn’s Taxonomy.<br />
<br />
B. The data to be processed travels linearly across all processing elements with each processing element executing a different instruction and passing it on to the next processing element. This line of data is cited to be a single “stream.”<br />
<br />
C. The data stream sends the same data to all processing elements for every cycle.<br />
<br />
D. Flynn based his description of MISD architecture on the Pipeline architecture in his paper that introduced Flynn’s Taxonomy.<br />
<br />
''A2: B, The data to be processed travels linearly across all processing elements with each processing element executing a different instruction and passing it on to the next processing element. This line of data is cited to be a single “stream.”''<br />
<br />
<br />
'''Q3: Why is the Pipeline architecture NOT considered a true MISD architecture by many experts?'''<br />
<br />
A. This is incorrect; there is consensus that the Pipeline architecture is synonymous with the MISD architecture described by Flynn’s Taxonomy.<br />
<br />
B. There is only a single, global information/control stream in the Pipeline architecture.<br />
<br />
C. The data passed to each processing element is different as it reflects the result of the processing of all the previous processing elements. Flynn’s Taxonomy describes the same data going to every processing element.<br />
<br />
D. It has been proven that it is impossible to build a MISD architecture parallel computer as described by Flynn’s Taxonomy.<br />
<br />
''A3: C, The data passed to each processing element is different as it reflects the result of the processing of all the previous processing elements. Flynn’s Taxonomy describes the same data going to every processing element.''<br />
<br />
<br />
'''Q4: True or False? MISD architecture is the most common parallel computing architecture used for general computing tasks.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A4: B, False, it is often not the best suited architecture for general computing tasks and as such not common outside of academic research.''<br />
<br />
<br />
'''Q5: When classifying a particular parallel architecture, it was noted that there was only one physical connection to provide all information streams, is it possible for this architecture to be classified as MISD architecture? (Note that there are three choices, you need to select the correct reason for the Yes/No/Maybe answer.)'''<br />
<br />
A. Yes, Flynn’s taxonomy does not require the number of physical connections in the hardware correspond to the number of logical streams.<br />
<br />
B. Maybe, we do not have enough information about the architecture to rule out the possibility of this parallel computer having a MISD architecture. We will need to see how the connections are physically laid out on the chip.<br />
<br />
C. No, there must be multiple physical information streams.<br />
<br />
''A5. A, Yes, Flynn’s taxonomy does not require the number of physical connections in the hardware correspond to the number of logical streams.''<br />
<br />
<br />
'''Q6. True or False? To be a MISD architecture, it requires that every processing element is identical.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A6. B, False, it is not required by definition of Flynn’s Taxonomy that all processing elements are identical. See example in the wiki reading about the fault tolerant flight control system on the Boeing 777.''<br />
<br />
<br />
'''Q7: Which of these is an application of the MISD architecture?'''<br />
<br />
A. Keyword search<br />
<br />
B. Controlling an assembly line.<br />
<br />
C. Home computing<br />
<br />
D. Image edge detection.<br />
<br />
''A7: D, Image edge detection.''<br />
<br />
<br />
'''Q8. True or False? The systolic array is another name for the MISD architecture.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A8. B, False, systolic arrays may be implemented using a MISD architecture but they also may be other architectures such as SIMD and MIMD.''<br />
<br />
<br />
'''Q9. True or False? MISD is the considered the easiest parallel architecture to scale.'''<br />
<br />
A. True<br />
<br />
B. False<br />
<br />
''A9. False, MIMD and SIMD are often used because of advantages in scaling.''<br />
<br />
<br />
'''Q10. Flynn’s Taxonomy relies on the number of logical information streams to classify the types of parallel architectures. What is the difference between a logical information stream and a physical information stream?'''<br />
<br />
A. A logical information stream is a connection between the ALU and the location of the data that is responsible for passing information about boolean (logical) computations, where the physical information stream it the connection between computer and other parts of the network.<br />
<br />
B. Physical information streams are determined by how the information is physically transported around chip while logical information streams are one layer of abstraction above the physical information streams. Logical streams describe conceptually how the information on the stream can be treated.<br />
<br />
C. Logical and physical information streams transport different kinds of information but are no different in implementation.<br />
<br />
D. There is no difference; the two terms can be used interchangeably.<br />
<br />
''A10. B, Physical information streams are determined by how the information is physically transported around chip while logical information streams are one layer of abstraction above the physical information streams. Logical streams describe conceptually how the information on the stream can be treated.''</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/1a_sp&diff=71798CSC/ECE 506 Spring 2013/1a sp2013-02-04T23:22:56Z<p>Schakra8: /* General-purpose systolic array */</p>
<hr />
<div>==Multiple Instruction Single Data (MISD) Architecture==<br />
<br />
Multiple Instruction Single Data, or MISD, architecture is one of the four general categories of multi-processor or multi-core architectures described by Flynn's Taxonomy.<br />
<br />
==MISD Architectures within the Context of Flynn’s Taxonomy<ref>http://en.wikipedia.org/wiki/Flynn's_taxonomy</ref><ref>http://www.phy.ornl.gov/csep/ca/node11.html</ref>==<br />
<br />
Flynn's Taxonomy categorizes parallel computing architectures based on the number of information streams that flowed into the processor.<br />
According to Flynn's Taxonomy, there are two kinds of logical streams of information that flows into a parallel computer, the Instruction Stream and the Data stream. Note that for the purposes of classification using Flynn’s Taxonomy, it does not matter whether or not the streams are physically independent of each other on the hardware, only that the streams can be logically treated as independent.<br />
Based on how many of each type of stream flows into the parallel computer (single vs. multiple), Flynn's Taxonomy divides parallel architectures into four categories. They are Single Instruction Single Data (SISD), Single Instruction Multiple Data (SIMD), Multiple Instruction Single Data (MISD), and Multiple Instruction Multiple Data (MIMD).<br />
<br />
The four classifications defined by Flynn are based upon the number of concurrent instruction (or control) and data streams available in the architecture are<ref>https://computing.llnl.gov/tutorials/parallel_comp/#Flynn</ref><br />
<br />
[[Image:Flynn's Taxonomy.PNG|thumb|center|400px|Figure 1. [http://en.wikipedia.org/wiki/Michael_J._Flynn Flynn]'s Taxonomy [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
==Overview of the MISD Architecture==<br />
<br />
In order to be categorized as MISD architecture, there must be at least two Instruction streams and there can only be one Data stream as shown on Figure 2. As per the description according to Flynn’s Taxonomy, each processing elements should receive the same data and execute different instructions on the same data simultaneously.<br />
<br />
[[Image:MISD.PNG|thumb|right|100px|Figure 2. MISD Architecture [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
There are very few implementations of the classical MISD architecture as described by Flynn’s Taxonomy, none of which are commercial, general purpose parallel computers. Instead, MISD architectures are mainly used to create custom hardware in order to solve specific problems. The majority of the implementations have been created are in academia.<br />
Over the years, several architectures have been described as MISD even though the architectures do not completely follow the description in Flynn’s Taxonomy. One common example is the Pipeline architecture where the data is passed from processing element to processing element and each processing element executes a different instruction. It is often argued that this is not a true MISD architecture as the data changes as it moves through the pipeline. Similarly, other architectures or applications exist which may be called MISD by some while others disagree. The following section looks at some of those implementations.<br />
<br />
Pipeline architectures belong to this type, though a purist might say that the data is different after processing by each stage in the pipeline. Fault-tolerant computers executing the same instructions redundantly in order to detect and mask errors, in a manner known as task replication, may be considered to belong to this type. Not many instances of this architecture exist, as MIMD and SIMD are often more appropriate for common data parallel techniques. Specifically, they allow better scaling and use of computational resources than MISD does. <br />
<br />
However, one prominent example of MISD in computing is the Space Shuttle flight control computers. Another example of this machine is the systolic array, such as the [http://www.cs.cmu.edu/~iwarp/ CMU iWrap] [BORKAR et al., 1990]. All the elements in this array are controlled by a global clock. On each cycle, an element will read a piece of data from one of its neighbors, perform a simple operation (e.g. add the incoming element to a stored value), and prepare a value to be written to a neighbor on the next step.<br />
<br />
==Implementations of MISD architecture==<br />
<br />
Some major implementations of MISD are in the fields of systolic arrays, pattern matching, flight control systems and fault tolerance systems. These MISD implementations are discussed below.<br />
<br />
===Systolic Array===<br />
<br />
A systolic array is an arrangement of processors in an array where data flows synchronously across the array between neighbors, usually with different data flowing in different directions. Each Processor at each step takes in data from one or more neighbors, processes it and, in the next step, outputs results in the opposite direction.<br />
<br />
The systolic array paradigm, data-stream-driven by data counters, is the counterpart of the [http://en.wikipedia.org/wiki/Von_Neumann_model von Neumann paradigm], instruction-stream-driven by a program counter. Because a systolic array usually sends and receives multiple data streams, and multiple data counters are needed to generate these data streams, it supports data parallelism. The name derives from analogy with the regular pumping of blood by the heart.<ref>http://en.wikipedia.org/wiki/Systolic_array</ref><br />
<br />
====Type of Systolic Arrays<ref>Johnson, K.T.; Hurson, A.R.; Shirazi, B.; , "General-purpose systolic arrays," Computer , vol.26, no.11, pp.20-31, Nov. 1993 doi: 10.1109/2.241423</ref>====<br />
<br />
=====Special-purpose systolic array=====<br />
[[Image:systolic_1.png|thumb|right|250px|Figure 3: The algorithm for the sum of a scalar product, computed in systolic element [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
[[Image:systolic_2.png|thumb|right|250px|Figure 4: The systolic product of two 3x3 matrices [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
An array of hardwired systolic processing elements tailored for a specific application. Typically, many tens or hundreds of cells fit on a single chip. One of the major applications of special-purpose systolic array is in matrix operations. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Systolic_1.png Figure 3] illustrates the algorithm for the sum of a scalar product, computed in a single systolic element. Here, a’s and b’s are synchronously shifted through the processing element to be available for next element. These data synchronously exits the processing element unmodified for the next element. The sum of the products is then shifted out of the accumulator.<br />
<br />
=====General-purpose systolic array=====<br />
<br />
It is an array of systolic processing elements, which gets adapted to a variety of applications via programming or reconfiguration. Array topologies can be either programmable or reconfigurable. Likewise, array cells are either programmable or reconfigurable. This is referred to as Systolic topologies.<br />
<br />
======<font size="2">Programmable Systolic Array</font>======<br />
<br />
It is an array of programmable systolic elements that operates either in SIMD or MIMD fashion. Either the arrays interconnect or each processing unit is programmable and a program controls dataflow through the elements. Programmable systolic arrays are programmable either at a high level or a low level. At either level, programmable arrays can be categorized as either SIMD or MIMD machines.<br />
<br />
[[Image:systolic_4.png|thumb|right|250px|Figure 5: General organization of MIMD programmable linear systolic arrays [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
<br />
For the latter approach, the workstation downloads a program to each MIMD (Figure 4) systolic cell. Each cell may be loaded with a different program, or all the cells in the array may be loaded with the same program. Each cell's architecture is somewhat similar to the conventional von Neumann architecture: It contains a control unit, an ALU, and local memory. MIMD systolic cells have more local memory than their SIMD counterparts to support the von Neumann-style organization.<br />
<br />
This architecture is defined as Multiple Instruction Multiple Data (MIMD) architecture in [http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf 5]. The architecture has multiple instruction streams for the PEs and a single data stream passing through all the PEs. Thus, it can also be defined as Multiple Instruction Single Data (MISD) architecture. The architecture of Systolic array configuration are controversial as explained in the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Architecture_of_systolic_arrays_as_against_MISD_architecture section 4.1.2.]<br />
<br />
======<font size="2">Reconfigurable Systolic Array</font>======<br />
[[Image:reconfig.jpg|thumb|right|250px|Figure 6: Block Diagram of the RSA Architecture [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
It is an array of systolic elements that can be programmed at the lowest level. Recent gate density advances in FPGA technology have produced a low-level, reconfigurable systolic array architecture that bridges the gap between special-purpose arrays and the more versatile, programmable general-purpose arrays. The FPGA architecture is unusual because a single hardware platform can be logically reconfigured as an exact duplicate of a special-purpose systolic array. <br />
<br />
The RSA circuit design is based on systolic array architecture consisting of PEs interconnected via SWs as depicted in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Reconfig.jpg Figure 6]. The homogeneous characteristic of the Reconfigurable Systolic Array (RSA) architecture, where each reconfigurable processing element (PE) cell is connected to its nearest neighbors via configurable switch (SW) elements, enables array expansion for parallel processing and facilitates time sharing computation of high-throughput data by individual PEs. Both the PEs and SWs can be reconfigured dynamically with the former as an arithmetic processor and the latter as a flexible router linking the neighboring PE cells. The RSA shifts reconfiguration and input signals into the PEs and SWs on separate data bus which enables the circuit to continue its operation while the reconfiguration is in process.<br />
<br />
====Architecture of systolic arrays as against MISD architecture====<br />
[[Image:comp.png|thumb|right|250px|Figure 7.Comparison between Architecture of systolic arrays and MISD]]<br />
<br />
As from the above mentioned configurations of the Systolic Arrays, it is seen that generally the configurations have multiple processing elements executing different instructions from dedicated instruction streams for each processing element. There is a single data stream that connects the adjacent PEs. Thus, systolic array can be defined as an MISD architecture.<br />
<br />
Many authors say that as the data read as input by one processing element is processed data output of the adjacent PE. The data stream cannot be considered as single because all the data paths do not carry the same data to all the PEs. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Comp.png Figure 7] shows the difference between the Data Stream for Systolic Arrays and the MISD architecture. Thus the systolic array should be considered as “Multiple Data” architecture and not Single Data architecture.<br />
<br />
===Fault Tolerant Systems<ref>http://en.wikipedia.org/wiki/Fault-tolerant_computer_system#Types_of_fault_tolerance</ref>===<br />
<br />
The fault tolerant systems are designed to handle the possible failures in software, hardware or interfaces. The hardware faults include hard disk failures, input or output device failures, etc. and the software and interface faults include driver failures; operator errors, installing unexpected software etc. The hardware faults can be detected and identified by implementing redundant hardware and multiple backups. The software faults can be tolerable by removing the program errors by executing the software redundantly or by implementing small programs that take over the tasks that crash or generate errors.<br />
<br />
[[Image:fault.png|thumb|right|250px|Figure 8 MISD as fault tolerant architecture]]<br />
<br />
In computer systems, the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Single_Instruction.2C_Multiple_Data_streams_.28SIMD.29 SIMD], [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD] and [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instruction.2C_Multiple_Data_streams_.28MIMD.29 MIMD] architectures facilitate the implementation of the fault tolerance systems by multiple instruction streams or multiple data streams or both. Fault tolerance on computations can be implemented by multiple processors (likely with different architectures) executing the algorithms on the same set of data. The output of each processor is compared with that of the others and M out of N majority voting method is used to determine the faulty processor. Thus MISD architecture is utilized to get the fault tolerance on critical computations.<br />
<br />
There are various examples of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture] being used as fault tolerant architecture. The major examples being flight control systems, nuclear power plants, satellite systems, super collider experiment systems, etc. Here, the flight control system is explained as an example of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture].<br />
<br />
====The Flight Control System – MISD Example for fault tolerance====<br />
<br />
A [http://en.wikipedia.org/wiki/Fly-by-wire Fly-By-Wire] system is used to replace the manual flight control by an electronic control interface. The movements of the flight control in the cockpit are converted to electronic signals and are transmitted to the actuators by wires. The control computers use the feedback from the sensors to compute and control the movement of the actuators to provide the expected response. These computers also perform the task to stabilize the aircraft and perform other tasks without the knowledge of the pilot. Flight control systems must meet extremely high levels of accuracy and functional integrity.<br />
<br />
There are redundant flight control computers present in the flight control system. If one of the flight-control computers crashes, gets damaged or is affected by electromagnetic pulses, the other computer can overrule the faulty one and hence the flight of the aircraft is unharmed. [[Image:fig13.png|thumb|right|250px|Figure 9: Architecture of triple redundant 777 primary flight computer [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-7 6]]]The number of redundant flight control computers is generally more than two, so that any computer whose results disagree with the others is ruled out to be faulty and is either ignored or rebooted.<br />
<br />
=====Multiple Processors Implementation in Boeing 777<ref>Yeh, Y.C.;, "Triple-triple redundant 777 primary flight computer," Aerospace Applications Conference, 1996. Proceedings., 1996 IEEE , vol.1, no., pp.293-307 vol.1, 3-10 Feb 1996<br />
doi: 10.1109/AERO.1996.495891</ref>=====<br />
<br />
In modern computers, the redundant flight control computations are carried out by multiprocessor systems. The triple redundant 777 primary flight computer, has the architecture as shown in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Fig13.png Figure 9].<br />
<br />
The system has three primary flight control computers, each of them having three lanes with different processors. The flight control program is compiled for each of the processors which get the input data from the same data bus but drive the output on their individual control bus. Thus each processor executes different instructions but they process the same data. Thus, it is the best suited example of Multiple Instruction Single Data (MISD) architecture.<br />
<br />
The three processors selected for the flight control system of [http://en.wikipedia.org/wiki/Boeing_777 Boeing 777] were [http://en.wikipedia.org/wiki/Intel_80486 Intel 80486], [http://en.wikipedia.org/wiki/Motorola_68040 Motorola 68040] and [http://en.wikipedia.org/wiki/AMD_Am29000 AMD 29050]. The dissimilar processors lead to dissimilar interface hardware circuits and compilers. Each lane of the flight control computer is data synchronized with the other lanes so that all of the lanes read the same frame of data from the flight sensors. As the outputs of each lane can be different, the median value of the outputs is used to select the output of the lane to be considered. The lane which has the median value select hardware selected is said to be in “command mode” whereas the other lanes are said to be in “monitoring mode”. It receives the data from the other Primary Flight Computer (PFC) lanes and performs a median select of the outputs. This provides a fault blocking mechanism before the fault detection and identification by the cross-lane monitoring system. Thus, the MISD based multi computer architecture is capable of detecting generic errors in compilers or in complex hardware devices providing assurance beyond reasonable doubt of the dependability of the Fly-By-Wire system.<br />
<br />
The above mentioned system clearly has individual Instruction Streams as the architecture of each processor is different, thus different instruction sets and different instruction streams. These processors have frame synchronized input data which means they have same set of data to work upon which is fed from a single data stream. Thus the flight control system can be classified under MISD architecture.<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System<ref>M.Gajer;,“ Parallel Image Processing on The TMS320C80 Multiprocessor System”, http://focus.ti.com/pdfs/univ/11-CommunicationAudioSpeech.pdf</ref>===<br />
<br />
The image pre-processing system of this Multiprocessor, the pre-image processing time needed to be made shorter. To achieve the aim of reducing the pre-processing time, different types of image processing architectures for ‘C80 were presented. For example MIMD architecture is used for image histogram equalization; SIMD architecture for median filtering; MISD architecture for directional edge detection.<br />
<br />
The MISD architecture can be explained as follows. Here, each processor performs different operations on the same data set. It is mentioned at many places that MISD architecture was defined only for the sake of completeness of classification due to difficulty in practice to find a computer based on MISD architecture<ref>TMS320C80 System Level Synopsis, Texas Instruments, 1995</ref>. But in case of Image Processing, this assumption need not be true as here making many different operations on same image is rare and hence MISD architecture is a best fit for such cases. As an example of implementation of image processing operations for the MISD architecture the edge detection operation will be considered. The edges should be detected in four different directions (north, south, west and east) by the medium of four different filter masks. Each of the parallel processors of the TMS320C80 calculates one of the directional edges. Finally one obtains four images with detected edges, each in different direction.<br />
<br />
The image processing in computer is very often a multistage process composed of many stages. For example, an algorithm detecting contours of the objects in an image. The algorithm for achieving the before mentioned is composed of four basic image processing operation. The image acquired from the camera in the first step undergoes low-pass filtering, during which it is smoothed and noise is also eliminated. In the next step image is thresholding is performed and each pixel gets black or white depending on that if its grey level is below or above threshold value. The resulting image is a binary image from which edges are detected by the usage of the Laplace filter. In the last step the image with detected contours undergoes logic filtering in the purpose of elimination of single black pixels placed on the white background. For implementing the above processes parallel, a five-stage pipeline was used. The first stage of this pipeline constitutes the master processor which accounts for control of the image acquisition process. Further pipe line stages were implemented for 4 parallel processors as mentioned in table 1.<br />
<table border=1><br />
<tr><td>Stage</td><td>Processor</td><td>Operation</td></tr><br />
<tr><td>1</td><td>MP</td><td>Image acquisition</td></tr><br />
<tr><td>2</td><td>PP0/td><td>Low-pass Filtering</td></tr><br />
<tr><td>3</td><td>PP1</td><td>Thresholding</td></tr><br />
<tr><td>4</td><td>PP2</td><td>Laplace Filtering</td></tr><br />
<tr><td>5</td><td>PP3</td><td>Logic Filtering</td></tr> <br />
</table><br />
Table 1: Pipeline structure for image edge detection[8]<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System<ref>Arne Halaas, Børge Svingen, Magnar Nedland, Pål Sætrom, Ola Snøve, Jr., and Olaf René Birkeland. 2004. A recursive MISD architecture for pattern matching. IEEE Trans. Very Large Scale Integr. Syst. 12, 7 (July 2004), 727-734. DOI=10.1109/TVLSI.2004.830918 http://dx.doi.org/10.1109/TVLSI.2004.830918</ref>===<br />
<br />
The situations where several patterns are required to be matched concurrently in data and no persistent index can be created for look-up requires the use of Online multipattern approximate searching. This algorithm requires evaluation of the hit function H(Si,P) which is the function working on strings and pattern,and requires partitioning P into smaller components and combining the individual results. A recursive approach is required and there exists a need to evaluate the Si value in parallel. Thus it helps in using an architecture with two complete binary trees which take care of the data distribution and result processing. The architecture is implemented with as many PE's that are practically possible within the implementation constraints. The data from the stream is fed to the root node of the data distribution tree and simultaneously distributed to several collection of PE's belonging to separate queries thus implementing the MISD architecture with a single data stream and multiple instruction streams.<br />
<br />
=='''References'''==<br />
<references/><br />
<br />
=='''Glossary'''==<br />
*'''CMU''': Carnegie Mellon University<br />
<br />
*'''CU''': Control Unit<br />
<br />
*'''DS''': Data Stream<br />
<br />
*'''Fly-By-Wire''': System that replaces the conventional manual flight controls of an aircraft with an electronic interface<br />
<br />
*'''FPGA''': Field Programmable Gate Array<br />
<br />
*'''Heterogeneous Systems''': A multiprocessor system with different kind of processors<br />
<br />
*'''Homogeneous System''': A multiprocessor system with same kind of processors<br />
<br />
*'''ILP''': Instruction Level Parallelism<br />
<br />
*'''IS''': Instruction Stream<br />
<br />
*'''MIMD''': Multiple Instruction Multiple Data<br />
<br />
*'''MISD''': Multiple Instruction Single Data<br />
<br />
*'''PE''': Processing Element<br />
<br />
*'''PFC''': Primary Flight Computer<br />
<br />
*'''SAPO''': Short for Samočinný počítač<br />
<br />
*'''SIMD''': Single Instruction Multiple Data<br />
<br />
*'''SISD''': Single Instruction Single Data<br />
<br />
*'''VLSI''': Very Large Scale Integration</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/1a_sp&diff=71796CSC/ECE 506 Spring 2013/1a sp2013-02-04T23:19:55Z<p>Schakra8: </p>
<hr />
<div>==Multiple Instruction Single Data (MISD) Architecture==<br />
<br />
Multiple Instruction Single Data, or MISD, architecture is one of the four general categories of multi-processor or multi-core architectures described by Flynn's Taxonomy.<br />
<br />
==MISD Architectures within the Context of Flynn’s Taxonomy<ref>http://en.wikipedia.org/wiki/Flynn's_taxonomy</ref><ref>http://www.phy.ornl.gov/csep/ca/node11.html</ref>==<br />
<br />
Flynn's Taxonomy categorizes parallel computing architectures based on the number of information streams that flowed into the processor.<br />
According to Flynn's Taxonomy, there are two kinds of logical streams of information that flows into a parallel computer, the Instruction Stream and the Data stream. Note that for the purposes of classification using Flynn’s Taxonomy, it does not matter whether or not the streams are physically independent of each other on the hardware, only that the streams can be logically treated as independent.<br />
Based on how many of each type of stream flows into the parallel computer (single vs. multiple), Flynn's Taxonomy divides parallel architectures into four categories. They are Single Instruction Single Data (SISD), Single Instruction Multiple Data (SIMD), Multiple Instruction Single Data (MISD), and Multiple Instruction Multiple Data (MIMD).<br />
<br />
The four classifications defined by Flynn are based upon the number of concurrent instruction (or control) and data streams available in the architecture are<ref>https://computing.llnl.gov/tutorials/parallel_comp/#Flynn</ref><br />
<br />
[[Image:Flynn's Taxonomy.PNG|thumb|center|400px|Figure 1. [http://en.wikipedia.org/wiki/Michael_J._Flynn Flynn]'s Taxonomy [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
==Overview of the MISD Architecture==<br />
<br />
In order to be categorized as MISD architecture, there must be at least two Instruction streams and there can only be one Data stream as shown on Figure 2. As per the description according to Flynn’s Taxonomy, each processing elements should receive the same data and execute different instructions on the same data simultaneously.<br />
<br />
[[Image:MISD.PNG|thumb|right|100px|Figure 2. MISD Architecture [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
There are very few implementations of the classical MISD architecture as described by Flynn’s Taxonomy, none of which are commercial, general purpose parallel computers. Instead, MISD architectures are mainly used to create custom hardware in order to solve specific problems. The majority of the implementations have been created are in academia.<br />
Over the years, several architectures have been described as MISD even though the architectures do not completely follow the description in Flynn’s Taxonomy. One common example is the Pipeline architecture where the data is passed from processing element to processing element and each processing element executes a different instruction. It is often argued that this is not a true MISD architecture as the data changes as it moves through the pipeline. Similarly, other architectures or applications exist which may be called MISD by some while others disagree. The following section looks at some of those implementations.<br />
<br />
Pipeline architectures belong to this type, though a purist might say that the data is different after processing by each stage in the pipeline. Fault-tolerant computers executing the same instructions redundantly in order to detect and mask errors, in a manner known as task replication, may be considered to belong to this type. Not many instances of this architecture exist, as MIMD and SIMD are often more appropriate for common data parallel techniques. Specifically, they allow better scaling and use of computational resources than MISD does. <br />
<br />
However, one prominent example of MISD in computing is the Space Shuttle flight control computers. Another example of this machine is the systolic array, such as the [http://www.cs.cmu.edu/~iwarp/ CMU iWrap] [BORKAR et al., 1990]. All the elements in this array are controlled by a global clock. On each cycle, an element will read a piece of data from one of its neighbors, perform a simple operation (e.g. add the incoming element to a stored value), and prepare a value to be written to a neighbor on the next step.<br />
<br />
==Implementations of MISD architecture==<br />
<br />
Some major implementations of MISD are in the fields of systolic arrays, pattern matching, flight control systems and fault tolerance systems. These MISD implementations are discussed below.<br />
<br />
===Systolic Array===<br />
<br />
A systolic array is an arrangement of processors in an array where data flows synchronously across the array between neighbors, usually with different data flowing in different directions. Each Processor at each step takes in data from one or more neighbors, processes it and, in the next step, outputs results in the opposite direction.<br />
<br />
The systolic array paradigm, data-stream-driven by data counters, is the counterpart of the [http://en.wikipedia.org/wiki/Von_Neumann_model von Neumann paradigm], instruction-stream-driven by a program counter. Because a systolic array usually sends and receives multiple data streams, and multiple data counters are needed to generate these data streams, it supports data parallelism. The name derives from analogy with the regular pumping of blood by the heart.<ref>http://en.wikipedia.org/wiki/Systolic_array</ref><br />
<br />
====Type of Systolic Arrays<ref>Johnson, K.T.; Hurson, A.R.; Shirazi, B.; , "General-purpose systolic arrays," Computer , vol.26, no.11, pp.20-31, Nov. 1993 doi: 10.1109/2.241423</ref>====<br />
<br />
=====Special-purpose systolic array=====<br />
[[Image:systolic_1.png|thumb|right|250px|Figure 3: The algorithm for the sum of a scalar product, computed in systolic element [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
[[Image:systolic_2.png|thumb|right|250px|Figure 4: The systolic product of two 3x3 matrices [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
An array of hardwired systolic processing elements tailored for a specific application. Typically, many tens or hundreds of cells fit on a single chip. One of the major applications of special-purpose systolic array is in matrix operations. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Systolic_1.png Figure 3] illustrates the algorithm for the sum of a scalar product, computed in a single systolic element. Here, a’s and b’s are synchronously shifted through the processing element to be available for next element. These data synchronously exits the processing element unmodified for the next element. The sum of the products is then shifted out of the accumulator.<br />
<br />
=====General-purpose systolic array=====<br />
<br />
It is an array of systolic processing elements, which gets adapted to a variety of applications via programming or reconfiguration. Array topologies can be either programmable or reconfigurable. Likewise, array cells are either programmable or reconfigurable. This is referred to as Systolic topologies.<br />
<br />
======Programmable Systolic Array======<br />
<br />
It is an array of programmable systolic elements that operates either in SIMD or MIMD fashion. Either the arrays interconnect or each processing unit is programmable and a program controls dataflow through the elements. Programmable systolic arrays are programmable either at a high level or a low level. At either level, programmable arrays can be categorized as either SIMD or MIMD machines.<br />
<br />
[[Image:systolic_4.png|thumb|right|250px|Figure 5: General organization of MIMD programmable linear systolic arrays [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
<br />
For the latter approach, the workstation downloads a program to each MIMD (Figure 4) systolic cell. Each cell may be loaded with a different program, or all the cells in the array may be loaded with the same program. Each cell's architecture is somewhat similar to the conventional von Neumann architecture: It contains a control unit, an ALU, and local memory. MIMD systolic cells have more local memory than their SIMD counterparts to support the von Neumann-style organization.<br />
<br />
This architecture is defined as Multiple Instruction Multiple Data (MIMD) architecture in [http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf 5]. The architecture has multiple instruction streams for the PEs and a single data stream passing through all the PEs. Thus, it can also be defined as Multiple Instruction Single Data (MISD) architecture. The architecture of Systolic array configuration are controversial as explained in the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Architecture_of_systolic_arrays_as_against_MISD_architecture section 4.1.2.]<br />
<br />
======Reconfigurable Systolic Array======<br />
[[Image:reconfig.jpg|thumb|right|250px|Figure 6: Block Diagram of the RSA Architecture [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
It is an array of systolic elements that can be programmed at the lowest level. Recent gate density advances in FPGA technology have produced a low-level, reconfigurable systolic array architecture that bridges the gap between special-purpose arrays and the more versatile, programmable general-purpose arrays. The FPGA architecture is unusual because a single hardware platform can be logically reconfigured as an exact duplicate of a special-purpose systolic array. <br />
<br />
The RSA circuit design is based on systolic array architecture consisting of PEs interconnected via SWs as depicted in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Reconfig.jpg Figure 6]. The homogeneous characteristic of the Reconfigurable Systolic Array (RSA) architecture, where each reconfigurable processing element (PE) cell is connected to its nearest neighbors via configurable switch (SW) elements, enables array expansion for parallel processing and facilitates time sharing computation of high-throughput data by individual PEs. Both the PEs and SWs can be reconfigured dynamically with the former as an arithmetic processor and the latter as a flexible router linking the neighboring PE cells. The RSA shifts reconfiguration and input signals into the PEs and SWs on separate data bus which enables the circuit to continue its operation while the reconfiguration is in process.<br />
<br />
====Architecture of systolic arrays as against MISD architecture====<br />
[[Image:comp.png|thumb|right|250px|Figure 7.Comparison between Architecture of systolic arrays and MISD]]<br />
<br />
As from the above mentioned configurations of the Systolic Arrays, it is seen that generally the configurations have multiple processing elements executing different instructions from dedicated instruction streams for each processing element. There is a single data stream that connects the adjacent PEs. Thus, systolic array can be defined as an MISD architecture.<br />
<br />
Many authors say that as the data read as input by one processing element is processed data output of the adjacent PE. The data stream cannot be considered as single because all the data paths do not carry the same data to all the PEs. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Comp.png Figure 7] shows the difference between the Data Stream for Systolic Arrays and the MISD architecture. Thus the systolic array should be considered as “Multiple Data” architecture and not Single Data architecture.<br />
<br />
===Fault Tolerant Systems<ref>http://en.wikipedia.org/wiki/Fault-tolerant_computer_system#Types_of_fault_tolerance</ref>===<br />
<br />
The fault tolerant systems are designed to handle the possible failures in software, hardware or interfaces. The hardware faults include hard disk failures, input or output device failures, etc. and the software and interface faults include driver failures; operator errors, installing unexpected software etc. The hardware faults can be detected and identified by implementing redundant hardware and multiple backups. The software faults can be tolerable by removing the program errors by executing the software redundantly or by implementing small programs that take over the tasks that crash or generate errors.<br />
<br />
[[Image:fault.png|thumb|right|250px|Figure 8 MISD as fault tolerant architecture]]<br />
<br />
In computer systems, the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Single_Instruction.2C_Multiple_Data_streams_.28SIMD.29 SIMD], [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD] and [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instruction.2C_Multiple_Data_streams_.28MIMD.29 MIMD] architectures facilitate the implementation of the fault tolerance systems by multiple instruction streams or multiple data streams or both. Fault tolerance on computations can be implemented by multiple processors (likely with different architectures) executing the algorithms on the same set of data. The output of each processor is compared with that of the others and M out of N majority voting method is used to determine the faulty processor. Thus MISD architecture is utilized to get the fault tolerance on critical computations.<br />
<br />
There are various examples of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture] being used as fault tolerant architecture. The major examples being flight control systems, nuclear power plants, satellite systems, super collider experiment systems, etc. Here, the flight control system is explained as an example of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture].<br />
<br />
====The Flight Control System – MISD Example for fault tolerance====<br />
<br />
A [http://en.wikipedia.org/wiki/Fly-by-wire Fly-By-Wire] system is used to replace the manual flight control by an electronic control interface. The movements of the flight control in the cockpit are converted to electronic signals and are transmitted to the actuators by wires. The control computers use the feedback from the sensors to compute and control the movement of the actuators to provide the expected response. These computers also perform the task to stabilize the aircraft and perform other tasks without the knowledge of the pilot. Flight control systems must meet extremely high levels of accuracy and functional integrity.<br />
<br />
There are redundant flight control computers present in the flight control system. If one of the flight-control computers crashes, gets damaged or is affected by electromagnetic pulses, the other computer can overrule the faulty one and hence the flight of the aircraft is unharmed. [[Image:fig13.png|thumb|right|250px|Figure 9: Architecture of triple redundant 777 primary flight computer [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-7 6]]]The number of redundant flight control computers is generally more than two, so that any computer whose results disagree with the others is ruled out to be faulty and is either ignored or rebooted.<br />
<br />
=====Multiple Processors Implementation in Boeing 777<ref>Yeh, Y.C.;, "Triple-triple redundant 777 primary flight computer," Aerospace Applications Conference, 1996. Proceedings., 1996 IEEE , vol.1, no., pp.293-307 vol.1, 3-10 Feb 1996<br />
doi: 10.1109/AERO.1996.495891</ref>=====<br />
<br />
In modern computers, the redundant flight control computations are carried out by multiprocessor systems. The triple redundant 777 primary flight computer, has the architecture as shown in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Fig13.png Figure 9].<br />
<br />
The system has three primary flight control computers, each of them having three lanes with different processors. The flight control program is compiled for each of the processors which get the input data from the same data bus but drive the output on their individual control bus. Thus each processor executes different instructions but they process the same data. Thus, it is the best suited example of Multiple Instruction Single Data (MISD) architecture.<br />
<br />
The three processors selected for the flight control system of [http://en.wikipedia.org/wiki/Boeing_777 Boeing 777] were [http://en.wikipedia.org/wiki/Intel_80486 Intel 80486], [http://en.wikipedia.org/wiki/Motorola_68040 Motorola 68040] and [http://en.wikipedia.org/wiki/AMD_Am29000 AMD 29050]. The dissimilar processors lead to dissimilar interface hardware circuits and compilers. Each lane of the flight control computer is data synchronized with the other lanes so that all of the lanes read the same frame of data from the flight sensors. As the outputs of each lane can be different, the median value of the outputs is used to select the output of the lane to be considered. The lane which has the median value select hardware selected is said to be in “command mode” whereas the other lanes are said to be in “monitoring mode”. It receives the data from the other Primary Flight Computer (PFC) lanes and performs a median select of the outputs. This provides a fault blocking mechanism before the fault detection and identification by the cross-lane monitoring system. Thus, the MISD based multi computer architecture is capable of detecting generic errors in compilers or in complex hardware devices providing assurance beyond reasonable doubt of the dependability of the Fly-By-Wire system.<br />
<br />
The above mentioned system clearly has individual Instruction Streams as the architecture of each processor is different, thus different instruction sets and different instruction streams. These processors have frame synchronized input data which means they have same set of data to work upon which is fed from a single data stream. Thus the flight control system can be classified under MISD architecture.<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System<ref>M.Gajer;,“ Parallel Image Processing on The TMS320C80 Multiprocessor System”, http://focus.ti.com/pdfs/univ/11-CommunicationAudioSpeech.pdf</ref>===<br />
<br />
The image pre-processing system of this Multiprocessor, the pre-image processing time needed to be made shorter. To achieve the aim of reducing the pre-processing time, different types of image processing architectures for ‘C80 were presented. For example MIMD architecture is used for image histogram equalization; SIMD architecture for median filtering; MISD architecture for directional edge detection.<br />
<br />
The MISD architecture can be explained as follows. Here, each processor performs different operations on the same data set. It is mentioned at many places that MISD architecture was defined only for the sake of completeness of classification due to difficulty in practice to find a computer based on MISD architecture<ref>TMS320C80 System Level Synopsis, Texas Instruments, 1995</ref>. But in case of Image Processing, this assumption need not be true as here making many different operations on same image is rare and hence MISD architecture is a best fit for such cases. As an example of implementation of image processing operations for the MISD architecture the edge detection operation will be considered. The edges should be detected in four different directions (north, south, west and east) by the medium of four different filter masks. Each of the parallel processors of the TMS320C80 calculates one of the directional edges. Finally one obtains four images with detected edges, each in different direction.<br />
<br />
The image processing in computer is very often a multistage process composed of many stages. For example, an algorithm detecting contours of the objects in an image. The algorithm for achieving the before mentioned is composed of four basic image processing operation. The image acquired from the camera in the first step undergoes low-pass filtering, during which it is smoothed and noise is also eliminated. In the next step image is thresholding is performed and each pixel gets black or white depending on that if its grey level is below or above threshold value. The resulting image is a binary image from which edges are detected by the usage of the Laplace filter. In the last step the image with detected contours undergoes logic filtering in the purpose of elimination of single black pixels placed on the white background. For implementing the above processes parallel, a five-stage pipeline was used. The first stage of this pipeline constitutes the master processor which accounts for control of the image acquisition process. Further pipe line stages were implemented for 4 parallel processors as mentioned in table 1.<br />
<table border=1><br />
<tr><td>Stage</td><td>Processor</td><td>Operation</td></tr><br />
<tr><td>1</td><td>MP</td><td>Image acquisition</td></tr><br />
<tr><td>2</td><td>PP0/td><td>Low-pass Filtering</td></tr><br />
<tr><td>3</td><td>PP1</td><td>Thresholding</td></tr><br />
<tr><td>4</td><td>PP2</td><td>Laplace Filtering</td></tr><br />
<tr><td>5</td><td>PP3</td><td>Logic Filtering</td></tr> <br />
</table><br />
Table 1: Pipeline structure for image edge detection[8]<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System<ref>Arne Halaas, Børge Svingen, Magnar Nedland, Pål Sætrom, Ola Snøve, Jr., and Olaf René Birkeland. 2004. A recursive MISD architecture for pattern matching. IEEE Trans. Very Large Scale Integr. Syst. 12, 7 (July 2004), 727-734. DOI=10.1109/TVLSI.2004.830918 http://dx.doi.org/10.1109/TVLSI.2004.830918</ref>===<br />
<br />
The situations where several patterns are required to be matched concurrently in data and no persistent index can be created for look-up requires the use of Online multipattern approximate searching. This algorithm requires evaluation of the hit function H(Si,P) which is the function working on strings and pattern,and requires partitioning P into smaller components and combining the individual results. A recursive approach is required and there exists a need to evaluate the Si value in parallel. Thus it helps in using an architecture with two complete binary trees which take care of the data distribution and result processing. The architecture is implemented with as many PE's that are practically possible within the implementation constraints. The data from the stream is fed to the root node of the data distribution tree and simultaneously distributed to several collection of PE's belonging to separate queries thus implementing the MISD architecture with a single data stream and multiple instruction streams.<br />
<br />
=='''References'''==<br />
<references/><br />
<br />
=='''Glossary'''==<br />
*'''CMU''': Carnegie Mellon University<br />
<br />
*'''CU''': Control Unit<br />
<br />
*'''DS''': Data Stream<br />
<br />
*'''Fly-By-Wire''': System that replaces the conventional manual flight controls of an aircraft with an electronic interface<br />
<br />
*'''FPGA''': Field Programmable Gate Array<br />
<br />
*'''Heterogeneous Systems''': A multiprocessor system with different kind of processors<br />
<br />
*'''Homogeneous System''': A multiprocessor system with same kind of processors<br />
<br />
*'''ILP''': Instruction Level Parallelism<br />
<br />
*'''IS''': Instruction Stream<br />
<br />
*'''MIMD''': Multiple Instruction Multiple Data<br />
<br />
*'''MISD''': Multiple Instruction Single Data<br />
<br />
*'''PE''': Processing Element<br />
<br />
*'''PFC''': Primary Flight Computer<br />
<br />
*'''SAPO''': Short for Samočinný počítač<br />
<br />
*'''SIMD''': Single Instruction Multiple Data<br />
<br />
*'''SISD''': Single Instruction Single Data<br />
<br />
*'''VLSI''': Very Large Scale Integration</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/1a_sp&diff=71794CSC/ECE 506 Spring 2013/1a sp2013-02-04T23:14:44Z<p>Schakra8: /* Edge Detection of Images on TMS320C80 Multiprocessor System */</p>
<hr />
<div>==Multiple Instruction Single Data (MISD) Architecture==<br />
<br />
Multiple Instruction Single Data, or MISD, architecture is one of the four general categories of multi-processor or multi-core architectures described by Flynn's Taxonomy.<br />
<br />
==MISD Architectures within the Context of Flynn’s Taxonomy<ref>http://en.wikipedia.org/wiki/Flynn's_taxonomy</ref><ref>http://www.phy.ornl.gov/csep/ca/node11.html</ref>==<br />
<br />
Flynn's Taxonomy categorizes parallel computing architectures based on the number of information streams that flowed into the processor.<br />
According to Flynn's Taxonomy, there are two kinds of logical streams of information that flows into a parallel computer, the Instruction Stream and the Data stream. Note that for the purposes of classification using Flynn’s Taxonomy, it does not matter whether or not the streams are physically independent of each other on the hardware, only that the streams can be logically treated as independent.<br />
Based on how many of each type of stream flows into the parallel computer (single vs. multiple), Flynn's Taxonomy divides parallel architectures into four categories. They are Single Instruction Single Data (SISD), Single Instruction Multiple Data (SIMD), Multiple Instruction Single Data (MISD), and Multiple Instruction Multiple Data (MIMD).<br />
<br />
The four classifications defined by Flynn are based upon the number of concurrent instruction (or control) and data streams available in the architecture are<ref>https://computing.llnl.gov/tutorials/parallel_comp/#Flynn</ref><br />
<br />
[[Image:Flynn's Taxonomy.PNG|thumb|center|400px|Figure 1. [http://en.wikipedia.org/wiki/Michael_J._Flynn Flynn]'s Taxonomy [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
==Overview of the MISD Architecture==<br />
<br />
In order to be categorized as MISD architecture, there must be at least two Instruction streams and there can only be one Data stream as shown on Figure 2. As per the description according to Flynn’s Taxonomy, each processing elements should receive the same data and execute different instructions on the same data simultaneously.<br />
<br />
[[Image:MISD.PNG|thumb|right|100px|Figure 2. MISD Architecture [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
There are very few implementations of the classical MISD architecture as described by Flynn’s Taxonomy, none of which are commercial, general purpose parallel computers. Instead, MISD architectures are mainly used to create custom hardware in order to solve specific problems. The majority of the implementations have been created are in academia.<br />
Over the years, several architectures have been described as MISD even though the architectures do not completely follow the description in Flynn’s Taxonomy. One common example is the Pipeline architecture where the data is passed from processing element to processing element and each processing element executes a different instruction. It is often argued that this is not a true MISD architecture as the data changes as it moves through the pipeline. Similarly, other architectures or applications exist which may be called MISD by some while others disagree. The following section looks at some of those implementations.<br />
<br />
Pipeline architectures belong to this type, though a purist might say that the data is different after processing by each stage in the pipeline. Fault-tolerant computers executing the same instructions redundantly in order to detect and mask errors, in a manner known as task replication, may be considered to belong to this type. Not many instances of this architecture exist, as MIMD and SIMD are often more appropriate for common data parallel techniques. Specifically, they allow better scaling and use of computational resources than MISD does. <br />
<br />
However, one prominent example of MISD in computing is the Space Shuttle flight control computers. Another example of this machine is the systolic array, such as the [http://www.cs.cmu.edu/~iwarp/ CMU iWrap] [BORKAR et al., 1990]. All the elements in this array are controlled by a global clock. On each cycle, an element will read a piece of data from one of its neighbors, perform a simple operation (e.g. add the incoming element to a stored value), and prepare a value to be written to a neighbor on the next step.<br />
<br />
==Implementations of MISD architecture==<br />
<br />
Some major implementations of MISD are in the fields of systolic arrays, pattern matching, flight control systems and fault tolerance systems. These MISD implementations are discussed below.<br />
<br />
===Systolic Array===<br />
<br />
A systolic array is an arrangement of processors in an array where data flows synchronously across the array between neighbors, usually with different data flowing in different directions. Each Processor at each step takes in data from one or more neighbors, processes it and, in the next step, outputs results in the opposite direction.<br />
<br />
The systolic array paradigm, data-stream-driven by data counters, is the counterpart of the [http://en.wikipedia.org/wiki/Von_Neumann_model von Neumann paradigm], instruction-stream-driven by a program counter. Because a systolic array usually sends and receives multiple data streams, and multiple data counters are needed to generate these data streams, it supports data parallelism. The name derives from analogy with the regular pumping of blood by the heart.<ref>http://en.wikipedia.org/wiki/Systolic_array</ref><br />
<br />
====Type of Systolic Arrays<ref>Johnson, K.T.; Hurson, A.R.; Shirazi, B.; , "General-purpose systolic arrays," Computer , vol.26, no.11, pp.20-31, Nov. 1993 doi: 10.1109/2.241423</ref>====<br />
<br />
=====Special-purpose systolic array=====<br />
[[Image:systolic_1.png|thumb|right|250px|Figure 3: The algorithm for the sum of a scalar product, computed in systolic element [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
[[Image:systolic_2.png|thumb|right|250px|Figure 4: The systolic product of two 3x3 matrices [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
An array of hardwired systolic processing elements tailored for a specific application. Typically, many tens or hundreds of cells fit on a single chip. One of the major applications of special-purpose systolic array is in matrix operations. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Systolic_1.png Figure 3] illustrates the algorithm for the sum of a scalar product, computed in a single systolic element. Here, a’s and b’s are synchronously shifted through the processing element to be available for next element. These data synchronously exits the processing element unmodified for the next element. The sum of the products is then shifted out of the accumulator.<br />
<br />
=====General-purpose systolic array=====<br />
<br />
It is an array of systolic processing elements, which gets adapted to a variety of applications via programming or reconfiguration. Array topologies can be either programmable or reconfigurable. Likewise, array cells are either programmable or reconfigurable. This is referred to as Systolic topologies.<br />
<br />
======Programmable Systolic Array======<br />
<br />
It is an array of programmable systolic elements that operates either in SIMD or MIMD fashion. Either the arrays interconnect or each processing unit is programmable and a program controls dataflow through the elements. Programmable systolic arrays are programmable either at a high level or a low level. At either level, programmable arrays can be categorized as either SIMD or MIMD machines.<br />
<br />
[[Image:systolic_4.png|thumb|right|250px|Figure 5: General organization of MIMD programmable linear systolic arrays [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
<br />
For the latter approach, the workstation downloads a program to each MIMD (Figure 4) systolic cell. Each cell may be loaded with a different program, or all the cells in the array may be loaded with the same program. Each cell's architecture is somewhat similar to the conventional von Neumann architecture: It contains a control unit, an ALU, and local memory. MIMD systolic cells have more local memory than their SIMD counterparts to support the von Neumann-style organization.<br />
<br />
This architecture is defined as Multiple Instruction Multiple Data (MIMD) architecture in [http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf 5]. The architecture has multiple instruction streams for the PEs and a single data stream passing through all the PEs. Thus, it can also be defined as Multiple Instruction Single Data (MISD) architecture. The architecture of Systolic array configuration are controversial as explained in the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Architecture_of_systolic_arrays_as_against_MISD_architecture section 4.1.2.]<br />
<br />
======Reconfigurable Systolic Array======<br />
[[Image:reconfig.jpg|thumb|right|250px|Figure 6: Block Diagram of the RSA Architecture [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
It is an array of systolic elements that can be programmed at the lowest level. Recent gate density advances in FPGA technology have produced a low-level, reconfigurable systolic array architecture that bridges the gap between special-purpose arrays and the more versatile, programmable general-purpose arrays. The FPGA architecture is unusual because a single hardware platform can be logically reconfigured as an exact duplicate of a special-purpose systolic array. <br />
<br />
The RSA circuit design is based on systolic array architecture consisting of PEs interconnected via SWs as depicted in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Reconfig.jpg Figure 6]. The homogeneous characteristic of the Reconfigurable Systolic Array (RSA) architecture, where each reconfigurable processing element (PE) cell is connected to its nearest neighbors via configurable switch (SW) elements, enables array expansion for parallel processing and facilitates time sharing computation of high-throughput data by individual PEs. Both the PEs and SWs can be reconfigured dynamically with the former as an arithmetic processor and the latter as a flexible router linking the neighboring PE cells. The RSA shifts reconfiguration and input signals into the PEs and SWs on separate data bus which enables the circuit to continue its operation while the reconfiguration is in process.<br />
<br />
====Architecture of systolic arrays as against MISD architecture====<br />
[[Image:comp.png|thumb|right|250px|Figure 7.Comparison between Architecture of systolic arrays and MISD]]<br />
<br />
As from the above mentioned configurations of the Systolic Arrays, it is seen that generally the configurations have multiple processing elements executing different instructions from dedicated instruction streams for each processing element. There is a single data stream that connects the adjacent PEs. Thus, systolic array can be defined as an MISD architecture.<br />
<br />
Many authors say that as the data read as input by one processing element is processed data output of the adjacent PE. The data stream cannot be considered as single because all the data paths do not carry the same data to all the PEs. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Comp.png Figure 7] shows the difference between the Data Stream for Systolic Arrays and the MISD architecture. Thus the systolic array should be considered as “Multiple Data” architecture and not Single Data architecture.<br />
<br />
===Fault Tolerant Systems<ref>http://en.wikipedia.org/wiki/Fault-tolerant_computer_system#Types_of_fault_tolerance</ref>===<br />
<br />
The fault tolerant systems are designed to handle the possible failures in software, hardware or interfaces. The hardware faults include hard disk failures, input or output device failures, etc. and the software and interface faults include driver failures; operator errors, installing unexpected software etc. The hardware faults can be detected and identified by implementing redundant hardware and multiple backups. The software faults can be tolerable by removing the program errors by executing the software redundantly or by implementing small programs that take over the tasks that crash or generate errors.<br />
<br />
[[Image:fault.png|thumb|right|250px|Figure 8 MISD as fault tolerant architecture]]<br />
<br />
In computer systems, the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Single_Instruction.2C_Multiple_Data_streams_.28SIMD.29 SIMD], [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD] and [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instruction.2C_Multiple_Data_streams_.28MIMD.29 MIMD] architectures facilitate the implementation of the fault tolerance systems by multiple instruction streams or multiple data streams or both. Fault tolerance on computations can be implemented by multiple processors (likely with different architectures) executing the algorithms on the same set of data. The output of each processor is compared with that of the others and M out of N majority voting method is used to determine the faulty processor. Thus MISD architecture is utilized to get the fault tolerance on critical computations.<br />
<br />
There are various examples of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture] being used as fault tolerant architecture. The major examples being flight control systems, nuclear power plants, satellite systems, super collider experiment systems, etc. Here, the flight control system is explained as an example of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture].<br />
<br />
====The Flight Control System – MISD Example for fault tolerance====<br />
<br />
A [http://en.wikipedia.org/wiki/Fly-by-wire Fly-By-Wire] system is used to replace the manual flight control by an electronic control interface. The movements of the flight control in the cockpit are converted to electronic signals and are transmitted to the actuators by wires. The control computers use the feedback from the sensors to compute and control the movement of the actuators to provide the expected response. These computers also perform the task to stabilize the aircraft and perform other tasks without the knowledge of the pilot. Flight control systems must meet extremely high levels of accuracy and functional integrity.<br />
<br />
There are redundant flight control computers present in the flight control system. If one of the flight-control computers crashes, gets damaged or is affected by electromagnetic pulses, the other computer can overrule the faulty one and hence the flight of the aircraft is unharmed. [[Image:fig13.png|thumb|right|250px|Figure 9: Architecture of triple redundant 777 primary flight computer [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-7 6]]]The number of redundant flight control computers is generally more than two, so that any computer whose results disagree with the others is ruled out to be faulty and is either ignored or rebooted.<br />
<br />
=====Multiple Processors Implementation in Boeing 777<ref>Yeh, Y.C.;, "Triple-triple redundant 777 primary flight computer," Aerospace Applications Conference, 1996. Proceedings., 1996 IEEE , vol.1, no., pp.293-307 vol.1, 3-10 Feb 1996<br />
doi: 10.1109/AERO.1996.495891</ref>=====<br />
<br />
In modern computers, the redundant flight control computations are carried out by multiprocessor systems. The triple redundant 777 primary flight computer, has the architecture as shown in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Fig13.png Figure 9].<br />
<br />
The system has three primary flight control computers, each of them having three lanes with different processors. The flight control program is compiled for each of the processors which get the input data from the same data bus but drive the output on their individual control bus. Thus each processor executes different instructions but they process the same data. Thus, it is the best suited example of Multiple Instruction Single Data (MISD) architecture.<br />
<br />
The three processors selected for the flight control system of [http://en.wikipedia.org/wiki/Boeing_777 Boeing 777] were [http://en.wikipedia.org/wiki/Intel_80486 Intel 80486], [http://en.wikipedia.org/wiki/Motorola_68040 Motorola 68040] and [http://en.wikipedia.org/wiki/AMD_Am29000 AMD 29050]. The dissimilar processors lead to dissimilar interface hardware circuits and compilers. Each lane of the flight control computer is data synchronized with the other lanes so that all of the lanes read the same frame of data from the flight sensors. As the outputs of each lane can be different, the median value of the outputs is used to select the output of the lane to be considered. The lane which has the median value select hardware selected is said to be in “command mode” whereas the other lanes are said to be in “monitoring mode”. It receives the data from the other Primary Flight Computer (PFC) lanes and performs a median select of the outputs. This provides a fault blocking mechanism before the fault detection and identification by the cross-lane monitoring system. Thus, the MISD based multi computer architecture is capable of detecting generic errors in compilers or in complex hardware devices providing assurance beyond reasonable doubt of the dependability of the Fly-By-Wire system.<br />
<br />
The above mentioned system clearly has individual Instruction Streams as the architecture of each processor is different, thus different instruction sets and different instruction streams. These processors have frame synchronized input data which means they have same set of data to work upon which is fed from a single data stream. Thus the flight control system can be classified under MISD architecture.<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System===<br />
<br />
The image pre-processing system of this Multiprocessor <ref>M.Gajer;,“ Parallel Image Processing on The TMS320C80 Multiprocessor System”, http://focus.ti.com/pdfs/univ/11-CommunicationAudioSpeech.pdf</ref> , the pre-image processing time needed to be made shorter. To achieve the aim of reducing the pre-processing time, different types of image processing architectures for ‘C80 were presented. For example MIMD architecture is used for image histogram equalization; SIMD architecture for median filtering; MISD architecture for directional edge detection.<br />
<br />
The MISD architecture can be explained as follows. Here, each processor performs different operations on the same data set. It is mentioned at many places that MISD architecture was defined only for the sake of completeness of classification due to difficulty in practice to find a computer based on MISD architecture<ref>TMS320C80 System Level Synopsis, Texas Instruments, 1995</ref>. But in case of Image Processing, this assumption need not be true as here making many different operations on same image is rare and hence MISD architecture is a best fit for such cases. As an example of implementation of image processing operations for the MISD architecture the edge detection operation will be considered. The edges should be detected in four different directions (north, south, west and east) by the medium of four different filter masks. Each of the parallel processors of the TMS320C80 calculates one of the directional edges. Finally one obtains four images with detected edges, each in different direction.<br />
<br />
The image processing in computer is very often a multistage process composed of many stages. For example, an algorithm detecting contours of the objects in an image. The algorithm for achieving the before mentioned is composed of four basic image processing operation. The image acquired from the camera in the first step undergoes low-pass filtering, during which it is smoothed and noise is also eliminated. In the next step image is thresholding is performed and each pixel gets black or white depending on that if its grey level is below or above threshold value. The resulting image is a binary image from which edges are detected by the usage of the Laplace filter. In the last step the image with detected contours undergoes logic filtering in the purpose of elimination of single black pixels placed on the white background. For implementing the above processes parallel, a five-stage pipeline was used. The first stage of this pipeline constitutes the master processor which accounts for control of the image acquisition process. Further pipe line stages were implemented for 4 parallel processors as mentioned in table 1.<br />
<table border=1><br />
<tr><td>Stage</td><td>Processor</td><td>Operation</td></tr><br />
<tr><td>1</td><td>MP</td><td>Image acquisition</td></tr><br />
<tr><td>2</td><td>PP0/td><td>Low-pass Filtering</td></tr><br />
<tr><td>3</td><td>PP1</td><td>Thresholding</td></tr><br />
<tr><td>4</td><td>PP2</td><td>Laplace Filtering</td></tr><br />
<tr><td>5</td><td>PP3</td><td>Logic Filtering</td></tr> <br />
</table><br />
Table 1: Pipeline structure for image edge detection[8]<br />
<br />
=='''References'''==<br />
<references/><br />
<br />
=='''Glossary'''==<br />
*'''CMU''': Carnegie Mellon University<br />
<br />
*'''CU''': Control Unit<br />
<br />
*'''DS''': Data Stream<br />
<br />
*'''Fly-By-Wire''': System that replaces the conventional manual flight controls of an aircraft with an electronic interface<br />
<br />
*'''FPGA''': Field Programmable Gate Array<br />
<br />
*'''Heterogeneous Systems''': A multiprocessor system with different kind of processors<br />
<br />
*'''Homogeneous System''': A multiprocessor system with same kind of processors<br />
<br />
*'''ILP''': Instruction Level Parallelism<br />
<br />
*'''IS''': Instruction Stream<br />
<br />
*'''MIMD''': Multiple Instruction Multiple Data<br />
<br />
*'''MISD''': Multiple Instruction Single Data<br />
<br />
*'''PE''': Processing Element<br />
<br />
*'''PFC''': Primary Flight Computer<br />
<br />
*'''SAPO''': Short for Samočinný počítač<br />
<br />
*'''SIMD''': Single Instruction Multiple Data<br />
<br />
*'''SISD''': Single Instruction Single Data<br />
<br />
*'''VLSI''': Very Large Scale Integration</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/1a_sp&diff=71793CSC/ECE 506 Spring 2013/1a sp2013-02-04T23:13:03Z<p>Schakra8: </p>
<hr />
<div>==Multiple Instruction Single Data (MISD) Architecture==<br />
<br />
Multiple Instruction Single Data, or MISD, architecture is one of the four general categories of multi-processor or multi-core architectures described by Flynn's Taxonomy.<br />
<br />
==MISD Architectures within the Context of Flynn’s Taxonomy<ref>http://en.wikipedia.org/wiki/Flynn's_taxonomy</ref><ref>http://www.phy.ornl.gov/csep/ca/node11.html</ref>==<br />
<br />
Flynn's Taxonomy categorizes parallel computing architectures based on the number of information streams that flowed into the processor.<br />
According to Flynn's Taxonomy, there are two kinds of logical streams of information that flows into a parallel computer, the Instruction Stream and the Data stream. Note that for the purposes of classification using Flynn’s Taxonomy, it does not matter whether or not the streams are physically independent of each other on the hardware, only that the streams can be logically treated as independent.<br />
Based on how many of each type of stream flows into the parallel computer (single vs. multiple), Flynn's Taxonomy divides parallel architectures into four categories. They are Single Instruction Single Data (SISD), Single Instruction Multiple Data (SIMD), Multiple Instruction Single Data (MISD), and Multiple Instruction Multiple Data (MIMD).<br />
<br />
The four classifications defined by Flynn are based upon the number of concurrent instruction (or control) and data streams available in the architecture are<ref>https://computing.llnl.gov/tutorials/parallel_comp/#Flynn</ref><br />
<br />
[[Image:Flynn's Taxonomy.PNG|thumb|center|400px|Figure 1. [http://en.wikipedia.org/wiki/Michael_J._Flynn Flynn]'s Taxonomy [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
==Overview of the MISD Architecture==<br />
<br />
In order to be categorized as MISD architecture, there must be at least two Instruction streams and there can only be one Data stream as shown on Figure 2. As per the description according to Flynn’s Taxonomy, each processing elements should receive the same data and execute different instructions on the same data simultaneously.<br />
<br />
[[Image:MISD.PNG|thumb|right|100px|Figure 2. MISD Architecture [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
There are very few implementations of the classical MISD architecture as described by Flynn’s Taxonomy, none of which are commercial, general purpose parallel computers. Instead, MISD architectures are mainly used to create custom hardware in order to solve specific problems. The majority of the implementations have been created are in academia.<br />
Over the years, several architectures have been described as MISD even though the architectures do not completely follow the description in Flynn’s Taxonomy. One common example is the Pipeline architecture where the data is passed from processing element to processing element and each processing element executes a different instruction. It is often argued that this is not a true MISD architecture as the data changes as it moves through the pipeline. Similarly, other architectures or applications exist which may be called MISD by some while others disagree. The following section looks at some of those implementations.<br />
<br />
Pipeline architectures belong to this type, though a purist might say that the data is different after processing by each stage in the pipeline. Fault-tolerant computers executing the same instructions redundantly in order to detect and mask errors, in a manner known as task replication, may be considered to belong to this type. Not many instances of this architecture exist, as MIMD and SIMD are often more appropriate for common data parallel techniques. Specifically, they allow better scaling and use of computational resources than MISD does. <br />
<br />
However, one prominent example of MISD in computing is the Space Shuttle flight control computers. Another example of this machine is the systolic array, such as the [http://www.cs.cmu.edu/~iwarp/ CMU iWrap] [BORKAR et al., 1990]. All the elements in this array are controlled by a global clock. On each cycle, an element will read a piece of data from one of its neighbors, perform a simple operation (e.g. add the incoming element to a stored value), and prepare a value to be written to a neighbor on the next step.<br />
<br />
==Implementations of MISD architecture==<br />
<br />
Some major implementations of MISD are in the fields of systolic arrays, pattern matching, flight control systems and fault tolerance systems. These MISD implementations are discussed below.<br />
<br />
===Systolic Array===<br />
<br />
A systolic array is an arrangement of processors in an array where data flows synchronously across the array between neighbors, usually with different data flowing in different directions. Each Processor at each step takes in data from one or more neighbors, processes it and, in the next step, outputs results in the opposite direction.<br />
<br />
The systolic array paradigm, data-stream-driven by data counters, is the counterpart of the [http://en.wikipedia.org/wiki/Von_Neumann_model von Neumann paradigm], instruction-stream-driven by a program counter. Because a systolic array usually sends and receives multiple data streams, and multiple data counters are needed to generate these data streams, it supports data parallelism. The name derives from analogy with the regular pumping of blood by the heart.<ref>http://en.wikipedia.org/wiki/Systolic_array</ref><br />
<br />
====Type of Systolic Arrays<ref>Johnson, K.T.; Hurson, A.R.; Shirazi, B.; , "General-purpose systolic arrays," Computer , vol.26, no.11, pp.20-31, Nov. 1993 doi: 10.1109/2.241423</ref>====<br />
<br />
=====Special-purpose systolic array=====<br />
[[Image:systolic_1.png|thumb|right|250px|Figure 3: The algorithm for the sum of a scalar product, computed in systolic element [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
[[Image:systolic_2.png|thumb|right|250px|Figure 4: The systolic product of two 3x3 matrices [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
An array of hardwired systolic processing elements tailored for a specific application. Typically, many tens or hundreds of cells fit on a single chip. One of the major applications of special-purpose systolic array is in matrix operations. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Systolic_1.png Figure 3] illustrates the algorithm for the sum of a scalar product, computed in a single systolic element. Here, a’s and b’s are synchronously shifted through the processing element to be available for next element. These data synchronously exits the processing element unmodified for the next element. The sum of the products is then shifted out of the accumulator.<br />
<br />
=====General-purpose systolic array=====<br />
<br />
It is an array of systolic processing elements, which gets adapted to a variety of applications via programming or reconfiguration. Array topologies can be either programmable or reconfigurable. Likewise, array cells are either programmable or reconfigurable. This is referred to as Systolic topologies.<br />
<br />
======Programmable Systolic Array======<br />
<br />
It is an array of programmable systolic elements that operates either in SIMD or MIMD fashion. Either the arrays interconnect or each processing unit is programmable and a program controls dataflow through the elements. Programmable systolic arrays are programmable either at a high level or a low level. At either level, programmable arrays can be categorized as either SIMD or MIMD machines.<br />
<br />
[[Image:systolic_4.png|thumb|right|250px|Figure 5: General organization of MIMD programmable linear systolic arrays [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
<br />
For the latter approach, the workstation downloads a program to each MIMD (Figure 4) systolic cell. Each cell may be loaded with a different program, or all the cells in the array may be loaded with the same program. Each cell's architecture is somewhat similar to the conventional von Neumann architecture: It contains a control unit, an ALU, and local memory. MIMD systolic cells have more local memory than their SIMD counterparts to support the von Neumann-style organization.<br />
<br />
This architecture is defined as Multiple Instruction Multiple Data (MIMD) architecture in [http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf 5]. The architecture has multiple instruction streams for the PEs and a single data stream passing through all the PEs. Thus, it can also be defined as Multiple Instruction Single Data (MISD) architecture. The architecture of Systolic array configuration are controversial as explained in the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Architecture_of_systolic_arrays_as_against_MISD_architecture section 4.1.2.]<br />
<br />
======Reconfigurable Systolic Array======<br />
[[Image:reconfig.jpg|thumb|right|250px|Figure 6: Block Diagram of the RSA Architecture [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
It is an array of systolic elements that can be programmed at the lowest level. Recent gate density advances in FPGA technology have produced a low-level, reconfigurable systolic array architecture that bridges the gap between special-purpose arrays and the more versatile, programmable general-purpose arrays. The FPGA architecture is unusual because a single hardware platform can be logically reconfigured as an exact duplicate of a special-purpose systolic array. <br />
<br />
The RSA circuit design is based on systolic array architecture consisting of PEs interconnected via SWs as depicted in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Reconfig.jpg Figure 6]. The homogeneous characteristic of the Reconfigurable Systolic Array (RSA) architecture, where each reconfigurable processing element (PE) cell is connected to its nearest neighbors via configurable switch (SW) elements, enables array expansion for parallel processing and facilitates time sharing computation of high-throughput data by individual PEs. Both the PEs and SWs can be reconfigured dynamically with the former as an arithmetic processor and the latter as a flexible router linking the neighboring PE cells. The RSA shifts reconfiguration and input signals into the PEs and SWs on separate data bus which enables the circuit to continue its operation while the reconfiguration is in process.<br />
<br />
====Architecture of systolic arrays as against MISD architecture====<br />
[[Image:comp.png|thumb|right|250px|Figure 7.Comparison between Architecture of systolic arrays and MISD]]<br />
<br />
As from the above mentioned configurations of the Systolic Arrays, it is seen that generally the configurations have multiple processing elements executing different instructions from dedicated instruction streams for each processing element. There is a single data stream that connects the adjacent PEs. Thus, systolic array can be defined as an MISD architecture.<br />
<br />
Many authors say that as the data read as input by one processing element is processed data output of the adjacent PE. The data stream cannot be considered as single because all the data paths do not carry the same data to all the PEs. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Comp.png Figure 7] shows the difference between the Data Stream for Systolic Arrays and the MISD architecture. Thus the systolic array should be considered as “Multiple Data” architecture and not Single Data architecture.<br />
<br />
===Fault Tolerant Systems<ref>http://en.wikipedia.org/wiki/Fault-tolerant_computer_system#Types_of_fault_tolerance</ref>===<br />
<br />
The fault tolerant systems are designed to handle the possible failures in software, hardware or interfaces. The hardware faults include hard disk failures, input or output device failures, etc. and the software and interface faults include driver failures; operator errors, installing unexpected software etc. The hardware faults can be detected and identified by implementing redundant hardware and multiple backups. The software faults can be tolerable by removing the program errors by executing the software redundantly or by implementing small programs that take over the tasks that crash or generate errors.<br />
<br />
[[Image:fault.png|thumb|right|250px|Figure 8 MISD as fault tolerant architecture]]<br />
<br />
In computer systems, the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Single_Instruction.2C_Multiple_Data_streams_.28SIMD.29 SIMD], [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD] and [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instruction.2C_Multiple_Data_streams_.28MIMD.29 MIMD] architectures facilitate the implementation of the fault tolerance systems by multiple instruction streams or multiple data streams or both. Fault tolerance on computations can be implemented by multiple processors (likely with different architectures) executing the algorithms on the same set of data. The output of each processor is compared with that of the others and M out of N majority voting method is used to determine the faulty processor. Thus MISD architecture is utilized to get the fault tolerance on critical computations.<br />
<br />
There are various examples of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture] being used as fault tolerant architecture. The major examples being flight control systems, nuclear power plants, satellite systems, super collider experiment systems, etc. Here, the flight control system is explained as an example of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture].<br />
<br />
====The Flight Control System – MISD Example for fault tolerance====<br />
<br />
A [http://en.wikipedia.org/wiki/Fly-by-wire Fly-By-Wire] system is used to replace the manual flight control by an electronic control interface. The movements of the flight control in the cockpit are converted to electronic signals and are transmitted to the actuators by wires. The control computers use the feedback from the sensors to compute and control the movement of the actuators to provide the expected response. These computers also perform the task to stabilize the aircraft and perform other tasks without the knowledge of the pilot. Flight control systems must meet extremely high levels of accuracy and functional integrity.<br />
<br />
There are redundant flight control computers present in the flight control system. If one of the flight-control computers crashes, gets damaged or is affected by electromagnetic pulses, the other computer can overrule the faulty one and hence the flight of the aircraft is unharmed. [[Image:fig13.png|thumb|right|250px|Figure 9: Architecture of triple redundant 777 primary flight computer [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-7 6]]]The number of redundant flight control computers is generally more than two, so that any computer whose results disagree with the others is ruled out to be faulty and is either ignored or rebooted.<br />
<br />
=====Multiple Processors Implementation in Boeing 777<ref>Yeh, Y.C.;, "Triple-triple redundant 777 primary flight computer," Aerospace Applications Conference, 1996. Proceedings., 1996 IEEE , vol.1, no., pp.293-307 vol.1, 3-10 Feb 1996<br />
doi: 10.1109/AERO.1996.495891</ref>=====<br />
<br />
In modern computers, the redundant flight control computations are carried out by multiprocessor systems. The triple redundant 777 primary flight computer, has the architecture as shown in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Fig13.png Figure 9].<br />
<br />
The system has three primary flight control computers, each of them having three lanes with different processors. The flight control program is compiled for each of the processors which get the input data from the same data bus but drive the output on their individual control bus. Thus each processor executes different instructions but they process the same data. Thus, it is the best suited example of Multiple Instruction Single Data (MISD) architecture.<br />
<br />
The three processors selected for the flight control system of [http://en.wikipedia.org/wiki/Boeing_777 Boeing 777] were [http://en.wikipedia.org/wiki/Intel_80486 Intel 80486], [http://en.wikipedia.org/wiki/Motorola_68040 Motorola 68040] and [http://en.wikipedia.org/wiki/AMD_Am29000 AMD 29050]. The dissimilar processors lead to dissimilar interface hardware circuits and compilers. Each lane of the flight control computer is data synchronized with the other lanes so that all of the lanes read the same frame of data from the flight sensors. As the outputs of each lane can be different, the median value of the outputs is used to select the output of the lane to be considered. The lane which has the median value select hardware selected is said to be in “command mode” whereas the other lanes are said to be in “monitoring mode”. It receives the data from the other Primary Flight Computer (PFC) lanes and performs a median select of the outputs. This provides a fault blocking mechanism before the fault detection and identification by the cross-lane monitoring system. Thus, the MISD based multi computer architecture is capable of detecting generic errors in compilers or in complex hardware devices providing assurance beyond reasonable doubt of the dependability of the Fly-By-Wire system.<br />
<br />
The above mentioned system clearly has individual Instruction Streams as the architecture of each processor is different, thus different instruction sets and different instruction streams. These processors have frame synchronized input data which means they have same set of data to work upon which is fed from a single data stream. Thus the flight control system can be classified under MISD architecture.<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System===<br />
<br />
The image pre-processing system of this Multiprocessor <ref>M.Gajer;,“ Parallel Image Processing on The TMS320C80 Multiprocessor System”, http://focus.ti.com/pdfs/univ/11-CommunicationAudioSpeech.pdf</ref> , the pre-image processing time needed to be made shorter. To achieve the aim of reducing the pre-processing time, different types of image processing architectures for ‘C80 were presented. For example MIMD architecture is used for image histogram equalization; SIMD architecture for median filtering; MISD architecture for directional edge detection.<br />
<br />
The MISD architecture can be explained as follows. Here, each processor performs different operations on the same data set. It is mentioned at many places that MISD architecture was defined only for the sake of completeness of classification due to difficulty in practice to find a computer based on MISD architecture<ref>TMS320C80 System Level Synopsis, Texas Instruments, 1995</ref>. But in case of Image Processing, this assumption need not be true as here making many different operations on same image is rare and hence MISD architecture is a best fit for such cases. As an example of implementation of image processing operations for the MISD architecture the edge detection operation will be considered. The edges should be detected in four different directions (north, south, west and east) by the medium of four different filter masks. Each of the parallel processors of the TMS320C80 calculates one of the directional edges. Finally one obtains four images with detected edges, each in different direction.<br />
<br />
The image processing in computer is very often a multistage process composed of many stages. For example, an algorithm detecting contours of the objects in an image. The algorithm for achieving the before mentioned is composed of four basic image processing operation. The image acquired from the camera in the first step undergoes low-pass filtering, during which it is smoothed and noise is also eliminated. In the next step image is thresholding is performed and each pixel gets black or white depending on that if its grey level is below or above threshold value. The resulting image is a binary image from which edges are detected by the usage of the Laplace filter. In the last step the image with detected contours undergoes logic filtering in the purpose of elimination of single black pixels placed on the white background. For implementing the above processes parallel, a five-stage pipeline was used. The first stage of this pipeline constitutes the master processor which accounts for control of the image acquisition process. Further pipe line stages were implemented for 4 parallel processors as mentioned in table 1.<br />
<table border=1><br />
<tr><td>Stage</td><td>Processor</td><td>Operation</td></tr><br />
<tr><td>1</td><td>MP</td><td>Image acquisition</td></tr><br />
<tr><td>2</td><td>PP0/td><td>Low-pass Filtering</td></tr><br />
<tr><td>3</td><td>PP1</td><td>Thresholding</td></tr><br />
<tr><td>4</td><td>PP2</td><td>Laplace Filtering</td></tr><br />
<tr><td>5</td><td>PP3</td><td>Logic Filtering</td></tr> <br />
</table><br />
Table 1: Pipeline structure for image edge detection <ref>M.Gajer;,“ Parallel Image Processing on The TMS320C80 Multiprocessor System”, http://focus.ti.com/pdfs/univ/11-CommunicationAudioSpeech.pdf</ref><br />
<br />
=='''References'''==<br />
<references/><br />
<br />
=='''Glossary'''==<br />
*'''CMU''': Carnegie Mellon University<br />
<br />
*'''CU''': Control Unit<br />
<br />
*'''DS''': Data Stream<br />
<br />
*'''Fly-By-Wire''': System that replaces the conventional manual flight controls of an aircraft with an electronic interface<br />
<br />
*'''FPGA''': Field Programmable Gate Array<br />
<br />
*'''Heterogeneous Systems''': A multiprocessor system with different kind of processors<br />
<br />
*'''Homogeneous System''': A multiprocessor system with same kind of processors<br />
<br />
*'''ILP''': Instruction Level Parallelism<br />
<br />
*'''IS''': Instruction Stream<br />
<br />
*'''MIMD''': Multiple Instruction Multiple Data<br />
<br />
*'''MISD''': Multiple Instruction Single Data<br />
<br />
*'''PE''': Processing Element<br />
<br />
*'''PFC''': Primary Flight Computer<br />
<br />
*'''SAPO''': Short for Samočinný počítač<br />
<br />
*'''SIMD''': Single Instruction Multiple Data<br />
<br />
*'''SISD''': Single Instruction Single Data<br />
<br />
*'''VLSI''': Very Large Scale Integration</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/1a_sp&diff=71792CSC/ECE 506 Spring 2013/1a sp2013-02-04T23:10:39Z<p>Schakra8: </p>
<hr />
<div>==Multiple Instruction Single Data (MISD) Architecture==<br />
<br />
Multiple Instruction Single Data, or MISD, architecture is one of the four general categories of multi-processor or multi-core architectures described by Flynn's Taxonomy.<br />
<br />
==MISD Architectures within the Context of Flynn’s Taxonomy<ref>http://en.wikipedia.org/wiki/Flynn's_taxonomy</ref><ref>http://www.phy.ornl.gov/csep/ca/node11.html</ref>==<br />
<br />
Flynn's Taxonomy categorizes parallel computing architectures based on the number of information streams that flowed into the processor.<br />
According to Flynn's Taxonomy, there are two kinds of logical streams of information that flows into a parallel computer, the Instruction Stream and the Data stream. Note that for the purposes of classification using Flynn’s Taxonomy, it does not matter whether or not the streams are physically independent of each other on the hardware, only that the streams can be logically treated as independent.<br />
Based on how many of each type of stream flows into the parallel computer (single vs. multiple), Flynn's Taxonomy divides parallel architectures into four categories. They are Single Instruction Single Data (SISD), Single Instruction Multiple Data (SIMD), Multiple Instruction Single Data (MISD), and Multiple Instruction Multiple Data (MIMD).<br />
<br />
The four classifications defined by Flynn are based upon the number of concurrent instruction (or control) and data streams available in the architecture are<ref>https://computing.llnl.gov/tutorials/parallel_comp/#Flynn</ref><br />
<br />
[[Image:Flynn's Taxonomy.PNG|thumb|center|400px|Figure 1. [http://en.wikipedia.org/wiki/Michael_J._Flynn Flynn]'s Taxonomy [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
==Overview of the MISD Architecture==<br />
<br />
In order to be categorized as MISD architecture, there must be at least two Instruction streams and there can only be one Data stream as shown on Figure 2. As per the description according to Flynn’s Taxonomy, each processing elements should receive the same data and execute different instructions on the same data simultaneously.<br />
<br />
[[Image:MISD.PNG|thumb|right|100px|Figure 2. MISD Architecture [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
There are very few implementations of the classical MISD architecture as described by Flynn’s Taxonomy, none of which are commercial, general purpose parallel computers. Instead, MISD architectures are mainly used to create custom hardware in order to solve specific problems. The majority of the implementations have been created are in academia.<br />
Over the years, several architectures have been described as MISD even though the architectures do not completely follow the description in Flynn’s Taxonomy. One common example is the Pipeline architecture where the data is passed from processing element to processing element and each processing element executes a different instruction. It is often argued that this is not a true MISD architecture as the data changes as it moves through the pipeline. Similarly, other architectures or applications exist which may be called MISD by some while others disagree. The following section looks at some of those implementations.<br />
<br />
Pipeline architectures belong to this type, though a purist might say that the data is different after processing by each stage in the pipeline. Fault-tolerant computers executing the same instructions redundantly in order to detect and mask errors, in a manner known as task replication, may be considered to belong to this type. Not many instances of this architecture exist, as MIMD and SIMD are often more appropriate for common data parallel techniques. Specifically, they allow better scaling and use of computational resources than MISD does. <br />
<br />
However, one prominent example of MISD in computing is the Space Shuttle flight control computers. Another example of this machine is the systolic array, such as the [http://www.cs.cmu.edu/~iwarp/ CMU iWrap] [BORKAR et al., 1990]. All the elements in this array are controlled by a global clock. On each cycle, an element will read a piece of data from one of its neighbors, perform a simple operation (e.g. add the incoming element to a stored value), and prepare a value to be written to a neighbor on the next step.<br />
<br />
==Implementations of MISD architecture==<br />
<br />
Some major implementations of MISD are in the fields of systolic arrays, pattern matching, flight control systems and fault tolerance systems. These MISD implementations are discussed below.<br />
<br />
===Systolic Array===<br />
<br />
A systolic array is an arrangement of processors in an array where data flows synchronously across the array between neighbors, usually with different data flowing in different directions. Each Processor at each step takes in data from one or more neighbors, processes it and, in the next step, outputs results in the opposite direction.<br />
<br />
The systolic array paradigm, data-stream-driven by data counters, is the counterpart of the [http://en.wikipedia.org/wiki/Von_Neumann_model von Neumann paradigm], instruction-stream-driven by a program counter. Because a systolic array usually sends and receives multiple data streams, and multiple data counters are needed to generate these data streams, it supports data parallelism. The name derives from analogy with the regular pumping of blood by the heart.<ref>http://en.wikipedia.org/wiki/Systolic_array</ref><br />
<br />
====Type of Systolic Arrays<ref>Johnson, K.T.; Hurson, A.R.; Shirazi, B.; , "General-purpose systolic arrays," Computer , vol.26, no.11, pp.20-31, Nov. 1993 doi: 10.1109/2.241423</ref>====<br />
<br />
=====Special-purpose systolic array=====<br />
[[Image:systolic_1.png|thumb|right|250px|Figure 3: The algorithm for the sum of a scalar product, computed in systolic element [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
[[Image:systolic_2.png|thumb|right|250px|Figure 4: The systolic product of two 3x3 matrices [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
An array of hardwired systolic processing elements tailored for a specific application. Typically, many tens or hundreds of cells fit on a single chip. One of the major applications of special-purpose systolic array is in matrix operations. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Systolic_1.png Figure 3] illustrates the algorithm for the sum of a scalar product, computed in a single systolic element. Here, a’s and b’s are synchronously shifted through the processing element to be available for next element. These data synchronously exits the processing element unmodified for the next element. The sum of the products is then shifted out of the accumulator.<br />
<br />
=====General-purpose systolic array=====<br />
<br />
It is an array of systolic processing elements, which gets adapted to a variety of applications via programming or reconfiguration. Array topologies can be either programmable or reconfigurable. Likewise, array cells are either programmable or reconfigurable. This is referred to as Systolic topologies.<br />
<br />
======Programmable Systolic Array======<br />
<br />
It is an array of programmable systolic elements that operates either in SIMD or MIMD fashion. Either the arrays interconnect or each processing unit is programmable and a program controls dataflow through the elements. Programmable systolic arrays are programmable either at a high level or a low level. At either level, programmable arrays can be categorized as either SIMD or MIMD machines.<br />
<br />
[[Image:systolic_4.png|thumb|right|250px|Figure 5: General organization of MIMD programmable linear systolic arrays [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
<br />
For the latter approach, the workstation downloads a program to each MIMD (Figure 4) systolic cell. Each cell may be loaded with a different program, or all the cells in the array may be loaded with the same program. Each cell's architecture is somewhat similar to the conventional von Neumann architecture: It contains a control unit, an ALU, and local memory. MIMD systolic cells have more local memory than their SIMD counterparts to support the von Neumann-style organization.<br />
<br />
This architecture is defined as Multiple Instruction Multiple Data (MIMD) architecture in [http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf 5]. The architecture has multiple instruction streams for the PEs and a single data stream passing through all the PEs. Thus, it can also be defined as Multiple Instruction Single Data (MISD) architecture. The architecture of Systolic array configuration are controversial as explained in the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Architecture_of_systolic_arrays_as_against_MISD_architecture section 4.1.2.]<br />
<br />
======Reconfigurable Systolic Array======<br />
[[Image:reconfig.jpg|thumb|right|250px|Figure 6: Block Diagram of the RSA Architecture [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
It is an array of systolic elements that can be programmed at the lowest level. Recent gate density advances in FPGA technology have produced a low-level, reconfigurable systolic array architecture that bridges the gap between special-purpose arrays and the more versatile, programmable general-purpose arrays. The FPGA architecture is unusual because a single hardware platform can be logically reconfigured as an exact duplicate of a special-purpose systolic array. <br />
<br />
The RSA circuit design is based on systolic array architecture consisting of PEs interconnected via SWs as depicted in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Reconfig.jpg Figure 6]. The homogeneous characteristic of the Reconfigurable Systolic Array (RSA) architecture, where each reconfigurable processing element (PE) cell is connected to its nearest neighbors via configurable switch (SW) elements, enables array expansion for parallel processing and facilitates time sharing computation of high-throughput data by individual PEs. Both the PEs and SWs can be reconfigured dynamically with the former as an arithmetic processor and the latter as a flexible router linking the neighboring PE cells. The RSA shifts reconfiguration and input signals into the PEs and SWs on separate data bus which enables the circuit to continue its operation while the reconfiguration is in process.<br />
<br />
====Architecture of systolic arrays as against MISD architecture====<br />
[[Image:comp.png|thumb|right|250px|Figure 7.Comparison between Architecture of systolic arrays and MISD]]<br />
<br />
As from the above mentioned configurations of the Systolic Arrays, it is seen that generally the configurations have multiple processing elements executing different instructions from dedicated instruction streams for each processing element. There is a single data stream that connects the adjacent PEs. Thus, systolic array can be defined as an MISD architecture.<br />
<br />
Many authors say that as the data read as input by one processing element is processed data output of the adjacent PE. The data stream cannot be considered as single because all the data paths do not carry the same data to all the PEs. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Comp.png Figure 7] shows the difference between the Data Stream for Systolic Arrays and the MISD architecture. Thus the systolic array should be considered as “Multiple Data” architecture and not Single Data architecture.<br />
<br />
===Fault Tolerant Systems<ref>http://en.wikipedia.org/wiki/Fault-tolerant_computer_system#Types_of_fault_tolerance</ref>===<br />
<br />
The fault tolerant systems are designed to handle the possible failures in software, hardware or interfaces. The hardware faults include hard disk failures, input or output device failures, etc. and the software and interface faults include driver failures; operator errors, installing unexpected software etc. The hardware faults can be detected and identified by implementing redundant hardware and multiple backups. The software faults can be tolerable by removing the program errors by executing the software redundantly or by implementing small programs that take over the tasks that crash or generate errors.<br />
<br />
[[Image:fault.png|thumb|right|250px|Figure 8 MISD as fault tolerant architecture]]<br />
<br />
In computer systems, the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Single_Instruction.2C_Multiple_Data_streams_.28SIMD.29 SIMD], [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD] and [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instruction.2C_Multiple_Data_streams_.28MIMD.29 MIMD] architectures facilitate the implementation of the fault tolerance systems by multiple instruction streams or multiple data streams or both. Fault tolerance on computations can be implemented by multiple processors (likely with different architectures) executing the algorithms on the same set of data. The output of each processor is compared with that of the others and M out of N majority voting method is used to determine the faulty processor. Thus MISD architecture is utilized to get the fault tolerance on critical computations.<br />
<br />
There are various examples of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture] being used as fault tolerant architecture. The major examples being flight control systems, nuclear power plants, satellite systems, super collider experiment systems, etc. Here, the flight control system is explained as an example of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture].<br />
<br />
====The Flight Control System – MISD Example for fault tolerance====<br />
<br />
A [http://en.wikipedia.org/wiki/Fly-by-wire Fly-By-Wire] system is used to replace the manual flight control by an electronic control interface. The movements of the flight control in the cockpit are converted to electronic signals and are transmitted to the actuators by wires. The control computers use the feedback from the sensors to compute and control the movement of the actuators to provide the expected response. These computers also perform the task to stabilize the aircraft and perform other tasks without the knowledge of the pilot. Flight control systems must meet extremely high levels of accuracy and functional integrity.<br />
<br />
There are redundant flight control computers present in the flight control system. If one of the flight-control computers crashes, gets damaged or is affected by electromagnetic pulses, the other computer can overrule the faulty one and hence the flight of the aircraft is unharmed. [[Image:fig13.png|thumb|right|250px|Figure 9: Architecture of triple redundant 777 primary flight computer [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-7 6]]]The number of redundant flight control computers is generally more than two, so that any computer whose results disagree with the others is ruled out to be faulty and is either ignored or rebooted.<br />
<br />
=====Multiple Processors Implementation in Boeing 777<ref>Yeh, Y.C.;, "Triple-triple redundant 777 primary flight computer," Aerospace Applications Conference, 1996. Proceedings., 1996 IEEE , vol.1, no., pp.293-307 vol.1, 3-10 Feb 1996<br />
doi: 10.1109/AERO.1996.495891</ref>=====<br />
<br />
In modern computers, the redundant flight control computations are carried out by multiprocessor systems. The triple redundant 777 primary flight computer, has the architecture as shown in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Fig13.png Figure 9].<br />
<br />
The system has three primary flight control computers, each of them having three lanes with different processors. The flight control program is compiled for each of the processors which get the input data from the same data bus but drive the output on their individual control bus. Thus each processor executes different instructions but they process the same data. Thus, it is the best suited example of Multiple Instruction Single Data (MISD) architecture.<br />
<br />
The three processors selected for the flight control system of [http://en.wikipedia.org/wiki/Boeing_777 Boeing 777] were [http://en.wikipedia.org/wiki/Intel_80486 Intel 80486], [http://en.wikipedia.org/wiki/Motorola_68040 Motorola 68040] and [http://en.wikipedia.org/wiki/AMD_Am29000 AMD 29050]. The dissimilar processors lead to dissimilar interface hardware circuits and compilers. Each lane of the flight control computer is data synchronized with the other lanes so that all of the lanes read the same frame of data from the flight sensors. As the outputs of each lane can be different, the median value of the outputs is used to select the output of the lane to be considered. The lane which has the median value select hardware selected is said to be in “command mode” whereas the other lanes are said to be in “monitoring mode”. It receives the data from the other Primary Flight Computer (PFC) lanes and performs a median select of the outputs. This provides a fault blocking mechanism before the fault detection and identification by the cross-lane monitoring system. Thus, the MISD based multi computer architecture is capable of detecting generic errors in compilers or in complex hardware devices providing assurance beyond reasonable doubt of the dependability of the Fly-By-Wire system.<br />
<br />
The above mentioned system clearly has individual Instruction Streams as the architecture of each processor is different, thus different instruction sets and different instruction streams. These processors have frame synchronized input data which means they have same set of data to work upon which is fed from a single data stream. Thus the flight control system can be classified under MISD architecture.<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System===<br />
<br />
The image pre-processing system of this Multiprocessor <ref>M.Gajer;,“ Parallel Image Processing on The TMS320C80 Multiprocessor System”, http://focus.ti.com/pdfs/univ/11-CommunicationAudioSpeech.pdf</ref> , the pre-image processing time needed to be made shorter. To achieve the aim of reducing the pre-processing time, different types of image processing architectures for ‘C80 were presented. For example MIMD architecture is used for image histogram equalization; SIMD architecture for median filtering; MISD architecture for directional edge detection.<br />
<br />
The MISD architecture can be explained as follows. Here, each processor performs different operations on the same data set. It is mentioned at many places that MISD architecture was defined only for the sake of completeness of classification due to difficulty in practice to find a computer based on MISD architecture<ref>TMS320C80 System Level Synopsis, Texas Instruments, 1995</ref>. But in case of Image Processing, this assumption need not be true as here making many different operations on same image is rare and hence MISD architecture is a best fit for such cases. As an example of implementation of image processing operations for the MISD architecture the edge detection operation will be considered. The edges should be detected in four different directions (north, south, west and east) by the medium of four different filter masks. Each of the parallel processors of the TMS320C80 calculates one of the directional edges. Finally one obtains four images with detected edges, each in different direction.<br />
<br />
The image processing in computer is very often a multistage process composed of many stages. For example, an algorithm detecting contours of the objects in an image. The algorithm for achieving the before mentioned is composed of four basic image processing operation. The image acquired from the camera in the first step undergoes low-pass filtering, during which it is smoothed and noise is also eliminated. In the next step image is thresholding is performed and each pixel gets black or white depending on that if its grey level is below or above threshold value. The resulting image is a binary image from which edges are detected by the usage of the Laplace filter. In the last step the image with detected contours undergoes logic filtering in the purpose of elimination of single black pixels placed on the white background. For implementing the above processes parallel, a five-stage pipeline was used. The first stage of this pipeline constitutes the master processor which accounts for control of the image acquisition process. Further pipe line stages were implemented for 4 parallel processors as mentioned in table 1.<br />
<table><br />
<tr><td>Stage</td><td>Processor</td><td>Operation</td></tr><br />
1 MP Image acquisition<br />
2 PP0 Low-pass Filtering<br />
3 PP1 Thresholding<br />
4 PP2 Laplace Filtering<br />
5 PP3 Logic Filtering<br />
</table><br />
Table 1: Pipeline structure for image edge detection [9]<br />
<br />
=='''References'''==<br />
<references/><br />
<br />
=='''Glossary'''==<br />
*'''CMU''': Carnegie Mellon University<br />
<br />
*'''CU''': Control Unit<br />
<br />
*'''DS''': Data Stream<br />
<br />
*'''Fly-By-Wire''': System that replaces the conventional manual flight controls of an aircraft with an electronic interface<br />
<br />
*'''FPGA''': Field Programmable Gate Array<br />
<br />
*'''Heterogeneous Systems''': A multiprocessor system with different kind of processors<br />
<br />
*'''Homogeneous System''': A multiprocessor system with same kind of processors<br />
<br />
*'''ILP''': Instruction Level Parallelism<br />
<br />
*'''IS''': Instruction Stream<br />
<br />
*'''MIMD''': Multiple Instruction Multiple Data<br />
<br />
*'''MISD''': Multiple Instruction Single Data<br />
<br />
*'''PE''': Processing Element<br />
<br />
*'''PFC''': Primary Flight Computer<br />
<br />
*'''SAPO''': Short for Samočinný počítač<br />
<br />
*'''SIMD''': Single Instruction Multiple Data<br />
<br />
*'''SISD''': Single Instruction Single Data<br />
<br />
*'''VLSI''': Very Large Scale Integration</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/1a_sp&diff=71787CSC/ECE 506 Spring 2013/1a sp2013-02-04T23:05:44Z<p>Schakra8: </p>
<hr />
<div>==Multiple Instruction Single Data (MISD) Architecture==<br />
<br />
Multiple Instruction Single Data, or MISD, architecture is one of the four general categories of multi-processor or multi-core architectures described by Flynn's Taxonomy.<br />
<br />
==MISD Architectures within the Context of Flynn’s Taxonomy<ref>http://en.wikipedia.org/wiki/Flynn's_taxonomy</ref><ref>http://www.phy.ornl.gov/csep/ca/node11.html</ref>==<br />
<br />
Flynn's Taxonomy categorizes parallel computing architectures based on the number of information streams that flowed into the processor.<br />
According to Flynn's Taxonomy, there are two kinds of logical streams of information that flows into a parallel computer, the Instruction Stream and the Data stream. Note that for the purposes of classification using Flynn’s Taxonomy, it does not matter whether or not the streams are physically independent of each other on the hardware, only that the streams can be logically treated as independent.<br />
Based on how many of each type of stream flows into the parallel computer (single vs. multiple), Flynn's Taxonomy divides parallel architectures into four categories. They are Single Instruction Single Data (SISD), Single Instruction Multiple Data (SIMD), Multiple Instruction Single Data (MISD), and Multiple Instruction Multiple Data (MIMD).<br />
<br />
The four classifications defined by Flynn are based upon the number of concurrent instruction (or control) and data streams available in the architecture are<ref>https://computing.llnl.gov/tutorials/parallel_comp/#Flynn</ref><br />
<br />
[[Image:Flynn's Taxonomy.PNG|thumb|center|400px|Figure 1. [http://en.wikipedia.org/wiki/Michael_J._Flynn Flynn]'s Taxonomy [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
==Overview of the MISD Architecture==<br />
<br />
In order to be categorized as MISD architecture, there must be at least two Instruction streams and there can only be one Data stream as shown on Figure 2. As per the description according to Flynn’s Taxonomy, each processing elements should receive the same data and execute different instructions on the same data simultaneously.<br />
<br />
[[Image:MISD.PNG|thumb|right|100px|Figure 2. MISD Architecture [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
There are very few implementations of the classical MISD architecture as described by Flynn’s Taxonomy, none of which are commercial, general purpose parallel computers. Instead, MISD architectures are mainly used to create custom hardware in order to solve specific problems. The majority of the implementations have been created are in academia.<br />
Over the years, several architectures have been described as MISD even though the architectures do not completely follow the description in Flynn’s Taxonomy. One common example is the Pipeline architecture where the data is passed from processing element to processing element and each processing element executes a different instruction. It is often argued that this is not a true MISD architecture as the data changes as it moves through the pipeline. Similarly, other architectures or applications exist which may be called MISD by some while others disagree. The following section looks at some of those implementations.<br />
<br />
Pipeline architectures belong to this type, though a purist might say that the data is different after processing by each stage in the pipeline. Fault-tolerant computers executing the same instructions redundantly in order to detect and mask errors, in a manner known as task replication, may be considered to belong to this type. Not many instances of this architecture exist, as MIMD and SIMD are often more appropriate for common data parallel techniques. Specifically, they allow better scaling and use of computational resources than MISD does. <br />
<br />
However, one prominent example of MISD in computing is the Space Shuttle flight control computers. Another example of this machine is the systolic array, such as the [http://www.cs.cmu.edu/~iwarp/ CMU iWrap] [BORKAR et al., 1990]. All the elements in this array are controlled by a global clock. On each cycle, an element will read a piece of data from one of its neighbors, perform a simple operation (e.g. add the incoming element to a stored value), and prepare a value to be written to a neighbor on the next step.<br />
<br />
==Implementations of MISD architecture==<br />
<br />
Some major implementations of MISD are in the fields of systolic arrays, pattern matching, flight control systems and fault tolerance systems. These MISD implementations are discussed below.<br />
<br />
===Systolic Array===<br />
<br />
A systolic array is an arrangement of processors in an array where data flows synchronously across the array between neighbors, usually with different data flowing in different directions. Each Processor at each step takes in data from one or more neighbors, processes it and, in the next step, outputs results in the opposite direction.<br />
<br />
The systolic array paradigm, data-stream-driven by data counters, is the counterpart of the [http://en.wikipedia.org/wiki/Von_Neumann_model von Neumann paradigm], instruction-stream-driven by a program counter. Because a systolic array usually sends and receives multiple data streams, and multiple data counters are needed to generate these data streams, it supports data parallelism. The name derives from analogy with the regular pumping of blood by the heart.<ref>http://en.wikipedia.org/wiki/Systolic_array</ref><br />
<br />
====Type of Systolic Arrays<ref>Johnson, K.T.; Hurson, A.R.; Shirazi, B.; , "General-purpose systolic arrays," Computer , vol.26, no.11, pp.20-31, Nov. 1993 doi: 10.1109/2.241423</ref>====<br />
<br />
=====Special-purpose systolic array=====<br />
[[Image:systolic_1.png|thumb|right|250px|Figure 3: The algorithm for the sum of a scalar product, computed in systolic element [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
[[Image:systolic_2.png|thumb|right|250px|Figure 4: The systolic product of two 3x3 matrices [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
An array of hardwired systolic processing elements tailored for a specific application. Typically, many tens or hundreds of cells fit on a single chip. One of the major applications of special-purpose systolic array is in matrix operations. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Systolic_1.png Figure 3] illustrates the algorithm for the sum of a scalar product, computed in a single systolic element. Here, a’s and b’s are synchronously shifted through the processing element to be available for next element. These data synchronously exits the processing element unmodified for the next element. The sum of the products is then shifted out of the accumulator.<br />
<br />
=====General-purpose systolic array=====<br />
<br />
It is an array of systolic processing elements, which gets adapted to a variety of applications via programming or reconfiguration. Array topologies can be either programmable or reconfigurable. Likewise, array cells are either programmable or reconfigurable. This is referred to as Systolic topologies.<br />
<br />
======Programmable Systolic Array======<br />
<br />
It is an array of programmable systolic elements that operates either in SIMD or MIMD fashion. Either the arrays interconnect or each processing unit is programmable and a program controls dataflow through the elements. Programmable systolic arrays are programmable either at a high level or a low level. At either level, programmable arrays can be categorized as either SIMD or MIMD machines.<br />
<br />
[[Image:systolic_4.png|thumb|right|250px|Figure 5: General organization of MIMD programmable linear systolic arrays [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
<br />
For the latter approach, the workstation downloads a program to each MIMD (Figure 4) systolic cell. Each cell may be loaded with a different program, or all the cells in the array may be loaded with the same program. Each cell's architecture is somewhat similar to the conventional von Neumann architecture: It contains a control unit, an ALU, and local memory. MIMD systolic cells have more local memory than their SIMD counterparts to support the von Neumann-style organization.<br />
<br />
This architecture is defined as Multiple Instruction Multiple Data (MIMD) architecture in [http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf 5]. The architecture has multiple instruction streams for the PEs and a single data stream passing through all the PEs. Thus, it can also be defined as Multiple Instruction Single Data (MISD) architecture. The architecture of Systolic array configuration are controversial as explained in the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Architecture_of_systolic_arrays_as_against_MISD_architecture section 4.1.2.]<br />
<br />
======Reconfigurable Systolic Array======<br />
[[Image:reconfig.jpg|thumb|right|250px|Figure 6: Block Diagram of the RSA Architecture [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
It is an array of systolic elements that can be programmed at the lowest level. Recent gate density advances in FPGA technology have produced a low-level, reconfigurable systolic array architecture that bridges the gap between special-purpose arrays and the more versatile, programmable general-purpose arrays. The FPGA architecture is unusual because a single hardware platform can be logically reconfigured as an exact duplicate of a special-purpose systolic array. <br />
<br />
The RSA circuit design is based on systolic array architecture consisting of PEs interconnected via SWs as depicted in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Reconfig.jpg Figure 6]. The homogeneous characteristic of the Reconfigurable Systolic Array (RSA) architecture, where each reconfigurable processing element (PE) cell is connected to its nearest neighbors via configurable switch (SW) elements, enables array expansion for parallel processing and facilitates time sharing computation of high-throughput data by individual PEs. Both the PEs and SWs can be reconfigured dynamically with the former as an arithmetic processor and the latter as a flexible router linking the neighboring PE cells. The RSA shifts reconfiguration and input signals into the PEs and SWs on separate data bus which enables the circuit to continue its operation while the reconfiguration is in process.<br />
<br />
====Architecture of systolic arrays as against MISD architecture====<br />
[[Image:comp.png|thumb|right|250px|Figure 7.Comparison between Architecture of systolic arrays and MISD]]<br />
<br />
As from the above mentioned configurations of the Systolic Arrays, it is seen that generally the configurations have multiple processing elements executing different instructions from dedicated instruction streams for each processing element. There is a single data stream that connects the adjacent PEs. Thus, systolic array can be defined as an MISD architecture.<br />
<br />
Many authors say that as the data read as input by one processing element is processed data output of the adjacent PE. The data stream cannot be considered as single because all the data paths do not carry the same data to all the PEs. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Comp.png Figure 7] shows the difference between the Data Stream for Systolic Arrays and the MISD architecture. Thus the systolic array should be considered as “Multiple Data” architecture and not Single Data architecture.<br />
<br />
===Fault Tolerant Systems<ref>http://en.wikipedia.org/wiki/Fault-tolerant_computer_system#Types_of_fault_tolerance</ref>===<br />
<br />
The fault tolerant systems are designed to handle the possible failures in software, hardware or interfaces. The hardware faults include hard disk failures, input or output device failures, etc. and the software and interface faults include driver failures; operator errors, installing unexpected software etc. The hardware faults can be detected and identified by implementing redundant hardware and multiple backups. The software faults can be tolerable by removing the program errors by executing the software redundantly or by implementing small programs that take over the tasks that crash or generate errors.<br />
<br />
[[Image:fault.png|thumb|right|250px|Figure 8 MISD as fault tolerant architecture]]<br />
<br />
In computer systems, the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Single_Instruction.2C_Multiple_Data_streams_.28SIMD.29 SIMD], [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD] and [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instruction.2C_Multiple_Data_streams_.28MIMD.29 MIMD] architectures facilitate the implementation of the fault tolerance systems by multiple instruction streams or multiple data streams or both. Fault tolerance on computations can be implemented by multiple processors (likely with different architectures) executing the algorithms on the same set of data. The output of each processor is compared with that of the others and M out of N majority voting method is used to determine the faulty processor. Thus MISD architecture is utilized to get the fault tolerance on critical computations.<br />
<br />
There are various examples of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture] being used as fault tolerant architecture. The major examples being flight control systems, nuclear power plants, satellite systems, super collider experiment systems, etc. Here, the flight control system is explained as an example of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture].<br />
<br />
====The Flight Control System – MISD Example for fault tolerance====<br />
<br />
A [http://en.wikipedia.org/wiki/Fly-by-wire Fly-By-Wire] system is used to replace the manual flight control by an electronic control interface. The movements of the flight control in the cockpit are converted to electronic signals and are transmitted to the actuators by wires. The control computers use the feedback from the sensors to compute and control the movement of the actuators to provide the expected response. These computers also perform the task to stabilize the aircraft and perform other tasks without the knowledge of the pilot. Flight control systems must meet extremely high levels of accuracy and functional integrity.<br />
<br />
There are redundant flight control computers present in the flight control system. If one of the flight-control computers crashes, gets damaged or is affected by electromagnetic pulses, the other computer can overrule the faulty one and hence the flight of the aircraft is unharmed. [[Image:fig13.png|thumb|right|250px|Figure 9: Architecture of triple redundant 777 primary flight computer [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-7 6]]]The number of redundant flight control computers is generally more than two, so that any computer whose results disagree with the others is ruled out to be faulty and is either ignored or rebooted.<br />
<br />
=====Multiple Processors Implementation in Boeing 777<ref>Yeh, Y.C.;, "Triple-triple redundant 777 primary flight computer," Aerospace Applications Conference, 1996. Proceedings., 1996 IEEE , vol.1, no., pp.293-307 vol.1, 3-10 Feb 1996<br />
doi: 10.1109/AERO.1996.495891</ref>=====<br />
<br />
In modern computers, the redundant flight control computations are carried out by multiprocessor systems. The triple redundant 777 primary flight computer, has the architecture as shown in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Fig13.png Figure 9].<br />
<br />
The system has three primary flight control computers, each of them having three lanes with different processors. The flight control program is compiled for each of the processors which get the input data from the same data bus but drive the output on their individual control bus. Thus each processor executes different instructions but they process the same data. Thus, it is the best suited example of Multiple Instruction Single Data (MISD) architecture.<br />
<br />
The three processors selected for the flight control system of [http://en.wikipedia.org/wiki/Boeing_777 Boeing 777] were [http://en.wikipedia.org/wiki/Intel_80486 Intel 80486], [http://en.wikipedia.org/wiki/Motorola_68040 Motorola 68040] and [http://en.wikipedia.org/wiki/AMD_Am29000 AMD 29050]. The dissimilar processors lead to dissimilar interface hardware circuits and compilers. Each lane of the flight control computer is data synchronized with the other lanes so that all of the lanes read the same frame of data from the flight sensors. As the outputs of each lane can be different, the median value of the outputs is used to select the output of the lane to be considered. The lane which has the median value select hardware selected is said to be in “command mode” whereas the other lanes are said to be in “monitoring mode”. It receives the data from the other Primary Flight Computer (PFC) lanes and performs a median select of the outputs. This provides a fault blocking mechanism before the fault detection and identification by the cross-lane monitoring system. Thus, the MISD based multi computer architecture is capable of detecting generic errors in compilers or in complex hardware devices providing assurance beyond reasonable doubt of the dependability of the Fly-By-Wire system.<br />
<br />
The above mentioned system clearly has individual Instruction Streams as the architecture of each processor is different, thus different instruction sets and different instruction streams. These processors have frame synchronized input data which means they have same set of data to work upon which is fed from a single data stream. Thus the flight control system can be classified under MISD architecture.<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System===<br />
<br />
The image pre-processing system of this Multiprocessor [9] , the pre-image processing time needed to be made shorter. To achieve the aim of reducing the pre-processing time, different types of image processing architectures for ‘C80 were presented. For example MIMD architecture is used for image histogram equalization; SIMD architecture for median filtering; MISD architecture for directional edge detection.<br />
<br />
The MISD architecture can be explained as follows. Here, each processor performs different operations on the same data set. It is mentioned at many places that MISD architecture was defined only for the sake of completeness of classification due to difficulty in practice to find a computer based on MISD architecture [10]. But in case of Image Processing, this assumption need not be true as here making many different operations on same image is rare and hence MISD architecture is a best fit for such cases. As an example of implementation of image processing operations for the MISD architecture the edge detection operation will be considered. The edges should be detected in four different directions (north, south, west and east) by the medium of four different filter masks. Each of the parallel processors of the TMS320C80 calculates one of the directional edges. Finally one obtains four images with detected edges, each in different direction.<br />
<br />
The image processing in computer is very often a multistage process composed of many stages. For example, an algorithm detecting contours of the objects in an image. The algorithm for achieving the before mentioned is composed of four basic image processing operation. The image acquired from the camera in the first step undergoes low-pass filtering, during which it is smoothed and noise is also eliminated. In the next step image is thresholding is performed and each pixel gets black or white depending on that if its grey level is below or above threshold value. The resulting image is a binary image from which edges are detected by the usage of the Laplace filter. In the last step the image with detected contours undergoes logic filtering in the purpose of elimination of single black pixels placed on the white background. For implementing the above processes parallel, a five-stage pipeline was used. The first stage of this pipeline constitutes the master processor which accounts for control of the image acquisition process. Further pipe line stages were implemented for 4 parallel processors as mentioned in table 1.<br />
<br />
Stage Processor Operation<br />
1 MP Image acquisition<br />
2 PP0 Low-pass Filtering<br />
3 PP1 Thresholding<br />
4 PP2 Laplace Filtering<br />
5 PP3 Logic Filtering<br />
Table 1: Pipeline structure for image edge detection [9]<br />
<br />
=='''References'''==<br />
<references/><br />
<br />
=='''Glossary'''==<br />
*'''CMU''': Carnegie Mellon University<br />
<br />
*'''CU''': Control Unit<br />
<br />
*'''DS''': Data Stream<br />
<br />
*'''Fly-By-Wire''': System that replaces the conventional manual flight controls of an aircraft with an electronic interface<br />
<br />
*'''FPGA''': Field Programmable Gate Array<br />
<br />
*'''Heterogeneous Systems''': A multiprocessor system with different kind of processors<br />
<br />
*'''Homogeneous System''': A multiprocessor system with same kind of processors<br />
<br />
*'''ILP''': Instruction Level Parallelism<br />
<br />
*'''IS''': Instruction Stream<br />
<br />
*'''MIMD''': Multiple Instruction Multiple Data<br />
<br />
*'''MISD''': Multiple Instruction Single Data<br />
<br />
*'''PE''': Processing Element<br />
<br />
*'''PFC''': Primary Flight Computer<br />
<br />
*'''SAPO''': Short for Samočinný počítač<br />
<br />
*'''SIMD''': Single Instruction Multiple Data<br />
<br />
*'''SISD''': Single Instruction Single Data<br />
<br />
*'''VLSI''': Very Large Scale Integration</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/1a_sp&diff=71786CSC/ECE 506 Spring 2013/1a sp2013-02-04T23:04:57Z<p>Schakra8: </p>
<hr />
<div>==Multiple Instruction Single Data (MISD) Architecture==<br />
<br />
Multiple Instruction Single Data, or MISD, architecture is one of the four general categories of multi-processor or multi-core architectures described by Flynn's Taxonomy.<br />
<br />
==MISD Architectures within the Context of Flynn’s Taxonomy<ref>http://en.wikipedia.org/wiki/Flynn's_taxonomy</ref><ref>http://www.phy.ornl.gov/csep/ca/node11.html</ref>==<br />
<br />
Flynn's Taxonomy categorizes parallel computing architectures based on the number of information streams that flowed into the processor.<br />
According to Flynn's Taxonomy, there are two kinds of logical streams of information that flows into a parallel computer, the Instruction Stream and the Data stream. Note that for the purposes of classification using Flynn’s Taxonomy, it does not matter whether or not the streams are physically independent of each other on the hardware, only that the streams can be logically treated as independent.<br />
Based on how many of each type of stream flows into the parallel computer (single vs. multiple), Flynn's Taxonomy divides parallel architectures into four categories. They are Single Instruction Single Data (SISD), Single Instruction Multiple Data (SIMD), Multiple Instruction Single Data (MISD), and Multiple Instruction Multiple Data (MIMD).<br />
<br />
The four classifications defined by Flynn are based upon the number of concurrent instruction (or control) and data streams available in the architecture are<ref>https://computing.llnl.gov/tutorials/parallel_comp/#Flynn</ref><br />
<br />
[[Image:Flynn's Taxonomy.PNG|thumb|center|400px|Figure 1. [http://en.wikipedia.org/wiki/Michael_J._Flynn Flynn]'s Taxonomy [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
==Overview of the MISD Architecture==<br />
<br />
In order to be categorized as MISD architecture, there must be at least two Instruction streams and there can only be one Data stream as shown on Figure 2. As per the description according to Flynn’s Taxonomy, each processing elements should receive the same data and execute different instructions on the same data simultaneously.<br />
<br />
[[Image:MISD.PNG|thumb|right|100px|Figure 2. MISD Architecture [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
There are very few implementations of the classical MISD architecture as described by Flynn’s Taxonomy, none of which are commercial, general purpose parallel computers. Instead, MISD architectures are mainly used to create custom hardware in order to solve specific problems. The majority of the implementations have been created are in academia.<br />
Over the years, several architectures have been described as MISD even though the architectures do not completely follow the description in Flynn’s Taxonomy. One common example is the Pipeline architecture where the data is passed from processing element to processing element and each processing element executes a different instruction. It is often argued that this is not a true MISD architecture as the data changes as it moves through the pipeline. Similarly, other architectures or applications exist which may be called MISD by some while others disagree. The following section looks at some of those implementations.<br />
<br />
Pipeline architectures belong to this type, though a purist might say that the data is different after processing by each stage in the pipeline. Fault-tolerant computers executing the same instructions redundantly in order to detect and mask errors, in a manner known as task replication, may be considered to belong to this type. Not many instances of this architecture exist, as MIMD and SIMD are often more appropriate for common data parallel techniques. Specifically, they allow better scaling and use of computational resources than MISD does. <br />
<br />
However, one prominent example of MISD in computing is the Space Shuttle flight control computers. Another example of this machine is the systolic array, such as the [http://www.cs.cmu.edu/~iwarp/ CMU iWrap] [BORKAR et al., 1990]. All the elements in this array are controlled by a global clock. On each cycle, an element will read a piece of data from one of its neighbors, perform a simple operation (e.g. add the incoming element to a stored value), and prepare a value to be written to a neighbor on the next step.<br />
<br />
==Implementations of MISD architecture==<br />
<br />
Some major implementations of MISD are in the fields of systolic arrays, pattern matching, flight control systems and fault tolerance systems. These MISD implementations are discussed below.<br />
<br />
===Systolic Array===<br />
<br />
A systolic array is an arrangement of processors in an array where data flows synchronously across the array between neighbors, usually with different data flowing in different directions. Each Processor at each step takes in data from one or more neighbors, processes it and, in the next step, outputs results in the opposite direction.<br />
<br />
The systolic array paradigm, data-stream-driven by data counters, is the counterpart of the [http://en.wikipedia.org/wiki/Von_Neumann_model von Neumann paradigm], instruction-stream-driven by a program counter. Because a systolic array usually sends and receives multiple data streams, and multiple data counters are needed to generate these data streams, it supports data parallelism. The name derives from analogy with the regular pumping of blood by the heart.<ref>http://en.wikipedia.org/wiki/Systolic_array</ref><br />
<br />
====Type of Systolic Arrays<ref>http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf General Purpose Systolic Arrays </ref>====<br />
<br />
=====Special-purpose systolic array=====<br />
[[Image:systolic_1.png|thumb|right|250px|Figure 3: The algorithm for the sum of a scalar product, computed in systolic element [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
[[Image:systolic_2.png|thumb|right|250px|Figure 4: The systolic product of two 3x3 matrices [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
An array of hardwired systolic processing elements tailored for a specific application. Typically, many tens or hundreds of cells fit on a single chip. One of the major applications of special-purpose systolic array is in matrix operations. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Systolic_1.png Figure 3] illustrates the algorithm for the sum of a scalar product, computed in a single systolic element. Here, a’s and b’s are synchronously shifted through the processing element to be available for next element. These data synchronously exits the processing element unmodified for the next element. The sum of the products is then shifted out of the accumulator.<br />
<br />
=====General-purpose systolic array=====<br />
<br />
It is an array of systolic processing elements, which gets adapted to a variety of applications via programming or reconfiguration. Array topologies can be either programmable or reconfigurable. Likewise, array cells are either programmable or reconfigurable. This is referred to as Systolic topologies.<br />
<br />
======Programmable Systolic Array======<br />
<br />
It is an array of programmable systolic elements that operates either in SIMD or MIMD fashion. Either the arrays interconnect or each processing unit is programmable and a program controls dataflow through the elements. Programmable systolic arrays are programmable either at a high level or a low level. At either level, programmable arrays can be categorized as either SIMD or MIMD machines.<br />
<br />
[[Image:systolic_4.png|thumb|right|250px|Figure 5: General organization of MIMD programmable linear systolic arrays [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
<br />
For the latter approach, the workstation downloads a program to each MIMD (Figure 4) systolic cell. Each cell may be loaded with a different program, or all the cells in the array may be loaded with the same program. Each cell's architecture is somewhat similar to the conventional von Neumann architecture: It contains a control unit, an ALU, and local memory. MIMD systolic cells have more local memory than their SIMD counterparts to support the von Neumann-style organization.<br />
<br />
This architecture is defined as Multiple Instruction Multiple Data (MIMD) architecture in [http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf 5]. The architecture has multiple instruction streams for the PEs and a single data stream passing through all the PEs. Thus, it can also be defined as Multiple Instruction Single Data (MISD) architecture. The architecture of Systolic array configuration are controversial as explained in the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Architecture_of_systolic_arrays_as_against_MISD_architecture section 4.1.2.]<br />
<br />
======Reconfigurable Systolic Array======<br />
[[Image:reconfig.jpg|thumb|right|250px|Figure 6: Block Diagram of the RSA Architecture [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
It is an array of systolic elements that can be programmed at the lowest level. Recent gate density advances in FPGA technology have produced a low-level, reconfigurable systolic array architecture that bridges the gap between special-purpose arrays and the more versatile, programmable general-purpose arrays. The FPGA architecture is unusual because a single hardware platform can be logically reconfigured as an exact duplicate of a special-purpose systolic array. <br />
<br />
The RSA circuit design is based on systolic array architecture consisting of PEs interconnected via SWs as depicted in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Reconfig.jpg Figure 6]. The homogeneous characteristic of the Reconfigurable Systolic Array (RSA) architecture, where each reconfigurable processing element (PE) cell is connected to its nearest neighbors via configurable switch (SW) elements, enables array expansion for parallel processing and facilitates time sharing computation of high-throughput data by individual PEs. Both the PEs and SWs can be reconfigured dynamically with the former as an arithmetic processor and the latter as a flexible router linking the neighboring PE cells. The RSA shifts reconfiguration and input signals into the PEs and SWs on separate data bus which enables the circuit to continue its operation while the reconfiguration is in process.<br />
<br />
====Architecture of systolic arrays as against MISD architecture====<br />
[[Image:comp.png|thumb|right|250px|Figure 7.Comparison between Architecture of systolic arrays and MISD]]<br />
<br />
As from the above mentioned configurations of the Systolic Arrays, it is seen that generally the configurations have multiple processing elements executing different instructions from dedicated instruction streams for each processing element. There is a single data stream that connects the adjacent PEs. Thus, systolic array can be defined as an MISD architecture.<br />
<br />
Many authors say that as the data read as input by one processing element is processed data output of the adjacent PE. The data stream cannot be considered as single because all the data paths do not carry the same data to all the PEs. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Comp.png Figure 7] shows the difference between the Data Stream for Systolic Arrays and the MISD architecture. Thus the systolic array should be considered as “Multiple Data” architecture and not Single Data architecture.<br />
<br />
===Fault Tolerant Systems<ref>http://en.wikipedia.org/wiki/Fault-tolerant_computer_system#Types_of_fault_tolerance</ref>===<br />
<br />
The fault tolerant systems are designed to handle the possible failures in software, hardware or interfaces. The hardware faults include hard disk failures, input or output device failures, etc. and the software and interface faults include driver failures; operator errors, installing unexpected software etc. The hardware faults can be detected and identified by implementing redundant hardware and multiple backups. The software faults can be tolerable by removing the program errors by executing the software redundantly or by implementing small programs that take over the tasks that crash or generate errors.<br />
<br />
[[Image:fault.png|thumb|right|250px|Figure 8 MISD as fault tolerant architecture]]<br />
<br />
In computer systems, the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Single_Instruction.2C_Multiple_Data_streams_.28SIMD.29 SIMD], [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD] and [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instruction.2C_Multiple_Data_streams_.28MIMD.29 MIMD] architectures facilitate the implementation of the fault tolerance systems by multiple instruction streams or multiple data streams or both. Fault tolerance on computations can be implemented by multiple processors (likely with different architectures) executing the algorithms on the same set of data. The output of each processor is compared with that of the others and M out of N majority voting method is used to determine the faulty processor. Thus MISD architecture is utilized to get the fault tolerance on critical computations.<br />
<br />
There are various examples of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture] being used as fault tolerant architecture. The major examples being flight control systems, nuclear power plants, satellite systems, super collider experiment systems, etc. Here, the flight control system is explained as an example of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture].<br />
<br />
====The Flight Control System – MISD Example for fault tolerance====<br />
<br />
A [http://en.wikipedia.org/wiki/Fly-by-wire Fly-By-Wire] system is used to replace the manual flight control by an electronic control interface. The movements of the flight control in the cockpit are converted to electronic signals and are transmitted to the actuators by wires. The control computers use the feedback from the sensors to compute and control the movement of the actuators to provide the expected response. These computers also perform the task to stabilize the aircraft and perform other tasks without the knowledge of the pilot. Flight control systems must meet extremely high levels of accuracy and functional integrity.<br />
<br />
There are redundant flight control computers present in the flight control system. If one of the flight-control computers crashes, gets damaged or is affected by electromagnetic pulses, the other computer can overrule the faulty one and hence the flight of the aircraft is unharmed. [[Image:fig13.png|thumb|right|250px|Figure 9: Architecture of triple redundant 777 primary flight computer [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-7 6]]]The number of redundant flight control computers is generally more than two, so that any computer whose results disagree with the others is ruled out to be faulty and is either ignored or rebooted.<br />
<br />
=====Multiple Processors Implementation in Boeing 777<ref>Yeh, Y.C.;, "Triple-triple redundant 777 primary flight computer," Aerospace Applications Conference, 1996. Proceedings., 1996 IEEE , vol.1, no., pp.293-307 vol.1, 3-10 Feb 1996<br />
doi: 10.1109/AERO.1996.495891</ref>=====<br />
<br />
In modern computers, the redundant flight control computations are carried out by multiprocessor systems. The triple redundant 777 primary flight computer, has the architecture as shown in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Fig13.png Figure 9].<br />
<br />
The system has three primary flight control computers, each of them having three lanes with different processors. The flight control program is compiled for each of the processors which get the input data from the same data bus but drive the output on their individual control bus. Thus each processor executes different instructions but they process the same data. Thus, it is the best suited example of Multiple Instruction Single Data (MISD) architecture.<br />
<br />
The three processors selected for the flight control system of [http://en.wikipedia.org/wiki/Boeing_777 Boeing 777] were [http://en.wikipedia.org/wiki/Intel_80486 Intel 80486], [http://en.wikipedia.org/wiki/Motorola_68040 Motorola 68040] and [http://en.wikipedia.org/wiki/AMD_Am29000 AMD 29050]. The dissimilar processors lead to dissimilar interface hardware circuits and compilers. Each lane of the flight control computer is data synchronized with the other lanes so that all of the lanes read the same frame of data from the flight sensors. As the outputs of each lane can be different, the median value of the outputs is used to select the output of the lane to be considered. The lane which has the median value select hardware selected is said to be in “command mode” whereas the other lanes are said to be in “monitoring mode”. It receives the data from the other Primary Flight Computer (PFC) lanes and performs a median select of the outputs. This provides a fault blocking mechanism before the fault detection and identification by the cross-lane monitoring system. Thus, the MISD based multi computer architecture is capable of detecting generic errors in compilers or in complex hardware devices providing assurance beyond reasonable doubt of the dependability of the Fly-By-Wire system.<br />
<br />
The above mentioned system clearly has individual Instruction Streams as the architecture of each processor is different, thus different instruction sets and different instruction streams. These processors have frame synchronized input data which means they have same set of data to work upon which is fed from a single data stream. Thus the flight control system can be classified under MISD architecture.<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System===<br />
<br />
The image pre-processing system of this Multiprocessor [9] , the pre-image processing time needed to be made shorter. To achieve the aim of reducing the pre-processing time, different types of image processing architectures for ‘C80 were presented. For example MIMD architecture is used for image histogram equalization; SIMD architecture for median filtering; MISD architecture for directional edge detection.<br />
<br />
The MISD architecture can be explained as follows. Here, each processor performs different operations on the same data set. It is mentioned at many places that MISD architecture was defined only for the sake of completeness of classification due to difficulty in practice to find a computer based on MISD architecture [10]. But in case of Image Processing, this assumption need not be true as here making many different operations on same image is rare and hence MISD architecture is a best fit for such cases. As an example of implementation of image processing operations for the MISD architecture the edge detection operation will be considered. The edges should be detected in four different directions (north, south, west and east) by the medium of four different filter masks. Each of the parallel processors of the TMS320C80 calculates one of the directional edges. Finally one obtains four images with detected edges, each in different direction.<br />
<br />
The image processing in computer is very often a multistage process composed of many stages. For example, an algorithm detecting contours of the objects in an image. The algorithm for achieving the before mentioned is composed of four basic image processing operation. The image acquired from the camera in the first step undergoes low-pass filtering, during which it is smoothed and noise is also eliminated. In the next step image is thresholding is performed and each pixel gets black or white depending on that if its grey level is below or above threshold value. The resulting image is a binary image from which edges are detected by the usage of the Laplace filter. In the last step the image with detected contours undergoes logic filtering in the purpose of elimination of single black pixels placed on the white background. For implementing the above processes parallel, a five-stage pipeline was used. The first stage of this pipeline constitutes the master processor which accounts for control of the image acquisition process. Further pipe line stages were implemented for 4 parallel processors as mentioned in table 1.<br />
<br />
Stage Processor Operation<br />
1 MP Image acquisition<br />
2 PP0 Low-pass Filtering<br />
3 PP1 Thresholding<br />
4 PP2 Laplace Filtering<br />
5 PP3 Logic Filtering<br />
Table 1: Pipeline structure for image edge detection [9]<br />
<br />
=='''References'''==<br />
<references/><br />
<br />
=='''Glossary'''==<br />
*'''CMU''': Carnegie Mellon University<br />
<br />
*'''CU''': Control Unit<br />
<br />
*'''DS''': Data Stream<br />
<br />
*'''Fly-By-Wire''': System that replaces the conventional manual flight controls of an aircraft with an electronic interface<br />
<br />
*'''FPGA''': Field Programmable Gate Array<br />
<br />
*'''Heterogeneous Systems''': A multiprocessor system with different kind of processors<br />
<br />
*'''Homogeneous System''': A multiprocessor system with same kind of processors<br />
<br />
*'''ILP''': Instruction Level Parallelism<br />
<br />
*'''IS''': Instruction Stream<br />
<br />
*'''MIMD''': Multiple Instruction Multiple Data<br />
<br />
*'''MISD''': Multiple Instruction Single Data<br />
<br />
*'''PE''': Processing Element<br />
<br />
*'''PFC''': Primary Flight Computer<br />
<br />
*'''SAPO''': Short for Samočinný počítač<br />
<br />
*'''SIMD''': Single Instruction Multiple Data<br />
<br />
*'''SISD''': Single Instruction Single Data<br />
<br />
*'''VLSI''': Very Large Scale Integration</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/1a_sp&diff=71782CSC/ECE 506 Spring 2013/1a sp2013-02-04T23:01:25Z<p>Schakra8: </p>
<hr />
<div>==Multiple Instruction Single Data (MISD) Architecture==<br />
<br />
Multiple Instruction Single Data, or MISD, architecture is one of the four general categories of multi-processor or multi-core architectures described by Flynn's Taxonomy.<br />
<br />
==MISD Architectures within the Context of Flynn’s Taxonomy<ref>http://en.wikipedia.org/wiki/Flynn's_taxonomy</ref><ref>http://www.phy.ornl.gov/csep/ca/node11.html</ref>==<br />
<br />
Flynn's Taxonomy categorizes parallel computing architectures based on the number of information streams that flowed into the processor.<br />
According to Flynn's Taxonomy, there are two kinds of logical streams of information that flows into a parallel computer, the Instruction Stream and the Data stream. Note that for the purposes of classification using Flynn’s Taxonomy, it does not matter whether or not the streams are physically independent of each other on the hardware, only that the streams can be logically treated as independent.<br />
Based on how many of each type of stream flows into the parallel computer (single vs. multiple), Flynn's Taxonomy divides parallel architectures into four categories. They are Single Instruction Single Data (SISD), Single Instruction Multiple Data (SIMD), Multiple Instruction Single Data (MISD), and Multiple Instruction Multiple Data (MIMD).<br />
<br />
The four classifications defined by Flynn are based upon the number of concurrent instruction (or control) and data streams available in the architecture are<ref>https://computing.llnl.gov/tutorials/parallel_comp/#Flynn</ref><br />
<br />
[[Image:Flynn's Taxonomy.PNG|thumb|center|400px|Figure 1. [http://en.wikipedia.org/wiki/Michael_J._Flynn Flynn]'s Taxonomy [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
==Overview of the MISD Architecture==<br />
<br />
In order to be categorized as MISD architecture, there must be at least two Instruction streams and there can only be one Data stream as shown on Figure 2. As per the description according to Flynn’s Taxonomy, each processing elements should receive the same data and execute different instructions on the same data simultaneously.<br />
<br />
[[Image:MISD.PNG|thumb|right|100px|Figure 2. MISD Architecture [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
There are very few implementations of the classical MISD architecture as described by Flynn’s Taxonomy, none of which are commercial, general purpose parallel computers. Instead, MISD architectures are mainly used to create custom hardware in order to solve specific problems. The majority of the implementations have been created are in academia.<br />
Over the years, several architectures have been described as MISD even though the architectures do not completely follow the description in Flynn’s Taxonomy. One common example is the Pipeline architecture where the data is passed from processing element to processing element and each processing element executes a different instruction. It is often argued that this is not a true MISD architecture as the data changes as it moves through the pipeline. Similarly, other architectures or applications exist which may be called MISD by some while others disagree. The following section looks at some of those implementations.<br />
<br />
Pipeline architectures belong to this type, though a purist might say that the data is different after processing by each stage in the pipeline. Fault-tolerant computers executing the same instructions redundantly in order to detect and mask errors, in a manner known as task replication, may be considered to belong to this type. Not many instances of this architecture exist, as MIMD and SIMD are often more appropriate for common data parallel techniques. Specifically, they allow better scaling and use of computational resources than MISD does. <br />
<br />
However, one prominent example of MISD in computing is the Space Shuttle flight control computers. Another example of this machine is the systolic array, such as the [http://www.cs.cmu.edu/~iwarp/ CMU iWrap] [BORKAR et al., 1990]. All the elements in this array are controlled by a global clock. On each cycle, an element will read a piece of data from one of its neighbors, perform a simple operation (e.g. add the incoming element to a stored value), and prepare a value to be written to a neighbor on the next step.<br />
<br />
==Implementations of MISD architecture==<br />
<br />
Some major implementations of MISD are in the fields of systolic arrays, pattern matching, flight control systems and fault tolerance systems. These MISD implementations are discussed below.<br />
<br />
===Systolic Array===<br />
<br />
A systolic array is an arrangement of processors in an array where data flows synchronously across the array between neighbors, usually with different data flowing in different directions. Each Processor at each step takes in data from one or more neighbors, processes it and, in the next step, outputs results in the opposite direction.<br />
<br />
The systolic array paradigm, data-stream-driven by data counters, is the counterpart of the [http://en.wikipedia.org/wiki/Von_Neumann_model von Neumann paradigm], instruction-stream-driven by a program counter. Because a systolic array usually sends and receives multiple data streams, and multiple data counters are needed to generate these data streams, it supports data parallelism. The name derives from analogy with the regular pumping of blood by the heart.<ref>http://en.wikipedia.org/wiki/Systolic_array</ref><br />
<br />
====Type of Systolic Arrays<ref>http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf General Purpose Systolic Arrays </ref>====<br />
<br />
=====Special-purpose systolic array=====<br />
[[Image:systolic_1.png|thumb|right|250px|Figure 3: The algorithm for the sum of a scalar product, computed in systolic element [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
[[Image:systolic_2.png|thumb|right|250px|Figure 4: The systolic product of two 3x3 matrices [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
An array of hardwired systolic processing elements tailored for a specific application. Typically, many tens or hundreds of cells fit on a single chip. One of the major applications of special-purpose systolic array is in matrix operations. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Systolic_1.png Figure 3] illustrates the algorithm for the sum of a scalar product, computed in a single systolic element. Here, a’s and b’s are synchronously shifted through the processing element to be available for next element. These data synchronously exits the processing element unmodified for the next element. The sum of the products is then shifted out of the accumulator.<br />
<br />
=====General-purpose systolic array=====<br />
<br />
It is an array of systolic processing elements, which gets adapted to a variety of applications via programming or reconfiguration. Array topologies can be either programmable or reconfigurable. Likewise, array cells are either programmable or reconfigurable. This is referred to as Systolic topologies.<br />
<br />
======Programmable Systolic Array======<br />
<br />
It is an array of programmable systolic elements that operates either in SIMD or MIMD fashion. Either the arrays interconnect or each processing unit is programmable and a program controls dataflow through the elements. Programmable systolic arrays are programmable either at a high level or a low level. At either level, programmable arrays can be categorized as either SIMD or MIMD machines.<br />
<br />
[[Image:systolic_4.png|thumb|right|250px|Figure 5: General organization of MIMD programmable linear systolic arrays [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
<br />
For the latter approach, the workstation downloads a program to each MIMD (Figure 4) systolic cell. Each cell may be loaded with a different program, or all the cells in the array may be loaded with the same program. Each cell's architecture is somewhat similar to the conventional von Neumann architecture: It contains a control unit, an ALU, and local memory. MIMD systolic cells have more local memory than their SIMD counterparts to support the von Neumann-style organization.<br />
<br />
This architecture is defined as Multiple Instruction Multiple Data (MIMD) architecture in [http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf 5]. The architecture has multiple instruction streams for the PEs and a single data stream passing through all the PEs. Thus, it can also be defined as Multiple Instruction Single Data (MISD) architecture. The architecture of Systolic array configuration are controversial as explained in the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Architecture_of_systolic_arrays_as_against_MISD_architecture section 4.1.2.]<br />
<br />
======Reconfigurable Systolic Array======<br />
[[Image:reconfig.jpg|thumb|right|250px|Figure 6: Block Diagram of the RSA Architecture [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
It is an array of systolic elements that can be programmed at the lowest level. Recent gate density advances in FPGA technology have produced a low-level, reconfigurable systolic array architecture that bridges the gap between special-purpose arrays and the more versatile, programmable general-purpose arrays. The FPGA architecture is unusual because a single hardware platform can be logically reconfigured as an exact duplicate of a special-purpose systolic array. <br />
<br />
The RSA circuit design is based on systolic array architecture consisting of PEs interconnected via SWs as depicted in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Reconfig.jpg Figure 6]. The homogeneous characteristic of the Reconfigurable Systolic Array (RSA) architecture, where each reconfigurable processing element (PE) cell is connected to its nearest neighbors via configurable switch (SW) elements, enables array expansion for parallel processing and facilitates time sharing computation of high-throughput data by individual PEs. Both the PEs and SWs can be reconfigured dynamically with the former as an arithmetic processor and the latter as a flexible router linking the neighboring PE cells. The RSA shifts reconfiguration and input signals into the PEs and SWs on separate data bus which enables the circuit to continue its operation while the reconfiguration is in process.<br />
<br />
====Architecture of systolic arrays as against MISD architecture====<br />
[[Image:comp.png|thumb|right|250px|Figure 7.Comparison between Architecture of systolic arrays and MISD]]<br />
<br />
As from the above mentioned configurations of the Systolic Arrays, it is seen that generally the configurations have multiple processing elements executing different instructions from dedicated instruction streams for each processing element. There is a single data stream that connects the adjacent PEs. Thus, systolic array can be defined as an MISD architecture.<br />
<br />
Many authors say that as the data read as input by one processing element is processed data output of the adjacent PE. The data stream cannot be considered as single because all the data paths do not carry the same data to all the PEs. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Comp.png Figure 7] shows the difference between the Data Stream for Systolic Arrays and the MISD architecture. Thus the systolic array should be considered as “Multiple Data” architecture and not Single Data architecture.<br />
<br />
===Fault Tolerant Systems<ref>http://en.wikipedia.org/wiki/Fault-tolerant_computer_system#Types_of_fault_tolerance</ref>===<br />
<br />
The fault tolerant systems are designed to handle the possible failures in software, hardware or interfaces. The hardware faults include hard disk failures, input or output device failures, etc. and the software and interface faults include driver failures; operator errors, installing unexpected software etc. The hardware faults can be detected and identified by implementing redundant hardware and multiple backups. The software faults can be tolerable by removing the program errors by executing the software redundantly or by implementing small programs that take over the tasks that crash or generate errors.<br />
<br />
[[Image:fault.png|thumb|right|250px|Figure 8 MISD as fault tolerant architecture]]<br />
<br />
In computer systems, the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Single_Instruction.2C_Multiple_Data_streams_.28SIMD.29 SIMD], [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD] and [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instruction.2C_Multiple_Data_streams_.28MIMD.29 MIMD] architectures facilitate the implementation of the fault tolerance systems by multiple instruction streams or multiple data streams or both. Fault tolerance on computations can be implemented by multiple processors (likely with different architectures) executing the algorithms on the same set of data. The output of each processor is compared with that of the others and M out of N majority voting method is used to determine the faulty processor. Thus MISD architecture is utilized to get the fault tolerance on critical computations.<br />
<br />
There are various examples of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture] being used as fault tolerant architecture. The major examples being flight control systems, nuclear power plants, satellite systems, super collider experiment systems, etc. Here, the flight control system is explained as an example of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture].<br />
<br />
====The Flight Control System – MISD Example for fault tolerance====<br />
<br />
A [http://en.wikipedia.org/wiki/Fly-by-wire Fly-By-Wire] system is used to replace the manual flight control by an electronic control interface. The movements of the flight control in the cockpit are converted to electronic signals and are transmitted to the actuators by wires. The control computers use the feedback from the sensors to compute and control the movement of the actuators to provide the expected response. These computers also perform the task to stabilize the aircraft and perform other tasks without the knowledge of the pilot. Flight control systems must meet extremely high levels of accuracy and functional integrity.<br />
<br />
There are redundant flight control computers present in the flight control system. If one of the flight-control computers crashes, gets damaged or is affected by electromagnetic pulses, the other computer can overrule the faulty one and hence the flight of the aircraft is unharmed. [[Image:fig13.png|thumb|right|250px|Figure 9: Architecture of triple redundant 777 primary flight computer [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-7 6]]]The number of redundant flight control computers is generally more than two, so that any computer whose results disagree with the others is ruled out to be faulty and is either ignored or rebooted.<br />
<br />
=====Multiple Processors Implementation in Boeing 777<ref>http://www.citemaster.net/getdoc/8767/R8.pdf Y.C. (Bob) Yeh, Boeing Commercial Airplane Group, "Triple-Triple Redundant 777 Primary Flight Computer" </ref>=====<br />
<br />
In modern computers, the redundant flight control computations are carried out by multiprocessor systems. The triple redundant 777 primary flight computer, has the architecture as shown in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Fig13.png Figure 9].<br />
<br />
The system has three primary flight control computers, each of them having three lanes with different processors. The flight control program is compiled for each of the processors which get the input data from the same data bus but drive the output on their individual control bus. Thus each processor executes different instructions but they process the same data. Thus, it is the best suited example of Multiple Instruction Single Data (MISD) architecture.<br />
<br />
The three processors selected for the flight control system of [http://en.wikipedia.org/wiki/Boeing_777 Boeing 777] were [http://en.wikipedia.org/wiki/Intel_80486 Intel 80486], [http://en.wikipedia.org/wiki/Motorola_68040 Motorola 68040] and [http://en.wikipedia.org/wiki/AMD_Am29000 AMD 29050]. The dissimilar processors lead to dissimilar interface hardware circuits and compilers. Each lane of the flight control computer is data synchronized with the other lanes so that all of the lanes read the same frame of data from the flight sensors. As the outputs of each lane can be different, the median value of the outputs is used to select the output of the lane to be considered. The lane which has the median value select hardware selected is said to be in “command mode” whereas the other lanes are said to be in “monitoring mode”. It receives the data from the other Primary Flight Computer (PFC) lanes and performs a median select of the outputs. This provides a fault blocking mechanism before the fault detection and identification by the cross-lane monitoring system. Thus, the MISD based multi computer architecture is capable of detecting generic errors in compilers or in complex hardware devices providing assurance beyond reasonable doubt of the dependability of the Fly-By-Wire system.<br />
<br />
The above mentioned system clearly has individual Instruction Streams as the architecture of each processor is different, thus different instruction sets and different instruction streams. These processors have frame synchronized input data which means they have same set of data to work upon which is fed from a single data stream. Thus the flight control system can be classified under MISD architecture.<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System===<br />
<br />
The image pre-processing system of this Multiprocessor [9] , the pre-image processing time needed to be made shorter. To achieve the aim of reducing the pre-processing time, different types of image processing architectures for ‘C80 were presented. For example MIMD architecture is used for image histogram equalization; SIMD architecture for median filtering; MISD architecture for directional edge detection.<br />
<br />
The MISD architecture can be explained as follows. Here, each processor performs different operations on the same data set. It is mentioned at many places that MISD architecture was defined only for the sake of completeness of classification due to difficulty in practice to find a computer based on MISD architecture [10]. But in case of Image Processing, this assumption need not be true as here making many different operations on same image is rare and hence MISD architecture is a best fit for such cases. As an example of implementation of image processing operations for the MISD architecture the edge detection operation will be considered. The edges should be detected in four different directions (north, south, west and east) by the medium of four different filter masks. Each of the parallel processors of the TMS320C80 calculates one of the directional edges. Finally one obtains four images with detected edges, each in different direction.<br />
<br />
The image processing in computer is very often a multistage process composed of many stages. For example, an algorithm detecting contours of the objects in an image. The algorithm for achieving the before mentioned is composed of four basic image processing operation. The image acquired from the camera in the first step undergoes low-pass filtering, during which it is smoothed and noise is also eliminated. In the next step image is thresholding is performed and each pixel gets black or white depending on that if its grey level is below or above threshold value. The resulting image is a binary image from which edges are detected by the usage of the Laplace filter. In the last step the image with detected contours undergoes logic filtering in the purpose of elimination of single black pixels placed on the white background. For implementing the above processes parallel, a five-stage pipeline was used. The first stage of this pipeline constitutes the master processor which accounts for control of the image acquisition process. Further pipe line stages were implemented for 4 parallel processors as mentioned in table 1.<br />
<br />
Stage Processor Operation<br />
1 MP Image acquisition<br />
2 PP0 Low-pass Filtering<br />
3 PP1 Thresholding<br />
4 PP2 Laplace Filtering<br />
5 PP3 Logic Filtering<br />
Table 1: Pipeline structure for image edge detection [9]<br />
<br />
=='''References'''==<br />
<references/><br />
<br />
=='''Glossary'''==<br />
*'''CMU''': Carnegie Mellon University<br />
<br />
*'''CU''': Control Unit<br />
<br />
*'''DS''': Data Stream<br />
<br />
*'''Fly-By-Wire''': System that replaces the conventional manual flight controls of an aircraft with an electronic interface<br />
<br />
*'''FPGA''': Field Programmable Gate Array<br />
<br />
*'''Heterogeneous Systems''': A multiprocessor system with different kind of processors<br />
<br />
*'''Homogeneous System''': A multiprocessor system with same kind of processors<br />
<br />
*'''ILP''': Instruction Level Parallelism<br />
<br />
*'''IS''': Instruction Stream<br />
<br />
*'''MIMD''': Multiple Instruction Multiple Data<br />
<br />
*'''MISD''': Multiple Instruction Single Data<br />
<br />
*'''PE''': Processing Element<br />
<br />
*'''PFC''': Primary Flight Computer<br />
<br />
*'''SAPO''': Short for Samočinný počítač<br />
<br />
*'''SIMD''': Single Instruction Multiple Data<br />
<br />
*'''SISD''': Single Instruction Single Data<br />
<br />
*'''VLSI''': Very Large Scale Integration</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/1a_sp&diff=71781CSC/ECE 506 Spring 2013/1a sp2013-02-04T23:00:48Z<p>Schakra8: </p>
<hr />
<div>==Multiple Instruction Single Data (MISD) Architecture==<br />
<br />
Multiple Instruction Single Data, or MISD, architecture is one of the four general categories of multi-processor or multi-core architectures described by Flynn's Taxonomy.<br />
<br />
==MISD Architectures within the Context of Flynn’s Taxonomy<ref>http://en.wikipedia.org/wiki/Flynn's_taxonomy</ref><ref>http://www.phy.ornl.gov/csep/ca/node11.html</ref>==<br />
<br />
Flynn's Taxonomy categorizes parallel computing architectures based on the number of information streams that flowed into the processor.<br />
According to Flynn's Taxonomy, there are two kinds of logical streams of information that flows into a parallel computer, the Instruction Stream and the Data stream. Note that for the purposes of classification using Flynn’s Taxonomy, it does not matter whether or not the streams are physically independent of each other on the hardware, only that the streams can be logically treated as independent.<br />
Based on how many of each type of stream flows into the parallel computer (single vs. multiple), Flynn's Taxonomy divides parallel architectures into four categories. They are Single Instruction Single Data (SISD), Single Instruction Multiple Data (SIMD), Multiple Instruction Single Data (MISD), and Multiple Instruction Multiple Data (MIMD).<br />
<br />
The four classifications defined by Flynn are based upon the number of concurrent instruction (or control) and data streams available in the architecture are<ref>https://computing.llnl.gov/tutorials/parallel_comp/#Flynn</ref><br />
<br />
[[Image:Flynn's Taxonomy.PNG|thumb|center|400px|Figure 1. [http://en.wikipedia.org/wiki/Michael_J._Flynn Flynn]'s Taxonomy [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
==Overview of the MISD Architecture==<br />
<br />
In order to be categorized as MISD architecture, there must be at least two Instruction streams and there can only be one Data stream as shown on Figure 2. As per the description according to Flynn’s Taxonomy, each processing elements should receive the same data and execute different instructions on the same data simultaneously.<br />
<br />
[[Image:MISD.PNG|thumb|right|100px|Figure 2. MISD Architecture [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
There are very few implementations of the classical MISD architecture as described by Flynn’s Taxonomy, none of which are commercial, general purpose parallel computers. Instead, MISD architectures are mainly used to create custom hardware in order to solve specific problems. The majority of the implementations have been created are in academia.<br />
Over the years, several architectures have been described as MISD even though the architectures do not completely follow the description in Flynn’s Taxonomy. One common example is the Pipeline architecture where the data is passed from processing element to processing element and each processing element executes a different instruction. It is often argued that this is not a true MISD architecture as the data changes as it moves through the pipeline. Similarly, other architectures or applications exist which may be called MISD by some while others disagree. The following section looks at some of those implementations.<br />
<br />
Pipeline architectures belong to this type, though a purist might say that the data is different after processing by each stage in the pipeline. Fault-tolerant computers executing the same instructions redundantly in order to detect and mask errors, in a manner known as task replication, may be considered to belong to this type. Not many instances of this architecture exist, as MIMD and SIMD are often more appropriate for common data parallel techniques. Specifically, they allow better scaling and use of computational resources than MISD does. <br />
<br />
However, one prominent example of MISD in computing is the Space Shuttle flight control computers. Another example of this machine is the systolic array, such as the [http://www.cs.cmu.edu/~iwarp/ CMU iWrap] [BORKAR et al., 1990]. All the elements in this array are controlled by a global clock. On each cycle, an element will read a piece of data from one of its neighbors, perform a simple operation (e.g. add the incoming element to a stored value), and prepare a value to be written to a neighbor on the next step.<br />
<br />
==Implementations of MISD architecture==<br />
<br />
Some major implementations of MISD are in the fields of systolic arrays, pattern matching, flight control systems and fault tolerance systems. These MISD implementations are discussed below.<br />
<br />
===Systolic Array===<br />
<br />
A systolic array is an arrangement of processors in an array where data flows synchronously across the array between neighbors, usually with different data flowing in different directions. Each Processor at each step takes in data from one or more neighbors, processes it and, in the next step, outputs results in the opposite direction.<br />
<br />
The systolic array paradigm, data-stream-driven by data counters, is the counterpart of the [http://en.wikipedia.org/wiki/Von_Neumann_model von Neumann paradigm], instruction-stream-driven by a program counter. Because a systolic array usually sends and receives multiple data streams, and multiple data counters are needed to generate these data streams, it supports data parallelism. The name derives from analogy with the regular pumping of blood by the heart.<ref>http://en.wikipedia.org/wiki/Systolic_array</ref><br />
<br />
====Type of Systolic Arrays<ref>http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf General Purpose Systolic Arrays </ref>====<br />
<br />
=====Special-purpose systolic array=====<br />
[[Image:systolic_1.png|thumb|right|250px|Figure 3: The algorithm for the sum of a scalar product, computed in systolic element [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
[[Image:systolic_2.png|thumb|right|250px|Figure 7: The systolic product of two 3x3 matrices [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
An array of hardwired systolic processing elements tailored for a specific application. Typically, many tens or hundreds of cells fit on a single chip. One of the major applications of special-purpose systolic array is in matrix operations. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Systolic_1.png Figure 3] illustrates the algorithm for the sum of a scalar product, computed in a single systolic element. Here, a’s and b’s are synchronously shifted through the processing element to be available for next element. These data synchronously exits the processing element unmodified for the next element. The sum of the products is then shifted out of the accumulator.<br />
<br />
=====General-purpose systolic array=====<br />
<br />
It is an array of systolic processing elements, which gets adapted to a variety of applications via programming or reconfiguration. Array topologies can be either programmable or reconfigurable. Likewise, array cells are either programmable or reconfigurable. This is referred to as Systolic topologies.<br />
<br />
======Programmable Systolic Array======<br />
<br />
It is an array of programmable systolic elements that operates either in SIMD or MIMD fashion. Either the arrays interconnect or each processing unit is programmable and a program controls dataflow through the elements. Programmable systolic arrays are programmable either at a high level or a low level. At either level, programmable arrays can be categorized as either SIMD or MIMD machines.<br />
<br />
[[Image:systolic_4.png|thumb|right|250px|Figure 5: General organization of MIMD programmable linear systolic arrays [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
<br />
For the latter approach, the workstation downloads a program to each MIMD (Figure 4) systolic cell. Each cell may be loaded with a different program, or all the cells in the array may be loaded with the same program. Each cell's architecture is somewhat similar to the conventional von Neumann architecture: It contains a control unit, an ALU, and local memory. MIMD systolic cells have more local memory than their SIMD counterparts to support the von Neumann-style organization.<br />
<br />
This architecture is defined as Multiple Instruction Multiple Data (MIMD) architecture in [http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf 5]. The architecture has multiple instruction streams for the PEs and a single data stream passing through all the PEs. Thus, it can also be defined as Multiple Instruction Single Data (MISD) architecture. The architecture of Systolic array configuration are controversial as explained in the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Architecture_of_systolic_arrays_as_against_MISD_architecture section 4.1.2.]<br />
<br />
======Reconfigurable Systolic Array======<br />
[[Image:reconfig.jpg|thumb|right|250px|Figure 6: Block Diagram of the RSA Architecture [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
It is an array of systolic elements that can be programmed at the lowest level. Recent gate density advances in FPGA technology have produced a low-level, reconfigurable systolic array architecture that bridges the gap between special-purpose arrays and the more versatile, programmable general-purpose arrays. The FPGA architecture is unusual because a single hardware platform can be logically reconfigured as an exact duplicate of a special-purpose systolic array. <br />
<br />
The RSA circuit design is based on systolic array architecture consisting of PEs interconnected via SWs as depicted in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Reconfig.jpg Figure 6]. The homogeneous characteristic of the Reconfigurable Systolic Array (RSA) architecture, where each reconfigurable processing element (PE) cell is connected to its nearest neighbors via configurable switch (SW) elements, enables array expansion for parallel processing and facilitates time sharing computation of high-throughput data by individual PEs. Both the PEs and SWs can be reconfigured dynamically with the former as an arithmetic processor and the latter as a flexible router linking the neighboring PE cells. The RSA shifts reconfiguration and input signals into the PEs and SWs on separate data bus which enables the circuit to continue its operation while the reconfiguration is in process.<br />
<br />
====Architecture of systolic arrays as against MISD architecture====<br />
[[Image:comp.png|thumb|right|250px|Figure 7.Comparison between Architecture of systolic arrays and MISD]]<br />
<br />
As from the above mentioned configurations of the Systolic Arrays, it is seen that generally the configurations have multiple processing elements executing different instructions from dedicated instruction streams for each processing element. There is a single data stream that connects the adjacent PEs. Thus, systolic array can be defined as an MISD architecture.<br />
<br />
Many authors say that as the data read as input by one processing element is processed data output of the adjacent PE. The data stream cannot be considered as single because all the data paths do not carry the same data to all the PEs. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Comp.png Figure 7] shows the difference between the Data Stream for Systolic Arrays and the MISD architecture. Thus the systolic array should be considered as “Multiple Data” architecture and not Single Data architecture.<br />
<br />
===Fault Tolerant Systems<ref>http://en.wikipedia.org/wiki/Fault-tolerant_computer_system#Types_of_fault_tolerance</ref>===<br />
<br />
The fault tolerant systems are designed to handle the possible failures in software, hardware or interfaces. The hardware faults include hard disk failures, input or output device failures, etc. and the software and interface faults include driver failures; operator errors, installing unexpected software etc. The hardware faults can be detected and identified by implementing redundant hardware and multiple backups. The software faults can be tolerable by removing the program errors by executing the software redundantly or by implementing small programs that take over the tasks that crash or generate errors.<br />
<br />
[[Image:fault.png|thumb|right|250px|Figure 8 MISD as fault tolerant architecture]]<br />
<br />
In computer systems, the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Single_Instruction.2C_Multiple_Data_streams_.28SIMD.29 SIMD], [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD] and [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instruction.2C_Multiple_Data_streams_.28MIMD.29 MIMD] architectures facilitate the implementation of the fault tolerance systems by multiple instruction streams or multiple data streams or both. Fault tolerance on computations can be implemented by multiple processors (likely with different architectures) executing the algorithms on the same set of data. The output of each processor is compared with that of the others and M out of N majority voting method is used to determine the faulty processor. Thus MISD architecture is utilized to get the fault tolerance on critical computations.<br />
<br />
There are various examples of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture] being used as fault tolerant architecture. The major examples being flight control systems, nuclear power plants, satellite systems, super collider experiment systems, etc. Here, the flight control system is explained as an example of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture].<br />
<br />
====The Flight Control System – MISD Example for fault tolerance====<br />
<br />
A [http://en.wikipedia.org/wiki/Fly-by-wire Fly-By-Wire] system is used to replace the manual flight control by an electronic control interface. The movements of the flight control in the cockpit are converted to electronic signals and are transmitted to the actuators by wires. The control computers use the feedback from the sensors to compute and control the movement of the actuators to provide the expected response. These computers also perform the task to stabilize the aircraft and perform other tasks without the knowledge of the pilot. Flight control systems must meet extremely high levels of accuracy and functional integrity.<br />
<br />
There are redundant flight control computers present in the flight control system. If one of the flight-control computers crashes, gets damaged or is affected by electromagnetic pulses, the other computer can overrule the faulty one and hence the flight of the aircraft is unharmed. [[Image:fig13.png|thumb|right|250px|Figure 9: Architecture of triple redundant 777 primary flight computer [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-7 6]]]The number of redundant flight control computers is generally more than two, so that any computer whose results disagree with the others is ruled out to be faulty and is either ignored or rebooted.<br />
<br />
=====Multiple Processors Implementation in Boeing 777<ref>http://www.citemaster.net/getdoc/8767/R8.pdf Y.C. (Bob) Yeh, Boeing Commercial Airplane Group, "Triple-Triple Redundant 777 Primary Flight Computer" </ref>=====<br />
<br />
In modern computers, the redundant flight control computations are carried out by multiprocessor systems. The triple redundant 777 primary flight computer, has the architecture as shown in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Fig13.png Figure 9].<br />
<br />
The system has three primary flight control computers, each of them having three lanes with different processors. The flight control program is compiled for each of the processors which get the input data from the same data bus but drive the output on their individual control bus. Thus each processor executes different instructions but they process the same data. Thus, it is the best suited example of Multiple Instruction Single Data (MISD) architecture.<br />
<br />
The three processors selected for the flight control system of [http://en.wikipedia.org/wiki/Boeing_777 Boeing 777] were [http://en.wikipedia.org/wiki/Intel_80486 Intel 80486], [http://en.wikipedia.org/wiki/Motorola_68040 Motorola 68040] and [http://en.wikipedia.org/wiki/AMD_Am29000 AMD 29050]. The dissimilar processors lead to dissimilar interface hardware circuits and compilers. Each lane of the flight control computer is data synchronized with the other lanes so that all of the lanes read the same frame of data from the flight sensors. As the outputs of each lane can be different, the median value of the outputs is used to select the output of the lane to be considered. The lane which has the median value select hardware selected is said to be in “command mode” whereas the other lanes are said to be in “monitoring mode”. It receives the data from the other Primary Flight Computer (PFC) lanes and performs a median select of the outputs. This provides a fault blocking mechanism before the fault detection and identification by the cross-lane monitoring system. Thus, the MISD based multi computer architecture is capable of detecting generic errors in compilers or in complex hardware devices providing assurance beyond reasonable doubt of the dependability of the Fly-By-Wire system.<br />
<br />
The above mentioned system clearly has individual Instruction Streams as the architecture of each processor is different, thus different instruction sets and different instruction streams. These processors have frame synchronized input data which means they have same set of data to work upon which is fed from a single data stream. Thus the flight control system can be classified under MISD architecture.<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System===<br />
<br />
The image pre-processing system of this Multiprocessor [9] , the pre-image processing time needed to be made shorter. To achieve the aim of reducing the pre-processing time, different types of image processing architectures for ‘C80 were presented. For example MIMD architecture is used for image histogram equalization; SIMD architecture for median filtering; MISD architecture for directional edge detection.<br />
<br />
The MISD architecture can be explained as follows. Here, each processor performs different operations on the same data set. It is mentioned at many places that MISD architecture was defined only for the sake of completeness of classification due to difficulty in practice to find a computer based on MISD architecture [10]. But in case of Image Processing, this assumption need not be true as here making many different operations on same image is rare and hence MISD architecture is a best fit for such cases. As an example of implementation of image processing operations for the MISD architecture the edge detection operation will be considered. The edges should be detected in four different directions (north, south, west and east) by the medium of four different filter masks. Each of the parallel processors of the TMS320C80 calculates one of the directional edges. Finally one obtains four images with detected edges, each in different direction.<br />
<br />
The image processing in computer is very often a multistage process composed of many stages. For example, an algorithm detecting contours of the objects in an image. The algorithm for achieving the before mentioned is composed of four basic image processing operation. The image acquired from the camera in the first step undergoes low-pass filtering, during which it is smoothed and noise is also eliminated. In the next step image is thresholding is performed and each pixel gets black or white depending on that if its grey level is below or above threshold value. The resulting image is a binary image from which edges are detected by the usage of the Laplace filter. In the last step the image with detected contours undergoes logic filtering in the purpose of elimination of single black pixels placed on the white background. For implementing the above processes parallel, a five-stage pipeline was used. The first stage of this pipeline constitutes the master processor which accounts for control of the image acquisition process. Further pipe line stages were implemented for 4 parallel processors as mentioned in table 1.<br />
<br />
Stage Processor Operation<br />
1 MP Image acquisition<br />
2 PP0 Low-pass Filtering<br />
3 PP1 Thresholding<br />
4 PP2 Laplace Filtering<br />
5 PP3 Logic Filtering<br />
Table 1: Pipeline structure for image edge detection [9]<br />
<br />
=='''References'''==<br />
<references/><br />
<br />
=='''Glossary'''==<br />
*'''CMU''': Carnegie Mellon University<br />
<br />
*'''CU''': Control Unit<br />
<br />
*'''DS''': Data Stream<br />
<br />
*'''Fly-By-Wire''': System that replaces the conventional manual flight controls of an aircraft with an electronic interface<br />
<br />
*'''FPGA''': Field Programmable Gate Array<br />
<br />
*'''Heterogeneous Systems''': A multiprocessor system with different kind of processors<br />
<br />
*'''Homogeneous System''': A multiprocessor system with same kind of processors<br />
<br />
*'''ILP''': Instruction Level Parallelism<br />
<br />
*'''IS''': Instruction Stream<br />
<br />
*'''MIMD''': Multiple Instruction Multiple Data<br />
<br />
*'''MISD''': Multiple Instruction Single Data<br />
<br />
*'''PE''': Processing Element<br />
<br />
*'''PFC''': Primary Flight Computer<br />
<br />
*'''SAPO''': Short for Samočinný počítač<br />
<br />
*'''SIMD''': Single Instruction Multiple Data<br />
<br />
*'''SISD''': Single Instruction Single Data<br />
<br />
*'''VLSI''': Very Large Scale Integration</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/1a_sp&diff=71780CSC/ECE 506 Spring 2013/1a sp2013-02-04T22:58:41Z<p>Schakra8: /* The Flight Control System – MISD Example for fault tolerance */</p>
<hr />
<div>==Multiple Instruction Single Data (MISD) Architecture==<br />
<br />
Multiple Instruction Single Data, or MISD, architecture is one of the four general categories of multi-processor or multi-core architectures described by Flynn's Taxonomy.<br />
<br />
==MISD Architectures within the Context of Flynn’s Taxonomy<ref>http://en.wikipedia.org/wiki/Flynn's_taxonomy</ref><ref>http://www.phy.ornl.gov/csep/ca/node11.html</ref>==<br />
<br />
Flynn's Taxonomy categorizes parallel computing architectures based on the number of information streams that flowed into the processor.<br />
According to Flynn's Taxonomy, there are two kinds of logical streams of information that flows into a parallel computer, the Instruction Stream and the Data stream. Note that for the purposes of classification using Flynn’s Taxonomy, it does not matter whether or not the streams are physically independent of each other on the hardware, only that the streams can be logically treated as independent.<br />
Based on how many of each type of stream flows into the parallel computer (single vs. multiple), Flynn's Taxonomy divides parallel architectures into four categories. They are Single Instruction Single Data (SISD), Single Instruction Multiple Data (SIMD), Multiple Instruction Single Data (MISD), and Multiple Instruction Multiple Data (MIMD).<br />
<br />
The four classifications defined by Flynn are based upon the number of concurrent instruction (or control) and data streams available in the architecture are<ref>https://computing.llnl.gov/tutorials/parallel_comp/#Flynn</ref><br />
<br />
[[Image:Flynn's Taxonomy.PNG|thumb|center|400px|Figure 1. [http://en.wikipedia.org/wiki/Michael_J._Flynn Flynn]'s Taxonomy [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
==Overview of the MISD Architecture==<br />
<br />
In order to be categorized as MISD architecture, there must be at least two Instruction streams and there can only be one Data stream as shown on Figure 2. As per the description according to Flynn’s Taxonomy, each processing elements should receive the same data and execute different instructions on the same data simultaneously.<br />
<br />
[[Image:MISD.PNG|thumb|right|100px|Figure 2. MISD Architecture [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
There are very few implementations of the classical MISD architecture as described by Flynn’s Taxonomy, none of which are commercial, general purpose parallel computers. Instead, MISD architectures are mainly used to create custom hardware in order to solve specific problems. The majority of the implementations have been created are in academia.<br />
Over the years, several architectures have been described as MISD even though the architectures do not completely follow the description in Flynn’s Taxonomy. One common example is the Pipeline architecture where the data is passed from processing element to processing element and each processing element executes a different instruction. It is often argued that this is not a true MISD architecture as the data changes as it moves through the pipeline. Similarly, other architectures or applications exist which may be called MISD by some while others disagree. The following section looks at some of those implementations.<br />
<br />
Pipeline architectures belong to this type, though a purist might say that the data is different after processing by each stage in the pipeline. Fault-tolerant computers executing the same instructions redundantly in order to detect and mask errors, in a manner known as task replication, may be considered to belong to this type. Not many instances of this architecture exist, as MIMD and SIMD are often more appropriate for common data parallel techniques. Specifically, they allow better scaling and use of computational resources than MISD does. <br />
<br />
However, one prominent example of MISD in computing is the Space Shuttle flight control computers. Another example of this machine is the systolic array, such as the [http://www.cs.cmu.edu/~iwarp/ CMU iWrap] [BORKAR et al., 1990]. All the elements in this array are controlled by a global clock. On each cycle, an element will read a piece of data from one of its neighbors, perform a simple operation (e.g. add the incoming element to a stored value), and prepare a value to be written to a neighbor on the next step.<br />
<br />
==Implementations of MISD architecture==<br />
<br />
Some major implementations of MISD are in the fields of systolic arrays, pattern matching, flight control systems and fault tolerance systems. These MISD implementations are discussed below.<br />
<br />
===Systolic Array===<br />
<br />
A systolic array is an arrangement of processors in an array where data flows synchronously across the array between neighbors, usually with different data flowing in different directions. Each Processor at each step takes in data from one or more neighbors, processes it and, in the next step, outputs results in the opposite direction.<br />
<br />
The systolic array paradigm, data-stream-driven by data counters, is the counterpart of the [http://en.wikipedia.org/wiki/Von_Neumann_model von Neumann paradigm], instruction-stream-driven by a program counter. Because a systolic array usually sends and receives multiple data streams, and multiple data counters are needed to generate these data streams, it supports data parallelism. The name derives from analogy with the regular pumping of blood by the heart.<ref>http://en.wikipedia.org/wiki/Systolic_array</ref><br />
<br />
====Type of Systolic Arrays<ref>http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf General Purpose Systolic Arrays </ref>====<br />
<br />
=====Special-purpose systolic array=====<br />
[[Image:systolic_1.png|thumb|right|250px|Figure 3: The algorithm for the sum of a scalar product, computed in systolic element [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
[[Image:systolic_2.png|thumb|right|250px|Figure 7: The systolic product of two 3x3 matrices [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
An array of hardwired systolic processing elements tailored for a specific application. Typically, many tens or hundreds of cells fit on a single chip. One of the major applications of special-purpose systolic array is in matrix operations. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Systolic_1.png Figure 3] illustrates the algorithm for the sum of a scalar product, computed in a single systolic element. Here, a’s and b’s are synchronously shifted through the processing element to be available for next element. These data synchronously exits the processing element unmodified for the next element. The sum of the products is then shifted out of the accumulator.<br />
<br />
=====General-purpose systolic array=====<br />
<br />
It is an array of systolic processing elements, which gets adapted to a variety of applications via programming or reconfiguration. Array topologies can be either programmable or reconfigurable. Likewise, array cells are either programmable or reconfigurable. This is referred to as Systolic topologies.<br />
<br />
======Programmable Systolic Array======<br />
<br />
It is an array of programmable systolic elements that operates either in SIMD or MIMD fashion. Either the arrays interconnect or each processing unit is programmable and a program controls dataflow through the elements. Programmable systolic arrays are programmable either at a high level or a low level. At either level, programmable arrays can be categorized as either SIMD or MIMD machines.<br />
<br />
[[Image:systolic_4.png|thumb|right|250px|Figure 5: General organization of MIMD programmable linear systolic arrays [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
<br />
For the latter approach, the workstation downloads a program to each MIMD (Figure 4) systolic cell. Each cell may be loaded with a different program, or all the cells in the array may be loaded with the same program. Each cell's architecture is somewhat similar to the conventional von Neumann architecture: It contains a control unit, an ALU, and local memory. MIMD systolic cells have more local memory than their SIMD counterparts to support the von Neumann-style organization.<br />
<br />
This architecture is defined as Multiple Instruction Multiple Data (MIMD) architecture in [http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf 5]. The architecture has multiple instruction streams for the PEs and a single data stream passing through all the PEs. Thus, it can also be defined as Multiple Instruction Single Data (MISD) architecture. The architecture of Systolic array configuration are controversial as explained in the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Architecture_of_systolic_arrays_as_against_MISD_architecture section 4.1.2.]<br />
<br />
======Reconfigurable Systolic Array======<br />
[[Image:reconfig.jpg|thumb|right|250px|Figure 6: Block Diagram of the RSA Architecture [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
It is an array of systolic elements that can be programmed at the lowest level. Recent gate density advances in FPGA technology have produced a low-level, reconfigurable systolic array architecture that bridges the gap between special-purpose arrays and the more versatile, programmable general-purpose arrays. The FPGA architecture is unusual because a single hardware platform can be logically reconfigured as an exact duplicate of a special-purpose systolic array. <br />
<br />
The RSA circuit design is based on systolic array architecture consisting of PEs interconnected via SWs as depicted in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Reconfig.jpg Figure 6]. The homogeneous characteristic of the Reconfigurable Systolic Array (RSA) architecture, where each reconfigurable processing element (PE) cell is connected to its nearest neighbors via configurable switch (SW) elements, enables array expansion for parallel processing and facilitates time sharing computation of high-throughput data by individual PEs. Both the PEs and SWs can be reconfigured dynamically with the former as an arithmetic processor and the latter as a flexible router linking the neighboring PE cells. The RSA shifts reconfiguration and input signals into the PEs and SWs on separate data bus which enables the circuit to continue its operation while the reconfiguration is in process.<br />
<br />
====Architecture of systolic arrays as against MISD architecture====<br />
[[Image:comp.png|thumb|right|250px|Figure 7.Comparison between Architecture of systolic arrays and MISD]]<br />
<br />
As from the above mentioned configurations of the Systolic Arrays, it is seen that generally the configurations have multiple processing elements executing different instructions from dedicated instruction streams for each processing element. There is a single data stream that connects the adjacent PEs. Thus, systolic array can be defined as an MISD architecture.<br />
<br />
Many authors say that as the data read as input by one processing element is processed data output of the adjacent PE. The data stream cannot be considered as single because all the data paths do not carry the same data to all the PEs. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Comp.png Figure 7] shows the difference between the Data Stream for Systolic Arrays and the MISD architecture. Thus the systolic array should be considered as “Multiple Data” architecture and not Single Data architecture.<br />
<br />
===Fault Tolerant Systems<ref>http://en.wikipedia.org/wiki/Fault-tolerant_computer_system#Types_of_fault_tolerance</ref>===<br />
<br />
The fault tolerant systems are designed to handle the possible failures in software, hardware or interfaces. The hardware faults include hard disk failures, input or output device failures, etc. and the software and interface faults include driver failures; operator errors, installing unexpected software etc. The hardware faults can be detected and identified by implementing redundant hardware and multiple backups. The software faults can be tolerable by removing the program errors by executing the software redundantly or by implementing small programs that take over the tasks that crash or generate errors.<br />
<br />
[[Image:fault.png|thumb|right|250px|Figure 12 MISD as fault tolerant architecture]]<br />
<br />
In computer systems, the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Single_Instruction.2C_Multiple_Data_streams_.28SIMD.29 SIMD], [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD] and [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instruction.2C_Multiple_Data_streams_.28MIMD.29 MIMD] architectures facilitate the implementation of the fault tolerance systems by multiple instruction streams or multiple data streams or both. Fault tolerance on computations can be implemented by multiple processors (likely with different architectures) executing the algorithms on the same set of data. The output of each processor is compared with that of the others and M out of N majority voting method is used to determine the faulty processor. Thus MISD architecture is utilized to get the fault tolerance on critical computations.<br />
<br />
There are various examples of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture] being used as fault tolerant architecture. The major examples being flight control systems, nuclear power plants, satellite systems, super collider experiment systems, etc. Here, the flight control system is explained as an example of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture].<br />
<br />
====The Flight Control System – MISD Example for fault tolerance====<br />
<br />
A [http://en.wikipedia.org/wiki/Fly-by-wire Fly-By-Wire] system is used to replace the manual flight control by an electronic control interface. The movements of the flight control in the cockpit are converted to electronic signals and are transmitted to the actuators by wires. The control computers use the feedback from the sensors to compute and control the movement of the actuators to provide the expected response. These computers also perform the task to stabilize the aircraft and perform other tasks without the knowledge of the pilot. Flight control systems must meet extremely high levels of accuracy and functional integrity.<br />
<br />
There are redundant flight control computers present in the flight control system. If one of the flight-control computers crashes, gets damaged or is affected by electromagnetic pulses, the other computer can overrule the faulty one and hence the flight of the aircraft is unharmed. [[Image:fig13.png|thumb|right|250px|Figure 9: Architecture of triple redundant 777 primary flight computer [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-7 6]]]The number of redundant flight control computers is generally more than two, so that any computer whose results disagree with the others is ruled out to be faulty and is either ignored or rebooted.[[Image:fig14.png|thumb|right|250px|Figure 14: PFC with instruction and data streams]]<br />
<br />
=====Multiple Processors Implementation in Boeing 777<ref>http://www.citemaster.net/getdoc/8767/R8.pdf Y.C. (Bob) Yeh, Boeing Commercial Airplane Group, "Triple-Triple Redundant 777 Primary Flight Computer" </ref>=====<br />
<br />
In modern computers, the redundant flight control computations are carried out by multiprocessor systems. The triple redundant 777 primary flight computer, has the architecture as shown in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Fig13.png Figure 9].<br />
<br />
The system has three primary flight control computers, each of them having three lanes with different processors. The flight control program is compiled for each of the processors which get the input data from the same data bus but drive the output on their individual control bus. Thus each processor executes different instructions but they process the same data. Thus, it is the best suited example of Multiple Instruction Single Data (MISD) architecture.<br />
<br />
The three processors selected for the flight control system of [http://en.wikipedia.org/wiki/Boeing_777 Boeing 777] were [http://en.wikipedia.org/wiki/Intel_80486 Intel 80486], [http://en.wikipedia.org/wiki/Motorola_68040 Motorola 68040] and [http://en.wikipedia.org/wiki/AMD_Am29000 AMD 29050]. The dissimilar processors lead to dissimilar interface hardware circuits and compilers. Each lane of the flight control computer is data synchronized with the other lanes so that all of the lanes read the same frame of data from the flight sensors. As the outputs of each lane can be different, the median value of the outputs is used to select the output of the lane to be considered. The lane which has the median value select hardware selected is said to be in “command mode” whereas the other lanes are said to be in “monitoring mode”. It receives the data from the other Primary Flight Computer (PFC) lanes and performs a median select of the outputs. This provides a fault blocking mechanism before the fault detection and identification by the cross-lane monitoring system. Thus, the MISD based multi computer architecture is capable of detecting generic errors in compilers or in complex hardware devices providing assurance beyond reasonable doubt of the dependability of the Fly-By-Wire system.<br />
<br />
The above mentioned system clearly has individual Instruction Streams as the architecture of each processor is different, thus different instruction sets and different instruction streams. These processors have frame synchronized input data which means they have same set of data to work upon which is fed from a single data stream. Thus the flight control system can be classified under MISD architecture.<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System===<br />
<br />
The image pre-processing system of this Multiprocessor [9] , the pre-image processing time needed to be made shorter. To achieve the aim of reducing the pre-processing time, different types of image processing architectures for ‘C80 were presented. For example MIMD architecture is used for image histogram equalization; SIMD architecture for median filtering; MISD architecture for directional edge detection.<br />
<br />
The MISD architecture can be explained as follows. Here, each processor performs different operations on the same data set. It is mentioned at many places that MISD architecture was defined only for the sake of completeness of classification due to difficulty in practice to find a computer based on MISD architecture [10]. But in case of Image Processing, this assumption need not be true as here making many different operations on same image is rare and hence MISD architecture is a best fit for such cases. As an example of implementation of image processing operations for the MISD architecture the edge detection operation will be considered. The edges should be detected in four different directions (north, south, west and east) by the medium of four different filter masks. Each of the parallel processors of the TMS320C80 calculates one of the directional edges. Finally one obtains four images with detected edges, each in different direction.<br />
<br />
The image processing in computer is very often a multistage process composed of many stages. For example, an algorithm detecting contours of the objects in an image. The algorithm for achieving the before mentioned is composed of four basic image processing operation. The image acquired from the camera in the first step undergoes low-pass filtering, during which it is smoothed and noise is also eliminated. In the next step image is thresholding is performed and each pixel gets black or white depending on that if its grey level is below or above threshold value. The resulting image is a binary image from which edges are detected by the usage of the Laplace filter. In the last step the image with detected contours undergoes logic filtering in the purpose of elimination of single black pixels placed on the white background. For implementing the above processes parallel, a five-stage pipeline was used. The first stage of this pipeline constitutes the master processor which accounts for control of the image acquisition process. Further pipe line stages were implemented for 4 parallel processors as mentioned in table 1.<br />
<br />
Stage Processor Operation<br />
1 MP Image acquisition<br />
2 PP0 Low-pass Filtering<br />
3 PP1 Thresholding<br />
4 PP2 Laplace Filtering<br />
5 PP3 Logic Filtering<br />
Table 1: Pipeline structure for image edge detection [9]<br />
<br />
=='''References'''==<br />
<references/><br />
<br />
=='''Glossary'''==<br />
*'''CMU''': Carnegie Mellon University<br />
<br />
*'''CU''': Control Unit<br />
<br />
*'''DS''': Data Stream<br />
<br />
*'''Fly-By-Wire''': System that replaces the conventional manual flight controls of an aircraft with an electronic interface<br />
<br />
*'''FPGA''': Field Programmable Gate Array<br />
<br />
*'''Heterogeneous Systems''': A multiprocessor system with different kind of processors<br />
<br />
*'''Homogeneous System''': A multiprocessor system with same kind of processors<br />
<br />
*'''ILP''': Instruction Level Parallelism<br />
<br />
*'''IS''': Instruction Stream<br />
<br />
*'''MIMD''': Multiple Instruction Multiple Data<br />
<br />
*'''MISD''': Multiple Instruction Single Data<br />
<br />
*'''PE''': Processing Element<br />
<br />
*'''PFC''': Primary Flight Computer<br />
<br />
*'''SAPO''': Short for Samočinný počítač<br />
<br />
*'''SIMD''': Single Instruction Multiple Data<br />
<br />
*'''SISD''': Single Instruction Single Data<br />
<br />
*'''VLSI''': Very Large Scale Integration</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/1a_sp&diff=71778CSC/ECE 506 Spring 2013/1a sp2013-02-04T22:54:26Z<p>Schakra8: /* Architecture of systolic arrays as against MISD architecture */</p>
<hr />
<div>==Multiple Instruction Single Data (MISD) Architecture==<br />
<br />
Multiple Instruction Single Data, or MISD, architecture is one of the four general categories of multi-processor or multi-core architectures described by Flynn's Taxonomy.<br />
<br />
==MISD Architectures within the Context of Flynn’s Taxonomy<ref>http://en.wikipedia.org/wiki/Flynn's_taxonomy</ref><ref>http://www.phy.ornl.gov/csep/ca/node11.html</ref>==<br />
<br />
Flynn's Taxonomy categorizes parallel computing architectures based on the number of information streams that flowed into the processor.<br />
According to Flynn's Taxonomy, there are two kinds of logical streams of information that flows into a parallel computer, the Instruction Stream and the Data stream. Note that for the purposes of classification using Flynn’s Taxonomy, it does not matter whether or not the streams are physically independent of each other on the hardware, only that the streams can be logically treated as independent.<br />
Based on how many of each type of stream flows into the parallel computer (single vs. multiple), Flynn's Taxonomy divides parallel architectures into four categories. They are Single Instruction Single Data (SISD), Single Instruction Multiple Data (SIMD), Multiple Instruction Single Data (MISD), and Multiple Instruction Multiple Data (MIMD).<br />
<br />
The four classifications defined by Flynn are based upon the number of concurrent instruction (or control) and data streams available in the architecture are<ref>https://computing.llnl.gov/tutorials/parallel_comp/#Flynn</ref><br />
<br />
[[Image:Flynn's Taxonomy.PNG|thumb|center|400px|Figure 1. [http://en.wikipedia.org/wiki/Michael_J._Flynn Flynn]'s Taxonomy [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
==Overview of the MISD Architecture==<br />
<br />
In order to be categorized as MISD architecture, there must be at least two Instruction streams and there can only be one Data stream as shown on Figure 2. As per the description according to Flynn’s Taxonomy, each processing elements should receive the same data and execute different instructions on the same data simultaneously.<br />
<br />
[[Image:MISD.PNG|thumb|right|100px|Figure 2. MISD Architecture [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
There are very few implementations of the classical MISD architecture as described by Flynn’s Taxonomy, none of which are commercial, general purpose parallel computers. Instead, MISD architectures are mainly used to create custom hardware in order to solve specific problems. The majority of the implementations have been created are in academia.<br />
Over the years, several architectures have been described as MISD even though the architectures do not completely follow the description in Flynn’s Taxonomy. One common example is the Pipeline architecture where the data is passed from processing element to processing element and each processing element executes a different instruction. It is often argued that this is not a true MISD architecture as the data changes as it moves through the pipeline. Similarly, other architectures or applications exist which may be called MISD by some while others disagree. The following section looks at some of those implementations.<br />
<br />
Pipeline architectures belong to this type, though a purist might say that the data is different after processing by each stage in the pipeline. Fault-tolerant computers executing the same instructions redundantly in order to detect and mask errors, in a manner known as task replication, may be considered to belong to this type. Not many instances of this architecture exist, as MIMD and SIMD are often more appropriate for common data parallel techniques. Specifically, they allow better scaling and use of computational resources than MISD does. <br />
<br />
However, one prominent example of MISD in computing is the Space Shuttle flight control computers. Another example of this machine is the systolic array, such as the [http://www.cs.cmu.edu/~iwarp/ CMU iWrap] [BORKAR et al., 1990]. All the elements in this array are controlled by a global clock. On each cycle, an element will read a piece of data from one of its neighbors, perform a simple operation (e.g. add the incoming element to a stored value), and prepare a value to be written to a neighbor on the next step.<br />
<br />
==Implementations of MISD architecture==<br />
<br />
Some major implementations of MISD are in the fields of systolic arrays, pattern matching, flight control systems and fault tolerance systems. These MISD implementations are discussed below.<br />
<br />
===Systolic Array===<br />
<br />
A systolic array is an arrangement of processors in an array where data flows synchronously across the array between neighbors, usually with different data flowing in different directions. Each Processor at each step takes in data from one or more neighbors, processes it and, in the next step, outputs results in the opposite direction.<br />
<br />
The systolic array paradigm, data-stream-driven by data counters, is the counterpart of the [http://en.wikipedia.org/wiki/Von_Neumann_model von Neumann paradigm], instruction-stream-driven by a program counter. Because a systolic array usually sends and receives multiple data streams, and multiple data counters are needed to generate these data streams, it supports data parallelism. The name derives from analogy with the regular pumping of blood by the heart.<ref>http://en.wikipedia.org/wiki/Systolic_array</ref><br />
<br />
====Type of Systolic Arrays<ref>http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf General Purpose Systolic Arrays </ref>====<br />
<br />
=====Special-purpose systolic array=====<br />
[[Image:systolic_1.png|thumb|right|250px|Figure 3: The algorithm for the sum of a scalar product, computed in systolic element [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
[[Image:systolic_2.png|thumb|right|250px|Figure 7: The systolic product of two 3x3 matrices [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
An array of hardwired systolic processing elements tailored for a specific application. Typically, many tens or hundreds of cells fit on a single chip. One of the major applications of special-purpose systolic array is in matrix operations. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Systolic_1.png Figure 3] illustrates the algorithm for the sum of a scalar product, computed in a single systolic element. Here, a’s and b’s are synchronously shifted through the processing element to be available for next element. These data synchronously exits the processing element unmodified for the next element. The sum of the products is then shifted out of the accumulator.<br />
<br />
=====General-purpose systolic array=====<br />
<br />
It is an array of systolic processing elements, which gets adapted to a variety of applications via programming or reconfiguration. Array topologies can be either programmable or reconfigurable. Likewise, array cells are either programmable or reconfigurable. This is referred to as Systolic topologies.<br />
<br />
======Programmable Systolic Array======<br />
<br />
It is an array of programmable systolic elements that operates either in SIMD or MIMD fashion. Either the arrays interconnect or each processing unit is programmable and a program controls dataflow through the elements. Programmable systolic arrays are programmable either at a high level or a low level. At either level, programmable arrays can be categorized as either SIMD or MIMD machines.<br />
<br />
[[Image:systolic_4.png|thumb|right|250px|Figure 5: General organization of MIMD programmable linear systolic arrays [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
<br />
For the latter approach, the workstation downloads a program to each MIMD (Figure 4) systolic cell. Each cell may be loaded with a different program, or all the cells in the array may be loaded with the same program. Each cell's architecture is somewhat similar to the conventional von Neumann architecture: It contains a control unit, an ALU, and local memory. MIMD systolic cells have more local memory than their SIMD counterparts to support the von Neumann-style organization.<br />
<br />
This architecture is defined as Multiple Instruction Multiple Data (MIMD) architecture in [http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf 5]. The architecture has multiple instruction streams for the PEs and a single data stream passing through all the PEs. Thus, it can also be defined as Multiple Instruction Single Data (MISD) architecture. The architecture of Systolic array configuration are controversial as explained in the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Architecture_of_systolic_arrays_as_against_MISD_architecture section 4.1.2.]<br />
<br />
======Reconfigurable Systolic Array======<br />
[[Image:reconfig.jpg|thumb|right|250px|Figure 6: Block Diagram of the RSA Architecture [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
It is an array of systolic elements that can be programmed at the lowest level. Recent gate density advances in FPGA technology have produced a low-level, reconfigurable systolic array architecture that bridges the gap between special-purpose arrays and the more versatile, programmable general-purpose arrays. The FPGA architecture is unusual because a single hardware platform can be logically reconfigured as an exact duplicate of a special-purpose systolic array. <br />
<br />
The RSA circuit design is based on systolic array architecture consisting of PEs interconnected via SWs as depicted in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Reconfig.jpg Figure 6]. The homogeneous characteristic of the Reconfigurable Systolic Array (RSA) architecture, where each reconfigurable processing element (PE) cell is connected to its nearest neighbors via configurable switch (SW) elements, enables array expansion for parallel processing and facilitates time sharing computation of high-throughput data by individual PEs. Both the PEs and SWs can be reconfigured dynamically with the former as an arithmetic processor and the latter as a flexible router linking the neighboring PE cells. The RSA shifts reconfiguration and input signals into the PEs and SWs on separate data bus which enables the circuit to continue its operation while the reconfiguration is in process.<br />
<br />
====Architecture of systolic arrays as against MISD architecture====<br />
[[Image:comp.png|thumb|right|250px|Figure 7.Comparison between Architecture of systolic arrays and MISD]]<br />
<br />
As from the above mentioned configurations of the Systolic Arrays, it is seen that generally the configurations have multiple processing elements executing different instructions from dedicated instruction streams for each processing element. There is a single data stream that connects the adjacent PEs. Thus, systolic array can be defined as an MISD architecture.<br />
<br />
Many authors say that as the data read as input by one processing element is processed data output of the adjacent PE. The data stream cannot be considered as single because all the data paths do not carry the same data to all the PEs. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Comp.png Figure 7] shows the difference between the Data Stream for Systolic Arrays and the MISD architecture. Thus the systolic array should be considered as “Multiple Data” architecture and not Single Data architecture.<br />
<br />
===Fault Tolerant Systems<ref>http://en.wikipedia.org/wiki/Fault-tolerant_computer_system#Types_of_fault_tolerance</ref>===<br />
<br />
The fault tolerant systems are designed to handle the possible failures in software, hardware or interfaces. The hardware faults include hard disk failures, input or output device failures, etc. and the software and interface faults include driver failures; operator errors, installing unexpected software etc. The hardware faults can be detected and identified by implementing redundant hardware and multiple backups. The software faults can be tolerable by removing the program errors by executing the software redundantly or by implementing small programs that take over the tasks that crash or generate errors.<br />
<br />
[[Image:fault.png|thumb|right|250px|Figure 12 MISD as fault tolerant architecture]]<br />
<br />
In computer systems, the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Single_Instruction.2C_Multiple_Data_streams_.28SIMD.29 SIMD], [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD] and [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instruction.2C_Multiple_Data_streams_.28MIMD.29 MIMD] architectures facilitate the implementation of the fault tolerance systems by multiple instruction streams or multiple data streams or both. Fault tolerance on computations can be implemented by multiple processors (likely with different architectures) executing the algorithms on the same set of data. The output of each processor is compared with that of the others and M out of N majority voting method is used to determine the faulty processor. Thus MISD architecture is utilized to get the fault tolerance on critical computations.<br />
<br />
There are various examples of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture] being used as fault tolerant architecture. The major examples being flight control systems, nuclear power plants, satellite systems, super collider experiment systems, etc. Here, the flight control system is explained as an example of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture].<br />
<br />
====The Flight Control System – MISD Example for fault tolerance====<br />
<br />
A [http://en.wikipedia.org/wiki/Fly-by-wire Fly-By-Wire] system is used to replace the manual flight control by an electronic control interface. The movements of the flight control in the cockpit are converted to electronic signals and are transmitted to the actuators by wires. The control computers use the feedback from the sensors to compute and control the movement of the actuators to provide the expected response. These computers also perform the task to stabilize the aircraft and perform other tasks without the knowledge of the pilot. Flight control systems must meet extremely high levels of accuracy and functional integrity.<br />
<br />
There are redundant flight control computers present in the flight control system. If one of the flight-control computers crashes, gets damaged or is affected by electromagnetic pulses, the other computer can overrule the faulty one and hence the flight of the aircraft is unharmed. [[Image:fig13.png|thumb|right|250px|Figure 13: Architecture of triple redundant 777 primary flight computer [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-7 6]]]The number of redundant flight control computers is generally more than two, so that any computer whose results disagree with the others is ruled out to be faulty and is either ignored or rebooted.[[Image:fig14.png|thumb|right|250px|Figure 14: PFC with instruction and data streams]]<br />
<br />
=====Multiple Processors Implementation in Boeing 777<ref>http://www.citemaster.net/getdoc/8767/R8.pdf Y.C. (Bob) Yeh, Boeing Commercial Airplane Group, "Triple-Triple Redundant 777 Primary Flight Computer" </ref>=====<br />
<br />
In modern computers, the redundant flight control computations are carried out by multiprocessor systems. The triple redundant 777 primary flight computer, has the architecture as shown in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Fig13.png Figure 13].<br />
<br />
The system has three primary flight control computers, each of them having three lanes with different processors. The flight control program is compiled for each of the processors which get the input data from the same data bus but drive the output on their individual control bus. Thus each processor executes different instructions but they process the same data. Thus, it is the best suited example of Multiple Instruction Single Data (MISD) architecture.<br />
<br />
The three processors selected for the flight control system of [http://en.wikipedia.org/wiki/Boeing_777 Boeing 777] were [http://en.wikipedia.org/wiki/Intel_80486 Intel 80486], [http://en.wikipedia.org/wiki/Motorola_68040 Motorola 68040] and [http://en.wikipedia.org/wiki/AMD_Am29000 AMD 29050]. The dissimilar processors lead to dissimilar interface hardware circuits and compilers. Each lane of the flight control computer is data synchronized with the other lanes so that all of the lanes read the same frame of data from the flight sensors. As the outputs of each lane can be different, the median value of the outputs is used to select the output of the lane to be considered. The lane which has the median value select hardware selected is said to be in “command mode” whereas the other lanes are said to be in “monitoring mode”. It receives the data from the other Primary Flight Computer (PFC) lanes and performs a median select of the outputs. This provides a fault blocking mechanism before the fault detection and identification by the cross-lane monitoring system. Thus, the MISD based multi computer architecture is capable of detecting generic errors in compilers or in complex hardware devices providing assurance beyond reasonable doubt of the dependability of the Fly-By-Wire system.<br />
<br />
The above mentioned system clearly has individual Instruction Streams as the architecture of each processor is different, thus different instruction sets and different instruction streams. These processors have frame synchronized input data which means they have same set of data to work upon which is fed from a single data stream. Thus the flight control system can be classified under MISD architecture.<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System===<br />
<br />
The image pre-processing system of this Multiprocessor [9] , the pre-image processing time needed to be made shorter. To achieve the aim of reducing the pre-processing time, different types of image processing architectures for ‘C80 were presented. For example MIMD architecture is used for image histogram equalization; SIMD architecture for median filtering; MISD architecture for directional edge detection.<br />
<br />
The MISD architecture can be explained as follows. Here, each processor performs different operations on the same data set. It is mentioned at many places that MISD architecture was defined only for the sake of completeness of classification due to difficulty in practice to find a computer based on MISD architecture [10]. But in case of Image Processing, this assumption need not be true as here making many different operations on same image is rare and hence MISD architecture is a best fit for such cases. As an example of implementation of image processing operations for the MISD architecture the edge detection operation will be considered. The edges should be detected in four different directions (north, south, west and east) by the medium of four different filter masks. Each of the parallel processors of the TMS320C80 calculates one of the directional edges. Finally one obtains four images with detected edges, each in different direction.<br />
<br />
The image processing in computer is very often a multistage process composed of many stages. For example, an algorithm detecting contours of the objects in an image. The algorithm for achieving the before mentioned is composed of four basic image processing operation. The image acquired from the camera in the first step undergoes low-pass filtering, during which it is smoothed and noise is also eliminated. In the next step image is thresholding is performed and each pixel gets black or white depending on that if its grey level is below or above threshold value. The resulting image is a binary image from which edges are detected by the usage of the Laplace filter. In the last step the image with detected contours undergoes logic filtering in the purpose of elimination of single black pixels placed on the white background. For implementing the above processes parallel, a five-stage pipeline was used. The first stage of this pipeline constitutes the master processor which accounts for control of the image acquisition process. Further pipe line stages were implemented for 4 parallel processors as mentioned in table 1.<br />
<br />
Stage Processor Operation<br />
1 MP Image acquisition<br />
2 PP0 Low-pass Filtering<br />
3 PP1 Thresholding<br />
4 PP2 Laplace Filtering<br />
5 PP3 Logic Filtering<br />
Table 1: Pipeline structure for image edge detection [9]<br />
<br />
=='''References'''==<br />
<references/><br />
<br />
=='''Glossary'''==<br />
*'''CMU''': Carnegie Mellon University<br />
<br />
*'''CU''': Control Unit<br />
<br />
*'''DS''': Data Stream<br />
<br />
*'''Fly-By-Wire''': System that replaces the conventional manual flight controls of an aircraft with an electronic interface<br />
<br />
*'''FPGA''': Field Programmable Gate Array<br />
<br />
*'''Heterogeneous Systems''': A multiprocessor system with different kind of processors<br />
<br />
*'''Homogeneous System''': A multiprocessor system with same kind of processors<br />
<br />
*'''ILP''': Instruction Level Parallelism<br />
<br />
*'''IS''': Instruction Stream<br />
<br />
*'''MIMD''': Multiple Instruction Multiple Data<br />
<br />
*'''MISD''': Multiple Instruction Single Data<br />
<br />
*'''PE''': Processing Element<br />
<br />
*'''PFC''': Primary Flight Computer<br />
<br />
*'''SAPO''': Short for Samočinný počítač<br />
<br />
*'''SIMD''': Single Instruction Multiple Data<br />
<br />
*'''SISD''': Single Instruction Single Data<br />
<br />
*'''VLSI''': Very Large Scale Integration</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/1a_sp&diff=71777CSC/ECE 506 Spring 2013/1a sp2013-02-04T22:53:49Z<p>Schakra8: /* General-purpose systolic array */</p>
<hr />
<div>==Multiple Instruction Single Data (MISD) Architecture==<br />
<br />
Multiple Instruction Single Data, or MISD, architecture is one of the four general categories of multi-processor or multi-core architectures described by Flynn's Taxonomy.<br />
<br />
==MISD Architectures within the Context of Flynn’s Taxonomy<ref>http://en.wikipedia.org/wiki/Flynn's_taxonomy</ref><ref>http://www.phy.ornl.gov/csep/ca/node11.html</ref>==<br />
<br />
Flynn's Taxonomy categorizes parallel computing architectures based on the number of information streams that flowed into the processor.<br />
According to Flynn's Taxonomy, there are two kinds of logical streams of information that flows into a parallel computer, the Instruction Stream and the Data stream. Note that for the purposes of classification using Flynn’s Taxonomy, it does not matter whether or not the streams are physically independent of each other on the hardware, only that the streams can be logically treated as independent.<br />
Based on how many of each type of stream flows into the parallel computer (single vs. multiple), Flynn's Taxonomy divides parallel architectures into four categories. They are Single Instruction Single Data (SISD), Single Instruction Multiple Data (SIMD), Multiple Instruction Single Data (MISD), and Multiple Instruction Multiple Data (MIMD).<br />
<br />
The four classifications defined by Flynn are based upon the number of concurrent instruction (or control) and data streams available in the architecture are<ref>https://computing.llnl.gov/tutorials/parallel_comp/#Flynn</ref><br />
<br />
[[Image:Flynn's Taxonomy.PNG|thumb|center|400px|Figure 1. [http://en.wikipedia.org/wiki/Michael_J._Flynn Flynn]'s Taxonomy [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
==Overview of the MISD Architecture==<br />
<br />
In order to be categorized as MISD architecture, there must be at least two Instruction streams and there can only be one Data stream as shown on Figure 2. As per the description according to Flynn’s Taxonomy, each processing elements should receive the same data and execute different instructions on the same data simultaneously.<br />
<br />
[[Image:MISD.PNG|thumb|right|100px|Figure 2. MISD Architecture [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
There are very few implementations of the classical MISD architecture as described by Flynn’s Taxonomy, none of which are commercial, general purpose parallel computers. Instead, MISD architectures are mainly used to create custom hardware in order to solve specific problems. The majority of the implementations have been created are in academia.<br />
Over the years, several architectures have been described as MISD even though the architectures do not completely follow the description in Flynn’s Taxonomy. One common example is the Pipeline architecture where the data is passed from processing element to processing element and each processing element executes a different instruction. It is often argued that this is not a true MISD architecture as the data changes as it moves through the pipeline. Similarly, other architectures or applications exist which may be called MISD by some while others disagree. The following section looks at some of those implementations.<br />
<br />
Pipeline architectures belong to this type, though a purist might say that the data is different after processing by each stage in the pipeline. Fault-tolerant computers executing the same instructions redundantly in order to detect and mask errors, in a manner known as task replication, may be considered to belong to this type. Not many instances of this architecture exist, as MIMD and SIMD are often more appropriate for common data parallel techniques. Specifically, they allow better scaling and use of computational resources than MISD does. <br />
<br />
However, one prominent example of MISD in computing is the Space Shuttle flight control computers. Another example of this machine is the systolic array, such as the [http://www.cs.cmu.edu/~iwarp/ CMU iWrap] [BORKAR et al., 1990]. All the elements in this array are controlled by a global clock. On each cycle, an element will read a piece of data from one of its neighbors, perform a simple operation (e.g. add the incoming element to a stored value), and prepare a value to be written to a neighbor on the next step.<br />
<br />
==Implementations of MISD architecture==<br />
<br />
Some major implementations of MISD are in the fields of systolic arrays, pattern matching, flight control systems and fault tolerance systems. These MISD implementations are discussed below.<br />
<br />
===Systolic Array===<br />
<br />
A systolic array is an arrangement of processors in an array where data flows synchronously across the array between neighbors, usually with different data flowing in different directions. Each Processor at each step takes in data from one or more neighbors, processes it and, in the next step, outputs results in the opposite direction.<br />
<br />
The systolic array paradigm, data-stream-driven by data counters, is the counterpart of the [http://en.wikipedia.org/wiki/Von_Neumann_model von Neumann paradigm], instruction-stream-driven by a program counter. Because a systolic array usually sends and receives multiple data streams, and multiple data counters are needed to generate these data streams, it supports data parallelism. The name derives from analogy with the regular pumping of blood by the heart.<ref>http://en.wikipedia.org/wiki/Systolic_array</ref><br />
<br />
====Type of Systolic Arrays<ref>http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf General Purpose Systolic Arrays </ref>====<br />
<br />
=====Special-purpose systolic array=====<br />
[[Image:systolic_1.png|thumb|right|250px|Figure 3: The algorithm for the sum of a scalar product, computed in systolic element [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
[[Image:systolic_2.png|thumb|right|250px|Figure 7: The systolic product of two 3x3 matrices [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
An array of hardwired systolic processing elements tailored for a specific application. Typically, many tens or hundreds of cells fit on a single chip. One of the major applications of special-purpose systolic array is in matrix operations. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Systolic_1.png Figure 3] illustrates the algorithm for the sum of a scalar product, computed in a single systolic element. Here, a’s and b’s are synchronously shifted through the processing element to be available for next element. These data synchronously exits the processing element unmodified for the next element. The sum of the products is then shifted out of the accumulator.<br />
<br />
=====General-purpose systolic array=====<br />
<br />
It is an array of systolic processing elements, which gets adapted to a variety of applications via programming or reconfiguration. Array topologies can be either programmable or reconfigurable. Likewise, array cells are either programmable or reconfigurable. This is referred to as Systolic topologies.<br />
<br />
======Programmable Systolic Array======<br />
<br />
It is an array of programmable systolic elements that operates either in SIMD or MIMD fashion. Either the arrays interconnect or each processing unit is programmable and a program controls dataflow through the elements. Programmable systolic arrays are programmable either at a high level or a low level. At either level, programmable arrays can be categorized as either SIMD or MIMD machines.<br />
<br />
[[Image:systolic_4.png|thumb|right|250px|Figure 5: General organization of MIMD programmable linear systolic arrays [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
<br />
For the latter approach, the workstation downloads a program to each MIMD (Figure 4) systolic cell. Each cell may be loaded with a different program, or all the cells in the array may be loaded with the same program. Each cell's architecture is somewhat similar to the conventional von Neumann architecture: It contains a control unit, an ALU, and local memory. MIMD systolic cells have more local memory than their SIMD counterparts to support the von Neumann-style organization.<br />
<br />
This architecture is defined as Multiple Instruction Multiple Data (MIMD) architecture in [http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf 5]. The architecture has multiple instruction streams for the PEs and a single data stream passing through all the PEs. Thus, it can also be defined as Multiple Instruction Single Data (MISD) architecture. The architecture of Systolic array configuration are controversial as explained in the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Architecture_of_systolic_arrays_as_against_MISD_architecture section 4.1.2.]<br />
<br />
======Reconfigurable Systolic Array======<br />
[[Image:reconfig.jpg|thumb|right|250px|Figure 6: Block Diagram of the RSA Architecture [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
It is an array of systolic elements that can be programmed at the lowest level. Recent gate density advances in FPGA technology have produced a low-level, reconfigurable systolic array architecture that bridges the gap between special-purpose arrays and the more versatile, programmable general-purpose arrays. The FPGA architecture is unusual because a single hardware platform can be logically reconfigured as an exact duplicate of a special-purpose systolic array. <br />
<br />
The RSA circuit design is based on systolic array architecture consisting of PEs interconnected via SWs as depicted in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Reconfig.jpg Figure 6]. The homogeneous characteristic of the Reconfigurable Systolic Array (RSA) architecture, where each reconfigurable processing element (PE) cell is connected to its nearest neighbors via configurable switch (SW) elements, enables array expansion for parallel processing and facilitates time sharing computation of high-throughput data by individual PEs. Both the PEs and SWs can be reconfigured dynamically with the former as an arithmetic processor and the latter as a flexible router linking the neighboring PE cells. The RSA shifts reconfiguration and input signals into the PEs and SWs on separate data bus which enables the circuit to continue its operation while the reconfiguration is in process.<br />
<br />
====Architecture of systolic arrays as against MISD architecture====<br />
[[Image:comp.png|thumb|right|250px|Figure 11.Comparison between Architecture of systolic arrays and MISD]]<br />
<br />
As from the above mentioned configurations of the Systolic Arrays, it is seen that generally the configurations have multiple processing elements executing different instructions from dedicated instruction streams for each processing element. There is a single data stream that connects the adjacent PEs. Thus, systolic array can be defined as an MISD architecture.<br />
<br />
Many authors say that as the data read as input by one processing element is processed data output of the adjacent PE. The data stream cannot be considered as single because all the data paths do not carry the same data to all the PEs. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Comp.png Figure 11] shows the difference between the Data Stream for Systolic Arrays and the MISD architecture. Thus the systolic array should be considered as “Multiple Data” architecture and not Single Data architecture.<br />
<br />
===Fault Tolerant Systems<ref>http://en.wikipedia.org/wiki/Fault-tolerant_computer_system#Types_of_fault_tolerance</ref>===<br />
<br />
The fault tolerant systems are designed to handle the possible failures in software, hardware or interfaces. The hardware faults include hard disk failures, input or output device failures, etc. and the software and interface faults include driver failures; operator errors, installing unexpected software etc. The hardware faults can be detected and identified by implementing redundant hardware and multiple backups. The software faults can be tolerable by removing the program errors by executing the software redundantly or by implementing small programs that take over the tasks that crash or generate errors.<br />
<br />
[[Image:fault.png|thumb|right|250px|Figure 12 MISD as fault tolerant architecture]]<br />
<br />
In computer systems, the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Single_Instruction.2C_Multiple_Data_streams_.28SIMD.29 SIMD], [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD] and [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instruction.2C_Multiple_Data_streams_.28MIMD.29 MIMD] architectures facilitate the implementation of the fault tolerance systems by multiple instruction streams or multiple data streams or both. Fault tolerance on computations can be implemented by multiple processors (likely with different architectures) executing the algorithms on the same set of data. The output of each processor is compared with that of the others and M out of N majority voting method is used to determine the faulty processor. Thus MISD architecture is utilized to get the fault tolerance on critical computations.<br />
<br />
There are various examples of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture] being used as fault tolerant architecture. The major examples being flight control systems, nuclear power plants, satellite systems, super collider experiment systems, etc. Here, the flight control system is explained as an example of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture].<br />
<br />
====The Flight Control System – MISD Example for fault tolerance====<br />
<br />
A [http://en.wikipedia.org/wiki/Fly-by-wire Fly-By-Wire] system is used to replace the manual flight control by an electronic control interface. The movements of the flight control in the cockpit are converted to electronic signals and are transmitted to the actuators by wires. The control computers use the feedback from the sensors to compute and control the movement of the actuators to provide the expected response. These computers also perform the task to stabilize the aircraft and perform other tasks without the knowledge of the pilot. Flight control systems must meet extremely high levels of accuracy and functional integrity.<br />
<br />
There are redundant flight control computers present in the flight control system. If one of the flight-control computers crashes, gets damaged or is affected by electromagnetic pulses, the other computer can overrule the faulty one and hence the flight of the aircraft is unharmed. [[Image:fig13.png|thumb|right|250px|Figure 13: Architecture of triple redundant 777 primary flight computer [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-7 6]]]The number of redundant flight control computers is generally more than two, so that any computer whose results disagree with the others is ruled out to be faulty and is either ignored or rebooted.[[Image:fig14.png|thumb|right|250px|Figure 14: PFC with instruction and data streams]]<br />
<br />
=====Multiple Processors Implementation in Boeing 777<ref>http://www.citemaster.net/getdoc/8767/R8.pdf Y.C. (Bob) Yeh, Boeing Commercial Airplane Group, "Triple-Triple Redundant 777 Primary Flight Computer" </ref>=====<br />
<br />
In modern computers, the redundant flight control computations are carried out by multiprocessor systems. The triple redundant 777 primary flight computer, has the architecture as shown in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Fig13.png Figure 13].<br />
<br />
The system has three primary flight control computers, each of them having three lanes with different processors. The flight control program is compiled for each of the processors which get the input data from the same data bus but drive the output on their individual control bus. Thus each processor executes different instructions but they process the same data. Thus, it is the best suited example of Multiple Instruction Single Data (MISD) architecture.<br />
<br />
The three processors selected for the flight control system of [http://en.wikipedia.org/wiki/Boeing_777 Boeing 777] were [http://en.wikipedia.org/wiki/Intel_80486 Intel 80486], [http://en.wikipedia.org/wiki/Motorola_68040 Motorola 68040] and [http://en.wikipedia.org/wiki/AMD_Am29000 AMD 29050]. The dissimilar processors lead to dissimilar interface hardware circuits and compilers. Each lane of the flight control computer is data synchronized with the other lanes so that all of the lanes read the same frame of data from the flight sensors. As the outputs of each lane can be different, the median value of the outputs is used to select the output of the lane to be considered. The lane which has the median value select hardware selected is said to be in “command mode” whereas the other lanes are said to be in “monitoring mode”. It receives the data from the other Primary Flight Computer (PFC) lanes and performs a median select of the outputs. This provides a fault blocking mechanism before the fault detection and identification by the cross-lane monitoring system. Thus, the MISD based multi computer architecture is capable of detecting generic errors in compilers or in complex hardware devices providing assurance beyond reasonable doubt of the dependability of the Fly-By-Wire system.<br />
<br />
The above mentioned system clearly has individual Instruction Streams as the architecture of each processor is different, thus different instruction sets and different instruction streams. These processors have frame synchronized input data which means they have same set of data to work upon which is fed from a single data stream. Thus the flight control system can be classified under MISD architecture.<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System===<br />
<br />
The image pre-processing system of this Multiprocessor [9] , the pre-image processing time needed to be made shorter. To achieve the aim of reducing the pre-processing time, different types of image processing architectures for ‘C80 were presented. For example MIMD architecture is used for image histogram equalization; SIMD architecture for median filtering; MISD architecture for directional edge detection.<br />
<br />
The MISD architecture can be explained as follows. Here, each processor performs different operations on the same data set. It is mentioned at many places that MISD architecture was defined only for the sake of completeness of classification due to difficulty in practice to find a computer based on MISD architecture [10]. But in case of Image Processing, this assumption need not be true as here making many different operations on same image is rare and hence MISD architecture is a best fit for such cases. As an example of implementation of image processing operations for the MISD architecture the edge detection operation will be considered. The edges should be detected in four different directions (north, south, west and east) by the medium of four different filter masks. Each of the parallel processors of the TMS320C80 calculates one of the directional edges. Finally one obtains four images with detected edges, each in different direction.<br />
<br />
The image processing in computer is very often a multistage process composed of many stages. For example, an algorithm detecting contours of the objects in an image. The algorithm for achieving the before mentioned is composed of four basic image processing operation. The image acquired from the camera in the first step undergoes low-pass filtering, during which it is smoothed and noise is also eliminated. In the next step image is thresholding is performed and each pixel gets black or white depending on that if its grey level is below or above threshold value. The resulting image is a binary image from which edges are detected by the usage of the Laplace filter. In the last step the image with detected contours undergoes logic filtering in the purpose of elimination of single black pixels placed on the white background. For implementing the above processes parallel, a five-stage pipeline was used. The first stage of this pipeline constitutes the master processor which accounts for control of the image acquisition process. Further pipe line stages were implemented for 4 parallel processors as mentioned in table 1.<br />
<br />
Stage Processor Operation<br />
1 MP Image acquisition<br />
2 PP0 Low-pass Filtering<br />
3 PP1 Thresholding<br />
4 PP2 Laplace Filtering<br />
5 PP3 Logic Filtering<br />
Table 1: Pipeline structure for image edge detection [9]<br />
<br />
=='''References'''==<br />
<references/><br />
<br />
=='''Glossary'''==<br />
*'''CMU''': Carnegie Mellon University<br />
<br />
*'''CU''': Control Unit<br />
<br />
*'''DS''': Data Stream<br />
<br />
*'''Fly-By-Wire''': System that replaces the conventional manual flight controls of an aircraft with an electronic interface<br />
<br />
*'''FPGA''': Field Programmable Gate Array<br />
<br />
*'''Heterogeneous Systems''': A multiprocessor system with different kind of processors<br />
<br />
*'''Homogeneous System''': A multiprocessor system with same kind of processors<br />
<br />
*'''ILP''': Instruction Level Parallelism<br />
<br />
*'''IS''': Instruction Stream<br />
<br />
*'''MIMD''': Multiple Instruction Multiple Data<br />
<br />
*'''MISD''': Multiple Instruction Single Data<br />
<br />
*'''PE''': Processing Element<br />
<br />
*'''PFC''': Primary Flight Computer<br />
<br />
*'''SAPO''': Short for Samočinný počítač<br />
<br />
*'''SIMD''': Single Instruction Multiple Data<br />
<br />
*'''SISD''': Single Instruction Single Data<br />
<br />
*'''VLSI''': Very Large Scale Integration</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/1a_sp&diff=71776CSC/ECE 506 Spring 2013/1a sp2013-02-04T22:53:02Z<p>Schakra8: /* General-purpose systolic array */</p>
<hr />
<div>==Multiple Instruction Single Data (MISD) Architecture==<br />
<br />
Multiple Instruction Single Data, or MISD, architecture is one of the four general categories of multi-processor or multi-core architectures described by Flynn's Taxonomy.<br />
<br />
==MISD Architectures within the Context of Flynn’s Taxonomy<ref>http://en.wikipedia.org/wiki/Flynn's_taxonomy</ref><ref>http://www.phy.ornl.gov/csep/ca/node11.html</ref>==<br />
<br />
Flynn's Taxonomy categorizes parallel computing architectures based on the number of information streams that flowed into the processor.<br />
According to Flynn's Taxonomy, there are two kinds of logical streams of information that flows into a parallel computer, the Instruction Stream and the Data stream. Note that for the purposes of classification using Flynn’s Taxonomy, it does not matter whether or not the streams are physically independent of each other on the hardware, only that the streams can be logically treated as independent.<br />
Based on how many of each type of stream flows into the parallel computer (single vs. multiple), Flynn's Taxonomy divides parallel architectures into four categories. They are Single Instruction Single Data (SISD), Single Instruction Multiple Data (SIMD), Multiple Instruction Single Data (MISD), and Multiple Instruction Multiple Data (MIMD).<br />
<br />
The four classifications defined by Flynn are based upon the number of concurrent instruction (or control) and data streams available in the architecture are<ref>https://computing.llnl.gov/tutorials/parallel_comp/#Flynn</ref><br />
<br />
[[Image:Flynn's Taxonomy.PNG|thumb|center|400px|Figure 1. [http://en.wikipedia.org/wiki/Michael_J._Flynn Flynn]'s Taxonomy [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
==Overview of the MISD Architecture==<br />
<br />
In order to be categorized as MISD architecture, there must be at least two Instruction streams and there can only be one Data stream as shown on Figure 2. As per the description according to Flynn’s Taxonomy, each processing elements should receive the same data and execute different instructions on the same data simultaneously.<br />
<br />
[[Image:MISD.PNG|thumb|right|100px|Figure 2. MISD Architecture [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
There are very few implementations of the classical MISD architecture as described by Flynn’s Taxonomy, none of which are commercial, general purpose parallel computers. Instead, MISD architectures are mainly used to create custom hardware in order to solve specific problems. The majority of the implementations have been created are in academia.<br />
Over the years, several architectures have been described as MISD even though the architectures do not completely follow the description in Flynn’s Taxonomy. One common example is the Pipeline architecture where the data is passed from processing element to processing element and each processing element executes a different instruction. It is often argued that this is not a true MISD architecture as the data changes as it moves through the pipeline. Similarly, other architectures or applications exist which may be called MISD by some while others disagree. The following section looks at some of those implementations.<br />
<br />
Pipeline architectures belong to this type, though a purist might say that the data is different after processing by each stage in the pipeline. Fault-tolerant computers executing the same instructions redundantly in order to detect and mask errors, in a manner known as task replication, may be considered to belong to this type. Not many instances of this architecture exist, as MIMD and SIMD are often more appropriate for common data parallel techniques. Specifically, they allow better scaling and use of computational resources than MISD does. <br />
<br />
However, one prominent example of MISD in computing is the Space Shuttle flight control computers. Another example of this machine is the systolic array, such as the [http://www.cs.cmu.edu/~iwarp/ CMU iWrap] [BORKAR et al., 1990]. All the elements in this array are controlled by a global clock. On each cycle, an element will read a piece of data from one of its neighbors, perform a simple operation (e.g. add the incoming element to a stored value), and prepare a value to be written to a neighbor on the next step.<br />
<br />
==Implementations of MISD architecture==<br />
<br />
Some major implementations of MISD are in the fields of systolic arrays, pattern matching, flight control systems and fault tolerance systems. These MISD implementations are discussed below.<br />
<br />
===Systolic Array===<br />
<br />
A systolic array is an arrangement of processors in an array where data flows synchronously across the array between neighbors, usually with different data flowing in different directions. Each Processor at each step takes in data from one or more neighbors, processes it and, in the next step, outputs results in the opposite direction.<br />
<br />
The systolic array paradigm, data-stream-driven by data counters, is the counterpart of the [http://en.wikipedia.org/wiki/Von_Neumann_model von Neumann paradigm], instruction-stream-driven by a program counter. Because a systolic array usually sends and receives multiple data streams, and multiple data counters are needed to generate these data streams, it supports data parallelism. The name derives from analogy with the regular pumping of blood by the heart.<ref>http://en.wikipedia.org/wiki/Systolic_array</ref><br />
<br />
====Type of Systolic Arrays<ref>http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf General Purpose Systolic Arrays </ref>====<br />
<br />
=====Special-purpose systolic array=====<br />
[[Image:systolic_1.png|thumb|right|250px|Figure 3: The algorithm for the sum of a scalar product, computed in systolic element [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
[[Image:systolic_2.png|thumb|right|250px|Figure 7: The systolic product of two 3x3 matrices [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
An array of hardwired systolic processing elements tailored for a specific application. Typically, many tens or hundreds of cells fit on a single chip. One of the major applications of special-purpose systolic array is in matrix operations. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Systolic_1.png Figure 3] illustrates the algorithm for the sum of a scalar product, computed in a single systolic element. Here, a’s and b’s are synchronously shifted through the processing element to be available for next element. These data synchronously exits the processing element unmodified for the next element. The sum of the products is then shifted out of the accumulator.<br />
<br />
=====General-purpose systolic array=====<br />
<br />
It is an array of systolic processing elements, which gets adapted to a variety of applications via programming or reconfiguration. Array topologies can be either programmable or reconfigurable. Likewise, array cells are either programmable or reconfigurable. This is referred to as Systolic topologies.<br />
<br />
======Programmable Systolic Array======<br />
<br />
It is an array of programmable systolic elements that operates either in SIMD or MIMD fashion. Either the arrays interconnect or each processing unit is programmable and a program controls dataflow through the elements. Programmable systolic arrays are programmable either at a high level or a low level. At either level, programmable arrays can be categorized as either SIMD or MIMD machines.<br />
<br />
[[Image:systolic_4.png|thumb|right|250px|Figure 5: General organization of MIMD programmable linear systolic arrays [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
<br />
For the latter approach, the workstation downloads a program to each MIMD (Figure 4) systolic cell. Each cell may be loaded with a different program, or all the cells in the array may be loaded with the same program. Each cell's architecture is somewhat similar to the conventional von Neumann architecture: It contains a control unit, an ALU, and local memory. MIMD systolic cells have more local memory than their SIMD counterparts to support the von Neumann-style organization.<br />
<br />
This architecture is defined as Multiple Instruction Multiple Data (MIMD) architecture in [http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf 5]. The architecture has multiple instruction streams for the PEs and a single data stream passing through all the PEs. Thus, it can also be defined as Multiple Instruction Single Data (MISD) architecture. The architecture of Systolic array configuration are controversial as explained in the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Architecture_of_systolic_arrays_as_against_MISD_architecture section 4.1.2.]<br />
<br />
======Reconfigurable Systolic Array======<br />
[[Image:reconfig.jpg|thumb|right|250px|Figure 10: Block Diagram of the RSA Architecture [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
It is an array of systolic elements that can be programmed at the lowest level. Recent gate density advances in FPGA technology have produced a low-level, reconfigurable systolic array architecture that bridges the gap between special-purpose arrays and the more versatile, programmable general-purpose arrays. The FPGA architecture is unusual because a single hardware platform can be logically reconfigured as an exact duplicate of a special-purpose systolic array. <br />
<br />
The RSA circuit design is based on systolic array architecture consisting of PEs interconnected via SWs as depicted in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Reconfig.jpg Figure 10]. The homogeneous characteristic of the Reconfigurable Systolic Array (RSA) architecture, where each reconfigurable processing element (PE) cell is connected to its nearest neighbors via configurable switch (SW) elements, enables array expansion for parallel processing and facilitates time sharing computation of high-throughput data by individual PEs. Both the PEs and SWs can be reconfigured dynamically with the former as an arithmetic processor and the latter as a flexible router linking the neighboring PE cells. The RSA shifts reconfiguration and input signals into the PEs and SWs on separate data bus which enables the circuit to continue its operation while the reconfiguration is in process.<br />
<br />
====Architecture of systolic arrays as against MISD architecture====<br />
[[Image:comp.png|thumb|right|250px|Figure 11.Comparison between Architecture of systolic arrays and MISD]]<br />
<br />
As from the above mentioned configurations of the Systolic Arrays, it is seen that generally the configurations have multiple processing elements executing different instructions from dedicated instruction streams for each processing element. There is a single data stream that connects the adjacent PEs. Thus, systolic array can be defined as an MISD architecture.<br />
<br />
Many authors say that as the data read as input by one processing element is processed data output of the adjacent PE. The data stream cannot be considered as single because all the data paths do not carry the same data to all the PEs. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Comp.png Figure 11] shows the difference between the Data Stream for Systolic Arrays and the MISD architecture. Thus the systolic array should be considered as “Multiple Data” architecture and not Single Data architecture.<br />
<br />
===Fault Tolerant Systems<ref>http://en.wikipedia.org/wiki/Fault-tolerant_computer_system#Types_of_fault_tolerance</ref>===<br />
<br />
The fault tolerant systems are designed to handle the possible failures in software, hardware or interfaces. The hardware faults include hard disk failures, input or output device failures, etc. and the software and interface faults include driver failures; operator errors, installing unexpected software etc. The hardware faults can be detected and identified by implementing redundant hardware and multiple backups. The software faults can be tolerable by removing the program errors by executing the software redundantly or by implementing small programs that take over the tasks that crash or generate errors.<br />
<br />
[[Image:fault.png|thumb|right|250px|Figure 12 MISD as fault tolerant architecture]]<br />
<br />
In computer systems, the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Single_Instruction.2C_Multiple_Data_streams_.28SIMD.29 SIMD], [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD] and [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instruction.2C_Multiple_Data_streams_.28MIMD.29 MIMD] architectures facilitate the implementation of the fault tolerance systems by multiple instruction streams or multiple data streams or both. Fault tolerance on computations can be implemented by multiple processors (likely with different architectures) executing the algorithms on the same set of data. The output of each processor is compared with that of the others and M out of N majority voting method is used to determine the faulty processor. Thus MISD architecture is utilized to get the fault tolerance on critical computations.<br />
<br />
There are various examples of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture] being used as fault tolerant architecture. The major examples being flight control systems, nuclear power plants, satellite systems, super collider experiment systems, etc. Here, the flight control system is explained as an example of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture].<br />
<br />
====The Flight Control System – MISD Example for fault tolerance====<br />
<br />
A [http://en.wikipedia.org/wiki/Fly-by-wire Fly-By-Wire] system is used to replace the manual flight control by an electronic control interface. The movements of the flight control in the cockpit are converted to electronic signals and are transmitted to the actuators by wires. The control computers use the feedback from the sensors to compute and control the movement of the actuators to provide the expected response. These computers also perform the task to stabilize the aircraft and perform other tasks without the knowledge of the pilot. Flight control systems must meet extremely high levels of accuracy and functional integrity.<br />
<br />
There are redundant flight control computers present in the flight control system. If one of the flight-control computers crashes, gets damaged or is affected by electromagnetic pulses, the other computer can overrule the faulty one and hence the flight of the aircraft is unharmed. [[Image:fig13.png|thumb|right|250px|Figure 13: Architecture of triple redundant 777 primary flight computer [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-7 6]]]The number of redundant flight control computers is generally more than two, so that any computer whose results disagree with the others is ruled out to be faulty and is either ignored or rebooted.[[Image:fig14.png|thumb|right|250px|Figure 14: PFC with instruction and data streams]]<br />
<br />
=====Multiple Processors Implementation in Boeing 777<ref>http://www.citemaster.net/getdoc/8767/R8.pdf Y.C. (Bob) Yeh, Boeing Commercial Airplane Group, "Triple-Triple Redundant 777 Primary Flight Computer" </ref>=====<br />
<br />
In modern computers, the redundant flight control computations are carried out by multiprocessor systems. The triple redundant 777 primary flight computer, has the architecture as shown in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Fig13.png Figure 13].<br />
<br />
The system has three primary flight control computers, each of them having three lanes with different processors. The flight control program is compiled for each of the processors which get the input data from the same data bus but drive the output on their individual control bus. Thus each processor executes different instructions but they process the same data. Thus, it is the best suited example of Multiple Instruction Single Data (MISD) architecture.<br />
<br />
The three processors selected for the flight control system of [http://en.wikipedia.org/wiki/Boeing_777 Boeing 777] were [http://en.wikipedia.org/wiki/Intel_80486 Intel 80486], [http://en.wikipedia.org/wiki/Motorola_68040 Motorola 68040] and [http://en.wikipedia.org/wiki/AMD_Am29000 AMD 29050]. The dissimilar processors lead to dissimilar interface hardware circuits and compilers. Each lane of the flight control computer is data synchronized with the other lanes so that all of the lanes read the same frame of data from the flight sensors. As the outputs of each lane can be different, the median value of the outputs is used to select the output of the lane to be considered. The lane which has the median value select hardware selected is said to be in “command mode” whereas the other lanes are said to be in “monitoring mode”. It receives the data from the other Primary Flight Computer (PFC) lanes and performs a median select of the outputs. This provides a fault blocking mechanism before the fault detection and identification by the cross-lane monitoring system. Thus, the MISD based multi computer architecture is capable of detecting generic errors in compilers or in complex hardware devices providing assurance beyond reasonable doubt of the dependability of the Fly-By-Wire system.<br />
<br />
The above mentioned system clearly has individual Instruction Streams as the architecture of each processor is different, thus different instruction sets and different instruction streams. These processors have frame synchronized input data which means they have same set of data to work upon which is fed from a single data stream. Thus the flight control system can be classified under MISD architecture.<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System===<br />
<br />
The image pre-processing system of this Multiprocessor [9] , the pre-image processing time needed to be made shorter. To achieve the aim of reducing the pre-processing time, different types of image processing architectures for ‘C80 were presented. For example MIMD architecture is used for image histogram equalization; SIMD architecture for median filtering; MISD architecture for directional edge detection.<br />
<br />
The MISD architecture can be explained as follows. Here, each processor performs different operations on the same data set. It is mentioned at many places that MISD architecture was defined only for the sake of completeness of classification due to difficulty in practice to find a computer based on MISD architecture [10]. But in case of Image Processing, this assumption need not be true as here making many different operations on same image is rare and hence MISD architecture is a best fit for such cases. As an example of implementation of image processing operations for the MISD architecture the edge detection operation will be considered. The edges should be detected in four different directions (north, south, west and east) by the medium of four different filter masks. Each of the parallel processors of the TMS320C80 calculates one of the directional edges. Finally one obtains four images with detected edges, each in different direction.<br />
<br />
The image processing in computer is very often a multistage process composed of many stages. For example, an algorithm detecting contours of the objects in an image. The algorithm for achieving the before mentioned is composed of four basic image processing operation. The image acquired from the camera in the first step undergoes low-pass filtering, during which it is smoothed and noise is also eliminated. In the next step image is thresholding is performed and each pixel gets black or white depending on that if its grey level is below or above threshold value. The resulting image is a binary image from which edges are detected by the usage of the Laplace filter. In the last step the image with detected contours undergoes logic filtering in the purpose of elimination of single black pixels placed on the white background. For implementing the above processes parallel, a five-stage pipeline was used. The first stage of this pipeline constitutes the master processor which accounts for control of the image acquisition process. Further pipe line stages were implemented for 4 parallel processors as mentioned in table 1.<br />
<br />
Stage Processor Operation<br />
1 MP Image acquisition<br />
2 PP0 Low-pass Filtering<br />
3 PP1 Thresholding<br />
4 PP2 Laplace Filtering<br />
5 PP3 Logic Filtering<br />
Table 1: Pipeline structure for image edge detection [9]<br />
<br />
=='''References'''==<br />
<references/><br />
<br />
=='''Glossary'''==<br />
*'''CMU''': Carnegie Mellon University<br />
<br />
*'''CU''': Control Unit<br />
<br />
*'''DS''': Data Stream<br />
<br />
*'''Fly-By-Wire''': System that replaces the conventional manual flight controls of an aircraft with an electronic interface<br />
<br />
*'''FPGA''': Field Programmable Gate Array<br />
<br />
*'''Heterogeneous Systems''': A multiprocessor system with different kind of processors<br />
<br />
*'''Homogeneous System''': A multiprocessor system with same kind of processors<br />
<br />
*'''ILP''': Instruction Level Parallelism<br />
<br />
*'''IS''': Instruction Stream<br />
<br />
*'''MIMD''': Multiple Instruction Multiple Data<br />
<br />
*'''MISD''': Multiple Instruction Single Data<br />
<br />
*'''PE''': Processing Element<br />
<br />
*'''PFC''': Primary Flight Computer<br />
<br />
*'''SAPO''': Short for Samočinný počítač<br />
<br />
*'''SIMD''': Single Instruction Multiple Data<br />
<br />
*'''SISD''': Single Instruction Single Data<br />
<br />
*'''VLSI''': Very Large Scale Integration</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/1a_sp&diff=71775CSC/ECE 506 Spring 2013/1a sp2013-02-04T22:52:28Z<p>Schakra8: /* Programmable Systolic Array */</p>
<hr />
<div>==Multiple Instruction Single Data (MISD) Architecture==<br />
<br />
Multiple Instruction Single Data, or MISD, architecture is one of the four general categories of multi-processor or multi-core architectures described by Flynn's Taxonomy.<br />
<br />
==MISD Architectures within the Context of Flynn’s Taxonomy<ref>http://en.wikipedia.org/wiki/Flynn's_taxonomy</ref><ref>http://www.phy.ornl.gov/csep/ca/node11.html</ref>==<br />
<br />
Flynn's Taxonomy categorizes parallel computing architectures based on the number of information streams that flowed into the processor.<br />
According to Flynn's Taxonomy, there are two kinds of logical streams of information that flows into a parallel computer, the Instruction Stream and the Data stream. Note that for the purposes of classification using Flynn’s Taxonomy, it does not matter whether or not the streams are physically independent of each other on the hardware, only that the streams can be logically treated as independent.<br />
Based on how many of each type of stream flows into the parallel computer (single vs. multiple), Flynn's Taxonomy divides parallel architectures into four categories. They are Single Instruction Single Data (SISD), Single Instruction Multiple Data (SIMD), Multiple Instruction Single Data (MISD), and Multiple Instruction Multiple Data (MIMD).<br />
<br />
The four classifications defined by Flynn are based upon the number of concurrent instruction (or control) and data streams available in the architecture are<ref>https://computing.llnl.gov/tutorials/parallel_comp/#Flynn</ref><br />
<br />
[[Image:Flynn's Taxonomy.PNG|thumb|center|400px|Figure 1. [http://en.wikipedia.org/wiki/Michael_J._Flynn Flynn]'s Taxonomy [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
==Overview of the MISD Architecture==<br />
<br />
In order to be categorized as MISD architecture, there must be at least two Instruction streams and there can only be one Data stream as shown on Figure 2. As per the description according to Flynn’s Taxonomy, each processing elements should receive the same data and execute different instructions on the same data simultaneously.<br />
<br />
[[Image:MISD.PNG|thumb|right|100px|Figure 2. MISD Architecture [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
There are very few implementations of the classical MISD architecture as described by Flynn’s Taxonomy, none of which are commercial, general purpose parallel computers. Instead, MISD architectures are mainly used to create custom hardware in order to solve specific problems. The majority of the implementations have been created are in academia.<br />
Over the years, several architectures have been described as MISD even though the architectures do not completely follow the description in Flynn’s Taxonomy. One common example is the Pipeline architecture where the data is passed from processing element to processing element and each processing element executes a different instruction. It is often argued that this is not a true MISD architecture as the data changes as it moves through the pipeline. Similarly, other architectures or applications exist which may be called MISD by some while others disagree. The following section looks at some of those implementations.<br />
<br />
Pipeline architectures belong to this type, though a purist might say that the data is different after processing by each stage in the pipeline. Fault-tolerant computers executing the same instructions redundantly in order to detect and mask errors, in a manner known as task replication, may be considered to belong to this type. Not many instances of this architecture exist, as MIMD and SIMD are often more appropriate for common data parallel techniques. Specifically, they allow better scaling and use of computational resources than MISD does. <br />
<br />
However, one prominent example of MISD in computing is the Space Shuttle flight control computers. Another example of this machine is the systolic array, such as the [http://www.cs.cmu.edu/~iwarp/ CMU iWrap] [BORKAR et al., 1990]. All the elements in this array are controlled by a global clock. On each cycle, an element will read a piece of data from one of its neighbors, perform a simple operation (e.g. add the incoming element to a stored value), and prepare a value to be written to a neighbor on the next step.<br />
<br />
==Implementations of MISD architecture==<br />
<br />
Some major implementations of MISD are in the fields of systolic arrays, pattern matching, flight control systems and fault tolerance systems. These MISD implementations are discussed below.<br />
<br />
===Systolic Array===<br />
<br />
A systolic array is an arrangement of processors in an array where data flows synchronously across the array between neighbors, usually with different data flowing in different directions. Each Processor at each step takes in data from one or more neighbors, processes it and, in the next step, outputs results in the opposite direction.<br />
<br />
The systolic array paradigm, data-stream-driven by data counters, is the counterpart of the [http://en.wikipedia.org/wiki/Von_Neumann_model von Neumann paradigm], instruction-stream-driven by a program counter. Because a systolic array usually sends and receives multiple data streams, and multiple data counters are needed to generate these data streams, it supports data parallelism. The name derives from analogy with the regular pumping of blood by the heart.<ref>http://en.wikipedia.org/wiki/Systolic_array</ref><br />
<br />
====Type of Systolic Arrays<ref>http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf General Purpose Systolic Arrays </ref>====<br />
<br />
=====Special-purpose systolic array=====<br />
[[Image:systolic_1.png|thumb|right|250px|Figure 3: The algorithm for the sum of a scalar product, computed in systolic element [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
[[Image:systolic_2.png|thumb|right|250px|Figure 7: The systolic product of two 3x3 matrices [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
An array of hardwired systolic processing elements tailored for a specific application. Typically, many tens or hundreds of cells fit on a single chip. One of the major applications of special-purpose systolic array is in matrix operations. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Systolic_1.png Figure 3] illustrates the algorithm for the sum of a scalar product, computed in a single systolic element. Here, a’s and b’s are synchronously shifted through the processing element to be available for next element. These data synchronously exits the processing element unmodified for the next element. The sum of the products is then shifted out of the accumulator.<br />
<br />
=====General-purpose systolic array=====<br />
<br />
It is an array of systolic processing elements, which gets adapted to a variety of applications via programming or reconfiguration. Array topologies can be either programmable or reconfigurable. Likewise, array cells are either programmable or reconfigurable. This is referred to as Systolic topologies.<br />
<br />
======Programmable Systolic Array======<br />
<br />
It is an array of programmable systolic elements that operates either in SIMD or MIMD fashion. Either the arrays interconnect or each processing unit is programmable and a program controls dataflow through the elements. Programmable systolic arrays are programmable either at a high level or a low level. At either level, programmable arrays can be categorized as either SIMD or MIMD machines.<br />
<br />
[[Image:systolic_4.png|thumb|right|250px|Figure 4: General organization of MIMD programmable linear systolic arrays [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
<br />
For the latter approach, the workstation downloads a program to each MIMD (Figure 4) systolic cell. Each cell may be loaded with a different program, or all the cells in the array may be loaded with the same program. Each cell's architecture is somewhat similar to the conventional von Neumann architecture: It contains a control unit, an ALU, and local memory. MIMD systolic cells have more local memory than their SIMD counterparts to support the von Neumann-style organization.<br />
<br />
This architecture is defined as Multiple Instruction Multiple Data (MIMD) architecture in [http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf 5]. The architecture has multiple instruction streams for the PEs and a single data stream passing through all the PEs. Thus, it can also be defined as Multiple Instruction Single Data (MISD) architecture. The architecture of Systolic array configuration are controversial as explained in the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Architecture_of_systolic_arrays_as_against_MISD_architecture section 4.1.2.]<br />
<br />
======Reconfigurable Systolic Array======<br />
[[Image:reconfig.jpg|thumb|right|250px|Figure 10: Block Diagram of the RSA Architecture [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
It is an array of systolic elements that can be programmed at the lowest level. Recent gate density advances in FPGA technology have produced a low-level, reconfigurable systolic array architecture that bridges the gap between special-purpose arrays and the more versatile, programmable general-purpose arrays. The FPGA architecture is unusual because a single hardware platform can be logically reconfigured as an exact duplicate of a special-purpose systolic array. <br />
<br />
The RSA circuit design is based on systolic array architecture consisting of PEs interconnected via SWs as depicted in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Reconfig.jpg Figure 10]. The homogeneous characteristic of the Reconfigurable Systolic Array (RSA) architecture, where each reconfigurable processing element (PE) cell is connected to its nearest neighbors via configurable switch (SW) elements, enables array expansion for parallel processing and facilitates time sharing computation of high-throughput data by individual PEs. Both the PEs and SWs can be reconfigured dynamically with the former as an arithmetic processor and the latter as a flexible router linking the neighboring PE cells. The RSA shifts reconfiguration and input signals into the PEs and SWs on separate data bus which enables the circuit to continue its operation while the reconfiguration is in process.<br />
<br />
====Architecture of systolic arrays as against MISD architecture====<br />
[[Image:comp.png|thumb|right|250px|Figure 11.Comparison between Architecture of systolic arrays and MISD]]<br />
<br />
As from the above mentioned configurations of the Systolic Arrays, it is seen that generally the configurations have multiple processing elements executing different instructions from dedicated instruction streams for each processing element. There is a single data stream that connects the adjacent PEs. Thus, systolic array can be defined as an MISD architecture.<br />
<br />
Many authors say that as the data read as input by one processing element is processed data output of the adjacent PE. The data stream cannot be considered as single because all the data paths do not carry the same data to all the PEs. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Comp.png Figure 11] shows the difference between the Data Stream for Systolic Arrays and the MISD architecture. Thus the systolic array should be considered as “Multiple Data” architecture and not Single Data architecture.<br />
<br />
===Fault Tolerant Systems<ref>http://en.wikipedia.org/wiki/Fault-tolerant_computer_system#Types_of_fault_tolerance</ref>===<br />
<br />
The fault tolerant systems are designed to handle the possible failures in software, hardware or interfaces. The hardware faults include hard disk failures, input or output device failures, etc. and the software and interface faults include driver failures; operator errors, installing unexpected software etc. The hardware faults can be detected and identified by implementing redundant hardware and multiple backups. The software faults can be tolerable by removing the program errors by executing the software redundantly or by implementing small programs that take over the tasks that crash or generate errors.<br />
<br />
[[Image:fault.png|thumb|right|250px|Figure 12 MISD as fault tolerant architecture]]<br />
<br />
In computer systems, the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Single_Instruction.2C_Multiple_Data_streams_.28SIMD.29 SIMD], [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD] and [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instruction.2C_Multiple_Data_streams_.28MIMD.29 MIMD] architectures facilitate the implementation of the fault tolerance systems by multiple instruction streams or multiple data streams or both. Fault tolerance on computations can be implemented by multiple processors (likely with different architectures) executing the algorithms on the same set of data. The output of each processor is compared with that of the others and M out of N majority voting method is used to determine the faulty processor. Thus MISD architecture is utilized to get the fault tolerance on critical computations.<br />
<br />
There are various examples of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture] being used as fault tolerant architecture. The major examples being flight control systems, nuclear power plants, satellite systems, super collider experiment systems, etc. Here, the flight control system is explained as an example of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture].<br />
<br />
====The Flight Control System – MISD Example for fault tolerance====<br />
<br />
A [http://en.wikipedia.org/wiki/Fly-by-wire Fly-By-Wire] system is used to replace the manual flight control by an electronic control interface. The movements of the flight control in the cockpit are converted to electronic signals and are transmitted to the actuators by wires. The control computers use the feedback from the sensors to compute and control the movement of the actuators to provide the expected response. These computers also perform the task to stabilize the aircraft and perform other tasks without the knowledge of the pilot. Flight control systems must meet extremely high levels of accuracy and functional integrity.<br />
<br />
There are redundant flight control computers present in the flight control system. If one of the flight-control computers crashes, gets damaged or is affected by electromagnetic pulses, the other computer can overrule the faulty one and hence the flight of the aircraft is unharmed. [[Image:fig13.png|thumb|right|250px|Figure 13: Architecture of triple redundant 777 primary flight computer [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-7 6]]]The number of redundant flight control computers is generally more than two, so that any computer whose results disagree with the others is ruled out to be faulty and is either ignored or rebooted.[[Image:fig14.png|thumb|right|250px|Figure 14: PFC with instruction and data streams]]<br />
<br />
=====Multiple Processors Implementation in Boeing 777<ref>http://www.citemaster.net/getdoc/8767/R8.pdf Y.C. (Bob) Yeh, Boeing Commercial Airplane Group, "Triple-Triple Redundant 777 Primary Flight Computer" </ref>=====<br />
<br />
In modern computers, the redundant flight control computations are carried out by multiprocessor systems. The triple redundant 777 primary flight computer, has the architecture as shown in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Fig13.png Figure 13].<br />
<br />
The system has three primary flight control computers, each of them having three lanes with different processors. The flight control program is compiled for each of the processors which get the input data from the same data bus but drive the output on their individual control bus. Thus each processor executes different instructions but they process the same data. Thus, it is the best suited example of Multiple Instruction Single Data (MISD) architecture.<br />
<br />
The three processors selected for the flight control system of [http://en.wikipedia.org/wiki/Boeing_777 Boeing 777] were [http://en.wikipedia.org/wiki/Intel_80486 Intel 80486], [http://en.wikipedia.org/wiki/Motorola_68040 Motorola 68040] and [http://en.wikipedia.org/wiki/AMD_Am29000 AMD 29050]. The dissimilar processors lead to dissimilar interface hardware circuits and compilers. Each lane of the flight control computer is data synchronized with the other lanes so that all of the lanes read the same frame of data from the flight sensors. As the outputs of each lane can be different, the median value of the outputs is used to select the output of the lane to be considered. The lane which has the median value select hardware selected is said to be in “command mode” whereas the other lanes are said to be in “monitoring mode”. It receives the data from the other Primary Flight Computer (PFC) lanes and performs a median select of the outputs. This provides a fault blocking mechanism before the fault detection and identification by the cross-lane monitoring system. Thus, the MISD based multi computer architecture is capable of detecting generic errors in compilers or in complex hardware devices providing assurance beyond reasonable doubt of the dependability of the Fly-By-Wire system.<br />
<br />
The above mentioned system clearly has individual Instruction Streams as the architecture of each processor is different, thus different instruction sets and different instruction streams. These processors have frame synchronized input data which means they have same set of data to work upon which is fed from a single data stream. Thus the flight control system can be classified under MISD architecture.<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System===<br />
<br />
The image pre-processing system of this Multiprocessor [9] , the pre-image processing time needed to be made shorter. To achieve the aim of reducing the pre-processing time, different types of image processing architectures for ‘C80 were presented. For example MIMD architecture is used for image histogram equalization; SIMD architecture for median filtering; MISD architecture for directional edge detection.<br />
<br />
The MISD architecture can be explained as follows. Here, each processor performs different operations on the same data set. It is mentioned at many places that MISD architecture was defined only for the sake of completeness of classification due to difficulty in practice to find a computer based on MISD architecture [10]. But in case of Image Processing, this assumption need not be true as here making many different operations on same image is rare and hence MISD architecture is a best fit for such cases. As an example of implementation of image processing operations for the MISD architecture the edge detection operation will be considered. The edges should be detected in four different directions (north, south, west and east) by the medium of four different filter masks. Each of the parallel processors of the TMS320C80 calculates one of the directional edges. Finally one obtains four images with detected edges, each in different direction.<br />
<br />
The image processing in computer is very often a multistage process composed of many stages. For example, an algorithm detecting contours of the objects in an image. The algorithm for achieving the before mentioned is composed of four basic image processing operation. The image acquired from the camera in the first step undergoes low-pass filtering, during which it is smoothed and noise is also eliminated. In the next step image is thresholding is performed and each pixel gets black or white depending on that if its grey level is below or above threshold value. The resulting image is a binary image from which edges are detected by the usage of the Laplace filter. In the last step the image with detected contours undergoes logic filtering in the purpose of elimination of single black pixels placed on the white background. For implementing the above processes parallel, a five-stage pipeline was used. The first stage of this pipeline constitutes the master processor which accounts for control of the image acquisition process. Further pipe line stages were implemented for 4 parallel processors as mentioned in table 1.<br />
<br />
Stage Processor Operation<br />
1 MP Image acquisition<br />
2 PP0 Low-pass Filtering<br />
3 PP1 Thresholding<br />
4 PP2 Laplace Filtering<br />
5 PP3 Logic Filtering<br />
Table 1: Pipeline structure for image edge detection [9]<br />
<br />
=='''References'''==<br />
<references/><br />
<br />
=='''Glossary'''==<br />
*'''CMU''': Carnegie Mellon University<br />
<br />
*'''CU''': Control Unit<br />
<br />
*'''DS''': Data Stream<br />
<br />
*'''Fly-By-Wire''': System that replaces the conventional manual flight controls of an aircraft with an electronic interface<br />
<br />
*'''FPGA''': Field Programmable Gate Array<br />
<br />
*'''Heterogeneous Systems''': A multiprocessor system with different kind of processors<br />
<br />
*'''Homogeneous System''': A multiprocessor system with same kind of processors<br />
<br />
*'''ILP''': Instruction Level Parallelism<br />
<br />
*'''IS''': Instruction Stream<br />
<br />
*'''MIMD''': Multiple Instruction Multiple Data<br />
<br />
*'''MISD''': Multiple Instruction Single Data<br />
<br />
*'''PE''': Processing Element<br />
<br />
*'''PFC''': Primary Flight Computer<br />
<br />
*'''SAPO''': Short for Samočinný počítač<br />
<br />
*'''SIMD''': Single Instruction Multiple Data<br />
<br />
*'''SISD''': Single Instruction Single Data<br />
<br />
*'''VLSI''': Very Large Scale Integration</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/1a_sp&diff=71774CSC/ECE 506 Spring 2013/1a sp2013-02-04T22:48:55Z<p>Schakra8: /* MISD Architectures within the Context of Flynn’s Taxonomyhttp://en.wikipedia.org/wiki/Flynn's_taxonomyhttp://www.phy.ornl.gov/csep/ca/node11.html */</p>
<hr />
<div>==Multiple Instruction Single Data (MISD) Architecture==<br />
<br />
Multiple Instruction Single Data, or MISD, architecture is one of the four general categories of multi-processor or multi-core architectures described by Flynn's Taxonomy.<br />
<br />
==MISD Architectures within the Context of Flynn’s Taxonomy<ref>http://en.wikipedia.org/wiki/Flynn's_taxonomy</ref><ref>http://www.phy.ornl.gov/csep/ca/node11.html</ref>==<br />
<br />
Flynn's Taxonomy categorizes parallel computing architectures based on the number of information streams that flowed into the processor.<br />
According to Flynn's Taxonomy, there are two kinds of logical streams of information that flows into a parallel computer, the Instruction Stream and the Data stream. Note that for the purposes of classification using Flynn’s Taxonomy, it does not matter whether or not the streams are physically independent of each other on the hardware, only that the streams can be logically treated as independent.<br />
Based on how many of each type of stream flows into the parallel computer (single vs. multiple), Flynn's Taxonomy divides parallel architectures into four categories. They are Single Instruction Single Data (SISD), Single Instruction Multiple Data (SIMD), Multiple Instruction Single Data (MISD), and Multiple Instruction Multiple Data (MIMD).<br />
<br />
The four classifications defined by Flynn are based upon the number of concurrent instruction (or control) and data streams available in the architecture are<ref>https://computing.llnl.gov/tutorials/parallel_comp/#Flynn</ref><br />
<br />
[[Image:Flynn's Taxonomy.PNG|thumb|center|400px|Figure 1. [http://en.wikipedia.org/wiki/Michael_J._Flynn Flynn]'s Taxonomy [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
==Overview of the MISD Architecture==<br />
<br />
In order to be categorized as MISD architecture, there must be at least two Instruction streams and there can only be one Data stream as shown on Figure 2. As per the description according to Flynn’s Taxonomy, each processing elements should receive the same data and execute different instructions on the same data simultaneously.<br />
<br />
[[Image:MISD.PNG|thumb|right|100px|Figure 2. MISD Architecture [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
There are very few implementations of the classical MISD architecture as described by Flynn’s Taxonomy, none of which are commercial, general purpose parallel computers. Instead, MISD architectures are mainly used to create custom hardware in order to solve specific problems. The majority of the implementations have been created are in academia.<br />
Over the years, several architectures have been described as MISD even though the architectures do not completely follow the description in Flynn’s Taxonomy. One common example is the Pipeline architecture where the data is passed from processing element to processing element and each processing element executes a different instruction. It is often argued that this is not a true MISD architecture as the data changes as it moves through the pipeline. Similarly, other architectures or applications exist which may be called MISD by some while others disagree. The following section looks at some of those implementations.<br />
<br />
Pipeline architectures belong to this type, though a purist might say that the data is different after processing by each stage in the pipeline. Fault-tolerant computers executing the same instructions redundantly in order to detect and mask errors, in a manner known as task replication, may be considered to belong to this type. Not many instances of this architecture exist, as MIMD and SIMD are often more appropriate for common data parallel techniques. Specifically, they allow better scaling and use of computational resources than MISD does. <br />
<br />
However, one prominent example of MISD in computing is the Space Shuttle flight control computers. Another example of this machine is the systolic array, such as the [http://www.cs.cmu.edu/~iwarp/ CMU iWrap] [BORKAR et al., 1990]. All the elements in this array are controlled by a global clock. On each cycle, an element will read a piece of data from one of its neighbors, perform a simple operation (e.g. add the incoming element to a stored value), and prepare a value to be written to a neighbor on the next step.<br />
<br />
==Implementations of MISD architecture==<br />
<br />
Some major implementations of MISD are in the fields of systolic arrays, pattern matching, flight control systems and fault tolerance systems. These MISD implementations are discussed below.<br />
<br />
===Systolic Array===<br />
<br />
A systolic array is an arrangement of processors in an array where data flows synchronously across the array between neighbors, usually with different data flowing in different directions. Each Processor at each step takes in data from one or more neighbors, processes it and, in the next step, outputs results in the opposite direction.<br />
<br />
The systolic array paradigm, data-stream-driven by data counters, is the counterpart of the [http://en.wikipedia.org/wiki/Von_Neumann_model von Neumann paradigm], instruction-stream-driven by a program counter. Because a systolic array usually sends and receives multiple data streams, and multiple data counters are needed to generate these data streams, it supports data parallelism. The name derives from analogy with the regular pumping of blood by the heart.<ref>http://en.wikipedia.org/wiki/Systolic_array</ref><br />
<br />
====Type of Systolic Arrays<ref>http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf General Purpose Systolic Arrays </ref>====<br />
<br />
=====Special-purpose systolic array=====<br />
[[Image:systolic_1.png|thumb|right|250px|Figure 3: The algorithm for the sum of a scalar product, computed in systolic element [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
[[Image:systolic_2.png|thumb|right|250px|Figure 7: The systolic product of two 3x3 matrices [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
An array of hardwired systolic processing elements tailored for a specific application. Typically, many tens or hundreds of cells fit on a single chip. One of the major applications of special-purpose systolic array is in matrix operations. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Systolic_1.png Figure 3] illustrates the algorithm for the sum of a scalar product, computed in a single systolic element. Here, a’s and b’s are synchronously shifted through the processing element to be available for next element. These data synchronously exits the processing element unmodified for the next element. The sum of the products is then shifted out of the accumulator.<br />
<br />
=====General-purpose systolic array=====<br />
<br />
It is an array of systolic processing elements, which gets adapted to a variety of applications via programming or reconfiguration. Array topologies can be either programmable or reconfigurable. Likewise, array cells are either programmable or reconfigurable. This is referred to as Systolic topologies.<br />
<br />
======Programmable Systolic Array======<br />
<br />
It is an array of programmable systolic elements that operates either in SIMD or MIMD fashion. Either the arrays interconnect or each processing unit is programmable and a program controls dataflow through the elements. Programmable systolic arrays are programmable either at a high level or a low level. At either level, programmable arrays can be categorized as either SIMD or MIMD machines.<br />
<br />
[[Image:systolic_4.png|thumb|right|250px|Figure 9: General organization of MIMD programmable linear systolic arrays [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
<br />
For the latter approach, the workstation downloads a program to each MIMD (Figure 7) systolic cell. Each cell may be loaded with a different program, or all the cells in the array may be loaded with the same program. Each cell's architecture is somewhat similar to the conventional von Neumann architecture: It contains a control unit, an ALU, and local memory. MIMD systolic cells have more local memory than their SIMD counterparts to support the von Neumann-style organization.<br />
<br />
This architecture is defined as Multiple Instruction Multiple Data (MIMD) architecture in [http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf 5]. The architecture has multiple instruction streams for the PEs and a single data stream passing through all the PEs. Thus, it can also be defined as Multiple Instruction Single Data (MISD) architecture. The architecture of Systolic array configuration are controversial as explained in the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Architecture_of_systolic_arrays_as_against_MISD_architecture section 4.1.2.]<br />
<br />
======Reconfigurable Systolic Array======<br />
[[Image:reconfig.jpg|thumb|right|250px|Figure 10: Block Diagram of the RSA Architecture [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
It is an array of systolic elements that can be programmed at the lowest level. Recent gate density advances in FPGA technology have produced a low-level, reconfigurable systolic array architecture that bridges the gap between special-purpose arrays and the more versatile, programmable general-purpose arrays. The FPGA architecture is unusual because a single hardware platform can be logically reconfigured as an exact duplicate of a special-purpose systolic array. <br />
<br />
The RSA circuit design is based on systolic array architecture consisting of PEs interconnected via SWs as depicted in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Reconfig.jpg Figure 10]. The homogeneous characteristic of the Reconfigurable Systolic Array (RSA) architecture, where each reconfigurable processing element (PE) cell is connected to its nearest neighbors via configurable switch (SW) elements, enables array expansion for parallel processing and facilitates time sharing computation of high-throughput data by individual PEs. Both the PEs and SWs can be reconfigured dynamically with the former as an arithmetic processor and the latter as a flexible router linking the neighboring PE cells. The RSA shifts reconfiguration and input signals into the PEs and SWs on separate data bus which enables the circuit to continue its operation while the reconfiguration is in process.<br />
<br />
====Architecture of systolic arrays as against MISD architecture====<br />
[[Image:comp.png|thumb|right|250px|Figure 11.Comparison between Architecture of systolic arrays and MISD]]<br />
<br />
As from the above mentioned configurations of the Systolic Arrays, it is seen that generally the configurations have multiple processing elements executing different instructions from dedicated instruction streams for each processing element. There is a single data stream that connects the adjacent PEs. Thus, systolic array can be defined as an MISD architecture.<br />
<br />
Many authors say that as the data read as input by one processing element is processed data output of the adjacent PE. The data stream cannot be considered as single because all the data paths do not carry the same data to all the PEs. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Comp.png Figure 11] shows the difference between the Data Stream for Systolic Arrays and the MISD architecture. Thus the systolic array should be considered as “Multiple Data” architecture and not Single Data architecture.<br />
<br />
===Fault Tolerant Systems<ref>http://en.wikipedia.org/wiki/Fault-tolerant_computer_system#Types_of_fault_tolerance</ref>===<br />
<br />
The fault tolerant systems are designed to handle the possible failures in software, hardware or interfaces. The hardware faults include hard disk failures, input or output device failures, etc. and the software and interface faults include driver failures; operator errors, installing unexpected software etc. The hardware faults can be detected and identified by implementing redundant hardware and multiple backups. The software faults can be tolerable by removing the program errors by executing the software redundantly or by implementing small programs that take over the tasks that crash or generate errors.<br />
<br />
[[Image:fault.png|thumb|right|250px|Figure 12 MISD as fault tolerant architecture]]<br />
<br />
In computer systems, the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Single_Instruction.2C_Multiple_Data_streams_.28SIMD.29 SIMD], [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD] and [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instruction.2C_Multiple_Data_streams_.28MIMD.29 MIMD] architectures facilitate the implementation of the fault tolerance systems by multiple instruction streams or multiple data streams or both. Fault tolerance on computations can be implemented by multiple processors (likely with different architectures) executing the algorithms on the same set of data. The output of each processor is compared with that of the others and M out of N majority voting method is used to determine the faulty processor. Thus MISD architecture is utilized to get the fault tolerance on critical computations.<br />
<br />
There are various examples of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture] being used as fault tolerant architecture. The major examples being flight control systems, nuclear power plants, satellite systems, super collider experiment systems, etc. Here, the flight control system is explained as an example of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture].<br />
<br />
====The Flight Control System – MISD Example for fault tolerance====<br />
<br />
A [http://en.wikipedia.org/wiki/Fly-by-wire Fly-By-Wire] system is used to replace the manual flight control by an electronic control interface. The movements of the flight control in the cockpit are converted to electronic signals and are transmitted to the actuators by wires. The control computers use the feedback from the sensors to compute and control the movement of the actuators to provide the expected response. These computers also perform the task to stabilize the aircraft and perform other tasks without the knowledge of the pilot. Flight control systems must meet extremely high levels of accuracy and functional integrity.<br />
<br />
There are redundant flight control computers present in the flight control system. If one of the flight-control computers crashes, gets damaged or is affected by electromagnetic pulses, the other computer can overrule the faulty one and hence the flight of the aircraft is unharmed. [[Image:fig13.png|thumb|right|250px|Figure 13: Architecture of triple redundant 777 primary flight computer [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-7 6]]]The number of redundant flight control computers is generally more than two, so that any computer whose results disagree with the others is ruled out to be faulty and is either ignored or rebooted.[[Image:fig14.png|thumb|right|250px|Figure 14: PFC with instruction and data streams]]<br />
<br />
=====Multiple Processors Implementation in Boeing 777<ref>http://www.citemaster.net/getdoc/8767/R8.pdf Y.C. (Bob) Yeh, Boeing Commercial Airplane Group, "Triple-Triple Redundant 777 Primary Flight Computer" </ref>=====<br />
<br />
In modern computers, the redundant flight control computations are carried out by multiprocessor systems. The triple redundant 777 primary flight computer, has the architecture as shown in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Fig13.png Figure 13].<br />
<br />
The system has three primary flight control computers, each of them having three lanes with different processors. The flight control program is compiled for each of the processors which get the input data from the same data bus but drive the output on their individual control bus. Thus each processor executes different instructions but they process the same data. Thus, it is the best suited example of Multiple Instruction Single Data (MISD) architecture.<br />
<br />
The three processors selected for the flight control system of [http://en.wikipedia.org/wiki/Boeing_777 Boeing 777] were [http://en.wikipedia.org/wiki/Intel_80486 Intel 80486], [http://en.wikipedia.org/wiki/Motorola_68040 Motorola 68040] and [http://en.wikipedia.org/wiki/AMD_Am29000 AMD 29050]. The dissimilar processors lead to dissimilar interface hardware circuits and compilers. Each lane of the flight control computer is data synchronized with the other lanes so that all of the lanes read the same frame of data from the flight sensors. As the outputs of each lane can be different, the median value of the outputs is used to select the output of the lane to be considered. The lane which has the median value select hardware selected is said to be in “command mode” whereas the other lanes are said to be in “monitoring mode”. It receives the data from the other Primary Flight Computer (PFC) lanes and performs a median select of the outputs. This provides a fault blocking mechanism before the fault detection and identification by the cross-lane monitoring system. Thus, the MISD based multi computer architecture is capable of detecting generic errors in compilers or in complex hardware devices providing assurance beyond reasonable doubt of the dependability of the Fly-By-Wire system.<br />
<br />
The above mentioned system clearly has individual Instruction Streams as the architecture of each processor is different, thus different instruction sets and different instruction streams. These processors have frame synchronized input data which means they have same set of data to work upon which is fed from a single data stream. Thus the flight control system can be classified under MISD architecture.<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System===<br />
<br />
The image pre-processing system of this Multiprocessor [9] , the pre-image processing time needed to be made shorter. To achieve the aim of reducing the pre-processing time, different types of image processing architectures for ‘C80 were presented. For example MIMD architecture is used for image histogram equalization; SIMD architecture for median filtering; MISD architecture for directional edge detection.<br />
<br />
The MISD architecture can be explained as follows. Here, each processor performs different operations on the same data set. It is mentioned at many places that MISD architecture was defined only for the sake of completeness of classification due to difficulty in practice to find a computer based on MISD architecture [10]. But in case of Image Processing, this assumption need not be true as here making many different operations on same image is rare and hence MISD architecture is a best fit for such cases. As an example of implementation of image processing operations for the MISD architecture the edge detection operation will be considered. The edges should be detected in four different directions (north, south, west and east) by the medium of four different filter masks. Each of the parallel processors of the TMS320C80 calculates one of the directional edges. Finally one obtains four images with detected edges, each in different direction.<br />
<br />
The image processing in computer is very often a multistage process composed of many stages. For example, an algorithm detecting contours of the objects in an image. The algorithm for achieving the before mentioned is composed of four basic image processing operation. The image acquired from the camera in the first step undergoes low-pass filtering, during which it is smoothed and noise is also eliminated. In the next step image is thresholding is performed and each pixel gets black or white depending on that if its grey level is below or above threshold value. The resulting image is a binary image from which edges are detected by the usage of the Laplace filter. In the last step the image with detected contours undergoes logic filtering in the purpose of elimination of single black pixels placed on the white background. For implementing the above processes parallel, a five-stage pipeline was used. The first stage of this pipeline constitutes the master processor which accounts for control of the image acquisition process. Further pipe line stages were implemented for 4 parallel processors as mentioned in table 1.<br />
<br />
Stage Processor Operation<br />
1 MP Image acquisition<br />
2 PP0 Low-pass Filtering<br />
3 PP1 Thresholding<br />
4 PP2 Laplace Filtering<br />
5 PP3 Logic Filtering<br />
Table 1: Pipeline structure for image edge detection [9]<br />
<br />
=='''References'''==<br />
<references/><br />
<br />
=='''Glossary'''==<br />
*'''CMU''': Carnegie Mellon University<br />
<br />
*'''CU''': Control Unit<br />
<br />
*'''DS''': Data Stream<br />
<br />
*'''Fly-By-Wire''': System that replaces the conventional manual flight controls of an aircraft with an electronic interface<br />
<br />
*'''FPGA''': Field Programmable Gate Array<br />
<br />
*'''Heterogeneous Systems''': A multiprocessor system with different kind of processors<br />
<br />
*'''Homogeneous System''': A multiprocessor system with same kind of processors<br />
<br />
*'''ILP''': Instruction Level Parallelism<br />
<br />
*'''IS''': Instruction Stream<br />
<br />
*'''MIMD''': Multiple Instruction Multiple Data<br />
<br />
*'''MISD''': Multiple Instruction Single Data<br />
<br />
*'''PE''': Processing Element<br />
<br />
*'''PFC''': Primary Flight Computer<br />
<br />
*'''SAPO''': Short for Samočinný počítač<br />
<br />
*'''SIMD''': Single Instruction Multiple Data<br />
<br />
*'''SISD''': Single Instruction Single Data<br />
<br />
*'''VLSI''': Very Large Scale Integration</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/1a_sp&diff=71771CSC/ECE 506 Spring 2013/1a sp2013-02-04T22:47:31Z<p>Schakra8: /* Implementations of MISD architecture */</p>
<hr />
<div>==Multiple Instruction Single Data (MISD) Architecture==<br />
<br />
Multiple Instruction Single Data, or MISD, architecture is one of the four general categories of multi-processor or multi-core architectures described by Flynn's Taxonomy.<br />
<br />
==MISD Architectures within the Context of Flynn’s Taxonomy<ref>http://en.wikipedia.org/wiki/Flynn's_taxonomy</ref><ref>http://www.phy.ornl.gov/csep/ca/node11.html</ref>==<br />
<br />
Flynn's Taxonomy categorizes parallel computing architectures based on the number of information streams that flowed into the processor.<br />
According to Flynn's Taxonomy, there are two kinds of logical streams of information that flows into a parallel computer, the Instruction Stream and the Data stream. Note that for the purposes of classification using Flynn’s Taxonomy, it does not matter whether or not the streams are physically independent of each other on the hardware, only that the streams can be logically treated as independent.<br />
Based on how many of each type of stream flows into the parallel computer (single vs. multiple), Flynn's Taxonomy divides parallel architectures into four categories. They are Single Instruction Single Data (SISD), Single Instruction Multiple Data (SIMD), Multiple Instruction Single Data (MISD), and Multiple Instruction Multiple Data (MIMD).<br />
<br />
The four classifications defined by Flynn are based upon the number of concurrent instruction (or control) and data streams available in the architecture are<ref>https://computing.llnl.gov/tutorials/parallel_comp/#Flynn</ref><br />
<br />
[[Image:Flynn's Taxonomy.PNG|thumb|center|400px|Figure 1. [http://en.wikipedia.org/wiki/Michael_J._Flynn Flynn]'s Taxonomy [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]] <br />
<br />
==Overview of the MISD Architecture==<br />
<br />
In order to be categorized as MISD architecture, there must be at least two Instruction streams and there can only be one Data stream as shown on Figure 2. As per the description according to Flynn’s Taxonomy, each processing elements should receive the same data and execute different instructions on the same data simultaneously.<br />
<br />
[[Image:MISD.PNG|thumb|right|100px|Figure 2. MISD Architecture [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
There are very few implementations of the classical MISD architecture as described by Flynn’s Taxonomy, none of which are commercial, general purpose parallel computers. Instead, MISD architectures are mainly used to create custom hardware in order to solve specific problems. The majority of the implementations have been created are in academia.<br />
Over the years, several architectures have been described as MISD even though the architectures do not completely follow the description in Flynn’s Taxonomy. One common example is the Pipeline architecture where the data is passed from processing element to processing element and each processing element executes a different instruction. It is often argued that this is not a true MISD architecture as the data changes as it moves through the pipeline. Similarly, other architectures or applications exist which may be called MISD by some while others disagree. The following section looks at some of those implementations.<br />
<br />
Pipeline architectures belong to this type, though a purist might say that the data is different after processing by each stage in the pipeline. Fault-tolerant computers executing the same instructions redundantly in order to detect and mask errors, in a manner known as task replication, may be considered to belong to this type. Not many instances of this architecture exist, as MIMD and SIMD are often more appropriate for common data parallel techniques. Specifically, they allow better scaling and use of computational resources than MISD does. <br />
<br />
However, one prominent example of MISD in computing is the Space Shuttle flight control computers. Another example of this machine is the systolic array, such as the [http://www.cs.cmu.edu/~iwarp/ CMU iWrap] [BORKAR et al., 1990]. All the elements in this array are controlled by a global clock. On each cycle, an element will read a piece of data from one of its neighbors, perform a simple operation (e.g. add the incoming element to a stored value), and prepare a value to be written to a neighbor on the next step.<br />
<br />
==Implementations of MISD architecture==<br />
<br />
Some major implementations of MISD are in the fields of systolic arrays, pattern matching, flight control systems and fault tolerance systems. These MISD implementations are discussed below.<br />
<br />
===Systolic Array===<br />
<br />
A systolic array is an arrangement of processors in an array where data flows synchronously across the array between neighbors, usually with different data flowing in different directions. Each Processor at each step takes in data from one or more neighbors, processes it and, in the next step, outputs results in the opposite direction.<br />
<br />
The systolic array paradigm, data-stream-driven by data counters, is the counterpart of the [http://en.wikipedia.org/wiki/Von_Neumann_model von Neumann paradigm], instruction-stream-driven by a program counter. Because a systolic array usually sends and receives multiple data streams, and multiple data counters are needed to generate these data streams, it supports data parallelism. The name derives from analogy with the regular pumping of blood by the heart.<ref>http://en.wikipedia.org/wiki/Systolic_array</ref><br />
<br />
====Type of Systolic Arrays<ref>http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf General Purpose Systolic Arrays </ref>====<br />
<br />
=====Special-purpose systolic array=====<br />
[[Image:systolic_1.png|thumb|right|250px|Figure 3: The algorithm for the sum of a scalar product, computed in systolic element [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
[[Image:systolic_2.png|thumb|right|250px|Figure 7: The systolic product of two 3x3 matrices [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
An array of hardwired systolic processing elements tailored for a specific application. Typically, many tens or hundreds of cells fit on a single chip. One of the major applications of special-purpose systolic array is in matrix operations. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Systolic_1.png Figure 3] illustrates the algorithm for the sum of a scalar product, computed in a single systolic element. Here, a’s and b’s are synchronously shifted through the processing element to be available for next element. These data synchronously exits the processing element unmodified for the next element. The sum of the products is then shifted out of the accumulator.<br />
<br />
=====General-purpose systolic array=====<br />
<br />
It is an array of systolic processing elements, which gets adapted to a variety of applications via programming or reconfiguration. Array topologies can be either programmable or reconfigurable. Likewise, array cells are either programmable or reconfigurable. This is referred to as Systolic topologies.<br />
<br />
======Programmable Systolic Array======<br />
<br />
It is an array of programmable systolic elements that operates either in SIMD or MIMD fashion. Either the arrays interconnect or each processing unit is programmable and a program controls dataflow through the elements. Programmable systolic arrays are programmable either at a high level or a low level. At either level, programmable arrays can be categorized as either SIMD or MIMD machines.<br />
<br />
[[Image:systolic_4.png|thumb|right|250px|Figure 9: General organization of MIMD programmable linear systolic arrays [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
<br />
For the latter approach, the workstation downloads a program to each MIMD (Figure 7) systolic cell. Each cell may be loaded with a different program, or all the cells in the array may be loaded with the same program. Each cell's architecture is somewhat similar to the conventional von Neumann architecture: It contains a control unit, an ALU, and local memory. MIMD systolic cells have more local memory than their SIMD counterparts to support the von Neumann-style organization.<br />
<br />
This architecture is defined as Multiple Instruction Multiple Data (MIMD) architecture in [http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf 5]. The architecture has multiple instruction streams for the PEs and a single data stream passing through all the PEs. Thus, it can also be defined as Multiple Instruction Single Data (MISD) architecture. The architecture of Systolic array configuration are controversial as explained in the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Architecture_of_systolic_arrays_as_against_MISD_architecture section 4.1.2.]<br />
<br />
======Reconfigurable Systolic Array======<br />
[[Image:reconfig.jpg|thumb|right|250px|Figure 10: Block Diagram of the RSA Architecture [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
It is an array of systolic elements that can be programmed at the lowest level. Recent gate density advances in FPGA technology have produced a low-level, reconfigurable systolic array architecture that bridges the gap between special-purpose arrays and the more versatile, programmable general-purpose arrays. The FPGA architecture is unusual because a single hardware platform can be logically reconfigured as an exact duplicate of a special-purpose systolic array. <br />
<br />
The RSA circuit design is based on systolic array architecture consisting of PEs interconnected via SWs as depicted in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Reconfig.jpg Figure 10]. The homogeneous characteristic of the Reconfigurable Systolic Array (RSA) architecture, where each reconfigurable processing element (PE) cell is connected to its nearest neighbors via configurable switch (SW) elements, enables array expansion for parallel processing and facilitates time sharing computation of high-throughput data by individual PEs. Both the PEs and SWs can be reconfigured dynamically with the former as an arithmetic processor and the latter as a flexible router linking the neighboring PE cells. The RSA shifts reconfiguration and input signals into the PEs and SWs on separate data bus which enables the circuit to continue its operation while the reconfiguration is in process.<br />
<br />
====Architecture of systolic arrays as against MISD architecture====<br />
[[Image:comp.png|thumb|right|250px|Figure 11.Comparison between Architecture of systolic arrays and MISD]]<br />
<br />
As from the above mentioned configurations of the Systolic Arrays, it is seen that generally the configurations have multiple processing elements executing different instructions from dedicated instruction streams for each processing element. There is a single data stream that connects the adjacent PEs. Thus, systolic array can be defined as an MISD architecture.<br />
<br />
Many authors say that as the data read as input by one processing element is processed data output of the adjacent PE. The data stream cannot be considered as single because all the data paths do not carry the same data to all the PEs. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Comp.png Figure 11] shows the difference between the Data Stream for Systolic Arrays and the MISD architecture. Thus the systolic array should be considered as “Multiple Data” architecture and not Single Data architecture.<br />
<br />
===Fault Tolerant Systems<ref>http://en.wikipedia.org/wiki/Fault-tolerant_computer_system#Types_of_fault_tolerance</ref>===<br />
<br />
The fault tolerant systems are designed to handle the possible failures in software, hardware or interfaces. The hardware faults include hard disk failures, input or output device failures, etc. and the software and interface faults include driver failures; operator errors, installing unexpected software etc. The hardware faults can be detected and identified by implementing redundant hardware and multiple backups. The software faults can be tolerable by removing the program errors by executing the software redundantly or by implementing small programs that take over the tasks that crash or generate errors.<br />
<br />
[[Image:fault.png|thumb|right|250px|Figure 12 MISD as fault tolerant architecture]]<br />
<br />
In computer systems, the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Single_Instruction.2C_Multiple_Data_streams_.28SIMD.29 SIMD], [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD] and [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instruction.2C_Multiple_Data_streams_.28MIMD.29 MIMD] architectures facilitate the implementation of the fault tolerance systems by multiple instruction streams or multiple data streams or both. Fault tolerance on computations can be implemented by multiple processors (likely with different architectures) executing the algorithms on the same set of data. The output of each processor is compared with that of the others and M out of N majority voting method is used to determine the faulty processor. Thus MISD architecture is utilized to get the fault tolerance on critical computations.<br />
<br />
There are various examples of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture] being used as fault tolerant architecture. The major examples being flight control systems, nuclear power plants, satellite systems, super collider experiment systems, etc. Here, the flight control system is explained as an example of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture].<br />
<br />
====The Flight Control System – MISD Example for fault tolerance====<br />
<br />
A [http://en.wikipedia.org/wiki/Fly-by-wire Fly-By-Wire] system is used to replace the manual flight control by an electronic control interface. The movements of the flight control in the cockpit are converted to electronic signals and are transmitted to the actuators by wires. The control computers use the feedback from the sensors to compute and control the movement of the actuators to provide the expected response. These computers also perform the task to stabilize the aircraft and perform other tasks without the knowledge of the pilot. Flight control systems must meet extremely high levels of accuracy and functional integrity.<br />
<br />
There are redundant flight control computers present in the flight control system. If one of the flight-control computers crashes, gets damaged or is affected by electromagnetic pulses, the other computer can overrule the faulty one and hence the flight of the aircraft is unharmed. [[Image:fig13.png|thumb|right|250px|Figure 13: Architecture of triple redundant 777 primary flight computer [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-7 6]]]The number of redundant flight control computers is generally more than two, so that any computer whose results disagree with the others is ruled out to be faulty and is either ignored or rebooted.[[Image:fig14.png|thumb|right|250px|Figure 14: PFC with instruction and data streams]]<br />
<br />
=====Multiple Processors Implementation in Boeing 777<ref>http://www.citemaster.net/getdoc/8767/R8.pdf Y.C. (Bob) Yeh, Boeing Commercial Airplane Group, "Triple-Triple Redundant 777 Primary Flight Computer" </ref>=====<br />
<br />
In modern computers, the redundant flight control computations are carried out by multiprocessor systems. The triple redundant 777 primary flight computer, has the architecture as shown in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Fig13.png Figure 13].<br />
<br />
The system has three primary flight control computers, each of them having three lanes with different processors. The flight control program is compiled for each of the processors which get the input data from the same data bus but drive the output on their individual control bus. Thus each processor executes different instructions but they process the same data. Thus, it is the best suited example of Multiple Instruction Single Data (MISD) architecture.<br />
<br />
The three processors selected for the flight control system of [http://en.wikipedia.org/wiki/Boeing_777 Boeing 777] were [http://en.wikipedia.org/wiki/Intel_80486 Intel 80486], [http://en.wikipedia.org/wiki/Motorola_68040 Motorola 68040] and [http://en.wikipedia.org/wiki/AMD_Am29000 AMD 29050]. The dissimilar processors lead to dissimilar interface hardware circuits and compilers. Each lane of the flight control computer is data synchronized with the other lanes so that all of the lanes read the same frame of data from the flight sensors. As the outputs of each lane can be different, the median value of the outputs is used to select the output of the lane to be considered. The lane which has the median value select hardware selected is said to be in “command mode” whereas the other lanes are said to be in “monitoring mode”. It receives the data from the other Primary Flight Computer (PFC) lanes and performs a median select of the outputs. This provides a fault blocking mechanism before the fault detection and identification by the cross-lane monitoring system. Thus, the MISD based multi computer architecture is capable of detecting generic errors in compilers or in complex hardware devices providing assurance beyond reasonable doubt of the dependability of the Fly-By-Wire system.<br />
<br />
The above mentioned system clearly has individual Instruction Streams as the architecture of each processor is different, thus different instruction sets and different instruction streams. These processors have frame synchronized input data which means they have same set of data to work upon which is fed from a single data stream. Thus the flight control system can be classified under MISD architecture.<br />
<br />
===Edge Detection of Images on TMS320C80 Multiprocessor System===<br />
<br />
The image pre-processing system of this Multiprocessor [9] , the pre-image processing time needed to be made shorter. To achieve the aim of reducing the pre-processing time, different types of image processing architectures for ‘C80 were presented. For example MIMD architecture is used for image histogram equalization; SIMD architecture for median filtering; MISD architecture for directional edge detection.<br />
<br />
The MISD architecture can be explained as follows. Here, each processor performs different operations on the same data set. It is mentioned at many places that MISD architecture was defined only for the sake of completeness of classification due to difficulty in practice to find a computer based on MISD architecture [10]. But in case of Image Processing, this assumption need not be true as here making many different operations on same image is rare and hence MISD architecture is a best fit for such cases. As an example of implementation of image processing operations for the MISD architecture the edge detection operation will be considered. The edges should be detected in four different directions (north, south, west and east) by the medium of four different filter masks. Each of the parallel processors of the TMS320C80 calculates one of the directional edges. Finally one obtains four images with detected edges, each in different direction.<br />
<br />
The image processing in computer is very often a multistage process composed of many stages. For example, an algorithm detecting contours of the objects in an image. The algorithm for achieving the before mentioned is composed of four basic image processing operation. The image acquired from the camera in the first step undergoes low-pass filtering, during which it is smoothed and noise is also eliminated. In the next step image is thresholding is performed and each pixel gets black or white depending on that if its grey level is below or above threshold value. The resulting image is a binary image from which edges are detected by the usage of the Laplace filter. In the last step the image with detected contours undergoes logic filtering in the purpose of elimination of single black pixels placed on the white background. For implementing the above processes parallel, a five-stage pipeline was used. The first stage of this pipeline constitutes the master processor which accounts for control of the image acquisition process. Further pipe line stages were implemented for 4 parallel processors as mentioned in table 1.<br />
<br />
Stage Processor Operation<br />
1 MP Image acquisition<br />
2 PP0 Low-pass Filtering<br />
3 PP1 Thresholding<br />
4 PP2 Laplace Filtering<br />
5 PP3 Logic Filtering<br />
Table 1: Pipeline structure for image edge detection [9]<br />
<br />
=='''References'''==<br />
<references/><br />
<br />
=='''Glossary'''==<br />
*'''CMU''': Carnegie Mellon University<br />
<br />
*'''CU''': Control Unit<br />
<br />
*'''DS''': Data Stream<br />
<br />
*'''Fly-By-Wire''': System that replaces the conventional manual flight controls of an aircraft with an electronic interface<br />
<br />
*'''FPGA''': Field Programmable Gate Array<br />
<br />
*'''Heterogeneous Systems''': A multiprocessor system with different kind of processors<br />
<br />
*'''Homogeneous System''': A multiprocessor system with same kind of processors<br />
<br />
*'''ILP''': Instruction Level Parallelism<br />
<br />
*'''IS''': Instruction Stream<br />
<br />
*'''MIMD''': Multiple Instruction Multiple Data<br />
<br />
*'''MISD''': Multiple Instruction Single Data<br />
<br />
*'''PE''': Processing Element<br />
<br />
*'''PFC''': Primary Flight Computer<br />
<br />
*'''SAPO''': Short for Samočinný počítač<br />
<br />
*'''SIMD''': Single Instruction Multiple Data<br />
<br />
*'''SISD''': Single Instruction Single Data<br />
<br />
*'''VLSI''': Very Large Scale Integration</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/1a_sp&diff=71769CSC/ECE 506 Spring 2013/1a sp2013-02-04T22:46:00Z<p>Schakra8: /* Overview of the MISD Architecture */</p>
<hr />
<div>==Multiple Instruction Single Data (MISD) Architecture==<br />
<br />
Multiple Instruction Single Data, or MISD, architecture is one of the four general categories of multi-processor or multi-core architectures described by Flynn's Taxonomy.<br />
<br />
==MISD Architectures within the Context of Flynn’s Taxonomy<ref>http://en.wikipedia.org/wiki/Flynn's_taxonomy</ref><ref>http://www.phy.ornl.gov/csep/ca/node11.html</ref>==<br />
<br />
Flynn's Taxonomy categorizes parallel computing architectures based on the number of information streams that flowed into the processor.<br />
According to Flynn's Taxonomy, there are two kinds of logical streams of information that flows into a parallel computer, the Instruction Stream and the Data stream. Note that for the purposes of classification using Flynn’s Taxonomy, it does not matter whether or not the streams are physically independent of each other on the hardware, only that the streams can be logically treated as independent.<br />
Based on how many of each type of stream flows into the parallel computer (single vs. multiple), Flynn's Taxonomy divides parallel architectures into four categories. They are Single Instruction Single Data (SISD), Single Instruction Multiple Data (SIMD), Multiple Instruction Single Data (MISD), and Multiple Instruction Multiple Data (MIMD).<br />
<br />
The four classifications defined by Flynn are based upon the number of concurrent instruction (or control) and data streams available in the architecture are<ref>https://computing.llnl.gov/tutorials/parallel_comp/#Flynn</ref><br />
<br />
[[Image:Flynn's Taxonomy.PNG|thumb|center|400px|Figure 1. [http://en.wikipedia.org/wiki/Michael_J._Flynn Flynn]'s Taxonomy [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]] <br />
<br />
==Overview of the MISD Architecture==<br />
<br />
In order to be categorized as MISD architecture, there must be at least two Instruction streams and there can only be one Data stream as shown on Figure 2. As per the description according to Flynn’s Taxonomy, each processing elements should receive the same data and execute different instructions on the same data simultaneously.<br />
<br />
[[Image:MISD.PNG|thumb|right|100px|Figure 2. MISD Architecture [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
There are very few implementations of the classical MISD architecture as described by Flynn’s Taxonomy, none of which are commercial, general purpose parallel computers. Instead, MISD architectures are mainly used to create custom hardware in order to solve specific problems. The majority of the implementations have been created are in academia.<br />
Over the years, several architectures have been described as MISD even though the architectures do not completely follow the description in Flynn’s Taxonomy. One common example is the Pipeline architecture where the data is passed from processing element to processing element and each processing element executes a different instruction. It is often argued that this is not a true MISD architecture as the data changes as it moves through the pipeline. Similarly, other architectures or applications exist which may be called MISD by some while others disagree. The following section looks at some of those implementations.<br />
<br />
Pipeline architectures belong to this type, though a purist might say that the data is different after processing by each stage in the pipeline. Fault-tolerant computers executing the same instructions redundantly in order to detect and mask errors, in a manner known as task replication, may be considered to belong to this type. Not many instances of this architecture exist, as MIMD and SIMD are often more appropriate for common data parallel techniques. Specifically, they allow better scaling and use of computational resources than MISD does. <br />
<br />
However, one prominent example of MISD in computing is the Space Shuttle flight control computers. Another example of this machine is the systolic array, such as the [http://www.cs.cmu.edu/~iwarp/ CMU iWrap] [BORKAR et al., 1990]. All the elements in this array are controlled by a global clock. On each cycle, an element will read a piece of data from one of its neighbors, perform a simple operation (e.g. add the incoming element to a stored value), and prepare a value to be written to a neighbor on the next step.<br />
<br />
==Implementations of MISD architecture==<br />
<br />
Some major implementations of MISD are in the fields of systolic arrays, pattern matching, flight control systems and fault tolerance systems. These MISD implementations are discussed below.<br />
<br />
===1. Systolic Array===<br />
<br />
A systolic array is an arrangement of processors in an array where data flows synchronously across the array between neighbors, usually with different data flowing in different directions. Each Processor at each step takes in data from one or more neighbors, processes it and, in the next step, outputs results in the opposite direction.<br />
<br />
The systolic array paradigm, data-stream-driven by data counters, is the counterpart of the [http://en.wikipedia.org/wiki/Von_Neumann_model von Neumann paradigm], instruction-stream-driven by a program counter. Because a systolic array usually sends and receives multiple data streams, and multiple data counters are needed to generate these data streams, it supports data parallelism. The name derives from analogy with the regular pumping of blood by the heart.<ref>http://en.wikipedia.org/wiki/Systolic_array</ref><br />
<br />
====1.1 Type of Systolic Arrays<ref>http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf General Purpose Systolic Arrays </ref>====<br />
<br />
=====1.1.1 Special-purpose systolic array=====<br />
[[Image:systolic_1.png|thumb|right|250px|Figure 3: The algorithm for the sum of a scalar product, computed in systolic element [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
[[Image:systolic_2.png|thumb|right|250px|Figure 7: The systolic product of two 3x3 matrices [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
An array of hardwired systolic processing elements tailored for a specific application. Typically, many tens or hundreds of cells fit on a single chip. One of the major applications of special-purpose systolic array is in matrix operations. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Systolic_1.png Figure 3] illustrates the algorithm for the sum of a scalar product, computed in a single systolic element. Here, a’s and b’s are synchronously shifted through the processing element to be available for next element. These data synchronously exits the processing element unmodified for the next element. The sum of the products is then shifted out of the accumulator.<br />
<br />
=====1.1.2 General-purpose systolic array=====<br />
<br />
It is an array of systolic processing elements, which gets adapted to a variety of applications via programming or reconfiguration. Array topologies can be either programmable or reconfigurable. Likewise, array cells are either programmable or reconfigurable. This is referred to as Systolic topologies.<br />
<br />
=====1.1.2.1 Programmable Systolic Array=====<br />
<br />
It is an array of programmable systolic elements that operates either in SIMD or MIMD fashion. Either the arrays interconnect or each processing unit is programmable and a program controls dataflow through the elements. Programmable systolic arrays are programmable either at a high level or a low level. At either level, programmable arrays can be categorized as either SIMD or MIMD machines.<br />
<br />
[[Image:systolic_4.png|thumb|right|250px|Figure 9: General organization of MIMD programmable linear systolic arrays [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
<br />
For the latter approach, the workstation downloads a program to each MIMD (Figure 7) systolic cell. Each cell may be loaded with a different program, or all the cells in the array may be loaded with the same program. Each cell's architecture is somewhat similar to the conventional von Neumann architecture: It contains a control unit, an ALU, and local memory. MIMD systolic cells have more local memory than their SIMD counterparts to support the von Neumann-style organization.<br />
<br />
This architecture is defined as Multiple Instruction Multiple Data (MIMD) architecture in [http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf 5]. The architecture has multiple instruction streams for the PEs and a single data stream passing through all the PEs. Thus, it can also be defined as Multiple Instruction Single Data (MISD) architecture. The architecture of Systolic array configuration are controversial as explained in the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Architecture_of_systolic_arrays_as_against_MISD_architecture section 4.1.2.]<br />
<br />
=====1.1.2.1 Reconfigurable Systolic Array=====<br />
[[Image:reconfig.jpg|thumb|right|250px|Figure 10: Block Diagram of the RSA Architecture [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
It is an array of systolic elements that can be programmed at the lowest level. Recent gate density advances in FPGA technology have produced a low-level, reconfigurable systolic array architecture that bridges the gap between special-purpose arrays and the more versatile, programmable general-purpose arrays. The FPGA architecture is unusual because a single hardware platform can be logically reconfigured as an exact duplicate of a special-purpose systolic array. <br />
<br />
The RSA circuit design is based on systolic array architecture consisting of PEs interconnected via SWs as depicted in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Reconfig.jpg Figure 10]. The homogeneous characteristic of the Reconfigurable Systolic Array (RSA) architecture, where each reconfigurable processing element (PE) cell is connected to its nearest neighbors via configurable switch (SW) elements, enables array expansion for parallel processing and facilitates time sharing computation of high-throughput data by individual PEs. Both the PEs and SWs can be reconfigured dynamically with the former as an arithmetic processor and the latter as a flexible router linking the neighboring PE cells. The RSA shifts reconfiguration and input signals into the PEs and SWs on separate data bus which enables the circuit to continue its operation while the reconfiguration is in process.<br />
<br />
====1.2 Architecture of systolic arrays as against MISD architecture====<br />
[[Image:comp.png|thumb|right|250px|Figure 11.Comparison between Architecture of systolic arrays and MISD]]<br />
<br />
As from the above mentioned configurations of the Systolic Arrays, it is seen that generally the configurations have multiple processing elements executing different instructions from dedicated instruction streams for each processing element. There is a single data stream that connects the adjacent PEs. Thus, systolic array can be defined as an MISD architecture.<br />
<br />
Many authors say that as the data read as input by one processing element is processed data output of the adjacent PE. The data stream cannot be considered as single because all the data paths do not carry the same data to all the PEs. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Comp.png Figure 11] shows the difference between the Data Stream for Systolic Arrays and the MISD architecture. Thus the systolic array should be considered as “Multiple Data” architecture and not Single Data architecture.<br />
<br />
===2. Fault Tolerant Systems<ref>http://en.wikipedia.org/wiki/Fault-tolerant_computer_system#Types_of_fault_tolerance</ref>===<br />
<br />
The fault tolerant systems are designed to handle the possible failures in software, hardware or interfaces. The hardware faults include hard disk failures, input or output device failures, etc. and the software and interface faults include driver failures; operator errors, installing unexpected software etc. The hardware faults can be detected and identified by implementing redundant hardware and multiple backups. The software faults can be tolerable by removing the program errors by executing the software redundantly or by implementing small programs that take over the tasks that crash or generate errors.<br />
<br />
[[Image:fault.png|thumb|right|250px|Figure 12 MISD as fault tolerant architecture]]<br />
<br />
In computer systems, the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Single_Instruction.2C_Multiple_Data_streams_.28SIMD.29 SIMD], [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD] and [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instruction.2C_Multiple_Data_streams_.28MIMD.29 MIMD] architectures facilitate the implementation of the fault tolerance systems by multiple instruction streams or multiple data streams or both. Fault tolerance on computations can be implemented by multiple processors (likely with different architectures) executing the algorithms on the same set of data. The output of each processor is compared with that of the others and M out of N majority voting method is used to determine the faulty processor. Thus MISD architecture is utilized to get the fault tolerance on critical computations.<br />
<br />
There are various examples of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture] being used as fault tolerant architecture. The major examples being flight control systems, nuclear power plants, satellite systems, super collider experiment systems, etc. Here, the flight control system is explained as an example of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture].<br />
<br />
====2.1 The Flight Control System – MISD Example for fault tolerance====<br />
<br />
A [http://en.wikipedia.org/wiki/Fly-by-wire Fly-By-Wire] system is used to replace the manual flight control by an electronic control interface. The movements of the flight control in the cockpit are converted to electronic signals and are transmitted to the actuators by wires. The control computers use the feedback from the sensors to compute and control the movement of the actuators to provide the expected response. These computers also perform the task to stabilize the aircraft and perform other tasks without the knowledge of the pilot. Flight control systems must meet extremely high levels of accuracy and functional integrity.<br />
<br />
There are redundant flight control computers present in the flight control system. If one of the flight-control computers crashes, gets damaged or is affected by electromagnetic pulses, the other computer can overrule the faulty one and hence the flight of the aircraft is unharmed. [[Image:fig13.png|thumb|right|250px|Figure 13: Architecture of triple redundant 777 primary flight computer [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-7 6]]]The number of redundant flight control computers is generally more than two, so that any computer whose results disagree with the others is ruled out to be faulty and is either ignored or rebooted.[[Image:fig14.png|thumb|right|250px|Figure 14: PFC with instruction and data streams]]<br />
<br />
=====Multiple Processors Implementation in Boeing 777<ref>http://www.citemaster.net/getdoc/8767/R8.pdf Y.C. (Bob) Yeh, Boeing Commercial Airplane Group, "Triple-Triple Redundant 777 Primary Flight Computer" </ref>=====<br />
<br />
In modern computers, the redundant flight control computations are carried out by multiprocessor systems. The triple redundant 777 primary flight computer, has the architecture as shown in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Fig13.png Figure 13].<br />
<br />
The system has three primary flight control computers, each of them having three lanes with different processors. The flight control program is compiled for each of the processors which get the input data from the same data bus but drive the output on their individual control bus. Thus each processor executes different instructions but they process the same data. Thus, it is the best suited example of Multiple Instruction Single Data (MISD) architecture.<br />
<br />
The three processors selected for the flight control system of [http://en.wikipedia.org/wiki/Boeing_777 Boeing 777] were [http://en.wikipedia.org/wiki/Intel_80486 Intel 80486], [http://en.wikipedia.org/wiki/Motorola_68040 Motorola 68040] and [http://en.wikipedia.org/wiki/AMD_Am29000 AMD 29050]. The dissimilar processors lead to dissimilar interface hardware circuits and compilers. Each lane of the flight control computer is data synchronized with the other lanes so that all of the lanes read the same frame of data from the flight sensors. As the outputs of each lane can be different, the median value of the outputs is used to select the output of the lane to be considered. The lane which has the median value select hardware selected is said to be in “command mode” whereas the other lanes are said to be in “monitoring mode”. It receives the data from the other Primary Flight Computer (PFC) lanes and performs a median select of the outputs. This provides a fault blocking mechanism before the fault detection and identification by the cross-lane monitoring system. Thus, the MISD based multi computer architecture is capable of detecting generic errors in compilers or in complex hardware devices providing assurance beyond reasonable doubt of the dependability of the Fly-By-Wire system.<br />
<br />
The above mentioned system clearly has individual Instruction Streams as the architecture of each processor is different, thus different instruction sets and different instruction streams. These processors have frame synchronized input data which means they have same set of data to work upon which is fed from a single data stream. Thus the flight control system can be classified under MISD architecture.<br />
<br />
===3. Edge Detection of Images on TMS320C80 Multiprocessor System===<br />
<br />
The image pre-processing system of this Multiprocessor [9] , the pre-image processing time needed to be made shorter. To achieve the aim of reducing the pre-processing time, different types of image processing architectures for ‘C80 were presented. For example MIMD architecture is used for image histogram equalization; SIMD architecture for median filtering; MISD architecture for directional edge detection.<br />
<br />
The MISD architecture can be explained as follows. Here, each processor performs different operations on the same data set. It is mentioned at many places that MISD architecture was defined only for the sake of completeness of classification due to difficulty in practice to find a computer based on MISD architecture [10]. But in case of Image Processing, this assumption need not be true as here making many different operations on same image is rare and hence MISD architecture is a best fit for such cases. As an example of implementation of image processing operations for the MISD architecture the edge detection operation will be considered. The edges should be detected in four different directions (north, south, west and east) by the medium of four different filter masks. Each of the parallel processors of the TMS320C80 calculates one of the directional edges. Finally one obtains four images with detected edges, each in different direction.<br />
<br />
The image processing in computer is very often a multistage process composed of many stages. For example, an algorithm detecting contours of the objects in an image. The algorithm for achieving the before mentioned is composed of four basic image processing operation. The image acquired from the camera in the first step undergoes low-pass filtering, during which it is smoothed and noise is also eliminated. In the next step image is thresholding is performed and each pixel gets black or white depending on that if its grey level is below or above threshold value. The resulting image is a binary image from which edges are detected by the usage of the Laplace filter. In the last step the image with detected contours undergoes logic filtering in the purpose of elimination of single black pixels placed on the white background. For implementing the above processes parallel, a five-stage pipeline was used. The first stage of this pipeline constitutes the master processor which accounts for control of the image acquisition process. Further pipe line stages were implemented for 4 parallel processors as mentioned in table 1.<br />
<br />
Stage Processor Operation<br />
1 MP Image acquisition<br />
2 PP0 Low-pass Filtering<br />
3 PP1 Thresholding<br />
4 PP2 Laplace Filtering<br />
5 PP3 Logic Filtering<br />
Table 1: Pipeline structure for image edge detection [9]<br />
<br />
<br />
=='''References'''==<br />
<references/><br />
<br />
=='''Glossary'''==<br />
*'''CMU''': Carnegie Mellon University<br />
<br />
*'''CU''': Control Unit<br />
<br />
*'''DS''': Data Stream<br />
<br />
*'''Fly-By-Wire''': System that replaces the conventional manual flight controls of an aircraft with an electronic interface<br />
<br />
*'''FPGA''': Field Programmable Gate Array<br />
<br />
*'''Heterogeneous Systems''': A multiprocessor system with different kind of processors<br />
<br />
*'''Homogeneous System''': A multiprocessor system with same kind of processors<br />
<br />
*'''ILP''': Instruction Level Parallelism<br />
<br />
*'''IS''': Instruction Stream<br />
<br />
*'''MIMD''': Multiple Instruction Multiple Data<br />
<br />
*'''MISD''': Multiple Instruction Single Data<br />
<br />
*'''PE''': Processing Element<br />
<br />
*'''PFC''': Primary Flight Computer<br />
<br />
*'''SAPO''': Short for Samočinný počítač<br />
<br />
*'''SIMD''': Single Instruction Multiple Data<br />
<br />
*'''SISD''': Single Instruction Single Data<br />
<br />
*'''VLSI''': Very Large Scale Integration</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_506_Spring_2013/1a_sp&diff=71768CSC/ECE 506 Spring 2013/1a sp2013-02-04T22:43:44Z<p>Schakra8: Created page with "==Multiple Instruction Single Data (MISD) Architecture== Multiple Instruction Single Data, or MISD, architecture is one of the four general categories of multi-processor or mult..."</p>
<hr />
<div>==Multiple Instruction Single Data (MISD) Architecture==<br />
<br />
Multiple Instruction Single Data, or MISD, architecture is one of the four general categories of multi-processor or multi-core architectures described by Flynn's Taxonomy.<br />
<br />
==MISD Architectures within the Context of Flynn’s Taxonomy<ref>http://en.wikipedia.org/wiki/Flynn's_taxonomy</ref><ref>http://www.phy.ornl.gov/csep/ca/node11.html</ref>==<br />
<br />
Flynn's Taxonomy categorizes parallel computing architectures based on the number of information streams that flowed into the processor.<br />
According to Flynn's Taxonomy, there are two kinds of logical streams of information that flows into a parallel computer, the Instruction Stream and the Data stream. Note that for the purposes of classification using Flynn’s Taxonomy, it does not matter whether or not the streams are physically independent of each other on the hardware, only that the streams can be logically treated as independent.<br />
Based on how many of each type of stream flows into the parallel computer (single vs. multiple), Flynn's Taxonomy divides parallel architectures into four categories. They are Single Instruction Single Data (SISD), Single Instruction Multiple Data (SIMD), Multiple Instruction Single Data (MISD), and Multiple Instruction Multiple Data (MIMD).<br />
<br />
The four classifications defined by Flynn are based upon the number of concurrent instruction (or control) and data streams available in the architecture are<ref>https://computing.llnl.gov/tutorials/parallel_comp/#Flynn</ref><br />
<br />
[[Image:Flynn's Taxonomy.PNG|thumb|center|400px|Figure 1. [http://en.wikipedia.org/wiki/Michael_J._Flynn Flynn]'s Taxonomy [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]] <br />
<br />
===Overview of the MISD Architecture===<br />
<br />
In order to be categorized as MISD architecture, there must be at least two Instruction streams and there can only be one Data stream as shown on Figure 2. As per the description according to Flynn’s Taxonomy, each processing elements should receive the same data and execute different instructions on the same data simultaneously.<br />
<br />
[[Image:MISD.PNG|thumb|right|100px|Figure 2. MISD Architecture [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-0 1]]]]<br />
<br />
There are very few implementations of the classical MISD architecture as described by Flynn’s Taxonomy, none of which are commercial, general purpose parallel computers. Instead, MISD architectures are mainly used to create custom hardware in order to solve specific problems. The majority of the implementations have been created are in academia.<br />
Over the years, several architectures have been described as MISD even though the architectures do not completely follow the description in Flynn’s Taxonomy. One common example is the Pipeline architecture where the data is passed from processing element to processing element and each processing element executes a different instruction. It is often argued that this is not a true MISD architecture as the data changes as it moves through the pipeline. Similarly, other architectures or applications exist which may be called MISD by some while others disagree. The following section looks at some of those implementations.<br />
<br />
Pipeline architectures belong to this type, though a purist might say that the data is different after processing by each stage in the pipeline. Fault-tolerant computers executing the same instructions redundantly in order to detect and mask errors, in a manner known as task replication, may be considered to belong to this type. Not many instances of this architecture exist, as MIMD and SIMD are often more appropriate for common data parallel techniques. Specifically, they allow better scaling and use of computational resources than MISD does. <br />
<br />
However, one prominent example of MISD in computing is the Space Shuttle flight control computers. Another example of this machine is the systolic array, such as the [http://www.cs.cmu.edu/~iwarp/ CMU iWrap] [BORKAR et al., 1990]. All the elements in this array are controlled by a global clock. On each cycle, an element will read a piece of data from one of its neighbors, perform a simple operation (e.g. add the incoming element to a stored value), and prepare a value to be written to a neighbor on the next step.<br />
<br />
==Implementations of MISD architecture==<br />
<br />
Some major implementations of MISD are in the fields of systolic arrays, pattern matching, flight control systems and fault tolerance systems. These MISD implementations are discussed below.<br />
<br />
===1. Systolic Array===<br />
<br />
A systolic array is an arrangement of processors in an array where data flows synchronously across the array between neighbors, usually with different data flowing in different directions. Each Processor at each step takes in data from one or more neighbors, processes it and, in the next step, outputs results in the opposite direction.<br />
<br />
The systolic array paradigm, data-stream-driven by data counters, is the counterpart of the [http://en.wikipedia.org/wiki/Von_Neumann_model von Neumann paradigm], instruction-stream-driven by a program counter. Because a systolic array usually sends and receives multiple data streams, and multiple data counters are needed to generate these data streams, it supports data parallelism. The name derives from analogy with the regular pumping of blood by the heart.<ref>http://en.wikipedia.org/wiki/Systolic_array</ref><br />
<br />
====1.1 Type of Systolic Arrays<ref>http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf General Purpose Systolic Arrays </ref>====<br />
<br />
=====1.1.1 Special-purpose systolic array=====<br />
[[Image:systolic_1.png|thumb|right|250px|Figure 3: The algorithm for the sum of a scalar product, computed in systolic element [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
[[Image:systolic_2.png|thumb|right|250px|Figure 7: The systolic product of two 3x3 matrices [[http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]]<br />
<br />
An array of hardwired systolic processing elements tailored for a specific application. Typically, many tens or hundreds of cells fit on a single chip. One of the major applications of special-purpose systolic array is in matrix operations. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Systolic_1.png Figure 3] illustrates the algorithm for the sum of a scalar product, computed in a single systolic element. Here, a’s and b’s are synchronously shifted through the processing element to be available for next element. These data synchronously exits the processing element unmodified for the next element. The sum of the products is then shifted out of the accumulator.<br />
<br />
=====1.1.2 General-purpose systolic array=====<br />
<br />
It is an array of systolic processing elements, which gets adapted to a variety of applications via programming or reconfiguration. Array topologies can be either programmable or reconfigurable. Likewise, array cells are either programmable or reconfigurable. This is referred to as Systolic topologies.<br />
<br />
=====1.1.2.1 Programmable Systolic Array=====<br />
<br />
It is an array of programmable systolic elements that operates either in SIMD or MIMD fashion. Either the arrays interconnect or each processing unit is programmable and a program controls dataflow through the elements. Programmable systolic arrays are programmable either at a high level or a low level. At either level, programmable arrays can be categorized as either SIMD or MIMD machines.<br />
<br />
[[Image:systolic_4.png|thumb|right|250px|Figure 9: General organization of MIMD programmable linear systolic arrays [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
<br />
For the latter approach, the workstation downloads a program to each MIMD (Figure 7) systolic cell. Each cell may be loaded with a different program, or all the cells in the array may be loaded with the same program. Each cell's architecture is somewhat similar to the conventional von Neumann architecture: It contains a control unit, an ALU, and local memory. MIMD systolic cells have more local memory than their SIMD counterparts to support the von Neumann-style organization.<br />
<br />
This architecture is defined as Multiple Instruction Multiple Data (MIMD) architecture in [http://home.engineering.iastate.edu/~zambreno/classes/cpre583/documents/JohHur93A.pdf 5]. The architecture has multiple instruction streams for the PEs and a single data stream passing through all the PEs. Thus, it can also be defined as Multiple Instruction Single Data (MISD) architecture. The architecture of Systolic array configuration are controversial as explained in the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Architecture_of_systolic_arrays_as_against_MISD_architecture section 4.1.2.]<br />
<br />
=====1.1.2.1 Reconfigurable Systolic Array=====<br />
[[Image:reconfig.jpg|thumb|right|250px|Figure 10: Block Diagram of the RSA Architecture [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-4 5]]]<br />
It is an array of systolic elements that can be programmed at the lowest level. Recent gate density advances in FPGA technology have produced a low-level, reconfigurable systolic array architecture that bridges the gap between special-purpose arrays and the more versatile, programmable general-purpose arrays. The FPGA architecture is unusual because a single hardware platform can be logically reconfigured as an exact duplicate of a special-purpose systolic array. <br />
<br />
The RSA circuit design is based on systolic array architecture consisting of PEs interconnected via SWs as depicted in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Reconfig.jpg Figure 10]. The homogeneous characteristic of the Reconfigurable Systolic Array (RSA) architecture, where each reconfigurable processing element (PE) cell is connected to its nearest neighbors via configurable switch (SW) elements, enables array expansion for parallel processing and facilitates time sharing computation of high-throughput data by individual PEs. Both the PEs and SWs can be reconfigured dynamically with the former as an arithmetic processor and the latter as a flexible router linking the neighboring PE cells. The RSA shifts reconfiguration and input signals into the PEs and SWs on separate data bus which enables the circuit to continue its operation while the reconfiguration is in process.<br />
<br />
====1.2 Architecture of systolic arrays as against MISD architecture====<br />
[[Image:comp.png|thumb|right|250px|Figure 11.Comparison between Architecture of systolic arrays and MISD]]<br />
<br />
As from the above mentioned configurations of the Systolic Arrays, it is seen that generally the configurations have multiple processing elements executing different instructions from dedicated instruction streams for each processing element. There is a single data stream that connects the adjacent PEs. Thus, systolic array can be defined as an MISD architecture.<br />
<br />
Many authors say that as the data read as input by one processing element is processed data output of the adjacent PE. The data stream cannot be considered as single because all the data paths do not carry the same data to all the PEs. [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Comp.png Figure 11] shows the difference between the Data Stream for Systolic Arrays and the MISD architecture. Thus the systolic array should be considered as “Multiple Data” architecture and not Single Data architecture.<br />
<br />
===2. Fault Tolerant Systems<ref>http://en.wikipedia.org/wiki/Fault-tolerant_computer_system#Types_of_fault_tolerance</ref>===<br />
<br />
The fault tolerant systems are designed to handle the possible failures in software, hardware or interfaces. The hardware faults include hard disk failures, input or output device failures, etc. and the software and interface faults include driver failures; operator errors, installing unexpected software etc. The hardware faults can be detected and identified by implementing redundant hardware and multiple backups. The software faults can be tolerable by removing the program errors by executing the software redundantly or by implementing small programs that take over the tasks that crash or generate errors.<br />
<br />
[[Image:fault.png|thumb|right|250px|Figure 12 MISD as fault tolerant architecture]]<br />
<br />
In computer systems, the [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Single_Instruction.2C_Multiple_Data_streams_.28SIMD.29 SIMD], [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD] and [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instruction.2C_Multiple_Data_streams_.28MIMD.29 MIMD] architectures facilitate the implementation of the fault tolerance systems by multiple instruction streams or multiple data streams or both. Fault tolerance on computations can be implemented by multiple processors (likely with different architectures) executing the algorithms on the same set of data. The output of each processor is compared with that of the others and M out of N majority voting method is used to determine the faulty processor. Thus MISD architecture is utilized to get the fault tolerance on critical computations.<br />
<br />
There are various examples of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture] being used as fault tolerant architecture. The major examples being flight control systems, nuclear power plants, satellite systems, super collider experiment systems, etc. Here, the flight control system is explained as an example of [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#Multiple_Instructions.2C_Single_Data_stream_.28MISD.29 MISD architecture].<br />
<br />
====2.1 The Flight Control System – MISD Example for fault tolerance====<br />
<br />
A [http://en.wikipedia.org/wiki/Fly-by-wire Fly-By-Wire] system is used to replace the manual flight control by an electronic control interface. The movements of the flight control in the cockpit are converted to electronic signals and are transmitted to the actuators by wires. The control computers use the feedback from the sensors to compute and control the movement of the actuators to provide the expected response. These computers also perform the task to stabilize the aircraft and perform other tasks without the knowledge of the pilot. Flight control systems must meet extremely high levels of accuracy and functional integrity.<br />
<br />
There are redundant flight control computers present in the flight control system. If one of the flight-control computers crashes, gets damaged or is affected by electromagnetic pulses, the other computer can overrule the faulty one and hence the flight of the aircraft is unharmed. [[Image:fig13.png|thumb|right|250px|Figure 13: Architecture of triple redundant 777 primary flight computer [http://expertiza.csc.ncsu.edu/wiki/index.php/CSC/ECE_506_Spring_2012/1c_dm#cite_note-7 6]]]The number of redundant flight control computers is generally more than two, so that any computer whose results disagree with the others is ruled out to be faulty and is either ignored or rebooted.[[Image:fig14.png|thumb|right|250px|Figure 14: PFC with instruction and data streams]]<br />
<br />
=====Multiple Processors Implementation in Boeing 777<ref>http://www.citemaster.net/getdoc/8767/R8.pdf Y.C. (Bob) Yeh, Boeing Commercial Airplane Group, "Triple-Triple Redundant 777 Primary Flight Computer" </ref>=====<br />
<br />
In modern computers, the redundant flight control computations are carried out by multiprocessor systems. The triple redundant 777 primary flight computer, has the architecture as shown in [http://expertiza.csc.ncsu.edu/wiki/index.php/File:Fig13.png Figure 13].<br />
<br />
The system has three primary flight control computers, each of them having three lanes with different processors. The flight control program is compiled for each of the processors which get the input data from the same data bus but drive the output on their individual control bus. Thus each processor executes different instructions but they process the same data. Thus, it is the best suited example of Multiple Instruction Single Data (MISD) architecture.<br />
<br />
The three processors selected for the flight control system of [http://en.wikipedia.org/wiki/Boeing_777 Boeing 777] were [http://en.wikipedia.org/wiki/Intel_80486 Intel 80486], [http://en.wikipedia.org/wiki/Motorola_68040 Motorola 68040] and [http://en.wikipedia.org/wiki/AMD_Am29000 AMD 29050]. The dissimilar processors lead to dissimilar interface hardware circuits and compilers. Each lane of the flight control computer is data synchronized with the other lanes so that all of the lanes read the same frame of data from the flight sensors. As the outputs of each lane can be different, the median value of the outputs is used to select the output of the lane to be considered. The lane which has the median value select hardware selected is said to be in “command mode” whereas the other lanes are said to be in “monitoring mode”. It receives the data from the other Primary Flight Computer (PFC) lanes and performs a median select of the outputs. This provides a fault blocking mechanism before the fault detection and identification by the cross-lane monitoring system. Thus, the MISD based multi computer architecture is capable of detecting generic errors in compilers or in complex hardware devices providing assurance beyond reasonable doubt of the dependability of the Fly-By-Wire system.<br />
<br />
The above mentioned system clearly has individual Instruction Streams as the architecture of each processor is different, thus different instruction sets and different instruction streams. These processors have frame synchronized input data which means they have same set of data to work upon which is fed from a single data stream. Thus the flight control system can be classified under MISD architecture.<br />
<br />
===3. Edge Detection of Images on TMS320C80 Multiprocessor System===<br />
<br />
The image pre-processing system of this Multiprocessor [9] , the pre-image processing time needed to be made shorter. To achieve the aim of reducing the pre-processing time, different types of image processing architectures for ‘C80 were presented. For example MIMD architecture is used for image histogram equalization; SIMD architecture for median filtering; MISD architecture for directional edge detection.<br />
<br />
The MISD architecture can be explained as follows. Here, each processor performs different operations on the same data set. It is mentioned at many places that MISD architecture was defined only for the sake of completeness of classification due to difficulty in practice to find a computer based on MISD architecture [10]. But in case of Image Processing, this assumption need not be true as here making many different operations on same image is rare and hence MISD architecture is a best fit for such cases. As an example of implementation of image processing operations for the MISD architecture the edge detection operation will be considered. The edges should be detected in four different directions (north, south, west and east) by the medium of four different filter masks. Each of the parallel processors of the TMS320C80 calculates one of the directional edges. Finally one obtains four images with detected edges, each in different direction.<br />
<br />
The image processing in computer is very often a multistage process composed of many stages. For example, an algorithm detecting contours of the objects in an image. The algorithm for achieving the before mentioned is composed of four basic image processing operation. The image acquired from the camera in the first step undergoes low-pass filtering, during which it is smoothed and noise is also eliminated. In the next step image is thresholding is performed and each pixel gets black or white depending on that if its grey level is below or above threshold value. The resulting image is a binary image from which edges are detected by the usage of the Laplace filter. In the last step the image with detected contours undergoes logic filtering in the purpose of elimination of single black pixels placed on the white background. For implementing the above processes parallel, a five-stage pipeline was used. The first stage of this pipeline constitutes the master processor which accounts for control of the image acquisition process. Further pipe line stages were implemented for 4 parallel processors as mentioned in table 1.<br />
<br />
Stage Processor Operation<br />
1 MP Image acquisition<br />
2 PP0 Low-pass Filtering<br />
3 PP1 Thresholding<br />
4 PP2 Laplace Filtering<br />
5 PP3 Logic Filtering<br />
Table 1: Pipeline structure for image edge detection [9]<br />
<br />
<br />
=='''References'''==<br />
<references/><br />
<br />
=='''Glossary'''==<br />
*'''CMU''': Carnegie Mellon University<br />
<br />
*'''CU''': Control Unit<br />
<br />
*'''DS''': Data Stream<br />
<br />
*'''Fly-By-Wire''': System that replaces the conventional manual flight controls of an aircraft with an electronic interface<br />
<br />
*'''FPGA''': Field Programmable Gate Array<br />
<br />
*'''Heterogeneous Systems''': A multiprocessor system with different kind of processors<br />
<br />
*'''Homogeneous System''': A multiprocessor system with same kind of processors<br />
<br />
*'''ILP''': Instruction Level Parallelism<br />
<br />
*'''IS''': Instruction Stream<br />
<br />
*'''MIMD''': Multiple Instruction Multiple Data<br />
<br />
*'''MISD''': Multiple Instruction Single Data<br />
<br />
*'''PE''': Processing Element<br />
<br />
*'''PFC''': Primary Flight Computer<br />
<br />
*'''SAPO''': Short for Samočinný počítač<br />
<br />
*'''SIMD''': Single Instruction Multiple Data<br />
<br />
*'''SISD''': Single Instruction Single Data<br />
<br />
*'''VLSI''': Very Large Scale Integration</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=ECE506_Main_Page&diff=71734ECE506 Main Page2013-02-04T20:14:41Z<p>Schakra8: /* Supplements to Solihin Text */</p>
<hr />
<div>This page serves as a portal for all wiki material related to CSC506 and ECE506. Link to any new wiki pages from this page, and add links to any current pages.<br />
<br />
=Supplements to Solihin Text=<br />
<br />
Post links to the textbook supplements in this section.<br />
*Chapter 2 [[CSC/ECE 506 Spring 2011/ch2 dm | CSC/ECE 506 Spring 2011/ch2 dm]]<br />
*Chapter 2 [[Parallel_Programming_Models | Parallel Programming Models]]<br />
*Chapter 2 (Still being revised) [[CSC/ECE 506 Spring 2011/ch2 cl | CSC/ECE 506 Spring 2011/ch2 cl]]<br />
*Chapter 2a [[ CSC/ECE 506 Spring 2011/ch2a mc | Current Data-Parallel Architectures ]]<br />
*Chapter 2a [[ CSC/ECE 506 Spring_2012/2a va ]]<br />
*Chapter 2b [[CSC/ECE 506 Spring 2012/ch2b cm | CSC/ECE 506 Spring 2012/ch2b cm]]<br />
*Chapter 2b [[ECE506_CSC/ECE_506_Spring_2012/2b_az | CSC/ECE 506 Spring 2012/2b az - Data-Parallel Processing with the AMD HD 6900 Series Graphics Processing Unit]]<br />
*Chapter 3 (Final Revision) [[ CSC/ECE 506 Spring 2011/ch3 ab | Parallel Architecture Mechanisms and Programming Models ]]<br />
*Chapter 4a[[ CSC/ECE 506 Spring 2011/ch4a ob | Parallelization of Nelder Mead Algorithm ]]<br />
*Chapter 4a (Under Construction) [[ CSC/ECE_506_Spring_2011/ch4a_bm | Parallelization of Algorithms ]]<br />
*Chapter 4a [[ CSC/ECE 506 Spring 2011/ch4a zz | CSC/ECE 506 Spring 2011/ch4a zz ]]<br />
*Chapter 4b [[Chapter 4b CSC/ECE 506 Spring 2011 / ch4b]]<br />
*Chapter 5a [[ CSC/ECE 506 Spring 2012/ch5a ja | CSC/ECE 506 Spring 2012/ch5a ja ]]<br />
*Chapter 9a [[CSC/ECE 506 Spring 2012/ch9a cm | CSC/ECE 506 Spring 2012/ch9a cm]]<br />
*Chapter 6a (Under Construction) [[ CSC/ECE 506 Spring 2011/ch6a jp | CSC/ECE 506 Spring 2011/ch6a jp ]]<br />
*Chapter 6a (Under Construction) [[ CSC/ECE 506 Spring 2011/ch6a ep | CSC/ECE 506 Spring 2011/ch6a ep ]]<br />
*Chapter 6b (Ready for First Review) [[CSC/ECE 506 Spring 2011/ch6b ab | CSC/ECE 506 Spring 2011/ch6b ab]]<br />
*Chapter 7 (Under Construction) [[CSC/ECE 506 Spring 2011/ch7 jp | CSC/ECE 506 Spring 2011/ch7 jp]]<br />
*Chapter 8 [[CSC/ECE 506 Spring 2011/ch8 mc | CSC/ECE 506 Spring 2011/ch8 mc]]<br />
*Chapter 10 (Under Construction) [[CSC/ECE 506 Spring 2011/ch10 sb | CSC/ECE 506 Spring 2011/ch10 sb]]<br />
*Chapter 10 [[CSC/ECE 506 Spring 2012/ch10 sj | CSC/ECE 506 Spring 2012/ch10 sj]]<br />
*Chapter 10a [[CSC/ECE_506_Spring_2011/ch10a_dc | CSC/ECE_506_Spring_2011/ch10a_dc]]<br />
*Chapter 11 [[CSC/ECE_506_Spring_2011/ch11_BB_EP | Chapter 11 Supplement]]<br />
*Chapter 11 [[Scalable_Coherent_Interface | SCI (Scalable Coherent Interface) ]]<br />
*Chapter 12 [[ CSC/ECE 506 Spring 2011/ch12 ob | Interconnection Network Topologies and Routing Algorithms]]<br />
*Chapter 12 (Ready for Final Review) [[ CSC/ECE 506 Spring 2011/ch12 aj | Interconnection Network Topologies and Routing Algorithms]]<br />
*Chapter 12 [[ CSC/ECE 506 Spring 2011/ch12 | Interconnection Network Topologies]]<br />
*[[CSC/ECE 506 Spring 2012/1a ry]]<br />
*[[CSC/ECE 506 Spring 2012/1c dm]]<br />
*[[CSC/ECE 506 Spring 2012/1c cl]]<br />
*[[CSC/ECE 506 Spring 2012/1a mw]]<br />
*[[CSC/ECE 506 Spring 2012/3a yw]]<br />
*[[CSC/ECE 506 Spring 2012/7b yw]]<br />
*[[CSC/ECE 506 Spring 2012/3b sk]]<br />
*[[CSC/ECE 506 Spring 2012/4b rs]]<br />
*[[CSC/ECE 506 Spring 2012/6b am]]<br />
*[[CSC/ECE 506 Spring 2012/8a cj]]<br />
*[[CSC/ECE 506 Spring 2012/10a dr]]<br />
*[[CSC/ECE 506 Spring 2012/10a jp]]<br />
*[[CSC/ECE 506 Spring 2012/9a ms]]<br />
*[[CSC/ECE 506 Spring 2012/10b sr]]<br />
*[[CSC/ECE 506 Spring 2012/12b jh]]<br />
*[[CSC/ECE 506 Spring 2010/8a fu]]<br />
*[[CSC/ECE 506 Spring 2010/8a sk]]<br />
*[[CSC/ECE 506 Spring 2012/11a ht]]<br />
*[[CSC/ECE 506 Spring 2013/1b dj]]<br />
*[[CSC/ECE 506 Spring 2013/1a sp]]<br />
*Chapter 11a [[ECE506_CSC/ECE_506_Spring_2012/11a_az | CSC/ECE 506 Spring 2012/11a az - Performance of DSM system]]</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_517_Fall_2012/ch2a_2w23_sr&diff=69003CSC/ECE 517 Fall 2012/ch2a 2w23 sr2012-10-27T03:59:07Z<p>Schakra8: /* Advantages */</p>
<hr />
<div>''Responsibility-Driven Design''<br />
== Introduction ==<br />
[http://en.wikipedia.org/wiki/Object-oriented_programming Object-oriented programming(OOP) language] enables software to be reusable, refinable, testable, maintainable, and extensible by supporting [http://en.wikipedia.org/wiki/Encapsulation_(object-oriented_programming) encapsulation]. In order to realize these advantages, encapsulation should be maximized during the design process. Other techniques attempt to enforce encapsulation during the implementation phase. This is too late in the software life-cycle to achieve maximum benefits.<br />
<br />
Responsibility-driven design(RRD) is an alternate design method to focus on increasing the encapsulation by viewing a program in terms of the client/server model.<ref>http://dl.acm.org/citation.cfm?doid=74877.74885 Object-oriented design: a responsibility-driven approach</ref> This method was proposed by [http://en.wikipedia.org/wiki/Rebecca_Wirfs-Brock Rebecca Wirfs-Brock] and Brian Wilkerson.<br />
<br />
== Definition ==<br />
Responsibility-Driven Design is defined as follows:<br />
<br />
It is inspired by the [http://en.wikipedia.org/wiki/Client%E2%80%93server_model client/server model]. It focuses on the contract by asking:<br />
<br />
* What actions is this object responsible for?<br />
* What information does this object share?<ref>http://dl.acm.org/citation.cfm?doid=74877.74885 Object-oriented design: a responsibility-driven approach</ref><br />
<br />
== Responsibility-Driven Design principles and analysis==<br />
<br />
'''''Principles'''''<br />
<br />
The basic principles of Responsibility-Driven Design are:<br />
<br />
*<b>Maximize Abstraction</b><br />
<br />
Initially hide the distinction between data and behavior.<br />
<br />
Think of objects responsibilities for “knowing”, “doing”,and “deciding”.<br />
<br />
*<b>Distribute Behavior</b><br />
<br />
Promote a delegated control architecture.<br />
<br />
Make objects smart i.e. have them behave intelligently, not just hold bundles of data.<br />
<br />
*<b>Preserve Flexibility</b><br />
<br />
Design objects such that interior details can be readily changed.<br />
<br />
'''''Analysis'''''<br />
<br />
The analysis stage of Responsible-Driven Design consists of three phases:<br />
*<b>System Definition:</b> High-level view of system<br />
*<b>Detailed Description:</b> Detailed view of development process, functional requirements, and non-functional requirements.<br />
*<b>Object Analysis:</b> Construction of domain objects。<ref>http://www.wirfs-brock.com/ RRD principles and constructs</ref><br />
<br />
<br />
== Example ==<br />
<br />
The basic example of Responsibility-driven design is Point Of Sale Terminal (POST). It is basically driven by use cases and requirements which include domain models and system operations derived from usecases. It then assigns responsibilities to objects/classes for use cases realization. Suppose the given use case for POST is represented in Figure 1. Then the RDD methodical context for POST is given by Figure 2. <br />
<br />
<br />
[[File:RDD2.jpg]]<br />
<br />
Figure 1. [http://www.site.uottawa.ca/~ssome/Cours/SEG3202/rdd.pdf Usecase diagram of POST]<br />
<br />
[[File:rdd1.jpg]]<br />
<br />
Figure 2. [http://www.site.uottawa.ca/~ssome/Cours/SEG3202/rdd.pdf RDD Methodological context of POST Usecase]<br />
<br />
Here, the figure 2. shows the considerations made while designing a model which is responsibility driven for POST. It includes business modelling,requirements and design of functionality.<br />
<br />
== Advantages ==<br />
*'''''Increases encapsulation'''''<br />
The responsibility-driven approach emphasizes the encapsulation of both the structure and behavior of objects by focusing on the contractual responsibilities of a class.It improves encapsulation with respect to subclass clients, ensuring that the inherited behavior is part of the contract of the subclass. The benefit is increased encapsulation, since the specification of the exact way in which a request is carried out is private to the server. Encapsulation is compromised when the structural details of an object become part of the interface. Responsibility-driven designs maximize encapsulation by ignoring the structural details.<br />
*'''''Facilitates polymorphism'''''<br />
By encouraging the designer to focus on responsibilities independently of implementation, the responsibility driven approach can help a designer identify standard protocols (message names), which facilitates polymorphism. It makes abstract classes easier to identify. One difficulty in identifying abstract classes lies in determining what parts of the protocol of existing classes are parts of the type of those classes, and what parts are implementation details. Because the protocol of a class includes only those messages which form the type of the class, this problem is eliminated.<br />
<br />
== Comparison of Responsibility-Driven Design with Data-Driven Design ==<br />
<br />
Adapting abstract data type design methods to object-oriented programming results in a data-driven design. On the other hand, Responsibility-driven design is in direct contrast with Data-driven design, which promotes defining the behavior of a class along the data that it holds. <br />
<br />
'''''Why responsibility-driven design over data-driven design?'''''<br />
<br />
While data-driven design does prevent coupling of data and functionality, in some cases, data-driven programming has been argued to lead to bad object-oriented design, especially when dealing with more abstract data. This is because a purely data-driven object or entity is defined by the way it is represented. Any attempt to change the structure of the object would immediately break the functions that rely on it. As a, perhaps trivial, example, one might represent driving directions as a series of intersections (two intersecting streets) where the driver must turn right or left. If an intersection (in the United States) is represented in data by the zip code (5-digit number) and two street names (strings of text), you might run into bugs when you discover a city where streets intersect multiple times. While this example may be over simplified, restructuring of data is fairly common problem in software engineering, either to eliminate bugs, increase efficiency, or support new features. In these cases responsibility-driven design may be promoted as a better approach, where functionality and data can be coupled together, so functions don't have to deal with the data representation itself.<br />
<br />
'''''Example of Responsibility-Driven v/s Data-Driven Design'''''<br />
<br />
Let us consider a class of raster image objects which stores visual images as two-dimensional array of bits. The operations of raster image include basic image manipulations like rotation, orientation, scaling etc. Now, we can define a RasterImage class which will include methods for each possible operation it can perform:<br />
class Rasterlmage is <br />
bitArray : ByteArray; <br />
width : Integer; <br />
height : Integer;<br />
rotateBy(degree : Integer) : void; <br />
scaleBy(x : Integer, y : Integer) : void; <br />
bitAt(index : Point) : Integer; <br />
bitAtPut(index : Point, bitValue : Integer) : void;<br />
Rasterlmage structure: <br />
bitArray : ByteArray; <br />
bitArray(bits : ByteArray) : void; <br />
width0 : Integer; <br />
width(aNumber : Integer) : void; <br />
height0 : Integer; <br />
height(aNumber : Integer) : void; <br />
end Rasterlmage;<br />
<br />
The same requirement can be modified through responsibility driven desgn. The responsibilities of RasterImage class are: knowing and maintaining the smallest rectangle that completely contains its image, knowing and maintaining all individual bit values within its image and rotating and scaling the image. Therefore, the RasterImage class based on responsibilities is as follows:<br />
<br />
class Rasterlmage is <br />
boundingRectangle() : Rectangle; <br />
boundingRectangIe(bounds : Rectangle) : void; <br />
bitAt(index : Point) : Integer; <br />
bitAtPut(index : Point, bitvalue : Integer) : void; <br />
scaleBy(x : Integer, y : Integer) : void; <br />
rotateBy(degrees : Integer) : void; <br />
end Rasterlmage;<br />
<br />
== Conclusion ==<br />
Responsibility-Driven Design is a way to design software that emphasizes modeling of objects’ roles, responsibilities, and collaborations. Its principle is to maximize abstraction, delegate control and preserve flexibility. It increases encapsulation by focusing on the contractual responsibilities of a class and facilitates polymorphism by encouraging the designer to focus on responsibilities independently of implementation.<br />
<br />
== References ==<br />
<br />
<references/><br />
<br />
== External Links ==<br />
1. [http://www.ece.uprm.edu/~borges/CRC.pdf Responsibility Driven Design]<br />
<br />
2. [http://nccastaff.bournemouth.ac.uk/jmacey/CA1/Papers/Responsibility-Driven%20Design.pdf Object-Oriented Design: A Responsibility-Driven Approach ]</div>Schakra8https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_517_Fall_2012/ch2a_2w23_sr&diff=69002CSC/ECE 517 Fall 2012/ch2a 2w23 sr2012-10-27T03:58:42Z<p>Schakra8: /* Advantages */</p>
<hr />
<div>''Responsibility-Driven Design''<br />
== Introduction ==<br />
[http://en.wikipedia.org/wiki/Object-oriented_programming Object-oriented programming(OOP) language] enables software to be reusable, refinable, testable, maintainable, and extensible by supporting [http://en.wikipedia.org/wiki/Encapsulation_(object-oriented_programming) encapsulation]. In order to realize these advantages, encapsulation should be maximized during the design process. Other techniques attempt to enforce encapsulation during the implementation phase. This is too late in the software life-cycle to achieve maximum benefits.<br />
<br />
Responsibility-driven design(RRD) is an alternate design method to focus on increasing the encapsulation by viewing a program in terms of the client/server model.<ref>http://dl.acm.org/citation.cfm?doid=74877.74885 Object-oriented design: a responsibility-driven approach</ref> This method was proposed by [http://en.wikipedia.org/wiki/Rebecca_Wirfs-Brock Rebecca Wirfs-Brock] and Brian Wilkerson.<br />
<br />
== Definition ==<br />
Responsibility-Driven Design is defined as follows:<br />
<br />
It is inspired by the [http://en.wikipedia.org/wiki/Client%E2%80%93server_model client/server model]. It focuses on the contract by asking:<br />
<br />
* What actions is this object responsible for?<br />
* What information does this object share?<ref>http://dl.acm.org/citation.cfm?doid=74877.74885 Object-oriented design: a responsibility-driven approach</ref><br />
<br />
== Responsibility-Driven Design principles and analysis==<br />
<br />
'''''Principles'''''<br />
<br />
The basic principles of Responsibility-Driven Design are:<br />
<br />
*<b>Maximize Abstraction</b><br />
<br />
Initially hide the distinction between data and behavior.<br />
<br />
Think of objects responsibilities for “knowing”, “doing”,and “deciding”.<br />
<br />
*<b>Distribute Behavior</b><br />
<br />
Promote a delegated control architecture.<br />
<br />
Make objects smart i.e. have them behave intelligently, not just hold bundles of data.<br />
<br />
*<b>Preserve Flexibility</b><br />
<br />
Design objects such that interior details can be readily changed.<br />
<br />
'''''Analysis'''''<br />
<br />
The analysis stage of Responsible-Driven Design consists of three phases:<br />
*<b>System Definition:</b> High-level view of system<br />
*<b>Detailed Description:</b> Detailed view of development process, functional requirements, and non-functional requirements.<br />
*<b>Object Analysis:</b> Construction of domain objects。<ref>http://www.wirfs-brock.com/ RRD principles and constructs</ref><br />
<br />
<br />
== Example ==<br />
<br />
The basic example of Responsibility-driven design is Point Of Sale Terminal (POST). It is basically driven by use cases and requirements which include domain models and system operations derived from usecases. It then assigns responsibilities to objects/classes for use cases realization. Suppose the given use case for POST is represented in Figure 1. Then the RDD methodical context for POST is given by Figure 2. <br />
<br />
<br />
[[File:RDD2.jpg]]<br />
<br />
Figure 1. [http://www.site.uottawa.ca/~ssome/Cours/SEG3202/rdd.pdf Usecase diagram of POST]<br />
<br />
[[File:rdd1.jpg]]<br />
<br />
Figure 2. [http://www.site.uottawa.ca/~ssome/Cours/SEG3202/rdd.pdf RDD Methodological context of POST Usecase]<br />
<br />
Here, the figure 2. shows the considerations made while designing a model which is responsibility driven for POST. It includes business modelling,requirements and design of functionality.<br />
<br />
== Advantages ==<br />
*'''''Increased encapsulation'''''<br />
The responsibility-driven approach emphasizes the encapsulation of both the structure and behavior of objects by focusing on the contractual responsibilities of a class.It improves encapsulation with respect to subclass clients, ensuring that the inherited behavior is part of the contract of the subclass. The benefit is increased encapsulation, since the specification of the exact way in which a request is carried out is private to the server. Encapsulation is compromised when the structural details of an object become part of the interface. Responsibility-driven designs maximize encapsulation by ignoring the structural details.<br />
*'''''Facilitates polymorphism'''''<br />
By encouraging the designer to focus on responsibilities independently of implementation, the responsibility driven approach can help a designer identify standard protocols (message names), which facilitates polymorphism. It makes abstract classes easier to identify. One difficulty in identifying abstract classes lies in determining what parts of the protocol of existing classes are parts of the type of those classes, and what parts are implementation details. Because the protocol of a class includes only those messages which form the type of the class, this problem is eliminated.<br />
<br />
== Comparison of Responsibility-Driven Design with Data-Driven Design ==<br />
<br />
Adapting abstract data type design methods to object-oriented programming results in a data-driven design. On the other hand, Responsibility-driven design is in direct contrast with Data-driven design, which promotes defining the behavior of a class along the data that it holds. <br />
<br />
'''''Why responsibility-driven design over data-driven design?'''''<br />
<br />
While data-driven design does prevent coupling of data and functionality, in some cases, data-driven programming has been argued to lead to bad object-oriented design, especially when dealing with more abstract data. This is because a purely data-driven object or entity is defined by the way it is represented. Any attempt to change the structure of the object would immediately break the functions that rely on it. As a, perhaps trivial, example, one might represent driving directions as a series of intersections (two intersecting streets) where the driver must turn right or left. If an intersection (in the United States) is represented in data by the zip code (5-digit number) and two street names (strings of text), you might run into bugs when you discover a city where streets intersect multiple times. While this example may be over simplified, restructuring of data is fairly common problem in software engineering, either to eliminate bugs, increase efficiency, or support new features. In these cases responsibility-driven design may be promoted as a better approach, where functionality and data can be coupled together, so functions don't have to deal with the data representation itself.<br />
<br />
'''''Example of Responsibility-Driven v/s Data-Driven Design'''''<br />
<br />
Let us consider a class of raster image objects which stores visual images as two-dimensional array of bits. The operations of raster image include basic image manipulations like rotation, orientation, scaling etc. Now, we can define a RasterImage class which will include methods for each possible operation it can perform:<br />
class Rasterlmage is <br />
bitArray : ByteArray; <br />
width : Integer; <br />
height : Integer;<br />
rotateBy(degree : Integer) : void; <br />
scaleBy(x : Integer, y : Integer) : void; <br />
bitAt(index : Point) : Integer; <br />
bitAtPut(index : Point, bitValue : Integer) : void;<br />
Rasterlmage structure: <br />
bitArray : ByteArray; <br />
bitArray(bits : ByteArray) : void; <br />
width0 : Integer; <br />
width(aNumber : Integer) : void; <br />
height0 : Integer; <br />
height(aNumber : Integer) : void; <br />
end Rasterlmage;<br />
<br />
The same requirement can be modified through responsibility driven desgn. The responsibilities of RasterImage class are: knowing and maintaining the smallest rectangle that completely contains its image, knowing and maintaining all individual bit values within its image and rotating and scaling the image. Therefore, the RasterImage class based on responsibilities is as follows:<br />
<br />
class Rasterlmage is <br />
boundingRectangle() : Rectangle; <br />
boundingRectangIe(bounds : Rectangle) : void; <br />
bitAt(index : Point) : Integer; <br />
bitAtPut(index : Point, bitvalue : Integer) : void; <br />
scaleBy(x : Integer, y : Integer) : void; <br />
rotateBy(degrees : Integer) : void; <br />
end Rasterlmage;<br />
<br />
== Conclusion ==<br />
Responsibility-Driven Design is a way to design software that emphasizes modeling of objects’ roles, responsibilities, and collaborations. Its principle is to maximize abstraction, delegate control and preserve flexibility. It increases encapsulation by focusing on the contractual responsibilities of a class and facilitates polymorphism by encouraging the designer to focus on responsibilities independently of implementation.<br />
<br />
== References ==<br />
<br />
<references/><br />
<br />
== External Links ==<br />
1. [http://www.ece.uprm.edu/~borges/CRC.pdf Responsibility Driven Design]<br />
<br />
2. [http://nccastaff.bournemouth.ac.uk/jmacey/CA1/Papers/Responsibility-Driven%20Design.pdf Object-Oriented Design: A Responsibility-Driven Approach ]</div>Schakra8