CSC/ECE 506 Spring 2010/ch 2 aj/Data Parallel Programming

Data-Parallel Programming Model

Introduction

Relationship with SIMD

Examples

// a simple code
sum = 0;
for (i = 0; i < a.length; i++)
{
   a[i] = a[i] * i;
   sum = sum + a[i];
}

// data parallel programming: let each PE perform the same task on different pieces of distributed data
pe_id = getid();
my_sum = 0;
for (i = pe_id; i < a.length; i += number_of_pe)
{
   a[i] = a[i] * i;
   my_sum = my_sum + a[i];
}

// data reorganization via message passing
if (pe_id != 0) send_msg (0, my_sum);
else
   for (i = 1; i < number_of_pe; i++)
   {
      recv_msg (i, temp);
      my_sum = my_sum + temp;
   }
sum = my_sum;

// data reorganization via shared memory
shared sum;
lock();
sum = sum + my_sum;
unlock();
barrier;

References

1) Old textbook

2) Internet article