CSC/ECE 506 Spring 2012/1a hv - Revision history

Vmarapa: /* Interconnects */

2012-02-07T04:14:00Z

Interconnects

← Older revision		Revision as of 04:14, 7 February 2012
Line 151:		Line 151:

	Supercomputers are known to have a large number of processors and they need to work together. [http://en.wikipedia.org/wiki/Interconnection Interconnects] help connects different nodes in a super computer and they have the capacity to carry high bandwidth at the same time maintaining a very low latency. A node in a supercomputer usually denotes a communication end point, that is it maybe visualized as compute nodes, I/O nodes, service nodes and network nodes. In supercomputers where message passing plays a key role, the interconnects are kept as shot as possible just like the way electronic circuits were setup in a ‘C’ shape in the Cray systems to ensure that there is little latency. Since the interconnect plays a key role, supercomputers do not use wireless networking because of the probability of a higher latency which will be involved.<ref>[http://compnetworking.about.com/library/weekly/aa051902d.htm Computer networking interconnects article.]</ref>		Supercomputers are known to have a large number of processors and they need to work together. [http://en.wikipedia.org/wiki/Interconnection Interconnects] help connects different nodes in a super computer and they have the capacity to carry high bandwidth at the same time maintaining a very low latency. A node in a supercomputer usually denotes a communication end point, that is it maybe visualized as compute nodes, I/O nodes, service nodes and network nodes. In supercomputers where message passing plays a key role, the interconnects are kept as shot as possible just like the way electronic circuits were setup in a ‘C’ shape in the Cray systems to ensure that there is little latency. Since the interconnect plays a key role, supercomputers do not use wireless networking because of the probability of a higher latency which will be involved.<ref>[http://compnetworking.about.com/library/weekly/aa051902d.htm Computer networking interconnects article.]</ref>


	[[File:Interconnect_share.jpg\|300px\|thumb\|right\| Interconnect trends. source: http://www.top500.org/overtime/list/38/conn]]		[[File:Interconnect_share.jpg\|300px\|thumb\|right\| Interconnect trends. source: http://www.top500.org/overtime/list/38/conn]]

Vmarapa: /* Supercomputer Evolution */

2012-02-07T04:06:58Z

Supercomputer Evolution

← Older revision		Revision as of 04:06, 7 February 2012
Line 5:		Line 5:
	[[Image:Comp8.jpg\|thumb\|right\|400px\|This is Jaguar, currently rated the fifth most powerful computer in the world, with 54 teraflops. It uses 7,832 AMD Barcelona quad-core Opteron processors]]		[[Image:Comp8.jpg\|thumb\|right\|400px\|This is Jaguar, currently rated the fifth most powerful computer in the world, with 54 teraflops. It uses 7,832 AMD Barcelona quad-core Opteron processors]]

	The United States government has played ~~the~~ key role in the development and use of supercomputers, During World War II, the US Army paid for the construction of [http://en.wikipedia.org/wiki/ENIAC ENIAC] in order to speed the calculations of artillery tables. In the 30 years after World War II, the US government used high-performance computers to design nuclear weapons, break codes, and perform other security-related applications.		The United States government has played a key role in the development and use of supercomputers, During World War II, the US Army paid for the construction of [http://en.wikipedia.org/wiki/ENIAC ENIAC] in order to speed the calculations of artillery tables. In the 30 years after World War II, the US government used high-performance computers to design nuclear weapons, break codes, and perform other security-related applications.

	A supercomputer is generally considered to be the front-line “cutting-edge” in terms of processing capacity (number crunching) and computational speed at the time it is built, but with the pace of development yesterdays' supercomputers become regular servers today.		A supercomputer is generally considered to be the front-line “cutting-edge” in terms of processing capacity (number crunching) and computational speed at the time it is built, but with the pace of development yesterdays' supercomputers become regular servers today.

Vmarapa: /* Worlds fastest supercomputer - The K Computer */

2012-02-07T04:04:26Z

Worlds fastest supercomputer - The K Computer

← Older revision		Revision as of 04:04, 7 February 2012
Line 216:		Line 216:

	[[File:K_computer.jpg\|300px\|thumb\|right\| The K computer rack. source: http://www.fujitsu.com/global/about/tech/k/explore/]]		[[File:K_computer.jpg\|300px\|thumb\|right\| The K computer rack. source: http://www.fujitsu.com/global/about/tech/k/explore/]]
	[http://en.wikipedia.org/wiki/K_computer Japans K supercomputer] is currently the worlds fastest supercomputer and with major upgrades over the past few months, its speed is quadruple times faster than its closest competitor. The K computer, K - “kei” which stands for 10 quadrillion is a product of Fujitsu built at the [http://www.riken.go.jp/engn/r-world/research/lab/aics/index.html RIKEN Advanced Institute for Computational Science] campus in Japan. The K computer topped the supercomputer rankings in June’11 and managed to stay ahead of competition till date. A surprising aspect though is that the K computer is yet to become fully operational. There is little doubt in saying that the K computer is here to stay on top of the supercomputer charts for a while. But as history of supercomputing shows, the top supercomputer today is most of the times no where to be found in the top 10 in a few years of time. The K computer even though it has the highest power consumption (nearly 10 megawatts, equivalent to powering up ten thousand homes) is one of the most efficient systems on the list in terms of Mflops/watt delivered.<ref>[http://top500.org/lists/2011/11/press-release top500.org article]</ref><ref>[http://en.wikipedia.org/wiki/K_computer K Computer wiki]</ref>		[http://en.wikipedia.org/wiki/K_computer Japans K supercomputer] is currently the worlds fastest supercomputer and with major upgrades over the past few months, its speed is quadruple times faster than its closest competitor. The K computer, K - “kei” which stands for 10 quadrillion is a product of Fujitsu built at the [http://www.riken.go.jp/engn/r-world/research/lab/aics/index.html RIKEN Advanced Institute for Computational Science] campus in Japan. The K computer topped the supercomputer rankings in June’11 and managed to stay ahead of competition till date. A surprising aspect though is that the K computer is yet to become fully operational. There is little doubt in saying that the K computer is here to stay on top of the supercomputer charts for a while. But as history of supercomputing shows, the top supercomputer today is most of the times no where to be found in the top 10 in a few years of time. The K computer even though it has the highest power consumption (nearly 10 megawatts, equivalent to powering up ten thousand homes) is one of the most efficient systems on the list in terms of Mflops/watt delivered.<ref>[http://top500.org/lists/2011/11/press-release top500.org K computer article]</ref><ref>[http://en.wikipedia.org/wiki/K_computer K Computer wiki]</ref>

Vmarapa: /* Growing use of GPUs in supercomputing and the Barcelona Supercomputing Center */

2012-02-07T04:03:25Z

Growing use of GPUs in supercomputing and the Barcelona Supercomputing Center

← Older revision		Revision as of 04:03, 7 February 2012
Line 362:		Line 362:


	In the following years, the development of hybrid architecture will center around GPU and cell co-processor architectures. The focus, apart from higher computing speeds, will also concentrate on lesser power consumption. The IBM BlueGene family of supercomputers, as a result occupy the top 4 spots in the ~~Green500~~.org list of power efficient supercomputers. The Barcelona SuperComputing Center comes in at 7.<ref>[http://www-03.ibm.com/press/us/en/pressrelease/20210.wss Barcelona SuperComputing Center]</ref>		In the following years, the development of hybrid architecture will center around GPU and cell co-processor architectures. The focus, apart from higher computing speeds, will also concentrate on lesser power consumption. The IBM BlueGene family of supercomputers, as a result occupy the top 4 spots in the [http://www.green500.org/ Green500] list of power efficient supercomputers. The Barcelona SuperComputing Center comes in at 7.<ref>[http://www-03.ibm.com/press/us/en/pressrelease/20210.wss Barcelona SuperComputing Center]</ref>

Vmarapa: /* Supercomputer Programming Models */

2012-02-07T04:02:26Z

Supercomputer Programming Models

← Older revision		Revision as of 04:02, 7 February 2012
Line 228:		Line 228:

	== Supercomputer Programming Models ==		== Supercomputer Programming Models ==
	The parallel architectures of supercomputers often dictate the use of special programming techniques to exploit their speed. The base language of supercomputer code is, in general, Fortran or C, using special libraries to share data between nodes. Now environments such as [http://en.wikipedia.org/wiki/Parallel_Virtual_Machine PVM] and [http://en.wikipedia.org/wiki/Message_Passing_Interface MPI] for loosely connected clusters and [openmp.org/ OpenMP] for tightly coordinated shared memory machines are used. Significant effort is required to optimize a problem for the interconnect characteristics of the machine it is run on. The aim is to prevent any of the CPUs from wasting time waiting on data from other nodes.		The parallel architectures of supercomputers often dictate the use of special programming techniques to exploit their speed. The base language of supercomputer code is, in general, Fortran or C, using special libraries to share data between nodes. Now environments such as [http://en.wikipedia.org/wiki/Parallel_Virtual_Machine PVM] and [http://en.wikipedia.org/wiki/Message_Passing_Interface MPI] for loosely connected clusters and [http://openmp.org/ OpenMP] for tightly coordinated shared memory machines are used. Significant effort is required to optimize a problem for the interconnect characteristics of the machine it is run on. The aim is to prevent any of the CPUs from wasting time waiting on data from other nodes.

	Now we will discuss briefly regarding the programming languages mentioned above.		Now we will discuss briefly regarding the programming languages mentioned above.

	1) '''Fortran''' previously known as [en.wikipedia.org/wiki/Fortran FORTRAN] is a general-purpose, procedural, imperative programming language that is especially suited to computation like numeric and scientific computing. It was originally developed by IBM in the 1950s for scientific and engineering applications,then became very dominant in this area of programming early on and has been in use for over half a century in very much computationally intensive areas such as numerical weather prediction, finite element analysis, [http://en.wikipedia.org/wiki/Computational_fluid_dynamics computational fluid dynamics] (CFD), computational physics, and computational chemistry. It is one of the most popular and highly preferred language in the area of high-performance computing and is the language used for programs that benchmark and rank the world's fastest supercomputers.		1) '''Fortran''' previously known as [http://en.wikipedia.org/wiki/Fortran FORTRAN] is a general-purpose, procedural, imperative programming language that is especially suited to computation like numeric and scientific computing. It was originally developed by IBM in the 1950s for scientific and engineering applications,then became very dominant in this area of programming early on and has been in use for over half a century in very much computationally intensive areas such as numerical weather prediction, finite element analysis, [http://en.wikipedia.org/wiki/Computational_fluid_dynamics computational fluid dynamics] (CFD), computational physics, and computational chemistry. It is one of the most popular and highly preferred language in the area of high-performance computing and is the language used for programs that benchmark and rank the world's fastest supercomputers.

	Fortran a blend derived from The IBM Mathematical Formula Translating System encompasses a lineage of versions, each of which evolved to add extensions to the language while usually retaining compatibility with previous versions. Successive versions have added support for processing of character-based data (FORTRAN 77), array programming, modular programming and object-based programming (Fortran 90 / 95), and object-oriented and generic programming (Fortran 2003).		Fortran a blend derived from The IBM Mathematical Formula Translating System encompasses a lineage of versions, each of which evolved to add extensions to the language while usually retaining compatibility with previous versions. Successive versions have added support for processing of character-based data (FORTRAN 77), array programming, modular programming and object-based programming (Fortran 90 / 95), and object-oriented and generic programming (Fortran 2003).

Vmarapa: /* Interconnects */

2012-02-07T04:01:51Z

Interconnects

← Older revision		Revision as of 04:01, 7 February 2012
Line 155:		Line 155:
	[[File:Interconnect_share.jpg\|300px\|thumb\|right\| Interconnect trends. source: http://www.top500.org/overtime/list/38/conn]]		[[File:Interconnect_share.jpg\|300px\|thumb\|right\| Interconnect trends. source: http://www.top500.org/overtime/list/38/conn]]

	Notice the trend in the type of interconnects being used in supercomputers, the early days used the [http://en.wikipedia.org/wiki/SP-1_switch SP switches] now owned by IBM, then the trend shifted to [en.wikipedia.org/wiki/Non-Uniform_Memory_Access NUMA] links and the most popular interconnects these days are the [en.wikipedia.org/wiki/InfiniBand InfiniBand].<ref>[http://en.wikipedia.org/wiki/InfiniBand Infiniband]</ref> This trend is because InfiniBand offers high throughput, low latency, high [en.wikipedia.org/wiki/Quality_of_service QoS], failover etc. The future generations of super computers however might use the ethernet fabric architectures instead of InfiniBand because of the various advantages they have to offer.		Notice the trend in the type of interconnects being used in supercomputers, the early days used the [http://en.wikipedia.org/wiki/SP-1_switch SP switches] now owned by IBM, then the trend shifted to [http://en.wikipedia.org/wiki/Non-Uniform_Memory_Access NUMA] links and the most popular interconnects these days are the [http://en.wikipedia.org/wiki/InfiniBand InfiniBand].<ref>[http://en.wikipedia.org/wiki/InfiniBand Infiniband]</ref> This trend is because InfiniBand offers high throughput, low latency, high [http://en.wikipedia.org/wiki/Quality_of_service QoS], failover etc. The future generations of super computers however might use the ethernet fabric architectures instead of InfiniBand because of the various advantages they have to offer.

	=== System Architecture: Comparing Crays to IBM and Japan's K computer ===		=== System Architecture: Comparing Crays to IBM and Japan's K computer ===

Vmarapa: /* Supercomputer Design */

2012-02-07T04:00:54Z

Supercomputer Design

← Older revision		Revision as of 04:00, 7 February 2012
Line 103:		Line 103:
	area network and running programs written for parallel processing. The other approach,		area network and running programs written for parallel processing. The other approach,
	called vector processing, is to develop specialized hardware to solve complex calculations.		called vector processing, is to develop specialized hardware to solve complex calculations.
	This technique was employed in the [en.wikipedia.org/wiki/Earth_Simulator Earth Simulator], a Japanese supercomputer introduced in		This technique was employed in the [http://en.wikipedia.org/wiki/Earth_Simulator Earth Simulator], a Japanese supercomputer introduced in
	2002 that utilizes 640 nodes composed of 5104 specialized processors to execute 35.6 trillion		2002 that utilizes 640 nodes composed of 5104 specialized processors to execute 35.6 trillion
	mathematical operations per second. it is used to analyze earthquake, weather patterns,		mathematical operations per second. it is used to analyze earthquake, weather patterns,

Vmarapa: /* Supercomputer History in Japan */

2012-02-07T04:00:27Z

Supercomputer History in Japan

← Older revision		Revision as of 04:00, 7 February 2012
Line 64:		Line 64:
	At the end of 1987, NEC improved its supercomputer family with the introduction of A-series which gave improvements to the memory and I/O bandwidth. The top model, the SX-2A, had the same theoretical peak performance as the SX-2. Several low-range models were also announced but today none of them qualify for the TOP500.		At the end of 1987, NEC improved its supercomputer family with the introduction of A-series which gave improvements to the memory and I/O bandwidth. The top model, the SX-2A, had the same theoretical peak performance as the SX-2. Several low-range models were also announced but today none of them qualify for the TOP500.

	In 1989 NEC announced a rather aggressive new model, the [en.wikipedia.org/wiki/SX-3_supercomputer SX-3], with several important changes. The vector cycle time was brought down to 2.9 ns, the number of pipelines was doubled, but most significantly NEC added multiprocessing capability to its new series. It contained four independent arithmetic processors each with a scalar and a vector processing unit and NEC increased its performance by more than one order of magnitude of 22 Gflop/s from 1.33 on the SX-2A. The combination of these features made SX-3 the most powerful vector processor in the world. The total memory bandwidth was subdivided into two halves which in turn featured two vector load and one vector store paths per pipeline set as well as one scalar load and one scalar store path. This gave a total memory bandwidth to the vector units of about 66 GB/s. Like its predecessors, the SX-3 was to offer the memory bandwidth needed to sustain peak performance unless most operands were contained in the vector registers.		In 1989 NEC announced a rather aggressive new model, the [http://en.wikipedia.org/wiki/SX-3_supercomputer SX-3], with several important changes. The vector cycle time was brought down to 2.9 ns, the number of pipelines was doubled, but most significantly NEC added multiprocessing capability to its new series. It contained four independent arithmetic processors each with a scalar and a vector processing unit and NEC increased its performance by more than one order of magnitude of 22 Gflop/s from 1.33 on the SX-2A. The combination of these features made SX-3 the most powerful vector processor in the world. The total memory bandwidth was subdivided into two halves which in turn featured two vector load and one vector store paths per pipeline set as well as one scalar load and one scalar store path. This gave a total memory bandwidth to the vector units of about 66 GB/s. Like its predecessors, the SX-3 was to offer the memory bandwidth needed to sustain peak performance unless most operands were contained in the vector registers.

	In 1992 NEC announced the SX-3R with a couple of improvements compared to the first version. The clock was further reduced to 2.5 ns, so that the peak performance increased to 6.4 Gflop/s per processor		In 1992 NEC announced the SX-3R with a couple of improvements compared to the first version. The clock was further reduced to 2.5 ns, so that the peak performance increased to 6.4 Gflop/s per processor
Line 80:		Line 80:
	=== The VPP-500 series ===		=== The VPP-500 series ===

	In 1993 Fujitsu sprung a surprise to the world by announcing a Vector Parallel Processor (VPP) series that was designed for reaching in the range of hundreds of Gflop/s. At the core of the system is a combined [en.wikipedia.org/wiki/BiCMOS Ga-As/Bi-CMOS] processor, based largely on the original design of the VP-200. The processor chips gate delay was made as low as 60 ps in the Ga-As chips by using the most advanced hardware technology available. The resulting cycle time was 9.5 ns. The processor has four independent pipelines each capable of executing two Multiply-Add instructions in parallel resulting in a peak speed of 1.7 Gflop/s per processor. Each processor board is equipped with 256 Megabytes of central memory.		In 1993 Fujitsu sprung a surprise to the world by announcing a Vector Parallel Processor (VPP) series that was designed for reaching in the range of hundreds of Gflop/s. At the core of the system is a combined [http://en.wikipedia.org/wiki/BiCMOS Ga-As/Bi-CMOS] processor, based largely on the original design of the VP-200. The processor chips gate delay was made as low as 60 ps in the Ga-As chips by using the most advanced hardware technology available. The resulting cycle time was 9.5 ns. The processor has four independent pipelines each capable of executing two Multiply-Add instructions in parallel resulting in a peak speed of 1.7 Gflop/s per processor. Each processor board is equipped with 256 Megabytes of central memory.

	The most amazing part of the VPP-500 is the capability to interconnect up to 222 processors via a cross-bar network with two independent (read/write) connections, each operating at 400 MB/s. The total memory is addressed via virtual shared memory primitives. The system is meant to be front-ended by a VP-2x00 system that handles input/output and permanent file store, and job queue logistics.		The most amazing part of the VPP-500 is the capability to interconnect up to 222 processors via a cross-bar network with two independent (read/write) connections, each operating at 400 MB/s. The total memory is addressed via virtual shared memory primitives. The system is meant to be front-ended by a VP-2x00 system that handles input/output and permanent file store, and job queue logistics.

	As mentioned in the introduction, an early version of this system called the [en.wikipedia.org/wiki/Numerical_Wind_Tunnel Numeric Wind Tunnel], was developed together with NAL. This early version of the VPP-500 (with 140 processors) is today the fastest supercomputer in the world and stands out at the beginning of the TOP500 due to a value that is twice that of the TMC CM-5/1024 installed at Los Alamo.		As mentioned in the introduction, an early version of this system called the [http://en.wikipedia.org/wiki/Numerical_Wind_Tunnel Numeric Wind Tunnel], was developed together with NAL. This early version of the VPP-500 (with 140 processors) is today the fastest supercomputer in the world and stands out at the beginning of the TOP500 due to a value that is twice that of the TMC CM-5/1024 installed at Los Alamo.

	=== Hitachi's Supercomputers ===		=== Hitachi's Supercomputers ===

Vmarapa: /* The future of supercomputing */

2012-02-07T03:59:08Z

The future of supercomputing

← Older revision		Revision as of 03:59, 7 February 2012
Line 352:		Line 352:


	Recently, The [www.top500.org/site/2540 Barcelona Supercomputing Center] (BSC) announced plans to build a supercomputer on an [http://en.wikipedia.org/wiki/ARM_architecture ARM-based architecture]. This was the first radical step taken towards building a supercomputer with drastic power efficient requirements. "BSC is planning to build the first ARM supercomputer, accelerated by CUDA GPUs<ref>[http://5601-blogs-nvidia-com.voxcdn.com/wp-content/uploads/2011/11/IDC-Exascale-Executive-Brief_Nov2011.pdf BSC ARM supercomputer]</ref>, for scientific research. This prototype system will use NVIDIA’s quad-core ARM-based Tegra 3 system-on-a-chip, along with NVIDIA CUDA GPUs on a hardware board designed by SECO., to accelerate a variety of scientific research projects". The ARM architecture, popular for its use in cellphones for its low power consumption. It has been documented that GPU based heterogeneous devices are set to replace x86 processors because of their power efficiency. More and more supercomputers today are using so-called 'accelarator technology' along with their CPUs. In 2008 By this time, 28% of the HPC sites were using accelerator technology		Recently, The [http://www.top500.org/site/2540 Barcelona Supercomputing Center] (BSC) announced plans to build a supercomputer on an [http://en.wikipedia.org/wiki/ARM_architecture ARM-based architecture]. This was the first radical step taken towards building a supercomputer with drastic power efficient requirements. "BSC is planning to build the first ARM supercomputer, accelerated by CUDA GPUs<ref>[http://5601-blogs-nvidia-com.voxcdn.com/wp-content/uploads/2011/11/IDC-Exascale-Executive-Brief_Nov2011.pdf BSC ARM supercomputer]</ref>, for scientific research. This prototype system will use NVIDIA’s quad-core ARM-based Tegra 3 system-on-a-chip, along with NVIDIA CUDA GPUs on a hardware board designed by SECO., to accelerate a variety of scientific research projects". The ARM architecture, popular for its use in cellphones for its low power consumption. It has been documented that GPU based heterogeneous devices are set to replace x86 processors because of their power efficiency. More and more supercomputers today are using so-called 'accelarator technology' along with their CPUs. In 2008 By this time, 28% of the HPC sites were using accelerator technology
	— a threefold increase from two years earlier — and nearly all of these accelerators were GPUs. "9% of HPC sites were using some form of accelerator technology alongside CPUs in their installed systems.By this time, 28% of the HPC sites were using accelerator technology — a threefold increase from two years earlier — and nearly all of these accelerators were GPUs. Although GPUs represent only about 5% of the processor counts in heterogeneous systems, their numbers are growing rapidly".<ref>[http://5601-blogs-nvidia-com.voxcdn.com/wp-content/uploads/2011/11/IDC-Exascale-Executive-Brief_Nov2011.pdf IDC Exascale Brief]</ref>		— a threefold increase from two years earlier — and nearly all of these accelerators were GPUs. "9% of HPC sites were using some form of accelerator technology alongside CPUs in their installed systems.By this time, 28% of the HPC sites were using accelerator technology — a threefold increase from two years earlier — and nearly all of these accelerators were GPUs. Although GPUs represent only about 5% of the processor counts in heterogeneous systems, their numbers are growing rapidly".<ref>[http://5601-blogs-nvidia-com.voxcdn.com/wp-content/uploads/2011/11/IDC-Exascale-Executive-Brief_Nov2011.pdf IDC Exascale Brief]</ref>

Vmarapa: /* Message to the reader */

2012-02-07T03:51:22Z

Message to the reader

← Older revision		Revision as of 03:51, 7 February 2012
Line 377:		Line 377:

	== Message to the reader==		== Message to the reader==
	This article is the combined effort of teams of CSC 506 in Spring '11 and '12. Majority of this page was developed by the '11 team. The '12 team has implemented in whole the sections 7,8,9,12.5,13(except last 3 paragraphs), 14. Proper references have been provided for all the work the '12 team is responsible for.		This article is the combined effort of teams of CSC 506 in Spring '11 and '12. Majority of this page was developed by the '11 team. The '12 team has implemented in whole the sections 7,8,9,12.5,13(except last 3 paragraphs), 14. Proper references have been provided for all the work the '12 team is responsible for. We have also added in-line definitions to many terms in the material written by the '12 team and the '11 team. We have also updated the top 10 supercomputer list to point to the new one and references are more accurate.

	== References (created by the authors of old wiki article written in 2011) ==		== References (created by the authors of old wiki article written in 2011) ==