CSC/ECE 506 Spring 2015/1b DL - Revision history

Dtlee2 at 01:47, 3 February 2015

2015-02-03T01:47:15Z

← Older revision		Revision as of 01:47, 3 February 2015
Line 31:		Line 31:
	<p>The need to perform complex mathematical tasks demanded by several applications of the type discussed in the preceding paragraph is beyond the capabilities of a single or a group of processing units. A large number of processing units need to cooperate and collaborate to accomplish tasks of such magnitude in a timely fashion. This concept of a '''massive parallel system''' has become an inherent feature of contemporary supercomputers. In order for supercomputing nodes to run tasks in parallel, a software solution is used to provide a messaging API for communication between threads of a given task. OpenMP is considered the standard for applications using different levels of parallelism including loop-level, nested, and task level as part of the OpenMP 4.0 specification<ref>https://computing.llnl.gov/tutorials/parallel_comp</ref>.</p>		<p>The need to perform complex mathematical tasks demanded by several applications of the type discussed in the preceding paragraph is beyond the capabilities of a single or a group of processing units. A large number of processing units need to cooperate and collaborate to accomplish tasks of such magnitude in a timely fashion. This concept of a '''massive parallel system''' has become an inherent feature of contemporary supercomputers. In order for supercomputing nodes to run tasks in parallel, a software solution is used to provide a messaging API for communication between threads of a given task. OpenMP is considered the standard for applications using different levels of parallelism including loop-level, nested, and task level as part of the OpenMP 4.0 specification<ref>https://computing.llnl.gov/tutorials/parallel_comp</ref>.</p>

	[[File:NodeSocketCores.jpg\|frame\|left]]		[[File:NodeSocketCores.jpg\|frame\|left\|Taken from https://computing.llnl.gov/tutorials/parallel_comp/]]


Line 64:		Line 64:

	==== STREAM ====		==== STREAM ====
	<p>Unlike the LINPACK benchmark suite which assess processing performance, the STREAM benchmark measures the sustainable memory bandwidth and computation rate of simple vector kernels. STREAM uses large datasets that exceed the size of a system’s immediate memory caches which is especially prevalent in applications using large-scale vector arrays<ref>http://www.cs.virginia.edu/stream</ref>. With CPU processors able to process data at quicker speeds, computational performance is less of a concern in the STREAM benchmark as memory latency can manifest in CPU idle times. STREAM tests the memory bandwidth by using COPY, SCALE, SUM, and TRIAD operations ordered by its computational complexity. This scale of complexity of each operation can be used to mimic real world applications when the data is stored in a location that is not quickly accessible. Memory bandwidth performance is measured in GB/s which can then further be extrapolated by the performance per CPU core.<ref>http://www.admin-magazine.com/HPC/Articles/Finding-Memory-Bottlenecks-with-Stream</ref></p>		<p>Unlike the LINPACK benchmark suite which assess processing performance, the STREAM benchmark measures the sustainable memory bandwidth and computation rate of simple vector kernels. STREAM uses large datasets that exceed the size of a system’s immediate memory caches which is especially prevalent in applications using large-scale vector arrays<ref>http://www.cs.virginia.edu/stream</ref>. With CPU processors able to process data at quicker speeds, computational performance is less of a concern in the STREAM benchmark as memory latency can manifest in CPU idle times. </p>

	[[File:StreamMemoryBanwidth_4-28-14.png\|~~center~~]]		[[File:StreamMemoryBanwidth_4-28-14.png\|frame\|left\|Stream Memory Bandwidth - 2014 - Top 20 Listings]]

			<p>STREAM tests the memory bandwidth by using COPY, SCALE, SUM, and TRIAD operations ordered by its computational complexity. This scale of complexity of each operation can be used to mimic real world applications when the data is stored in a location that is not quickly accessible. Memory bandwidth performance is measured in GB/s which can then further be extrapolated by the performance per CPU core.<ref>http://www.admin-magazine.com/HPC/Articles/Finding-Memory-Bottlenecks-with-Stream</ref></p>

	==== Graph500 ====		==== Graph500 ====
	<p>The Graph500 organization proposes an alternative evaluation of supercomputing performance to the standard Top500 by introducing a new metric identified as traversed edges per second (TEPs)<ref>http://www.graph500.org</ref>. The justification on the Graph500 benchmark is to address data intensive applications designed for 3D simulations which is a commonly recognized pitfall of the LINPACK algorithm.</p>		<p>The Graph500 organization proposes an alternative evaluation of supercomputing performance to the standard Top500 by introducing a new metric identified as traversed edges per second (TEPs)<ref>http://www.graph500.org</ref>. The justification on the Graph500 benchmark is to address data intensive applications designed for 3D simulations which is a commonly recognized pitfall of the LINPACK algorithm.</p>

	[[File:Graph500_11-2014_listings.png\|frame\|middle\|Graph 500 - November 2014 - Top 10 Listings]]		[[File:Graph500_11-2014_listings.png\|frame\|middle\|800px\|Graph 500 - November 2014 - Top 10 Listings]]

	<p>The Graph500 algorithm starts by generating an edge list for a large vector NxM graph predefined by the number of vertices and the edge factor (ratio of the edges to vertices). Using the edge list is generated, a list of tuples each containing an endpoint and an edge weight is assigned to construct a graph matrix. Using a random generated search key, the benchmark measures the time it takes to access data stored at a given vertex in TEPs by taking the number of traversed edges divided by the access time.</p>		<p>The Graph500 algorithm starts by generating an edge list for a large vector NxM graph predefined by the number of vertices and the edge factor (ratio of the edges to vertices). Using the edge list is generated, a list of tuples each containing an endpoint and an edge weight is assigned to construct a graph matrix. Using a random generated search key, the benchmark measures the time it takes to access data stored at a given vertex in TEPs by taking the number of traversed edges divided by the access time.</p>

Dtlee2: added pictures; need to format better

2015-02-02T23:45:23Z

added pictures; need to format better

← Older revision		Revision as of 23:45, 2 February 2015
Line 26:		Line 26:
	=== Taxonomy ===		=== Taxonomy ===
	<p>Supercomputers are a collection of inter-networked nodes with each node having several CPUs (central processing units) in them. An individual CPU contains multiple cores with each core able to handle a unique thread of execution. Supercomputers are able to process data in parallel by having multiple processors run different sections of the same program concurrently resulting in quicker run times. Furthermore, multi-core processors allow for simultaneous handling of multiple tasks in parallel. The largest supercomputers can have processors numbering hundreds of thousands <ref>https://computing.llnl.gov/tutorials/parallel_comp</ref> that are able to achieve speeds up to the petascale.</p>		<p>Supercomputers are a collection of inter-networked nodes with each node having several CPUs (central processing units) in them. An individual CPU contains multiple cores with each core able to handle a unique thread of execution. Supercomputers are able to process data in parallel by having multiple processors run different sections of the same program concurrently resulting in quicker run times. Furthermore, multi-core processors allow for simultaneous handling of multiple tasks in parallel. The largest supercomputers can have processors numbering hundreds of thousands <ref>https://computing.llnl.gov/tutorials/parallel_comp</ref> that are able to achieve speeds up to the petascale.</p>

	~~[[Image:NodeSocketCores.jpg\|thumb\|left]]~~

	<p>The majority of supercomputing architectures operate on some variant of a Linux distribution for its operating system and is typically optimized for the hardware components in the node. The basis for using Linux is because of its scalability and ease of configuration which can simplify the deployment of hundreds of thousands of CPUs significantly trivial. It also provides a cost effective solution and allows administrators more flexibility in terms of customization options so installation can be tailored to the application environment <ref>http://www.unixmen.com/why-do-super-computers-use-linux</ref>.</p>		<p>The majority of supercomputing architectures operate on some variant of a Linux distribution for its operating system and is typically optimized for the hardware components in the node. The basis for using Linux is because of its scalability and ease of configuration which can simplify the deployment of hundreds of thousands of CPUs significantly trivial. It also provides a cost effective solution and allows administrators more flexibility in terms of customization options so installation can be tailored to the application environment <ref>http://www.unixmen.com/why-do-super-computers-use-linux</ref>.</p>

	<p>The need to perform complex mathematical tasks demanded by several applications of the type discussed in the preceding paragraph is beyond the capabilities of a single or a group of processing units. A large number of processing units need to cooperate and collaborate to accomplish tasks of such magnitude in a timely fashion. This concept of a '''massive parallel system''' has become an inherent feature of contemporary supercomputers. In order for supercomputing nodes to run tasks in parallel, a software solution is used to provide a messaging API for communication between threads of a given task. OpenMP is considered the standard for applications using different levels of parallelism including loop-level, nested, and task level as part of the OpenMP 4.0 specification<ref>https://computing.llnl.gov/tutorials/parallel_comp</ref>.</p>		<p>The need to perform complex mathematical tasks demanded by several applications of the type discussed in the preceding paragraph is beyond the capabilities of a single or a group of processing units. A large number of processing units need to cooperate and collaborate to accomplish tasks of such magnitude in a timely fashion. This concept of a '''massive parallel system''' has become an inherent feature of contemporary supercomputers. In order for supercomputing nodes to run tasks in parallel, a software solution is used to provide a messaging API for communication between threads of a given task. OpenMP is considered the standard for applications using different levels of parallelism including loop-level, nested, and task level as part of the OpenMP 4.0 specification<ref>https://computing.llnl.gov/tutorials/parallel_comp</ref>.</p>

			[[File:NodeSocketCores.jpg\|frame\|left]]


	=== Design Considerations ===		=== Design Considerations ===
Line 65:		Line 66:
	<p>Unlike the LINPACK benchmark suite which assess processing performance, the STREAM benchmark measures the sustainable memory bandwidth and computation rate of simple vector kernels. STREAM uses large datasets that exceed the size of a system’s immediate memory caches which is especially prevalent in applications using large-scale vector arrays<ref>http://www.cs.virginia.edu/stream</ref>. With CPU processors able to process data at quicker speeds, computational performance is less of a concern in the STREAM benchmark as memory latency can manifest in CPU idle times. STREAM tests the memory bandwidth by using COPY, SCALE, SUM, and TRIAD operations ordered by its computational complexity. This scale of complexity of each operation can be used to mimic real world applications when the data is stored in a location that is not quickly accessible. Memory bandwidth performance is measured in GB/s which can then further be extrapolated by the performance per CPU core.<ref>http://www.admin-magazine.com/HPC/Articles/Finding-Memory-Bottlenecks-with-Stream</ref></p>		<p>Unlike the LINPACK benchmark suite which assess processing performance, the STREAM benchmark measures the sustainable memory bandwidth and computation rate of simple vector kernels. STREAM uses large datasets that exceed the size of a system’s immediate memory caches which is especially prevalent in applications using large-scale vector arrays<ref>http://www.cs.virginia.edu/stream</ref>. With CPU processors able to process data at quicker speeds, computational performance is less of a concern in the STREAM benchmark as memory latency can manifest in CPU idle times. STREAM tests the memory bandwidth by using COPY, SCALE, SUM, and TRIAD operations ordered by its computational complexity. This scale of complexity of each operation can be used to mimic real world applications when the data is stored in a location that is not quickly accessible. Memory bandwidth performance is measured in GB/s which can then further be extrapolated by the performance per CPU core.<ref>http://www.admin-magazine.com/HPC/Articles/Finding-Memory-Bottlenecks-with-Stream</ref></p>

	[[~~Image~~:StreamMemoryBanwidth_4-28-14.png~~\|thumb~~\|center]]		[[File:StreamMemoryBanwidth_4-28-14.png\|center]]

	==== Graph500 ====		==== Graph500 ====
	<p>The Graph500 organization proposes an alternative evaluation of supercomputing performance to the standard Top500 by introducing a new metric identified as traversed edges per second (TEPs)<ref>http://www.graph500.org</ref>. The justification on the Graph500 benchmark is to address data intensive applications designed for 3D simulations which is a commonly recognized pitfall of the LINPACK algorithm.</p>		<p>The Graph500 organization proposes an alternative evaluation of supercomputing performance to the standard Top500 by introducing a new metric identified as traversed edges per second (TEPs)<ref>http://www.graph500.org</ref>. The justification on the Graph500 benchmark is to address data intensive applications designed for 3D simulations which is a commonly recognized pitfall of the LINPACK algorithm.</p>

	[[~~Image~~:Graph500_11-2014_listings.png\|~~thumb~~\|~~left~~]]		[[File:Graph500_11-2014_listings.png\|frame\|middle\|Graph 500 - November 2014 - Top 10 Listings]]

	<p>The Graph500 algorithm starts by generating an edge list for a large vector NxM graph predefined by the number of vertices and the edge factor (ratio of the edges to vertices). Using the edge list is generated, a list of tuples each containing an endpoint and an edge weight is assigned to construct a graph matrix. Using a random generated search key, the benchmark measures the time it takes to access data stored at a given vertex in TEPs by taking the number of traversed edges divided by the access time.</p>		<p>The Graph500 algorithm starts by generating an edge list for a large vector NxM graph predefined by the number of vertices and the edge factor (ratio of the edges to vertices). Using the edge list is generated, a list of tuples each containing an endpoint and an edge weight is assigned to construct a graph matrix. Using a random generated search key, the benchmark measures the time it takes to access data stored at a given vertex in TEPs by taking the number of traversed edges divided by the access time.</p>

Dtlee2: Grammar revisions

2015-02-02T19:37:32Z

Grammar revisions

← Older revision		Revision as of 19:37, 2 February 2015
Line 13:		Line 13:

	=== On demand supercomputing and cloud ===		=== On demand supercomputing and cloud ===
	<p>The current paradigm in the cloud computing<ref>http://en.wikipedia.org/wiki/Cloud_computing</ref> ~~area~~ is ~~theavailability~~ of supercomputing ~~application~~ to ubiquitous computer usage. ~~Computing large~~ analytical calculations using supercomputers available to ~~small~~ industries ~~or individuals~~ using a cloud environment ~~will~~ change the dynamics of process of manufacturing in small scale industries. On-demand supercomputing will enable companies and enterprises to significantly decrease the development and evaluation time of their ~~prototype~~ helping shape their financial infrastructure while also leading to quicker product deployments in ~~the market~~<ref>http://arstechnica.com/information-technology/2013/11/18-hours-33k-and-156314-cores-amazon-cloud-hpc-hits-a-petaflop/</ref>.</p>		<p>The current paradigm in the cloud computing<ref>http://en.wikipedia.org/wiki/Cloud_computing</ref> industry is the availability of supercomputing applications to ubiquitous computer usage. Large analytical calculations using supercomputers have become available to smaller industries using a cloud environment and it is expected to change the dynamics of process manufacturing in these small scale industries. On-demand supercomputing will also enable companies and enterprises to significantly decrease the development and evaluation time of their prototypes helping shape their financial infrastructure while also leading to quicker product deployments in their respective markets<ref>http://arstechnica.com/information-technology/2013/11/18-hours-33k-and-156314-cores-amazon-cloud-hpc-hits-a-petaflop/</ref>.</p>

	<p>The [http://www.open-power.org/ OpenPOWER] Foundation is a prominent example in discussing future growth trends. It is a consortium initiated by IBM in collaboration between Google, [http://www.tyan.com/ Tyan], [http://www.tyan.com/ Nvidia] and [http://www.mellanox.com/ Mellanox] as its founding members. IBM is opening up its POWER Architecture technology on a liberal license which will enable the vendors build and configure their own customized servers, and also help design their networking and storage hardware for cloud computing and data centers<ref>http://en.wikipedia.org/wiki/OpenPOWER_Foundation</ref>.</p>		<p>The [http://www.open-power.org/ OpenPOWER] Foundation is a prominent example in discussing future growth trends. It is a consortium initiated by IBM in collaboration between Google, [http://www.tyan.com/ Tyan], [http://www.tyan.com/ Nvidia] and [http://www.mellanox.com/ Mellanox] as its founding members. IBM is opening up its POWER Architecture technology on a liberal license which will enable the vendors build and configure their own customized servers, and also help design their networking and storage hardware for cloud computing and data centers<ref>http://en.wikipedia.org/wiki/OpenPOWER_Foundation</ref>.</p>

Dtlee2 at 19:34, 2 February 2015

2015-02-02T19:34:00Z

← Older revision		Revision as of 19:34, 2 February 2015
Line 3:		Line 3:

	== Introduction ==		== Introduction ==
	<p>A [http://dictionary.reference.com/browse/supercomputer supercomputer] is a computer at the leading edge of state of the art processing capacity specifically designed for fast calculation speeds. The early 1960’s saw the advent of such machines and in the 1970’s, systems comprising a few processors were used which subsequently increased to thousands during the 1990’s and by the end of 20th century, massive parallel supercomputers with tens of thousands of processors with extremely high processing speed came into prominence<ref>http://en.wikipedia.org/wiki/Supercomputer#History</ref>. Supercomputers play important role in the field of computational science as well as in cryptanalysis. They are also used in the quantum mechanics, weather forecasting and climate research and oil and gas exploration. Earlier supercomputer architectures were aimed at exploiting parallelism at the processor level like vector processing followed by multiprocessor systems with shared memory. With an increasing demand for more complex and faster computations, processors with shared memory architectures were not enough in terms of processing capabilities which paved the way for hybrid structures such as clusters of multi-node mesh networks where each node serves as a multiprocessing element<ref>http://en.wikipedia.org/wiki/Supercomputer#Performance_measurement</ref>.</p>		<p>A [http://dictionary.reference.com/browse/supercomputer supercomputer] is a computer at the leading edge of state of the art processing capacity specifically designed for fast calculation speeds. The early 1960’s saw the advent of such machines and in the 1970’s, systems comprising a few processors were used which subsequently increased to thousands during the 1990’s and by the end of 20th century, massive parallel supercomputers with tens of thousands of processors with extremely high processing speed came into prominence<ref>http://en.wikipedia.org/wiki/Supercomputer#History</ref>. Supercomputers play an important role in the field of computational science as well as in cryptanalysis. They are also used in the quantum mechanics, weather forecasting and climate research and oil and gas exploration. Earlier supercomputer architectures were aimed at exploiting parallelism at the processor level like vector processing followed by multiprocessor systems with shared memory. With an increasing demand for more complex and faster computations, processors with shared memory architectures were not enough in terms of processing capabilities which paved the way for hybrid structures such as clusters of multi-node mesh networks where each node serves as a multiprocessing element<ref>http://en.wikipedia.org/wiki/Supercomputer#Performance_measurement</ref>.</p>

	== Supercomputing Applications ==		== Supercomputing Applications ==

Dtlee2: Grammar checks; changes to wording in paragraphs

2015-02-02T19:30:26Z

Grammar checks; changes to wording in paragraphs

Show changes

Dtlee2: added a few TODO comments for next submission; made a few formatting changes

2015-01-28T03:27:03Z

added a few TODO comments for next submission; made a few formatting changes

Show changes

Dtlee2: added a few hyperlinks/references

2015-01-28T02:04:07Z

added a few hyperlinks/references

Dtlee2: added changes to characteristis of supercomputers section

2015-01-28T01:46:55Z

added changes to characteristis of supercomputers section

← Older revision		Revision as of 01:46, 28 January 2015
Line 22:		Line 22:
	<p>The definition of a supercomputer established in the previous section gives some insights into the nature and features of such computing systems. As a result, certain patterns or a combination of features emerge, which are unique to supercomputers. This section discusses those features and characteristics<ref>http://royal.pingdom.com/2012/11/13/new-top-supercomputer-dumps-cores-and-increases-power-efficiency/</ref></p>		<p>The definition of a supercomputer established in the previous section gives some insights into the nature and features of such computing systems. As a result, certain patterns or a combination of features emerge, which are unique to supercomputers. This section discusses those features and characteristics<ref>http://royal.pingdom.com/2012/11/13/new-top-supercomputer-dumps-cores-and-increases-power-efficiency/</ref></p>

	=== ~~Implementation and Organization~~ ===		=== Taxonomy ===
	*'''Massive Parallel System:'''		<p>Supercomputers are a collection of inter-networked nodes with each node having several CPUs in them. An individual CPU contains multiple cores with each core able to handle a unique thread of execution. Supercomputers are able to process data in parallel by having multiple processors run different sections of the same program concurrently resulting in quicker run times. Multi-core processors allow for simultaneous handling of multiple tasks. The largest supercomputers can have processors numbering hundreds of thousands <ref>https://computing.llnl.gov/tutorials/parallel_comp</ref> that are able to achieve speeds on the petascale.</p>
	<p>~~The need to perform complex mathematical tasks demanded by~~ several ~~applications of the type discussed~~ in ~~the preceding paragraph is beyond the capabilities of a single or~~ a ~~group~~ of ~~processing units~~. ~~A large number of processing units need~~ to ~~cooperate and collaborate to accomplish tasks~~ of ~~such magnitude~~ in ~~a timely fashion~~. ~~This concept~~ of ~~a '''massive parallel system''' has become an inherent feature~~ of ~~contemporary supercomputers~~.</p>

			<p>The majority of supercomputing architectures operate on some variant of a Linux distribution for its operating system and is typically optimized for the hardware components in the node. The basis for using Linux is because of its scalability and ease of configuration which can simplify the deployment of hundreds of thousands of CPUs significantly trivial. It also provides a cost effective solution and allows administrators more flexibility in terms of customization options so installation can be tailored to the application environment <ref>http://www.unixmen.com/why-do-super-computers-use-linux</ref>.</p>

			<p>The need to perform complex mathematical tasks demanded by several applications of the type discussed in the preceding paragraph is beyond the capabilities of a single or a group of processing units. A large number of processing units need to cooperate and collaborate to accomplish tasks of such magnitude in a timely fashion. This concept of a '''massive parallel system''' has become an inherent feature of contemporary supercomputers. In order for supercomputing nodes to run tasks in parallel, a software solution is used to provide a messaging API for communication between threads of a given task. OpenMP is considered the standard for applications using different levels of parallelism including loop-level, nested, and task level as part of the OpenMP 4.0 specification<ref>https://computing.llnl.gov/tutorials/parallel_comp</ref>.</p>

	=== Processors ===		=== Processors ===

Dtlee2: added to supercomputing applications section

2015-01-28T01:42:18Z

added to supercomputing applications section

← Older revision		Revision as of 01:42, 28 January 2015
Line 6:		Line 6:

	== Supercomputing Applications ==		== Supercomputing Applications ==
			<p>Supercomputers are used by many different organizations functioning in both the private and public sectors. In the public sector, the U.S. federal government uses supercomputing resources in areas of medical research, weather simulations, and space technology development. The private sector has also seen growing usage of supercomputer environments and are seen commonly within academic and financial institutions, or telecommunications and manufacturing companies. Below lists applications used today:</p>

			<p>Supercomputing capabilities allows scientists to study simulations based on statistical modelling of large ordered data sets on phenomena that would otherwise be impossible to dissect in real time. For instance, the [http://en.wikipedia.org/wiki/Human_Brain_Project Human Brain Project] is a research project aimed to simulate brain activity which would foster research into neurology and brain development [7]. According to some sources, Japan's K computer (ranked 4th on Top500's September 2014 listing, 10.5 petaflops/s) took 40 minutes to replicate only a second worth of brain activity<ref>http://www.livescience.com/42561-supercomputer-models-brain-activity.html</ref></p>

			<p>Supercomputing is also prominent in weather forecasting with meteorological models becoming increasingly data intensive. A typical workload is composed of data assimilation, deterministic forecast models and ensemble forecast models. In addition, specialized models may be applied for areas such as extreme events, air quality, ocean waves and road conditions. Quicker and more accurate responses to sudden fluctuations in climate changes are made possible by advents in the supercomputing industry.<ref><http://www.cray.com/blog/the-critical-role-of-supercomputers-in-weather-forecasting</ref></p>

	=== On demand supercomputing and cloud ===		=== On demand supercomputing and cloud ===
	<p>The current paradigm in the cloud computing<ref>http://en.wikipedia.org/wiki/Cloud_computing</ref> tribe will be the availability of supercomputing application to ubiquitous computer. Computing big analytical calculations using supercomputer available to small industries or individuals using cloud will change the dynamics of process of manufacturing in small scale industries. On demand supercomputing will enable companies and enterprises to significantly decrease the development and evaluation time of their prototype and this will help them change their costing structure along with decrease in time to market a product<ref>http://arstechnica.com/information-technology/2013/11/18-hours-33k-and-156314-cores-amazon-cloud-hpc-hits-a-petaflop/</ref>.</p>		<p>The current paradigm in the cloud computing<ref>http://en.wikipedia.org/wiki/Cloud_computing</ref> tribe will be the availability of supercomputing application to ubiquitous computer. Computing big analytical calculations using supercomputer available to small industries or individuals using cloud will change the dynamics of process of manufacturing in small scale industries. On demand supercomputing will enable companies and enterprises to significantly decrease the development and evaluation time of their prototype and this will help them change their costing structure along with decrease in time to market a product<ref>http://arstechnica.com/information-technology/2013/11/18-hours-33k-and-156314-cores-amazon-cloud-hpc-hits-a-petaflop/</ref>.</p>
	<p>The [http://www.open-power.org/ OpenPOWER] Foundation is one a prominent example which in a way represents the shape of things to come. It is a consortium initiated by IBM in collaboration with Google, [http://www.tyan.com/ Tyan], [http://www.tyan.com/ Nvidia] and [http://www.mellanox.com/ Mellanox] as the founding members. IBM is opening up its POWER Architecture technology on a liberal license which will enable the server vendor ecosystem build and configure their own customized server, networking and storage hardware for cloud computing and data centers<ref>http://en.wikipedia.org/wiki/OpenPOWER_Foundation</ref>.</p>		<p>The [http://www.open-power.org/ OpenPOWER] Foundation is one a prominent example which in a way represents the shape of things to come. It is a consortium initiated by IBM in collaboration with Google, [http://www.tyan.com/ Tyan], [http://www.tyan.com/ Nvidia] and [http://www.mellanox.com/ Mellanox] as the founding members. IBM is opening up its POWER Architecture technology on a liberal license which will enable the server vendor ecosystem build and configure their own customized server, networking and storage hardware for cloud computing and data centers<ref>http://en.wikipedia.org/wiki/OpenPOWER_Foundation</ref>.</p>
	<p>Though the cloud a viable platform for medium scale applications; interconnects and bandwidth still is a major bottleneck to executing large scale tightly coupled high performance computing applications on the cloud. The large scale interest and migration to optical networks does instill some hope but it is highly unlikely that the cloud can ever replace supercomputers<ref>http://www.computer.org/portal/web/computingnow/archive/september2012</ref><ref>http://gigaom.com/2011/11/14/how-the-cloud-is-reshaping-supercomputers/</ref>.</p>		<p>Though the cloud a viable platform for medium scale applications; interconnects and bandwidth still is a major bottleneck to executing large scale tightly coupled high performance computing applications on the cloud. The large scale interest and migration to optical networks does instill some hope but it is highly unlikely that the cloud can ever replace supercomputers<ref>http://www.computer.org/portal/web/computingnow/archive/september2012</ref><ref>http://gigaom.com/2011/11/14/how-the-cloud-is-reshaping-supercomputers/</ref>.</p>


	== Characteristics of supercomputers ==		== Characteristics of supercomputers ==
	[[Image:4.jpg\|thumb\|right\|center\|upright\|500px\|<b>Tianhe-2 - The fastest supercomputer as of 2013</b> http://www.engineering.com/DesignerEdge/DesignerEdgeArticles/ArticleID/6676/Tianhe-2-Tops-Supercomputer-List.aspx]]		[[Image:4.jpg\|thumb\|right\|center\|upright\|500px\|<b>Tianhe-2 - The fastest supercomputer as of 2013</b> http://www.engineering.com/DesignerEdge/DesignerEdgeArticles/ArticleID/6676/Tianhe-2-Tops-Supercomputer-List.aspx]]

Dtlee2: Added subsections for hpcg, stream, graph500 under baseline section

2015-01-26T05:01:35Z

Added subsections for hpcg, stream, graph500 under baseline section

← Older revision		Revision as of 05:01, 26 January 2015
Line 6:		Line 6:

	== Supercomputing Applications ==		== Supercomputing Applications ==

	=== On demand supercomputing and cloud ===		=== On demand supercomputing and cloud ===
	<p>The current paradigm in the cloud computing<ref>http://en.wikipedia.org/wiki/Cloud_computing</ref> tribe will be the availability of supercomputing application to ubiquitous computer. Computing big analytical calculations using supercomputer available to small industries or individuals using cloud will change the dynamics of process of manufacturing in small scale industries. On demand supercomputing will enable companies and enterprises to significantly decrease the development and evaluation time of their prototype and this will help them change their costing structure along with decrease in time to market a product<ref>http://arstechnica.com/information-technology/2013/11/18-hours-33k-and-156314-cores-amazon-cloud-hpc-hits-a-petaflop/</ref>.</p>		<p>The current paradigm in the cloud computing<ref>http://en.wikipedia.org/wiki/Cloud_computing</ref> tribe will be the availability of supercomputing application to ubiquitous computer. Computing big analytical calculations using supercomputer available to small industries or individuals using cloud will change the dynamics of process of manufacturing in small scale industries. On demand supercomputing will enable companies and enterprises to significantly decrease the development and evaluation time of their prototype and this will help them change their costing structure along with decrease in time to market a product<ref>http://arstechnica.com/information-technology/2013/11/18-hours-33k-and-156314-cores-amazon-cloud-hpc-hits-a-petaflop/</ref>.</p>
Line 16:		Line 15:

	=== Implementation and Organization ===		=== Implementation and Organization ===

	*'''Massive Parallel System:'''		*'''Massive Parallel System:'''
	<p>The need to perform complex mathematical tasks demanded by several applications of the type discussed in the preceding paragraph is beyond the capabilities of a single or a group of processing units. A large number of processing units need to cooperate and collaborate to accomplish tasks of such magnitude in a timely fashion. This concept of a '''massive parallel system''' has become an inherent feature of contemporary supercomputers.</p>		<p>The need to perform complex mathematical tasks demanded by several applications of the type discussed in the preceding paragraph is beyond the capabilities of a single or a group of processing units. A large number of processing units need to cooperate and collaborate to accomplish tasks of such magnitude in a timely fashion. This concept of a '''massive parallel system''' has become an inherent feature of contemporary supercomputers.</p>
Line 22:		Line 20:

	=== Processors ===		=== Processors ===

	*'''Processing Speed:'''		*'''Processing Speed:'''
	<p>The common feature uniting the [http://en.wikipedia.org/wiki/Human_Genome_Project Human Genome Project], the [http://home.web.cern.ch/topics/large-hadron-collider Large Hadron collider] and other such challenging scientific experiments of our era has been their demand for massive computing power. The ever accelerating growth of our knowledgebase coupled with the IT revolution has made '''processing speed''' as the most critical parameter in characterizing supercomputers. This hunger for increasing processing speed is evident from the fact that the faster supercomputer today is a thousand times faster than the fastest supercomputer a decade ago. As supercomputers find more and more applications is a wide horizon of fields such as meteorology, defense and biology; this demand for high computing power is far for subsiding.</P>		<p>The common feature uniting the [http://en.wikipedia.org/wiki/Human_Genome_Project Human Genome Project], the [http://home.web.cern.ch/topics/large-hadron-collider Large Hadron collider] and other such challenging scientific experiments of our era has been their demand for massive computing power. The ever accelerating growth of our knowledgebase coupled with the IT revolution has made '''processing speed''' as the most critical parameter in characterizing supercomputers. This hunger for increasing processing speed is evident from the fact that the faster supercomputer today is a thousand times faster than the fastest supercomputer a decade ago. As supercomputers find more and more applications is a wide horizon of fields such as meteorology, defense and biology; this demand for high computing power is far for subsiding.</P>

	=== Design Considerations ===		=== Design Considerations ===

	*'''Power consumption:'''		*'''Power consumption:'''
	<p>The system comprising of hundreds and thousands of processing elements entails a huge demand of electrical power. As a result '''power consumption''' becomes an important factor in designing supercomputers with performance per watt being a critical metric. With the rapid increase in the usage of such systems, strides are being made towards designing more power efficient systems<ref>http://asmarterplanet.com/blog/2010/07/energy-efficiency-key-to-supercomputing-future.html</ref>. Along with the list of top 500 supercomputers, the top500 organization also publishes a [http://www.green500.org/ Green500] list which ranks the systems with their power efficiency measured in [http://www.webopedia.com/TERM/F/FLOPS.html FLOPS] per watt.</p>		<p>The system comprising of hundreds and thousands of processing elements entails a huge demand of electrical power. As a result '''power consumption''' becomes an important factor in designing supercomputers with performance per watt being a critical metric. With the rapid increase in the usage of such systems, strides are being made towards designing more power efficient systems<ref>http://asmarterplanet.com/blog/2010/07/energy-efficiency-key-to-supercomputing-future.html</ref>. Along with the list of top 500 supercomputers, the top500 organization also publishes a [http://www.green500.org/ Green500] list which ranks the systems with their power efficiency measured in [http://www.webopedia.com/TERM/F/FLOPS.html FLOPS] per watt.</p>
Line 38:		Line 34:

	== Performance evaluation ==		== Performance evaluation ==
			<p>As supercomputers become more complex and powerful, measuring the performance of a particular supercomputer by just observing the specifications is rather tricky and is most likely to produce erroneous results. Benchmarks are dedicated programs which compare the characteristics of different supercomputers and are designed to mimic a particular type of workload on a component or system. Benchmarks provide a uniform framework to assess different characteristics of computer hardware such as floating point operations performance of a [http://www.webopedia.com/TERM/C/CPU.html CPU].</p>

	<p>As supercomputers become more complex and powerful, measuring the performance of a particular supercomputer by just observing the specifications is rather tricky and is most likely to produce erroneous results. Benchmarks are dedicated programs which compare the characteristics of different supercomputers and are designed to mimic a particular type of workload on a component or system. Benchmarks provide a uniform framework to assess different characteristics of computer hardware such as floating point operations performance of a [http://www.webopedia.com/TERM/C/CPU.html CPU].</p>		=== Benchmarks ===

			==== LINPACK ====
	<p>Amongst the various types of the benchmarks available in the market, kernel types such as [http://www.netlib.org/linpack/ LINPACK] are specially designed to check the performance of the supercomputers. [http://www.netlib.org/linpack/ LINPACK] benchmark is a simple program that factors and solves a large dense system of linear equations using Gaussian elimination with partial pivoting. Supercomputers are compared using [http://www.webopedia.com/TERM/F/FLOPS.html FLOPS]– floating point operations per second. In addition, a software package called LINPACK is a standard approach to testing or benchmarking supercomputers by solving a dense system of linear equations using the Gauss method. However, LINPACK benchmarking software is not only used to benchmark supercomputers, it can also be used to benchmark a typical user computer.</p>		<p>Amongst the various types of the benchmarks available in the market, kernel types such as [http://www.netlib.org/linpack/ LINPACK] are specially designed to check the performance of the supercomputers. [http://www.netlib.org/linpack/ LINPACK] benchmark is a simple program that factors and solves a large dense system of linear equations using Gaussian elimination with partial pivoting. Supercomputers are compared using [http://www.webopedia.com/TERM/F/FLOPS.html FLOPS]– floating point operations per second. In addition, a software package called LINPACK is a standard approach to testing or benchmarking supercomputers by solving a dense system of linear equations using the Gauss method. However, LINPACK benchmarking software is not only used to benchmark supercomputers, it can also be used to benchmark a typical user computer.</p>
	[[Image:super9.png\|thumb\|center\|upright\|800px\|<b>The 10 fastest supercomputers as of 2013</b>		[[Image:super9.png\|thumb\|center\|upright\|800px\|<b>The 10 fastest supercomputers as of 2013</b>
Line 46:		Line 44:
	<p>Certain new benchmarks introduced in November 2013 when TOP500 was updated. LINPACK is becoming obsolete as it measures the speed and efficiency of the linear equations calculations and fails when it comes to evaluating computations which are nonlinear in nature. A majority of differential equation calculations also require high bandwidth and low latency and access of data using irregular patterns. As a consequence, the founder of LINPACK has introduced new benchmark called high performance conjugate gradient (HPCG)<ref>http://www.sandia.gov/~maherou/docs/HPCG-Benchmark.pdf</ref> which is related to data access patterns and computations which relate closely to contemporary applications. Transcending to this new benchmark will help in rating computers and guiding their implementations in a direction which will better impact the performance improvement for real application rather than a blind race towards achieving the top spot on top500.</p>		<p>Certain new benchmarks introduced in November 2013 when TOP500 was updated. LINPACK is becoming obsolete as it measures the speed and efficiency of the linear equations calculations and fails when it comes to evaluating computations which are nonlinear in nature. A majority of differential equation calculations also require high bandwidth and low latency and access of data using irregular patterns. As a consequence, the founder of LINPACK has introduced new benchmark called high performance conjugate gradient (HPCG)<ref>http://www.sandia.gov/~maherou/docs/HPCG-Benchmark.pdf</ref> which is related to data access patterns and computations which relate closely to contemporary applications. Transcending to this new benchmark will help in rating computers and guiding their implementations in a direction which will better impact the performance improvement for real application rather than a blind race towards achieving the top spot on top500.</p>

	=== ~~Benchmarks~~ ===		==== High Performance Conjugate Gradients (HPCG) ====
			<p>In efforts to address supercomputing performance for practical environments, the HPCG benchmark was developed so that computational and data access patterns were designed to match realistically with functions of commonly used applications seen in the high performance computing industry. Instead of looking at measurable performance of scalability of an application, the shift in the HPCG benchmark was to measure performance of a given application collectivity in a scaled environment. </p>

			<p>A key design limitation seen with the LINPACK was that it was incapable of measuring irregular data access patterns which is a common element in graph theory and accounts for much of the workload processed by a supercomputer. HPCG uses an algorithm composing of many irregular data access operations and many levels of recursion that are used in high performance computing environments. </p>

			==== STREAM ====
			<p>Unlike the LINPACK benchmark suite which assess processing performance, the STREAM benchmark measures the sustainable memory bandwidth and computation rate of simple vector kernels. STREAM uses large datasets that exceed the size of a system’s immediate memory caches which is especially prevalent in applications using large-scale vector arrays. With CPU processors able to process data at quicker speeds, computational performance is less of a concern in the STREAM benchmark as memory latency can manifest in CPU idle times.</p>

			<p>STREAM tests the memory bandwidth by using COPY, SCALE, SUM, and TRIAD operations ordered by its computational complexity. This scale of complexity of each operation can be used to mimic real world applications when the data is stored in a location that is not quickly accessible. Memory bandwidth performance is measured in GB/s which can then further be extrapolated by the performance per CPU core.</p>

			==== Graph500 ====
			<p>The Graph500 organization proposes an alternative evaluation of supercomputing performance to the standard Top500 by introducing a new metric identified as traversed edges per second (TEPs). The justification on the Graph500 benchmark is to address data intensive applications designed for 3D simulations which is a commonly recognized pitfall of the LINPACK algorithm.</p>

			<p>The Graph500 algorithm starts by generating an edge list for a large vector NxM graph predefined by the number of vertices and the edge factor (ratio of the edges to vertices). Using the edge list is generated, a list of tuples each containing an endpoint and an edge weight is assigned to construct a graph matrix. Using a random generated search key, the benchmark measures the time it takes to access data stored at a given vertex in TEPs by taking the number of traversed edges divided by the access time.</p>


	== Supercomputer Architecture ==		== Supercomputer Architecture ==

	The architecture of parallel systems<ref>https://computing.llnl.gov/tutorials/parallel_comp/#Whatis</ref> can be classified on the basis of the hardware level at which they support parallelism. This is usually determined by the manner in which the processing elements are connected with each other, the level at which they communicate the memory resources they share.		The architecture of parallel systems<ref>https://computing.llnl.gov/tutorials/parallel_comp/#Whatis</ref> can be classified on the basis of the hardware level at which they support parallelism. This is usually determined by the manner in which the processing elements are connected with each other, the level at which they communicate the memory resources they share.

Line 75:		Line 82:

	=== Cluster Computing ===		=== Cluster Computing ===

	~~<p></p>~~
	[[Image:3a.png\|thumb\|right\|center\|upright\|600px\|<b>A Generic Cluster System</b>		[[Image:3a.png\|thumb\|right\|center\|upright\|600px\|<b>A Generic Cluster System</b>
	http://en.wikipedia.org/wiki/File:Beowulf.png]]		http://en.wikipedia.org/wiki/File:Beowulf.png]]
Line 96:		Line 101:

	==== Other Cluster Types ====		==== Other Cluster Types ====

	<p>As mention in the previous section, a cluster is parallel computer system consisting of independent nodes each of which is capable of individual operation.</p>		<p>As mention in the previous section, a cluster is parallel computer system consisting of independent nodes each of which is capable of individual operation.</p>

Line 106:		Line 110:

	== Summary ==		== Summary ==

	<p>The supercomputer landscape of today is a very heterogeneous mix of varying architectures and infrastructures and this had made segregation of supercomputer into well defined subgroups rather arbitrary; there seem a lot many overlapping or hybrid examples. Supercomputing has found uses in a wide horizon of domains and their potential uses continue to increase at an exciting rate. High performance computing has made large scale simulations and calculations possible which has led to successful execution of some of humanities’ most ambitious projects and this has had profound impact on our well being.</p>		<p>The supercomputer landscape of today is a very heterogeneous mix of varying architectures and infrastructures and this had made segregation of supercomputer into well defined subgroups rather arbitrary; there seem a lot many overlapping or hybrid examples. Supercomputing has found uses in a wide horizon of domains and their potential uses continue to increase at an exciting rate. High performance computing has made large scale simulations and calculations possible which has led to successful execution of some of humanities’ most ambitious projects and this has had profound impact on our well being.</p>

← Older revision		Revision as of 02:04, 28 January 2015
Line 10:		Line 10:
	<p>Supercomputing capabilities allows scientists to study simulations based on statistical modelling of large ordered data sets on phenomena that would otherwise be impossible to dissect in real time. For instance, the [http://en.wikipedia.org/wiki/Human_Brain_Project Human Brain Project] is a research project aimed to simulate brain activity which would foster research into neurology and brain development [7]. According to some sources, Japan's K computer (ranked 4th on Top500's September 2014 listing, 10.5 petaflops/s) took 40 minutes to replicate only a second worth of brain activity<ref>http://www.livescience.com/42561-supercomputer-models-brain-activity.html</ref></p>		<p>Supercomputing capabilities allows scientists to study simulations based on statistical modelling of large ordered data sets on phenomena that would otherwise be impossible to dissect in real time. For instance, the [http://en.wikipedia.org/wiki/Human_Brain_Project Human Brain Project] is a research project aimed to simulate brain activity which would foster research into neurology and brain development [7]. According to some sources, Japan's K computer (ranked 4th on Top500's September 2014 listing, 10.5 petaflops/s) took 40 minutes to replicate only a second worth of brain activity<ref>http://www.livescience.com/42561-supercomputer-models-brain-activity.html</ref></p>

	<p>Supercomputing is also prominent in weather forecasting with meteorological models becoming increasingly data intensive. A typical workload is composed of data assimilation, deterministic forecast models and ensemble forecast models. In addition, specialized models may be applied for areas such as extreme events, air quality, ocean waves and road conditions. Quicker and more accurate responses to sudden fluctuations in climate changes are made possible by advents in the supercomputing industry.<ref><http://www.cray.com/blog/the-critical-role-of-supercomputers-in-weather-forecasting</ref></p>		<p>Supercomputing is also prominent in weather forecasting with meteorological models becoming increasingly data intensive<ref><http://www.cray.com/blog/the-critical-role-of-supercomputers-in-weather-forecasting</ref>. A typical workload is composed of data assimilation, deterministic forecast models and ensemble forecast models. In addition, specialized models may be applied for areas such as extreme events, air quality, ocean waves and road conditions. Quicker and more accurate responses to sudden fluctuations in climate changes are made possible by advents in the supercomputing industry.<ref><http://www.cray.com/blog/the-critical-role-of-supercomputers-in-weather-forecasting</ref></p>

	=== On demand supercomputing and cloud ===		=== On demand supercomputing and cloud ===
Line 55:		Line 55:

	==== High Performance Conjugate Gradients (HPCG) ====		==== High Performance Conjugate Gradients (HPCG) ====
	<p>In efforts to address supercomputing performance for practical environments, the HPCG benchmark was developed so that computational and data access patterns were designed to match realistically with functions of commonly used applications seen in the high performance computing industry. Instead of looking at measurable performance of scalability of an application, the shift in the HPCG benchmark was to measure performance of a given application collectivity in a scaled environment. </p>		<p>In efforts to address supercomputing performance for practical environments, the HPCG benchmark was developed so that computational and data access patterns were designed to match realistically with functions of commonly used applications seen in the high performance computing industry. Instead of looking at measurable performance of scalability of an application, the shift in the HPCG benchmark was to measure performance of a given application collectivity in a scaled environment.<ref>http://hpcg-benchmark.org</ref> </p>

	<p>A key design limitation seen with the LINPACK was that it was incapable of measuring irregular data access patterns which is a common element in graph theory and accounts for much of the workload processed by a supercomputer. HPCG uses an algorithm composing of many irregular data access operations and many levels of recursion that are used in high performance computing environments. </p>		<p>A key design limitation seen with the LINPACK was that it was incapable of measuring irregular data access patterns which is a common element in graph theory and accounts for much of the workload processed by a supercomputer. HPCG uses an algorithm composing of many irregular data access operations and many levels of recursion that are used in high performance computing environments. </p>

	==== STREAM ====		==== STREAM ====
	<p>Unlike the LINPACK benchmark suite which assess processing performance, the STREAM benchmark measures the sustainable memory bandwidth and computation rate of simple vector kernels. STREAM uses large datasets that exceed the size of a system’s immediate memory caches which is especially prevalent in applications using large-scale vector arrays. With CPU processors able to process data at quicker speeds, computational performance is less of a concern in the STREAM benchmark as memory latency can manifest in CPU idle times.</p>		<p>Unlike the LINPACK benchmark suite which assess processing performance, the STREAM benchmark measures the sustainable memory bandwidth and computation rate of simple vector kernels. STREAM uses large datasets that exceed the size of a system’s immediate memory caches which is especially prevalent in applications using large-scale vector arrays<ref>http://www.cs.virginia.edu/stream</ref>. With CPU processors able to process data at quicker speeds, computational performance is less of a concern in the STREAM benchmark as memory latency can manifest in CPU idle times.</p>

	<p>STREAM tests the memory bandwidth by using COPY, SCALE, SUM, and TRIAD operations ordered by its computational complexity. This scale of complexity of each operation can be used to mimic real world applications when the data is stored in a location that is not quickly accessible. Memory bandwidth performance is measured in GB/s which can then further be extrapolated by the performance per CPU core.</p>		<p>STREAM tests the memory bandwidth by using COPY, SCALE, SUM, and TRIAD operations ordered by its computational complexity. This scale of complexity of each operation can be used to mimic real world applications when the data is stored in a location that is not quickly accessible. Memory bandwidth performance is measured in GB/s which can then further be extrapolated by the performance per CPU core.<ref>http://www.admin-magazine.com/HPC/Articles/Finding-Memory-Bottlenecks-with-Stream</ref></p>

	==== Graph500 ====		==== Graph500 ====
	<p>The Graph500 organization proposes an alternative evaluation of supercomputing performance to the standard Top500 by introducing a new metric identified as traversed edges per second (TEPs). The justification on the Graph500 benchmark is to address data intensive applications designed for 3D simulations which is a commonly recognized pitfall of the LINPACK algorithm.</p>		<p>The Graph500 organization proposes an alternative evaluation of supercomputing performance to the standard Top500 by introducing a new metric identified as traversed edges per second (TEPs)<ref>http://www.graph500.org</ref>. The justification on the Graph500 benchmark is to address data intensive applications designed for 3D simulations which is a commonly recognized pitfall of the LINPACK algorithm.</p>

	<p>The Graph500 algorithm starts by generating an edge list for a large vector NxM graph predefined by the number of vertices and the edge factor (ratio of the edges to vertices). Using the edge list is generated, a list of tuples each containing an endpoint and an edge weight is assigned to construct a graph matrix. Using a random generated search key, the benchmark measures the time it takes to access data stored at a given vertex in TEPs by taking the number of traversed edges divided by the access time.</p>		<p>The Graph500 algorithm starts by generating an edge list for a large vector NxM graph predefined by the number of vertices and the edge factor (ratio of the edges to vertices). Using the edge list is generated, a list of tuples each containing an endpoint and an edge weight is assigned to construct a graph matrix. Using a random generated search key, the benchmark measures the time it takes to access data stored at a given vertex in TEPs by taking the number of traversed edges divided by the access time.</p>