14 Distributed Computing
615
Supercomputers as High Core-Count Clusters The original design of supercomputers was based on a rather small number of the fastest single core processors available
at the time, interconnected with custom very low latency links. In addition, the
design usually included a much larger amount of RAM than in standard computers.
The design of supercomputers has changed completely over the last 20 years. 8 The
main driver of this change is the difficulty in building processors with ever shorter
clock cycles. The so-called “clock speed wall” has been hit, and practically all
processors operate within a narrow range around a few GHz. Hence, the only way
to make a supercomputer faster is through parallelism. Modern supercomputers are
in fact huge clusters of relatively standard multi-core processors. Low latency links
continue to be used and their hardware costs have become much lower. In addition,
“latency hiding” techniques sometimes allow the use of standard networks, such as
Ethernet. Finally, the amount of RAM per core does not differ much from standard
computers. Hence, executing an HTC workload on a supercomputer targeted for
HPC is no longer wasteful. At worst, only the low latency interconnect will be
underutilized.
Increase of Workloads with Low Input–Output Particle physics has traditionally
been a heavy user of HTC systems because they were the least expensive architecture for executing detector track reconstruction and analysis, applications with
a relatively low CPU to input–output ratio. The needs for CPU for reconstruction
and analysis have grown, however, as detectors have become much more complex.
In parallel, the precision needed in the simulations has vastly increased. These
high CPU, low input–output applications currently represent the largest computing
demand of a modern particle physics detector. Hence, the global workload profile
has moved closer to HPC in the last decades.
14.2 Functional Decomposition of a Distributed Computing
Environment
It is useful to introduce a functional decomposition, or reference framework, to
discuss distributed computing systems. The decomposition that has dominated
particle physics data processing since the 1990s, is the “SHIFT Model”, introduced
in the early 1990s by Robertson and collaborators. 9 It will be described here
with some updates of terminology. The basic assumption in the SHIFT framework
is that a Local Area Network (LAN) can be built with enough capacity and
flexibility so that the rest of the elements in the distributed environment can
8 Xie, Xianghui, et al. “Evolution of supercomputers.” Frontiers of Computer Science in China 4.4
(2010): 428–436.
9 J.P. Baud, et al. “SHIFT, the Scalable Heterogeneous Integrated Facility for HEP Computing”,
Proc. Conference on Computing in the High Energy Physics, CHEP91, Tsukuba, Japan, Universal
Academic Press.
615
Supercomputers as High Core-Count Clusters The original design of supercomputers was based on a rather small number of the fastest single core processors available
at the time, interconnected with custom very low latency links. In addition, the
design usually included a much larger amount of RAM than in standard computers.
The design of supercomputers has changed completely over the last 20 years. 8 The
main driver of this change is the difficulty in building processors with ever shorter
clock cycles. The so-called “clock speed wall” has been hit, and practically all
processors operate within a narrow range around a few GHz. Hence, the only way
to make a supercomputer faster is through parallelism. Modern supercomputers are
in fact huge clusters of relatively standard multi-core processors. Low latency links
continue to be used and their hardware costs have become much lower. In addition,
“latency hiding” techniques sometimes allow the use of standard networks, such as
Ethernet. Finally, the amount of RAM per core does not differ much from standard
computers. Hence, executing an HTC workload on a supercomputer targeted for
HPC is no longer wasteful. At worst, only the low latency interconnect will be
underutilized.
Increase of Workloads with Low Input–Output Particle physics has traditionally
been a heavy user of HTC systems because they were the least expensive architecture for executing detector track reconstruction and analysis, applications with
a relatively low CPU to input–output ratio. The needs for CPU for reconstruction
and analysis have grown, however, as detectors have become much more complex.
In parallel, the precision needed in the simulations has vastly increased. These
high CPU, low input–output applications currently represent the largest computing
demand of a modern particle physics detector. Hence, the global workload profile
has moved closer to HPC in the last decades.
14.2 Functional Decomposition of a Distributed Computing
Environment
It is useful to introduce a functional decomposition, or reference framework, to
discuss distributed computing systems. The decomposition that has dominated
particle physics data processing since the 1990s, is the “SHIFT Model”, introduced
in the early 1990s by Robertson and collaborators. 9 It will be described here
with some updates of terminology. The basic assumption in the SHIFT framework
is that a Local Area Network (LAN) can be built with enough capacity and
flexibility so that the rest of the elements in the distributed environment can
8 Xie, Xianghui, et al. “Evolution of supercomputers.” Frontiers of Computer Science in China 4.4
(2010): 428–436.
9 J.P. Baud, et al. “SHIFT, the Scalable Heterogeneous Integrated Facility for HEP Computing”,
Proc. Conference on Computing in the High Energy Physics, CHEP91, Tsukuba, Japan, Universal
Academic Press.