14 Distributed Computing
637
coherent operation of all Tier-1 sites and allows data to flow between any pairing of
Tier-1 and Tier-2 centers. The actual situation is even more complex, as there can
be large size differences between Tier-1 or Tier-2 centers at different sites.
The computing models of the LHC experiments have undergone substantial
evolution in the last 10 years, influenced by the operational experience that has
been gained. The most important lesson is that reliably managing storage at sites
is a difficult task, requiring specialized skills not always available at resource
centers. In addition, the management of data and replica placement by the central
production teams of each experiment requires substantial human resources, and the
effort needed scales with the number of sites as opposed to the overall amount
of storage. Hence, a model where a limited number of large sites provide highly
reliable 10–100 Petabyte storage platforms, with the rest of the sites providing CPU
power with input–output via Remote Data Servers, is globally more manageable and
economical.
The original, static Tier hierarchy was motivated in part by the high costs and
technical difficulties of deploying Gigabit per second (Gbps) WAN networking.
However, networking in general and WAN in particular have had the fastest evolution of all computer industry technologies. In addition, Research and Education
Networks provide a wider variety of connection options, including 10 and 100 Gbps
links for specially demanding sites. This means that currently some Tier-2 sites have
equal networking capabilities to a Tier-1 site, whereas they may or may not achieve
the reliability levels of a Tier-1 site. It is convenient, then, to use these sites in a
more flexible manner through a more dynamic hierarchical model.
Hence, the WLCG model has evolved towards differentiating two types of sites:
Nucleation sites, which have high reliability and provide Petabyte-level storage,
and satellite sites, which are mostly expected to deliver CPU capacity. WAN
based input–output is accomplished through the custom xrootd or the standard http
protocols, and the storage is remotely managed by the central experiment project
teams via standard WebDAV protocols interacting with the Remote Data Servers
at nucleation sites. Sites are continuously monitored and their status as nucleation
points is defined dynamically according to their current reliability. In this model,
most of the Tier-1 sites and larger Tier-2 sites act as nucleation sites. A site with
reliability or availability problems will be temporarily removed as a nucleation site
and restored when monitoring reveals the issues have been resolved. Tier-1 sites
continue to have the differentiating feature of operating the robotic magnetic tape
libraries used for bulk-archiving of the data.
14.8 Current State-of-the-Art Technologies
The computing industry continues to evolve at a rapid pace. The decade of the 2010s
has seen the consolidation of a number of technologies and the emergence of an
important change in business models for the provision of computing services. These
are being integrated into the computing environments for particle physics.
637
coherent operation of all Tier-1 sites and allows data to flow between any pairing of
Tier-1 and Tier-2 centers. The actual situation is even more complex, as there can
be large size differences between Tier-1 or Tier-2 centers at different sites.
The computing models of the LHC experiments have undergone substantial
evolution in the last 10 years, influenced by the operational experience that has
been gained. The most important lesson is that reliably managing storage at sites
is a difficult task, requiring specialized skills not always available at resource
centers. In addition, the management of data and replica placement by the central
production teams of each experiment requires substantial human resources, and the
effort needed scales with the number of sites as opposed to the overall amount
of storage. Hence, a model where a limited number of large sites provide highly
reliable 10–100 Petabyte storage platforms, with the rest of the sites providing CPU
power with input–output via Remote Data Servers, is globally more manageable and
economical.
The original, static Tier hierarchy was motivated in part by the high costs and
technical difficulties of deploying Gigabit per second (Gbps) WAN networking.
However, networking in general and WAN in particular have had the fastest evolution of all computer industry technologies. In addition, Research and Education
Networks provide a wider variety of connection options, including 10 and 100 Gbps
links for specially demanding sites. This means that currently some Tier-2 sites have
equal networking capabilities to a Tier-1 site, whereas they may or may not achieve
the reliability levels of a Tier-1 site. It is convenient, then, to use these sites in a
more flexible manner through a more dynamic hierarchical model.
Hence, the WLCG model has evolved towards differentiating two types of sites:
Nucleation sites, which have high reliability and provide Petabyte-level storage,
and satellite sites, which are mostly expected to deliver CPU capacity. WAN
based input–output is accomplished through the custom xrootd or the standard http
protocols, and the storage is remotely managed by the central experiment project
teams via standard WebDAV protocols interacting with the Remote Data Servers
at nucleation sites. Sites are continuously monitored and their status as nucleation
points is defined dynamically according to their current reliability. In this model,
most of the Tier-1 sites and larger Tier-2 sites act as nucleation sites. A site with
reliability or availability problems will be temporarily removed as a nucleation site
and restored when monitoring reveals the issues have been resolved. Tier-1 sites
continue to have the differentiating feature of operating the robotic magnetic tape
libraries used for bulk-archiving of the data.
14.8 Current State-of-the-Art Technologies
The computing industry continues to evolve at a rapid pace. The decade of the 2010s
has seen the consolidation of a number of technologies and the emergence of an
important change in business models for the provision of computing services. These
are being integrated into the computing environments for particle physics.