14 Distributed Computing
633
come from providing simple CPU-intensive services with little input or output.
Providing reliable data-centric services has proven to be much more difficult, both
from the point of view of the operation of resource centers themselves and of the
Information and Control servers which provide the Grid interface for each center.
Nevertheless, the CERN Computer Center, the WLCG Tier-1 centers and a subset
of the WLCG Tier-2 centers do provide the needed reliable data-centric services
needed for nucleating the rest of resources (see below).
The European Grid Infrastructure (EGI) program, 69 financed in part by the EU,
has deployed the largest production Grid Infrastructure to date in collaboration with
the Worldwide LHC Computing Grid (see next section). Other Grid Infrastructures
provide support for particle and astroparticle physics, for example the Open Science
Grid (OSG) in the USA. The EGI Grid will be described as an example of a working
Grid used for daily work by particle physicists. EGI defines a Grid Infrastructure as
a set of services deployed over Internet that allows a large number of Resource
Centers, each with a different security and management domain, to be used in a
coherent fashion by a large number of users from different institutions grouped in
virtual organizations. Usually, a virtual organization corresponds to a project.
The basic services offered by the EGI Grid Infrastructure are based on the UMD
middleware distribution, 70 which is deployed in a managed way, including monitoring and fault detection, with the support of a distributed operations organization
provided by resource centers, many of which are associated to LHC computing
or national Research and Education networks. Resource centers deploy Remote
Data Servers using a variety of Disk Servers (possibly with magnetic tape back
ends) which expose a uniform interface to the Grid via UMD, whose instances
are called “Storage Elements”. Authorization and access control is accomplished
using the certificate method described earlier with tokens generated by the Virtual
Organization Membership Service (VOMS) upon presentation of X.509 certificates
issued by national research agencies and declaration of the virtual organization
(e.g. project) under which the user wishes to be authorized. The actual data flow
is predominantly through the ftp protocol, wrapped in tools such as gridftp (which
incorporates certificate authentication and multiple parallel transmission streams). 71
Each resource center implements a “Computing Element” (CE) service, a set of
Information and Control Servers which map a specific batch system to an abstract
69 See http://www.egi.eu. EGI operates the international coordination structure for Grid resource
centers across Europe. EGI also coordinates the maintenance and security support for the Grid
software originally developed in the 2000s by the EU Enabling Grids for E-sciencE (EGEE) series
of projects and the EU DataGrid project.
70 The Unified Middleware Distribution (UMD) maintained by EGI is based on software developed
in the EU European Middleware Infrastructure (EMI) project. The EMI software is the result of
integration and compatibility work done on the gLite middleware, developed within the EGEE
Project, the ARC middleware, developed by NorduGrid, and various other packages. See https://
wiki.egi.eu/wiki/Middleware
71 A more sophisticated Grid data access tool known as srm was used in the past but is largely being
abandoned due to maintenance and management issues.
633
come from providing simple CPU-intensive services with little input or output.
Providing reliable data-centric services has proven to be much more difficult, both
from the point of view of the operation of resource centers themselves and of the
Information and Control servers which provide the Grid interface for each center.
Nevertheless, the CERN Computer Center, the WLCG Tier-1 centers and a subset
of the WLCG Tier-2 centers do provide the needed reliable data-centric services
needed for nucleating the rest of resources (see below).
The European Grid Infrastructure (EGI) program, 69 financed in part by the EU,
has deployed the largest production Grid Infrastructure to date in collaboration with
the Worldwide LHC Computing Grid (see next section). Other Grid Infrastructures
provide support for particle and astroparticle physics, for example the Open Science
Grid (OSG) in the USA. The EGI Grid will be described as an example of a working
Grid used for daily work by particle physicists. EGI defines a Grid Infrastructure as
a set of services deployed over Internet that allows a large number of Resource
Centers, each with a different security and management domain, to be used in a
coherent fashion by a large number of users from different institutions grouped in
virtual organizations. Usually, a virtual organization corresponds to a project.
The basic services offered by the EGI Grid Infrastructure are based on the UMD
middleware distribution, 70 which is deployed in a managed way, including monitoring and fault detection, with the support of a distributed operations organization
provided by resource centers, many of which are associated to LHC computing
or national Research and Education networks. Resource centers deploy Remote
Data Servers using a variety of Disk Servers (possibly with magnetic tape back
ends) which expose a uniform interface to the Grid via UMD, whose instances
are called “Storage Elements”. Authorization and access control is accomplished
using the certificate method described earlier with tokens generated by the Virtual
Organization Membership Service (VOMS) upon presentation of X.509 certificates
issued by national research agencies and declaration of the virtual organization
(e.g. project) under which the user wishes to be authorized. The actual data flow
is predominantly through the ftp protocol, wrapped in tools such as gridftp (which
incorporates certificate authentication and multiple parallel transmission streams). 71
Each resource center implements a “Computing Element” (CE) service, a set of
Information and Control Servers which map a specific batch system to an abstract
69 See http://www.egi.eu. EGI operates the international coordination structure for Grid resource
centers across Europe. EGI also coordinates the maintenance and security support for the Grid
software originally developed in the 2000s by the EU Enabling Grids for E-sciencE (EGEE) series
of projects and the EU DataGrid project.
70 The Unified Middleware Distribution (UMD) maintained by EGI is based on software developed
in the EU European Middleware Infrastructure (EMI) project. The EMI software is the result of
integration and compatibility work done on the gLite middleware, developed within the EGEE
Project, the ARC middleware, developed by NorduGrid, and various other packages. See https://
wiki.egi.eu/wiki/Middleware
71 A more sophisticated Grid data access tool known as srm was used in the past but is largely being
abandoned due to maintenance and management issues.