630
M. Delfino
one-to-one relationship to raw data files, which in turn may have little to do with
physics analysis. Furthermore, it usually has output for all raw data records, which
depending on detector triggering and physics signal-to-noise conditions may be very
large compared to the optimal for physics analysis.
These considerations lead to the definition of “reduced” or “analysis” datasets
along two complementary lines. Data reduction takes the reconstruction output and
generates data structures optimized for analysis by removing information related
to reprocessing and “turning around” data structures related to tracks, vertices,
calorimeter clusters, etc. to be optimal for analysis. Particle identification by
correlation between different detector elements is often done (or re-done) during
data reduction and, when combined with Event Filtering, may be optimized for
particular physics processes (heavy quark identification, rejection of electrons, etc.).
The historical name for the output of data reduction is “Data Summary Tape”
or “DST” and successively smaller outputs have been called “mini”, “micro” or
even “nano” DSTs. 63 A nano-DST may simply consist of a list of “reconstructed
particles”, each stored with a few properties such as particle ID probabilities and
momentum components. A nano-DST is in some sense the experimental counterpart
to the event generator output.
DST files often do not mirror the raw and reconstructed file organization scheme.
At a minimum, DST files contain the DST records that correspond to many raw
files, in order to ease the management of large numbers of datasets and avoid
handling very small files. In accelerator experiments, DSTs may be organized by
running conditions. In fixed-target experiments where on-beam and off-beam data
are alternatively taken, separate DSTs for each may be produced. In astroparticle
experiments, which often behave like telescopes, DSTs will often be organized by
observed source, merging data from many observation days or nights. These reorganizations of data are special cases of event filtering.
Event filtering 64 may be performed before, after or during production of any
DST-like output. The purpose is to group together in particular files the DST output
most relevant for a particular analysis, thereby avoiding repetitive reading of the
rest of the data by analysis programs. Event filtering can be done according to
many criteria, but is most often performed as a pre-selection for particular physics
analyses, especially in situations of unfavorable signal-to-noise ratio such as the
LHC experiments. Output datasets are often referred to by the name of the physics
process pre-selected and are written in an identical format as the original DST-like
datasets.
63 The names of Data Reduction output datasets are not standardized. Therefore readers will have
to identify the corresponding dataset names for their respective project.
64 Event Filtering in this context should not be confused with the actions taken by high-level trigger
“Event Filters”.
M. Delfino
one-to-one relationship to raw data files, which in turn may have little to do with
physics analysis. Furthermore, it usually has output for all raw data records, which
depending on detector triggering and physics signal-to-noise conditions may be very
large compared to the optimal for physics analysis.
These considerations lead to the definition of “reduced” or “analysis” datasets
along two complementary lines. Data reduction takes the reconstruction output and
generates data structures optimized for analysis by removing information related
to reprocessing and “turning around” data structures related to tracks, vertices,
calorimeter clusters, etc. to be optimal for analysis. Particle identification by
correlation between different detector elements is often done (or re-done) during
data reduction and, when combined with Event Filtering, may be optimized for
particular physics processes (heavy quark identification, rejection of electrons, etc.).
The historical name for the output of data reduction is “Data Summary Tape”
or “DST” and successively smaller outputs have been called “mini”, “micro” or
even “nano” DSTs. 63 A nano-DST may simply consist of a list of “reconstructed
particles”, each stored with a few properties such as particle ID probabilities and
momentum components. A nano-DST is in some sense the experimental counterpart
to the event generator output.
DST files often do not mirror the raw and reconstructed file organization scheme.
At a minimum, DST files contain the DST records that correspond to many raw
files, in order to ease the management of large numbers of datasets and avoid
handling very small files. In accelerator experiments, DSTs may be organized by
running conditions. In fixed-target experiments where on-beam and off-beam data
are alternatively taken, separate DSTs for each may be produced. In astroparticle
experiments, which often behave like telescopes, DSTs will often be organized by
observed source, merging data from many observation days or nights. These reorganizations of data are special cases of event filtering.
Event filtering 64 may be performed before, after or during production of any
DST-like output. The purpose is to group together in particular files the DST output
most relevant for a particular analysis, thereby avoiding repetitive reading of the
rest of the data by analysis programs. Event filtering can be done according to
many criteria, but is most often performed as a pre-selection for particular physics
analyses, especially in situations of unfavorable signal-to-noise ratio such as the
LHC experiments. Output datasets are often referred to by the name of the physics
process pre-selected and are written in an identical format as the original DST-like
datasets.
63 The names of Data Reduction output datasets are not standardized. Therefore readers will have
to identify the corresponding dataset names for their respective project.
64 Event Filtering in this context should not be confused with the actions taken by high-level trigger
“Event Filters”.