Data Structures and Workflows for ICME
37
Fig. 10 Schematic of a SIMPL data structure storage scheme in an HDF5 file
access the array in the data structure and make it available for computation when the
filter runs. Importantly, filters are also capable of performing validity checks when
processing their parameters. This validation procedure is referred to as a preflight
state. During preflight, a filter performs the necessary validation checks, modifying
the data structure as needed by proxy: memory is not allocated and computations
are not run during preflight, but any object creation or removal is represented in the
data structure.
By creating a sequence of filters, a user instantiates a workflow that creates,
modifies, and saves a SIMPL data structure. This sequence of filters is referred
to as a pipeline. Pipelines orchestrate the task of requesting filters to preflight
themselves, ensuring that the overall pipeline is in a valid state before allowing
execution. Additionally, pipelines may be serialized using the JSON file format.
JSON enables a human-readable transfer format for saving user-defined pipelines;
the constituents of a JSON pipeline file are simply the sequence of filters with their
explicit parameter settings. Note that a JSON file can be encoded as a string; this
capability is leveraged to store the pipeline within the HDF5 SIMPL file schema as
a string dataset.
SIMPL provides an interface for defining self-contained groups of filters called
plugins. Programmatically, plugins are dynamically loaded libraries that comprise
a collection of SIMPL filters, along with any additional support code necessary for
the filters’ operation. DREAM.3D itself is simply a collection of SIMPL plugins
37
Fig. 10 Schematic of a SIMPL data structure storage scheme in an HDF5 file
access the array in the data structure and make it available for computation when the
filter runs. Importantly, filters are also capable of performing validity checks when
processing their parameters. This validation procedure is referred to as a preflight
state. During preflight, a filter performs the necessary validation checks, modifying
the data structure as needed by proxy: memory is not allocated and computations
are not run during preflight, but any object creation or removal is represented in the
data structure.
By creating a sequence of filters, a user instantiates a workflow that creates,
modifies, and saves a SIMPL data structure. This sequence of filters is referred
to as a pipeline. Pipelines orchestrate the task of requesting filters to preflight
themselves, ensuring that the overall pipeline is in a valid state before allowing
execution. Additionally, pipelines may be serialized using the JSON file format.
JSON enables a human-readable transfer format for saving user-defined pipelines;
the constituents of a JSON pipeline file are simply the sequence of filters with their
explicit parameter settings. Note that a JSON file can be encoded as a string; this
capability is leveraged to store the pipeline within the HDF5 SIMPL file schema as
a string dataset.
SIMPL provides an interface for defining self-contained groups of filters called
plugins. Programmatically, plugins are dynamically loaded libraries that comprise
a collection of SIMPL filters, along with any additional support code necessary for
the filters’ operation. DREAM.3D itself is simply a collection of SIMPL plugins