Multi-SPMD Programming Model with YML and XcalableMP
221
3 Multi-SPMD Programming Model
3.1 Overview
While most programming models consider MPI+X such as MPI+OpenMP, or
MPI+X 1 +X 2 · · · , we consider X 1 +MPI (or XMP) +X 2 and propose a multi-SPMD
(mSPMD) programming model where MPI programs and OpenMP+MPI programs
work together in the context of a workflow programming model. In other words,
tasks in a workflow are parallel programs written in XMP, MPI, or their hybrid with
OpenMP.
Figure 1 shows the overview of the mSPMD programming model. In the target
systems we have expected, there should be non-uniform memory access (NUMA),
general-purpose many-core CPUs, and accelerators such as GPU. We employ a
shared memory programming model within a node, or a group of cores, and
GPGPU programming on an accelerator. In a group of nodes, we have considered
a distributed parallel programming model. Between these groups of nodes, there is
a workflow programming model to manage and control several distributed parallel
programs and hybrid programs of the distributed parallel and shared programming
models. To realize this framework, we support XcalableMP (XMP) to describe
the distributed parallel programs in a workflow as well as MPI, which is a defacto standard for distributed parallel programming. For the shared programming
and GPGPU, as well as XMP+OpenMP, MPI+OpenMP, MPI+GPGPU such as
CUDA, OpenACC, we support a runtime library called StarPU. The StarPU
library[1], which is a task programming library for hybrid architectures, enables
us to implement heterogeneous applications in a uniform way. XMP provides an
extension to enable work-sharing among CPU cores and GPU [7]. YML[2–4]—a
development and execution environment for a scientific workflow—is used for the
workflow execution.
Fig. 1 An overview of multi-SPMD programming model
221
3 Multi-SPMD Programming Model
3.1 Overview
While most programming models consider MPI+X such as MPI+OpenMP, or
MPI+X 1 +X 2 · · · , we consider X 1 +MPI (or XMP) +X 2 and propose a multi-SPMD
(mSPMD) programming model where MPI programs and OpenMP+MPI programs
work together in the context of a workflow programming model. In other words,
tasks in a workflow are parallel programs written in XMP, MPI, or their hybrid with
OpenMP.
Figure 1 shows the overview of the mSPMD programming model. In the target
systems we have expected, there should be non-uniform memory access (NUMA),
general-purpose many-core CPUs, and accelerators such as GPU. We employ a
shared memory programming model within a node, or a group of cores, and
GPGPU programming on an accelerator. In a group of nodes, we have considered
a distributed parallel programming model. Between these groups of nodes, there is
a workflow programming model to manage and control several distributed parallel
programs and hybrid programs of the distributed parallel and shared programming
models. To realize this framework, we support XcalableMP (XMP) to describe
the distributed parallel programs in a workflow as well as MPI, which is a defacto standard for distributed parallel programming. For the shared programming
and GPGPU, as well as XMP+OpenMP, MPI+OpenMP, MPI+GPGPU such as
CUDA, OpenACC, we support a runtime library called StarPU. The StarPU
library[1], which is a task programming library for hybrid architectures, enables
us to implement heterogeneous applications in a uniform way. XMP provides an
extension to enable work-sharing among CPU cores and GPU [7]. YML[2–4]—a
development and execution environment for a scientific workflow—is used for the
workflow execution.
Fig. 1 An overview of multi-SPMD programming model