64
S. A. Khan et al.
X =
K
k=1
z k ◦ w k ◦ u k +
where Z and U and W are the latent variables corresponding to the three modes.
Several implementations of CP factorization have existed for quite some time now,
for example, the seminal implementation by Andersson and Bro [42]. Recently, CP
and other factorizations have gained substantial interest among the machine learning
community [43, 44], since recent developments addressed several methodological
challenges posed by multi-way data sets. More recently, an easy to use probabilistic
implementation of CP was presented by Khan and Ammad-ud-din [45]. The implementation automatically handles missing values in the data, hence making it applicable to a wide selection of real-world data sets. It also features automatic component
selection as well as visualization and prediction routines making both exploratory
and predictive analytics easier.
4.2.5 Multi-tensor Factorization
Multi-tensor factorization (MTF [40, 41]) is a new machine learning method designed
to capture relationships between a collection of tensor data sets. MTF jointly factorizes multiple tensors to learn a joint low-dimensional representation that models the
statistical dependencies between the tensors. Interestingly, MTF considers matrices
as tensors of order two, thus enabling joint factorization of both matrices and tensors.
This characteristic makes it possible to analyze novel data sets composed of matrices
as well as tensors in a single joint analysis.
MTF is designed to factorize multiple co-occurring data sets, with the objective of
distinguishing the shared and specific components regardless of their matrix or tensor
nature. This is achieved by modeling the entire variation of all data sets through a
common Factor analysis and CP-type factorization having two keys features. First,
the factorization is characterized by latent variables Z that are common between
all the views (tensor and matrices). This allows the factorization to capture crossdependencies regardless of the data view. Second, the loadings W controls which of
the patterns in Z are active in each of the views. Learning these W loadings makes
it possible to identify the dependency patterns in a truly data-driven fashion without
any prior information on dependency patterns.
Formally, for multiple paired tensors X
(t)
∈ R
N ×D t ×L , where t = 1…T, we
specify a joint model of matrices and tensor. An indicator variable β t identifies the
tensors (β t = 1) and matrices (β t = 2), MTF is formulated using normal distributions
and conjugate priors as:
x
(t)
n,d t ,l ∼ N
z n,k .w d t ,k .u l,k , (τ
(t)
)
−1
Z, U
(t)
∼ N (0, I)
S. A. Khan et al.
X =
K
k=1
z k ◦ w k ◦ u k +
where Z and U and W are the latent variables corresponding to the three modes.
Several implementations of CP factorization have existed for quite some time now,
for example, the seminal implementation by Andersson and Bro [42]. Recently, CP
and other factorizations have gained substantial interest among the machine learning
community [43, 44], since recent developments addressed several methodological
challenges posed by multi-way data sets. More recently, an easy to use probabilistic
implementation of CP was presented by Khan and Ammad-ud-din [45]. The implementation automatically handles missing values in the data, hence making it applicable to a wide selection of real-world data sets. It also features automatic component
selection as well as visualization and prediction routines making both exploratory
and predictive analytics easier.
4.2.5 Multi-tensor Factorization
Multi-tensor factorization (MTF [40, 41]) is a new machine learning method designed
to capture relationships between a collection of tensor data sets. MTF jointly factorizes multiple tensors to learn a joint low-dimensional representation that models the
statistical dependencies between the tensors. Interestingly, MTF considers matrices
as tensors of order two, thus enabling joint factorization of both matrices and tensors.
This characteristic makes it possible to analyze novel data sets composed of matrices
as well as tensors in a single joint analysis.
MTF is designed to factorize multiple co-occurring data sets, with the objective of
distinguishing the shared and specific components regardless of their matrix or tensor
nature. This is achieved by modeling the entire variation of all data sets through a
common Factor analysis and CP-type factorization having two keys features. First,
the factorization is characterized by latent variables Z that are common between
all the views (tensor and matrices). This allows the factorization to capture crossdependencies regardless of the data view. Second, the loadings W controls which of
the patterns in Z are active in each of the views. Learning these W loadings makes
it possible to identify the dependency patterns in a truly data-driven fashion without
any prior information on dependency patterns.
Formally, for multiple paired tensors X
(t)
∈ R
N ×D t ×L , where t = 1…T, we
specify a joint model of matrices and tensor. An indicator variable β t identifies the
tensors (β t = 1) and matrices (β t = 2), MTF is formulated using normal distributions
and conjugate priors as:
x
(t)
n,d t ,l ∼ N
z n,k .w d t ,k .u l,k , (τ
(t)
)
−1
Z, U
(t)
∼ N (0, I)