60
4 Advanced Neural Networks
one of the origins of the recent deep learning boom, it is often cited that a
convolutional neural network (AlexNet [39] 3 ) in an image recognition competition
using ImageNet [26] achieved a truly “mind-boggling” performance in 2012. Since
then, convolutional neural networks in image recognition have become as much the
norm as “sushi is tuna.” 4
4.1.2 Transposed Convolution
By the way, if we consider the convolution as the “interaction” between the image
x ij and the feature d I J as described above, it is unnatural to pay attention to only
the convolution operation (that produces something equivalent to d I J from the input
x ij ). Namely, it is natural to consider an operation whose input is the feature d I J and
output is the image x ij . This is considered to be just an operation by a transposed
matrix, if we look at the Hamiltonian (4.6) as a quadratic form of x and d by
appropriately changing the indices. This transposed convolution
d I J →
I J
d I J J I J,ij
(4.9)
would be used, for example, when we want to generate a cat-like image at ij by
an input of “cat-likeness.” See Fig. 4.4. Readers may have heard of the news that
a picture drawn by an artificial intelligence was sold at an auction at a high price.
This transposed convolution is a technique often used to implement neural networks
related to such image generation. 5 As an example, Fig. 4.5 shows the result of a
neural network called DCGAN (deep convolutional generative adversarial network)
[42], about an unsupervised learning of handwritten characters. 6 The neural network
is given only the input image of MNIST, and by learning its features (stopping,
hitting and flipping, etc.) well, it generates an image that mimics MNIST. We can
see how successful it is.
Checkerboard artifact
Transpose convolution is considered to be a natural method of generating highdimensional features from low-dimensional features as an inverse operation of
3 Alex is the name of the first author of this paper.
4 Speaking of tuna in the past, “pickling” was used to prevent rot, so it was common to eat lean
meat, and the toro part seemed to be worthless. However, thanks to the development of refrigeration
technology, fresh tuna can be delivered to areas far away from the sea, which has led to the spread of
toro. Similarly, if new technologies develop in machine learning, a “next-generation” architecture,
possibly beyond convolutional neural networks, may emerge. In fact, in recent years a new neural
network called Capsule Network [40] has attracted attention.
5 For readers who want to learn more, we suggest referring to [41], for example.
6 We will explain this method in detail, in Sect. 6.3.
4 Advanced Neural Networks
one of the origins of the recent deep learning boom, it is often cited that a
convolutional neural network (AlexNet [39] 3 ) in an image recognition competition
using ImageNet [26] achieved a truly “mind-boggling” performance in 2012. Since
then, convolutional neural networks in image recognition have become as much the
norm as “sushi is tuna.” 4
4.1.2 Transposed Convolution
By the way, if we consider the convolution as the “interaction” between the image
x ij and the feature d I J as described above, it is unnatural to pay attention to only
the convolution operation (that produces something equivalent to d I J from the input
x ij ). Namely, it is natural to consider an operation whose input is the feature d I J and
output is the image x ij . This is considered to be just an operation by a transposed
matrix, if we look at the Hamiltonian (4.6) as a quadratic form of x and d by
appropriately changing the indices. This transposed convolution
d I J →
I J
d I J J I J,ij
(4.9)
would be used, for example, when we want to generate a cat-like image at ij by
an input of “cat-likeness.” See Fig. 4.4. Readers may have heard of the news that
a picture drawn by an artificial intelligence was sold at an auction at a high price.
This transposed convolution is a technique often used to implement neural networks
related to such image generation. 5 As an example, Fig. 4.5 shows the result of a
neural network called DCGAN (deep convolutional generative adversarial network)
[42], about an unsupervised learning of handwritten characters. 6 The neural network
is given only the input image of MNIST, and by learning its features (stopping,
hitting and flipping, etc.) well, it generates an image that mimics MNIST. We can
see how successful it is.
Checkerboard artifact
Transpose convolution is considered to be a natural method of generating highdimensional features from low-dimensional features as an inverse operation of
3 Alex is the name of the first author of this paper.
4 Speaking of tuna in the past, “pickling” was used to prevent rot, so it was common to eat lean
meat, and the toro part seemed to be worthless. However, thanks to the development of refrigeration
technology, fresh tuna can be delivered to areas far away from the sea, which has led to the spread of
toro. Similarly, if new technologies develop in machine learning, a “next-generation” architecture,
possibly beyond convolutional neural networks, may emerge. In fact, in recent years a new neural
network called Capsule Network [40] has attracted attention.
5 For readers who want to learn more, we suggest referring to [41], for example.
6 We will explain this method in detail, in Sect. 6.3.