72
4 Advanced Neural Networks
Implementation Using Recurrent Neural Network
In the main text, we described that a recurrent neural network is Turing-complete.
This means that if we adjust the training parameters of the network well, we can
achieve something like (4.42). The paper [43] describes how to determine the weight
by hand to achieve the desired processing, while a recent deep learning model (using
a neural Turing machine (NTM) [53]) makes it possible to decide this automatically
by learning. In other words, the network is becoming able to implement the process
described above from the data itself. 12 The following shows the actual output of the
trained network when an input that is not in the training data is input. Although it did
not get it completely right because the training was not very good, it is still amazing
to see the output coming out in the correct order at the correct length (though the
values are slightly different).
[1, 1, 1, 2, 12] →
Correct answer [1, 1, 1, 2, 12]
NT M
[1, 1, 1, 1, 12]
[13, 3, 1, 2, 5, 9, 9] →
Correct answer [1, 2, 3, 5, 9, 9, 13]
NT M
[0, 3, 3, 3, 9, 13, 13]
[8, 10, 8, 3, 14, 4, 15, 5] →
Correct answer [3, 4, 5, 8, 8, 10, 14, 15]
NT M
[0, 4, 6, 10, 10, 14, 14, 14]
[5, 2, 8, 12, 0] →
Correct answer [0, 2, 5, 8, 12]
NT M
[2, 6, 6, 8, 12]
[5, 7, 12, 10, 2] →
Correct answer [2, 5, 7, 10, 12]
NT M
[2, 6, 6, 8, 12]
KdV Equation and Box-Ball System
Let us introduce another story, a physical system called a box-ball system. The boxball system is obtained by a specific discretization of the KdV equation 13 which is
obtained from the fluid Navier-Stokes equation when the waves are
• shallow waves, and
• propagating in only one direction.
12 The data is the supervised training data before and after the sorting. Here, about 800 data with a
sequence length of up to 4 are randomly generated in binary notation, and an LSTM is used as a
controller model.
13 The name comes from Korteweg and de Vries.
4 Advanced Neural Networks
Implementation Using Recurrent Neural Network
In the main text, we described that a recurrent neural network is Turing-complete.
This means that if we adjust the training parameters of the network well, we can
achieve something like (4.42). The paper [43] describes how to determine the weight
by hand to achieve the desired processing, while a recent deep learning model (using
a neural Turing machine (NTM) [53]) makes it possible to decide this automatically
by learning. In other words, the network is becoming able to implement the process
described above from the data itself. 12 The following shows the actual output of the
trained network when an input that is not in the training data is input. Although it did
not get it completely right because the training was not very good, it is still amazing
to see the output coming out in the correct order at the correct length (though the
values are slightly different).
[1, 1, 1, 2, 12] →
Correct answer [1, 1, 1, 2, 12]
NT M
[1, 1, 1, 1, 12]
[13, 3, 1, 2, 5, 9, 9] →
Correct answer [1, 2, 3, 5, 9, 9, 13]
NT M
[0, 3, 3, 3, 9, 13, 13]
[8, 10, 8, 3, 14, 4, 15, 5] →
Correct answer [3, 4, 5, 8, 8, 10, 14, 15]
NT M
[0, 4, 6, 10, 10, 14, 14, 14]
[5, 2, 8, 12, 0] →
Correct answer [0, 2, 5, 8, 12]
NT M
[2, 6, 6, 8, 12]
[5, 7, 12, 10, 2] →
Correct answer [2, 5, 7, 10, 12]
NT M
[2, 6, 6, 8, 12]
KdV Equation and Box-Ball System
Let us introduce another story, a physical system called a box-ball system. The boxball system is obtained by a specific discretization of the KdV equation 13 which is
obtained from the fluid Navier-Stokes equation when the waves are
• shallow waves, and
• propagating in only one direction.
12 The data is the supervised training data before and after the sorting. Here, about 800 data with a
sequence length of up to 4 are randomly generated in binary notation, and an LSTM is used as a
controller model.
13 The name comes from Korteweg and de Vries.