0.5. In the test, the output of all neurons was multiplied by only 0.5, So every time the
neural network is trying different structures, but the weights are shared between these
structures. Dropout greatly reduces overfitting and roughly doubles the number of
iterations required for convergence. Architecture of ALexNet runs in parallel on two
GPUs as shown in Fig. 2.
The neural network is considered to be a black-box operation. It is difficult to
understand its internal operating mechanism. If it cannot be solved, the improvement of
its network architecture can only rely on trial. ZF Net invented a convolutional network
visualization technology called deconvolutional network that allows researchers to
know what the convolutional layer is observing and what it is doing in the CNN. The
basic principle is that after each layer of the trained CNN network, a deconvnet layer
with the same filter settings as the original CNN network is attached, and the basic
working principle of deconvolution is shown in Fig. 3. If we want to check the activation value of a feature in a layer, save the activation value of the corresponding
feature layer of this layer, and set the other activation values in this layer to zero, then
the feature layer is used as the input of the deconvolution network [3]. One of the most
interesting places in deep learning is that the first layer responds to simple edges, the
next layer responds to textures, and the later layers respond to more complex object
parts, if stacked multiple layers, as the layers are deepened, the extracted information
becomes more complex and abstract. ZF Net provides CNN with a more intuitive
display capability, enabling researchers to better understand the internal workings of
CNN and provide guidance for further improving the CNN model. In addition, compared to AlexNet’s 11 Ã 11 filter, ZF Net uses a smaller 7 Ã 7 filter with smaller steps
size, which helps preserve the raw pixel information of the input data.
Fig. 2. Illustration of the architecture of ALexNet run in parallel on two GPUs
Research on System Architecture Based on Deep Learning …
517
neural network is trying different structures, but the weights are shared between these
structures. Dropout greatly reduces overfitting and roughly doubles the number of
iterations required for convergence. Architecture of ALexNet runs in parallel on two
GPUs as shown in Fig. 2.
The neural network is considered to be a black-box operation. It is difficult to
understand its internal operating mechanism. If it cannot be solved, the improvement of
its network architecture can only rely on trial. ZF Net invented a convolutional network
visualization technology called deconvolutional network that allows researchers to
know what the convolutional layer is observing and what it is doing in the CNN. The
basic principle is that after each layer of the trained CNN network, a deconvnet layer
with the same filter settings as the original CNN network is attached, and the basic
working principle of deconvolution is shown in Fig. 3. If we want to check the activation value of a feature in a layer, save the activation value of the corresponding
feature layer of this layer, and set the other activation values in this layer to zero, then
the feature layer is used as the input of the deconvolution network [3]. One of the most
interesting places in deep learning is that the first layer responds to simple edges, the
next layer responds to textures, and the later layers respond to more complex object
parts, if stacked multiple layers, as the layers are deepened, the extracted information
becomes more complex and abstract. ZF Net provides CNN with a more intuitive
display capability, enabling researchers to better understand the internal workings of
CNN and provide guidance for further improving the CNN model. In addition, compared to AlexNet’s 11 Ã 11 filter, ZF Net uses a smaller 7 Ã 7 filter with smaller steps
size, which helps preserve the raw pixel information of the input data.
Fig. 2. Illustration of the architecture of ALexNet run in parallel on two GPUs
Research on System Architecture Based on Deep Learning …
517