problem of excessive memory storage occupied by multiple cyclic scans will not occur,
and the encoding time and storage space occupation rate will be reduced.
AlexNet selects Re as CNN’s excitation function to solve the problem of increasing
gradient dispersion in network depth [3]. In the last several complete connection layers,
some neurons were randomly neglected to avoid overfitting of the model. Max pool
eliminates the blur effect of average pool. The step size is smaller than the size of the
pool core, but the high-level features are not easy to be classified. Based on the finetuning of AlexNet, ZFNet changes the first layer of AlexNet: The size of the filter is
changed from 11 Â 11 to 7 Â 7, and the step size is changed from 4 to 2, retaining
more features. Through deconvolution, the feature map reflects the architecture of
feature hierarchy. The first layer learns specific physical features, such as texture and
edge, while the second layer learns and identifies abstract features related to types.
Thus, the higher the features, the better the classification performance; however, the
higher up the hierarchy, the longer the training is needed before convergence. VGGNet
further studied the relationship between the depth and performance of the convolutional
neural network [4], using a 3 Â 3 small convolution kernel and a 2 Â 2 maximum
pooling layer, and repeated stacking. As the network deepens, the pool shrinks by half
and the number of channels doubles. While simplifying the structure of convolutional
neural network, a large number of training features are retained.
Generally, the idea of establishing an ideal optimal model is to increase the depth of
the model or the number of neurons. But there are some drawbacks:
• If the training data set is limited and the parameters are too many, overfitting is
likely to occur.
• The larger the network model structure is, the more complex the calculation will be,
making it difficult to carry out practical application.
• The deeper the network model structure is, the more easily the gradient is lost in the
deep layer, and the optimization model is difficult to be optimized.
• Degradation: Accuracy first increases with depth, and when saturation is reached,
the increase of model depth leads to the decrease of accuracy.
In order to solve these problems, we believe that non-uniform sparse data computer
software and hardware are inefficient. ResNet assumes that the learning target is
converted into the difference value F(x) = H(x)−x between the expected output H(x)
and the direct output x, and the initial result is defined as the input x [5]. The process is
shown in Fig. 1.
Fig. 1. Residual unit
314
T. Tang et al.
and the encoding time and storage space occupation rate will be reduced.
AlexNet selects Re as CNN’s excitation function to solve the problem of increasing
gradient dispersion in network depth [3]. In the last several complete connection layers,
some neurons were randomly neglected to avoid overfitting of the model. Max pool
eliminates the blur effect of average pool. The step size is smaller than the size of the
pool core, but the high-level features are not easy to be classified. Based on the finetuning of AlexNet, ZFNet changes the first layer of AlexNet: The size of the filter is
changed from 11 Â 11 to 7 Â 7, and the step size is changed from 4 to 2, retaining
more features. Through deconvolution, the feature map reflects the architecture of
feature hierarchy. The first layer learns specific physical features, such as texture and
edge, while the second layer learns and identifies abstract features related to types.
Thus, the higher the features, the better the classification performance; however, the
higher up the hierarchy, the longer the training is needed before convergence. VGGNet
further studied the relationship between the depth and performance of the convolutional
neural network [4], using a 3 Â 3 small convolution kernel and a 2 Â 2 maximum
pooling layer, and repeated stacking. As the network deepens, the pool shrinks by half
and the number of channels doubles. While simplifying the structure of convolutional
neural network, a large number of training features are retained.
Generally, the idea of establishing an ideal optimal model is to increase the depth of
the model or the number of neurons. But there are some drawbacks:
• If the training data set is limited and the parameters are too many, overfitting is
likely to occur.
• The larger the network model structure is, the more complex the calculation will be,
making it difficult to carry out practical application.
• The deeper the network model structure is, the more easily the gradient is lost in the
deep layer, and the optimization model is difficult to be optimized.
• Degradation: Accuracy first increases with depth, and when saturation is reached,
the increase of model depth leads to the decrease of accuracy.
In order to solve these problems, we believe that non-uniform sparse data computer
software and hardware are inefficient. ResNet assumes that the learning target is
converted into the difference value F(x) = H(x)−x between the expected output H(x)
and the direct output x, and the initial result is defined as the input x [5]. The process is
shown in Fig. 1.
Fig. 1. Residual unit
314
T. Tang et al.