The forward transmission process of the binary network can be divided into four steps:
first let the input pass through a Batch Normal, then binarize the input value, and binary
convolution and pooling are used. Finally output the classification through a classifier.
The weight is updated with full precision during the training process.
The forward transmission of binarized network:
x
k
¼ r BN W
k
b Á x
kÀ1
À
Á
À
Á ¼ sign BN W
k
b Á x
kÀ1
À
Á
À
Á
ð5Þ
The process of binarized model update is shown in Fig. 1.
6 Training and Discussion
6.1 Experiment Platform
The experimental environment is Ubuntu 16.04 LTS 64-bit system, using PyTorch as
the deep learning open-source framework and using python as the programming language. The computer memory is 64G, equipped with Intel Xeon(R) CPU E5-1640 v4
3.6 GHz x12 processor. The graphics card is a NVIDIA GTX1080Ti.
6.2 Parameter Selection in Experiment
The train and test dataset are divided into multiple batches in this paper; each batch has
32 images. The full-precision model uses SGD to optimize the model with a learning
rate of 0.005 and regularization coefficient of 5e−4. The learning rate become 0.1 times
of the original per 20 epoch. The binarized model uses Adam to optimize the model
with learning rate of 0.001, regularization coefficient of 1e−5. The learning rate
become 0.1 times of the original per 30 epoch.
In order to prevent over-fitting, four different proportions are set: train set: 80, 60,
30, 20%; test set: 20, 30, 60, 80%. The more the sample ratio, the smaller the influence
of experimental inherent on the results. At the same time, in order to make a fair
comparison, the hyper-parameters are standardized.
Fig. 1. Schematic diagram of binarized model training
Plant Diseases Identification Based on Binarized Neural Network
15
first let the input pass through a Batch Normal, then binarize the input value, and binary
convolution and pooling are used. Finally output the classification through a classifier.
The weight is updated with full precision during the training process.
The forward transmission of binarized network:
x
k
¼ r BN W
k
b Á x
kÀ1
À
Á
À
Á ¼ sign BN W
k
b Á x
kÀ1
À
Á
À
Á
ð5Þ
The process of binarized model update is shown in Fig. 1.
6 Training and Discussion
6.1 Experiment Platform
The experimental environment is Ubuntu 16.04 LTS 64-bit system, using PyTorch as
the deep learning open-source framework and using python as the programming language. The computer memory is 64G, equipped with Intel Xeon(R) CPU E5-1640 v4
3.6 GHz x12 processor. The graphics card is a NVIDIA GTX1080Ti.
6.2 Parameter Selection in Experiment
The train and test dataset are divided into multiple batches in this paper; each batch has
32 images. The full-precision model uses SGD to optimize the model with a learning
rate of 0.005 and regularization coefficient of 5e−4. The learning rate become 0.1 times
of the original per 20 epoch. The binarized model uses Adam to optimize the model
with learning rate of 0.001, regularization coefficient of 1e−5. The learning rate
become 0.1 times of the original per 30 epoch.
In order to prevent over-fitting, four different proportions are set: train set: 80, 60,
30, 20%; test set: 20, 30, 60, 80%. The more the sample ratio, the smaller the influence
of experimental inherent on the results. At the same time, in order to make a fair
comparison, the hyper-parameters are standardized.
Fig. 1. Schematic diagram of binarized model training
Plant Diseases Identification Based on Binarized Neural Network
15