256
X. Hao et al.
2 Approach
In this section, we present details of the proposed VAA for training the deep
CNN model. This method mainly consists of three phases. First, we train a base
network model till convergence using original samples. Afterward, we obtain the
attention map form the trained base network model and set the value of valuable
parts on the original sample to zero in order to obtain new samples. Finally, we
utilize these new samples and original samples to fine-tune the trained base
network model.
With the development of deep learning, many CNN models have been applied
to pedestrian retrieval such as CaffeNet, VGG, ResNet, and so on. We adopt
the ResNet-50 model [14] as the base network model and set the neuron number of fully connected (FC) layer to the identity number of pedestrian retrieval
database, i.e., 751 for Market-1501. The structure of the base network model is
shown in Fig. 1. During training, we treat pedestrian retrieval as a classification
task and use cross-entropy loss to fine-tune the network model. During testing,
we remove the FC layer and utilize the remaining layer to extract the feature of
pedestrian sample.
Fig. 1. Structure of the base network model.
After training the base network model, the next phase is to find valuable
parts on the pedestrian sample and set the pixel value of these parts to zero
in order to obtain new samples. Since there is no location annotation when
training the network model, we cannot directly find valuable parts of pedestrian
sample. Considering that the attention map can reflect the part of network model
concern, we can locate valuable parts of the pedestrian sample. Inspired by this,
we use the attention map to find valuable parts of the pedestrian sample. The
attention map AM n obtained at the nth layer of the network model is represented
as follows:
AM n =
i
(F M i )
2
,
(1)
where F M i represents the ith convolutional activation map in this layer. It is
worth mentioning that the operation is between elements.
X. Hao et al.
2 Approach
In this section, we present details of the proposed VAA for training the deep
CNN model. This method mainly consists of three phases. First, we train a base
network model till convergence using original samples. Afterward, we obtain the
attention map form the trained base network model and set the value of valuable
parts on the original sample to zero in order to obtain new samples. Finally, we
utilize these new samples and original samples to fine-tune the trained base
network model.
With the development of deep learning, many CNN models have been applied
to pedestrian retrieval such as CaffeNet, VGG, ResNet, and so on. We adopt
the ResNet-50 model [14] as the base network model and set the neuron number of fully connected (FC) layer to the identity number of pedestrian retrieval
database, i.e., 751 for Market-1501. The structure of the base network model is
shown in Fig. 1. During training, we treat pedestrian retrieval as a classification
task and use cross-entropy loss to fine-tune the network model. During testing,
we remove the FC layer and utilize the remaining layer to extract the feature of
pedestrian sample.
Fig. 1. Structure of the base network model.
After training the base network model, the next phase is to find valuable
parts on the pedestrian sample and set the pixel value of these parts to zero
in order to obtain new samples. Since there is no location annotation when
training the network model, we cannot directly find valuable parts of pedestrian
sample. Considering that the attention map can reflect the part of network model
concern, we can locate valuable parts of the pedestrian sample. Inspired by this,
we use the attention map to find valuable parts of the pedestrian sample. The
attention map AM n obtained at the nth layer of the network model is represented
as follows:
AM n =
i
(F M i )
2
,
(1)
where F M i represents the ith convolutional activation map in this layer. It is
worth mentioning that the operation is between elements.