using the fully convolutional network, similar to the general design of DenseBox [12].
The proposed model has two modules. The first module is used for feature extraction.
Since the hand size, shape, appearance, and illuminations have large variance, we
merge the feature maps from different scales to integrate both the global and the local
information. To reduce the computation overhead due to merging too many channels
on large feature maps, we use three successive 1 Â 1, 3 Â 3, 3 Â 3 convolutional
layers to lower the subsequent computational cost. The second module is the output
part, which includes two branches. The first branch outputs the confidence of each pixel
within the hand bounding boxes, and the second branch outputs the vertex coordinates
of the hand bounding boxes.
3.3 Loss Function
The loss function has two terms, as described in (1)
L ¼ L c þ L r :
ð1Þ
where L c evaluates the probability that a pixel locates inside the bounding box or not,
and L r evaluates the errors between the regressed position and the ground truth.
L c is defined using cross-entropy loss, given by
L c ¼ Àap
Ã
ð1 À pÞ
c log p À ð1 À aÞð1 À p
Ã
Þp
c logð1 À pÞ:
ð2Þ
where p
* represents the true class of a pixel, p represents the predicted probability of a
pixel inside the hand bounding box, a ¼ 1 À
Nðp
à ¼1Þ
Nðp Ã Þ is used to balance the impacts of
the positive and negative samples, and c is the adjustable parameter. In our experiments, the best choice of c is c ¼ 2.
L r is defined using smoothed-L1 loss, given by
L r ¼ smoothed L1 C i À C
Ã
i
À
Á
ð3Þ
smoothed L1 ðxÞ ¼
0:5x
2
jxj\1
jxj À 0:5 others
&
:
ð4Þ
where C i and C
Ã
i represent regressed vertex coordinates of the hand bounding box and
the ground truth, respectively.
3.4 Implementations
The size of input images is 128 Â 128. The network is trained end to end using Adam
optimization algorithm with minibatch size 24. The initial learning rate is 10
−3
, decays
10 times every 20,000 minibatches, and stops at 10
−5 .
Hand Detection Based on Multi-scale Fully Convolutional Networks
23
The proposed model has two modules. The first module is used for feature extraction.
Since the hand size, shape, appearance, and illuminations have large variance, we
merge the feature maps from different scales to integrate both the global and the local
information. To reduce the computation overhead due to merging too many channels
on large feature maps, we use three successive 1 Â 1, 3 Â 3, 3 Â 3 convolutional
layers to lower the subsequent computational cost. The second module is the output
part, which includes two branches. The first branch outputs the confidence of each pixel
within the hand bounding boxes, and the second branch outputs the vertex coordinates
of the hand bounding boxes.
3.3 Loss Function
The loss function has two terms, as described in (1)
L ¼ L c þ L r :
ð1Þ
where L c evaluates the probability that a pixel locates inside the bounding box or not,
and L r evaluates the errors between the regressed position and the ground truth.
L c is defined using cross-entropy loss, given by
L c ¼ Àap
Ã
ð1 À pÞ
c log p À ð1 À aÞð1 À p
Ã
Þp
c logð1 À pÞ:
ð2Þ
where p
* represents the true class of a pixel, p represents the predicted probability of a
pixel inside the hand bounding box, a ¼ 1 À
Nðp
à ¼1Þ
Nðp Ã Þ is used to balance the impacts of
the positive and negative samples, and c is the adjustable parameter. In our experiments, the best choice of c is c ¼ 2.
L r is defined using smoothed-L1 loss, given by
L r ¼ smoothed L1 C i À C
Ã
i
À
Á
ð3Þ
smoothed L1 ðxÞ ¼
0:5x
2
jxj\1
jxj À 0:5 others
&
:
ð4Þ
where C i and C
Ã
i represent regressed vertex coordinates of the hand bounding box and
the ground truth, respectively.
3.4 Implementations
The size of input images is 128 Â 128. The network is trained end to end using Adam
optimization algorithm with minibatch size 24. The initial learning rate is 10
−3
, decays
10 times every 20,000 minibatches, and stops at 10
−5 .
Hand Detection Based on Multi-scale Fully Convolutional Networks
23