[15], etc. And, you probably need a high-performance GPU computing cluster, and
sometimes, this training takes several weeks or even months to make the output of the
loss function converge.
So, how can we train neural networks to develop our own image processing
applications without such a GPU computing cluster or lots of training samples? One
solution is transfer learning. Transfer learning is to transfer the model parameters that
have been learned from other places to the new model to help train the new model.
Due to the explosive growth of research related to neural network in recent years,
people often associate transfer learning with training of neural network. These two
concepts are unrelated at the beginning. Transfer learning is a branch of machine
learning, and many transfer methods did not need to use neural network. But now,
people find that deep neural network model has strong transferability. Because DNN is
a hierarchical representation of data obtained through pretrain and then classified with
high-level semantic classification. The low-level semantic features (such as texture,
edge, color information, etc.) are at the bottom of the model. Such features are actually
invariable in different classification tasks, and the real difference is the high-level
features.
If you apply the same network architecture to implement a vision task, you can
usually download weights that someone else has trained on the same network as
initialization, rather than training it from random values. And use transfer learning to
sort of transfer knowledge from some of these very large public data sets to our own
problem. Usually, this transfer process can greatly reduce the convergence time of the
model.
Let us see an example of medical image recognition, as shown in Fig. 3. If we need
building a cell detector to recognize colon cancer histology images, we were prepared
to recognize four clinically meaningful cells: epithelial nuclei, inflammatory nuclei,
fibroblasts, and miscellaneous nuclei. And we assume that each cropped image patch
contains only one cell. So, we have a classification problem with four classes. But our
Epithelial Nuclei
Inflammatory Nuclei
Fibroblast Nuclei
Miscellaneous Nuclei
Hidden layer
Softmax layer
Weight downloaded from the Internet
Transfer learning
Training images
Fig. 3. A sample of transfer learning
A Guideline for Object Detection Using Convolutional …
161
sometimes, this training takes several weeks or even months to make the output of the
loss function converge.
So, how can we train neural networks to develop our own image processing
applications without such a GPU computing cluster or lots of training samples? One
solution is transfer learning. Transfer learning is to transfer the model parameters that
have been learned from other places to the new model to help train the new model.
Due to the explosive growth of research related to neural network in recent years,
people often associate transfer learning with training of neural network. These two
concepts are unrelated at the beginning. Transfer learning is a branch of machine
learning, and many transfer methods did not need to use neural network. But now,
people find that deep neural network model has strong transferability. Because DNN is
a hierarchical representation of data obtained through pretrain and then classified with
high-level semantic classification. The low-level semantic features (such as texture,
edge, color information, etc.) are at the bottom of the model. Such features are actually
invariable in different classification tasks, and the real difference is the high-level
features.
If you apply the same network architecture to implement a vision task, you can
usually download weights that someone else has trained on the same network as
initialization, rather than training it from random values. And use transfer learning to
sort of transfer knowledge from some of these very large public data sets to our own
problem. Usually, this transfer process can greatly reduce the convergence time of the
model.
Let us see an example of medical image recognition, as shown in Fig. 3. If we need
building a cell detector to recognize colon cancer histology images, we were prepared
to recognize four clinically meaningful cells: epithelial nuclei, inflammatory nuclei,
fibroblasts, and miscellaneous nuclei. And we assume that each cropped image patch
contains only one cell. So, we have a classification problem with four classes. But our
Epithelial Nuclei
Inflammatory Nuclei
Fibroblast Nuclei
Miscellaneous Nuclei
Hidden layer
Softmax layer
Weight downloaded from the Internet
Transfer learning
Training images
Fig. 3. A sample of transfer learning
A Guideline for Object Detection Using Convolutional …
161