However, the first-generation decision tree had some disadvantages such as
overfitting problem. Furthermore, when applying ID3 on a large-scale dataset,
only one attribute could be tested for making decision at a time, which means
using ID3 in performing large-scale machine learning task could be timeconsuming.
C4.5: C4.5 decision tree algorithm is developed by Quinlan (1993). As the improved
version of ID3 algorithm, C4.5 handles both categorical value and numeric value,
adopting gain ratio as the rule of splitting. C4.5 decision tree made a lot of
improvements based on ID3; its new features include handling both continuous
and discrete attributes, handling training data with missing attribute values,
handling attributes with differing costs, and pruning trees after creation.
CART: Classification and Regression Trees (CART) is an umbrella term to refer to
the classification tree (the predicted outcome is the class to which the data
belongs) and regression tree (the predicted outcome can be considered a real
number), first introduced by Breiman et al. (1984). CART uses towing criteria as
the rule of splitting where it is measured with Gini Impurity when splitting nodes
instead of Shannon entropy. Besides, CART can easily handle outliers. And like
C4.5, CART can deal with both categorical and numeric values. Based on its
advantages of performing classification and regression tasks, CART has been
shown to be very effective for land cover classification (Sexton et al. 2013).
Other decision tree programs that have been used in machine learning-based
agricultural information extraction include See5/C5.0, which is the successive version of C4.5 (Quinlan 2003), S-Plus, which covers a great number of data mining
functions, including decision tree/regression tree module, and R language, which
provides an integrated environment for statistical analysis.
6.5.3 Artificial Neural Network
Artificial neural network is another machine learning approach which is extensively
applied in image analysis and computer vision. The concept of computational
model for neural networks was firstly proposed in McCulloch and Pitts (1943).
Artificial neural network is inspired by Hubel and Wiesel (1959). Fukushima and
Miyake (1982) proposed the concept of convolutional neural network. In 1989, the
backpropagation algorithm was applied to a deep convolutional neural network for
performing handwritten ZIP code recognition (LeCun et al. 1989) which is the
early application of deep learning technology. In recent years, with the rapid
development of computer hardware like graphics processing units (GPU) which
could significantly accelerate the process of network training, deep learning technology and deep convolutional nets became the new favorite in both research
community and industry. As described in LeCun et al. (2015), “Deep
convolutional nets have brought about breakthroughs in processing images,
video, speech and audio.” In image processing and computer vision area, deep
94
C. Zhang and L. Lin
overfitting problem. Furthermore, when applying ID3 on a large-scale dataset,
only one attribute could be tested for making decision at a time, which means
using ID3 in performing large-scale machine learning task could be timeconsuming.
C4.5: C4.5 decision tree algorithm is developed by Quinlan (1993). As the improved
version of ID3 algorithm, C4.5 handles both categorical value and numeric value,
adopting gain ratio as the rule of splitting. C4.5 decision tree made a lot of
improvements based on ID3; its new features include handling both continuous
and discrete attributes, handling training data with missing attribute values,
handling attributes with differing costs, and pruning trees after creation.
CART: Classification and Regression Trees (CART) is an umbrella term to refer to
the classification tree (the predicted outcome is the class to which the data
belongs) and regression tree (the predicted outcome can be considered a real
number), first introduced by Breiman et al. (1984). CART uses towing criteria as
the rule of splitting where it is measured with Gini Impurity when splitting nodes
instead of Shannon entropy. Besides, CART can easily handle outliers. And like
C4.5, CART can deal with both categorical and numeric values. Based on its
advantages of performing classification and regression tasks, CART has been
shown to be very effective for land cover classification (Sexton et al. 2013).
Other decision tree programs that have been used in machine learning-based
agricultural information extraction include See5/C5.0, which is the successive version of C4.5 (Quinlan 2003), S-Plus, which covers a great number of data mining
functions, including decision tree/regression tree module, and R language, which
provides an integrated environment for statistical analysis.
6.5.3 Artificial Neural Network
Artificial neural network is another machine learning approach which is extensively
applied in image analysis and computer vision. The concept of computational
model for neural networks was firstly proposed in McCulloch and Pitts (1943).
Artificial neural network is inspired by Hubel and Wiesel (1959). Fukushima and
Miyake (1982) proposed the concept of convolutional neural network. In 1989, the
backpropagation algorithm was applied to a deep convolutional neural network for
performing handwritten ZIP code recognition (LeCun et al. 1989) which is the
early application of deep learning technology. In recent years, with the rapid
development of computer hardware like graphics processing units (GPU) which
could significantly accelerate the process of network training, deep learning technology and deep convolutional nets became the new favorite in both research
community and industry. As described in LeCun et al. (2015), “Deep
convolutional nets have brought about breakthroughs in processing images,
video, speech and audio.” In image processing and computer vision area, deep
94
C. Zhang and L. Lin