244
F. Firouzi et al.
5.4.5 Support Vector Machine ......................................................... 286
5.4.6 Decision Tree Classifier ......................................................... 290
5.4.7 Ensembles ........................................................................ 292
5.5 Dimensionality Reduction ................................................................ 296
5.6 Artificial Neural Networks................................................................ 297
5.6.1 Neural Network Models ......................................................... 297
5.6.2 Train a Neural Network Model.................................................. 298
5.6.3 Activation Function .............................................................. 300
5.6.4 Softmax Function ................................................................ 302
5.6.5 Convolution Neural Networks................................................... 304
5.7 Clustering .................................................................................. 310
5.7.1 K-Means Clustering ............................................................. 310
5.7.2 Hierarchical Clustering .......................................................... 311
5.8 Summary .................................................................................. 312
References ........................................................................................ 312
5.1 Fundamental of Machine Learning
Learning consists of such a broad range of processes that it is hard to define
precisely. In the dictionary, learning is defined as “to gain knowledge, or skill in,
or understanding of, by study, experience, or instruction” and “modification of a
behavioral tendency by experience.” In contrast to zoologists and psychologists
specialized in defining learning from perspectives of biological behaviors, we focus
on the learning process in machines. Many concepts in machine learning are brought
from the efforts of psychologists to make more precise their theories of human
learning through computational models. On the other side, during the development
of machine learning, some concepts or technologies may also inspire certain aspects
of biological learning [2–4].
As for the machine, we may impose the characteristic that a machine learns
whenever the system changes its structure, programmed in such a manner that
its expected future performance improves. The machine can only learn from the
history of its inputs or in response to external information, where more information
or changes shall render a more accurate response model. Some of these changes
(e.g., augmenting a database by adding an event) fall easily within the province of
other disciplines and are not essentially better understood for being called learning.
For instance, when an image recognition machine can eventually differentiate cats
from dogs after seeing several pictures of cats and dogs, we feel quite justified in
that case to say that the machine has learned. A typical machine learning model or
algorithm is similar to the following (see Fig. 5.1). A machine learning process
consists of three main components: input, machine learning model, and output.
Given the information, i.e., inputs, such as precipitation, humidity, and temperature,
we would like to execute our task, i.e., output, predicting the weather to be either
sunny or rainy. The core of this prediction process is a machine learning model,
a.k.a how can we find an appropriate model to accurately map all combination of
F. Firouzi et al.
5.4.5 Support Vector Machine ......................................................... 286
5.4.6 Decision Tree Classifier ......................................................... 290
5.4.7 Ensembles ........................................................................ 292
5.5 Dimensionality Reduction ................................................................ 296
5.6 Artificial Neural Networks................................................................ 297
5.6.1 Neural Network Models ......................................................... 297
5.6.2 Train a Neural Network Model.................................................. 298
5.6.3 Activation Function .............................................................. 300
5.6.4 Softmax Function ................................................................ 302
5.6.5 Convolution Neural Networks................................................... 304
5.7 Clustering .................................................................................. 310
5.7.1 K-Means Clustering ............................................................. 310
5.7.2 Hierarchical Clustering .......................................................... 311
5.8 Summary .................................................................................. 312
References ........................................................................................ 312
5.1 Fundamental of Machine Learning
Learning consists of such a broad range of processes that it is hard to define
precisely. In the dictionary, learning is defined as “to gain knowledge, or skill in,
or understanding of, by study, experience, or instruction” and “modification of a
behavioral tendency by experience.” In contrast to zoologists and psychologists
specialized in defining learning from perspectives of biological behaviors, we focus
on the learning process in machines. Many concepts in machine learning are brought
from the efforts of psychologists to make more precise their theories of human
learning through computational models. On the other side, during the development
of machine learning, some concepts or technologies may also inspire certain aspects
of biological learning [2–4].
As for the machine, we may impose the characteristic that a machine learns
whenever the system changes its structure, programmed in such a manner that
its expected future performance improves. The machine can only learn from the
history of its inputs or in response to external information, where more information
or changes shall render a more accurate response model. Some of these changes
(e.g., augmenting a database by adding an event) fall easily within the province of
other disciplines and are not essentially better understood for being called learning.
For instance, when an image recognition machine can eventually differentiate cats
from dogs after seeing several pictures of cats and dogs, we feel quite justified in
that case to say that the machine has learned. A typical machine learning model or
algorithm is similar to the following (see Fig. 5.1). A machine learning process
consists of three main components: input, machine learning model, and output.
Given the information, i.e., inputs, such as precipitation, humidity, and temperature,
we would like to execute our task, i.e., output, predicting the weather to be either
sunny or rainy. The core of this prediction process is a machine learning model,
a.k.a how can we find an appropriate model to accurately map all combination of