ANNs consist of two elements (i) formal neurons and (ii) connections between
the neurons. The way that individual artificial neurons are interconnected is called
topology, architecture or graph of an ANN. The full potential and calculation
capabilities become clear when we start to interconnect neurons into ANNs.
Neurons are arranged in layers, where at least two layers of neurons (an input layer
and an output layer) are required for construction of a neural network. In any
architecture every neuron of a layer is connected to every neuron of the following
layer, and no intra-layer connections exist. This property allows referred to them as
‘‘multilayer perceptrons’’ although the classical perception contains only a single
neuron (Minsky and Papert 1969). The fact that interconnection can be done in
numerous ways results in different possible topologies that are divided into two
basic types (Fig. 2.2). In the feed-forward topology (a) the information flows from
inputs to outputs in only one direction and in the recurrent topology (b) some
information flows not in only one direction from input to output but also in
opposite direction. Fully connected feed-forward networks are by far the most
frequently used neural networks for nonlinear modeling, pattern recognition,
classification, signal filtering, and forecasting (Sumpter et al. 1994).
Formal neurons transform a numerical input to an output value, and the neuron
connections represent numerical weight values. The weights and neurons’ internal
variables (termed bias or threshold values) are free variables of the system which
must be determined in the so-called ‘‘training stage’’ of the network development.
This training set the proper response of an ANN according to the characteristics of
the problem to be solved.
Input layer
Hiddenlayer
Output layer
Input layer
Hiddenlayer
Output layer
(a)
(b)
Fig. 2.2 Architecture of simple artificial neural networks: a Feed-forward (FNN) and b recurrent
(RNN) topology of an artificial neural network
62
M. A. Vázquez-Cruz et al.
the neurons. The way that individual artificial neurons are interconnected is called
topology, architecture or graph of an ANN. The full potential and calculation
capabilities become clear when we start to interconnect neurons into ANNs.
Neurons are arranged in layers, where at least two layers of neurons (an input layer
and an output layer) are required for construction of a neural network. In any
architecture every neuron of a layer is connected to every neuron of the following
layer, and no intra-layer connections exist. This property allows referred to them as
‘‘multilayer perceptrons’’ although the classical perception contains only a single
neuron (Minsky and Papert 1969). The fact that interconnection can be done in
numerous ways results in different possible topologies that are divided into two
basic types (Fig. 2.2). In the feed-forward topology (a) the information flows from
inputs to outputs in only one direction and in the recurrent topology (b) some
information flows not in only one direction from input to output but also in
opposite direction. Fully connected feed-forward networks are by far the most
frequently used neural networks for nonlinear modeling, pattern recognition,
classification, signal filtering, and forecasting (Sumpter et al. 1994).
Formal neurons transform a numerical input to an output value, and the neuron
connections represent numerical weight values. The weights and neurons’ internal
variables (termed bias or threshold values) are free variables of the system which
must be determined in the so-called ‘‘training stage’’ of the network development.
This training set the proper response of an ANN according to the characteristics of
the problem to be solved.
Input layer
Hiddenlayer
Output layer
Input layer
Hiddenlayer
Output layer
(a)
(b)
Fig. 2.2 Architecture of simple artificial neural networks: a Feed-forward (FNN) and b recurrent
(RNN) topology of an artificial neural network
62
M. A. Vázquez-Cruz et al.