�
�
189
Logic Programming and Artificial Neural Networks
v k (t + Δt), is calculated by means of an output function whose argument is
p k (t). In fact, the activation function we will use most often in our work is the
weighted sum of the inputs minus the threshold. In other words, in most of
u
o
our discussions p k (t) =
j∈I k
w kj v j (t) − θ k ∈ R. We say that a unit N k
becomes active at time t if p k (t) ≥ 0. On the other hand, we consider a number
of different types of units distinguished mainly by their output function, as
follows. A unit is said to be a binary threshold unit if its output function is a
threshold function or Heaviside function H, so that
1 if p k (t) ≥ 0,
v k (t + Δt) = H(p k (t)) =
0 otherwise.
A unit is said to be a linear unit if its output function is the identity as a
function of p k (t) and its threshold θ is 0. A unit is said to be a sigmoidal unit
or a squashing unit if its output function φ is non-decreasing and is such that
lim x→∞ φ(x) = 1 and lim x→−∞ φ(x) = 0. Such functions are called squashing
functions.
•
We will only consider connectionist networks where the units can be organized in layers, although a variant of this will be encountered in Section 7.6.
A layer is a vector of units. An n-layer feedforward network F consists of the
input layer, n − 2 hidden layers, and the output layer, where n ≥ 2. Each unit
occurring in the i-th layer is connected to each unit occurring in the (i+1)-st
layer, 1 ≤ i < n. Let r and s be the number of units occurring in the input
and output layers, respectively. A connectionist network F is called a multilayer feedforward network if it is an n-layer feedforward network for some
n. A multilayer feedforward network F computes a function f F : R
r → R
s ,
called the input-output mapping of F or the network function of F, as follows. The input vector (the argument of f F ) is presented to the input layer
at time t 0 and propagated through the hidden layers to the output layer. At
each time point, all units update their potential and value, as noted above.
At time t 0 + (n − 1)Δt, the output vector (the image under f F of the input
vector) is read off the output layer.
For a 3-layer feedforward network with r linear units in the input layer,
squashing units in the hidden layer, and a single linear unit in the output
layer, the input-output function of the network as described in the previous
paragraph can thus be obtained as a mapping f : R
r → R with
f (x 1 , . . . , x r ) =
c j φ
w ji x i − θ j ,
j
i
where c j is the weight associated with the connection from the j-th unit of the
hidden layer to the single unit in the output layer, φ is the squashing output
function of the units in the hidden layer, w ji is the weight associated with the
�
189
Logic Programming and Artificial Neural Networks
v k (t + Δt), is calculated by means of an output function whose argument is
p k (t). In fact, the activation function we will use most often in our work is the
weighted sum of the inputs minus the threshold. In other words, in most of
u
o
our discussions p k (t) =
j∈I k
w kj v j (t) − θ k ∈ R. We say that a unit N k
becomes active at time t if p k (t) ≥ 0. On the other hand, we consider a number
of different types of units distinguished mainly by their output function, as
follows. A unit is said to be a binary threshold unit if its output function is a
threshold function or Heaviside function H, so that
1 if p k (t) ≥ 0,
v k (t + Δt) = H(p k (t)) =
0 otherwise.
A unit is said to be a linear unit if its output function is the identity as a
function of p k (t) and its threshold θ is 0. A unit is said to be a sigmoidal unit
or a squashing unit if its output function φ is non-decreasing and is such that
lim x→∞ φ(x) = 1 and lim x→−∞ φ(x) = 0. Such functions are called squashing
functions.
•
We will only consider connectionist networks where the units can be organized in layers, although a variant of this will be encountered in Section 7.6.
A layer is a vector of units. An n-layer feedforward network F consists of the
input layer, n − 2 hidden layers, and the output layer, where n ≥ 2. Each unit
occurring in the i-th layer is connected to each unit occurring in the (i+1)-st
layer, 1 ≤ i < n. Let r and s be the number of units occurring in the input
and output layers, respectively. A connectionist network F is called a multilayer feedforward network if it is an n-layer feedforward network for some
n. A multilayer feedforward network F computes a function f F : R
r → R
s ,
called the input-output mapping of F or the network function of F, as follows. The input vector (the argument of f F ) is presented to the input layer
at time t 0 and propagated through the hidden layers to the output layer. At
each time point, all units update their potential and value, as noted above.
At time t 0 + (n − 1)Δt, the output vector (the image under f F of the input
vector) is read off the output layer.
For a 3-layer feedforward network with r linear units in the input layer,
squashing units in the hidden layer, and a single linear unit in the output
layer, the input-output function of the network as described in the previous
paragraph can thus be obtained as a mapping f : R
r → R with
f (x 1 , . . . , x r ) =
c j φ
w ji x i − θ j ,
j
i
where c j is the weight associated with the connection from the j-th unit of the
hidden layer to the single unit in the output layer, φ is the squashing output
function of the units in the hidden layer, w ji is the weight associated with the