422
J. Wu et al.
earliest firing neuron, which signals the detection of the earliest local discriminative
feature (a property of the Tempotron).
The Tempotron learning rule follows a stochastic gradient descent method for
synaptic weight updates: the desired output neuron triggers a weight update whenever
it fails to fire on samples with matching class label or when the wrong output neurons
fire erroneously on samples from other classes. When the desired output neuron i
fails to fire, long-term potentiation (LTP) update with cost function V thr - V t
max
i
is
triggered. Similarly, long-term depression (LTD) update with cost function V t
max
i
-
V thr is triggered when the wrong output neuron fires erroneously. The Tempotron
update rule is defined as follows:
w i j =
⎧
⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨
⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩
λ
t
( f )
j
i
K (t
max
i
− t
( f )
j ), i f LT P
− λ
t
( f )
j
i
K (t
max
i
− t
( f )
j ), i f LT D
0,
other wise
(10)
where λ denotes a constant learning rate and t
max
i
refers to the time instant when
the postsynaptic neuron i reaches its maximum membrane potential over the pattern
duration. The t
( f )
j
are spike times of spike emitted by the pre-synaptic neuron j. The
synaptic weights are only updated at the time instant of t
max
i
. For LTD weight update,
the t
max
i
is also the spike time since the post-spike computations are ignored.
Inspired by the maximum-margin classifier [19, 29], we introduce a hard margin
to the V thr and denote this learning rule as the Maximum-Margin Tempotron.
During the training phase, the term is either added to or deducted from the V thr
of the desired or wrong output neurons, respectively. Consequently, for the desired
neuron i, a spike is generated at t if
V i (t) = V thr + and
d
dt
V i (t) > 0
(11)
For the other (wrong) neurons, a spike is generated if
V i (t) = V thr − and
d
dt
V i (t) > 0
( 1 2 )
The desired output neuron will fire only when it has observed strong evidence that
causes its V t max to rise above V thr by a margin of . Similarly, the other neurons
will be discouraged to fire and maintain its membrane potential by a margin
below V thr . This additional margin imposes a harder constraint during training
and encourages the SNN classifier to find more discriminative features in the input
spike patterns. Therefore, during testing, when the hard margin is removed from
J. Wu et al.
earliest firing neuron, which signals the detection of the earliest local discriminative
feature (a property of the Tempotron).
The Tempotron learning rule follows a stochastic gradient descent method for
synaptic weight updates: the desired output neuron triggers a weight update whenever
it fails to fire on samples with matching class label or when the wrong output neurons
fire erroneously on samples from other classes. When the desired output neuron i
fails to fire, long-term potentiation (LTP) update with cost function V thr - V t
max
i
is
triggered. Similarly, long-term depression (LTD) update with cost function V t
max
i
-
V thr is triggered when the wrong output neuron fires erroneously. The Tempotron
update rule is defined as follows:
w i j =
⎧
⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨
⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩
λ
t
( f )
j
i
K (t
max
i
− t
( f )
j ), i f LT P
− λ
t
( f )
j
i
K (t
max
i
− t
( f )
j ), i f LT D
0,
other wise
(10)
where λ denotes a constant learning rate and t
max
i
refers to the time instant when
the postsynaptic neuron i reaches its maximum membrane potential over the pattern
duration. The t
( f )
j
are spike times of spike emitted by the pre-synaptic neuron j. The
synaptic weights are only updated at the time instant of t
max
i
. For LTD weight update,
the t
max
i
is also the spike time since the post-spike computations are ignored.
Inspired by the maximum-margin classifier [19, 29], we introduce a hard margin
to the V thr and denote this learning rule as the Maximum-Margin Tempotron.
During the training phase, the term is either added to or deducted from the V thr
of the desired or wrong output neurons, respectively. Consequently, for the desired
neuron i, a spike is generated at t if
V i (t) = V thr + and
d
dt
V i (t) > 0
(11)
For the other (wrong) neurons, a spike is generated if
V i (t) = V thr − and
d
dt
V i (t) > 0
( 1 2 )
The desired output neuron will fire only when it has observed strong evidence that
causes its V t max to rise above V thr by a margin of . Similarly, the other neurons
will be discouraged to fire and maintain its membrane potential by a margin
below V thr . This additional margin imposes a harder constraint during training
and encourages the SNN classifier to find more discriminative features in the input
spike patterns. Therefore, during testing, when the hard margin is removed from