Contemporary industries alleviate big data demands by relying on these machine
learning algorithms distributed across hundreds of servers and graphics processing
units (GPUs), which are discussed in a later section, in an attempt to emulate a
brain’s capability of organizing unstructured data. However, CMOS-based neural
networks are trending towards costly power consumption as transfer speeds between
buffer memory and logic components as well as miniaturization capabilities are
approaching fundamental limits.
2.4 Reservoir Computing
Reservoir computing (RC) is an emerging paradigm that promotes computing using
the intrinsic nonlinear dynamics of an excited system called a reservoir. Maass et al.
initially proposed a version of RC called the Liquid State Machine (LSM) as a model
for cortical microcircuits. Independently, Jaeger introduced a variation of RC called
the Echo State Machine (ESM) as an alternative RNN approach for control tasks.
Variations of both LSM and ESM have been proposed for many different machine
learning and system control tasks. Büsing et al. conducted a comprehensive study of
reservoir performance using different metrics as a function of the node
connectivity K, the logarithm of the number of states per node m, and the variance
of the weights in the reservoir [26].
At its core, the RC paradigm utilizes a reservoir’s capability to project an input’s
information into a mathematical higher representation space, similar to a Fourier
transform. A variety of spatially distributed mathematical operations occur throughout the system according to system properties and dimensionality. Computation
occurs as a recursive learning algorithm that inscribes a filter on the system such that
the projection spans the correct mathematical operations and the desired process is
achieved. The reservoir essentially outputs a series of nonlinear transformations of
the input which are then trained at the output layer by synaptic weights using linear
regression. A system with sufficiently rich dynamics can remember perturbations by
an external input over time which compared to other approaches, has many key
advantages of using RC such as:
1. Computationally inexpensive training (low programming overhead).
2. Flexibility in the physical reservoir implementation (cost-effective fabrication).
3. A high tolerance to material variation, defects, and faults (robustness).
These factors make RC particularly suitable for emerging unconventional computing paradigms, such as computing using physical phenomena [27] and selfassembled electronic architectures [28]. The high nonlinearity, and thus the high
dimensionality, ensures convergence of this algorithm in practical time. Reservoir
computing differs from earlier attempts at computing with random assemblies of
nano-cells or switches, e.g., Tour et al. [29]. Such systems lacked a formal framework and required complex and time-consuming optimization steps in order to
208
R. Aguilera et al.
learning algorithms distributed across hundreds of servers and graphics processing
units (GPUs), which are discussed in a later section, in an attempt to emulate a
brain’s capability of organizing unstructured data. However, CMOS-based neural
networks are trending towards costly power consumption as transfer speeds between
buffer memory and logic components as well as miniaturization capabilities are
approaching fundamental limits.
2.4 Reservoir Computing
Reservoir computing (RC) is an emerging paradigm that promotes computing using
the intrinsic nonlinear dynamics of an excited system called a reservoir. Maass et al.
initially proposed a version of RC called the Liquid State Machine (LSM) as a model
for cortical microcircuits. Independently, Jaeger introduced a variation of RC called
the Echo State Machine (ESM) as an alternative RNN approach for control tasks.
Variations of both LSM and ESM have been proposed for many different machine
learning and system control tasks. Büsing et al. conducted a comprehensive study of
reservoir performance using different metrics as a function of the node
connectivity K, the logarithm of the number of states per node m, and the variance
of the weights in the reservoir [26].
At its core, the RC paradigm utilizes a reservoir’s capability to project an input’s
information into a mathematical higher representation space, similar to a Fourier
transform. A variety of spatially distributed mathematical operations occur throughout the system according to system properties and dimensionality. Computation
occurs as a recursive learning algorithm that inscribes a filter on the system such that
the projection spans the correct mathematical operations and the desired process is
achieved. The reservoir essentially outputs a series of nonlinear transformations of
the input which are then trained at the output layer by synaptic weights using linear
regression. A system with sufficiently rich dynamics can remember perturbations by
an external input over time which compared to other approaches, has many key
advantages of using RC such as:
1. Computationally inexpensive training (low programming overhead).
2. Flexibility in the physical reservoir implementation (cost-effective fabrication).
3. A high tolerance to material variation, defects, and faults (robustness).
These factors make RC particularly suitable for emerging unconventional computing paradigms, such as computing using physical phenomena [27] and selfassembled electronic architectures [28]. The high nonlinearity, and thus the high
dimensionality, ensures convergence of this algorithm in practical time. Reservoir
computing differs from earlier attempts at computing with random assemblies of
nano-cells or switches, e.g., Tour et al. [29]. Such systems lacked a formal framework and required complex and time-consuming optimization steps in order to
208
R. Aguilera et al.