1 Introduction
In 1965 Gordon Moore, co-founder and chairman of Intel, published a paper on the
number of transistors per integrated circuit (microchip) doubling every 2 years. This
trend became known as Moore’s Law and meant processing speed doubled every
2 years. Recently, Moore’s law shows signs of slowing down due to physical and
economic constraints. In the next few decades individual elements will approach the
scale of a few atoms and in turn the fundamental limits of miniaturization [1]. Feature
sizes are constrained by the optical diffraction limit which defines minimum feature
size by wavelength divided by two times the numerical aperture of the microscope
[2]. Even before physical boundaries are approached, the economical limitations of
continuing miniaturization and massive integration will be reached. With extremely
small features fabrication costs increase dramatically, making them unsuitable for
cost-effective mass production. Thus, as the end of Moore’s Law is approached [3]
processing power will no longer increase unless alternative individual elements or
architectures of integrated circuits are explored.
Modern computers ubiquitously use the Von Neumann architecture first
described by John von Neumann in 1945. This architecture separates memory and
processing components within integrated circuits [4]. Partitioning data and instruction storage from arithmetic/logic processing leads to a limitation of information
transfer known as the Von Neumann bottleneck [5]. When calculating more complex
problems requiring mass amounts of data or instructions from memory storage,
processors sit idle. In order to maximize information processing and ultimately
overcome the Von Neumann bottleneck, significant changes to computer architectures must be made.
Carver Mead, a scientist at Cal Tech that coined the term “Moore’s Law” became
known for his bio-inspired work in the mid 60s. Particularly his attempts to emulate
neural functionality directly into analog hardware implementations and pioneering
the “Neuromorphic” computation field. His neuromorphic research was specifically
based on analog metal oxide semiconductor (MOS) processors. By 1995 Mead and
collaborators demonstrated a single silicon transistor ‘synapse’ capable of analog
learning. However, the ensuing rapid development in digital microprocessors using
VLSI superseded early analog computing approaches. In turn, digital neural network
software led to today’s deep learning and machine learning algorithms.
Concurrently, the growth of the internet-of-things, cloud computing and the rapid
explosion of unstructured data has placed new demands on computers in our increasingly interconnected world. Examples of interconnected data are those from satellites,
sensors, economic markets, commerce, global climate patterns, social media and
consumer habits. This combinatorial complexity challenges the inherently serial Von
Neumann architecture of computers which, at their core, scale poorly into supercomputers requiring massive increases in hardware and energy consumption. Currently,
China’s Tianhe-2 supercomputer uses 18 MW at full power [6] and is still very far
from performing many tasks that a human brain can achieve using merely 20 W. In
terms of dealing with complex tasks, the scalability of today’s computers cannot keep
up in a realistic manner with the world in which we live.
202
R. Aguilera et al.
In 1965 Gordon Moore, co-founder and chairman of Intel, published a paper on the
number of transistors per integrated circuit (microchip) doubling every 2 years. This
trend became known as Moore’s Law and meant processing speed doubled every
2 years. Recently, Moore’s law shows signs of slowing down due to physical and
economic constraints. In the next few decades individual elements will approach the
scale of a few atoms and in turn the fundamental limits of miniaturization [1]. Feature
sizes are constrained by the optical diffraction limit which defines minimum feature
size by wavelength divided by two times the numerical aperture of the microscope
[2]. Even before physical boundaries are approached, the economical limitations of
continuing miniaturization and massive integration will be reached. With extremely
small features fabrication costs increase dramatically, making them unsuitable for
cost-effective mass production. Thus, as the end of Moore’s Law is approached [3]
processing power will no longer increase unless alternative individual elements or
architectures of integrated circuits are explored.
Modern computers ubiquitously use the Von Neumann architecture first
described by John von Neumann in 1945. This architecture separates memory and
processing components within integrated circuits [4]. Partitioning data and instruction storage from arithmetic/logic processing leads to a limitation of information
transfer known as the Von Neumann bottleneck [5]. When calculating more complex
problems requiring mass amounts of data or instructions from memory storage,
processors sit idle. In order to maximize information processing and ultimately
overcome the Von Neumann bottleneck, significant changes to computer architectures must be made.
Carver Mead, a scientist at Cal Tech that coined the term “Moore’s Law” became
known for his bio-inspired work in the mid 60s. Particularly his attempts to emulate
neural functionality directly into analog hardware implementations and pioneering
the “Neuromorphic” computation field. His neuromorphic research was specifically
based on analog metal oxide semiconductor (MOS) processors. By 1995 Mead and
collaborators demonstrated a single silicon transistor ‘synapse’ capable of analog
learning. However, the ensuing rapid development in digital microprocessors using
VLSI superseded early analog computing approaches. In turn, digital neural network
software led to today’s deep learning and machine learning algorithms.
Concurrently, the growth of the internet-of-things, cloud computing and the rapid
explosion of unstructured data has placed new demands on computers in our increasingly interconnected world. Examples of interconnected data are those from satellites,
sensors, economic markets, commerce, global climate patterns, social media and
consumer habits. This combinatorial complexity challenges the inherently serial Von
Neumann architecture of computers which, at their core, scale poorly into supercomputers requiring massive increases in hardware and energy consumption. Currently,
China’s Tianhe-2 supercomputer uses 18 MW at full power [6] and is still very far
from performing many tasks that a human brain can achieve using merely 20 W. In
terms of dealing with complex tasks, the scalability of today’s computers cannot keep
up in a realistic manner with the world in which we live.
202
R. Aguilera et al.