Language (OpenCL) platform has been designed to allowing programming through
a heterogeneous array of hardware components including FPGAs. Intel has demonstrated a convolutional neural network (CNN) implementation using this platform
which currently outperforms traditional CPU based CNNs with a significant increase
in efficiency [34]. Advances in software continue to expand the use of FPGAs for a
wide array of applications.
Continued developments in FPGA architectures have demonstrated their proficiency as accelerators in deep convolutional neural networks [35], Qiao et al. have
developed an FPGA accelerator compatible with Caffe software, which is currently
implemented in a wide array of CNNs. A comparison of their FPGA (Xilinx Zynq)
accelerator to a traditional CPU (Intel Xeon X5675) and GPU (Nvidia Tesla K20)
based CNNs shows FPGAs are superior in versatility and power efficiency to CPUs
[34]. While GPUs still offer the capability of much higher performance (486 Gflops
to the Zynq’s 77.8 Gflops), the large disparity in power consumption (235 W to the
Zynq’s 14.4 W) implicates FPGAs as the most power efficient device to be
implemented in the acceleration of CNNs. Future advances in GPU and FPGA
technology may alter this trend.
3.3 Graphics Processing Units (GPUs)
The term graphics processing unit, or GPU, was promulgated by NVIDIA in 1999 to
mark the release of the world’s first such device, the GeForce 256 [36]. This GPU
was touted as an incredible advancement in the world of computer hardware,
possessing approximately 23 million transistors (NVIDIA boasted at the time that
this was twice as complex as a Pentium III processor) and possessed 50 gigaflops of
floating-point calculation capability [36]. Nearly two decades later, GPUs have made
enormous strides in power and capability. The most powerful GPU as of 2017 is
NVIDIA’s Tesla V100, a device possessing 21 billion transistors, over 5000 cores
providing 120 teraflops of performance for deep learning applications while drawing
only 300 W of power [37]. Thus, these units have increased in processing power by
over four orders of magnitude over the last 18 years.
As GPU power increased over time, interest in them from individuals and
organizations outside the computer gaming community grew significantly. In the
quest to expand upon the capabilities of existing computing architectures and
platforms, academic research groups began evaluating GPU’s viability for
performing certain tasks previously delegated to supercomputers. In 2012,
researchers at the University of Toronto demonstrated that GPUs could be used to
drastically improve tasks related to computer vision and deep learning, such as
image reconstruction, by using the GPUs to run deep neural networks [38]. These
GPUs displayed significant performance gains relative to traditional computer
processor-based neural networks thanks to their massively parallel architecture
involving thousands of individual processor units and exceptional processor-tomemory bandwidth.
Atomic Switch Networks for Neuroarchitectonics: Past, Present, Future
211
a heterogeneous array of hardware components including FPGAs. Intel has demonstrated a convolutional neural network (CNN) implementation using this platform
which currently outperforms traditional CPU based CNNs with a significant increase
in efficiency [34]. Advances in software continue to expand the use of FPGAs for a
wide array of applications.
Continued developments in FPGA architectures have demonstrated their proficiency as accelerators in deep convolutional neural networks [35], Qiao et al. have
developed an FPGA accelerator compatible with Caffe software, which is currently
implemented in a wide array of CNNs. A comparison of their FPGA (Xilinx Zynq)
accelerator to a traditional CPU (Intel Xeon X5675) and GPU (Nvidia Tesla K20)
based CNNs shows FPGAs are superior in versatility and power efficiency to CPUs
[34]. While GPUs still offer the capability of much higher performance (486 Gflops
to the Zynq’s 77.8 Gflops), the large disparity in power consumption (235 W to the
Zynq’s 14.4 W) implicates FPGAs as the most power efficient device to be
implemented in the acceleration of CNNs. Future advances in GPU and FPGA
technology may alter this trend.
3.3 Graphics Processing Units (GPUs)
The term graphics processing unit, or GPU, was promulgated by NVIDIA in 1999 to
mark the release of the world’s first such device, the GeForce 256 [36]. This GPU
was touted as an incredible advancement in the world of computer hardware,
possessing approximately 23 million transistors (NVIDIA boasted at the time that
this was twice as complex as a Pentium III processor) and possessed 50 gigaflops of
floating-point calculation capability [36]. Nearly two decades later, GPUs have made
enormous strides in power and capability. The most powerful GPU as of 2017 is
NVIDIA’s Tesla V100, a device possessing 21 billion transistors, over 5000 cores
providing 120 teraflops of performance for deep learning applications while drawing
only 300 W of power [37]. Thus, these units have increased in processing power by
over four orders of magnitude over the last 18 years.
As GPU power increased over time, interest in them from individuals and
organizations outside the computer gaming community grew significantly. In the
quest to expand upon the capabilities of existing computing architectures and
platforms, academic research groups began evaluating GPU’s viability for
performing certain tasks previously delegated to supercomputers. In 2012,
researchers at the University of Toronto demonstrated that GPUs could be used to
drastically improve tasks related to computer vision and deep learning, such as
image reconstruction, by using the GPUs to run deep neural networks [38]. These
GPUs displayed significant performance gains relative to traditional computer
processor-based neural networks thanks to their massively parallel architecture
involving thousands of individual processor units and exceptional processor-tomemory bandwidth.
Atomic Switch Networks for Neuroarchitectonics: Past, Present, Future
211