435
performance computing or HPC) is used to model simulations or perform brute force
calculations. Supercomputers are research tools. Historically (in the 1970s) this was
weather forecasting and aerodynamics (planes, spacecraft, cars), then nuclear weapons simulations and radiation shielding modeling (the height of the Cold War in the
1980s), and in recent years the emphasis has been on cracking decryption and molecular dynamics modeling. For example, for weather forecasting (Webster 2012), it is
the sheer amount of data that needs to be processed. There are hundreds of thousands
of weather stations around the world, each one recording dozens of variables (wind
speed/direction, temperature, humidity, pollen count). All this data needs to be compared to last night’s data, last week’s data, and historical data, all of which must be
fetched from huge memory and storage banks. Generally, most of these calculations
can occur at the same time, and so massive parallelism means that weather forecasts
can be produced in seconds or minutes, rather than hours (Baillie et al. 1997).
Second, these computers are backed by distributed file systems which aim to
divide the data into multiple storage units, compute and save intermediate steps in
the disk, then merge the outputs into a single result. To understand why such a strategy is used, one needs to understand the performances of these units. CPUs or
GPUs are extremely fast components that can handle trillions of floating-point operations in a second. However, the data, which is used as the input, resides in the disks
that are thousands of times slower than the CPU and GPU units.
Since a computer system is as fast as its slowest component, historically, the
disks were the bottlenecks for operations that require large input or intermediate
data. These distributed file system approaches aim to address the speed issue by
dividing the data into smaller chunks, using them individually by each CPU/GPU
and thus reducing the cost of read/writes into disk (Howard et al. 1988). For example, Hadoop Map- Reduce framework uses this strategy (Borthakur 2007). The term
Map-Reduce refers to two separate and distinct tasks that Hadoop programs perform. The first is the map job, which takes a set of data and converts it into another
set of data, where individual elements are broken down into tuples (key/value pairs).
The reduce job takes the output from a map as input and combines those data tuples
into a smaller set of tuples and thus giving the results.
Due to the advances in cheap and abundant memory space, another recent
approach is using memory instead of disk to perform such distributed data processing. Although this second approach, namely, (Apache Spark 2015), is not completely
replacing the Hadoop (it still uses Hadoop Distributed File System), it uses inmemory processing of data on top of Hadoop’s Map-Reduce framework and thus
improves the performance since memory read/writes are thousands of times faster
than disk read/writes (Intel 2013; Jain & Somni 2013; Reed & Dongarra 2015).
Therefore, Spark’s in-memory processing delivers near real-time analytics for data
machine learning, sensors, data monitoring and analytics which is particularly important to address the Velocity issues of Big Data. For example, agricultural datasets,
which include soil properties, humidity, and so on, that are collected from the field
sensors in real time can be analyzed near real- time by using such an approach.
Finally, it is worth mentioning the current state-of-the-art approaches that
leverage these high-performance capabilities. One particular and often heard term
16 Computing
performance computing or HPC) is used to model simulations or perform brute force
calculations. Supercomputers are research tools. Historically (in the 1970s) this was
weather forecasting and aerodynamics (planes, spacecraft, cars), then nuclear weapons simulations and radiation shielding modeling (the height of the Cold War in the
1980s), and in recent years the emphasis has been on cracking decryption and molecular dynamics modeling. For example, for weather forecasting (Webster 2012), it is
the sheer amount of data that needs to be processed. There are hundreds of thousands
of weather stations around the world, each one recording dozens of variables (wind
speed/direction, temperature, humidity, pollen count). All this data needs to be compared to last night’s data, last week’s data, and historical data, all of which must be
fetched from huge memory and storage banks. Generally, most of these calculations
can occur at the same time, and so massive parallelism means that weather forecasts
can be produced in seconds or minutes, rather than hours (Baillie et al. 1997).
Second, these computers are backed by distributed file systems which aim to
divide the data into multiple storage units, compute and save intermediate steps in
the disk, then merge the outputs into a single result. To understand why such a strategy is used, one needs to understand the performances of these units. CPUs or
GPUs are extremely fast components that can handle trillions of floating-point operations in a second. However, the data, which is used as the input, resides in the disks
that are thousands of times slower than the CPU and GPU units.
Since a computer system is as fast as its slowest component, historically, the
disks were the bottlenecks for operations that require large input or intermediate
data. These distributed file system approaches aim to address the speed issue by
dividing the data into smaller chunks, using them individually by each CPU/GPU
and thus reducing the cost of read/writes into disk (Howard et al. 1988). For example, Hadoop Map- Reduce framework uses this strategy (Borthakur 2007). The term
Map-Reduce refers to two separate and distinct tasks that Hadoop programs perform. The first is the map job, which takes a set of data and converts it into another
set of data, where individual elements are broken down into tuples (key/value pairs).
The reduce job takes the output from a map as input and combines those data tuples
into a smaller set of tuples and thus giving the results.
Due to the advances in cheap and abundant memory space, another recent
approach is using memory instead of disk to perform such distributed data processing. Although this second approach, namely, (Apache Spark 2015), is not completely
replacing the Hadoop (it still uses Hadoop Distributed File System), it uses inmemory processing of data on top of Hadoop’s Map-Reduce framework and thus
improves the performance since memory read/writes are thousands of times faster
than disk read/writes (Intel 2013; Jain & Somni 2013; Reed & Dongarra 2015).
Therefore, Spark’s in-memory processing delivers near real-time analytics for data
machine learning, sensors, data monitoring and analytics which is particularly important to address the Velocity issues of Big Data. For example, agricultural datasets,
which include soil properties, humidity, and so on, that are collected from the field
sensors in real time can be analyzed near real- time by using such an approach.
Finally, it is worth mentioning the current state-of-the-art approaches that
leverage these high-performance capabilities. One particular and often heard term
16 Computing