434
2018, most users of Python are using package management tools (e.g., Anaconda n.d.)
to keep track of which packages are used in their application as well as help ease the
portability of their code. Also, Python has an option of using web browser- based IDEs
such as Jupyter notebooks which are getting increasingly popular due to their code
completion, portability, and presentation (web-based tutorials, etc.) capabilities.
Finally, it is worth noting that the scientific community is increasingly doing
their developments in the cloud because of the machine-independent development
capabilities as well as data management/storage options provided by these tools.
For example, Amazon Web Services (n.d.) provide data management by its
“Buckets (e.g., S3)” and its web-based development environment (e.g., AWS
Sagemaker) with ready to use data analytics tools embedded.
16.3 Challenges in Computer Science
As defined above, some computational problems are extremely hard to solve. In
addition, those may require large storage space to solve the problem. To make things
worse, “Big Data,” which is often defined by four V’s, namely, volume, variety,
velocity, and value, adds an additional challenge to the current approaches. Volume
refers to the increasing amount of data that are collected from different sources.
Variety refers to the different types of data that are collected from many sensors and
velocity refers to the speed of collection. Finally, the Value terms emphasize the
analytic application of the data and its potential associated value.
Most data related to FEW is Big Spatial Data with more challenging characteristics such as autocorrelation between samples, non-isotropy across dimensions, and
heterogeneity.
Agricultural datasets are great examples of big data (Bronson & Knezevic 2016).
These datasets which are often collected from multiple sources, including the soil
sensors in the field, remote sensing imagery from satellites as well as unmanned
aerial vehicles (UAV), water and energy consumption datasets (Eftelioglu et al.
2016b) for each individual field and meteorological datasets represents how the
agricultural data represents the variety aspect of the big data (Eftelioglu et al.
2016a). In addition, these datasets have huge volumes caused by the hundreds or
thousands of variables with high resolution over a large geographical area, multiplied by the finer temporal frequency of data collection. Finally, due to the advancements on cheap sensors and improved remote sensing capabilities thanks to the
UAVs, the data velocity is much faster than before. So far, this trend seems to continue to grow in the future.
To address the challenges of hard computational problems, as well as the issues,
arise with big data, several approaches are used.
First, from a hardware perspective, supercomputers were introduced to computer
science. These computers often have many CPUs or GPUs that are specialized in
doing a high number of parallel computations of smaller floating point or matrix
operations. For the most part, supercomputing (sometimes referred to as highE. Eftelioglu and S. Shekhar
2018, most users of Python are using package management tools (e.g., Anaconda n.d.)
to keep track of which packages are used in their application as well as help ease the
portability of their code. Also, Python has an option of using web browser- based IDEs
such as Jupyter notebooks which are getting increasingly popular due to their code
completion, portability, and presentation (web-based tutorials, etc.) capabilities.
Finally, it is worth noting that the scientific community is increasingly doing
their developments in the cloud because of the machine-independent development
capabilities as well as data management/storage options provided by these tools.
For example, Amazon Web Services (n.d.) provide data management by its
“Buckets (e.g., S3)” and its web-based development environment (e.g., AWS
Sagemaker) with ready to use data analytics tools embedded.
16.3 Challenges in Computer Science
As defined above, some computational problems are extremely hard to solve. In
addition, those may require large storage space to solve the problem. To make things
worse, “Big Data,” which is often defined by four V’s, namely, volume, variety,
velocity, and value, adds an additional challenge to the current approaches. Volume
refers to the increasing amount of data that are collected from different sources.
Variety refers to the different types of data that are collected from many sensors and
velocity refers to the speed of collection. Finally, the Value terms emphasize the
analytic application of the data and its potential associated value.
Most data related to FEW is Big Spatial Data with more challenging characteristics such as autocorrelation between samples, non-isotropy across dimensions, and
heterogeneity.
Agricultural datasets are great examples of big data (Bronson & Knezevic 2016).
These datasets which are often collected from multiple sources, including the soil
sensors in the field, remote sensing imagery from satellites as well as unmanned
aerial vehicles (UAV), water and energy consumption datasets (Eftelioglu et al.
2016b) for each individual field and meteorological datasets represents how the
agricultural data represents the variety aspect of the big data (Eftelioglu et al.
2016a). In addition, these datasets have huge volumes caused by the hundreds or
thousands of variables with high resolution over a large geographical area, multiplied by the finer temporal frequency of data collection. Finally, due to the advancements on cheap sensors and improved remote sensing capabilities thanks to the
UAVs, the data velocity is much faster than before. So far, this trend seems to continue to grow in the future.
To address the challenges of hard computational problems, as well as the issues,
arise with big data, several approaches are used.
First, from a hardware perspective, supercomputers were introduced to computer
science. These computers often have many CPUs or GPUs that are specialized in
doing a high number of parallel computations of smaller floating point or matrix
operations. For the most part, supercomputing (sometimes referred to as highE. Eftelioglu and S. Shekhar