6 Big Data
319
Fig. 6.2 Big Data life cycle
Capture
Organize
Integrate
Analyze
Act
BIG
DATA
manage, and manipulate vast amounts of data at the right speed and at the right time
in order to gain the most valuable insights.
The data growth in the past decade has been unprecedented. There are 2.5 quintillion bytes of data created each day, and this will only increase more rapidly with
the growth of the Internet of Things (IoT): an estimated 50 billion IoT devices will
be connected by 2020 [2]. With IoT’s network of RFID tags, machines, appliances,
smartphones, buildings, and many other devices with embedded technology that can
be accessed over the Internet, estimates are projecting between 40 and 50 Zettabytes
(or 2 70 bytes equaling 1,180,591,620,717,411,303,424 bytes) of data will be created
by 2020. Over 92% of the data in the world was generated in the last 2 years alone.
A large contributor to this skyrocketing data generation are Internet applications like
Snapchat, YouTube, Twitter, and Instagram [3].
In order to convert the vast amount of available data into insight, it is important
to consider the functional requirements for Big Data. Figure 6.2 illustrates a set of
iterative steps in the Big Data functional requirements life cycle. Data first needs to
be captured, then organized, and integrated. The integration process includes data
cleaning and preparation. Once integrated, data can be analyzed in order to solve the
business problems at hand. Often times this analysis includes developing predictive
models utilizing Data Science approaches. Finally, the organization can act based
on the outcome of the Big Data analysis, frequently via the implementation and
utilization of the predictive models.
6.2 Big Data Management and Computing Platforms
As the volume and velocity of data grows, so grows the need to manage and process
it. Optimization, enabling the rapid formulation and testing of many diverse models
and real-time operations, becomes essential, especially in the case of streaming
data. Distributed and parallel processing approaches are well suited for these
319
Fig. 6.2 Big Data life cycle
Capture
Organize
Integrate
Analyze
Act
BIG
DATA
manage, and manipulate vast amounts of data at the right speed and at the right time
in order to gain the most valuable insights.
The data growth in the past decade has been unprecedented. There are 2.5 quintillion bytes of data created each day, and this will only increase more rapidly with
the growth of the Internet of Things (IoT): an estimated 50 billion IoT devices will
be connected by 2020 [2]. With IoT’s network of RFID tags, machines, appliances,
smartphones, buildings, and many other devices with embedded technology that can
be accessed over the Internet, estimates are projecting between 40 and 50 Zettabytes
(or 2 70 bytes equaling 1,180,591,620,717,411,303,424 bytes) of data will be created
by 2020. Over 92% of the data in the world was generated in the last 2 years alone.
A large contributor to this skyrocketing data generation are Internet applications like
Snapchat, YouTube, Twitter, and Instagram [3].
In order to convert the vast amount of available data into insight, it is important
to consider the functional requirements for Big Data. Figure 6.2 illustrates a set of
iterative steps in the Big Data functional requirements life cycle. Data first needs to
be captured, then organized, and integrated. The integration process includes data
cleaning and preparation. Once integrated, data can be analyzed in order to solve the
business problems at hand. Often times this analysis includes developing predictive
models utilizing Data Science approaches. Finally, the organization can act based
on the outcome of the Big Data analysis, frequently via the implementation and
utilization of the predictive models.
6.2 Big Data Management and Computing Platforms
As the volume and velocity of data grows, so grows the need to manage and process
it. Optimization, enabling the rapid formulation and testing of many diverse models
and real-time operations, becomes essential, especially in the case of streaming
data. Distributed and parallel processing approaches are well suited for these