352
N. Balac
And type the text you would like to be counted – possible example:
Spark streaming is amazing!
This is a great example of a spark streaming application
Run
Execute
Run
#In another terminal run the
$ spark-submit mynetworkcount.py localhost 9999
The output on the screen will indicate the number of words counted.
This example illustrates the Spark Streaming process of ingesting data into a
Discretized Streaming framework. DStreams enable users to capture the data and
perform many different types of computations, as illustrated in this example by a
simple word count of the incoming data set. DStreaming and RDDs are a crucial set
of building blocks that enable construction of complex streaming applications with
Spark and Spark Streaming.
6.5 Big Data Analytics: Building the Data Pipeline
Several different maturity levels can be considered with regard to Big Data
Analytics. There are a number of organizations (DAMM, Gartner, IIA, HIMMS,
TDWI, IBM, etc.) that have defined their own version of analytics maturity
levels. However, they all agree on three general tiers. All organizations start with
raw data and move first to cleaned, standardized, and organized data. They next
progress to basic and advanced reporting. Finally, they may finally graduate to
building predictive models. This process highlights the levels of sophistication
in analytics moving from descriptive, to diagnostic, to predictive, and finally to
prescriptive modeling. Descriptive analytics help understand what has happened in
the past, while diagnostic analytics looks into reasons of why something might have
happened. Predictive analytics techniques build machine learning models to predict
what will happen. These models can then be fed into prescriptive models, which take
the process directly to decision making and action by recommending what should
be done under certain conditions.
6.5.1 Developing Predictive and Prescriptive Models
John Naisbitt famously said, “We are drowning in data, but starving for knowledge!”
A great quote that is made more amazing when one considers that it was made
in 1982. His observation rings ever more true today. While the scale of data has
changed, the need for skills, tools, and techniques to find meaning in the mayhem
of the Big Data world has not. It is costly to collect, store, and secure Big Data
properly, and real return on investment (ROI) hinges on the ability to extract
N. Balac
And type the text you would like to be counted – possible example:
Spark streaming is amazing!
This is a great example of a spark streaming application
Run
Execute
Run
#In another terminal run the
$ spark-submit mynetworkcount.py localhost 9999
The output on the screen will indicate the number of words counted.
This example illustrates the Spark Streaming process of ingesting data into a
Discretized Streaming framework. DStreams enable users to capture the data and
perform many different types of computations, as illustrated in this example by a
simple word count of the incoming data set. DStreaming and RDDs are a crucial set
of building blocks that enable construction of complex streaming applications with
Spark and Spark Streaming.
6.5 Big Data Analytics: Building the Data Pipeline
Several different maturity levels can be considered with regard to Big Data
Analytics. There are a number of organizations (DAMM, Gartner, IIA, HIMMS,
TDWI, IBM, etc.) that have defined their own version of analytics maturity
levels. However, they all agree on three general tiers. All organizations start with
raw data and move first to cleaned, standardized, and organized data. They next
progress to basic and advanced reporting. Finally, they may finally graduate to
building predictive models. This process highlights the levels of sophistication
in analytics moving from descriptive, to diagnostic, to predictive, and finally to
prescriptive modeling. Descriptive analytics help understand what has happened in
the past, while diagnostic analytics looks into reasons of why something might have
happened. Predictive analytics techniques build machine learning models to predict
what will happen. These models can then be fed into prescriptive models, which take
the process directly to decision making and action by recommending what should
be done under certain conditions.
6.5.1 Developing Predictive and Prescriptive Models
John Naisbitt famously said, “We are drowning in data, but starving for knowledge!”
A great quote that is made more amazing when one considers that it was made
in 1982. His observation rings ever more true today. While the scale of data has
changed, the need for skills, tools, and techniques to find meaning in the mayhem
of the Big Data world has not. It is costly to collect, store, and secure Big Data
properly, and real return on investment (ROI) hinges on the ability to extract