4 Architecting IoT Cloud
209
• Data Filtering, Parsing, and Transformation – As data moves from source to
destination, Logstash is able to filter and parse each event and identify named
fields to generate structure and then transform them into a common format to
facilitate quicker analysis and added business value.
• Data Transportation – Uses various outputs to route data where needed, providing greater pliancy, and allows a deluge of downstream use cases.
Kibana Elasticsearch often includes Kibana, an open-source analytics and data
visualization platform used to search, view, interact, and visually manipulate data
via charts, maps, and tables that are housed in Elasticsearch indices. Although
citizen data scientists may also perform basic data processing and analytics, Kibana
is not considered as a holistic machine learning framework. Figure 4.25 illustrates
how Logstash, Elasticsearch, and Kibana can create a pipeline to ingest, analyze,
and visualize data.
4.6.1.6 Which Database Is Right for Your IoT Project?
Cassandra Greatest strengths include scalability without sacrificing reliability;
Cassandra can be deployed across multiple servers without extensive extra work
because it can replicate with minimal configuration. Cassandra is easy to set up and
maintain regardless of data growth. It is also best used in industries where rapid
database growth is needed. Cassandra does offer easier growth than MongoDB and,
in general, is best for the following use cases:
• Sensor logs
• User preferences
• Geographic information
• Reporting systems
• Time-series data
• Write-heavy applications such as logging
MongoDB MongoDB works best for workloads containing highly unstructured
data. If you are not able to anticipate the scale or type of data you will be using,
Fig. 4.25 Logstash, Elasticsearch, and Kibana pipeline
209
• Data Filtering, Parsing, and Transformation – As data moves from source to
destination, Logstash is able to filter and parse each event and identify named
fields to generate structure and then transform them into a common format to
facilitate quicker analysis and added business value.
• Data Transportation – Uses various outputs to route data where needed, providing greater pliancy, and allows a deluge of downstream use cases.
Kibana Elasticsearch often includes Kibana, an open-source analytics and data
visualization platform used to search, view, interact, and visually manipulate data
via charts, maps, and tables that are housed in Elasticsearch indices. Although
citizen data scientists may also perform basic data processing and analytics, Kibana
is not considered as a holistic machine learning framework. Figure 4.25 illustrates
how Logstash, Elasticsearch, and Kibana can create a pipeline to ingest, analyze,
and visualize data.
4.6.1.6 Which Database Is Right for Your IoT Project?
Cassandra Greatest strengths include scalability without sacrificing reliability;
Cassandra can be deployed across multiple servers without extensive extra work
because it can replicate with minimal configuration. Cassandra is easy to set up and
maintain regardless of data growth. It is also best used in industries where rapid
database growth is needed. Cassandra does offer easier growth than MongoDB and,
in general, is best for the following use cases:
• Sensor logs
• User preferences
• Geographic information
• Reporting systems
• Time-series data
• Write-heavy applications such as logging
MongoDB MongoDB works best for workloads containing highly unstructured
data. If you are not able to anticipate the scale or type of data you will be using,
Fig. 4.25 Logstash, Elasticsearch, and Kibana pipeline