place during the period January–May 2018, as well as surveys using questionnaires
that were sent to representatives of drinking water companies that are part of the
Dutch joint research (BTO Bedrijfstakonderzoek in Dutch) [14].
Secondly, to gain insight on different approaches on data validation, a literature
review on faulty data detection techniques was performed, resulting in an overview
of available techniques that are relevant for the drinking water companies. This
overview differentiates between simple and complex techniques, and it also includes
the range of applications of each one. Based on the insight gained by the literature
review and the data gathered from the water companies, a data quality control is
proposed using simple techniques.
Based on the findings of the previous steps, an application in three cases focusing
on two types of problems in drinking water follows. The two problem types:
i. The detection of anomalies in volume flow rate, as an example of data validation
in water quantity
ii. Anomaly detection in datasets of temperature, turbidity and pH, as an example of
validation in water quality
The analysis of the case studies was performed in close cooperation with the
water companies. In this chapter, only data validation for one company, Company A,
is presented.
Finally, using the information collected from all previous steps, best practices and
issues regarding DQC by the water utilities are identified, as well as recommendations for future application of faulty detection techniques, along with ideas for future
research in the field of DQC in the Dutch drinking water sector.
1.3 Outline
A brief overview of the contents of the following sections is provided in this
paragraph. In §2, the necessary foundations and theoretical background in DQC is
defined, and an overview of current experiences of the drinking water companies is
provided. Having set the foundations, §3 contains the literature review on faulty data
detection techniques. This overview leads to a selection of techniques directly
applicable to Dutch water utilities, in Sect. 4, the form of a flowchart, to implement
simple techniques for DQC is presented.
In §4 are described the data obtained from the drinking water companies for
different studied cases and their results, derived from the application of simple
techniques following the proposed data validation. At the end of §4, the best
practices and issues found during the implementation of the cases is summarized.
Following the analysis, §5 contains the discussion and recommendations that highlight the need for future research. Finally, §6 describes the main conclusions drawn
from each case study, as well as general conclusions drawn from the application of
the methodology.
68
M. Castro-Gama et al.
that were sent to representatives of drinking water companies that are part of the
Dutch joint research (BTO Bedrijfstakonderzoek in Dutch) [14].
Secondly, to gain insight on different approaches on data validation, a literature
review on faulty data detection techniques was performed, resulting in an overview
of available techniques that are relevant for the drinking water companies. This
overview differentiates between simple and complex techniques, and it also includes
the range of applications of each one. Based on the insight gained by the literature
review and the data gathered from the water companies, a data quality control is
proposed using simple techniques.
Based on the findings of the previous steps, an application in three cases focusing
on two types of problems in drinking water follows. The two problem types:
i. The detection of anomalies in volume flow rate, as an example of data validation
in water quantity
ii. Anomaly detection in datasets of temperature, turbidity and pH, as an example of
validation in water quality
The analysis of the case studies was performed in close cooperation with the
water companies. In this chapter, only data validation for one company, Company A,
is presented.
Finally, using the information collected from all previous steps, best practices and
issues regarding DQC by the water utilities are identified, as well as recommendations for future application of faulty detection techniques, along with ideas for future
research in the field of DQC in the Dutch drinking water sector.
1.3 Outline
A brief overview of the contents of the following sections is provided in this
paragraph. In §2, the necessary foundations and theoretical background in DQC is
defined, and an overview of current experiences of the drinking water companies is
provided. Having set the foundations, §3 contains the literature review on faulty data
detection techniques. This overview leads to a selection of techniques directly
applicable to Dutch water utilities, in Sect. 4, the form of a flowchart, to implement
simple techniques for DQC is presented.
In §4 are described the data obtained from the drinking water companies for
different studied cases and their results, derived from the application of simple
techniques following the proposed data validation. At the end of §4, the best
practices and issues found during the implementation of the cases is summarized.
Following the analysis, §5 contains the discussion and recommendations that highlight the need for future research. Finally, §6 describes the main conclusions drawn
from each case study, as well as general conclusions drawn from the application of
the methodology.
68
M. Castro-Gama et al.