specific case study, the data used and the test applied are presented. Taking advantage of feedback sessions, it was possible to implement additional expert knowledge
in the determination of faulty data of two utilities. Only the analysis for Company A
will be discussed in this chapter.
4.2 Data and Proposed Data Validation for Company A
The utility currently monitors 73,000 variables (simultaneously) for drinking water
and 23,000 for wastewater. It is intended here to compare current Company A’s data
validation with a proposed data validation. For comparison purposes, the proposed
data validation is composed of simple tests. The four tests are:
• Verification of boundaries
• Verification of timestamps
• Verification of flat values
• Verification of jumps
Data was collected from two sources the Pumping Stations (PS) for the whole
system and Water Quality (WQ) data at Waste Water Treatment Plant (WWTP) I and
WWTP II. Data from the pumping stations was analysed only for the identification
Table 2 Overview of data and techniques used for validation
Company Datasets
Remarks
Action
Current
validation
Techniques
Company
A
Water
quality
WWTP I
production
One location:
Temperature, pH,
turbidity. Company A has a built
up system that
labels the data
with different
flags
Compare identification of
anomalies with
own system and
with registration
of maintenance
activities or
reported
incidents
Data is validated automatically by
the system
Simple test
confusion
matrix
Flow
City A
Pump
stations
Large-scale City
A. 5 pumping
stations. Company A has a built
up system that
labels the data
with different
flags
Data is
Validated
automatically
by the system
Data aggregation
regression
Knowledge
based
Energy
City A
Energy
provider
Large-scale City
A. 5 pumping
stations. Data
comes from a
third party. No
flags available for
this data
Data from a
third party not
validated by
Company A
Data aggregation
regression
Knowledge
based
A Bird’s-Eye View of Data Validation in the Drinking Water Industry of the. . .
85
in the determination of faulty data of two utilities. Only the analysis for Company A
will be discussed in this chapter.
4.2 Data and Proposed Data Validation for Company A
The utility currently monitors 73,000 variables (simultaneously) for drinking water
and 23,000 for wastewater. It is intended here to compare current Company A’s data
validation with a proposed data validation. For comparison purposes, the proposed
data validation is composed of simple tests. The four tests are:
• Verification of boundaries
• Verification of timestamps
• Verification of flat values
• Verification of jumps
Data was collected from two sources the Pumping Stations (PS) for the whole
system and Water Quality (WQ) data at Waste Water Treatment Plant (WWTP) I and
WWTP II. Data from the pumping stations was analysed only for the identification
Table 2 Overview of data and techniques used for validation
Company Datasets
Remarks
Action
Current
validation
Techniques
Company
A
Water
quality
WWTP I
production
One location:
Temperature, pH,
turbidity. Company A has a built
up system that
labels the data
with different
flags
Compare identification of
anomalies with
own system and
with registration
of maintenance
activities or
reported
incidents
Data is validated automatically by
the system
Simple test
confusion
matrix
Flow
City A
Pump
stations
Large-scale City
A. 5 pumping
stations. Company A has a built
up system that
labels the data
with different
flags
Data is
Validated
automatically
by the system
Data aggregation
regression
Knowledge
based
Energy
City A
Energy
provider
Large-scale City
A. 5 pumping
stations. Data
comes from a
third party. No
flags available for
this data
Data from a
third party not
validated by
Company A
Data aggregation
regression
Knowledge
based
A Bird’s-Eye View of Data Validation in the Drinking Water Industry of the. . .
85