sensing devices from multiple points within the production, transport and distribution chain and typically stored in a central repository.
To demonstrate this, Fig. 2 presents the data-to-information workflow typically
seen in the context of drinking water. Elements from the PDCA approach, as
analysed in §2.1, have been mapped, focusing on the Do-Check-Act-Plan parts of
the loop that describes the pathway from data to information (and, eventually at the
plan stage, knowledge). One may observe three distinct levels: acquisition of data
(level 1), followed by transformation and quality control (level 2) and finally
dissemination of the information produced by data (level 3). In the data acquisition
level, data is coming from sensors (possibly in real time) or can be fed from
periodical manual checks, such as local visits, regular sampling, etc. Such data can
be considered raw data, which means that they are stored as obtained by the sensors.
After acquisition, a common workflow inside a data warehouse is to upscale finescaled data through aggregation or averaging, in order to produce metrics and time
series at intervals meaningful to management or to identify extreme or periodic
events [32] and causal factors [1]. Prior to this step, raw data need to be cleaned of
errors belonging to the two types explained in Sect. 2.1. In the context of water
utilities, these errors could be due to sensor failure (maintenance problems, bias,
de-calibration, communication failure, physical damage due to catastrophes, etc.),
due to human mistakes (incorrect installation of measuring equipment, e.g. sensor
settings, unit conversions, not using the validation protocol issued by the manufacturer or forgetting registering information) and due to unexpected processes, phenomena and events in the monitored urban water system (electrical power outage,
failure of a pump).
Due to the numerous processes involved, data validation is not trivial but depends
on:
• The type of variable monitored
• The overall measurement and sensor/monitoring network conditions and more
specifically:
– The degree of system complexity
Larger systems may require larger sensor networks in this way more variables are measured simultaneously.
Sensors located far away from each other may be correlated or measure
completely different patterns with delays.
– The operational age of the sensor/monitoring network, which is translated in
the time length of available data
– The type and technology sensors/equipment used
Precision, accuracy, type of measurement, uncertainty of measurement
• The characteristics of the phenomenon being captured and more specifically:
– The type of problem (leakage detection, water balance closure, water quality,
etc.)
A Bird’s-Eye View of Data Validation in the Drinking Water Industry of the. . .
73
To demonstrate this, Fig. 2 presents the data-to-information workflow typically
seen in the context of drinking water. Elements from the PDCA approach, as
analysed in §2.1, have been mapped, focusing on the Do-Check-Act-Plan parts of
the loop that describes the pathway from data to information (and, eventually at the
plan stage, knowledge). One may observe three distinct levels: acquisition of data
(level 1), followed by transformation and quality control (level 2) and finally
dissemination of the information produced by data (level 3). In the data acquisition
level, data is coming from sensors (possibly in real time) or can be fed from
periodical manual checks, such as local visits, regular sampling, etc. Such data can
be considered raw data, which means that they are stored as obtained by the sensors.
After acquisition, a common workflow inside a data warehouse is to upscale finescaled data through aggregation or averaging, in order to produce metrics and time
series at intervals meaningful to management or to identify extreme or periodic
events [32] and causal factors [1]. Prior to this step, raw data need to be cleaned of
errors belonging to the two types explained in Sect. 2.1. In the context of water
utilities, these errors could be due to sensor failure (maintenance problems, bias,
de-calibration, communication failure, physical damage due to catastrophes, etc.),
due to human mistakes (incorrect installation of measuring equipment, e.g. sensor
settings, unit conversions, not using the validation protocol issued by the manufacturer or forgetting registering information) and due to unexpected processes, phenomena and events in the monitored urban water system (electrical power outage,
failure of a pump).
Due to the numerous processes involved, data validation is not trivial but depends
on:
• The type of variable monitored
• The overall measurement and sensor/monitoring network conditions and more
specifically:
– The degree of system complexity
Larger systems may require larger sensor networks in this way more variables are measured simultaneously.
Sensors located far away from each other may be correlated or measure
completely different patterns with delays.
– The operational age of the sensor/monitoring network, which is translated in
the time length of available data
– The type and technology sensors/equipment used
Precision, accuracy, type of measurement, uncertainty of measurement
• The characteristics of the phenomenon being captured and more specifically:
– The type of problem (leakage detection, water balance closure, water quality,
etc.)
A Bird’s-Eye View of Data Validation in the Drinking Water Industry of the. . .
73