14
J. Wittmann
All these steps of the design workflow have to be done with respect to the initial
intention for introducing the indicator. This argumentation follows in full analogy
the argumentation that good models have to be as detailed as necessary and as
abstract as possible to allow efficient model experiments on the one hand and to
meet the targets on the other hand. The same objective oriented design and usage
should be direct the design and the interpretation of indicators.
6 Test and Validation of Indicator Models
Even if the preceding strict distinction between the (dynamic) model of the real
system and the indicator and evaluation model based on it may be too theoretical for
some practical examples, it emphasizes that even for the simple use cases, at each
step of the argumentation and interpretation it must be differentiated whether this
step refers to an abstract representation of reality or whether it deals with evaluation
in the sense of an indicator model. The system model can do completely without
indicators and be executable, the indicator model as introduced in the previous
section, on the other hand, cannot work without the system model.
In addition to this differentiation in design, the differentiation between the system
model and the indicator model has a further advantage: firstly, it makes it obvious
that the indicator model must also be validated, and secondly, it constructively
determines the way in which such validation is to be carried out. Validation can
only take place according to the rules that apply to the validation of the system
model: There must be sufficient agreement with regard to the requirements between
the results derived from reality and the results generated with the aid of the indicator
model.
This throws us back to the fields of application and functionalities for indicators
classified at the beginning and hopefully also makes clear why this classification
work was carried out in advance. The validity of an indicator (system) can only be
validated in relation to the specific requirements to be specified for each individual
application. An abstract “better” or “worse” for one or the other indicator approach,
detached from the objective in the specific application, cannot exist.
Constructively and in addition to the work steps for the creation of an indicator,
the user has the task of specifying
(a) the purpose of the indicator
(b) the accuracy with which the indicator is intended to fulfil this purpose
(c) the examples to be used to test the correspondence between the results of the
indicator and reality.
In the field of software engineering, one speaks of the requirements for a system
(a), the degree to which these requirements must be fulfilled (b) and the test cases
with which the fulfilment of the requirements can/must be proven.
Strictly speaking, the task to develop an indicator begins much earlier than
normally assumed with an explicit specification of the validity criteria for the
J. Wittmann
All these steps of the design workflow have to be done with respect to the initial
intention for introducing the indicator. This argumentation follows in full analogy
the argumentation that good models have to be as detailed as necessary and as
abstract as possible to allow efficient model experiments on the one hand and to
meet the targets on the other hand. The same objective oriented design and usage
should be direct the design and the interpretation of indicators.
6 Test and Validation of Indicator Models
Even if the preceding strict distinction between the (dynamic) model of the real
system and the indicator and evaluation model based on it may be too theoretical for
some practical examples, it emphasizes that even for the simple use cases, at each
step of the argumentation and interpretation it must be differentiated whether this
step refers to an abstract representation of reality or whether it deals with evaluation
in the sense of an indicator model. The system model can do completely without
indicators and be executable, the indicator model as introduced in the previous
section, on the other hand, cannot work without the system model.
In addition to this differentiation in design, the differentiation between the system
model and the indicator model has a further advantage: firstly, it makes it obvious
that the indicator model must also be validated, and secondly, it constructively
determines the way in which such validation is to be carried out. Validation can
only take place according to the rules that apply to the validation of the system
model: There must be sufficient agreement with regard to the requirements between
the results derived from reality and the results generated with the aid of the indicator
model.
This throws us back to the fields of application and functionalities for indicators
classified at the beginning and hopefully also makes clear why this classification
work was carried out in advance. The validity of an indicator (system) can only be
validated in relation to the specific requirements to be specified for each individual
application. An abstract “better” or “worse” for one or the other indicator approach,
detached from the objective in the specific application, cannot exist.
Constructively and in addition to the work steps for the creation of an indicator,
the user has the task of specifying
(a) the purpose of the indicator
(b) the accuracy with which the indicator is intended to fulfil this purpose
(c) the examples to be used to test the correspondence between the results of the
indicator and reality.
In the field of software engineering, one speaks of the requirements for a system
(a), the degree to which these requirements must be fulfilled (b) and the test cases
with which the fulfilment of the requirements can/must be proven.
Strictly speaking, the task to develop an indicator begins much earlier than
normally assumed with an explicit specification of the validity criteria for the