3 Outline of an Evaluation Framework: Evaluation
Components
A transparent evaluation should highlight the individual components as clearly as
possible and formulate them well, so that the argumentative steps can be understood
and assessed. To show how complex evaluation processes can be, the different
evaluation components are presented (Table 1).
The terms ‘evaluandum’ or ‘evaluand’ are commonly used in the literature.
Taylor introduces the term ‘evaluatum’ as the object “which is graded or ranked”
(Taylor 1961, 4). The object(s) to be evaluated is/are called evaluandum or
evaluanda. Frequently, items are evaluated that are selected to represent an entire
item type. The evaluation statements obtained are then generalised and the selection
of the representative object must meet certain requirements. The object actually
selected for evaluation is called the evaluatum. Evaluandum and evaluatum can, but
do not have to, coincide. In the following, the term ‘object of evaluation’ is used for
both the evaluandum and the evaluatum and it is assumed that they coincide.
The various components can and should each be considered in their specific
peculiarities, and evaluation research has concentrated accordingly on these various
aspects and formulated corresponding recommendations that should be observed for
successful evaluations (Rossi et al. 2006; House 1980). In view of the extensive
considerations involved, these cannot be dealt with in detail here (see Ott/Reinmuth
in this volume for the role of values in evaluations; in Reinmuth 2020 the evaluation
components and the evaluation of rule systems is discussed at greater length).
Rather, the focus will be on the various arguments that can be linked to evaluations.
Table 1 Evaluation components
1—Evaluator
An agent (actors and agencies, e.g. students of an event) evaluates
2—Evaluandum
An action, an object (a house, a meal), an actor (a person, a pupil, a
doctor), an event (outbreak of the 2nd WK), unreal circumstances (the
action of Zeus), abstract objects (rules, teaching contributions) or hybrid
forms [policies (normative-axiological systems), programs (rules, agents,
events)]
3—Point of view
From a moral, aesthetic, rational, legal,... point of view
4—Respect
Under axiological or normative regard/respect
5—Yardstick
According to a yardstick (...)
6—Criteria
According to certain criteria (...)
7—Standards
Using certain standards (. . .)
8—Data/Evidence
On the basis of existing data, facts, assumptions, information on (characteristics of the object of evaluation)
9—Degree of
fulfilment
By checking the degree to which the criteria defined by the standards have
been met
10—Evaluation
judgment
And arriving at an evaluation judgment on the basis of the degree of
fulfilment and the weighing of the criteria
11—Evaluation
proposition
And the agent affirmatively expresses her judgement, where appropriate,
in an evaluation proposition.
A Framework for Evaluation of Normative Solutions to Environmental Problems
111
Components
A transparent evaluation should highlight the individual components as clearly as
possible and formulate them well, so that the argumentative steps can be understood
and assessed. To show how complex evaluation processes can be, the different
evaluation components are presented (Table 1).
The terms ‘evaluandum’ or ‘evaluand’ are commonly used in the literature.
Taylor introduces the term ‘evaluatum’ as the object “which is graded or ranked”
(Taylor 1961, 4). The object(s) to be evaluated is/are called evaluandum or
evaluanda. Frequently, items are evaluated that are selected to represent an entire
item type. The evaluation statements obtained are then generalised and the selection
of the representative object must meet certain requirements. The object actually
selected for evaluation is called the evaluatum. Evaluandum and evaluatum can, but
do not have to, coincide. In the following, the term ‘object of evaluation’ is used for
both the evaluandum and the evaluatum and it is assumed that they coincide.
The various components can and should each be considered in their specific
peculiarities, and evaluation research has concentrated accordingly on these various
aspects and formulated corresponding recommendations that should be observed for
successful evaluations (Rossi et al. 2006; House 1980). In view of the extensive
considerations involved, these cannot be dealt with in detail here (see Ott/Reinmuth
in this volume for the role of values in evaluations; in Reinmuth 2020 the evaluation
components and the evaluation of rule systems is discussed at greater length).
Rather, the focus will be on the various arguments that can be linked to evaluations.
Table 1 Evaluation components
1—Evaluator
An agent (actors and agencies, e.g. students of an event) evaluates
2—Evaluandum
An action, an object (a house, a meal), an actor (a person, a pupil, a
doctor), an event (outbreak of the 2nd WK), unreal circumstances (the
action of Zeus), abstract objects (rules, teaching contributions) or hybrid
forms [policies (normative-axiological systems), programs (rules, agents,
events)]
3—Point of view
From a moral, aesthetic, rational, legal,... point of view
4—Respect
Under axiological or normative regard/respect
5—Yardstick
According to a yardstick (...)
6—Criteria
According to certain criteria (...)
7—Standards
Using certain standards (. . .)
8—Data/Evidence
On the basis of existing data, facts, assumptions, information on (characteristics of the object of evaluation)
9—Degree of
fulfilment
By checking the degree to which the criteria defined by the standards have
been met
10—Evaluation
judgment
And arriving at an evaluation judgment on the basis of the degree of
fulfilment and the weighing of the criteria
11—Evaluation
proposition
And the agent affirmatively expresses her judgement, where appropriate,
in an evaluation proposition.
A Framework for Evaluation of Normative Solutions to Environmental Problems
111