156
L. Carlsen
2.3 The More Elaborate Analyses
In addition to the basic partial ordering tools some more elaborate analyses have
been used including average ranks (Bubley and Dyer 1999; De Loof et al. 2006;
Bruggemann and Patil 2011) and sensitivity analysis (Bruggemann and Patil 2011;
Bruggemann et al. 2014), the latter gives an insight in the relative importance of the
included indicators (Bruggemann and Patil 2011; Bruggemann et al. 2014).
The average ranking is expressed as average height from bottom (min.
Height = 1) to the top (max height = n, i.e., the maximum number of objects,
here n = 6) (Bruggemann and Annoni 2014). The average rank is generated by
calculating all linear order preserving sequences (set LE), the “linear extensions
of the original partial order. From LE_0 the statistical characterization for each
object is obtained. For example the characterization is calculated as the average
value an object has, taken all positions of this object within LE_0, the averaged
heights. It is clear that this procedure is computationally extremely difficult. Hence,
approximations were developed.
For the sentivity analysis (Bruggemann and Patil 2011; Bruggemann et al. 2014),
let Q be the set of all indicators, then taken all indicators of Q leads to a partial
order, which is called PO_0. The corresponding set of linear extensions is denoted
by LE_0. Leaving out one indicator of Q, say r j , then another partial order results,
which is denoted as PO_j.
Both partial orders can be described by an adjacent matrix, say A_0 for PO_0
and A_j for PO_j.
Taken the Euclidian Distance (squared) quantifies the role of indicator qj in
PO_0. This is a sensitivity measure for the indicators of set Q, describing the
structural changes of the partial order leaving one indicator out. This is not
immediately a measure of the sensitivity of the indicators for a ranking, because
the ranking is per definition a linear order and here derived over many interim steps.
If a linear order is obtained by all orders in LE_0, the set of linear extensions
taken from PO_0, then any PO_j will also lead to a corresponding set LE_j. And
this set is the more differing from LE_0 the larger the sensitivity is. Therefore the
ranking due to averaged heights is as more affected by indicator r j as larger its
sensitivity is.
For detail information on the single tool the cited literature should be consulted
as detailed description is outside the scope of the present paper.
2.4 Software
All partial order analyses were carried out using the PyHasse software (Bruggemann
et al. 2014). PyHasse is programmed using the interpreter language Python (version
2.6) (Hetland 2005; Weigend 2006; Ernesti and Kaiser 2008; Langtangen 2008;
L. Carlsen
2.3 The More Elaborate Analyses
In addition to the basic partial ordering tools some more elaborate analyses have
been used including average ranks (Bubley and Dyer 1999; De Loof et al. 2006;
Bruggemann and Patil 2011) and sensitivity analysis (Bruggemann and Patil 2011;
Bruggemann et al. 2014), the latter gives an insight in the relative importance of the
included indicators (Bruggemann and Patil 2011; Bruggemann et al. 2014).
The average ranking is expressed as average height from bottom (min.
Height = 1) to the top (max height = n, i.e., the maximum number of objects,
here n = 6) (Bruggemann and Annoni 2014). The average rank is generated by
calculating all linear order preserving sequences (set LE), the “linear extensions
of the original partial order. From LE_0 the statistical characterization for each
object is obtained. For example the characterization is calculated as the average
value an object has, taken all positions of this object within LE_0, the averaged
heights. It is clear that this procedure is computationally extremely difficult. Hence,
approximations were developed.
For the sentivity analysis (Bruggemann and Patil 2011; Bruggemann et al. 2014),
let Q be the set of all indicators, then taken all indicators of Q leads to a partial
order, which is called PO_0. The corresponding set of linear extensions is denoted
by LE_0. Leaving out one indicator of Q, say r j , then another partial order results,
which is denoted as PO_j.
Both partial orders can be described by an adjacent matrix, say A_0 for PO_0
and A_j for PO_j.
Taken the Euclidian Distance (squared) quantifies the role of indicator qj in
PO_0. This is a sensitivity measure for the indicators of set Q, describing the
structural changes of the partial order leaving one indicator out. This is not
immediately a measure of the sensitivity of the indicators for a ranking, because
the ranking is per definition a linear order and here derived over many interim steps.
If a linear order is obtained by all orders in LE_0, the set of linear extensions
taken from PO_0, then any PO_j will also lead to a corresponding set LE_j. And
this set is the more differing from LE_0 the larger the sensitivity is. Therefore the
ranking due to averaged heights is as more affected by indicator r j as larger its
sensitivity is.
For detail information on the single tool the cited literature should be consulted
as detailed description is outside the scope of the present paper.
2.4 Software
All partial order analyses were carried out using the PyHasse software (Bruggemann
et al. 2014). PyHasse is programmed using the interpreter language Python (version
2.6) (Hetland 2005; Weigend 2006; Ernesti and Kaiser 2008; Langtangen 2008;