Indicators in the Framework of Partial Order
23
the risk of not considering fundamental aspects of the phenomenon. In general,
multiple indicators make it possible to measure conceptual dimensions with greater
precision (multiple measurements make it possible to compensate for random
errors), accuracy and discriminant capacity. But, using too many of them can lead
to errors and disturbances in the measurement (information noise, like redundancy).
The selection must always be guided by the conceptual model, the basis of each
system.
Dealing with a multi-indicators system also raises the question of finding tools
and methods that allow us to analyse how they are related to each other. The
increasing dissemination and use of indicators in recent years have highlighted
the weakness of traditional approaches for their treatment. The multi-indicators
systems require approaches allowing more concise views able to summarizing the
complexity. In this case, using traditional statistical techniques (in particular, those
of dimensional reduction: principal component analysis, factor analysis, etc.) is
not functional. The guiding concept, as previously written, crossing all possible
strategies is synthesis (Maggino 2017b). It may concern two different aspects of
the system (Maggino 2009), the units (which aims at aggregating the individuals
value of one indicator observed at micro level; this synthesis should allow the
created macro units to be compared – social groups, age groups, geographic areas –
with reference to the indicators of interest) or the basic indicators (which aims at
aggregating the values referring to several indicators for each unit, micro or macro).
Focusing on the latter, synthesis can be faced through two different approaches,
aggregative-compensative and non-aggregative. The aggregative-compensative
approach 3 is the mainstream method to the synthesis. In fact, the term “aggregation”
is often use as a synonym of synthesis is and, implicitly or not, it is generally taken
for granted that “evaluation implies aggregation”. However, many critical issues
affect this approach. First of all, the treatment of ordinal data. To be aggregated and
processed in an effective way, we must consider them as “numbers”; thus, they must
be scaled to numerical values. Unfortunately, this often turns out to be inconsistent
with the nature of phenomena and produces results that may be largely arbitrary,
poorly meaningful and hardly interpretable.
Another question regards the relationships among indicators. According to
Fattore and Maggino (2014), it is clear that many data systems available to social
scientists often comprise weakly interdependent attributes; this situation is a major
obstacle to effective synthesis through aggregative procedures. The composite
indicators approach results inappropriate in these cases, because all its procedures
are aggregative and (even partially) compensative. Another problem is linked to the
interpretation of results. The values assumed by composite indexes often tend to
be representative of situations profoundly different from each other, as a result of
3 For a review of the aggregative-compensative approach, please see Maggino 2017b, Mazziotta
and Pareto 2017.
23
the risk of not considering fundamental aspects of the phenomenon. In general,
multiple indicators make it possible to measure conceptual dimensions with greater
precision (multiple measurements make it possible to compensate for random
errors), accuracy and discriminant capacity. But, using too many of them can lead
to errors and disturbances in the measurement (information noise, like redundancy).
The selection must always be guided by the conceptual model, the basis of each
system.
Dealing with a multi-indicators system also raises the question of finding tools
and methods that allow us to analyse how they are related to each other. The
increasing dissemination and use of indicators in recent years have highlighted
the weakness of traditional approaches for their treatment. The multi-indicators
systems require approaches allowing more concise views able to summarizing the
complexity. In this case, using traditional statistical techniques (in particular, those
of dimensional reduction: principal component analysis, factor analysis, etc.) is
not functional. The guiding concept, as previously written, crossing all possible
strategies is synthesis (Maggino 2017b). It may concern two different aspects of
the system (Maggino 2009), the units (which aims at aggregating the individuals
value of one indicator observed at micro level; this synthesis should allow the
created macro units to be compared – social groups, age groups, geographic areas –
with reference to the indicators of interest) or the basic indicators (which aims at
aggregating the values referring to several indicators for each unit, micro or macro).
Focusing on the latter, synthesis can be faced through two different approaches,
aggregative-compensative and non-aggregative. The aggregative-compensative
approach 3 is the mainstream method to the synthesis. In fact, the term “aggregation”
is often use as a synonym of synthesis is and, implicitly or not, it is generally taken
for granted that “evaluation implies aggregation”. However, many critical issues
affect this approach. First of all, the treatment of ordinal data. To be aggregated and
processed in an effective way, we must consider them as “numbers”; thus, they must
be scaled to numerical values. Unfortunately, this often turns out to be inconsistent
with the nature of phenomena and produces results that may be largely arbitrary,
poorly meaningful and hardly interpretable.
Another question regards the relationships among indicators. According to
Fattore and Maggino (2014), it is clear that many data systems available to social
scientists often comprise weakly interdependent attributes; this situation is a major
obstacle to effective synthesis through aggregative procedures. The composite
indicators approach results inappropriate in these cases, because all its procedures
are aggregative and (even partially) compensative. Another problem is linked to the
interpretation of results. The values assumed by composite indexes often tend to
be representative of situations profoundly different from each other, as a result of
3 For a review of the aggregative-compensative approach, please see Maggino 2017b, Mazziotta
and Pareto 2017.