92
to study (which is sometimes termed a signification purpose of a test; Wilson, 2018).
Thus, it could be argued that the interpretation of validity as being about whether a
test measures what it claims to measure is a special case of a broader focus on test
score interpretations and uses, as advocated by Messick and Kane. But especially
insofar as many test interpretations and uses do at least appear to depend on measurement claims, it is still necessary to articulate and justify claims about measurement separately and in addition to other claims about test interpretation and use
more broadly.
4.3.4 Causal perspectives on validity
In contrast to the perspective of Messick, Kane, and the Standards, other recent
scholarship on validity has more strongly emphasized understanding its semantics
in terms of (a) factual claims about true states of affairs, rather than judgments based
on available evidence, and (b) measurement, rather than interpretations and uses
more broadly. In particular, Borsboom and colleagues have developed an account of
validity that could be regarded as an extension of the earliest definition of the term
(i.e., validity is whether a test measures what it claims to measure): specifically, “a
test is a valid [measuring instrument] of a [property] if (a) the [property] exists and
(b) variation in the [property] causes variation in the outcomes of the test” (Borsboom
& Mellenbergh, 2004).
17
This perspective on validity emphasizes that whether or
not a test is valid as a measuring instrument of a property is a claim about the state
of affairs in the world, and its truth or falsity is independent of the evidence available at any given time, or the extent to which that evidence is found to be persuasive
by any given community of observers.
18
Borsboom and colleagues’ perspective on validity is also the most compatible
with the framework presented in this volume, where validity can be understood
(using terminology to be defined more precisely in later chapters) in terms of the
distinction between the intended and effective property measured by an instrument,
with ideal validity being definable as a perfect union between the two.
However, it seems fair to say that Borsboom and colleagues’ view still stands
outside the mainstream of thinking and discourse about validity (see, e.g., Newton,
2012, for a discussion). The dominant trends in thinking about validity over the
twentieth and early twenty-first centuries appear to roughly follow at least some
aspects of the historical progression of thinking about measurement found in the
philosophical and metrological literatures (and as described in previous sections of
17 This definition of could be viewed as a re-statement of what in Sect. 3.2.1 was referred
to as non-null instrument sensitivity.
18 Consistently with this perspective, Wilson (2005) has advocated that instrument development
efforts in the human sciences focus on the development of the definition of the property, and then
the specification of theory regarding how this property is related to test outcomes. This perspective
is explored further by Wilson (2013), and in Chap. 7 of this book.
4 Philosophical perspectives on measurement
to study (which is sometimes termed a signification purpose of a test; Wilson, 2018).
Thus, it could be argued that the interpretation of validity as being about whether a
test measures what it claims to measure is a special case of a broader focus on test
score interpretations and uses, as advocated by Messick and Kane. But especially
insofar as many test interpretations and uses do at least appear to depend on measurement claims, it is still necessary to articulate and justify claims about measurement separately and in addition to other claims about test interpretation and use
more broadly.
4.3.4 Causal perspectives on validity
In contrast to the perspective of Messick, Kane, and the Standards, other recent
scholarship on validity has more strongly emphasized understanding its semantics
in terms of (a) factual claims about true states of affairs, rather than judgments based
on available evidence, and (b) measurement, rather than interpretations and uses
more broadly. In particular, Borsboom and colleagues have developed an account of
validity that could be regarded as an extension of the earliest definition of the term
(i.e., validity is whether a test measures what it claims to measure): specifically, “a
test is a valid [measuring instrument] of a [property] if (a) the [property] exists and
(b) variation in the [property] causes variation in the outcomes of the test” (Borsboom
& Mellenbergh, 2004).
17
This perspective on validity emphasizes that whether or
not a test is valid as a measuring instrument of a property is a claim about the state
of affairs in the world, and its truth or falsity is independent of the evidence available at any given time, or the extent to which that evidence is found to be persuasive
by any given community of observers.
18
Borsboom and colleagues’ perspective on validity is also the most compatible
with the framework presented in this volume, where validity can be understood
(using terminology to be defined more precisely in later chapters) in terms of the
distinction between the intended and effective property measured by an instrument,
with ideal validity being definable as a perfect union between the two.
However, it seems fair to say that Borsboom and colleagues’ view still stands
outside the mainstream of thinking and discourse about validity (see, e.g., Newton,
2012, for a discussion). The dominant trends in thinking about validity over the
twentieth and early twenty-first centuries appear to roughly follow at least some
aspects of the historical progression of thinking about measurement found in the
philosophical and metrological literatures (and as described in previous sections of
17 This definition of
to as non-null instrument sensitivity.
18 Consistently with this perspective, Wilson (2005) has advocated that instrument development
efforts in the human sciences focus on the development of the definition of the property, and then
the specification of theory regarding how this property is related to test outcomes. This perspective
is explored further by Wilson (2013), and in Chap. 7 of this book.
4 Philosophical perspectives on measurement