89
One consequence of the focus on construct validity was increased attention to the
distinction between a test and the psychological property the test was designed to
measure, oftentimes referred to as a construct
14
(as opposed to the earlier, operationalist view that a test simply defined a property). This raised awareness of the possibility that multiple tests could measure the same property. Donald T. Campbell
and Donald W. Fiske (1959) popularized the idea of multimethod studies, and in so
doing added two new terms into the validity lexicon: convergent validity, reflecting
the idea that the results of measurements made with different instruments but of the
same property should exhibit high levels of agreement with one another (i.e., they
should “converge” on a common outcome), and discriminant validity, reflecting the
idea that the results of measurements of different properties should not be too highly
correlated with one another, even if they used similar kinds of instrumentation (i.e.,
it should be possible to empirically “discriminate” among theoretically distinct
properties). As a classic example, if extroversion and dominance are both assessed
via self-report and the reports of one’s family members, evidence for convergent
validity could take the form of showing that self-reports and family reports of extroversion are highly correlated (and similarly for dominance), and evidence for discriminant validity could take the form of showing that self-reports of extroversion
and self-reports of dominance are not so highly associated as to render them empirically redundant (and similarly for family reports).
4.3.3 An argument-based approach to validity
Starting in 1989, Samuel Messick offered a new perspective on validity that reflected
a significant shift from previous viewpoints in two important respects. First,
Messick’s view subsumed disparate lines of validity-related evidence under a generalized version of construct validity; thus, according to this view, (construct) validto specify how theoretical terms are defined implicitly through the role they play in networks of
lawful relations. However, as discussed briefly in Chap. 1, this project did not work for the simple
reason that “there were (and are) no nomological networks involving concepts like general intelligence” (Borsboom et al., 2009: p. 136).
14 Although many sources treat (or appear to treat) the term “construct” as synonymous with “property” (or at least “psychosocial property”), other sources also use it to refer to a concept or linguistic label that refers to a property (for a discussion, see, e.g., Slaney & Racine, 2013), and some
sources even do both simultaneously: for example, the Standards for Educational and Psychological
Testing (AERA, 2014), which are discussed further below, define a construct as “the concept or
characteristic that a test is designed to measure” (p. 11). To avoid confusion, we use the terms
“property” and “concept of property” rather than “construct”, except when specifically referring to
language used by others. We also discuss in Sect. 4.5 properties that are in some sense constructed
by human minds or human activities. Perhaps jarringly, in the terminology common in the human
sciences (and construct validity theory in particular), the term “construct” might or might not
imply that the property is thought of as having been constructed (see, e.g., Slaney, 2017).
4.3 The concept of validity in psychosocial measurement
One consequence of the focus on construct validity was increased attention to the
distinction between a test and the psychological property the test was designed to
measure, oftentimes referred to as a construct
14
(as opposed to the earlier, operationalist view that a test simply defined a property). This raised awareness of the possibility that multiple tests could measure the same property. Donald T. Campbell
and Donald W. Fiske (1959) popularized the idea of multimethod studies, and in so
doing added two new terms into the validity lexicon: convergent validity, reflecting
the idea that the results of measurements made with different instruments but of the
same property should exhibit high levels of agreement with one another (i.e., they
should “converge” on a common outcome), and discriminant validity, reflecting the
idea that the results of measurements of different properties should not be too highly
correlated with one another, even if they used similar kinds of instrumentation (i.e.,
it should be possible to empirically “discriminate” among theoretically distinct
properties). As a classic example, if extroversion and dominance are both assessed
via self-report and the reports of one’s family members, evidence for convergent
validity could take the form of showing that self-reports and family reports of extroversion are highly correlated (and similarly for dominance), and evidence for discriminant validity could take the form of showing that self-reports of extroversion
and self-reports of dominance are not so highly associated as to render them empirically redundant (and similarly for family reports).
4.3.3 An argument-based approach to validity
Starting in 1989, Samuel Messick offered a new perspective on validity that reflected
a significant shift from previous viewpoints in two important respects. First,
Messick’s view subsumed disparate lines of validity-related evidence under a generalized version of construct validity; thus, according to this view, (construct) validto specify how theoretical terms are defined implicitly through the role they play in networks of
lawful relations. However, as discussed briefly in Chap. 1, this project did not work for the simple
reason that “there were (and are) no nomological networks involving concepts like general intelligence” (Borsboom et al., 2009: p. 136).
14 Although many sources treat (or appear to treat) the term “construct” as synonymous with “property” (or at least “psychosocial property”), other sources also use it to refer to a concept or linguistic label that refers to a property (for a discussion, see, e.g., Slaney & Racine, 2013), and some
sources even do both simultaneously: for example, the Standards for Educational and Psychological
Testing (AERA, 2014), which are discussed further below, define a construct as “the concept or
characteristic that a test is designed to measure” (p. 11). To avoid confusion, we use the terms
“property” and “concept of property” rather than “construct”, except when specifically referring to
language used by others. We also discuss in Sect. 4.5 properties that are in some sense constructed
by human minds or human activities. Perhaps jarringly, in the terminology common in the human
sciences (and construct validity theory in particular), the term “construct” might or might not
imply that the property is thought of as having been constructed (see, e.g., Slaney, 2017).
4.3 The concept of validity in psychosocial measurement