xvi
people’s histories, and therefore their comprehensions, are unique, they are not chaotic. Similarities in peoples’ experiences lead to similarities in the resources they
develop. To borrow Wittgenstein’s term, there are family resemblances across people as to the resources they have developed in relation to certain linguistic, cultural,
and substantive patterns, and there are family resemblances across tasks as to the
resources people might bring to bear with regard to such patterns. The fit of an IRT
model and its person and item variables for a collection of responses to items from
people is a probabilistic pattern for the entire ensemble of data. The person variables
express, within this framework and data, tendencies of individuals—in the Rasch
model, for example—to perform well or poorly. The item variables are grand simplifications of patterns associated with individual items, looking across performances of all people to all items in the ensemble. Together, combining an item’s
variable with a person variable one person at a time approximates how a person with
that value would fare on that item.
Setting aside this IRT story for a moment, a sociocognitive perspective would
posit that given each person’s constellation of reading comprehension resources,
items tend to be harder or easier given, say, the complexity of the item’s syntax as it
relates to that individual’s history of experience. In contrast, in IRT, this is approximated across all persons by complexity as it applies in general, ignoring content,
context, and individuals’ histories. Similarly, from a sociocognitive perspective,
items tend to be harder or easier for a person as its vocabulary relates to that individual’s experience with those words and uses. In IRT, word frequencies from corpora take frequency across the texts as a proxy for word familiarity for each person.
Not really right, but better or worse in some applications, good enough for some
purposes in some contexts, and surely a step in the direction of understanding.
Similarity of experiences in a collection of persons that involve both the patterns
that are the target of a test and the myriad resources that are also necessary for performance brings us closer to both objectivity and intersubjectivity: What drives differences in persons’ performances is now mainly their resources for the targeted
capabilities, and what makes items difficult is similar for everyone involved. As
mentioned above, cognitive theory may further provide connections to typical processes and to features in items that predict their variables. Under these idealized
circumstances, the person variables of a suitable latent variable model are candidate
proxies for values of a property, perhaps further a property that may be argued as
measurable. Similar patterns in relationships among items may then arise more
widely across persons and tasks, enabling approximate calibrations across such
circumstances—a characteristic this book proposes that we require of measurement,
to add information beyond the observations at hand. However, we depart more often
and more substantially from these necessary conditions as persons become more
diverse, as tasks involve more varied knowledge patterns, and as performances
become more complex. In a simulation where every action can be logged, for example, we can observe differences among the knowledge and strategies people draw
on—differences previously hidden when only simple responses were recorded,
indeed differences that further question whether the underlying capabilities can be
characterized by different values of the same property.
Foreword
people’s histories, and therefore their comprehensions, are unique, they are not chaotic. Similarities in peoples’ experiences lead to similarities in the resources they
develop. To borrow Wittgenstein’s term, there are family resemblances across people as to the resources they have developed in relation to certain linguistic, cultural,
and substantive patterns, and there are family resemblances across tasks as to the
resources people might bring to bear with regard to such patterns. The fit of an IRT
model and its person and item variables for a collection of responses to items from
people is a probabilistic pattern for the entire ensemble of data. The person variables
express, within this framework and data, tendencies of individuals—in the Rasch
model, for example—to perform well or poorly. The item variables are grand simplifications of patterns associated with individual items, looking across performances of all people to all items in the ensemble. Together, combining an item’s
variable with a person variable one person at a time approximates how a person with
that value would fare on that item.
Setting aside this IRT story for a moment, a sociocognitive perspective would
posit that given each person’s constellation of reading comprehension resources,
items tend to be harder or easier given, say, the complexity of the item’s syntax as it
relates to that individual’s history of experience. In contrast, in IRT, this is approximated across all persons by complexity as it applies in general, ignoring content,
context, and individuals’ histories. Similarly, from a sociocognitive perspective,
items tend to be harder or easier for a person as its vocabulary relates to that individual’s experience with those words and uses. In IRT, word frequencies from corpora take frequency across the texts as a proxy for word familiarity for each person.
Not really right, but better or worse in some applications, good enough for some
purposes in some contexts, and surely a step in the direction of understanding.
Similarity of experiences in a collection of persons that involve both the patterns
that are the target of a test and the myriad resources that are also necessary for performance brings us closer to both objectivity and intersubjectivity: What drives differences in persons’ performances is now mainly their resources for the targeted
capabilities, and what makes items difficult is similar for everyone involved. As
mentioned above, cognitive theory may further provide connections to typical processes and to features in items that predict their variables. Under these idealized
circumstances, the person variables of a suitable latent variable model are candidate
proxies for values of a property, perhaps further a property that may be argued as
measurable. Similar patterns in relationships among items may then arise more
widely across persons and tasks, enabling approximate calibrations across such
circumstances—a characteristic this book proposes that we require of measurement,
to add information beyond the observations at hand. However, we depart more often
and more substantially from these necessary conditions as persons become more
diverse, as tasks involve more varied knowledge patterns, and as performances
become more complex. In a simulation where every action can be logged, for example, we can observe differences among the knowledge and strategies people draw
on—differences previously hidden when only simple responses were recorded,
indeed differences that further question whether the underlying capabilities can be
characterized by different values of the same property.
Foreword