xiii
processes, perhaps opening a black box, to move from a preliminary model to one
that is more sophisticated, more explanatory. In one case, emerging theories and
laboratory discoveries led early chemists to abandon phlogiston as a property, let
alone a measurable one. In another, the quantum revolution revealed how classical
measurement of, say, position and velocity jointly fails for subatomic particles; yet
the classical model remains an excellent approximation for measuring classical
effects. Advances in technology and learning science have put educational assessment at such a crossroads. Is educational measurement, with RCA as an example,
more like phlogiston or a Newtonian measure of velocity?
Opening the Black Box in Educational Measurement
Educational assessment finds itself today in the midst of revolution on multiple
fronts, one of which bears most directly on educational measurement. Advances in
our understandings of the nature, acquisition, and use of human cognitive capabilities force reconception of how we might conceive psychosocial attributes that
underlie assessment. At the same time, advances in technology, which open the door
to new forms of instrumentation, provide opportunities for both improving theories
and enrichening applications.
Reading comprehension was a good running example for this book. All readers
will have personally experienced comprehension through reading, and most have
taken standardized RCA tests consisting of the familiar “read a text and answer
questions about it” items. They are intuitively plausible instruments for which performance depends on a presumed property (RCA) that has values, and reading capabilities of individual persons are each characterized by such a value. Test developers
learned to produce texts and questions that were fairly predictable in difficulty and
fairly reliable in sorting out people who fared better or worse overall. Test-takers’
overall scores, usually in the form of sums of correct answers, are considered to
indicate, if noisily, their RCAs.
As this book notes, many of the capabilities we address in educational assessment, hence the properties they may be hypothesized to measure, vary over time and
place. Tests of RCA obviously vary for assessing test-takers in different languages,
but also in different cultures within the same nominal language, and language use
itself shows evolution over the decades as to style, popularity of constructions,
vocabulary, and topics. What might this mean for “measuring RCA”? Most of the
networking certification examinations I worked on with Cisco expired after 2 years,
as new technologies come online and the meaning of “networking competence”
evolves. This is a social consideration in determining whether such examinations
might assess an existing property; if so, it is one that changes over time, in ways that
individuals may or may not. Adding simulation tasks into the certification exams
along with multiple-choice items redefined what was assessed and what candidates
studied. Cognitive considerations, I think, are even thornier.
Foreword
processes, perhaps opening a black box, to move from a preliminary model to one
that is more sophisticated, more explanatory. In one case, emerging theories and
laboratory discoveries led early chemists to abandon phlogiston as a property, let
alone a measurable one. In another, the quantum revolution revealed how classical
measurement of, say, position and velocity jointly fails for subatomic particles; yet
the classical model remains an excellent approximation for measuring classical
effects. Advances in technology and learning science have put educational assessment at such a crossroads. Is educational measurement, with RCA as an example,
more like phlogiston or a Newtonian measure of velocity?
Opening the Black Box in Educational Measurement
Educational assessment finds itself today in the midst of revolution on multiple
fronts, one of which bears most directly on educational measurement. Advances in
our understandings of the nature, acquisition, and use of human cognitive capabilities force reconception of how we might conceive psychosocial attributes that
underlie assessment. At the same time, advances in technology, which open the door
to new forms of instrumentation, provide opportunities for both improving theories
and enrichening applications.
Reading comprehension was a good running example for this book. All readers
will have personally experienced comprehension through reading, and most have
taken standardized RCA tests consisting of the familiar “read a text and answer
questions about it” items. They are intuitively plausible instruments for which performance depends on a presumed property (RCA) that has values, and reading capabilities of individual persons are each characterized by such a value. Test developers
learned to produce texts and questions that were fairly predictable in difficulty and
fairly reliable in sorting out people who fared better or worse overall. Test-takers’
overall scores, usually in the form of sums of correct answers, are considered to
indicate, if noisily, their RCAs.
As this book notes, many of the capabilities we address in educational assessment, hence the properties they may be hypothesized to measure, vary over time and
place. Tests of RCA obviously vary for assessing test-takers in different languages,
but also in different cultures within the same nominal language, and language use
itself shows evolution over the decades as to style, popularity of constructions,
vocabulary, and topics. What might this mean for “measuring RCA”? Most of the
networking certification examinations I worked on with Cisco expired after 2 years,
as new technologies come online and the meaning of “networking competence”
evolves. This is a social consideration in determining whether such examinations
might assess an existing property; if so, it is one that changes over time, in ways that
individuals may or may not. Adding simulation tasks into the certification exams
along with multiple-choice items redefined what was assessed and what candidates
studied. Cognitive considerations, I think, are even thornier.
Foreword