xviii
Conclusion
Some 30 years ago, a different cross-disciplinary quest for fundamental concepts
across disparate fields transformed my thinking and my career. It was David
Schum’s decades-long inquiry into whether there might exist “a science of
evidence”,
4
underlying the reasoning under uncertainty found in varied forms and
sources of evidence, concerning disparate inferences and purposes, spanning philosophy, logic, probability, statistics, history, medicine, psychology, and other disciplines. The answer is yes, he concluded. His 1994 text The Evidential Foundations
of Probabilistic Reasoning is the most complete presentation of his findings.
5
I mentioned my interests in task design, validity, and cognition and assessment.
It turns out that all of these can be seen through the lenses of fundamental argumentation structures and reasoning principles, as applied to the evidence, the substance,
and the context of the domain of educational assessment. Familiar assessment practices and analytic methods can be re-understood as applications of these more basic
structures, likewise the concepts of reliability, validity, comparability, and generalizability. We can use the same framework to address the challenges and opportunities that developments in psychology, technology, and analytics continually present.
In each instance, there is serious work to be done: marshalling evidence, constructing arguments, building theory, generating and exploring alternative explanations. It
is these foundational structures that embody the deep principles of evidence and
inference that underlie educational assessment.
If we are to claim that the use of an assessment is producing measurements of
cognitive attributes (or at least to demonstrate the extent to which the essential properties of measurement are being approximated), then we must integrate (a) the fundamental structures of measurement, and (b) the suggestions for the kinds of
evidence and arguments, as this book lays them out, with these evidentiary-reasoning structures. We can, through this structure, come to understand whether, and if so
in what sense, and to what extent and in what contexts, an activity justifies the
interpretations and extrapolations that the very term educational measurement
implies. This book is that gratifying combination of painstaking philosophy, with
beneficial consequences, and a structure that supports the work that must be done in
a field like mine that can surely use it.
Educational Testing Service
Robert J. Mislevy
Princeton, NJ, USA
4 Schum, D.A. (2009). A science of evidence: Contributions from law and probability. Law,
Probability and Risk, 8(3), 197–231.
5 Schum, D.A. (1994). The evidential foundations of probabilistic reasoning. New York: Wiley.
Foreword
Conclusion
Some 30 years ago, a different cross-disciplinary quest for fundamental concepts
across disparate fields transformed my thinking and my career. It was David
Schum’s decades-long inquiry into whether there might exist “a science of
evidence”,
4
underlying the reasoning under uncertainty found in varied forms and
sources of evidence, concerning disparate inferences and purposes, spanning philosophy, logic, probability, statistics, history, medicine, psychology, and other disciplines. The answer is yes, he concluded. His 1994 text The Evidential Foundations
of Probabilistic Reasoning is the most complete presentation of his findings.
5
I mentioned my interests in task design, validity, and cognition and assessment.
It turns out that all of these can be seen through the lenses of fundamental argumentation structures and reasoning principles, as applied to the evidence, the substance,
and the context of the domain of educational assessment. Familiar assessment practices and analytic methods can be re-understood as applications of these more basic
structures, likewise the concepts of reliability, validity, comparability, and generalizability. We can use the same framework to address the challenges and opportunities that developments in psychology, technology, and analytics continually present.
In each instance, there is serious work to be done: marshalling evidence, constructing arguments, building theory, generating and exploring alternative explanations. It
is these foundational structures that embody the deep principles of evidence and
inference that underlie educational assessment.
If we are to claim that the use of an assessment is producing measurements of
cognitive attributes (or at least to demonstrate the extent to which the essential properties of measurement are being approximated), then we must integrate (a) the fundamental structures of measurement, and (b) the suggestions for the kinds of
evidence and arguments, as this book lays them out, with these evidentiary-reasoning structures. We can, through this structure, come to understand whether, and if so
in what sense, and to what extent and in what contexts, an activity justifies the
interpretations and extrapolations that the very term educational measurement
implies. This book is that gratifying combination of painstaking philosophy, with
beneficial consequences, and a structure that supports the work that must be done in
a field like mine that can surely use it.
Educational Testing Service
Robert J. Mislevy
Princeton, NJ, USA
4 Schum, D.A. (2009). A science of evidence: Contributions from law and probability. Law,
Probability and Risk, 8(3), 197–231.
5 Schum, D.A. (1994). The evidential foundations of probabilistic reasoning. New York: Wiley.
Foreword