A New Dimensionality Estimation Tool For Multiple Item Tests And A New Dif Analysis Paradigm Based On Multidimensionality And Construct Validity

Educational Measurement

Educational Measurement has been the bible in its field since the first edition was published by ACE in 1951. The importance of this fourth edition of Educational Measurement is to extensively update and extend the topics treated in the previous three editions. As such, the fourth edition documents progress in the field and provides critical guidance to the efforts of new generations of researchers and practitioners. Edited by Robert Brennan and jointly sponsored by the American Council on Education (ACE) and the National Council on Measurement in Education, the fourth edition provides in-depth treatments of critical measurement topics, and the chapter authors are acknowledged experts in their respective fields. Educational measurement researchers and practitioners will find this text essential, and those interested in statistics, psychology, business, and economics should also find this work to be of very strong interest. Topics covered are divided into three subject areas: theory and general principles; construction, administration, and scoring; and applications. The first part of the book covers the topics of validation, reliability, item response theory, scaling and norming, linking and equating, test fairness, and cognitive psychology. Part two includes chapters on test development, test administration, performance assessment, setting performance standards, and technology in testing. The final section includes chapters on second language testing, testing for accountability in K-12 schools, standardized assessment of individual achievement in K-12 schools, higher education admissions testing, monitoring educational progress, licensure and certification testing, and legal and ethical issues.

A New Dimensionality Estimation Tool for Multiple-item Tests and a New DIF Analysis Paradigm Based on Multidimensionality and Construct Validity

This thesis is concerned with two critical issues facing the testing industry today: dimensionality analysis and DIF (Differential Item Functioning) analysis. Chapter 1 develops the use of new dimensionally-sensitive proximity measures with Hierarchical Cluster Analysis and DIMTEST to estimate the dimensionality structure of tests. The results of simulation studies and real data analyses indicate that the new tool represents a significant step forward in the ability of dimensionality assessment tools to identify reliably the latent dimensionality structure of a set of items. Chapter 2 of the thesis develops a new DIF analysis paradigm that unifies the substantive and statistical DIF research camps by linking both camps to a theoretically sound and mathematically rigorous multidimensional conceptualization of DIF. The new paradigm is shown to have the potential to improve the understanding of the root causes of DIF through the testing of substantive DIF hypotheses, to reduce Type 1 error through a better understanding of the dimensionality of the matching criterion, and to increase power through the testing of bundles of items with similar content. Two new paradigm-based DIF analysis methods, one of which employed the new dimensionality estimation tool of Chapter 1, were described and applied to real data. The analyses demonstrated that the new paradigm-based methods offered insights that cannot be obtained from the standard one-item-at-a-time DIF analysis.

LSAT Dimensionality Analysis for the December 1991, June 1992, and October 1992 Administrations