Table of Contents | Search Technical Documentation | References
NAEP Technical DocumentationTreatment of Items in NAEP to Avoid Violations of Assumptions
NAEP provides information about the treatment of specific items in NAEP scales and evidence of differential item functioning (DIF) for specific items. Using current methodologies in psychometrics, the assumption of
conditional independence and the assumption that the data fit the
Item Response Theory (IRT) models are examined and controlled in NAEP in several ways. The assumptions of conditional independence and IRT model-fit are examined by considering the results of
DIF analyses,
item fit statistics, and
plots of empirical and theoretical item response functions. They are controlled by
treating missing and "not reached" responses appropriately, maintaining the
context and administration of items across assessments,
collapsing categories of
polytomous items when appropriate,
combining two or more items into a single item (cluster item), or making decisions about the inclusion or
deletion of an item in a scale based on data. The identification and amelioration of violations of IRT assumptions is an area of ongoing research in educational measurement. For example, studies have investigated local item dependence (Yen, 1993), assessed the fit of the item response function (Donoghue & Hombo, 1999; Hombo, Donoghue, & Thayer, 2000; Orlando & Thissen, 2000) and item parameter drift (Donoghue & Isham, 1998), and detected and described multidimensionality (e.g., Roussos, Stout, & Marden, 1998; Zhang & Stout, 1999).
Last updated 24 July 2024 (SK)