The Setting of Achievement Levels

Methods for NAEP standards setting

Different methods were used to set achievement levels in the various NAEP subjects assessed:

1. The achievement levels in mathematics were set in 1992, according to the modified Angoff process described in Appendix G to The 1992 NAEP Technical Report. The achievement levels were validated in 1996, according to the modified Angoff process described in Appendix F to The 1996 NAEP Technical Report. The achievement levels in mathematics were corrected for an information weighting error in 1996 as described in Appendix H to this technical report. Note that the 1992 and 1994 technical reports are not online; please Contact NAEP, specifying the publication and appendix needed.
2. The achievement levels in reading were set in 1992 and validated in 1994, according to the modified Angoff processes described in Appendix H to& The 1992 NAEP Technical Report, and in Appendix F to The 1994 NAEP Technical Report. The achievement levels in reading were also corrected for an information weighting error in 1996 as described in Appendix H to The 1996 NAEP Technical Report.
3. The achievement levels for the 1994 U.S. history and geography assessments were set according to the modified Angoff process described in Appendix G to The 1992 NAEP Technical Report.
4. Achievement level descriptions for the 1996 science assessment were developed for cut scores set by the National Assessment Governing Board, as described in Appendix G of The 1996 NAEP Technical Report.
5. The achievement levels for the 1998 civics and writing assessments were set according to the modified Angoff process (with Reckase chart feedback that encouraged the ratings to be consistent with the IRT scales) as described in Appendix J to The NAEP 1998 Technical Report.
6. The achievement levels for the 2005 grade 12 mathematics assessment were developed using a modified bookmark method with subscale feedback, as described in Developing Achievement Levels on the 2005 National Assessment of Educational Progress in Grade Twelve Mathematics: Process Report in the Governing Board publications about achievement levels.
7. The achievement levels for the 2007 grade 12 economics assessment were developed using a bookmark method, as described in Developing Achievement Levels on the 2006 National Assessment of Educational Progress in Grade Twelve Economics: Process Report in Governing Board publications.
8. The achievement levels for the 2009 science assessment were developed using a bookmark method, as described in Developing Achievement Levels on the 2009 National Assessment of Educational Progress in Science for Grades Four, Eight, and Twelve: Process Report and also the Technical Report, in Governing Board publications.
9. The achievement levels for the 2011 writing assessment were developed using a body of work method, as forecast in Developing Achievement Levels on the National Assessment of Educational Progress for Writing Grades 8 and 12 in 2011 and Grade 4 in 2013 in Governing Board publications.

Achievement levels have been defined for the NAEP economics, geography, mathematics, reading, and U.S. history composite scales and for univariate scales in civics, science, technology and engineering literacy (TEL), and writing.

The achievement levels for each subject area were adopted by the Board based on a standard-setting process that involved a cross-section of educators and interested citizens from across the nation as panelists. The panelists were asked to judge what students should know and be able to do relative to the body of content in the assessment framework for a given subject area. Details of the process used for setting achievement levels within each subject area are available from the National Assessment Governing Board website. Based on the results of the achievement-level-setting process, after the Board’s approval, a set of achievement-level cut scores were set for each subject area scale. The cut scores are the scores that define the boundaries between below NAEP Basic, NAEP Basic, NAEP Proficient, and NAEP Advanced performance levels at each grade. After the cut scores are set, the judges also select exemplar items that are good examples of the kinds of knowledge and skills that students in each achievement level can answer correctly.

More information about achievement levels

The NAEP technical documentation describes the achievement levels for each subject in the first chapter, Analysis and Scaling, in the section about Describing NAEP Scale Score Distributions.

Current legislation requires that NAEP achievement levels be considered trial; read about their status.