Validity Study of the NAEP Mathematics Assessment: Grades 4 and 8

Moreica Ortega and Fran Stancavage; University of California, Berkeley: Phil Daro; University of Illinois: Lizanne DeStefano; University of Colorado at Boulder: Robert Linn

Executive Summary

Since its founding in 1963, the National Assessment of Education Progress (NAEP) has made a unique contribution to our understanding of American education. It is the only source of information on the educational attainment of all U.S. students, and it is the only vehicle through which states can compare the progress of their students against a common standard. The current main NAEP mathematics trend line extends back to 1990, although there have been two limited revisions to the framework and corresponding incremental changes in the item pool since that time. The NAEP mathematics framework was last updated in 2001 for the 2005 assessment.

In spring 2006, the NAEP Validity Studies (NVS) Panel was asked by National Center for Education Statistics (NCES) to undertake a validity study of the current NAEP mathematics assessment. In particular, NCES asked the NVS Panel to answer the following questions:

Does the NAEP framework offer reasonable content and skill-based coverage compared to the assessments of states and other nations?
Does the NAEP item pool and assessment design accurately reflect the NAEP framework?
Is NAEP mathematically accurate and not unduly oriented to a particular curriculum, philosophy, or pedagogy?
Does NAEP properly consider the spread of abilities in the assessable population?
Does NAEP provide information that is representative of all students, including students who are unable to demonstrate their achievements on the standard assessment?

Because the framework for grade 12 mathematics was under revision at the time, the validity study was limited to grades 4 and 8.