A Validity Study of the NAEP Full Population Estimates

This paper documents research into the validity of imputation procedures used to calculate NAEP scores for excluded students and used to provide adjusted or full population estimates (FPEs) for NAEP assessments. The research used data from two special inclusion studies conducted in conjunction with the 2011 NAEP Mathematics Assessment in which operationally excluded students were assessed by offering them either a calculator accommodation or a test booklet made up of Knowledge and Skill Appropriate blocks that were easier than standard NAEP blocks.

By comparing actual test results (i.e., plausible values) and full population estimates for these operationally excluded students, the study was able to evaluate the extent of bias in the FPEs. Limitations of the study included the relatively small sample sizes yielded by the special inclusion studies, and the fact that the special inclusion samples were primarily from the higher end of the distribution of all excluded students.

The National Assessment of Educational Progress (NAEP) is the only nationally representative and continuing assessment of what America's students know and can do in various subject areas.