A Practical Guide on Designing and Conducting Impact Studies in Education: Lessons Learned From the What Works Clearinghouse (Phase I)

Drawing on our five years of experience developing What Works Clearinghouse (WWC) evidence standards and reviewing studies against those standards as well as current literature on the design of impact studies, we highlight in this paper some of the most critical issues in designing and conducting impact studies in education, and provide practical guidance on addressing these issues and avoiding common pitfalls.

This paper serves as a quick reference guide for researchers who design and conduct impact studies in education and related fields. Information presented in this paper may also help research users make better informed judgments about the quality of the research and the credibility of the evidence produced from the research.

The WWC Phase I Computation Tools contains a set of Excel programs that can be used to compute effects sizes under various scenarios and implement corrections for clustering and multiple comparisons based on findings from impact studies in education and related fields. The computations contained in these Excel programs are based on the Technical Details of WWC-Conducted Computations document created during the first phase of the WWC.