|
The review examined 232 studies to determine the strength of effectiveness for 29 Comprehensive School Reform (CSR) models. According to the authors, the evidence supports six primary findings:
There are a limited quantity and quality of studies supporting CSR achievement effects—Only 12 reform models are supported by five or more studies of their achievement effects. Only 4 models have been the subject of five or more third-party studies that used comparison groups. More than 40% of the analyses have used some type of quasi-experimental control group. Only seven studies of 3 CSR models, or about 3% of all studies of the achievement effects associated with CSR, have generated evidence from randomized experiments. In addition, many of the studies did not present sufficient detail to allow for replication of the findings. The authors suggest that these limitations are due to the recent emergence of CSR models.
Overall effects of CSR are greater than the effects of other interventions designed to serve similar purposes and student and school populations—Overall, CSR schools can be expected to score one eighth of a standard deviation, or 2.5 NCEs, higher on achievement tests than non-CSR schools.
Differences in the effectiveness of CSR are largely due to unmeasured program-specific and school-specific differences in implementation—Whether a model requires a component such as ongoing staff professional development or specific curricular materials does little to predict achievement outcomes. Similarly, diverse characteristics of schools, level of technical support, or cost of a model do not help predict variability of effect. However, requiring the active involvement of parents and the local community in school governance and improvement activities tends to result in worse outcomes than models not requiring these activities.
Methodological differences across the studies yielded greater differences in effects than the general programmatic components of the CSR models—For example, studies performed by the developer yielded considerably stronger effects than studies performed by others. This factor raises questions of developer bias.
Models meeting the highest standard of evidence are the only CSR models to have clearly established that their effects are relatively robust and the models can be expected to improve test scores—Models meeting the standard for the Strongest Evidence of Effectiveness category are distinguished by the quantity and generalizability of their outcomes, quality of evidence, and reliable effects on achievement. These models are Direct Instruction, the School Development Program, and Success for All.
Number of years of model implementation has important implications for understanding CSR effects on achievement—The strong effects of CSR beginning after the 5th year of implementation may be explained in two ways: a potential cumulative impact of CSR or a self-selection artifact.
|
(9.56 MB) 
