Some Missed Gist of School Choice Report
By Patrick J. Wolfe and John F. Witte, Op-Ed in The Milwaukee Journal Sentinel
We released a set of five baseline reports on the Milwaukee Parental Choice Program last month, the first new studies of the voucher program using individual student data since 1995. Since then, many stories and commentaries have been published. Some of those contained inaccurate, incomplete or misleading information.
First, our research project is supported by a large consortium of philanthropies with diverse positions regarding school choice but a uniform commitment to non-interference in the research. We would not conduct this research under any other conditions. Our funders include the Annie E. Casey, Joyce, Kern, Lynde and Harry Bradley, Robertson and Walton Family foundations.
We listed this complete set of funding organizations at the start of each of our five reports. Unfortunately, Alan J. Borsuk's Feb. 26 Journal Sentinel story about the studies ("Voucher study finds parity,") reported the names of only three of the six philanthropies. The omission created a false impression - subsequently repeated by Mary Bell ("Voucher school achievements are still not measurable," March 8) - that the evaluation is primarily backed by "pro-voucher" foundations.
That is simply not true.
Second, no reliable conclusions about the effectiveness of the choice program can or should be drawn from these initial descriptive data. We provided that important guidance throughout our reports. Nevertheless, many commentators chose to ignore it.
Two of our reports included information about test score data from students in the voucher program and in Milwaukee Public Schools. The first such study, called the Annual School Testing Summary Report, merely described the performance of certain groups of choice students and placed that performance in context by a rough comparison to groups of MPS students. Such descriptive information is interesting but cannot be the basis of any sound conclusions about the effectiveness of the voucher program. It merely shows what is, not how it came to be.
Our Longitudinal Educational Growth Study eventually will provide reliable information about the effectiveness of the voucher program. It will do so by tracking the academic gains of carefully matched samples of choice and MPS students over time. To start the five-year study, we had to place the voucher and MPS students in our sample on an equal footing, academically. The test scores of the two groups were closely matched to each other, by design.
We have essentially placed the two groups at a common starting point. It would be absurd to determine the winner of a race based on the positions of the competitors at the starting line. Similarly, no one should draw conclusions about the performance of the voucher program based on information from the initial baseline year of a longitudinal study. We aren't finished yet. We have just started.
Third, we were neither ordered nor allowed to present student achievement information for individual private schools participating in the voucher program. Nothing in the state law that makes the study possible said that we, as the researchers collecting and analyzing the data, had to describe the performance levels of the choice students at individual private schools. Our procedures for protecting the privacy and confidentiality of the students in our study prevent us from revealing specific information about any students or schools by name. These procedures are required by our university's research oversight body.
As we discussed repeatedly in our reports, school-level test score data would not be useful for evaluating Milwaukee's school choice program or the private schools that enroll voucher students. Student performance at the school level is the product of an unknown mix of student background and school factors. That is why no statistical evaluation of school vouchers has ever connected student test score information to individually named schools. It is also why the U.S. Department of Education is now encouraging states to use measures of student gains over time to satisfy the accountability provisions under No Child Left Behind.
MPS has emerged as a leader in moving to gain scores and educational "value added" as metrics for evaluating their schools. We applaud that approach and seek to emulate it in our own five-year evaluation of the choice program.
As the summary report on the baseline year of our study concluded, "Stay tuned."
Patrick J. Wolf and John F. Witte are co-principal investigators of the new Milwaukee Parental Choice Program evaluation. Wolf is professor of education reform and 21st century chair in school choice at the University of Arkansas-Fayetteville. Witte is professor of political science and public affairs at the University of Wisconsin-Madison.
The above op-ed appeared in the March 16, 2008 Milwaukee Journal Sentinel