Once more on vouchers
A blog post last week about Florida's corporate tax credit scholarships drew a lot of comments on the Gradebook, with many of them saying that the report in question was comparing apples to oranges.
The researcher who wrote the report, David Figlio of Northwestern University, concluded that voucher students made similar academic gains in private schools as did students who were eligible for the program but remained in public schools. We reported that here. Well, that raised an obvious question - and I took all kinds of heat for not addressing it - how can you compare the voucher students, who mostly take the Stanford Achievement Test, with public school students, who take the FCAT?
If you read the report, Figlio addresses that question:
"An additional complication is that public school students no longer take a directly comparable nationally norm-referenced test, making comparisons across sectors somewhat more challenging. Through the 2007-08 academic year, public school students took both the criterion-referenced FCAT as well as the norm-referenced Stanford Achievement Test, but the norm-referenced test administration was ended due to budgetary concerns. That said, it is still possible to make comparisons between program participants and non-participants by performing an analysis of the concordance between FCAT scores and Stanford Achievement Test scores. In principle, a concordance analysis predicts what the norm-referenced national percentile would have been, given the level of the FCAT score."
Another point that was made in the original blog post, but appears to have been overlooked: Figlio isn't claiming a causal relationship based on comparisons of voucher students and public school students. He's not saying that vouchers are the magic bullet here. He's evaluating how the students in the voucher program are doing academically - and making what limited comparisons he can.
He's clear in the report that there are caveats. One, as noted above, the tests are different. Two, the students who use the vouchers tend to be poorer than the students who remained in public school but were eligible for the program and three, there's selection bias. Families choose whether to participate in the program.
"One reason to not interpret differences in test score gains between public school students and FTC Scholarship Program students as causal per se involves the fact that students and families choose whether to participate in the program, and these choices introduce "selection bias" into any comparison of test score gains."
Perhaps some of this should have been clearer in the original blog post. Vouchers are controversial and it seems that every new study is immediately lauded by one side (vouchers are great!) or derided by the other (it was class size, it was peer grouping, it wasn't vouchers!) Readers can, of course, decide for themselves.
On this study, here are some of the for and against responses. From Diane Ravitch, respected education historian, is this. And from RedefinED, the blog associated with Step up for Students (which administers the vouchers) is this.
(I confess, too, that I didn't immediately notice all the comments on my original blog until I read RedefinED today. I would have addressed the questions sooner, if I had. Busy week around here.)