Florida Department of Education takes issue with Miami-Dade superintendent's statement
After watching Miami-Dade superintendent Alberto Carvalho's 10-minute commentary on school accountability (see post below), Florida Department of Education spokeswoman Meghan Collins told the Gradebook she felt compelled to respond.
Collins said that Carvalho misstated some key elements that deserved clarification.
First, she took issue with the superintendent's characterization of Florida Standards Assessment questions as not aligned to state standards. That's a common refrain among many critics of the tests.
"It's worth noting that for all but three, there was a connection to a standard," Collins said. "A majority of items did have an exact match."
The state's recent Alpine Testing Solutions validity study has been the source for both positions. Carvalho looked to item match charts in the document that showed "no standard match" for as high as 33 percent of questions, depending on the level. Collins said the report indicated that the reviewers did not make identical matches to the intended standards, but they did see connection to related standards.
The report states for third-grade language arts that "Panelists selected a different standard than the intended standard for 33% of the items." Similar language was used for other grade levels.
Whether that's good enough remains the most critical point of contention.
Next, Collins rejected Carvalho's criticism of the state decision to use test questions written for and field tested on Utah students. This did occur, Collins said, but she added that Florida experts reviewed the questions and removed ones that were not appropriate.
The Alpine report states "all items were reviewed by educators knowledgeable of Florida students and the Florida Standards to evaluate whether the items were appropriate for use within the FSA program."
It also finds: "While alignment to Florida standards was confirmed for the majority of items reviewed via the item review study, many were not confirmed, usually because these items focused on slightly different content
within the same anchor standards. It would be more appropriate to phase-out the items originally developed for use in Utah and replace them with items written to specifically target the Florida standards."
Collins said the department did remove some questions, and is continuing that work.
She next turned to Carvalho's statement that the department did not ask for lists of students and schools that were impacted by computerized testing problems. She said that was not true, adding that commissioner Pam Stewart repeated the request at the meeting where superintendents announced their lack of confidence in the accountability system.
Independent testing experts including Alpine's have said knowing that impact could make a difference in how the state sets its school grades. Carvalho suggested those grades are likely to drop dramatically -- last year's A could become this year's C, he said -- as the state sets grades based on performance without learning gains.
He criticized the plans to move ahead with school grades.
Collins responded by noting that the school grading formula and rule have yet to be completed. "So we don't actually know if they have fallen," she said.
Several districts have made projections based on recommended cut scores. But those scores also have not been approved. Collins noted that the state has said all along that it would have the work completed by winter, so they are on track and not late, another complaint Carvalho raised.
He said the grades -- which he and others have argued should be called off -- will come so late that students will be testing again when they learned how they did last year. There's really no contradicting that.
Collins did reiterate the department's stance that it will, and must, issue grades. She pointed to Florida Statute 1008.34, which states that "schools shall be graded." Superintendents and others have turned to that same law to note it also requires the use of learning gains in setting grades.
Since those gains can't be calculated with a new test, they argue, the grades should be cancelled or, if needed, all be deemed "incomplete," seeing how the data is not all available.
Collins said the law contemplated incomplete data. It states: "If a school does not have at least 10 students with complete data for one or more of the components listed in subparagraphs (b)1. and 2., those components may not be used in calculating the school's grade."
Watch for this debate to continue as the Florida Board of Education meets later this month, and lawmakers meet for additional committee sessions.