Use caution in reading Florida test validity study, author says
Florida's testing validity study, released Tuesday, seemed to have something for everybody within its 186 pages.
"Independent review confirms Florida Standards Assessment is valid," the Department of Education declared, saying the results would be used for school grades and teacher evaluations.
"Superintendents stand firm behind their initial position that the results of the Florida Standards Assessment (FSA) cannot fairly be used in teacher evaluations or to calculate A-F grades for public schools," the Florida Association of District School Superintendents asserted. "The report itself highlights and underscores the concerns of superintendents."
Andrew Wiley, senior pyschometrician for Alpine Testing Solutions, said the reviewers' intention was to "move people in the direction of using the entire report, and not just cherry picking information."
Within that report, Wiley said, the reviewers did find enough data to support using the test results at an aggregate level, such as school grades. However, he cautioned, many unknowns about the impacts to individual schools and students exist. That means more study would be appropriate, as the report suggests.
"We need to talk about how the aggregate scores can be used," he said. "It should be done with a lot of caution."
Wiley also shied from the idea of calling the test "valid."
"Validity is not a simple yes-no," he said.
He suggested that the more accurate conversation would focus on whether the interpretation and use of the test results is valid. That's why Alpine and its partner, edCount, issued several cautions within the report and recommended more work be done to improve the state's test and its administration.
"If Florida doesn't work hard, the danger is that cynicism (among many parents and educators) will continue to grow," Wiley said. "They need a plan to address the problems."
Alpine tried to stay away from making any policy recommendations, Wiley said, instead focusing its report on the details of the FSA, from the questions to outcomes. He acknowledged it can be easy to hone in on charts and highlights, or to fixate on the issues that support one position or another.
"I really would like to see people using the entire report," Wiley said.
Lawmakers, parents, school leaders and DOE officials have said they plan to continue seeking improvement to state testing, with the report as a guide. Stay tuned.