Are Florida's Level 5 students really 'highly likely to excel in the next grade level'?
Aiming to clarify what each Florida Standards Assessment score level means, the state Department of Education recently introduced a new score report that will go to parents.
Each level, 1-5, includes a brief description of what it represents. Level 1 is defined as "Inadequate: Highly likely to need substantial support for the next grade." Level 4 says, "Proficient: Likely to excel in the next grade."
To which Bob Schaeffer, the Sanibel Island-based public education director of FairTest, says, prove it.
As a rule, standardized tests are as good as what they're supposed to be used for. One key factor in that formula is their predictive validity, meaning, essentially, their ability to predict some future activity, such as success in a future grade level or on a similar, separate exam. Experts consider this criterion to be critical in the assessment process.
So when they see wording like "highly likely to excel in the next grade level" -- the language used to describe a Level 5 on the FSA -- they want to see the materials that back up the claim.
"It's hard to believe that such research exists," Schaeffer said via email, "because the FSA is so new that there is unlikely to be solid data about how students with a particular score perform in the next grade."
Department spokeswoman Meghan Collins answered that the state standards, by their very nature, are designed to ensure that students receive the necessary foundation to succeed from one grade level to the next. The test and its score levels help determine their success, she said.
"We know that all parents and guardians want to know how well their students have mastered the content they are expected to know, but we also understand that most are juggling a variety of other responsibilities, making time a precious commodity," Collins said via email. "Therefore, we sought to develop a score report that offers students and their families a high-level explanation, with links for those who need a more in-depth understanding of the report."
She pointed to a 239-page document, 2015 Achievement Level Descriptions, as a place where the detail-oriented could head for specifics. "Additionally, Department of Education staff is available to answer questions about any component of the Florida Standards Assessments," Collins said.
Schaeffer suggested that Collins' response missed the point. For one, he said, most parents are unlikely to sift through a lengthy document to determine the state's basis for a score, and whether it's accurate.
Moreover, he noted, the document to which Collins referred contained no references to the wording of "likely to excel," "inadequate" or the other terms.
"Thus, my question remains unanswered," Schaeffer said. "What is the evidence for the Department of Education's predictive validity claims about proficiency levels? Without such evidence, DOE is publishing unsupported 'propaganda,' not offering sound educational guidance."
The State Board of Education approved the cut scores last week. Score reports and school grades are due out in February.