Tuesday, December 12, 2017
Education

Study finds flaws in Florida's computerized testing, but state will use the results anyway

A $600,000 study of Florida's new computerized tests found problems in "just about every aspect" of their rollout last spring. It found "suspect" scores and concluded the testing fell short of the "normal rigor and standardization." What's more, the study was unable to pinpoint where the troubles occurred.

All in all, though, Alpine Testing Solutions suggested the Florida Standards Assessments did what they were intended to do — measure student performance on state standards — and said the state still could use the results to grade schools and evaluate teachers.

Education Commissioner Pam Stewart said she planned to do just that, though her department cautioned that this year's tests should not be the sole factor in making decisions on individual students.

"This is welcome news for all of us," Stewart told reporters in a conference call. "Now all Floridians can share my confidence in the assessment."

But they didn't.

Many said parts of the report simply verified their long-standing concerns.

Calling the tests valid seemed counterintuitive, given the flaws that were found and student experiences in the spring, Florida School Boards Association executive director Andrea Messina said.

"I recognize that scientifically it has been determined valid," she said. "But I think when parents read that … or when teachers hear it, it's going to undermine the credibility of the entire program. They all know firsthand the challenges they had."

University of North Carolina testing expert Greg Cizek said the state's decision to keep using the tests could be acceptable if the problems were spread widely across Florida. He said it could prove troublesome, though, if the difficulties affected students in certain areas.

"If the company did not do some sort of postmortem to determine how densely these students were nested, then you've got a real problem," he said. "If 1 percent of those students were concentrated in an area, that school is at a great disadvantage."

That's something the validity study couldn't determine with certainty, though.

Pasco County school superintendent Kurt Browning keyed in on the lack of precision. He was troubled by a line in the report stating "the precise magnitude of the problems is difficult to gauge with 100 percent accuracy," and that it was clear the spring FSA administration "did not meet the normal rigor and standardization expected."

"We think we're going to be assessing at one level, and they're telling us in their own words that we're not," Browning said. He repeated his opposition to using the results for school grades, "much less anything else."

Problems ranged from hackers interrupting the testing servers to students getting knocked out of their exams and struggling to log back in. Some counties reported no problems, while others had repeated troubles.

As a result, Alpine Testing recommended against using the results as the sole factor in determining individual consequences such as whether students should receive tutoring, be held back a grade or kept from graduating.

Stewart stressed that state tests never are used as the single guideline for any such decision.

Moreover, she noted, third-graders took a paper test, which had none of the implementation problems associated with the computer versions taken in other grade levels.

The state also waived the requirement that the FSA math end-of-course exams count as 15 percent of students' course grades, although they still must pass the algebra I test to graduate.

Citrus County School Board Chairman Thomas Kennedy wasn't thrilled with the state's decision. If the FSA can't be used for individual student decisions, "then it shouldn't be used for teachers' evaluations and schools' grades either," he said.

Bob Schaeffer, education director for the advocacy group Fair­Test and a Lee County resident, agreed that the disruptions were so serious as to make the test results unusable.

"The review process should not stop with the validity study," he said. "Florida's assessment policies must be completely overhauled. In addition, the state's taxpayers should receive a rebate from AIR," the company hired to administer the tests.

Schaeffer and others raised additional questions as well.

They pointed to a finding in the report that, for some subjects, only 65 percent of questions were aligned to state standards. Several questions focused on Utah standards, where the test was sampled, the report stated.

The report recommended replacing those items.

Stewart said a team of content experts in her department culled many items from the tests because of such issues, and the process will continue.

But the notion of not having all questions tied to Florida standards outraged Browning, the Pasco superintendent, who for months has joined colleagues across the state in criticizing the FSA field testing.

"You have got to be kidding me," Browning said. "We have been doing backflips with Florida standards, and now our assessment has a number of items that are based on Utah standards? Shame on AIR. Shame on them all."

Pasco parent and activist Deb Herbage argued that the study wouldn't have been necessary if the state had done all its work ahead of time, rather than rushing into testing.

She noted many items, such as proof of question alignment and validity, were promised in the state's contract with AIR.

But by the time the Legislature met in the spring, lawmakers deemed the study critical to the long-term viability of the state's accountability system. The public had witnessed myriad technical problems, and needed an independent review of the tests' value, the lawmakers said.

Now that the report is out, the challenge is to make improvements and move forward, leaders said.

"This validity study, combined with the Legislature's efforts during the 2015 session to reform student testing, has strengthened our school accountability system," House Speaker Steve Crisafulli, R-Merritt Island, said in a statement. "The Alpine study did highlight important problems that occurred with the administration of the FSA, and I believe the Department of Education and the test vendor must work together to resolve these issues moving forward."

Senate Education Committee Chairman John Legg, R-Trinity, who called for the study, said his panel would make the report its sole focus when it meets this month.

Contact Jeffrey S. Solochek at [email protected] or (813) 909-4614. Follow @JeffSolochek.

Comments
‘It’s like an insane nightmare’: Parents question private company hired to drive special needs kids to school

‘It’s like an insane nightmare’: Parents question private company hired to drive special needs kids to school

RIVERVIEW — As a foster parent with two sons of her own, Kayla Storey has learned all the tricks to get her kids out of bed and off to school every morning. But this year, Storey says she’s the one waking up every school day with a knot i...
Updated: 12 hours ago

University of Central Florida Greeks won’t hold social events, serve alcohol for 6 weeks this spring

ORLANDO — University of Central Florida fraternities and sororities won’t host social activities or any events with drinking for at least the first six weeks of the spring semester, up from the two-week ban on alcohol that has been in place in the pa...
Published: 12/08/17
Amid reports of rapes, beatings, cover-ups, grand jury to probe juvenile justice abuses

Amid reports of rapes, beatings, cover-ups, grand jury to probe juvenile justice abuses

Disturbed by stories about the rape of teens by supervisory staff, a pandemic of sometimes savage force, brutal beatdowns ordered by youth care workers and policies that permit the hiring of violent offenders, Miami-Dade’s state attorney wants to kno...
Published: 12/07/17
Henderson: Some basic facts about Hillsborough’s teacher pay imbroglio

Henderson: Some basic facts about Hillsborough’s teacher pay imbroglio

Hillsborough County’s public school teachers are horn-honking, voice-raising, sign-waving, foot-stomping mad, and I can’t blame them. They are paying for a problem they didn’t create. About one-third of the workforce was expecting to receive a $4,000...
Published: 12/07/17
In Watershed Ambassadors Program, Pasco students learn about natural Florida

In Watershed Ambassadors Program, Pasco students learn about natural Florida

SPRING HILL — On a small wooden dock at the Cross Bar Ranch, Cynthia Brinker gingerly pokes through the trappings in her fishing net, plucking out a tiny creature to examine close up. "What the heck is this?" the Weightman Middle School studen...
Published: 12/06/17
Updated: 12/07/17
Crognale named 2018 Hernando Principal of the Year

Crognale named 2018 Hernando Principal of the Year

BROOKSVILLE — For just a year and a half, Steve Crognale has been the principal at the Endeavor and Discovery Academies. But now, he’s been named the Hernando School District Principal of the Year for 2018. Endeavor serves students, most of them hig...
Published: 12/06/17
‘It’s like an insane nightmare’: Parents question private company hired to drive special needs kids to school

‘It’s like an insane nightmare’: Parents question private company hired to drive special needs kids to school

RIVERVIEW — As a foster parent with two sons of her own, Kayla Storey is skilled at calming first-day-of-school jitters. But this school year, Storey says she’s the one waking up every weekday with a knot in her stomach.It’s been there ever since th...
Updated: 12 hours ago
Why do universities handle sexual assault cases, anyway?

Why do universities handle sexual assault cases, anyway?

News stories about campus sexual assault often get the question, "Why do schools handle these cases, anyway?"Readers often wonder how universities got tasked with handling these convoluted cases in the first place. Where, they ask, do the police come...
Published: 12/05/17
Updated: 12/11/17
Hillsborough teachers keep the heat on after $92 bonus offer

Hillsborough teachers keep the heat on after $92 bonus offer

TAMPA — The second Hillsborough County School Board meeting in less than a month took place Tuesday against a backdrop of honking car horns, cheering teachers and audience members moving through the room in shifts.Dressed in blue union-issued T-shirt...
Published: 12/05/17
Updated: 12/06/17
Hillsborough school district names Teacher of the Year finalists

Hillsborough school district names Teacher of the Year finalists

Finalists were announced Tuesday for Hillsborough County Teacher of the year and other honors.Winners will be announced at a banquet on Jan. 16.The finalists for teacher of the year are: Jennifer Jackson, seventh grade science, Stewart Middle; Alexa ...
Published: 12/05/17