Advertisement
  1. Education

Glitches, cyberattacks deal blow to credibility of Florida's new school tests

The debut of standardized computer testing in Florida has proved anything but standard for the state’s eighth-, ninth- and 10th-graders, including these ninth-graders at Land O’Lakes High School.
Published Mar. 11, 2015

The debut of new standardized computer testing in Florida has proved anything but standard for the state's eighth-, ninth- and 10th-graders.

First came server problems caused by the state's vendor, American Institutes for Research. Then, according to officials, a cyberattack known as a "distributed denial of service," or DDOS, struck the system, and law enforcement is investigating.

Some kids took the test without a hitch. Some struggled to overcome the glitches. Some have yet to finish, days after starting.

The upshot has been doubt. Parents, educators and decisionmakers have demanded to know: Can the results ever be trusted?

"It's an excellent question — and one the Department of Education will need to answer empirically," said state Sen. Don Gaetz, a legislative leader on school issues and a former Okaloosa County superintendent.

Last year, Kansas leaders threw out results from their trial run of a statewide exam. Their situation was, in many ways, parallel to Florida's.

"Our first week we had problems that were really our fault," tied to server errors, said Marianne Perie, director of the University of Kansas Center for Educational Testing and Evaluation. "Then we had a period of time when we got hit by a DDOS."

Eventually, the state had a time of trouble-free testing.

But multiple studies of student performance led experts to recommend throwing out the scores.

One analysis showed that students who had problems with test access skipped an average of 15 percent of the questions, while those who had no difficulties skipped 1 percent. Another review matched students with similar academic and demographic characteristics, and found the children who tested while problems were occurring no longer compared to the ones who tested later.

"Bad scores are worse than no scores," Perie said, explaining why she recommended losing the scores.

As their name implies, standardized tests rely on uniform conditions in order to get usable information. The standards are considered a foundation of fairness, comparability and integrity in scoring.

It's difficult to test all students under identical circumstances, given human error, but schools try.

"It's what we don't know that kills us," said Scott Marion, associate director of the National Center for the Improvement of Educational Assessment.

Not knowing the full extent of the problems, and how they affected students, makes it difficult to analyze their effects, he said. Comparing the scores becomes problematic, making the data questionable.

"If you get to take your test and you get to go through it seamlessly, and I'm going through it and I don't, do you have an advantage? I think you probably do," Marion said. "That becomes an issue for comparability."

Florida could face that problem.

"Any sense of standard conditions seems to have been compromised by the events reported," said Steve Dunbar, a testing expert at the University of Iowa. "Problems of this frequency and magnitude are significant in how they might affect student performance on the test. They would not be tolerated in college admissions or certification testing."

The same could be said for school accountability testing that carries high stakes, he said. Florida's 10th-grade writing exam is part of the state's graduation requirement.

The Florida Department of Education has yet to address how it will handle results from this round of testing. Meanwhile, a growing number of critics have called upon state leaders to at least abandon attaching consequences to the scores, given the swirl of uncertainties.

Perie agreed that the state should make no decisions until it can run the statistics and determine if problems are real.

"They have to be able to compare (the first week) to a clean testing window," she said, adding that the evaluation must be conducted by an independent firm, not the state or its testing vendor.

At the same time, she added, "there's a difference between statistical analysis and credibility."

Perie referred to Oklahoma's 2014 testing cycle, during which vendor software did not work properly. A review uncovered no issues with the results, she said, but the public outcry prompted officials to throw out some student scores and fire the vendor.

"At a certain point, even if you don't like to give up on accountability, you're risking the credibility of the system," Marion said.

Miami-Dade superintendent Alberto Carvalho wondered whether Florida hadn't already hit that point.

He said the state rushed to put the Florida Standards Assessments into play, and its administration has been "less than smooth."

Even if everything goes well from here, Carvalho said, he questioned whether the problems so far had eroded the tests' credibility.

Will the state now push forward and still base school consequences on these scores? "That's a question that needs to be taken up," he said.

Times-Herald Tallahassee bureau reporter Kathleen McGrory contributed to this report. Contact Jeffrey S. Solochek at jsolochek@tampabay.com or (813) 909-4614. Follow @jeffsolochek.

ALSO IN THIS SECTION

  1. First-year Cox Elementary School teacher Kevin Knibbs, 33, of Dade City, answers questions about the components of time: hours, minutes, seconds, to his third-grade class on Monday, September 30, 2019, at the school in Date City. From left are students Angel Young, 8, Arlene Luna, 8, and Jahkia Gray, 8. DOUGLAS R. CLIFFORD  |  Tampa Bay Times
    A roundup of stories from around the state.
  2. Special Olympics athlete Lucas Yingling, 7, is cheered on by Pasco-Hernando State College basketball player Hayden Greene during a conditioning clinic held at Wendell Krinn Technical High in New Port Richey. Michele Miller
    Team members help Special Olympians hone their skills.
  3. First-year Cox Elementary School teacher Kevin Knibbs, 33, of Dade City, walks with students Angel Young, 8, left, and Kaivion Williams, 9, right, while en route to his third-grade class on Sept. 30 at the school in Date City. Knibbs decided to become an educator after working as a school custodian. DOUGLAS R. CLIFFORD  |  Tampa Bay Times
    Kevin Knibbs never thought about working with students — until he started interacting with them.
  4. Bayonet Point Middle School teacher Cynthia Thompson wants to become Pasco County schools superintendent. Courtesy of Cynthia Thompson
    Cynthia Thompson is a graduation enhancement instructor at Bayonet Point Middle School.
  5. Pasco School District headquarters in Land O' Lakes
    Steve Hegarty spent 10 years as Hillsborough schools public information officer before taking the police department post.
  6. Hillsborough County Superintendent Jeff Eakins, left, looks on while school board chair Tammy Shamburger speaks on newly raised concerns of a undiscovered cemetery for indigent African Americans that may be within the vicinity of King High School in Tampa, Florida on Friday, October 18, 2019.  OCTAVIO JONES  |  Times
    A roundup of stories from around the state.
  7. Joanne Glenn, Pasco eSchool principal, addresses the eSchool faculty on opening day of teacher preplanning week in 2018. Pasco eSchool is launching its first online dual-enrollment courses in conjunction with Pasco-Hernando State College in the second semester.  GAIL DIEDERICH | Special to the Times
    Students will have access to two sections of two courses — microapplications and public speaking.
  8. Challenger K-8 School students, from left, Jeremy Gonzalez, 13, Jackson Hoyt, 12, Benjamin Harper, 12, and Gianni Labdar, 12, finish meals consisting of fresh salads, quesadillas and nachos during a lunch service on Oct. 15 at the school in Spring Hill during the county's Fresh from Florida Plate Day event. DOUGLAS R. CLIFFORD  |  Tampa Bay Times
    Starting a farm-to-school initiative has been more complicated than district officials expected.
  9. Check tampabay.com for the latest breaking news and updates. Tampa Bay Times
    University police say a 25-year-old grad student enrolled at the University of Florida fell to her death Friday afternoon from near the top of the 8-story parking facility.
  10. Check tampabay.com for the latest breaking news and updates. Tampa Bay Times
    The 63-year-old crossing guard was hospitalized, according to the Pinellas County Sheriff’s Office.
Advertisement
Advertisement
Advertisement