Failed log-ins. Frozen screens. Server crashes. Service denials.
Students, teachers and administrators recall all too well the woes that plagued Florida's most ambitious attempt at computerized testing last spring. As this year's testing season approaches, they're working to avoid a repeat.
"We'll have a first glimpse of whether or not the issues have been resolved" when Florida Standards Assessments in writing begin Feb. 29, said Gisela Feild, research and assessment director Miami-Dade schools. "We hope we won't see the same problems again."
The Florida Department of Education, its testing vendor American Institutes for Research, along with districts and schools, have taken several steps to prevent such troubles. Those include expanding bandwidth, upgrading defenses against outside attacks and improving testing software.
Even with such moves, though, the department warned that students still might encounter interruptions beyond their control. And that, said FairTest public education director Bob Schaeffer, could hurt some children.
Imagine the impact when the screen goes blank for a seventh-grader taking a civics test required to get out of middle school, Schaeffer said. "For an emotional adolescent to experience that, it's a scary situation."
Yet there's almost no way to guarantee trouble-free computerized testing on a stage as large as Florida's, experts said.
That's because the undertaking is "not just a test, but a massive technology project" that involves so many moving parts in a decentralized system, said Doug Levin, founder of EdTech Strategies, a consulting firm.
The tests originate at the vendor's servers, move over the Internet through district providers, and enter schools with varying levels of networks, hardware and infrastructure.
"Some of the devices are going to be quite old. Some of the school networks won't be as strong," said Levin, who helped develop the nation's first education technology plan in 1996. "Inherently, it is a somewhat challenging endeavor."
No test is foolproof, whether on computers or paper, noted Marianne Perie, director of the University of Kansas Center for Educational Testing and Evaluation.
"The same year that we were hit by a cyber attack, UPS also lost a batch of Scantron answer sheets off the back of their truck," Perie said. "They were returned to us torn, dirty, soaked in motor oil and covered in tire tracks. We had to throw several dozen out because they were unreadable, and had to rebubble hundreds of others to get them to go through the machine."
Paper tests also can be more susceptible to cheating, more expensive to administer and less nuanced in depth, said Greg Cizek, a testing expert at the University of North Carolina. Still, many Florida educators have called for a return to paper and No. 2 pencil as a less time-consuming and less glitch-ridden way to check student knowledge.
They have yet to see the value in moving to computers, which were sold as a way to make testing go more smoothly and deliver results more quickly. Neither has happened in Florida.
"At present, there is just no way to get around the problems, the way technology currently is and the way schools are equipped to handle technology," Cizek said.
Florida has experienced interruptions since it first introduced computerized testing to a handful of students retaking the FCAT a decade ago. Regardless of which vendor, test or system was used, the state has seen servers crash, providers implement unsupported changes, even construction crews accidentally cut cables to schools.
And Florida is not alone.
Earlier this month, Tennessee canceled its computerized testing after one day amid major testing platform outages across the state. It is now sending all schools paper tests instead, at its vendor's expense.
"Despite the many improvements to the system in recent months, we lost confidence in the system's ability to perform consistently," said Tennessee Department of Education spokeswoman Chandler Hopper.
Indiana, Minnesota, Virginia and Montana are among several states that had computer testing interruptions in the past year.
"The fact that this has happened so often in Florida and around the country should be a wake-up call to policymakers to go slow on computerized testing and have backups available," Schaeffer said. "Be prepared would be our warning."
The state and districts are trying to prepare.
Florida Department of Education spokeswoman Meghan Collins provided a list of actions the state has taken to prevent problems. It improved its testing servers to combat cyber attacks, developed a system to warn students before a large amount of text is deleted, and enabled students to restore past versions of their responses.
"This is not an exhaustive list of improvements," Collins said via email. "But we hope that it does serve as a reassurance that we take very seriously the concerns expressed last year."
The state's testing vendor, AIR, declined to comment.
Local school officials were quick to note many of the problems students experienced last spring were not caused by districts. Still, they've made improvements where they can.
Pasco County added about 1,000 computers into schools for instruction and testing. That should allow the district to shorten its testing window, testing supervisor Mark Butler said.
The district upgraded networks and equipment, also installing a more advanced defensive service to fend off outside attacks.
Pinellas County schools expanded bandwidth for testing, while increasing its number of computers. Hillsborough County created a special log-in for each school to get into the testing platform, to work around the access troubles they had a year ago.
Miami-Dade, the state's largest school district, will do things like shut down access to YouTube and other sites that tax a school's Internet flow. It has enhanced its filters and alerts to outside attacks so it can move its testing data and, in many instances, avoid slowdowns too.
The district is considering whether to allow more wireless mobile devices for testing, added director of student assessment Sally Shay. She said the district's key concern remains maxing out the testing window, even with 105,000 machines and 180 technicians in the field to tackle issues as they arise.
Cizek, the UNC testing professor, said computerized assessments offered the promise of something better — certain types of questions that can't be delivered on paper, savings in time and money, exams adapted to students' abilities.
One of the reasons it hasn't fully delivered, he said, is because states focused on computerizing tests before they fully integrated technology into instruction. Had they done it the other way around, he added, they would have built up capacity to handle tests while students became accustomed to using the machines for academics.
"They were rushed into the marketplace based on political and ideological agendas, and not academic need and technological readiness," said Schaeffer of FairTest.
Given the history, district officials expected the debut of Florida's new system to hit bumps. But 2015 was "over and beyond anything we could have expected," Shay said.
Feild, the Miami-Dade testing director, said she hoped AIR and others learned enough to at least avoid the same troubles. She wouldn't rule out new ones.
"Hopefully it will be something different," she said, "preferably not something too severe."
Contact Jeffrey S. Solochek at [email protected] or (813) 909-4614. Follow @JeffSolochek.