Two weeks after the close of Florida's troubled spring testing season, a Utah-based firm won approval Friday to determine whether the state's new tests are valid.
Alpine Testing Solutions, partnering with edCount LLC of Washington, D.C., was the only group to bid on the project, which lawmakers mandated amid parent and educator complaints about the tests and their implementation. Their concerns included technology problems during testing, as well as the way the tests were assembled and sampled ahead of time.
State Sen. Jeremy Ring, a Broward County Democrat on a special selection panel, pressed company officials to explain how they would look into the technology issues, which kept some students from logging in and kicked others out of their tests midway through.
"Whether it could have been prevented or not — I happen to think very elementary mistakes were made — to students it didn't matter," Ring said. "At the end of the day, they couldn't get on."
Chad Buckendahl, one of the lead investigators, said the issue affects the credibility of the results and how they might be used.
He noted that the review would look at how the computer problems occurred, what processes were in place to handle them, and if they met industry standards. Additionally, he said, the study would assess how the problems affected student motivation, confidence and related measures.
That's just part of the overall study, said Tracey Hembry, who helped make the team's presentation to the state selection panel in Jacksonville. The project also will address such matters as whether the test content was properly aligned to Florida's academic standards and whether the questions were field-tested adequately with a representative sample of students.
Superintendents have challenged the field testing because it occurred primarily in Utah, which has a different demographic mix than Florida.
"It's all about thinking about validity, all the way through," Hembry told the panel.
But determining whether a test is valid won't be a straight "yes" or "no" proposition, Buckendahl added.
"Validity is a matter of degree," he said, noting that a series of reports will provide evidence of whether the tests worked as intended and expected in assessing student performance. "You're not going to get a simply black or white answer for this."
The fact that Alpine/edCount was the sole bidder for this project did not go unnoticed. Some testing reform activists, as well as at least one state lawmaker, implied that the short time frame for submitting proposals and finishing the work was designed to get certain results.
"Responding to the outrage, legislators said that a formal and fair evaluation would take place to see if the newly implemented statewide tests were valid. But, in true Florida fashion, the shenanigans continue," state Sen. Dwight Bullard, a Miami Democrat, wrote in a letter to the Tampa Bay Times.
Ring pointedly asked Buckendahl why his group was the only one to seek the work.
"Trying to get a proposal within a week and a half is pretty tough this time of year if you're already committed," Buckendahl replied, adding that the two groups were able to shuffle staff to meet the deadlines.
Representatives from two of six firms on the state's approved vendor list confirmed that they were too busy to bid on the project. They each vouched for Alpine and edCount as well-qualified to conduct the review.
Sen. John Legg, the Pasco County Republican who wrote the language calling for the validity study, laughed at the notion that he drew up his amendment to benefit a specific vendor.
"I am not smart enough to figure that out and be that clever," Legg said.
The $594,000 price tag was a bit high for his liking, and having a sole bidder wasn't his preference either. But Legg said the key was to get an independent reviewer on board to settle the tests' validity, which in turn will inform how to use the data, if at all.
St. Johns County school superintendent Joe Joyner, also on the selection panel, asked the group representatives if they had conflicts of interest.
Buckendahl said that Alpine had not worked with Florida's primary testing vendor, American Institutes for Research, and that it had not conducted any validity studies of this type of test. The company purposely stayed away from designing state tests associated with Common Core and related standards, too, he added.
On its website, edCount says it has worked in the past as a partner with AIR.
The next step is for the state to finish a contract with Alpine and edCount. After that, the group plans to begin its work with a preliminary report submitted by the end of July and the final by the Sept. 1 deadline set by lawmakers.
Contact Jeffrey S. Solochek at firstname.lastname@example.org or (813) 909-4614. Follow @jeffsolochek.