For years, critics have raised doubts about the credibility of the Florida Comprehensive Assessment Test. Now the federal government is asking a few questions, too.
Auditors with the U.S. Department of Education found discrepancies with how scanning machines read some of the answers that students bubble in, according to a new report based on 2008 test scores.
The report criticizes the Florida Department of Education for not better watchdogging its former testing contractor, CTB McGraw-Hill, and says some FCAT graders did not have required credentials.
The state's internal controls for FCAT scoring "provide reasonable assurance that assessment results are reliable," the report also says.
But the findings still gave FCAT critics fresh ammunition.
"If there are problems with how the test is scored … it's another reason to broaden our state's assessment formula," said Mark Hollis, a spokesman for the House Democratic Caucus.
The FCAT remains the backbone of the school accountability system put into place by then-Gov. Jeb Bush and maintained by Gov. Charlie Crist. But its use in grading schools and retaining students has made it unpopular.
Questions about scoring haven't helped. A few years ago, a review sparked by two Democratic lawmakers found some of the temporary workers hired to grade FCAT tests did not have degrees in the areas they were grading. In 2007, the state announced that one of the 2006 tests had been botched by human error.
The federal audit — the first ever done of the FCAT — found discrepancies in the "gridded responses" on nine of 50 FCAT test booklets that auditors selected at random.
Gridded responses are used on some FCAT math tests and require students to fill in several bubbles with each answer. In most of the cases cited as discrepancies, students changed their answers and appeared to clearly erase their initial responses, but the scanners still recorded something other than the revised answer.
The audit also noted:
• Fourteen of one contractor's 72 employees did not pass a screening test required to process the FCAT. A second contractor could not verify that 16 of its scorers met state requirements.
• Personal student information was "plainly visible" after being dumped in the trash at CTB's facility in Indianapolis.
• Federal officials had to issue subpoenas to CTB and two subcontractors to obtain test booklets and other materials.
The Florida Education Department disagreed with two of the three federal findings.
Among other objections, it said the auditors should have relied on original tests and not copies to determine whether there were scanning discrepancies. "Copies are susceptible to varying degrees of shading, and the human eye cannot consistently and accurately discriminate bubble intensity to the fine degree that today's scanning technology can," state officials wrote.
CTB officials who could respond to the audit findings could not be reached Friday.
In May, the Florida Education Department awarded a five-year FCAT contract — worth up to $254 million — to another company, Pearson. Department spokeswoman Kelsey Lehtomaa said in a written statement Friday that it is working with Pearson "to ensure that we have full compliance with the recommendations of the audit."
"There are areas we can and will improve," the statement said.
Ron Matus can be reached at [email protected] or (727) 893-8873.