Federal auditors find problems with FCAT scoring
Florida has some issues with FCAT scoring, including discrepancies with how scanning machines read some of the answers that students bubble in, according to this new audit by the U.S. Department of Education's Office of Inspector General.
The auditors say the Florida Department of Education's internal controls for FCAT scoring "provide reasonable assurance that assessment results are reliable." But they also say they found discrepancies in the "gridded responses" on nine of 50 FCAT test booklets they selected at random.
Gridded responses are used on some FCAT math tests and require students to fill in several bubbles with each answer. In most of the cases cited as discrepancies, students changed their answers and appeared to clearly erase their initial responses, but the scanners still recorded something other than the revised answer.
"As a result of the discrepancies identified, students may not be receiving proper credit for their responses based on the intensity, or lack thereof, of an an erasure," the audit report says. "Inaccurate scanning of gridded responses could affect the individual student's overall score and potentially otherwise impact the student given the high-stakes implications of the FCAT."
In light of other issues with FCAT scoring, FCAT critics said, "Here we go again."
"If there are problems with how the test is scored ... it's another reason to broaden our state's assessment formula," Mark Hollis, spokesman for the House Democratic Caucus, told The Gradebook.
The audit also criticizes the department for not better watchdogging its then-contractor, CTB McGraw-Hill. And it found that an old problem with FCAT testing still exists: The hiring of readers who don't meet state requirements for grading the FCAT writing test. The auditors found14 of one subcontractor's employees and 16 of another's should not have been allowed to score.
At one point during the audit, which began in June 2008 and is based on 2007-08 tests, federal officials issued subpoenas to CTB McGraw-Hill and two subcontractors to obtain test booklets and other materials.
The Florida DOE disagreed with two of the three federal findings. Among other objections, it says the auditors should have relied on original tests and not copies to determine whether there were scanning discrepancies. "Copies are susceptible to varying degrees of shading, and the human eye cannot consistently and accurately discriminate bubble intensity to the fine degree that today's scanning technology can," DOE officials wrote.
DOE spokeswoman Kelsey Lehtomaa issued this statement this evening: "This was the first Inspector General audit of the Department’s assessment scoring system and these findings were very helpful to us as we work to further evaluate and perfect our scoring system. As the report notes, Florida’s scoring system has been found reliable, however there are areas were we can and will improve. Florida’s students deserve an assessment system of the highest caliber and we’re in the process of addressing any issues raised by our partners at the Federal level. Our former contractor will no longer be administering the FCAT and we are already working with our new contractor to ensure that we have full compliance with the recommendations of the audit. Florida’s students can be assured our system is now and will continue to be a reliable system."
The chair of the House education committee, Rep. John Legg, R-New Port Richey, said in an e-mail that he has yet to read the audit in detail, but "my first review is that these issues are minor in nature, but could be come a significant issue if left unresolved."
He said the committee will be looking more closely at the audit and asking DOE for a response.