Confused by Florida's teacher scoring? So are top teachers

Patrick Boyko was recently named Teacher of the Year in  Hillsborough County. Yet his VAM score, and those of other highly regarded teachers, don't seem to add up.
Patrick Boyko was recently named Teacher of the Year in Hillsborough County. Yet his VAM score, and those of other highly regarded teachers, don't seem to add up.
Published Jan. 22, 2015

Patrick Boyko is a good teacher. He is probably even outstanding. A couple of weeks ago, Boyko was named the 2014 Hillsborough County Teacher of the Year.

He beat out a band director who has tripled the program's membership, a National Board Certified art teacher, and a third-generation Hillsborough educator who's bringing robotics into elementary schools.

"Patrick treasures his students and challenges them not only to be academically successful but also as future responsible adults," the Hillsborough Education Foundation wrote of his selection.

So one would reasonably expect that numbers used by the state to measure teacher effectiveness would be favorable to Boyko.

And yet.

Teacher performance data released last week by the Florida Department of Education paint a strange picture of the educators recognized as the best in Tampa Bay.

The numbers, known as "VAM" scores, use standardized test results to measure how much a teacher contributes to a student's growth. The letters refer to the "value-added model" for rating teachers, a formula that plays a significant role in their job evaluations even as questions continue to be raised about the scores' validity.

Boyko's VAM score for last school year was a minus 10.23 percent. That would mean his students scored 10 percent worse on the FCAT than typical, similar children across Florida — because of his teaching. In 2011-12, Boyko, a social studies teacher at Jefferson High in Tampa, earned a VAM score of minus 19.44 percent.

That score "would never reflect on what I do," he said.

"It's tough to see because I don't want to be represented by a negative number, but I am a highly effective teacher in regard to the Hillsborough County system, and they use hard data."

Scores for Teacher of the Year winners and finalists across the region fell all over the map. Pinellas County's 2014 Outstanding Educator Kevin Ford had a VAM score of 2.55 percent.

Stephanie Martanovic, a Pinellas finalist who won the "Fan Favorite Award," had a VAM score of minus 41.42 percent.

None of these educators teach classes that were tested on the FCAT. Defenders of VAM say the numbers still measure students' learning. But, in these cases, the question of accuracy looms large.

Ford's score of 2.55 percent came with a margin of error of 50.03 percentage points, a statistic the state refers to as "standard error."

A standard error larger than the actual VAM score means the teacher's contribution to students' learning growth can't be determined one way or the other, according to the DOE.

Standard error can be affected by the number of test takers in a class and how similar students' test scores were to each other. A small class with a lot of different scores would generate a high standard error.

"It just points to the lack of validity in this whole process," said Kim Black, president of the Pinellas Classroom Teachers Association, upon reviewing the teachers' scores.

In Pasco County, Teacher of the Year finalist James Washington received a score of minus 9.58 percent, with a standard of error of 20.17 percent. A ninth-grade language arts teacher at Zephyrhills High, Washington sponsors a group that provides free backpacks, haircuts and eye exams to students living in poverty.

Seven out of 10 of the Hernando Teacher of the Year finalists whose VAM scores were available had negative scores.

"Am I going to affix any value to a ranking of teachers in this district based on their VAM score? The answer is no, absolutely no, absolutely not," said Eric Williams, director of school improvement for the Hernando County School District. "I can't even ensure that the assessment is tied to the subject they teach."

Williams said the district's highest-rated employee probably would be a physical education teacher who interacts with a lot of students at a high-performing school.

When Florida signed on to the federal Race to the Top initiative, which encourages teacher performance pay among other reforms, districts were given until 2014-15 to develop student assessments tied to each educator. Hernando plans to roll out more than 600 such tests, Williams said.

But Florida later passed legislation forcing districts to tie teacher performance evaluations to student assessments much sooner. Until recently, the complicated VAM formula has exclusively drawn from the FCAT.

House Speaker Will Weatherford told the Tampa Bay Times editorial board last week that he likes the idea of having a value-added system. "I think the concept works," he said.

"I think a VAM model can work. I think it's been proven that it does work in some instances. But it's such a complex thing, I frankly couldn't even explain it to you."

Senate President Don Gaetz has questioned why some schools have dismal VAM scores (called "schoolwide" scores) but earn A or B grades from the state.

The Department of Education released VAM data Monday after losing a public records lawsuit by the Florida Times-Union. But not every teacher in the state was included. The department blocked out about 7 percent of teachers' names, according to the newspaper.

Monica Taylor, another finalist for Outstanding Educator in Pinellas, was one of those whose score was left out. She said she doesn't know the number, but thinks it was high because she was rated "highly effective" by the school district. As a media specialist at Woodlawn Elementary, her VAM score doesn't reflect what she does, she said.

"Whether you smile at a child or are stern with them … whether you can convince them to try a new book — how do you evaluate that?"

Another Pinellas finalist, Melissa Colgan, says her VAM score, before adjustments, gave her three out of a possible 50 points on her evaluation for the 2011-12 school year.

"When our scores first came out, I was actually proud to say I got three out of 50 points — like, woohoo! It was a joke," says Colgan, who works with special-needs students at Largo Middle.

She shared her score with Largo's media specialist, who two years ago was named one of the best at his job in the state. He also had received three points. "I said I must be doing something right," Colgan says, "because I got the same score as the media specialist of the year."

Lisa Gartner can be reached at Follow her on Twitter @lisagartner.