PIE Network Guest

“A decade ago, it was a pipe dream that you could talk systematically about which teachers are more or less effective,” said moderator Rick Hess in opening the session. “Now we’ve got some real, if modest, information on this score. But sometimes it seems we’ve taken these new tools and tried to swing at everything in sight, just because we have a hammer. How do we bring perspective to the use of value-added analysis and the issue of teacher quality?”

Biggest advances in improving teacher effectiveness

When asked about the biggest gains made over the past five years, Bryan Hassel of Public Impact said there has been a huge increase in understanding the difference in effectiveness among teachers and the impact the best ones have on improving student achievement. “We have made a down payment on new policies,” Hassel said. “We need to evaluate every teacher every year, and we need to make those evaluations matter. But that won’t be enough—we have a lot more to do.”

There has been “real headway” in using effectiveness as a tool for continuous improvement, said Paige Kowalski of the Data Quality Campaign. “We now have 35 states that are capable of calculating value-add. We have gotten policy makers engaged in the data.”

Sarah Almy of The Education Trust agreed that requiring student achievement for teacher evaluations in most states is “a big win”. But she added, “There are a lot of promises that have been made, but few have been implemented. Most of these are still just paper conversations.”

“The main thing we have gotten right is a huge shift from focusing on whole schools to focusing on individuals,” said Dan Goldhaber, director of the Center for Education Data and Research . “It’s a different world today. The focus on performance evaluations is what we got most right.” But Goldhaber said he doesn’t think schools have actually acted on the differences in teacher evaluations.

What are we getting wrong?

Goldhaber said he’s worried value-add is being used in a very algorithmic fashion. “We need to be more nuanced,” he said. “We need people who can make good decisions about performance. We have a system where people can not apply their judgment about the people who work for them.”

“Just doing evaluations does not translate to great teaching and learning,” Almy said. “We can’t assume that by identifying the most effective and least effective we will have great teachers in front of all kids. Most of us are making the assumption that once the data is available the distribution will happen, but we need to be more intentional.”

“We need to do a better job of making sure teachers understand what the data is and how to use it,” said Kowalski. She added that another big factor is the multiple and complex relationships between teachers and students.

Kowalski said state agencies haven’t proved very capable of communicating about the value-added model. “Advocates need to effectively communicate about what the data is and how it can be used,” she said.

“We haven’t zeroed in on excellent teachers, who produce well over a year’s worth of growth,” said Hassel. “Sure, it would be great to have average teachers instead of bad teachers, but average teachers don’t close achievement gaps. If twenty to twenty-five percent of teachers are excellent, only twenty to twenty-five percent of students will get great teachers.” By redesigning jobs and using technology, Hassel argued, “we could enable excellent teachers to reach three-fourths of students or more.”

Recommendations for the states

“States need to get out in front of messaging,” said Kowalski. “The first time a teacher loses a job because of data, it will all come crashing down.”

“People are working within a framework that doesn’t allow for professional judgment,” said Goldhaber. “It is a regulatory and cultural problem. We need good people in leadership positions to try to change it. “

“We need to accelerate identifying excellent teachers instead of waiting for air-tight systems to fire people,” said Hassel. “We need to clear barriers that keep excellent teachers from students. And we need to dramatically increase the will to give every student an excellent teacher.” One way could be to “dramatically increase financial rewards of success for districts and teachers. Secondly, policy makers could enact a new civil right to excellent teachers, requiring districts to give every child excellent teachers often enough to make progress.”

Almy advised advocates to “stick it out through implementation.” She said advocates should ask, “What will the state do when a district does not have a good distribution of effective teachers?”

Value-added and future performance

When Hess asked if value-added data could accurately predict how teachers might perform with different types of students in the future, Goldhaber replied, “We can be confident that what we are working on will be better at identifying great teachers than what we currently have, but what we currently have is a pretty low bar.” “I’d use value-added over teacher credentials in making a bet about whether an effective teacher this year will be one next year.”

Kowlaski said states are leaving the data issue up to the districts, which means inconsistent data is reaching the state. “If everyone is defining it differently and crunching the numbers differently that we don’t really know if the policy is working,” she said. “It’s critical to ensure consistency at the state level and provide consistency for districts.”

When asked about variability in value-added models, Hassel said districts shouldn’t lock in a system. “They should put something in place and make it better over time,” he said. “It will be better than what we have right now.”

Impact of value-added data on annual performance reports

Annual performance reports for educators “will not be different than the widget effect if districts are not interested in knowing how their teachers are different,” said Hassel. “If they don’t want the data, we won’t make progress.”

Kowlaski said districts are now asking thoughtful questions about the consequences of using data. “We have to train people to manage data and understand why and how—not just the what,” she said. “We need to dig in to the data to understand it and know if we are really doing it the right way.”Goldhaber said there should be some alignment between value-added data and classroom observation, but you wouldn’t expect results to be “perfectly aligned, nor would you want this.” “Value-added information can also be used to change the nature of conversations in schools, allowing supervisors to have tough conversations with teachers,” he said.

Should value-added data be publicly available?

“People have a right to aggregated district and school data,” said Kowlaski. “They should know why some areas are doing better than other— transparency can help parents understand.”

Goldhaber and Hassel both said they are opposed to publishing names with data. But after hearing ideas from audience member Patricia Levesque from A Foundation for Florida’s Future about ways to let parents know about their teacher’s effectiveness, Hassel said advocates need to think more about this topic before ruling out disclosures.