Value-Added and Social Desirability Bias
Chetty, Friedman, and Rockoff’s research on teacher value-added is widely interpreted as an argument for giving good teachers a fat raise. Obama’s 2012 State of the Union speech pulls a dollar figure right out of their abstract:
At a time when other countries are doubling down on education, tight budgets have forced states to lay off thousands of teachers. We know a good teacher can increase the lifetime income of a classroom by over $250,000. A great teacher can offer an escape from poverty to the child who dreams beyond his circumstance. Every person in this chamber can point to a teacher who changed the trajectory of their lives. Most teachers work tirelessly, with modest pay, sometimes digging into their own pocket for school supplies — just to make a difference.
Teachers matter. So instead of bashing them, or defending the status quo, let’s offer schools a deal. Give them the resources to keep good teachers on the job, and reward the best ones.
When I finally read the original paper, I was bemused to discover that Chetty et al.’s support for merit raises is muted at best.
Using Monte Carlo simulations analogous to those above, we estimate that retaining a teacher at the ninety-fifth percentile of the estimated VA [value-added] distribution (using three years of data) for an extra year would yield present value… [of] $212,000. In our data, roughly 9 percent of teachers in their third year do not return to the school district for a fourth year. The attrition rate is unrelated to teacher VA, consistent with the findings of Boyd et al. (2008). Clotfelter et al. (2008) estimate that a $1,800 bonus payment in North Carolina reduces attrition rates by 17 percent. Based on these estimates, a one-time bonus payment of $1,800 to high-VA teachers who return for a fourth year would increase retention rates in the next year by 1.5 percentage points and generate an average benefit of $3,180. The expected benefit of offering a bonus to even an excellent (ninety-fifth percentile) teacher is only modestly larger than the cost because one must pay bonuses to (100 − 9)/1.5 ≈ 60 additional teachers for every extra teacher retained.
The policy that dramatically passes the cost-benefit test is “deselection,” better known as firing bad teachers.
To quantify the value of improving teacher quality, we evaluate Hanushek’s (2009, 2011) proposal to replace teachers whose VA ratings are in the bottom 5 percent of the distribution with teachers of average quality.
[…]
Rothstein estimates that a policy which fires teachers if their estimated VA after three years falls below the fifth percentile would require a mean salary increase of 1.4 percent to equilibrate the teacher labor market.43 In our sample, mean teacher salaries were approximately $50,000, implying that annual salaries would have to be raised by approximately $700 for all teachers to compensate them for the additional risk. Based on our calculations above, the deselection policy would generate NPV gains of $184,000 per teacher deselected, or $9,250 for all teachers on average (because only 1 out of 20 teachers would actually be deselected). Hence, the estimated gains from this policy are more than ten times larger than the costs. [emphasis mine]
What’s up? I once again point my accusatory finger at Social Desirability Bias. Rewarding good teachers sounds a lot nicer than firing bad teachers. So when research comes along that potentially recommends both, pundits and politicians don’t coolly crunch the numbers. They leap to the recommendation that’s pleasing to the ear. So what if the original researchers find that firing bad teachers wins with flying colors? Move along folks, nothing to see here…
The post appeared first on Econlib.
I am not sure how “value added” by teacher is calculated, but it needs to be controlled for by the intellectual aptitude of the students, otherwise it is useless.
Variation in scores is more determined by the quality of the student than the quality of the teacher or school. Comparing test scores among teachers in any given year tells you very little. Comparing how much test scores increased for the same student from previous years is more useful.
"The policy that dramatically passes the cost-benefit test is “deselection,” better known as firing bad teachers."
My brother and sister, ten and eight years older than me, had all sorts of stories about a ridiculously bad economics teacher (in secondary education). When it was my turn to go to that school I realized none of the stories were exaggerated. And all students knew. His classes were a joke. A few years after graduating I came across a different teacher from that school and somehow the lousy econ teacher came up. The man was still around, but no longer allowed to teach any classes (what he did do I don't remember, but I expect not much). Turns out all other teachers also knew that he was useless. They had heard all the stories from students over the years. How could they not have. The guy had even been sent to additional workshops and trainings and whatnot, nothing helped. He spent 23 years 'teaching' at the same school, and more before that elsewhere, and only his retirement saved the next generation of students from having to sit through his classes without learning anything beyond what they got from the textbook. And he wasn't even the worst one (but the even more outrageous character had been hired on a one year temp contract, so at least there was one we never heard of again except through the anecdotes). Why did they keep him around? Hard to fire, hard to find a replacement, is my guess.