Some history and context on VAM in teacher evaluation

In the Columbia Journalism Review’s Tested: Covering schools in the age of micro-measurement, LynNell Hancock provides a rich survey of the history and context of the current debate over value-added modeling in teacher evaluation, with a particular focus on LA and NY.

Here are some key points from the critique:

1. In spite of their complexity, value-added models are based on very limited sources of data: who taught the students, without regard to how or under what conditions, and standardized tests, which are a very narrow and imperfect measure of learning,

No allowance is made for many “inside school” factors… Since the number is based on manipulating one-day snapshot tests—the value of which is a matter of debate—what does it really measure?

2. Value-added modeling is an imprecise method whose parameters and outcomes are highly dependent on the assumptions built into the model.

In February, two University of Colorado, Boulder researchers caused a dustup when they called the Times’s data “demonstrably inadequate.” After running the same data through their own methodology, controlling for added factors such as school demographics, the researchers found about half the reading teachers’ scores changed. On the extreme ends, about 8 percent were bumped from ineffective to effective, and 12 percent bumped the other way. To the researchers, the added factors were reasonable, and the fact that they changed the results so dramatically demonstrated the fragility of the value-added method.

3. Value-added modeling is inappropriate to use as grounds for firing teachers or calculating merit pay.

Nearly every economist who weighed in agreed that districts should not use these indicators to make high-stakes decisions, like whether to fire teachers or add bonuses to paychecks.

Further, it’s questionable how effective it is as a policy to focus simply on individual teacher quality, when poverty has a greater impact on a child’s learning:

The federal Coleman Report issued [in 1966] found that a child’s family economic status was the most telling predictor of school achievement. That stubborn fact remains discomfiting—but undisputed—among education researchers today.

These should all be familiar concerns by now. What this article adds is a much richer picture of the historical and political context for the many players in the debate. I’m deeply disturbed that NYS Supreme Court Judge Cynthia Kern ruled that “there is no requirement that data be reliable for it to be disclosed.” At least Trontz at the NY Times acknowledges the importance of publishing reliable information as opposed to spurious claims, except he seems to overlook all the arguments against the merits of the data:

If we find the data is so completely botched, or riddled with errors that it would be unfair to release it, then we would have to think very long and hard about releasing it.

That’s the whole point: applying value-added modeling to standardized test scores to fire or reward teachers is unreliable to the point of being unfair. Adding noise and confusion to the conversation isn’t “a net positive,” as Arthur Browne from The Daily News seems to believe; it degrades the discussion, at great harm to the individual teachers, their students, the institutions that house them, and the society that purports to sustain them and benefit from them.

Advertisements

Diluting the meaning of “highly qualified” teachers

Valerie Strauss posts:

Senators have included in key legislation language that would allow teachers still in training to be considered “highly qualified” so they can meet a standard set in the federal No Child Left Behind law.

In an era when the education mantra is that all kids deserve great teachers, some members of Congress want it to be the law of the land that a neophyte teacher who has demonstrated “satisfactory progress” toward full state certification is “highly qualified.”

Is it just me, or have I been transported to 1984? The original definition of “highly qualified teacher” in No Child Left Behind already represented what in most high-achieving countries would be a bare minimum qualification for beginning a teaching residency. Allowing teachers-in-training to be classified as “highly qualified” seems ridiculous on its face.

Strauss sees this as a giveaway to political darling Teach for America:

Teachers still in training programs are disproportionately concentrated in schools serving low-income students and students of color, the very children who need the very best the teaching profession has to offer. In California alone, nearly a quarter of such teachers work in schools with 98-100 percent of minority students, while some affluent districts have none. Half of California’s teachers still in training teach special education.

Allowing non-certified teachers to be considered “highly qualified” would be a gift to programs such as Teach for America, which gives newly graduated college students from elite institutions five weeks of summer training before sending them into low-performing schools.

%d bloggers like this: