So-called “data-driven” decision-making isn’t always smart
October 20, 2010 Leave a comment
ASCD writes about reformers’ obsession with data and how it’s leading to a new kind of stupid:
Today’s enthusiastic embrace of data has waltzed us directly from a petulant resistance to performance measures to a reflexive and unsophisticated reliance on a few simple metrics—namely, graduation rates, expenditures, and the reading and math test scores of students in grades 3 through 8. The result has been a nifty pirouette from one troubling mind-set to another; with nary a misstep, we have pivoted from the “old stupid” to the “new stupid.”
The article goes on to describe the three major characteristics of “the new stupid”:
- Misusing data
- Oversimplifying and over-applying findings from research
- Fixating on performance data and failing to consider management data
The importance of this last point is grossly underappreciated by reformers.
Existing achievement data are of limited utility for management purposes. State tests tend to provide results that are too coarse to offer more than a snapshot of student and school performance, and few district data systems link student achievement metrics to teachers, practices, or programs in a way that can help determine what is working. More significant, successful public and private organizations monitor their operations extensively and intensively. FedEx and UPS know at any given time where millions of packages are across the United States and around the globe. Yet few districts know how long it takes to respond to a teaching applicant, how frequently teachers use formative assessments, or how rapidly school requests for supplies are processed and fulfilled.
For all of our attention to testing and assessment, student achievement measures are largely irrelevant to judging the performance of many school district employees. It simply does not make sense to evaluate the performance of a payroll processor or human resources recruiter—or even a foreign language instructor—primarily on the basis of reading and math test scores for grades 3 through 8.
Just as hospitals employ large numbers of administrative and clinical personnel to support doctors and the military employs accountants, cooks, and lawyers to support its combat personnel, so schools have a “long tail” of support staff charged with ensuring that educators have the tools they need to be effective. Just as it makes more sense to judge the quality of army chefs on the quality of their kitchens and cuisines rather than on the outcome of combat operations, so it is more sensible to focus on how well district employees perform their prescribed tasks than on less direct measures of job performance. The tendency to casually focus on student achievement, especially given the testing system’s heavy emphasis on reading and math, allows a large number of employees to either be excused from results-driven accountability or be held accountable for activities over which they have no control. This undermines a performance mindset and promises to eventually erode confidence in management.
Very little of the education reform conversation focuses on school management and how it helps or hurts teachers. Through my work, I know teachers frequently have to work around last-minute schedule changes and other requests and wait out sometimes months-long processes for acquiring new printer toner or chairs that are the right height for the tables. The sanctity of the classroom is not respected, with PA announcements, administrators, fellow teachers, and visitors regularly breaking the flow of a lesson. Improving school management would likely go a long way to making the work of the teacher easier, but apparently teachers are the only school personnel who need to be held accountable for anything.