Behind the Statistical Mask: Data Obsession in Schools

“You can use all the quantitative data you can get, but you still have to distrust it and use your own intelligence and judgment.”
– Alvin Toffler

In a recent blog entry about the pace of life in our schools, I concluded with several cautions. One of these, I suggested, was against the growing obsession with testing data in education. I was not speaking of data that talented teachers instinctively use and good schools should systematically monitor, but the trending, frenzied determination to gather more and more standardised testing data that seems to increasingly dominate discourse on education today. While I agree with the ideal of data that informs schools, the proliferation of testing as the predominant source of data is something that is not serving students well. This is what I said:

Data-driven decision-making is the new buzz-phrase in education these days. I am all for informed support of students, but even the phrase “data-driven decision-making”should sound an alert to educators. We don’t need to be driven any further. An increased obsession with outcomes and results is not the way forward. The fact is, our schools are already swimming in data, but we have little time to evaluate its meaning or purpose. We do need data to inform learning in schools but, I would suggest, we need a people-centred approach to running our schools more than anything. We must get the order of these imperatives right.

Outside of the wider context of the focus on the pace of life in our schools, these words may seem like a diatribe against data informing education. It is not intended as such. However, I am not alone in having misgivings about the ever-increasing and vocal push for “data-driven schools”. I think we all know that, in schools, as in most walks of life, there are, essentially, three kinds of data:

Authentic Data: Supporting Students
This is what we strive for in our school. This is data that is clear, objective, and reveals important information that can, when carefully considered, inform decision-making and support student learning. From stats about attendance to cognitive assessments, this is always about doing what us best for the student.

Smokescreen Data: Obscured Vision
This is the school that is endlessly committed to gathering data, but may not do anything with it or choose to share it only selectively. Used badly, data can become a substitute for vision or a hiding place for those intent on pre-calculating every decision. Smokescreens are designed to protect decision makers.

Machine Data: Systemic Officialdom
This is the world of standardised tests, blind certainty, and high-stakes assessments. As one writer describes it, “This results in an increasingly unnecessary data burden requiring greater system complexity … Data risks becoming disconnected from its primary function – to inform teaching and learning – and, at worst, ends up as an exercise itself.”

Data is not unlike technology. It is a tool, not an end in itself. Why one uses technology and how it enhances learning is key. The same is true of data. The obsession with test scores, typical in the United States, is an example of data gone awry. In this context, I tend to agree with Alfie Kohn who has stated: “If all the earnest talk about “data” (in the context of educating children) doesn’t make you at least a little bit uneasy, it’s time to recharge your crap detector. Most assessment systems are based on an outdated behaviorist model that assumes nearly everything can — and should — be quantified. But the more educators allow themselves to be turned into accountants, the more trivial their teaching becomes and the more their assessments miss.”

The more we belabour the pre-eminence of “panacea” data that focuses on annual tests, the less time we will inevitably have for students. How teachers and parents are feeling is well summed up by a recent piece in The Guardian by Phoebe Doyle: “Teachers aren’t trained statisticians, data analysts. … What I want as a parent is a happy child, skipping to school, one that feels safe, nurtured, respected and one that loves learning. None of this can be achieved through data.” I would contend, however, that the right data used in the right ways can contribute significantly to the happiness of children.

Clearly, data that is explicitly intended and designed to support student learning is vital in schools today and we have an array of technologies that can support this process effectively. Many schools are inundated with data already and operate a patchwork of systems with no clear methodology for how to best organise and use this data. This is how schools have organically evolved. As the shift towards personalised learning grows, the ability for schools and students to monitor data that informs this learning will become increasingly crucial, but we need to consider some critical questions before we inadvertently and blindly rush into the data minefield. We must ask ourselves if we can:

  • clearly determine that the data gathered will support student learning
  • ascertain the key data that we need and gather it in efficient, unobtrusive ways
  • be sure that we do not create an officialdom that disrupts learning in the quest for data
  • determine how we will use the data, store it, and organise it professionally
  • ensure that human relationships and not data analysis remain our priority

For schools and national systems that have pursued data without fully considering these questions, the result is often not unlike that described by Brenda Dyck, writing for Education World: “Since I am an educator in the Age of Accountability, I resign myself to gathering and analyzing everything from classroom behavior to achievement test results and homework completion. But as I wade through the rows and rows of numbers in front of me, I speculate whether the time I’m spending will justify the time I’m not spending … developing meaningful relationships with my students, providing meaningful feedback … or even applying new learning.”

We must commit to systems that honour authentic data in support of student learning. If the goal is, as the 2015 Horizon Report contends, “to build better pedagogies, empower students to take an active part in their learning, target at-risk student populations, and assess factors affecting … student success,” then this is a goal worth pursuing. The work of people like Scott McLeod and Michael Fullan demonstrates that this important task can be done well and with powerful results.

In this age of accountability, we have a professional obligation to identify and appropriately use data to support and enhance student learning. But we must proceed with caution and clarity of purpose. As Michael Fullan and Lyn Sharratt rightly suggest: “Education is overloaded with programs and data. The growth of digital power has aided and abetted the spread of accountability-driven data—adequate yearly progress, test results for every child in every grade, common core standards, formative and summative assessments galore. Each data set shows a full continuum from below standard to exceed standards. Educators need to be able to put FACES on the data … and, to know what to do to help individual children behind the statistical mask.”

Our accountability is to students, not data. We must not lose sight of this salient truth. Like many fundamental truths, we don’t need data to support its veracity.